An object recognition method, apparatus and storage medium

Abstract

This application provides an object recognition method, apparatus, and storage medium, relating to the field of communication technology, to solve the technical problem in the prior art that it is impossible to accurately identify occluded objects. The object recognition method includes: generating multiple first edge images from multiple original images; each of the multiple first edge images including at least one target object; generating multiple images to be fused from the multiple first edge images; the multiple images to be fused including: an image where the target object is occluded and an image where the target object is not occluded; fusing each first edge image with the corresponding image to be fused to obtain multiple fused images; training an object recognition model based on the multiple fused images; the object recognition model is used to identify whether an occluded object in the image to be detected is a target object.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an object identification method, apparatus and storage medium. Background Technology

[0002] In recent years, with the continuous development of deep learning technology, it has been applied to various scenarios. For example, target detection technology in deep learning can be used to automate the detection of fiber optic construction sites, thus preventing problems from occurring.

[0003] However, due to the complex environment of construction sites, target objects are easily occluded. When a target object is occluded, existing target detection technologies cannot accurately identify the occluded object. Summary of the Invention

[0004] This application provides an object recognition method, apparatus, and storage medium to solve the technical problem in the prior art that it is impossible to accurately identify occluded objects.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, an object recognition method is provided, comprising: generating multiple first edge images from multiple original images; each of the multiple first edge images including at least one target object; generating multiple images to be fused from the multiple first edge images; the multiple images to be fused including: an image in which the target object is occluded and an image in which the target object is not occluded; for each first edge image, fusing the first edge image with the corresponding image to be fused to obtain multiple fused images; training an object recognition model from the multiple fused images; the object recognition model being used to identify whether an occluded object in the image to be detected is a target object.

[0007] Optionally, multiple images to be fused are generated based on multiple first edge images, including: obtaining the position of a target pixel with a pixel value of a first preset value in a preset target object; the preset target object is at least one target object among all target objects contained in the multiple first edge images; determining the size of the occlusion region of the preset target object based on the size of the preset target object; determining the position of the target pixel as the center position of the occlusion region, and determining the position of the occlusion region based on the size of the occlusion region; and performing occlusion processing on the original image to which the preset target object belongs based on the position of the occlusion region to obtain multiple images to be fused.

[0008] Optionally, based on the location of the occluded area, the original image to which the preset target object belongs is occluded to obtain multiple images to be fused, including: determining the location of the occluded area in the original image to which the preset target object belongs based on the location of the occluded area; converting the pixel values ​​of each pixel in the target area into preset pixel values ​​to obtain multiple images to be fused; the preset pixel values ​​are the average pixel values ​​of each pixel; and the target area is the occluded area in the original image to which the preset target object belongs.

[0009] Optionally, generating multiple first edge images based on multiple original images includes: acquiring multiple original images; each original image including at least one target object; converting the multiple original images into multiple second edge images using edge detection technology; and converting the pixel values ​​of pixels other than the target object in each second edge image into second preset values ​​to obtain multiple first edge images.

[0010] Optionally, for each first edge image, the first edge image is fused with the corresponding image to be fused to obtain multiple fused images, including: for each first edge image, multiple first edge images and multiple corresponding images to be fused are fused at the channel level to obtain multiple fused images; or, for each first edge image, features are extracted from the first edge image and the corresponding image to be fused, the extracted features are fused, and the fused features are convolved to obtain multiple fused images.

[0011] Optionally, it also includes: acquiring the image to be detected; inputting the image to be detected into an object recognition model to obtain a recognition result; the recognition result being whether the occluded object in the image to be detected is the target object.

[0012] In a second aspect, an object recognition device is provided, comprising: a processing unit; the processing unit being configured to generate a plurality of first edge images based on a plurality of original images; each of the plurality of first edge images including at least one target object; the processing unit being further configured to generate a plurality of images to be fused based on the plurality of first edge images; the plurality of images to be fused including: an image in which the target object is occluded and an image in which the target object is not occluded; the processing unit being further configured to fuse each first edge image with the corresponding image to be fused to obtain a plurality of fused images; the processing unit being further configured to train an object recognition model based on the plurality of fused images; the object recognition model being used to identify whether an occluded object in the image to be detected is a target object.

[0013] Optionally, the processing unit is specifically used for: obtaining the position of a target pixel with a pixel value of a first preset value in a preset target object; the preset target object is at least one target object among all target objects contained in multiple first edge images; determining the size of the occlusion region of the preset target object according to the size of the preset target object; determining the position of the target pixel as the center position of the occlusion region, and determining the position of the occlusion region according to the size of the occlusion region; and performing occlusion processing on the original image to which the preset target object belongs according to the position of the occlusion region to obtain multiple images to be fused.

[0014] Optionally, the processing unit is specifically used for: determining the location of the occluded region in the original image to which the preset target object belongs, based on the location of the occluded region; converting the pixel values ​​of each pixel in the target region into preset pixel values ​​to obtain multiple images to be fused; the preset pixel values ​​are the average pixel values ​​of each pixel; and the target region is the occluded region in the original image to which the preset target object belongs.

[0015] Optionally, the processing unit is specifically used for: acquiring multiple original images; each of the multiple original images includes at least one target object; converting the multiple original images into multiple second edge images using edge detection technology; and converting the pixel values ​​of pixels other than the target object in each second edge image into second preset values ​​to obtain multiple first edge images.

[0016] Optionally, the processing unit is specifically used to: for each first edge image, fuse multiple first edge images and the images to be fused corresponding to the multiple first edge images at the channel level to obtain multiple fused images; or, for each first edge image, extract features from the first edge image and the images to be fused corresponding to the first edge image, fuse the extracted features, and perform convolution processing on the fused features to obtain multiple fused images.

[0017] Optionally, it also includes: an acquisition unit; the acquisition unit is used to acquire the image to be detected; the processing unit is further used to input the image to be detected into an object recognition model to obtain a recognition result; the recognition result is whether the occluded object in the image to be detected is the target object.

[0018] Thirdly, an object recognition device is provided, including a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the object recognition device is running, the processor executes the computer execution instructions stored in the memory to cause the object recognition device to perform the object recognition method described in the first aspect.

[0019] The object identification device may be a network device or a component of a network device, such as a chip system within the network device. The chip system supports the network device in implementing the functions involved in the first aspect and any of its possible implementations, such as acquiring, determining, and transmitting data and / or information involved in the aforementioned object identification method. The chip system includes a chip, but may also include other discrete devices or circuit structures.

[0020] Fourthly, a computer-readable storage medium is provided, comprising computer-executable instructions that, when executed on a computer, cause the computer to perform the object recognition method described in the first aspect.

[0021] Fifthly, a computer program product is also provided, which includes computer instructions that, when executed on an object recognition device, cause the object recognition device to perform the object recognition method as described in the first aspect above.

[0022] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the object recognition device, or it may be packaged separately from the processor of the object recognition device; this application does not limit this.

[0023] The descriptions of the second, third, fourth, and fifth aspects of this application can be referenced to the detailed description of the first aspect.

[0024] In the embodiments of this application, the names of the aforementioned object identification devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. For example, the receiving unit may also be called a receiving module, receiver, etc. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalents.

[0025] The technical solution provided in this application brings at least the following beneficial effects:

[0026] Based on any of the foregoing aspects, this application provides an object recognition method, comprising: an electronic device generating a plurality of first edge images from a plurality of original images. Each of the plurality of first edge images includes at least one target object. Next, the electronic device generates a plurality of images to be fused from the plurality of first edge images. The plurality of images to be fused include: an image where the target object is occluded and an image where the target object is not occluded. Then, for each first edge image, the electronic device fuses the first edge image with the corresponding image to be fused to obtain a plurality of fused images. Subsequently, the electronic device can also train an object recognition model based on the plurality of fused images. The object recognition model is used to identify whether an occluded object in the image to be detected is a target object.

[0027] As shown above, electronic devices can train object recognition models by fusing images. Since the fused image includes multiple images of the target object that are occluded, the object recognition model trained on these multiple occluded images can accurately identify whether the occluded object is the target object.

[0028] Furthermore, since the fused image includes multiple first edge images, and each first edge image only uses black and white, the object recognition model trained using these first edge images can reduce the impact of excessive color differences when recognizing targets.

[0029] The beneficial effects of the first, second, third, fourth, and fifth aspects of this application can all be referred to in the analysis of the above-mentioned beneficial effects, and will not be repeated here. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the structure of an object recognition system provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram of the hardware structure of an object recognition device provided in this application embodiment. Figure 1 ;

[0032] Figure 3 A schematic diagram of the hardware structure of an object recognition device provided in this application embodiment. Figure 2 ;

[0033] Figure 4 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 1 ;

[0034] Figure 5 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 2 ;

[0035] Figure 6 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 3 ;

[0036] Figure 7 A schematic diagram illustrating a process for converting an original image into an image to be fused, provided as an embodiment of this application;

[0037] Figure 8 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 4 ;

[0038] Figure 9 This application provides a second edge image generated by the Canny operator in an embodiment of the present application;

[0039] Figure 10 This application provides a second edge image generated by the Sobel operator.

[0040] Figure 11 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 5 ;

[0041] Figure 12 A schematic diagram illustrating the fusion of a first edge image and an image to be fused, provided as an embodiment of this application;

[0042] Figure 13 A schematic diagram illustrating another method of fusing a first edge image and an image to be fused, provided in an embodiment of this application;

[0043] Figure 14 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 6 ;

[0044] Figure 15 A flowchart illustrating an object recognition method provided in this application embodiment. Figure 7 ;

[0045] Figure 16 This is a schematic diagram of the structure of an object recognition device provided in an embodiment of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0048] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0049] As described in the background section, in recent years, with the continuous development of deep learning technology, it has been applied to various scenarios. For example, target detection technology in deep learning can be used to automate the detection of fiber optic construction sites, thereby preventing problems from occurring.

[0050] However, due to the complex environment of construction sites, target objects are easily obscured. When a target object is obscured, existing target detection technologies cannot accurately identify it. Furthermore, significant color differences between multiple objects can also interfere with recognition.

[0051] To address the aforementioned problems, this application provides an object recognition method, comprising: an electronic device generating multiple first edge images from multiple original images. Each first edge image includes at least one target object. Next, the electronic device generates multiple images to be fused from the multiple first edge images. These multiple images to be fused include images where the target object is occluded and images where the target object is not occluded. Then, for each first edge image, the electronic device fuses the first edge image with the corresponding image to be fused to obtain multiple fused images. Subsequently, the electronic device can train an object recognition model based on the multiple fused images. The object recognition model is used to identify whether an occluded object in the image to be detected is a target object.

[0052] As shown above, electronic devices can train object recognition models by fusing images. Since the fused image includes multiple images of the target object that are occluded, the object recognition model trained on these multiple occluded images can accurately identify whether the occluded object is the target object.

[0053] Furthermore, since the fused image includes multiple first edge images, and each first edge image only uses black and white, the object recognition model trained using these first edge images can reduce the impact of excessive color differences when recognizing targets.

[0054] This object recognition method is applicable to object recognition systems. Figure 1 One structure of this object recognition system is shown. For example... Figure 1 As shown, the object recognition system includes an electronic device 101 and a data acquisition device 102.

[0055] The electronic device 101 and the data acquisition device 102 are connected in communication.

[0056] In this application, the data acquisition device 102 is used to acquire video data within the area and send the acquired video data to the electronic device 101.

[0057] Electronic device 101 can receive video data sent by data acquisition device 102 and process the video data to obtain multiple original images. It then generates multiple first edge images and multiple images to be fused from these original images. Next, electronic device 101 can fuse the multiple first edge images and the multiple images to be fused, and train an object recognition model using the fused images. In this way, electronic device 101 can use the object recognition model to identify whether a target object exists in the video data sent by data acquisition device 102.

[0058] Optionally, the data acquisition device 102 may be a monitoring device such as a camera or a terminal, and this embodiment of the application does not limit this.

[0059] Electronic device 101 can be a terminal, a server, or other types of electronic devices; this application embodiment does not limit this.

[0060] Optionally, when the physical device of electronic device 101 is a terminal, the terminal may be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The terminal may communicate with one or more core networks via a radio access network (RAN). The terminal may be a mobile terminal, such as a mobile phone (or "cellular" phone) and a computer with a mobile terminal, or a portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile device that exchanges voice and / or data with the radio access network, such as a mobile phone, tablet computer, laptop computer, netbook, or personal digital assistant (PDA).

[0061] Optionally, when the physical device of electronic device 101 is a server, the server can be one of the servers in a server cluster (composed of multiple servers), a chip in the server, a system-on-a-chip in the server, or a virtual machine (VM) deployed on a physical machine. This application embodiment does not limit this.

[0062] The basic hardware structure of electronic device 101 includes Figure 2 or Figure 3 The object recognition device shown includes the following components. Figure 2 and Figure 3 Taking the object recognition device shown as an example, the hardware structure of electronic device 101 will be introduced.

[0063] like Figure 2 The diagram shown is a hardware structure schematic of an object recognition device provided in an embodiment of this application. The object recognition device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, memory 22, and communication interface 23 are connected via the bus 24.

[0064] Processor 21 is the control center of the object recognition device. It can be a single processor or a collective term for multiple processing elements. For example, processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0065] As one embodiment, processor 21 may include one or more CPUs, for example Figure 2 CPU 0 and CPU 1 are shown in the diagram.

[0066] The memory 22 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0067] In one possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 via a bus 24 and is used to store instructions or program code. When the processor 21 calls and executes the instructions or program code stored in the memory 22, it can implement the object recognition method provided in the following embodiments of this application.

[0068] In this embodiment, the software programs stored in the memory 22 of the electronic device 101 are different, so the functions implemented by the electronic device 101 are different. The functions performed by each device will be described with reference to the following flowchart.

[0069] In another possible implementation, the memory 22 can also be integrated with the processor 21.

[0070] Communication interface 23 is used for the object recognition device to connect with other devices via a communication network, which may be Ethernet, wireless access network, wireless local area network (WLAN), etc. Communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0071] Bus 24 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0072] Figure 3 Another hardware structure of the object recognition device in an embodiment of this application is shown. For example... Figure 3 As shown, the object recognition device may include a processor 31 and a communication interface 32. The processor 31 is coupled to the communication interface 32.

[0073] The functions of processor 31 can be referred to in the description of processor 21 above. In addition, processor 31 also has a storage function, and can perform the functions of memory 22 mentioned above.

[0074] The communication interface 32 is used to provide data to the processor 31. The communication interface 32 can be an internal interface of the object recognition device or an external interface of the object recognition device (equivalent to communication interface 23).

[0075] It should be pointed out that, Figure 2 (or Figure 3 The structure shown in the diagram does not constitute a limitation on the object recognition device, except... Figure 2 (or Figure 3 In addition to the components shown in the diagram, the object recognition device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0076] The object recognition method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0077] The object recognition method provided in this application embodiment is applied to Figure 1 The electronic device 101 in the object recognition system shown, such as Figure 4 As shown, the object recognition method provided in this application includes:

[0078] S401, The electronic device generates multiple first edge images based on multiple original images.

[0079] Each of the multiple first edge images includes at least one target object. Each first edge image corresponds one-to-one with each original image.

[0080] Optionally, the data acquisition device can generate video data for different time periods within different areas and send the video data to the electronic device. Upon receiving the video data, the electronic device can convert it into video frames. Then, the electronic device can convert the video frames, including the target object, into video images. Afterward, staff can perform annotation operations on the video images using the electronic device. In this way, the electronic device can respond to the annotation operations performed by the staff, annotating the target objects in the video images to obtain the original images.

[0081] Optionally, the target objects in the video image can be labeled using label image (LabelImg) software, that is, the target objects in the video image are bounded by a target box.

[0082] Specifically, after labeling the target object in the video image, the electronic device can acquire multiple raw images. Since each of the multiple raw images includes at least one target object, the electronic device can convert the multiple raw images into multiple first edge images. Thus, each of the multiple first edge images includes at least one target object.

[0083] Optionally, in order to train an object recognition model using multiple raw images, the electronic device can divide the multiple raw images into a training set, a validation set, and a test set.

[0084] S402, The electronic device generates multiple images to be fused based on multiple first edge images.

[0085] The multiple images to be fused include: images of the target object that are occluded and images of the target object that are not occluded.

[0086] Specifically, since each of the multiple first edge images includes at least one target object, the electronic device can determine the position and size of the occlusion region used to occlude the target object based on the multiple first edge images. In this way, the electronic device can generate multiple images to be fused, each including the occluded target object, based on the multiple first edge images.

[0087] It should be noted that the multiple images to be fused in this application are used to train the object recognition model. In order to increase the diversity of the input of the object recognition model, the electronic device can select some target objects for occlusion processing according to a preset probability, that is, only occlude a part of the target objects.

[0088] Optionally, the electronic device can occlude all target objects in multiple first edge images, meaning all target objects in multiple images to be fused are occluded. Then, the electronic device can use multiple original images and multiple images to be fused together as input to train an object recognition model.

[0089] The electronic device can also select a portion of the first edge images from multiple first edge images according to a preset probability, and occlude all target objects in the portion of the first edge images to obtain multiple images to be fused corresponding to the portion of the first edge images. That is, a portion of the first edge images is used to generate multiple images to be fused, while the remaining portion of the first edge images is not used to generate images to be fused. Furthermore, all target objects in the generated images to be fused are occluded. Next, the electronic device can also determine the original image corresponding to the remaining portion of the first edge images as the images to be fused. Subsequently, the electronic device can determine the occluded images to be fused and the original images corresponding to the remaining portion of the first edge images as multiple images to be fused, so that the electronic device can train an object recognition model based on the multiple images to be fused.

[0090] The electronic device can also perform occlusion processing on all target objects in multiple first edge images according to a preset probability. That is, in a portion of the multiple images to be fused, the target objects in each image to be fused are occluded; in another portion, the target objects in each image to be fused are not occluded; and in the remaining portion, each image to be fused contains both occluded and unoccluded target objects. Then, the electronic device uses all the images to be fused as input to train the object recognition model.

[0091] S403. For each first edge image, the electronic device fuses the first edge image with the image to be fused corresponding to the first edge image to obtain multiple fused images.

[0092] Specifically, since the multiple images to be fused are generated from multiple first edge images, there is a one-to-one correspondence between the multiple first edge images and the multiple images to be fused. In this case, the electronic device can fuse each of the multiple first edge images with the corresponding image to be fused to obtain multiple fused images.

[0093] S404. The electronic device trains an object recognition model based on multiple fused images.

[0094] Among them, the object recognition model is used to identify whether an occluded object in the image to be detected is the target object.

[0095] Specifically, after generating multiple fused images, the electronic device can use these fused images as input for training an object recognition model. Since the fused image is obtained by fusing the image to be fused with a first edge image, and the image to be fused includes occluded target objects, the object recognition model trained using the fused image can identify whether the occluded object is the target object. Furthermore, because the first edge image is a black-and-white edge image, the impact of excessive color differences can be reduced when the object recognition model identifies objects.

[0096] Optionally, when annotating video images, staff can annotate multiple categories of target objects and indicate their categories to obtain the original image. In this way, the electronic device can train an object recognition model using the annotated target objects across multiple categories, enabling the model to identify target objects of various categories and output their categories. Subsequently, after identifying a target object, the object recognition model can obtain its location within the image to be detected and output that location.

[0097] It should be understood that, since the fused images include unoccluded target objects, the object recognition model can also identify whether unoccluded objects are target objects. Therefore, the object recognition model can determine whether a target image exists in the image to be detected.

[0098] In some embodiments, combined with Figure 4 ,like Figure 5 As shown, in the above S402, the electronic device generates multiple images to be fused based on multiple first edge images, specifically including:

[0099] S501. The electronic device acquires the position of a target pixel with a pixel value of a first preset value in a preset target object.

[0100] The preset target object is at least one of all target objects contained in the multiple first edge images.

[0101] Optionally, the first preset value can be 1.

[0102] Specifically, since the first edge image is an edge image, in which the pixel value of the edge of the object is 1, while the pixel value of the pixels in other parts is 0. Therefore, the electronic device can select a target pixel with a pixel value of 1 (i.e., the first preset value) in the preset target object, and can determine the target pixel as the center position of the occlusion area of ​​the preset target object.

[0103] Optionally, the electronic device may randomly select a pixel from the preset target objects that have a pixel value of 1 as the target pixel.

[0104] It should be noted that one target pixel corresponds to one preset target object.

[0105] S502. The electronic device determines the size of the occlusion area of ​​the preset target object based on the size of the preset target object.

[0106] Specifically, when the occlusion area is small, the occlusion effect on the target object is poor. Conversely, when the occlusion area is large, it will obscure a significant portion of the target object, resulting in poor model training performance. Therefore, electronic devices can determine the size of the occlusion area based on a preset ratio and the size of the preset target object. This ensures that the occlusion area achieves a good occlusion effect regardless of the target object's size.

[0107] Optionally, a preset target object may have one occlusion area or multiple occlusion areas, and this application embodiment does not limit this.

[0108] For example, suppose S r Let S represent the size of the occlusion region of the r-th preset target object in a first edge image, and let R be the number of preset target objects. r for:

[0109] S r =λa r ×λb r ; r∈(1,...,R)

[0110] Among them, a r b represents the length of the r-th preset target object. r Let λ represent the width of the r-th preset target object, and λ∈[0.25,0.5] represent the preset ratio.

[0111] S503, The electronic device determines the position of the target pixel as the center position of the occlusion area, and determines the position of the occlusion area according to the size of the occlusion area.

[0112] The center of the occluded region is defined as the pixel coordinates of the center point of the occluded region. The position of the occluded region is defined as the pixel coordinates of all pixels within the occluded region.

[0113] Specifically, since the target pixel is the pixel on the edge of the preset target object, the electronic device can determine the position of the target pixel as the center of the occlusion area. In this way, the occlusion area can occlude the edge of the preset target object, so that when the object recognition model identifies the occluded object, it does not need to identify the entire edge of the occluded object to determine that the occluded object is the target object.

[0114] After determining the center location of the obstructed area, the electronic device can determine the position of the obstructed area based on the center location and the size of the obstructed area.

[0115] Optionally, the occluded area can be rectangular or other shapes, and this application embodiment does not limit this.

[0116] For example, suppose N represents the pixel values ​​of all pixels in the first edge image, (h,w) represents the pixel coordinates of each pixel in the first edge image, and the number of preset target objects in the first edge image is R. Since the occlusion region is the occlusion region of the preset target object, the number of occlusion regions is also R. Then the pixel coordinates (h,w) of the center pixel of the r-th occlusion region are... r ,w r (i.e., the center location of the obstructed area in this application) is:

[0117] (h r ,w r )=f rand ((h,w)); r∈(1,...,R), (h,w)∈A r

[0118] stN h,w =1

[0119] Among them, f rand () is a random function, A r N represents all pixels of the r-th preset target object. h,w =1 is used to represent the pixel coordinates of pixels with a value of 1 in the first edge image, stN h,w =1 is used to represent f rand In ((h,w)), the pixel value of the pixel at (h,w) is 1.

[0120] S504. The electronic device performs occlusion processing on the original image to which the preset target object belongs, based on the location of the occlusion area, so as to obtain multiple images to be fused.

[0121] Specifically, since the first edge image is generated from the original image, the pixel positions (i.e., pixel coordinates) in the first edge image are the same as the pixel positions in the original image. In this case, the electronic device can determine the preset target object corresponding to the occlusion area based on the occlusion area. Then, the electronic device can determine the original image to which the preset target object belongs based on the preset target object corresponding to the occlusion area. Afterward, the electronic device can perform occlusion processing on the original image to which the preset target object belongs based on the pixel coordinates of all pixels in the occlusion area (i.e., the position of the occlusion area in this application) to obtain multiple images to be fused.

[0122] In some embodiments, combined with Figure 5 ,like Figure 6 As shown, in S504 above, the electronic device performs occlusion processing on the original image to which the preset target object belongs, based on the position of the occlusion area, to obtain multiple images to be fused, specifically including:

[0123] S601. The electronic device determines the location of the occluded area in the original image to which the preset target object belongs based on the location of the occluded area.

[0124] Specifically, since the pixel positions in the first edge image are the same as those in the original image, the electronic device can determine the pixel coordinates of all pixels in the occlusion region in the first edge image, which are then the pixel coordinates of all pixels in the occlusion region in the original image. In this way, the electronic device can generate an occlusion region in the original image based on the pixel coordinates of all pixels in the occlusion region in the original image, so that the occlusion region can occlude a preset target object in the original image.

[0125] S602, The electronic device converts the pixel values ​​of each pixel in the target area into preset pixel values ​​to obtain multiple images to be fused.

[0126] The preset pixel value is the average pixel value of all pixels. The target region is the occluded area in the original image to which the preset target object belongs.

[0127] Specifically, since the target region is the sum of at least one occluded region in an original image to which the preset target object belongs, the electronic device can convert the pixel values ​​of each pixel in the target region into preset pixel values, so that the color of the target region is converted to the color corresponding to the preset pixel values. In this way, the electronic device can occlude the preset target object based on the color difference between the color corresponding to the preset pixel values ​​and the color of the preset target object. Subsequently, the electronic device can determine the original image to be fused from which each pixel in the target region has been converted into the preset pixel values.

[0128] It should be understood that, in order to ensure the consistency of pixel values ​​across all pixels in the original image, the electronic device can determine the average pixel value of all pixels in all occluded regions of the original image and convert all pixels in all occluded regions of the original image into the average pixel value of all pixels in all occluded regions of the original image. In this way, the total pixel value of all pixels in each original image remains unchanged.

[0129] Optionally, the preset pixel value can also be any value, and this application embodiment does not limit it.

[0130] For example, assuming the number of occluded regions in an original image is R, then the pixel coordinates of the r-th occluded region are (h... i,r ,w j,r ); i∈(1,λa) r ),j∈(1,λb r ), r∈(1,...,R), each occluded region includes S r Given a set of pixels, the average pixel value K of each pixel in the entire occluded region (i.e., the target region in this application) of the original image is... r for:

[0131]

[0132] in, The pixel coordinates in the r-th occluded region are (h i ,w j The pixel value of the pixel point (i∈(1,λa)) is given by the fact that i∈(1,λa) r ),j∈(1,λb r ), r∈(1,...,R), therefore, the pixel coordinates are (h i ,w j The pixels in () are the pixels in the target region. r b represents the length of the r-th preset target object. r Let λ represent the width of the r-th preset target object, and λ∈[0.25,0.5] represent the preset ratio.

[0133] Afterwards, electronic devices can be based on K r Determine the images to be fused

[0134]

[0135] Where (h,w) represents the pixel coordinates of each pixel in the original image.

[0136] if h = h i,r ,w=w j,r (h,w) is used to represent the pixel coordinates of a pixel in the r-th occluded region. h,w Used to represent the pixel values ​​of each pixel in the region of the original image excluding the occluded area.

[0137] Another example, Figure 7 The diagram illustrates a process for converting a raw image into an image to be fused, as shown below. Figure 7 As shown, in obtaining the original image (i.e. Figure 7 After step a), the electronic device can use edge detection technology to separate the original image (which can be obtained by x∈R) from the edge. H×W×3 (representation) is converted into a second edge image (i.e.) Figure 7 (b) Next, the electronic device can remove information other than the target object from the second edge image to obtain the first edge image (i.e., Figure 7 (c)

[0138] Next, the electronic device can determine the center position of the occlusion region by using the edge of the preset target object in the first edge image, and determine the size of the occlusion region by using the size of the preset target object. Then, the electronic device can determine the pixel coordinates of all pixels in the occlusion region of the original image based on the center position and size of the occlusion region. Finally, the electronic device can determine the pixel values ​​of all pixels in the occlusion region of the original image as the pixel average to obtain the image to be fused (i.e.,...). Figure 7 The d in the middle can be obtained through express).

[0139] Both the original image and the image to be fused are H×W matrices.

[0140] It should be noted that the method by which electronic devices generate the image to be fused from the second edge image can be called the EdgeDrop method.

[0141] In some embodiments, combined with Figure 6 ,like Figure 8 As shown, in the above S401, the electronic device generates multiple first edge images based on multiple original images, specifically including:

[0142] S801, the electronic device acquires multiple raw images.

[0143] Each of the multiple original images includes at least one target object.

[0144] Specifically, since the electronic device trains its model using target objects in images, it can acquire original images that include at least one target object. Furthermore, because the electronic device cannot identify the target object among multiple objects in the original image, the target object in the original image is labeled. Thus, the electronic device can train an object recognition model using the labeled target objects in the original image, enabling the trained model to identify whether an object is the target object even if the target object is not labeled.

[0145] S802, the electronic device converts multiple original images into multiple second edge images using edge detection technology.

[0146] Among them, multiple original images and multiple second edge images correspond one-to-one.

[0147] Specifically, in order to train an object recognition model using the edges of the target object, an electronic device can convert multiple original images into multiple second edge images using edge detection technology. Since the second edge images include the edges of all objects in the original images, the electronic device can remove edges other than the target object to obtain a first edge map.

[0148] Optionally, the edge detection technique can be the Robert operator, the Prewitt operator, the Canny operator, or the Sobel operator.

[0149] It should be understood that since the purpose of generating the second edge image is to occlude the target object using its edges, it is not necessary to obtain precise edges. Therefore, the edge detection technique in this embodiment can be any edge detection technique.

[0150] For example, Figure 9 A second edge image generated by the Canny operator (i.e., the edge detection technique in this application) is shown, such as Figure 9As shown, when the edge detection technique is the Canny operator, the electronic device can perform Gaussian smoothing on the original image to obtain image A. Next, the electronic device can perform Non-Maximum Suppression (NMS) on the gradient magnitude based on the gradient direction of image A to obtain image B. Then, the electronic device can detect and connect the image edges in image B using a double thresholding algorithm to obtain the second edge image. Assuming the second edge image is M, then:

[0151] M = f c (x,z1,z2)

[0152] Here, z1 and z2 represent the two high and low thresholds of the double-threshold algorithm in the Canny operator, used to mark and connect edge pixels. z1 can be 200, and z2 can be 300. x represents the original image. The function f... c () is used to denote the function corresponding to the Canny operator.

[0153] Figure 10 A second edge image generated by the Sobel operator (i.e., the edge detection technique in this application) is shown, such as Figure 10 As shown, when the edge detection technique is the Sobel operator, the electronic device can determine the Sobel gradient of the original image in the horizontal direction (x). Next, the electronic device can determine the Sobel gradient of the original image in the vertical direction (y). The convolution kernel can be a 3×3 kernel during calculation. Subsequently, the electronic device can determine the sum of the Sobel gradients of the original image based on the Sobel gradients in the horizontal and vertical directions (x and y) to obtain the second edge image. The sum of the Sobel gradients of the original image can be determined through weighted fusion, with a weighting coefficient of 0.5.

[0154] S803, the electronic device converts the pixel values ​​of pixels other than the target object in each second edge image into second preset values ​​to obtain multiple first edge images.

[0155] Specifically, to remove the edges of objects other than the target object in the second edge image, the electronic device can convert the pixel values ​​of all pixels in the second edge image except the target object to 0 (i.e., the second preset value in this application). Since each second edge image only includes pixels with a pixel value of 1 (white) and pixels with a pixel value of 0 (black), all objects except the target object in each second edge image are converted to black to obtain a first edge image including the target object. In this way, the electronic device can perform fusion and training using the first edge image so that the object recognition model can recognize the target object based on the edges of the target object in the first edge image.

[0156] It should be understood that since the target object is the labeled object, the pixel values ​​of the target object's pixels will not change.

[0157] For example, assuming M represents the pixel value of each pixel in the second edge image, then the first edge image N is:

[0158] N = B × M; B ∈ {0, 1} H×W

[0159] Where B∈{0,1} H×W This represents a binary mask where B is 1 when the pixel coordinates of a pixel are within the target object, and 0 when the pixel coordinates of a pixel are outside the target object. H×W represents the image matrix.

[0160] In some embodiments, combined with Figure 8 ,like Figure 11 As shown, in the above S403, for each first edge image, the electronic device fuses the first edge image with the corresponding image to be fused to obtain multiple fused images, specifically including:

[0161] S1101. For each first edge image, the electronic device fuses multiple first edge images and the images to be fused corresponding to the multiple first edge images at the channel level to obtain multiple fused images.

[0162] Specifically, since the images to be fused are generated from the first edge images, the electronic device can fuse the multiple first edge images and their corresponding images at the channel level to obtain multiple fused images. In this way, the fused images include both the edges of the target object and any occluded target objects. Therefore, the object recognition model trained on the fused images can both identify occluded target objects and reduce the influence of color on recognition.

[0163] For example, assuming that the model training method used in this application embodiment is YOLOX technology, the parameters in the model training can be set as follows: the number of categories is set to 1, the input image size is set to 640*640, the number of iterations (epochs) is set to 300, the batch size is set to 8, and the loss function used in the model training process is consistent with the loss function of YOLOX technology.

[0164] in this case, Figure 12 A schematic diagram of fusing a first edge image and an image to be fused is shown, such as... Figure 12 As shown, after acquiring the first edge image and the image to be fused, the electronic device can determine the sizes of the first edge image and the image to be fused to a preset size (i.e., the input image size of 640*640 mentioned above). Next, the electronic device can convert the first edge image and the image to be fused into a 3D matrix form, meaning that the size of both the first edge image and the image to be fused is 640*640*3. Then, the electronic device fuses the first edge image converted to a 3D matrix and the corresponding image to be fused at the channel level to obtain a fused image. The size of the fused image is 640*640*6.

[0165] Subsequently, the electronic device can input the fused image into the object recognition model and train the object recognition model through the subsequent structure of YOLOX technology, so that the object recognition model can output the category and location of the target object in the image to be detected (i.e., the pixel coordinates of the target object in the image to be detected).

[0166] S1102. For each first edge image, the electronic device extracts features from the first edge image and the corresponding image to be fused, fuses the extracted features, and performs convolution processing on the fused features to obtain multiple fused images.

[0167] Optionally, the electronic device can extract features from the first edge image and the corresponding image to be fused, and process the extracted features using YOLOX technology.

[0168] Specifically, for each of the multiple first edge images, the electronic device can extract features from both the first edge image and its corresponding image to be fused. Next, the electronic device can fuse the features extracted from the first edge image and the features extracted from the image to be fused at the channel level to obtain fused features. Then, the electronic device can perform a convolution operation on the fused features to obtain multiple fused images.

[0169] In this way, the fused image includes both the edges of the target object and the occluded target object. Therefore, the object recognition model trained on the fused image can not only identify occluded target objects, but also reduce the influence of color on recognition.

[0170] For example, assuming the model training method used in this embodiment is YOLOX technology, and feature extraction is implemented through multiple residual modules in the Darknet53 feature extraction network of YOLOX technology, then the parameters in model training can be set as follows: number of classes set to 1, input image size set to 640*640, number of epochs set to 300, batch size set to 8, and the loss function used in model training is consistent with the loss function of YOLOX technology. Feature extraction of electronic devices is implemented through multiple residual modules in the Darknet53 feature extraction network of YOLOX technology.

[0171] in this case, Figure 13 This illustrates yet another method of fusing a first edge image and an image to be fused, such as... Figure 13 As shown, after acquiring the first edge image and the image to be fused, the electronic device can determine the size of the first edge image and the image to be fused to a preset size (i.e., the input image size 640*640 mentioned above). Then, the electronic device inputs the first edge image and the image to be fused into two traditional convolutional modules (Conv Blocks) respectively to obtain two feature images with a size of 640*640*32.

[0172] Next, the electronic device can input two feature images of size 640*640*32 into two first residual blocks respectively to obtain two feature images of size 320*320*64. Then, the electronic device can input the two feature images of size 320*320*64 into two second residual blocks respectively, and the second residual blocks process the input feature images twice to obtain two feature images of size 160*3160*128. Finally, the electronic device can input the two feature images of size 160*160*128 into two third residual blocks respectively, and the third residual blocks process the input feature images eight times to obtain two feature images of size 80*80*256.

[0173] Next, the electronic device can input the two 80*80*256 feature images into the concatenation module. The concatenation module then fuses the two 80*80*256 feature images at the channel level to obtain a single 80*80*512 feature image. Finally, the concatenation module performs a convolution operation on the fused 80*80*512 feature image to obtain a final 80*80*256 image.

[0174] Subsequently, the electronic device will integrate the image input object recognition model and train the object recognition model through the subsequent structure of YOLOX technology, so that the object recognition model can output the category and location of the target object in the image to be detected (i.e., the pixel coordinates of the target object in the image to be detected).

[0175] It should be noted that the electronic device extracts features from the first edge image and the image to be fused separately, but the weights are shared during the feature extraction process. That is, the weights are the same between the two traditional convolutional modules, the same between the two first residual modules, the same between the two second residual modules, and the same between the two third residual modules.

[0176] It should be understood that the YOLOX technology in this application embodiment does not require a data enhancement module: a mosaic module and a mixup module.

[0177] In some embodiments, combined with Figure 11 ,like Figure 14 As shown, after the object recognition model is trained, the object recognition method provided in this application embodiment further includes:

[0178] S1401, The electronic device acquires the image to be detected.

[0179] Specifically, to determine whether a target object exists in an area, the data acquisition device can acquire video data of the area. Then, the data acquisition device can send the acquired video data to an electronic device. The electronic device can receive the video data sent by the data acquisition device and convert it into multiple video frames. Next, the electronic device can convert the multiple video frames into multiple video images, i.e., multiple images to be detected.

[0180] S1402. The electronic device inputs the image to be detected into the object recognition model to obtain the recognition result.

[0181] The identification result is whether the occluded object in the image to be detected is the target object.

[0182] Specifically, after acquiring the image to be detected, the electronic device can input the image into a trained object recognition model to determine whether an occluded object in the image is the target object. In this way, even if the target object is occluded, the electronic device can still identify it, improving the recognition accuracy of the object recognition model.

[0183] It should be noted that when the target object is not present in the image to be detected, the object recognition model does not output the category and location of the target object. When an unoccluded target object exists in the image to be detected, the object recognition model can output the category and location of the unoccluded target object. When the target object in the image to be detected is occluded, the object recognition model can identify the occluded object as the target object and output the category and location of the target object.

[0184] Optionally, after receiving the recognition result, the electronic device can output a prompt message to remind staff.

[0185] Optionally, the prompts for staff can be sent via DingTalk, SMS, email, etc., and this application embodiment does not limit this.

[0186] In some embodiments, the above description mainly focuses on the various steps of the object recognition method provided in this application. The complete flow of the object recognition method provided in this application will be described below in conjunction with the above embodiments. Figure 15 As shown, the object recognition method provided in this application embodiment specifically includes:

[0187] S1501, The electronic device acquires multiple raw images.

[0188] Combination Figure 8 For a description of how electronic devices acquire multiple raw images, please refer to the relevant description of S801, which will not be repeated here.

[0189] S1502, The electronic device converts multiple original images into multiple second edge images using edge detection technology.

[0190] Each of the multiple first edge images includes at least one target object.

[0191] Combination Figure 8 For a description of how electronic devices convert multiple original images into multiple second edge images using edge detection technology, please refer to the relevant description of S802, which will not be repeated here.

[0192] S1503, the electronic device converts the pixel values ​​of pixels other than the target object in each second edge image into second preset values ​​to obtain multiple first edge images.

[0193] Combination Figure 8 The electronic device converts the pixel values ​​of pixels other than the target object in each second edge image into second preset values ​​to obtain multiple first edge images. For a related description, please refer to the relevant description of S803, which will not be repeated here.

[0194] S1504. The electronic device acquires the position of a target pixel with a pixel value of a first preset value in a preset target object.

[0195] The preset target object is at least one of all target objects contained in the multiple first edge images.

[0196] Combination Figure 5 The description of the location of the target pixel with a pixel value of the first preset value in the preset target object obtained by the electronic device can be found in the relevant description of S501, and will not be repeated here.

[0197] S1505. The electronic device determines the size of the occlusion area of ​​the preset target object based on the size of the preset target object.

[0198] Combination Figure 5 The description of how the electronic device determines the size of the occlusion area of ​​the preset target object based on the size of the preset target object can be found in the relevant description of S502, and will not be repeated here.

[0199] S1506. The electronic device determines the position of the target pixel as the center position of the occlusion area, and determines the position of the occlusion area according to the size of the occlusion area.

[0200] The center position of the occluded area is the pixel coordinate of the pixel at the center point of the occluded area.

[0201] Combination Figure 5 The electronic device determines the position of the target pixel as the center of the occlusion area, and determines the position of the occlusion area according to the size of the occlusion area. For related descriptions, please refer to the relevant description of S503, which will not be repeated here.

[0202] S1507. The electronic device determines the location of the occluded area in the original image to which the preset target object belongs based on the location of the occluded area.

[0203] Combination Figure 6 The description of how the electronic device determines the location of the occluded area in the original image to which the preset target object belongs based on the location of the occluded area can be found in the relevant description of S601, and will not be repeated here.

[0204] S1508 The electronic device converts the pixel values ​​of each pixel in the target area into preset pixel values ​​to obtain multiple images to be fused.

[0205] The preset pixel value is the average pixel value of all pixels. The target region is the occluded area in the original image to which the preset target object belongs.

[0206] Combination Figure 6 The electronic device converts the pixel values ​​of each pixel in the target area into preset pixel values ​​to obtain multiple images to be fused. For a related description, please refer to the relevant description of S602, which will not be repeated here.

[0207] S1509. For each first edge image, the electronic device fuses multiple first edge images and the images to be fused corresponding to the multiple first edge images at the channel level to obtain multiple fused images.

[0208] Alternatively, for each first edge image, the electronic device extracts features from the first edge image and the corresponding image to be fused, fuses the extracted features, and performs convolution processing on the fused features to obtain multiple fused images.

[0209] Combination Figure 11 For each first edge image, the electronic device fuses multiple first edge images and the corresponding images to be fused at the channel level to obtain multiple fused images. For a related description, please refer to the relevant description of S1101, which will not be repeated here.

[0210] Combination Figure 11 For each first edge image, the electronic device extracts features from the first edge image and the corresponding image to be fused, fuses the extracted features, and performs convolution processing on the fused features to obtain multiple fused images. For related descriptions, please refer to the relevant description of S1102, which will not be repeated here.

[0211] S1510: The electronic device trains an object recognition model based on multiple fused images.

[0212] Among them, the object recognition model is used to identify whether an occluded object in the image to be detected is the target object.

[0213] Combination Figure 4 For a description of the object recognition model trained by the electronic device based on multiple fused images, please refer to the relevant description of S404, which will not be repeated here.

[0214] S1511. The electronic device acquires the image to be detected.

[0215] Combination Figure 14 For a description of how electronic devices acquire images to be detected, please refer to the description in S1401. It will not be repeated here.

[0216] S1512. The electronic device inputs the image to be detected into the object recognition model to obtain the recognition result.

[0217] Combination Figure 14 The electronic device inputs the image to be detected into the object recognition model to obtain the recognition result. For a related description of the result, please refer to the relevant description of S1402. It will not be repeated here.

[0218] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0219] This application embodiment can divide the object recognition device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0220] like Figure 16 The diagram shown is a structural schematic of an object recognition device provided in an embodiment of this application. This object recognition device can be used to perform... Figure 4 , Figure 5 , Figure 6 , Figure 8 , Figure 11 , Figure 14 Any of the methods for object recognition shown in the examples. Figure 16 The object recognition device shown includes: a processing unit 1601;

[0221] Processing unit 1601 is configured to generate multiple first edge images based on multiple original images; each of the multiple first edge images includes at least one target object. For example, combining... Figure 4 The processing unit 1601 is used to execute S401.

[0222] The processing unit 1601 is further configured to generate multiple images to be fused based on multiple first edge images; the multiple images to be fused include: an image of the target object being occluded and an image of the target object not being occluded. For example, combining... Figure 4 The processing unit 1601 is used to execute S402.

[0223] The processing unit 1601 is further configured to fuse each first edge image with the corresponding image to be fused, thereby obtaining multiple fused images. For example, combining... Figure 4 The processing unit 1601 is used to execute S403.

[0224] The processing unit 1601 is also used to train an object recognition model based on multiple fused images; the object recognition model is used to identify whether an occluded object in the image to be detected is a target object. For example, combining... Figure 4 The processing unit 1601 is used to execute S404.

[0225] Optionally, the processing unit 1601 is specifically used for:

[0226] Obtain the position of a target pixel with a pixel value of a first preset value within a preset target object; the preset target object is at least one target object among all target objects contained in multiple first edge images. For example, combining... Figure 5 The processing unit 1601 is used to execute S501.

[0227] Based on the size of the preset target object, determine the size of the occlusion area of ​​the preset target object. For example, combined with... Figure 5 The processing unit 1601 is used to execute S502.

[0228] The center of the occlusion region is determined by positioning the target pixel, and the position of the occlusion region is determined based on its size. For example, combining... Figure 5 The processing unit 1601 is used to execute S503.

[0229] Based on the location of the occlusion area, the original image to which the preset target object belongs is occluded to obtain multiple images to be fused. For example, combining... Figure 5 The processing unit 1601 is used to execute S504.

[0230] Optionally, the processing unit 1601 is specifically used for:

[0231] Based on the location of the occluded area, determine the location of the occluded area in the original image to which the preset target object belongs. For example, combining... Figure 6 The processing unit 1601 is used to execute S601.

[0232] The pixel values ​​of each pixel in the target region are converted to preset pixel values ​​to obtain multiple images to be fused; the preset pixel values ​​are the average pixel values ​​of each pixel; the target region is the occluded area in the original image to which the preset target object belongs. For example, combining... Figure 6 The processing unit 1601 is used to execute S602.

[0233] Optionally, the processing unit 1601 is specifically used for:

[0234] Acquire multiple raw images; each raw image includes at least one target object. For example, combine... Figure 8 The processing unit 1601 is used to execute S801.

[0235] Multiple original images are converted into multiple second edge images using edge detection techniques. For example, combining... Figure 8 The processing unit 1601 is used to execute S802.

[0236] In each second edge image, the pixel values ​​of pixels other than the target object are converted to second preset values ​​to obtain multiple first edge images. For example, combining... Figure 8 The processing unit 1601 is used to execute S803.

[0237] Optionally, the processing unit 1601 is specifically used for:

[0238] For each first edge image, multiple first edge images and their corresponding images to be fused are fused at the channel level to obtain multiple fused images. For example, combining... Figure 11 The processing unit 1601 is used to execute S1101.

[0239] For each first edge image, features are extracted from both the first edge image and its corresponding image to be fused. The extracted features are then fused, and the fused features are convolved to obtain multiple fused images. For example, combining... Figure 11 The processing unit 1601 is used to execute S1102.

[0240] Optionally, it may also include: acquisition unit 1602;

[0241] The acquisition unit 1602 is used to acquire the image to be detected. For example, combined with... Figure 14 The acquisition unit 1602 is used to execute S1401.

[0242] The processing unit 1601 is further configured to input the image to be detected into an object recognition model to obtain a recognition result; the recognition result is whether the occluded object in the image to be detected is the target object. For example, combined with Figure 14The processing unit 1601 is used to execute S1402.

[0243] This application also provides a computer-readable storage medium, which includes computer-executable instructions. When the computer-executable instructions are run on a computer, the computer performs the object recognition method provided in the above embodiments.

[0244] This application also provides a computer program that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program can implement the object recognition method provided in the above embodiments.

[0245] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0247] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate; components shown as units may be one physical unit or multiple physical units, i.e., they may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0249] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An object recognition method, characterized in that, include: Generate multiple first edge images based on multiple original images; Each of the plurality of first edge images includes at least one target object; Obtain the position of a target pixel with a pixel value of a first preset value in a preset target object; the preset target object is at least one target object among all target objects contained in the plurality of first edge images; The size of the occlusion area of ​​the preset target object is determined based on the size of the preset target object; The position of the target pixel is determined as the center position of the occlusion area, and the position of the occlusion area is determined according to the size of the occlusion area; Based on the location of the occlusion area, the original image to which the preset target object belongs is occluded to obtain multiple images to be fused. The plurality of images to be fused includes: an image of the target object that is occluded and an image of the target object that is not occluded; For each first edge image, the first edge image is fused with the corresponding image to be fused to obtain multiple fused images; An object recognition model is obtained by training the multiple fused images; the object recognition model is used to identify whether an occluded object in the image to be detected is a target object.

2. The object recognition method according to claim 1, characterized in that, The step of occluding the original image to which the preset target object belongs based on the location of the occluded area to obtain the plurality of images to be merged includes: Based on the location of the occluded area, determine the location of the occluded area in the original image to which the preset target object belongs; The pixel values ​​of each pixel in the target region are converted into preset pixel values ​​to obtain the plurality of images to be fused; the preset pixel values ​​are the average pixel values ​​of each pixel; the target region is the occluded region in the original image to which the preset target object belongs.

3. The object recognition method according to claim 1, characterized in that, The step of generating multiple first edge images based on multiple original images includes: The plurality of original images are acquired; each of the plurality of original images includes at least one target object; The multiple original images are converted into multiple second edge images using edge detection technology; In each second edge image, the pixel values ​​of pixels other than the target object are converted into second preset values ​​to obtain the plurality of first edge images.

4. The object recognition method according to claim 1, characterized in that, For each first edge image, the first edge image is fused with the corresponding image to be fused to obtain multiple fused images, including: For each of the first edge images, the plurality of first edge images and the images to be fused corresponding to the plurality of first edge images are fused at the channel level to obtain the plurality of fused images; Alternatively, for each first edge image, feature extraction is performed on the first edge image and the corresponding image to be fused, the extracted features are fused, and the fused features are convolved to obtain the multiple fused images.

5. The object recognition method according to any one of claims 1-4, characterized in that, Also includes: Acquire the image to be detected; The image to be detected is input into the object recognition model to obtain the recognition result; The recognition result is whether the occluded object in the image to be detected is the target object.

6. An object recognition device, characterized in that, include: Processing unit; The processing unit is configured to generate multiple first edge images based on multiple original images; each of the multiple first edge images includes at least one target object. The processing unit is further configured to generate a plurality of images to be fused based on the plurality of first edge images; the plurality of images to be fused include: an image of the target object being occluded and an image of the target object not being occluded; The processing unit is further configured to obtain the position of a target pixel with a pixel value of a first preset value in a preset target object; the preset target object is at least one target object among all target objects contained in the plurality of first edge images; determine the size of the occlusion region of the preset target object according to the size of the preset target object; determine the position of the target pixel as the center position of the occlusion region, and determine the position of the occlusion region according to the size of the occlusion region; and perform occlusion processing on the original image to which the preset target object belongs according to the position of the occlusion region to obtain the plurality of images to be merged. The processing unit is further configured to fuse each first edge image with the image to be fused corresponding to the first edge image to obtain multiple fused images; The processing unit is further configured to train an object recognition model based on the plurality of fused images; the object recognition model is used to identify whether an occluded object in the image to be detected is a target object.

7. The object recognition device according to claim 6, characterized in that, The processing unit is specifically used for: Based on the location of the occluded area, determine the location of the occluded area in the original image to which the preset target object belongs; The pixel values ​​of each pixel in the target region are converted into preset pixel values ​​to obtain the plurality of images to be fused; the preset pixel values ​​are the average pixel values ​​of each pixel; the target region is the occluded region in the original image to which the preset target object belongs.

8. The object recognition device according to claim 6, characterized in that, The processing unit is specifically used for: The plurality of original images are acquired; each of the plurality of original images includes at least one target object; The multiple original images are converted into multiple second edge images using edge detection technology; In each second edge image, the pixel values ​​of pixels other than the target object are converted into second preset values ​​to obtain the plurality of first edge images.

9. The object recognition device according to claim 6, characterized in that, The processing unit is specifically used for: For each of the first edge images, the plurality of first edge images and the images to be fused corresponding to the plurality of first edge images are fused at the channel level to obtain the plurality of fused images; Alternatively, for each first edge image, feature extraction is performed on the first edge image and the corresponding image to be fused, the extracted features are fused, and the fused features are convolved to obtain the multiple fused images.

10. The object recognition device according to any one of claims 6-9, characterized in that, Also includes: Acquisition Unit; The acquisition unit is used to acquire the image to be detected; The processing unit is further configured to input the image to be detected into the object recognition model to obtain a recognition result; the recognition result is whether the occluded object in the image to be detected is the target object.

11. An object recognition device, characterized in that, It includes a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the object recognition device is running, the processor executes the computer execution instructions stored in the memory, so that the object recognition device performs the object recognition method as described in any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the object recognition method as described in any one of claims 1-5.

Images