A target identification method and related device

Abstract

The application provides a target recognition method and related device. The method comprises the following steps: first, obtaining a target image; second, obtaining an attention feature image by performing edge detection on the target image, and obtaining a color feature image by performing color feature extraction on the target image; third, performing feature fusion on the attention feature image and the color feature image to obtain a fused feature image; fourth, performing multi-scale learning on the fused feature image by using a salient feature learning network to generate a labeled image; and fifth, determining position information of a target object in the target image according to the labeled image. The method provided by the application improves the accuracy of target object recognition in the target image through edge detection, color extraction, multi-scale change and other processing methods.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target recognition method and related devices. Background Art

[0002] With the continuous development of computer technology, more and more scenarios require the recognition of moving target objects, for example, the recognition of target objects with moving trajectories in videos, or the recognition of game target objects with moving trajectories in games.

[0003] Traditional moving object recognition techniques, such as the frame difference method, take the difference between two or more frames to generate a difference image. Since the background pixel values ​​are small or zero after subtraction, while the moving target pixel values ​​are large after subtraction, the resulting difference image is binarized to detect the moving target. However, when the target object moves slowly, it is prone to missed detection. When the target object overlaps with other objects, it cannot be accurately identified, resulting in holes and other phenomena. Therefore, the accuracy of target object recognition using the frame difference method is low. Summary of the Invention

[0004] The embodiments of the present application provide a target recognition method and related devices, which improve the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0005] One aspect of the present application provides a target recognition method, comprising:

[0006] Acquire the target image;

[0007] The target image is processed through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, and the edge detection layer is used to identify the contour of the target object in the target image, where N is an integer greater than or equal to 1;

[0008] Extract color features of the target image to obtain a color feature image;

[0009] Fuse N attention feature images with color feature images to generate N fused feature images;

[0010] Perform multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M labeled images corresponding to each fused feature image, where M represents the number of scales for multi-scale learning and is an integer greater than 1.

[0011] The location information of the target object in the target image is determined according to the M annotated images corresponding to each fused feature image.

[0012] Another aspect of the present application provides a target recognition device, comprising:

[0013] A target image acquisition module is used to acquire a target image;

[0014] An edge detection module is configured to process the target image through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, and the edge detection layers are configured to identify the contours of the target object in the target image, where N is an integer greater than or equal to 1;

[0015] A color extraction module is used to extract color features of a target image to obtain a color feature image;

[0016] A feature fusion module is used to fuse N attention feature images with color feature images to generate N fused feature images;

[0017] A salient feature learning module is used to perform multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M labeled images corresponding to each fused feature image, where M represents the number of scales for multi-scale learning and is an integer greater than 1;

[0018] The target object positioning module is used to determine the position information of the target object in the target image according to the M annotated images corresponding to each fused feature image.

[0019] In another implementation of the embodiment of the present application, the salient feature learning network includes a pyramid pooling subnetwork and a salient feature extraction subnetwork, the pyramid pooling subnetwork includes P pooling layers, the input of each pooling layer is the output of the previous pooling layer, and P is an integer greater than 1;

[0020] The salient feature learning module includes a pyramid pooling submodule and a salient feature extraction submodule;

[0021] The pyramid pooling submodule is used to downsample each fused feature image through the pyramid pooling subnetwork to obtain P target pooled images corresponding to each fused feature image;

[0022] The salient feature extraction submodule is used to perform multi-scale learning on P target pooled images through the salient feature extraction subnetwork to obtain M annotated images corresponding to each fused feature image.

[0023] In another implementation of the embodiment of the present application, the salient feature extraction subnetwork includes a salient feature extraction layer and a first target feature annotation layer; the salient feature extraction submodule is further configured to:

[0024] Obtaining a first target pooling image and a second target pooling image from the P target pooling images, wherein the first target pooling image is the target pooling image with the lowest resolution among the P target pooling images, and the resolution of the second target pooling image is greater than the resolution of the first target pooling image;

[0025] Processing the first target pooled image through a salient feature extraction layer to obtain a first salient feature image, wherein the salient feature extraction layer is used to filter the background in the first target pooled image;

[0026] Performing feature fusion on the first target pooled image and the first salient feature image to generate a first fused image;

[0027] encoding the first fused image to generate a first encoded image;

[0028] Perform feature fusion on the second target pooled image and the first encoded image to generate a second fused image;

[0029] The second fused image is processed through the first target feature annotation layer to obtain a first annotated image, wherein the first annotated image includes a first annotated box of the target object.

[0030] In another implementation of the embodiment of the present application, the salient feature extraction submodule is further configured to:

[0031] Obtaining first smoothness information of a first target pooled image;

[0032] The background in the first target pooled image is filtered according to the first smoothness information to generate a first salient feature image.

[0033] In another implementation of the embodiment of the present application, the salient feature extraction submodule is further configured to:

[0034] performing convolution on the first salient feature image to generate a first convolved image;

[0035] Upsampling the first convolution image to generate a first upsampled image;

[0036] The first target pooled image and the first upsampled image are subjected to feature fusion to generate a first fused image.

[0037] In another implementation of the embodiment of the present application, the salient feature extraction submodule is further configured to:

[0038] Upsampling the first encoded image to generate a second upsampled image;

[0039] The second target pooled image is feature-fused with the second upsampled image to generate a second fused image.

[0040] In another implementation of the embodiment of the present application, the salient feature extraction subnetwork further includes a second target feature annotation layer; and the salient feature extraction submodule is further configured to:

[0041] Performing downsampling processing on the first fused image to generate a first downsampled image;

[0042] performing downsampling processing on the second fused image to generate a second downsampled image;

[0043] Performing feature fusion on the first down-sampled image and the second down-sampled image to generate a third fused image;

[0044] The third fused image is processed through the second target feature annotation layer to obtain a second annotated image, wherein the second annotated image includes a second annotated box of the target object.

[0045] In another implementation of the embodiment of the present application, the salient feature extraction sub-network further includes a third target feature annotation layer, a salient feature extraction sub-module, and is further configured to:

[0046] Performing convolution processing on the first salient feature image to generate a first convolution image;

[0047] performing downsampling processing on the obtained third fused image to generate a third downsampled image;

[0048] Performing feature fusion on the first convolution image and the third down-sampled image to generate a fourth fused image;

[0049] The fourth fused image is processed through the third target feature annotation layer to obtain a third annotated image, wherein the third annotated image includes a third annotated box of the target object.

[0050] In another implementation of the embodiment of the present application, the target object positioning module is further configured to:

[0051] Obtain N×M annotated images corresponding to N fused feature images, where each annotated image carries the annotation box of the target object;

[0052] Get N×M brightness information corresponding to N×M labeled images;

[0053] Determine a target annotated image based on N×M brightness information, wherein the target annotated image is the annotated image with the largest brightness information among the N×M annotated images;

[0054] According to the annotation box of the target object in the target annotated image, the position information of the target object in the target image is obtained.

[0055] In another implementation of the embodiment of the present application, the edge detection module is further configured to:

[0056] The contours of the target object in the target image are recognized through N edge detection layers different in N parameters to obtain N background-removed images;

[0057] The N background-removed images are respectively dilated and convolved to generate N attention feature images.

[0058] In another implementation manner of the embodiment of the application, the color extraction module is further configured to:

[0059] The target image is subjected to color feature extraction through a color feature extraction layer in the color feature extraction network to obtain a color feature matrix;

[0060] The color feature matrix is convolved to generate a color convolution matrix;

[0061] The color convolution matrix is down-sampled to generate a color sampling matrix;

[0062] The color feature image is generated according to the color sampling matrix.

[0063] In another implementation manner of the embodiment of the application, the feature fusion module is further configured to:

[0064] N attention feature matrices are generated according to the N attention feature images;

[0065] A color feature matrix is generated according to the color feature image;

[0066] The N attention feature matrices and the color feature matrix are respectively subjected to convolution calculation to generate N fusion feature matrices;

[0067] N fusion feature images are generated according to the N fusion feature matrices.

[0068] Another aspect of the application provides a computer device, comprising:

[0069] a memory, a transceiver, a processor and a bus system;

[0070] The memory is configured to store a program;

[0071] The processor is configured to execute the program in the memory, including executing the method of the above aspects;

[0072] The bus system is configured to connect the memory and the processor to enable the memory and the processor to communicate.

[0073] Another aspect of the application provides a computer readable storage medium, which stores instructions when running on a computer, so that the computer executes the method of the above aspects.

[0074] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided by the above aspects.

[0075] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0076] The present application provides a target recognition method and related devices, which include: first, obtaining a target image; second, processing the target image through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, and the edge detection layer is used to identify the outline of the target object in the target image; and extracting color features of the target image to obtain a color feature image; then, feature fusion of the N attention feature images and the color feature image is performed respectively to generate N fused feature images; then, multi-scale learning is performed on each of the N fused feature images through a salient feature learning network to obtain M labeled images corresponding to each fused feature image, wherein M represents the number of scales for multi-scale learning; finally, determining the position information of the target object in the target image according to the M labeled images corresponding to each fused feature image. The target recognition method provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate an annotated image. The position information of the target object in the target image is determined based on the annotated image. The method provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A schematic diagram of the architecture of a target recognition method system provided in one embodiment of the present application;

[0078] Figure 2 A flowchart of a target recognition method provided in one embodiment of the present application;

[0079] Figure 3 A schematic diagram of a target recognition method provided in one embodiment of the present application;

[0080] Figure 4 A flowchart of a target recognition method provided in another embodiment of the present application;

[0081] Figure 5 A schematic diagram of a salient feature learning network provided in one embodiment of the present application;

[0082] Figure 6 A flowchart of a target recognition method provided in another embodiment of the present application;

[0083] Figure 7 A schematic diagram of a salient feature extraction subnetwork provided in one embodiment of the present application;

[0084] Figure 8 A flowchart of a target recognition method provided in another embodiment of the present application;

[0085] Figure 9 A flowchart of a target recognition method provided in another embodiment of the present application;

[0086] Figure 10 A schematic diagram of a salient feature extraction subnetwork provided in another embodiment of the present application;

[0087] Figure 11 A flowchart of a target recognition method provided in another embodiment of the present application;

[0088] Figure 12 A schematic diagram of a salient feature extraction subnetwork provided in another embodiment of the present application;

[0089] Figure 13 A flowchart of a target recognition method provided in another embodiment of the present application;

[0090] Figure 14 A schematic diagram of a salient feature extraction subnetwork provided in another embodiment of the present application;

[0091] Figure 15 A flowchart of a target recognition method provided in another embodiment of the present application;

[0092] Figure 16 A schematic diagram of a salient feature extraction subnetwork provided in another embodiment of the present application;

[0093] Figure 17 A flowchart of a target recognition method provided in another embodiment of the present application;

[0094] Figure 18 A flowchart of a target recognition method provided in another embodiment of the present application;

[0095] Figure 19 A schematic diagram of an edge detection network provided in one embodiment of the present application;

[0096] Figure 20 A flowchart of a target recognition method provided in another embodiment of the present application;

[0097] Figure 21 A schematic diagram of a color feature extraction network provided in one embodiment of the present application;

[0098] Figure 22 A flowchart of a target recognition method provided in another embodiment of the present application;

[0099] Figure 23 A flowchart of a target recognition method for identifying and positioning a game object provided in another embodiment of the present application;

[0100] Figure 24 A schematic diagram of the structure of a target recognition device provided in one embodiment of the present application;

[0101] Figure 25 A schematic structural diagram of a target recognition device provided in another embodiment of the present application;

[0102] Figure 26 A schematic diagram of the server structure provided for an embodiment of the present application. DETAILED DESCRIPTION

[0103] The embodiment of the present application provides a target recognition method, which improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0104] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0105] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0106] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0107] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying and measuring objects, performing further image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0108] With the continuous advancement of computer technology, more and more scenarios require the identification of moving objects. When identifying moving objects in videos, manually verifying the object's trajectory and position in the video footage is labor-intensive and error-prone. Using frame difference methods for object recognition can easily lead to missed detections when the object moves slowly, and inaccurate identification when the object overlaps with other objects, resulting in gaps in the image.

[0109] In MOBA games (multiplayer online battle arena), it is necessary to identify and locate game objects. This is particularly challenging due to the fact that game objects can easily rotate and change size during gameplay. (For example, multiple game objects can release skills simultaneously or with a small time difference, resulting in variable rotation and partial occlusion of the game objects.) Using frame difference or mean shift target recognition methods for target object identification also makes it difficult to accurately identify the target game object due to the variable rotation and partial occlusion of the game object.

[0110] The target recognition method provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate an annotated image. The position information of the target object in the target image is determined based on the annotated image. The method provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0111] For easier understanding, see Figure 1 , Figure 1 This is an application environment diagram of the target recognition method in the embodiment of the present application, such as Figure 1 As shown, the target recognition method in the embodiment of the present application is applied to the target recognition system. The target recognition system includes: a server and a terminal device; wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, and the embodiment of the present application is not limited here.

[0112] The server first obtains a target image; secondly, processes the target image through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, and the edge detection layer is used to identify the contour of the target object in the target image; and, performs color feature extraction on the target image to obtain a color feature image; then, the N attention feature images are respectively fused with the color feature image to generate N fused feature images; then, multi-scale learning is performed on each of the N fused feature images through a salient feature learning network to obtain M annotated images corresponding to each fused feature image, wherein M represents the number of scales for multi-scale learning; finally, the position information of the target object in the target image is determined according to the M annotated images corresponding to each fused feature image.

[0113] The following describes the target recognition method in this application from the perspective of the server. Figure 2 , the target recognition method provided in the embodiment of the present application includes: step S110 to step S160. It should be noted that the present application does not limit the execution order of step S120 and step S130. Step S120 and step S130 can be executed in parallel, or step S120 can be executed first and then step S130, or step S130 can be executed first and then step S120. The embodiment of the present application takes the execution of step S120 and step S130 in parallel as an example. Specifically:

[0114] S110: Acquire a target image.

[0115] It can be understood that in the scenario of identifying a moving target object in a video, the target image is a video frame image in the target video, and the target image is obtained by extracting frames from the target video. The target image can be obtained by extracting frames from each frame in the target video, or by extracting frames from the target video at a fixed number of frames, for example, extracting video frame images every 5 frames as the target image.

[0116] In the scenario of target identification and position determination of game objects, since the game is in progress, the target image is the game screen at the current time / current moment, and the method provided in the embodiment of the present application needs to identify the target game object in the target image at all times to achieve real-time identification and position determination of the target game object.

[0117] S120. Process the target image through an edge detection network to obtain N attention feature images.

[0118] The edge detection network includes N edge detection layers with different parameters. The edge detection layer is used to identify the contour of the target object in the target image. N is an integer greater than or equal to 1.

[0119] It can be understood that the edge detection network includes N edge detection layers with different parameters, and the target image is used as the input of the N edge detection layers with different parameters. N attention feature images are output through the N edge detection layers with different parameters. Since the parameters of the N edge detection layers are different, the generated attention feature images have different recognition of the target object's contour. The difference in recognition of the target object's contour is mainly reflected in the different information contained in the recognized contour other than the target object. For example, the contour of the target object contains some background information, and the area of ​​the background information contained in the contour of the target object in the N attention feature images is different. Different target object recognition tasks correspond to different target objects.

[0120] S130 , extracting color features from the target image to obtain a color feature image.

[0121] It is understood that color feature extraction is performed on the target image through the color feature extraction network. Since the color information of the target object differs significantly from the background information, color feature extraction can be performed on the target image to obtain a color feature image. In the feature image, the color of the target object is opposite to the color of the background information. The color feature image clearly shows the approximate outline and position of the target object in the target image, similar to a heat map. The color depth at the target object's location is higher than the color depth of non-target objects.

[0122] S140 , respectively perform feature fusion on the N attention feature images and the color feature image to generate N fused feature images.

[0123] It can be understood that the N attention feature images and the color feature images are feature fused respectively. The fusion process of the attention feature image and the color feature image can be matrix multiplication or matrix convolution of the attention feature matrix corresponding to the attention feature image and the color feature matrix corresponding to the color feature image. The embodiment of the present application does not limit this.

[0124] S150 , performing multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M labeled images corresponding to each fused feature image.

[0125] Where M represents the number of scales for multi-scale learning, and M is an integer greater than 1.

[0126] It can be understood that by performing multi-scale learning on the N fused feature images through the salient feature learning network, N×M fused feature images corresponding to the N fused feature images are obtained, each fused feature image corresponds to M fused feature images, and the resolutions of the M fused feature images corresponding to each fused feature image are different.

[0127] S160 : Determine the position information of the target object in the target image according to the M annotated images corresponding to each fused feature image.

[0128] It is understood that each annotated image includes an annotated bounding box of the target object. Determining the target object's position in the target image based on the N×M annotated images corresponding to the N×M fused feature images can involve selecting an annotated image from the N×M annotated images as a reference for determining the target object's position in the target image, i.e., determining the target object's position in the target image based on the annotated bounding box of the selected annotated image.

[0129] For easier understanding, see Figure 3 , Figure 3 A schematic diagram of a target recognition method provided in an embodiment of the present application. The target image is used as the input of an edge detection network and a color feature extraction network respectively. The target image is processed by N edge detection layers with different parameters in the edge detection network to obtain N attention feature images. The target image is processed by a color feature extraction network to obtain a color feature image. The N attention feature images are fused with the color feature image respectively to generate N fused feature images. The N fused feature images are used as the input of a salient feature learning network, and each fused feature image is processed by the salient feature learning network to obtain N fused feature image sets, each of which includes M annotated images. The position information of the target object in the target image is determined based on the M annotated images corresponding to each fused feature image.

[0130] The target recognition method provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate an annotated image. The position information of the target object in the target image is determined based on the annotated image. The method provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0131] In this application Figure 2 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 4 The salient feature learning network includes a pyramid pooling subnetwork and a salient feature extraction subnetwork. The pyramid pooling subnetwork includes P pooling layers. The input of each pooling layer is the output of the previous pooling layer. P is an integer greater than 1. Step S150 further includes sub-steps S151 to S152. Specifically:

[0132] S151. Downsample each fused feature image through the pyramid pooling sub-network to obtain P target pooled images corresponding to each fused feature image.

[0133] It can be understood that the pyramid pooling network can be understood as arranging the P target pooled images obtained by pooling a fused feature image P times in a pyramid shape. The input of the first pooling is the fused feature image, and the input of each subsequent pooling is the output of the previous pooling. It can be understood that it is obtained by step-by-step pooling, and the pooling will not stop until a certain termination condition is reached. The bottom of the pyramid is a high-resolution representation of the fused feature image, and the top is a low-resolution approximation. The embodiment of the present application compares the layers of images to a pyramid. The higher the level, the smaller the image and the lower the resolution. The P target pooled images are the output results of the P pooling layers.

[0134] S152. Perform multi-scale learning on the P target pooled images through the salient feature extraction sub-network to obtain M labeled images corresponding to each fused feature image.

[0135] It can be understood that multi-scale learning is performed on P target pooled images through the salient feature extraction sub-network. Each scale learning is performed on the basis of the previous scale learning. After each scale learning, a labeled image is output. After M scale learning, M labeled images are obtained.

[0136] For easier understanding, see Figure 5 , Figure 5 Schematic diagram of a salient feature learning network provided in an embodiment of the present application. The salient feature learning network includes a pyramid pooling subnetwork and a salient feature extraction subnetwork. In the embodiment of the present application, the pyramid pooling subnetwork includes three pooling layers as an example. The fused feature image is input to the first pooling layer in the pyramid pooling subnetwork. The first pooling layer processes the fused feature image to obtain a first target pooling image. The first target pooling image is input to the second pooling layer and the salient feature extraction subnetwork. The second pooling layer processes the first target pooling image to obtain a second target pooling image. The second target pooling image is input to the third pooling layer and the salient feature extraction subnetwork. The third pooling layer processes the second target pooling image to obtain a third target pooling image. The third target pooling image is input to the salient feature extraction subnetwork. The salient feature extraction subnetwork performs multi-scale learning on the first target pooling image, the second target pooling image, and the third target pooling image to obtain M labeled images corresponding to the fused feature image.

[0137] The target identification method provided in the embodiments of the present application processes the fusion feature image through a salient feature learning network, generates a labeled image, and determines the position information of the target object in the target image according to the labeled image. The method provided in the embodiments of the present application introduces a pyramid pooling subnetwork into the salient feature learning network, which not only strengthens the learning of salient features, but also suppresses the weights of non-salient features. This helps to reduce the interference of pixel dispersion and the background with the appearance of the approximate template, and improves the accuracy of target object identification through the multi-scale change of the salient feature extraction subnetwork.

[0138] In the embodiments of the present application Figure 4 In one optional embodiment of the target identification method provided in the corresponding embodiments, refer to Figure 6 The salient feature extraction subnetwork includes a salient feature extraction layer and a first target feature labeling layer. The substep S152 further includes substep S1521 to substep S1526. Specifically:

[0139] S1521, obtaining a first target pooling image and a second target pooling image from the P target pooling images.

[0140] The first target pooling image is the target pooling image with the lowest resolution among the P target pooling images, and the resolution of the second target pooling image is higher than that of the first target pooling image.

[0141] It can be understood that the first target pooling image and the second target pooling image are generated by the fusion feature image through the pyramid pooling subnetwork. The first target pooling image is the output of the bottommost pooling layer in the pyramid pooling subnetwork. Since it has undergone P times of pooling, the resolution of the first target pooling image is the smallest among the resolutions of all target pooling images. The second target pooling image is the target pooling image other than the first target pooling image among all target pooling images. The second target pooling image has undergone fewer times of pooling than the first target pooling image, so the resolution of the second target pooling image is higher than that of the first target pooling image.

[0142] S1522, processing the first target pooling image through the salient feature extraction layer to obtain a first salient feature image.

[0143] The salient feature extraction layer is used to filter the background in the first target pooling image.

[0144] It can be understood that in the first target pooling image, the feature information of the target object is different from the feature information of the background. The background in the first target pooling image can be filtered out according to the feature information of the target object and the feature information of the background, so as to extract the target object as the first salient feature image.

[0145] The salient feature extraction layer can use the influencing factors of weights to suppress the learning of unnecessary feature weights. It can be regarded as a normalization process and can be calculated by the following formula:

[0146]

[0147] net j =∑ i＝0 (W i ×XB);

[0148] Among them, W i represents the channel attention weight, reflecting the importance of different channels, a i Represents the scaling factor from the forward neural network, if a i is the pixel normalization scaling factor, then W i represents the spatial attention weight, i is the scaling factor, J represents the type of scaling factor, net j is the first salient feature matrix corresponding to the first salient feature image, X represents the first target pooling matrix corresponding to the input first target pooled image, and B represents the bias. During the forward propagation, the bias allows the salient feature extraction layer to better integrate spatial attention and channel attention. In this way, the proposed salient feature extraction subnetwork can improve the ability to discriminate damage features by strengthening the salient feature extraction layer and suppressing the weights of non-salient features.

[0149] S1523: Perform feature fusion on the first target pooled image and the first salient feature image to generate a first fused image.

[0150] It is understood that the first target pooling matrix corresponding to the first target pooling image is matrix-fused with the first salient feature matrix corresponding to the first salient feature image to generate a first fused matrix, and the first fused image is generated based on the first fused matrix. The matrix fusion method can be matrix multiplication or matrix convolution, etc., which is not limited in this embodiment of the present application.

[0151] S1524: Encode the first fused image to generate a first encoded image.

[0152] It can be understood that encoding the first fused image may be performing convolution processing on the first fused image, then performing pooling processing on the result of the convolution processing, and finally performing feature vectorization on the result of the pooling processing to obtain the first encoded image.

[0153] S1525 . Perform feature fusion on the second target pooled image and the first encoded image to generate a second fused image.

[0154] It is understood that the second target pooling matrix corresponding to the second target pooling image is matrix-fused with the first encoding matrix corresponding to the first encoding image to generate a second fused matrix, and the second fused image is generated based on the second fused matrix. The matrix fusion method can be matrix multiplication or matrix convolution, etc., which is not limited in this embodiment of the present application.

[0155] S1526: Process the second fused image through the first target feature annotation layer to obtain a first annotated image.

[0156] The first annotated image includes a first annotated frame of the target object.

[0157] It can be understood that the second fused image is processed by the first target feature annotation layer, and the target object in the second fused image is annotated in the form of a first annotation frame to generate a first annotated image.

[0158] For easier understanding, see Figure 7 , Figure 7 Schematic diagram of the salient feature extraction subnetwork provided by the embodiment of the present application. The embodiment of the present application takes the pyramid pooling subnetwork including two layers of pooling layers as an example, generates a second target pooling image by the pooling layer of the first layer, and the second target pooling image is input to the pooling layer of the second layer to generate a first target pooling image. The first target pooling image undergoes two poolings, and the second target pooling image undergoes one pooling. The resolution of the first target pooling image after two poolings is less than the resolution of the second target pooling image after one pooling. The first target pooling image is input to the salient feature extraction layer, the background in the first target pooling image is filtered by the salient feature extraction layer to generate a first salient feature image, the first target pooling image and the first salient feature image are feature fused to generate a first fused image, the first fused image is encoded to generate a first encoded image, the second target pooling image is feature fused with the first encoded image to generate a second fused image, the second fused image is input to the first target feature annotation layer, and the target object in the second fused image is annotated by the first target feature annotation layer to generate a first annotated image.

[0159] The target recognition method provided in the embodiment of the present application realizes scale change and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0160] In this application Figure 6 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 8 Sub-step S1522 further includes sub-steps S15221 and S15222. Specifically:

[0161] S15221. Obtain first smoothness information of a first target pooled image.

[0162] S15222: Filter the background in the first target pooled image according to the first smoothness information to generate a first salient feature image.

[0163] It can be understood that in the first target pooled image, there is a difference in the smoothness of the target object and the smoothness of the background. The background in the first target pooled image can be filtered out based on the smoothness of the target object and the smoothness of the background in the first smoothness information of the first target pooled image to extract the target object as the first salient feature image.

[0164] The target recognition method provided in the embodiment of the present application improves the accuracy of labeling the target object in the first salient feature image by using the smoothness of the target pooled image as the basis for filtering out the background in the first target pooled image.

[0165] In this application Figure 6 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 9 Sub-step S1523 further includes sub-steps S15231 to S15233. Specifically:

[0166] S15231. Convolve the first salient feature image to generate a first convolution image.

[0167] S15232. Upsample the first convolution image to generate a first upsampled image.

[0168] It can be understood that after the first salient feature image is obtained through the salient feature extraction layer, the first salient feature image is convolved to prepare for scale expansion of the first salient feature image to prevent the first salient feature image from not being smoothed during upsampling. Convolving the first salient feature image is equivalent to smoothing before upsampling to obtain the first convolved image, and then upsampling the first convolved image to expand the scale.

[0169] S15233. Perform feature fusion on the first target pooled image and the first up-sampled image to generate a first fused image.

[0170] It is understood that the first target pooling matrix corresponding to the first target pooled image is matrix-fused with the first upsampling matrix corresponding to the first upsampled image to generate a first fused matrix, and the first fused image is generated based on the first fused matrix. The matrix fusion method can be matrix multiplication or matrix convolution, etc., which is not limited in this embodiment of the present application.

[0171] For easier understanding, see Figure 10 , Figure 10is a schematic diagram of the salient feature extraction subnetwork provided by the embodiment of the present application. The embodiment of the present application takes the pyramid pooling subnetwork including two pooling layers as an example, a second target pooling image is generated through the first layer of the pooling layer, the second target pooling image is input into the second layer of the pooling layer to generate a first target pooling image, the first target pooling image is subjected to twice pooling, and the second target pooling image is subjected to once pooling, the resolution of the first target pooling image subjected to twice pooling is smaller than the resolution of the second target pooling image subjected to once pooling. The first target pooling image is input into the salient feature extraction layer, the background in the first target pooling image is filtered through the salient feature extraction layer to generate a first salient feature image, the first salient feature image is subjected to convolution to generate a first convolution image, the first convolution image is subjected to upsampling to generate a first upsampling image, the first target pooling image and the first upsampling image are subjected to feature fusion to generate a first fusion image, the first fusion image is subjected to encoding to generate a first encoding image, the second target pooling image and the first encoding image are subjected to feature fusion to generate a second fusion image, and the second fusion image is input into the first target feature labeling layer to label the target object in the second fusion image through the first target feature labeling layer to generate a first labeling image.

[0172] The target recognition method provided by the embodiment of the present application realizes scale change and improves the accuracy of labeling the target object through the processes of salient feature extraction, feature fusion, encoding target feature labeling and the like on the target pooling object in the salient feature extraction subnetwork.

[0173] In the Figure 6 In the optional embodiment of the target recognition method provided by the corresponding embodiment, refer to Figure 11 The substep S1525 further includes a substep S15251 to a substep S15252. Specifically:

[0174] S15251, the first encoding image is subjected to upsampling to generate a second upsampling image.

[0175] It can be understood that after the first encoding image is obtained through encoding, the first encoding image is subjected to upsampling to expand the scale to generate a second upsampling image.

[0176] S15252, the second target pooling image and the second upsampling image are subjected to feature fusion to generate a second fusion image.

[0177] It can be understood that the second target pooling matrix corresponding to the second target pooling image and the second upsampling matrix corresponding to the second upsampling image are subjected to matrix fusion to generate a second fusion matrix, and the second fusion image is generated according to the second fusion matrix. The matrix fusion manner can be matrix multiplication or matrix convolution, which is not limited in the embodiment of the present application.

[0178] For easier understanding, see Figure 12 , Figure 12 Schematic diagram of a salient feature extraction subnetwork provided by an embodiment of the present application. The embodiment of the present application takes a pyramid pooling subnetwork including two pooling layers as an example, wherein a second target pooling image is generated by the pooling layer of the first layer, the second target pooling image is input to the pooling layer of the second layer to generate a first target pooling image, the first target pooling image undergoes two poolings, the second target pooling image undergoes one pooling, and the resolution of the first target pooling image after two poolings is less than the resolution of the second target pooling image after one pooling. The first target pooling image is input to the salient feature extraction layer, the background in the first target pooling image is filtered by the salient feature extraction layer to generate a first salient feature image, the first target pooling image and the first salient feature image are feature fused to generate a first fused image, the first fused image is encoded to generate a first encoded image, the first encoded image is upsampled to generate a second upsampled image, the second target pooling image and the second upsampled image are feature fused to generate a second fused image, the second fused image is input to the first target feature annotation layer, the target object in the second fused image is annotated by the first target feature annotation layer to generate a first annotated image.

[0179] It is understandable that the embodiment of the present application is described by taking the generation of one annotated image as an example, but the number of annotated images can be any integer greater than or equal to 1, and the present application does not impose any limitation thereto.

[0180] The target recognition method provided in the embodiment of the present application realizes scale change and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0181] In this application Figure 6 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 13 , the salient feature extraction sub-network also includes a second target feature annotation layer; after sub-step S1523, sub-steps S25231 to S25234 are also included. Specifically:

[0182] S25231. Perform downsampling processing on the first fused image to generate a first downsampled image.

[0183] S25232. Perform downsampling processing on the second fused image to generate a second downsampled image.

[0184] S25233. Perform feature fusion on the first down-sampled image and the second down-sampled image to generate a third fused image.

[0185] It can be understood that the first fusion image and the second fusion image are both down-sampled to generate a first down-sampled image and a second down-sampled image. The first down-sampled matrix corresponding to the first down-sampled image is fused with the second down-sampled matrix corresponding to the second down-sampled image to generate a third fusion matrix, and the third fusion matrix is used to generate a third fusion image. The matrix fusion manner can be matrix multiplication or matrix convolution, which is not limited in the embodiments of the present application.

[0186] S25234, the third fusion image is processed by the second target feature labeling layer to obtain a second labeled image.

[0187] The second labeled image includes a second labeled box of the target object.

[0188] It can be understood that the target object in the third fusion image is labeled in the form of a second labeled box by processing the third fusion image through the second target feature labeling layer to generate a second labeled image. Since the second labeled image is down-sampled twice relative to the first labeled image, the size of the second labeled image is smaller than the size of the first labeled image.

[0189] For ease of understanding, please refer to Figure 14 , Figure 14 is a schematic diagram of a salient feature extraction subnetwork provided by the embodiments of the present application. The embodiments of the present application take a pyramid pooling subnetwork including two pooling layers as an example. A second target pooling image is generated by the first layer of the pooling layer, the second target pooling image is input into the second layer of the pooling layer to generate a first target pooling image, the first target pooling image is processed by twice pooling, and the second target pooling image is processed by once pooling. The resolution of the first target pooling image processed by twice pooling is smaller than the resolution of the second target pooling image processed by once pooling. The first target pooling image is input into a salient feature extraction layer to filter the background in the first target pooling image by the salient feature extraction layer to generate a first salient feature image. The first target pooling image and the first salient feature image are fused to generate a first fusion image. The first fusion image is encoded to generate a first encoded image. The second target pooling image and the first encoded image are fused to generate a second fusion image. The second fusion image is input into the first target feature labeling layer to label the target object in the second fusion image by the first target feature labeling layer to generate a first labeled image. The first fusion image is down-sampled to generate a first down-sampled image, and the second fusion image is down-sampled to generate a second down-sampled image. The first down-sampled image and the second down-sampled image are fused to generate a third fusion image. The third fusion image is processed by the second target feature labeling layer to obtain a second labeled image. The size of the second labeled image is smaller than the size of the first labeled image.

[0190] It is understandable that the embodiment of the present application is described by taking the generation of two annotated images as an example, but the number of annotated images can be any integer greater than or equal to 1, and the present application does not impose any limitation thereto.

[0191] The target recognition method provided in the embodiment of the present application realizes multi-scale changes and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, and encoding target feature labeling on the target pooling object in the salient feature extraction subnetwork.

[0192] In this application Figure 13 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 15 , the salient feature extraction sub-network further includes a third target feature annotation layer, and the target recognition method further includes sub-steps S35231 to S35234. Specifically:

[0193] S35231. Perform convolution processing on the first salient feature image to generate a first convolution image.

[0194] It can be understood that after the first salient feature image is obtained by the salient feature extraction layer, the first salient feature image is smoothed by performing convolution on the first salient feature image.

[0195] S32532. Perform downsampling processing on the obtained third fused image to generate a third down-sampled image.

[0196] S35233. Perform feature fusion on the first convolution image and the third down-sampled image to generate a fourth fused image.

[0197] It is understood that the first convolution matrix corresponding to the first convolution image is feature-fused with the third downsampling matrix corresponding to the third downsampling image to generate a fourth fused matrix, and a fourth fused image is generated based on the fourth fused matrix. The matrix fusion method can be matrix multiplication or matrix convolution, etc., which is not limited in this embodiment of the present application.

[0198] S35234. Process the fourth fused image through the third target feature annotation layer to obtain a third annotated image.

[0199] The third annotated image includes a third annotated frame of the target object.

[0200] It can be understood that the fourth fused image is processed by the third target feature annotation layer, and the target object in the fourth fused image is annotated with a third annotation frame to generate a third annotated image. Because the third annotated image is downsampled once more than the second annotated image, the scale of the third annotated image is smaller than that of the second annotated image.

[0201] For easier understanding, see Figure 16 , Figure 16 Schematic diagram of the salient feature extraction subnetwork provided by the embodiment of the present application. The embodiment of the present application takes the pyramid pooling subnetwork including two pooling layers as an example, generates a second target pooling image by the pooling layer of the first layer, and the second target pooling image is input to the pooling layer of the second layer to generate a first target pooling image. The first target pooling image undergoes two poolings, and the second target pooling image undergoes one pooling. The resolution of the first target pooling image after two poolings is less than the resolution of the second target pooling image after one pooling. The first target pooling image is input to the salient feature extraction layer, the background in the first target pooling image is filtered by the salient feature extraction layer to generate a first salient feature image, the first target pooling image and the first salient feature image are feature fused to generate a first fused image, the first fused image is encoded to generate a first encoded image, the second target pooling image is feature fused with the first encoded image to generate a second fused image, the second fused image is input to the first target feature annotation layer, and the target object in the second fused image is annotated by the first target feature annotation layer to generate a first annotated image. The first fused image is downsampled to generate a first downsampled image, and the second fused image is downsampled to generate a second downsampled image. The first downsampled image and the second downsampled image are feature fused to generate a third fused image. The third fused image is processed through a second target feature annotation layer to generate a second annotated image. The first salient feature image is convolved to generate a first convolved image, and the third fused image is downsampled to generate a third downsampled image. The first convolved image and the third downsampled image are feature fused to generate a fourth fused image. The fourth fused image is processed through a third target feature annotation layer to generate a third annotated image. The scale of the third annotated image is smaller than that of the second annotated image, and the scale of the second annotated image is smaller than that of the first annotated image.

[0202] It is understandable that the embodiment of the present application is described by taking the generation of three annotated images as an example, but the number of annotated images can be any integer greater than or equal to 1, and the present application does not impose any limitation thereto.

[0203] The target recognition method provided in the embodiment of the present application realizes multi-scale changes and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, and encoding target feature labeling on the target pooling object in the salient feature extraction subnetwork.

[0204] In this application Figure 2 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 17 , step S160 further includes sub-steps S161 to S164. Specifically:

[0205] S161. Obtain N×M labeled images corresponding to N fused feature images.

[0206] Each annotated image carries the annotation box of the target object.

[0207] S162 : Obtain N×M pieces of brightness information corresponding to the N×M labeled images.

[0208] S163 : Determine a target annotated image according to the N×M brightness information.

[0209] Among them, the target annotated image is the annotated image with the largest brightness information among the N×M annotated images.

[0210] S164 , obtaining position information of the target object in the target image according to the annotation box of the target object in the target annotated image.

[0211] It can be understood that N×M annotated images correspond to M scales. The purpose of the embodiment of the present application is to find the most suitable scale of the target object. The brightness of the target object corresponding to the M scales is different. The most suitable scale of the target object is the scale corresponding to the maximum brightness of the target object, that is, the target annotated image is the annotated image with the largest brightness information among the N×M annotated images. According to the annotation box of the target object in the target annotated image, the position information of the target object in the target image is obtained.

[0212] The target recognition method provided in the embodiment of the present application realizes multi-scale changes and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, and encoding target feature labeling on the target pooling object in the salient feature extraction subnetwork.

[0213] In this application Figure 2 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 18 , step S120 further includes sub-steps S121 to S122. Specifically:

[0214] S121 , identifying the contour of the target object in the target image through N edge detection layers with different parameters to obtain N background-removed images.

[0215] It can be understood that the target image is processed by N edge detection layers with different parameters to remove the background in the target image and obtain the outline of the target object with the background removed. The edge detection layer can be a Canny edge detector, and the N Canny edge detectors with different parameters refer to Canny edge detectors with inconsistent edge filtering scales.

[0216] S122. Perform dilated convolution on the N background-removed images respectively to generate N attention feature images.

[0217] It's understandable that dilated convolution is performed on N background-removed images. The goal of dilated convolution is to expand the receptive field, better distinguish the target object from the background, and generate N attention feature images. These attention feature images contain interference from the background model and some edge interference that comes into contact with the target object. Dilated convolution can be a dilated convolution, which uses a value of 0 at the center of the convolution kernel. Using dilated convolution can maximize background separation in the target image and facilitate image segmentation of the target object.

[0218] The dilated convolution takes N background-removed images as input. These N background-removed images are extracted after two sets of dilated convolutions with a dilation rate of 2. After softmax calculation, N attention feature images are generated. The attention feature matrix corresponding to the attention feature image can be calculated using the following formula:

[0219] FF=softmax(F ex );

[0220] Among them, FF refers to the attention feature matrix corresponding to the attention feature image, F ex Represents the target matrix corresponding to the target image, and softmax is the softmax calculation.

[0221] See also Figure 19 , Figure 19 This is a schematic diagram of an edge detection network provided by an embodiment of the present application. The edge detection network includes N edge detection layers with different parameters. The target image is used as input to the N edge detection layers with different parameters. The N edge detection layers with different parameters output N background-removed images. The N background-removed images are then dilated and convolved to obtain N attention feature images.

[0222] The target recognition method provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, thereby solving the problem of low target object recognition rate caused by the influence of background motion on the target object motion, and improving the accuracy of target object recognition.

[0223] In this application Figure 2 In an optional embodiment of the target recognition method provided in the corresponding embodiment, please refer to Figure 20 , step S130 further includes sub-steps S131 to S134. Specifically:

[0224] S131. Perform color feature extraction on the target image through the color feature extraction layer in the color feature extraction network to obtain a color feature matrix.

[0225] It can be understood that the color feature extraction layer refers to an RGB channel used for color segmentation, which extracts color features of the target image to obtain a color feature matrix.

[0226] In S132, the color feature matrix is convolved to generate a color convolution matrix.

[0227] In S133, the color convolution matrix is down-sampled to generate a color sampling matrix.

[0228] In S134, the color feature image is generated according to the color sampling matrix.

[0229] It can be understood that the color feature matrix is convolved to generate a color convolution matrix, then the color convolution matrix is down-sampled to generate a color sampling matrix, and finally the color feature image is generated according to the color sampling matrix. The color feature image is a rich feature image of an rgb image.

[0230] For ease of understanding, please refer to Figure 21 , Figure 21 is a schematic diagram of a color feature extraction network provided by the embodiment of the present application. The color feature extraction network includes a color feature extraction layer. First, the target image is input to the color feature extraction layer in the color feature extraction network to obtain a color feature matrix. Then, the color feature matrix is convolved to generate a color convolution matrix. Then, the color convolution matrix is down-sampled to generate a color sampling matrix. Finally, the color feature image is generated according to the color sampling matrix. The color feature matrix corresponding to the color feature image can be calculated by the following formula:

[0231] F I = CONV(RGB, F ex );

[0232] Wherein, F I represents the color feature matrix corresponding to the color feature image, F ex represents the target matrix corresponding to the target image, RGB refers to the RGB segmentation channel, and CONV refers to convolution calculation.

[0233] The target recognition method provided by the embodiment of the present application extracts the color feature of the target image to obtain the color feature image, thereby improving the accuracy of target object recognition.

[0234] In the Figure 2 In one optional embodiment of the target recognition method provided by the corresponding embodiment of the present application, please refer to Figure 22 , step S140 further includes sub-step S141 to sub-step S144. Specifically:

[0235] S141, generating N attention feature matrices according to the N attention feature images.

[0236] S142, generating a color feature matrix according to the color feature image.

[0237] S143, respectively performing convolution calculation on the N attention feature matrices and the color feature matrix to generate N fusion feature matrices.

[0238] S144, generating N fusion feature images according to the N fusion feature matrices.

[0239] It can be understood that the feature fusion of the attention feature image and the color feature image can be matrix fusion of the attention feature matrix corresponding to the attention feature image and the color feature matrix corresponding to the color feature image to generate a fusion feature matrix, and generating a fusion feature image according to the fusion feature matrix. The matrix fusion manner can be matrix multiplication or matrix convolution, and the present application embodiment is not limited here. The fusion feature matrix corresponding to the fusion feature image can be calculated by the following formula:

[0240] F I =CONV(RGB,F ex );

[0241] FF=softmax(F ex );

[0242] F=F I ⊙FF;

[0243] Wherein, F I indicates the color feature matrix corresponding to the color feature image, F ex indicates the target matrix corresponding to the target image, RGB refers to the RGB segmentation channel, FF refers to the attention feature matrix corresponding to the attention feature image, F refers to the fusion feature matrix corresponding to the fusion feature image, CONV refers to convolution calculation, softmax refers to softmax calculation, and refers to convolution calculation.

[0244] After the fusion feature of the above formula, the potential background position information in the edge detection will enter the next fusion feature module FF and will play a role in the salient feature learning. After edge detection, the outline information of the target is roughly described, and after the salient feature learning network, the background can be filtered to the greatest extent, and then after different scale marking, the target object can be further positioned and marked.

[0245] The target identification method provided in the embodiments of the present application obtains an attention feature image through edge detection on a target image, and obtains a color feature image through color feature extraction on the target image, processes a fusion feature image obtained through feature fusion of the attention feature image and the color feature image by a salient feature learning network, generates a labeled image, and determines position information of a target object in the target image according to the labeled image. The method provided in the embodiments of the present application improves the accuracy of target object identification through edge detection, color extraction, multi-scale change and other processing methods.

[0246] For ease of understanding, the following will be combined with Figure 23 A target identification method applied to target identification and position determination of game objects is introduced.

[0247] First, a real-time game screen is acquired as a target image.

[0248] Second, the target image is input into N edge detection layers with different parameters in an edge detection network, N background-removed images are output through the N edge detection layers with different parameters, dilated convolution is performed on the N background-removed images, and N attention feature images are generated.

[0249] In addition, the target image is input into a color feature extraction layer in a color feature extraction network, color feature extraction is performed on the target image through the color feature extraction layer in the color feature extraction network, a color feature matrix is obtained, convolution is performed on the color feature matrix, a color convolution matrix is generated, down-sampling is performed on the color convolution matrix, a color sampling matrix is generated, and a color feature image is generated according to the color sampling matrix.

[0250] Then, N attention feature matrices are generated according to the N attention feature images, a color feature matrix is generated according to the color feature image, convolution calculation is respectively performed on the N attention feature matrices and the color feature matrix, N fusion feature matrices are generated, and N fusion feature images are generated according to the N fusion feature matrices.

[0251] Then, each fused feature image is downsampled using the pyramid pooling subnetwork in the salient feature learning network to obtain P target pooled images corresponding to each fused feature image. The first target pooled image, the second target pooled image, and the first smoothness information of the first target pooled image are obtained from the P target pooled images. The background in the first target pooled image is filtered based on the first smoothness information to generate a first salient feature image. The first salient feature image is convolved to generate a first convolved image. The first convolved image is upsampled to generate a first upsampled image. The first target pooled image and the first upsampled image are fused to generate a first fused image. The first fused image is encoded to generate a first encoded image. The first encoded image is upsampled to generate a second upsampled image. The second target pooled image and the second upsampled image are fused to generate a second fused image. The second fused image is processed through the first target feature annotation layer to obtain a first annotated image of scale 152×152.

[0252] The first fused image is downsampled to generate a first downsampled image. The second fused image is downsampled to generate a second downsampled image. The first downsampled image and the second downsampled image are feature-fused to generate a third fused image. The third fused image is processed through the second target feature annotation layer to obtain a second annotated image with a scale of 76×76.

[0253] The third fused image is downsampled to generate a third downsampled image, and the first convolution image and the third downsampled image are feature fused to generate a fourth fused image. The fourth fused image is processed through the third target feature annotation layer to obtain a third annotated image with a scale of 38×38.

[0254] Finally, the three brightness information corresponding to the first annotated image, the second annotated image, and the third annotated image are obtained. The annotated image corresponding to the maximum brightness information is obtained from the three brightness information. According to the annotation box of the target object in the target annotated image, the position information of the target object in the target image is obtained.

[0255] The target recognition method provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate a labeled image. The position information of the target object in the target image is determined based on the labeled image. The method provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes, and has a good target recognition effect for target recognition and position determination of game objects, with an accuracy rate of more than 92%.

[0256] The target recognition device in this application is described in detail below. Figure 24 . Figure 24 This is a schematic diagram of an embodiment of a target recognition device 10 in an embodiment of the present application. The target recognition device 10 includes:

[0257] The target image acquisition module 110 is used to acquire a target image.

[0258] The edge detection module 120 is used to process the target image through an edge detection network to obtain N attention feature images.

[0259] The edge detection network includes N edge detection layers with different parameters. The edge detection layer is used to identify the contour of the target object in the target image. N is an integer greater than or equal to 1.

[0260] The color extraction module 130 is used to extract color features from the target image to obtain a color feature image.

[0261] The feature fusion module 140 is used to perform feature fusion on the N attention feature images and the color feature image respectively to generate N fused feature images.

[0262] The salient feature learning module 150 is configured to perform multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M labeled images corresponding to each fused feature image.

[0263] Where M represents the number of scales for multi-scale learning, and M is an integer greater than 1;

[0264] The target object positioning module 160 is configured to determine the position information of the target object in the target image according to the M annotated images corresponding to each fused feature image.

[0265] The target recognition device provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate a labeled image. The position information of the target object in the target image is determined based on the labeled image. The device provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0266] In this application Figure 24In an optional embodiment of the target recognition device provided by the corresponding embodiment, the salient feature learning network includes a pyramid pooling subnetwork and a salient feature extraction subnetwork, the pyramid pooling subnetwork includes P pooling layers, the input of each pooling layer is the output of the previous pooling layer, and P is an integer greater than 1. Figure 25 The salient feature learning module 150 includes a pyramid pooling submodule 151 and a salient feature extraction submodule 152 .

[0267] The pyramid pooling submodule 151 is used to downsample each fused feature image through the pyramid pooling subnetwork to obtain P target pooled images corresponding to each fused feature image.

[0268] The salient feature extraction submodule 152 is used to perform multi-scale learning on the P target pooled images through the salient feature extraction subnetwork to obtain M labeled images corresponding to each fused feature image.

[0269] The target recognition device provided in the embodiment of the present application processes a fused feature image through a salient feature learning network to generate an annotated image, and determines the location information of the target object in the target image based on the annotated image. The device provided in the embodiment of the present application introduces a pyramid pooling subnetwork into the salient feature learning network, which not only strengthens the learning of salient features but also suppresses the weight of insignificant features. This helps to reduce interference from pixel dispersion and backgrounds that approximate the appearance of the template, and improves the accuracy of target object recognition by performing multi-scale changes through the salient feature extraction subnetwork.

[0270] In this application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the salient feature extraction submodule 152 is further configured to:

[0271] A first target pooling image and a second target pooling image are obtained from the P target pooling images.

[0272] The first target pooling image is the target pooling image with the lowest resolution among the P target pooling images, and the resolution of the second target pooling image is greater than the resolution of the first target pooling image.

[0273] The first target pooled image is processed by the salient feature extraction layer to obtain a first salient feature image.

[0274] The salient feature extraction layer is used to filter the background in the first target pooled image.

[0275] The first target pooled image and the first salient feature image are subjected to feature fusion to generate a first fused image.

[0276] The first fused image is encoded to generate a first encoded image.

[0277] The second target pooled image is feature-fused with the first encoded image to generate a second fused image.

[0278] The second fused image is processed through the first target feature annotation layer to obtain a first annotated image.

[0279] The first annotated image includes a first annotated frame of the target object.

[0280] The target recognition device provided in the embodiment of the present application realizes scale change and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0281] In this application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the salient feature extraction submodule 152 is further configured to:

[0282] Obtain first smoothness information of the first target pooled image.

[0283] The background in the first target pooled image is filtered according to the first smoothness information to generate a first salient feature image.

[0284] The target recognition device provided in the embodiment of the present application improves the accuracy of labeling the target object in the first salient feature image by using the smoothness of the target pooled image as the basis for filtering out the background in the first target pooled image.

[0285] In this application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the salient feature extraction submodule 152 is further configured to:

[0286] Convolution is performed on the first salient feature image to generate a first convolved image.

[0287] The first convolution image is upsampled to generate a first upsampled image.

[0288] The first target pooled image and the first upsampled image are subjected to feature fusion to generate a first fused image.

[0289] The target recognition device provided in the embodiment of the present application realizes scale change and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0290] In this application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the salient feature extraction submodule 152 is further configured to:

[0291] The first encoded image is up-sampled to generate a second up-sampled image.

[0292] The second target pooling image and the second up-sampled image are fused to generate a second fused image.

[0293] The target recognition device provided by the embodiments of the present application realizes scale change and improves the accuracy of target object labeling through the processes of significant feature extraction, feature fusion, and encoding target feature labeling on the target pooling object in the significant feature extraction subnetwork.

[0294] In the embodiments of the present application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiments, the significant feature extraction subnetwork further includes a second target feature labeling layer; and the significant feature extraction submodule 152 is further configured to:

[0295] The first fused image is down-sampled to generate a first down-sampled image.

[0296] The second fused image is down-sampled to generate a second down-sampled image.

[0297] The first down-sampled image and the second down-sampled image are fused to generate a third fused image.

[0298] The third fused image is processed by the second target feature labeling layer to obtain a second labeled image.

[0299] The second labeled image includes a second labeled frame of the target object.

[0300] The target recognition device provided by the embodiments of the present application realizes multi-scale change and improves the accuracy of target object labeling through the processes of significant feature extraction, feature fusion, and encoding target feature labeling on the target pooling object in the significant feature extraction subnetwork.

[0301] In the embodiments of the present application Figure 25 In an optional embodiment of the target recognition device provided by the corresponding embodiments, the significant feature extraction subnetwork further includes a third target feature labeling layer, and the significant feature extraction submodule 152 is further configured to:

[0302] The first significant feature image is convoluted to generate a first convoluted image.

[0303] The third fused image is down-sampled to generate a third down-sampled image.

[0304] The first convoluted image and the third down-sampled image are fused to generate a fourth fused image.

[0305] The fourth fused image is processed through the third target feature annotation layer to obtain a third annotated image.

[0306] The third annotated image includes a third annotated frame of the target object.

[0307] The target recognition device provided in the embodiment of the present application realizes multi-scale changes and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0308] In this application Figure 24 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the target object positioning module 160 is further configured to:

[0309] Obtain N×M labeled images corresponding to N fused feature images.

[0310] Each annotated image carries the annotation box of the target object.

[0311] Get N×M brightness information corresponding to N×M labeled images.

[0312] The target annotation image is determined based on N×M brightness information.

[0313] Among them, the target annotated image is the annotated image with the largest brightness information among the N×M annotated images.

[0314] According to the annotation box of the target object in the target annotation image, the position information of the target object in the target image is obtained.

[0315] The target recognition device provided in the embodiment of the present application realizes multi-scale changes and improves the accuracy of target object labeling by performing salient feature extraction, feature fusion, encoding target feature labeling and other processes on the target pooling object in the salient feature extraction subnetwork.

[0316] In this application Figure 24 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the edge detection module 120 is further configured to:

[0317] The contour of the target object in the target image is identified through N edge detection layers with different parameters to obtain N background-removed images.

[0318] Dilated convolution is performed on N background-removed images to generate N attention feature images.

[0319] The target recognition device provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, thereby solving the problem of low target object recognition rate caused by the influence of background motion on the target object motion, and improving the accuracy of target object recognition.

[0320] In this application Figure 24 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the color extraction module 130 is further configured to:

[0321] The color feature extraction layer in the color feature extraction network is used to extract the color features of the target image and obtain a color feature matrix.

[0322] Convolve the color feature matrix to generate a color convolution matrix.

[0323] Downsample the color convolution matrix to generate a color sampling matrix.

[0324] Generate a color feature image based on the color sampling matrix.

[0325] The target recognition device provided in the embodiment of the present application obtains a color feature image by extracting color features from the target image, thereby improving the accuracy of target object recognition.

[0326] In this application Figure 24 In an optional embodiment of the target recognition device provided by the corresponding embodiment, the feature fusion module 140 is further configured to:

[0327] Generate N attention feature matrices based on N attention feature images.

[0328] Generate a color feature matrix based on the color feature image.

[0329] Convolution calculations are performed on the N attention feature matrices and the color feature matrix respectively to generate N fusion feature matrices.

[0330] Generate N fused feature images according to the N fused feature matrices.

[0331] The target recognition device provided in the embodiment of the present application obtains an attention feature image by performing edge detection on the target image, and obtains a color feature image by performing color feature extraction on the target image. The fused feature image after feature fusion of the attention feature image and the color feature image is processed by a salient feature learning network to generate a labeled image. The position information of the target object in the target image is determined based on the labeled image. The device provided in the embodiment of the present application improves the accuracy of target object recognition through multiple processing methods such as edge detection, color extraction, and multi-scale changes.

[0332] Figure 26: This is a schematic diagram of a server structure provided in an embodiment of the present application. The server 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 322 (for example, one or more processors) and memories 332, and one or more storage media 330 (for example, one or more massive storage devices) for storing application programs 342 or data 344. Among them, the memories 332 and the storage media 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 322 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the server 300.

[0333] The server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input and output interfaces 358, and / or one or more operating systems 341, such as Windows Server 2003 or Windows Server 2003R. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0334] The steps performed by the server in the above embodiment can be based on the Figure 26 The server structure shown.

[0335] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0336] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0337] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0338] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0339] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0340] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target recognition method, characterized in that: include: Acquire the target image; Processing the target image through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, the edge detection layers are used to identify the contour of the target object in the target image, and N is an integer greater than or equal to 1; Performing color feature extraction on the target image to obtain a color feature image; Fusing the N attention feature images with the color feature image to generate N fused feature images; Performing multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M annotated images corresponding to each fused feature image, where M represents the number of scales for the multi-scale learning and is an integer greater than 1; The position information of the target object in the target image is determined according to the M annotated images corresponding to each fused feature image.

2. The target recognition method according to claim 1, wherein: The salient feature learning network includes a pyramid pooling subnetwork and a salient feature extraction subnetwork, wherein the pyramid pooling subnetwork includes P pooling layers, the input of each pooling layer is the output of the previous pooling layer, and P is an integer greater than 1; The performing multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M annotated images corresponding to each fused feature image includes: Downsampling each fused feature image through the pyramid pooling subnetwork to obtain P target pooled images corresponding to each fused feature image; Multi-scale learning is performed on the P target pooled images through the salient feature extraction subnetwork to obtain M labeled images corresponding to each fused feature image.

3. The target recognition method according to claim 2, wherein: The salient feature extraction subnetwork includes a salient feature extraction layer and a first target feature annotation layer; The performing multi-scale learning on the P target pooled images by the salient feature extraction sub-network to obtain M annotated images corresponding to each fused feature image includes: Obtaining a first target pooling image and a second target pooling image from the P target pooling images, wherein the first target pooling image is the target pooling image with the lowest resolution among the P target pooling images, and the resolution of the second target pooling image is greater than the resolution of the first target pooling image; Processing the first target pooled image through the salient feature extraction layer to obtain a first salient feature image, wherein the salient feature extraction layer is used to filter the background in the first target pooled image; Performing feature fusion on the first target pooled image and the first salient feature image to generate a first fused image; encoding the first fused image to generate a first encoded image; Performing feature fusion on the second target pooled image and the first encoded image to generate a second fused image; The second fused image is processed through a first target feature annotation layer to obtain a first annotated image, wherein the first annotated image includes a first annotated box of the target object.

4. The target recognition method according to claim 3, wherein: The step of processing the first target pooled image by the salient feature extraction layer to obtain a first salient feature image includes: Obtaining first smoothness information of the first target pooled image; The background in the first target pooled image is filtered according to the first smoothness information to generate the first salient feature image.

5. The target recognition method according to claim 3, wherein: The step of performing feature fusion on the first target pooled image and the first salient feature image to generate a first fused image includes: performing convolution on the first salient feature image to generate a first convolved image; Upsampling the first convolution image to generate a first upsampled image; Perform feature fusion on the first target pooled image and the first upsampled image to generate a first fused image.

6. The target recognition method according to claim 3, wherein: The step of fusing features of the second target pooled image with the first encoded image to generate a second fused image includes: Upsampling the first encoded image to generate a second upsampled image; Perform feature fusion on the second target pooled image and the second up-sampled image to generate a second fused image.

7. The target recognition method according to claim 3, wherein: The salient feature extraction subnetwork also includes a second target feature annotation layer; After generating the first fused image, the method further includes: performing downsampling processing on the first fused image to generate a first downsampled image; performing downsampling processing on the second fused image to generate a second downsampled image; Performing feature fusion on the first down-sampled image and the second down-sampled image to generate a third fused image; The third fused image is processed through the second target feature annotation layer to obtain a second annotated image, wherein the second annotated image includes a second annotated box of the target object.

8. The target recognition method according to claim 7, wherein: The salient feature extraction sub-network also includes a third target feature annotation layer, After the first target pooled image is processed by the salient feature extraction layer to obtain a first salient feature image, the method further includes: Performing convolution processing on the first salient feature image to generate a first convolution image; After obtaining the third fused image, the method further includes: performing downsampling processing on the obtained third fused image to generate a third downsampled image; Performing feature fusion on the first convolution image and the third down-sampled image to generate a fourth fused image; The fourth fused image is processed through the third target feature annotation layer to obtain a third annotated image, wherein the third annotated image includes a third annotated box of the target object.

9. The target recognition method according to claim 1, wherein: The determining the position information of the target object in the target image according to the M annotated images corresponding to each fused feature image includes: Obtain N×M annotated images corresponding to the N fused feature images, wherein each of the annotated images carries an annotated box of the target object; Obtaining N×M brightness information corresponding to the N×M labeled images; Determine a target annotated image based on the N×M brightness information, wherein the target annotated image is an annotated image with the largest brightness information among the N×M annotated images; According to the annotation box of the target object in the target annotated image, position information of the target object in the target image is obtained.

10. The target recognition method according to claim 1, wherein: The target image is processed by the edge detection network to obtain N attention feature images, including: Recognizing the contour of the target object in the target image through the N edge detection layers with different parameters to obtain N background-removed images; Perform dilated convolution on the N background-removed images respectively to generate N attention feature images.

11. The target recognition method according to claim 1, wherein: The step of extracting color features from the target image to obtain a color feature image includes: Performing color feature extraction on the target image through a color feature extraction layer in a color feature extraction network to obtain a color feature matrix; Convolving the color feature matrix to generate a color convolution matrix; Downsampling the color convolution matrix to generate a color sampling matrix; A color feature image is generated according to the color sampling matrix.

12. The target recognition method according to claim 1, wherein: The step of fusing the N attention feature images with the color feature image to generate N fused feature images comprises: Generating N attention feature matrices according to the N attention feature images; generating a color feature matrix according to the color feature image; Perform convolution calculations on the N attention feature matrices and the color feature matrix respectively to generate N fusion feature matrices; The N fused feature images are generated according to the N fused feature matrices.

13. A target recognition device, characterized in that: include: A target image acquisition module is used to acquire a target image; an edge detection module, configured to process the target image through an edge detection network to obtain N attention feature images, wherein the edge detection network includes N edge detection layers with different parameters, the edge detection layers are configured to identify the contour of the target object in the target image, and N is an integer greater than or equal to 1; A color extraction module is used to extract color features of the target image to obtain a color feature image; A feature fusion module, configured to perform feature fusion on the N attention feature images and the color feature image to generate N fused feature images; a salient feature learning module, configured to perform multi-scale learning on each of the N fused feature images through a salient feature learning network to obtain M annotated images corresponding to each fused feature image, where M represents the number of scales for the multi-scale learning and is an integer greater than 1; The target object positioning module is used to determine the position information of the target object in the target image according to the M labeled images corresponding to each fused feature image.

14. A computer device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is configured to execute the program in the memory, including executing the target recognition method according to any one of claims 1 to 12; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other. 15 . A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to execute the target recognition method according to claim 1 .

16. A computer program product comprising a computer program, characterized in that The computer program is used by a processor to execute the target recognition method according to any one of claims 1 to 12.

Images