Target detection method and device, and storage medium

Abstract

The application discloses a target detection method and device and a storage medium. The target detection method comprises the following steps: receiving a target image; inputting the target image into an attention feature fusion multi-scale single-shot detection (AFF-SSD) network structure to obtain a plurality of feature maps and at least one attention feature fusion map, wherein the AFF-SSD network structure is constructed based on a multi-scale single-shot detection (SSD) basic network structure, and the AFF-SSD network structure fuses at least two feature maps to obtain the attention feature fusion map; and performing feature recognition on the target image based on the attention feature fusion map and the feature maps. The method can realize accurate detection of small object targets.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically, to a target detection method, apparatus, and storage medium. Background Technology

[0002] In today's intelligent lifestyle, numerous smart products have emerged, and human-computer interaction has consistently been a primary research focus. From initial button interaction to touch interaction, and then to sensor-based interaction, the operation of intelligent products has become increasingly convenient and flexible. With the rise of artificial intelligence technology, target recognition and detection technologies have been widely applied in various interactive products. Facial recognition, gesture recognition and detection, fingerprint recognition and positioning, as well as various dynamic or static gestures, are all widely used in human-computer interaction.

[0003] In computer vision, a typical field of human-computer interaction, the development of deep learning has made the use of deep convolutional neural networks to identify targets in images a hot research topic. In 2016, Liu et al. proposed the Single Shot MultiBox Detector (SSD) algorithm at the ECCV conference. This algorithm extracts features using feature maps at different scales and uses regression to obtain the position and classification results of the target object. Compared with other target detection algorithms, this algorithm improves detection accuracy while meeting real-time requirements.

[0004] However, the SSD algorithm has the drawback of poor detection performance for small objects, which greatly limits its application. For example, in applications such as gesture recognition, fingerprint recognition, and face recognition, the detection accuracy of the SSD algorithm is not high.

[0005] Therefore, it is desirable to provide an improved target detection method to solve the above problems. Summary of the Invention

[0006] In view of the above problems, the purpose of the present invention is to provide a target detection method, apparatus and storage medium, thereby improving the detection accuracy of small object targets.

[0007] According to a first aspect of the present invention, a target detection method is provided, comprising:

[0008] Receive the target image;

[0009] The target image is input into an attention feature fusion multi-scale single-shot detection (AFF-SSD) network structure to obtain multiple feature maps and at least one attention feature fusion map. The AFF-SSD network structure fuses at least two of the feature maps to obtain the attention feature fusion map. The AFF-SSD network structure is built upon the basic multi-scale single-shot detection (SSD) network structure.

[0010] Feature recognition is performed on the target image based on the attention feature fusion map and the feature map.

[0011] Optionally, fusing at least two feature maps to obtain the attention feature fusion map includes:

[0012] The Conv4_3 layer and the Fc7 layer are fused to obtain a new Conv4_3 layer.

[0013] The Fc7 layer and the Conv8_2 layer are fused to obtain a new Fc7 layer.

[0014] The new Conv4_3 layer and the new Fc7 layer are respectively used as the attention feature fusion map.

[0015] Optionally, a method for feature fusion of at least two feature maps includes:

[0016] The nth feature map in the AFF-SSD network structure is processed to obtain the spatial attention map of the nth feature map, where n is a natural number;

[0017] The (n+1)th feature map in the AFF-SSD network structure is processed to obtain the channel attention map of the (n+1)th feature map; and

[0018] Based on the feature enhancement method, the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map are fused to obtain an initial fused map.

[0019] Wherein, the nth feature map is the feature map output by Conv4_3 layer, the (n+1)th feature map is the feature map output by Fc7 layer, or the nth feature map is the feature map output by Fc7 layer, and the (n+1)th feature map is the feature map output by Conv8_2 layer.

[0020] Optionally, the method for fusing at least two feature maps further includes: fusing the initial fusion map with the nth feature map to obtain the attention feature fusion map.

[0021] Optionally, the method for obtaining the spatial attention map of the nth feature map includes:

[0022] The nth feature map is convolved to obtain the first matrix;

[0023] The nth feature map is convolved to obtain the second matrix;

[0024] The first matrix is ​​rearranged in dimensions to obtain the third matrix;

[0025] The second matrix is ​​rearranged in dimensions and transposed to obtain the fourth matrix;

[0026] Multiply the third and fourth matrices to obtain the first feature map matrix;

[0027] The first feature map matrix is ​​normalized along the column direction; and

[0028] The first feature map matrix after normalization is rearranged in dimensions to obtain the spatial attention map of the nth feature map.

[0029] Optionally, the method for obtaining the channel attention map of the (n+1)th feature map includes:

[0030] Convolve the (n+1)th feature map to obtain the fifth matrix;

[0031] Convolve the (n+1)th feature map to obtain the sixth matrix;

[0032] The fifth matrix is ​​rearranged in dimensions and transposed to obtain the seventh matrix;

[0033] The sixth matrix is ​​rearranged in dimensions to obtain the eighth matrix;

[0034] Multiply the seventh and eighth matrices to obtain the second feature map matrix;

[0035] The second feature map matrix is ​​normalized along the column direction; and

[0036] The normalized second feature map matrix is ​​rearranged in dimensions to obtain the channel attention map of the (n+1)th feature map.

[0037] According to a second aspect of the present invention, a target detection device is provided, comprising:

[0038] The receiving module receives the target image;

[0039] The feature extraction module inputs the target image into an attention feature fusion multi-scale single-shot detection (AFF-SSD) network structure to obtain multiple feature maps and at least one attention feature fusion map. The AFF-SSD network structure fuses at least two of the feature maps to obtain the attention feature fusion map. The AFF-SSD network structure is built upon the basic multi-scale single-shot detection (SSD) network structure.

[0040] The recognition module performs feature recognition on the target image based on the attention feature fusion map and the feature map.

[0041] Optionally, the AFF-SSD network structure includes two attention feature fusion modules.

[0042] One of the attention feature fusion modules fuses the features of the Conv4_3 layer and the Fc7 layer to obtain a new Conv4_3 layer.

[0043] Another attention feature fusion module fuses the features of the Fc7 layer and the Conv8_2 layer to obtain a new Fc7 layer.

[0044] The new Conv4_3 layer and the new Fc7 layer serve as the attention feature fusion map, respectively.

[0045] Optionally, the attention feature fusion module includes:

[0046] The spatial attention module processes the nth feature map in the AFF-SSD network structure to obtain the spatial attention map of the nth feature map, where n is a natural number;

[0047] The channel attention module processes the (n+1)th feature map in the AFF-SSD network structure to obtain the channel attention map of the (n+1)th feature map; and

[0048] The fusion module fuses the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map using a feature enhancement method to obtain an initial fused map.

[0049] Wherein, the nth feature map is the feature map output by Conv4_3 layer, the (n+1)th feature map is the feature map output by Fc7 layer, or the nth feature map is the feature map output by Fc7 layer, and the (n+1)th feature map is the feature map output by Conv8_2 layer.

[0050] Optionally, the attention feature fusion module further includes: a splicing module, which performs feature fusion between the initial fusion map and the nth feature map to obtain the attention feature fusion map.

[0051] Optionally, the spatial attention module includes:

[0052] The first convolutional layer performs convolution on the nth feature map to obtain the first matrix;

[0053] The second convolutional layer performs convolution on the nth feature map to obtain the second matrix;

[0054] The first Reshape layer rearranges the dimensions of the first matrix to obtain the third matrix;

[0055] The second reshape layer rearranges the dimensions of the second matrix and transposes the matrix to obtain the fourth matrix;

[0056] The first multiplication module multiplies the third and fourth matrices to obtain the feature map matrix;

[0057] The first Softmax layer normalizes the feature map matrix along the column direction; and

[0058] The third Reshape layer rearranges the dimensions of the normalized feature map matrix to obtain the spatial attention map of the nth feature map.

[0059] Optionally, the channel attention module includes:

[0060] The third convolutional layer convolves the (n+1)th feature map to obtain the fifth matrix;

[0061] The fourth convolutional layer performs convolution on the (n+1)th feature map to obtain the sixth matrix;

[0062] The fourth reshape layer performs dimensional rearrangement and matrix transpose on the fifth matrix to obtain the seventh matrix;

[0063] The fifth reshape layer rearranges the dimensions of the sixth matrix to obtain the eighth matrix;

[0064] The second multiplication module multiplies the seventh and eighth matrices to obtain the second feature map matrix;

[0065] The second Softmax layer normalizes the second feature map matrix in the column direction; and

[0066] The sixth Reshape layer rearranges the dimensions of the normalized second feature map matrix to obtain the channel attention map of the (n+1)th feature map.

[0067] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that is executed by a processor to implement the method described above.

[0068] The target detection method, apparatus, and storage medium provided by this invention enhance the contextual information of the network by fusing feature maps in the AFF-SSD network structure, enabling different feature maps to complement each other and improving the correlation between distant pixels. This method can be applied to target recognition in scenarios such as human-computer interaction, and shows good detection performance for gesture recognition, fingerprint recognition, and face recognition.

[0069] In this technical solution, the SSD basic network structure mainly consists of Conv4_3, FC7, and Conv8_2 layers to detect small objects. Compared to Conv4_3, the FC7 layer has more semantic information, while the Conv4_3 layer has more feature information; similarly, compared to the FC7 layer, the Conv8_2 layer has more semantic information, while the FC7 layer has more feature information. Therefore, adding a channel attention module to the FC7 layer can extract more useful semantic information, and adding a spatial attention module to the Conv4_3 layer can enhance the extraction of important information and reduce the impact of noise. Then, the features from the two attention maps are fused, which not only extracts more useful feature information but also more semantic information, thus enhancing the detection capability for small objects. Attached Figure Description

[0070] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0071] Figure 1 A flowchart of a target detection method according to an embodiment of the present invention is shown;

[0072] Figure 2 A block diagram of a target detection device according to an embodiment of the present invention is shown;

[0073] Figure 3 A schematic diagram of the AFF-SSD network structure according to an embodiment of the present invention is shown;

[0074] Figure 4 A schematic diagram of an attention feature fusion module according to an embodiment of the present invention is shown;

[0075] Figure 5 A schematic diagram of a spatial attention module according to an embodiment of the present invention is shown;

[0076] Figure 6 A schematic diagram of a channel attention module according to an embodiment of the present invention is shown. Detailed Implementation

[0077] The present invention is described below based on embodiments, but the invention is not limited to these embodiments. In the detailed description of the invention below, certain specific details are described in detail. Those skilled in the art will fully understand the invention even without these details. To avoid obscuring the essence of the invention, well-known methods, processes, and procedures are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the possible system framework, functions, and operations of the methods and apparatus of embodiments of the present invention. The blocks in the flowcharts and block diagrams may represent a module, program segment, or simply a piece of code. These modules, program segments, and code are all executable instructions used to implement a specified logical function. It should also be noted that the executable instructions implementing the specified logical function can be recombined to generate new modules and program segments. Therefore, the blocks and their order in the accompanying drawings are only used to better illustrate the processes and steps of the embodiments and should not be construed as limiting the invention itself.

[0079] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0080] Figure 1 A flowchart of a target detection method according to an embodiment of the present invention is shown; Figure 3 A schematic diagram of the AFF-SSD network structure according to an embodiment of the present invention is shown; Figure 4 A schematic diagram of an attention feature fusion module according to an embodiment of the present invention is shown; Figure 5 A schematic diagram of a spatial attention module according to an embodiment of the present invention is shown; Figure 6 A schematic diagram of a channel attention module according to an embodiment of the present invention is shown.

[0081] like Figure 1 As shown, the target detection method includes steps S1 to S3.

[0082] In step S1, a target image is received. The target image has, for example, 300×300 pixels.

[0083] In step S2, the target image is input into the Attention Feature Fusion Single Shot MultiBox Detector (AFF-SSD) network structure to obtain multiple feature maps and at least one attention feature fusion map. The AFF-SSD network structure fuses at least two feature maps to obtain the attention feature fusion map. The AFF-SSD network structure is built on the SSD multi-scale single-shot detection network structure.

[0084] As an example, please refer to Figure 3 In each object detection step, feature fusion is performed twice. In each feature fusion, the two feature maps are fused to obtain an attention feature fusion map. Specifically, the Conv4_3 layer and the Fc7 layer are fused to obtain a new Conv4_3 layer; the Fc7 layer and the Conv8_2 layer are fused to obtain a new Fc7 layer; the new Conv4_3 layer and the new Fc7 layer are used as attention feature fusion maps respectively.

[0085] Optionally, the method for fusing at least two feature maps includes: processing the nth feature map in the AFF-SSD network structure to obtain a spatial attention map of the nth feature map, where n is a natural number; processing the (n+1)th feature map in the AFF-SSD network structure to obtain a channel attention map of the (n+1)th feature map; and fusing the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map based on a feature enhancement method to obtain an initial fused map. Optionally, it further includes: fusing the initial fused map with the nth feature map to obtain an attention feature fused map.

[0086] Optionally, the method for obtaining the spatial attention map of the nth feature map includes: convolving the nth feature map to obtain a first matrix; convolving the nth feature map to obtain a second matrix; rearranging the dimensions of the first matrix to obtain a third matrix; rearranging the dimensions of the second matrix and transposing the matrix to obtain a fourth matrix; multiplying the third matrix and the fourth matrix to obtain a first feature map matrix; normalizing the first feature map matrix in the column direction; and rearranging the dimensions of the normalized first feature map matrix to obtain the spatial attention map of the nth feature map.

[0087] Optionally, the method for obtaining the channel attention map of the (n+1)th feature map includes: convolving the (n+1)th feature map to obtain the fifth matrix; convolving the (n+1)th feature map to obtain the sixth matrix; performing dimensional rearrangement and matrix transpose on the fifth matrix to obtain the seventh matrix; performing dimensional rearrangement on the sixth matrix to obtain the eighth matrix; multiplying the seventh and eighth matrices to obtain the second feature map matrix; normalizing the second feature map matrix in the column direction; and performing dimensional rearrangement on the normalized second feature map matrix to obtain the channel attention map of the (n+1)th feature map.

[0088] In step S3, at least one attention feature fusion map and feature map obtained based on the AFF-SSD network structure are used to perform feature recognition on the target image.

[0089] The target detection method according to the present invention can be deployed on one or more servers. For example, different modules can be deployed on different servers to form a dedicated server. Alternatively, the same functional units, modules, or systems can be distributed across multiple servers to reduce load pressure. The servers include, but are not limited to, multiple PCs, PC servers, blade servers, supercomputers, etc., connected on the same local area network or via the Internet.

[0090] Figure 2 A block diagram of a target detection device according to an embodiment of the present invention is shown.

[0091] like Figure 2 As shown, the present invention also provides a target detection device 100, which includes: a receiving module 110 for receiving a target image; a feature extraction module 120 for inputting the target image into an attention feature fusion multi-scale single-shot detection (AFF-SSD) network structure to obtain multiple feature maps and at least one attention feature fusion map, wherein the AFF-SSD network structure fuses at least two feature maps to obtain an attention feature fusion map, and the AFF-SSD network structure is constructed based on the multi-scale single-shot detection (SSD) basic network structure; and a recognition module 130 for performing feature recognition on the target image based on at least one attention feature fusion map and feature map obtained by the AFF-SSD network structure.

[0092] For details, please refer to Figure 3The SSD basic network structure is based on the VGG16 network. The VGG16 network consists of 16 layers, specifically: two convolutional layers with 64 kernels each, two convolutional layers with 128 kernels each, three convolutional layers with 256 kernels each, six convolutional layers with 512 kernels each, two fully connected layers with 4096 neurons each, and one fully connected layer with 1000 neurons. The VGG16 network can be referenced from traditional techniques, and its specific structure will not be described here. The SSD basic network structure modifies the FC6 and FC7 fully connected layers of the VGG16 network into Conv6 convolutional layers (not shown, located between Conv4_3 and Fc7) and Fc7 convolutional layers (which can be simply referred to as Fc layers), and adds four more convolutional layers afterward.

[0093] In the feature extraction module 120, the AFF-SSD network structure used adds at least one attention feature fusion module to the basic SSD network structure. Figure 3 In the specific embodiment shown, the AFF-SSD network structure adds two attention feature fusion modules to the SSD basic network structure. One attention feature fusion module fuses the features of the Conv4_3 layer and the Fc7 layer to obtain a new Conv4_3 layer, and the other attention feature fusion module fuses the features of the Fc7 layer and the Conv8_2 layer to obtain a new Fc7 layer.

[0094] The AFF-SSD network architecture has six feature maps corresponding to six convolutional layers (Conv4_3, Fc7, Conv8_2, Conv9_2, Conv10_2, and Conv11_2). These six feature maps have different receptive fields, generating candidate boxes of different sizes to predict target objects of different sizes. In this network architecture, shallow feature maps (including the feature maps output by Conv4_3, Fc7, and Conv8_2) have small receptive fields and high resolution, generating small candidate boxes, and are therefore used to detect smaller target objects. Deep feature maps (including the feature maps output by Conv9_2, Conv10_2, and Conv11_2) have larger receptive fields, generating larger candidate boxes, and are therefore used to detect larger target objects.

[0095] The AFF-SSD network structure takes a target image of size 300×300 as input and downsamples it through pooling layers, gradually reducing the image size. The resulting images from the six feature maps are 38*38, 19*19, 10*10, 5*5, 3*3, and 1*1 pixels, respectively. Each feature map generates corresponding candidate boxes. Taking one feature map as an example, if the size of the feature map is M*N, the corresponding feature map is divided into M*N grids, and K candidate boxes are generated centered on the center point of each grid. Therefore, each feature map generates M*N*K candidate boxes. The entire network generates 8732 candidate boxes. Finally, non-maximum suppression (NMS) is used to filter out redundant bounding boxes and produce the final detection result.

[0096] As an example, the attention feature fusion module includes: a spatial attention module 121, which processes the nth feature map in the AFF-SSD network structure to obtain a spatial attention map of the nth feature map, where n is a natural number; a channel attention module 122, which processes the (n+1)th feature map in the AFF-SSD network structure to obtain a channel attention map of the (n+1)th feature map; and a fusion module 123, which fuses the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map based on a feature enhancement method to obtain an initial fused map. Optionally, the attention feature fusion module further includes: a splicing module 124, which fuses the initial fused map with the nth feature map to obtain an attention feature fusion map.

[0097] For details, please refer to Figure 4 The nth feature map and the (n+1)th feature map are input into the attention feature fusion module. The nth feature map is the feature map output from Conv4_3 layer, and the (n+1)th feature map is the feature map output from Fc7 layer, or the nth feature map is the feature map output from Fc7 layer, and the (n+1)th feature map is the feature map output from Conv8_2 layer.

[0098] In the spatial attention module 121, the input dimension Shape1 of the nth feature map is (W1, H1, C1), where W1 is the input width of the nth feature map, H1 is the input height of the nth feature map, and C1 is the number of input channels of the nth feature map, satisfying C1 = W1 * H1. First, the nth feature map is reduced to have 1 channel by two 1*1 convolutions, and the dimension becomes (W1, H1, 1). Then, the width and height are merged to obtain matrices C and D, where matrix C has dimensions ((W1*H1), 1) and matrix D has dimensions (1, (W1*H1)). Multiplying matrices C and D yields matrix E (dimensions (W1*H1), (W1*H1)). Softmax normalization is applied to matrix E along its columns to obtain the correlation between all feature points on the feature map and a given feature point. Expanding matrix E to (W1, H1, (W1*H1)) yields the spatial attention map of the nth feature map.

[0099] In the channel attention module 122, the input dimension Shape2 of the (n+1)th feature map is (W2, H2, C2), where W2 is the input width of the (n+1)th feature map, H2 is the input height of the (n+1)th feature map, and C2 is the number of input channels of the (n+1)th feature map. First, the number of input channels of the (n+1)th feature map is changed by passing two 1*1 convolutions, so that the changed number of channels C2' is equal to the number of input channels C1 of the nth feature map, that is, the number of channels C2' = C1 = W1*H1. Then, the width and height are merged to obtain matrices H and I; matrix H has dimensions ((W1*H1), (W2*H2)) and matrix I has dimensions ((W2*H2), (W1*H1)). Multiplying matrix H and matrix I yields matrix J with dimensions ((W1*H1), (W1*H1)). Softmax normalization is applied to matrix J along its column directions to obtain the influence of all feature points on a given feature point on the feature map. Expanding matrix J into (W1, H1, (W1*H1)), for a full-rank matrix J, it can also be represented as (C1 / 2, C1 / 2, C1), thus obtaining the channel attention map of the (n+1)th feature map.

[0100] Through the above steps, the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map, both with the same dimension, are obtained. Next, the fusion module 123 performs feature fusion (Add) on the spatial attention map of the nth feature map and the channel attention map of the (n+1)th feature map using a feature enhancement method, and then performs batch normalization (BN) to obtain an initial fusion map. Finally, to increase feature diversity, the stitching module 124 uses a channel stitching feature fusion method to fuse the initial fusion map with the features of the nth feature map, obtaining an attention feature fusion map.

[0101] As an example, the spatial attention module 121 includes: a first convolutional layer 1211, which convolves the nth feature map to obtain a first matrix; a second convolutional layer 1212, which convolves the nth feature map to obtain a second matrix; a first reshape layer 1213, which rearranges the dimensions of the first matrix to obtain a third matrix; a second reshape layer 1214, which rearranges the dimensions of the second matrix and transposes the matrix to obtain a fourth matrix; a first multiplication module 1215, which multiplies the third matrix and the fourth matrix to obtain a feature map matrix; a first softmax layer 1216, which normalizes the feature map matrix in the column direction; and a third reshape layer 1217, which rearranges the dimensions of the normalized feature map matrix to obtain a spatial attention map of the nth feature map.

[0102] For details, please refer to Figure 5 The spatial attention module 121 reduces noise and enhances the extraction of important feature information. The input nth feature map has dimensions (W1, H1, C1). The first convolutional layer 1211 and the second convolutional layer 1212 convolve the nth feature map to obtain feature maps A and B, respectively, with dimensions (W1, H1, 1). The first reshape layer 1213 rearranges the dimensions of feature map A to obtain matrix C, with dimensions ((W1*H1), 1). The second reshape layer 1214 rearranges the dimensions of feature map B and transposes the matrix to obtain matrix D, with dimensions [missing information]. (1, (W1*H1)); Then, the first multiplication module 1215 multiplies matrix C and matrix D to obtain the feature map matrix E with dimensions ((W1*H1), (W1*H1)), i.e. (C1, C1); The first Softmax layer 1216 normalizes matrix E in the column direction using softmax to obtain the spatial attention matrix; Finally, the third Reshape layer 1217 rearranges the dimensions of matrix E to obtain the spatial attention map with dimensions (W1, H1, (W1*H1)).

[0103] As an example, the channel attention module 122 includes: a third convolutional layer 1221, which convolves the (n+1)th feature map to obtain a fifth matrix; a fourth convolutional layer 1222, which convolves the (n+1)th feature map to obtain a sixth matrix; a fourth reshape layer 1223, which performs dimensional rearrangement and matrix transpose on the fifth matrix to obtain a seventh matrix; a fifth reshape layer 1224, which performs dimensional rearrangement on the sixth matrix to obtain an eighth matrix; a second multiplication module 1225, which multiplies the seventh and eighth matrices to obtain a second feature map matrix; a second softmax layer 1226, which normalizes the second feature map matrix in the column direction; and a sixth reshape layer 1227, which performs dimensional rearrangement on the normalized second feature map matrix to obtain the channel attention map of the (n+1)th feature map.

[0104] For details, please refer to Figure 6 The channel attention module 122 establishes dependencies between channels, enhances the expression of semantic information, and obtains the contour information of the feature map. The input dimension of the (n+1)th feature map is (W2, H2, C2). The third convolutional layer 1221 and the fourth convolutional layer 1222 change the number of channels by convolving the (n+1)th feature map, obtaining feature maps F and G. The number of channels in feature maps F and G is equal to the number of input channels in the nth feature map, thus the dimensions of feature maps F and G are (W2, H2, C1). The fourth reshape layer 1223 rearranges the dimensions of feature map F and transposes the matrix to obtain matrix H. Matrix H... The fifth Reshape layer 1224 rearranges the dimensions of the feature map G to obtain matrix I, whose dimensions are ((W2*H2), C1). Then, the second multiplication module 1225 multiplies matrix H and matrix I to obtain feature map matrix J with dimensions (C1, C1). The second Softmax layer 1226 normalizes matrix J in the column direction using softmax to obtain the channel attention matrix. The normalization operation is as follows:

[0105]

[0106] In the formula B ji H represents the correlation between the j-th pixel and the i-th pixel. The more similar their features are, the greater the correlation obtained by multiplying their transposes, resulting in a higher feature value on the attention feature image. ij Let J be the elements of matrix J. Finally, the sixth Reshape layer 1227 rearranges the dimensions of matrix J to obtain the channel attention map, which has dimensions (C1 / 2, C1 / 2, C1), that is, the dimensions of the channel attention map are (W1, H1, (W1*H1)).

[0107] The present invention also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the following: Figure 1 The method shown, Figure 1 For specific embodiments of the method shown, please refer to Figure 3-6 .

[0108] The target detection method, apparatus, and storage medium provided by this invention enhance the contextual information of the network by performing feature fusion on the feature maps in the AFF-SSD network structure, enabling different feature maps to complement each other and output more information (corresponding to more features), while also enhancing the correlation of distant pixels.

[0109] The technical solution provided in this application belongs to the fields of artificial intelligence and object detection. This technology can be applied to multi-scenario recognition in the field of human-computer interaction, enabling products to have convenient and flexible user operation perception modes. For example, applications such as gesture recognition, fingerprint recognition, face recognition, multi-finger tracking and recognition, and non-contact recognition can improve the user's interactive experience.

[0110] As used herein, the term "module" may refer to, be part of, or include the following: application-specific integrated circuit (ASIC), electronic circuitry, processor (shared, dedicated, or grouped) and / or memory (shared, dedicated, or grouped) executing one or more software or firmware programs, combinational logic circuitry, and / or other suitable components that provide the described functionality.

[0111] Those skilled in the art will understand that the various modules or units of the target detection method according to the present invention can be implemented by hardware, firmware, or software. Software includes, for example, coded programs written in various programming languages ​​such as JAVA, C / C++ / C#, and SQL. Although the steps and their order in the embodiments of the present invention are given in the method and method illustrations, the executable instructions that implement the specified logical function of the steps can be recombined to generate new steps. The order of the steps should not be limited to the order of steps in the method and method illustrations, and can be adjusted at any time according to functional needs. For example, some steps can be executed in parallel or in reverse order.

[0112] Furthermore, those skilled in the art will recognize that the structures and methods described in conjunction with the embodiments disclosed herein can be used with different configuration or adjustment methods to achieve the described functions for each structure or reasonable variations thereof, but such implementations should not be considered beyond the scope of this application. Moreover, it should be understood that the connection relationships between the various components of the amplifier in the foregoing figures in the embodiments of this application are illustrative examples and do not impose any limitations on the embodiments of this application.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method characterized by, The method comprises the following steps: receiving a target image; inputting the target image into an attention feature fusion multi-scale single shot detection (AFF-SSD) network structure to obtain a plurality of feature maps and at least one attention feature fusion map, wherein the AFF-SSD network structure is constructed based on a multi-scale single shot detection (SSD) basic network structure, and the AFF-SSD network structure performs feature fusion on at least two feature maps to obtain the attention feature fusion map; and performing feature recognition on the target image based on the attention feature fusion map and the feature maps, wherein the attention feature fusion map is obtained by the following steps: performing two different 1×1 convolutions on the nth feature map in parallel to obtain a first matrix and a second matrix, respectively, wherein n is a natural number; performing dimension rearrangement on the first matrix to obtain a third matrix, and performing dimension rearrangement and matrix transposition on the second matrix to obtain a fourth matrix; multiplying the third matrix and the fourth matrix to obtain a first feature map matrix, and performing Softmax normalization and dimension rearrangement on the first feature map matrix in the column direction to obtain a spatial attention map with the same size as the nth feature map; performing two different 1×1 convolutions on the (n+1)th feature map in parallel to obtain a fifth matrix and a sixth matrix; performing dimension rearrangement and matrix transposition on the fifth matrix to obtain a seventh matrix, and performing dimension rearrangement on the sixth matrix to obtain an eighth matrix; multiplying the seventh matrix and the eighth matrix to obtain a second feature map matrix, and performing Softmax normalization and dimension rearrangement on the second feature map matrix in the column direction to obtain a channel attention map consistent with the channel number of the nth feature map; adding the spatial attention map and the channel attention map by a feature enhancement method and batch normalization to obtain an initial fusion map; performing feature fusion on the initial fusion map and the nth feature map by a channel splicing feature fusion method to obtain the attention feature fusion map.

2. The object detection method of claim 1, wherein, The feature fusion on the at least two feature maps to obtain the attention feature fusion map comprises: performing feature fusion on a Conv4_3 layer and an Fc7 layer to obtain a new Conv4_3 layer; performing feature fusion on the Fc7 layer and a Conv8_2 layer to obtain a new Fc7 layer; respectively taking the new Conv4_3 layer and the new Fc7 layer as the attention feature fusion map.

3. The object detection method according to claim 1 or 2, characterized in that, The nth feature map is a feature map output by the Conv4_3 layer, and the (n+1)th feature map is a feature map output by the Fc7 layer, or the nth feature map is a feature map output by the Fc7 layer, and the (n+1)th feature map is a feature map output by the Conv8_2 layer.

4. A target detection apparatus characterized by comprising: The method comprises the following steps: a receiving module that receives a target image; The feature extraction module inputs the target image into an attention feature fusion multi-scale single-shot detection (AFF-SSD) network structure to obtain a plurality of feature maps and at least one attention feature fusion map, the AFF-SSD network structure being constructed by adding at least one attention feature fusion module to a multi-scale single-shot detection (SSD) base network structure, and the AFF-SSD network structure performing feature fusion on at least two feature maps to obtain the attention feature fusion map; and The recognition module performs feature recognition on the target image based on the attention feature fusion map and the feature maps. The attention feature fusion module includes: The spatial attention module is configured to perform two different 1×1 convolutions on the nth feature map in parallel to obtain a first matrix and a second matrix, respectively, n being a natural number; perform dimension rearrangement on the first matrix to obtain a third matrix, and perform dimension rearrangement and matrix transposition on the second matrix to obtain a fourth matrix; multiply the third matrix and the fourth matrix to obtain a first feature map matrix, and perform Softmax normalization and dimension rearrangement on the first feature map matrix in the column direction to obtain a spatial attention map with the same size as the nth feature map; The channel attention module is configured to perform two different 1×1 convolutions on the (n+1)th feature map in parallel to obtain a fifth matrix and a sixth matrix; perform dimension rearrangement and matrix transposition on the fifth matrix to obtain a seventh matrix, and perform dimension rearrangement on the sixth matrix to obtain an eighth matrix; multiply the seventh matrix and the eighth matrix to obtain a second feature map matrix, and perform Softmax normalization and dimension rearrangement on the second feature map matrix in the column direction to obtain a channel attention map with the same number of channels as the nth feature map; The fusion module adds and batch-normalizes the spatial attention map and the channel attention map by a feature enhancement method to obtain an initial fusion map; and The concatenation module performs feature fusion on the initial fusion map and the nth feature map by a channel concatenation feature fusion method to obtain the attention feature fusion map.

5. The object detection apparatus according to claim 4, characterized by The AFF-SSD network structure includes two attention feature fusion modules, wherein one attention feature fusion module performs feature fusion on a Conv4_3 layer and an Fc7 layer to obtain a new Conv4_3 layer, and another attention feature fusion module performs feature fusion on the Fc7 layer and a Conv8_2 layer to obtain a new Fc7 layer, the new Conv4_3 layer and the new Fc7 layer being the attention feature fusion map, respectively.

6. The object detection apparatus according to claim 4 or 5, characterized by The nth feature map is a feature map output by the Conv4_3 layer, and the (n+1)th feature map is a feature map output by the Fc7 layer, or the nth feature map is a feature map output by the Fc7 layer, and the (n+1)th feature map is a feature map output by the Conv8_2 layer.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the target detection method of any one of claims 1 to 3.

Images