A small target detection method and device based on feature map weighted attention fusion

Abstract

The present invention provides a small target detection method and device based on feature map weighted attention fusion. The detection method includes the following steps: acquiring an image to be detected; preprocessing the image to be detected by an improved imitation primary visual cortex model to obtain a primary feature map; extracting features from the primary feature map by a preset small target detection model to obtain m target feature maps, including: performing feature map size compression and feature extraction on the primary feature map in sequence m times to obtain m backbone feature maps and feature maps of interest of different sizes; performing attention fusion and feature extraction based on weighted channel splicing on the m backbone feature maps and feature maps of interest to obtain m target feature maps, performing feature fusion on the m target feature maps to obtain target detection results. The present invention improves the problem that the methods for small target detection in the prior art are poor in accuracy, easily cause false detection and missed detection, and cannot be applied to application scenarios with a large number of small targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence target detection, and in particular relates to a small target detection method and device based on feature map weighted attention fusion. Background Art

[0002] The task of target detection is to detect the target object of interest in a static image (or dynamic video), which can provide good conditions for subsequent target tracking, behavior prediction, etc. The target detection task is an important part of computer vision and is widely used in many fields such as unmanned driving, robot navigation, video surveillance, industrial inspection, aerospace, etc.

[0003] Existing target detection methods have achieved a lot of results, and typical research results include Faster-RCNN, SSD, CenterNet, YOLOv1~YOLOv5, YOLOX, etc. However, these methods are not very accurate in detecting small targets, resulting in high false detection and missed detection rates, and are difficult to apply to application scenarios with a large number of small targets. Summary of the invention

[0004] In view of the above analysis, the present invention aims to provide a small target detection method and device based on feature map weighted attention fusion, which is used to improve the problem that the existing methods for small target detection are not accurate, easily cause false detection and missed detection, and cannot be applied to application scenarios with a large number of small targets.

[0005] The purpose of the present invention is mainly achieved through the following technical solutions:

[0006] On the one hand, the present invention provides a small target detection method based on feature map weighted attention fusion, characterized in that it includes the following steps:

[0007] Acquire the image to be detected;

[0008] Preprocessing the image to be detected by using an improved simulated primary visual cortex model to obtain a primary feature map;

[0009] The primary feature map is subjected to feature extraction through a preset small target detection model to obtain m target feature maps; including: performing feature map size compression and feature extraction on the primary feature map m times in sequence to obtain m backbone feature maps and feature maps of interest of different sizes; performing attention fusion and feature extraction based on weighted channel splicing of the m backbone feature maps and feature maps of interest to obtain m target feature maps, where m is an integer greater than 1;

[0010] Feature fusion is performed on the m target feature maps to obtain a target detection result.

[0011] Furthermore, the improved simulated primary visual cortex model includes: a VoneBlock layer, a Conv layer and a Weighted_Sum_Attention layer;

[0012] The VoneBlock layer is used to extract bionic visual features from the image to be detected to obtain a bionic visual feature map;

[0013] The Conv layer is used to perform convolution feature extraction on the image to be detected to obtain a convolution feature map;

[0014] The Weighted_Sum_Attention layer is used to perform weighted addition of attention fusion on the bionic visual feature map and the convolution feature map to obtain the primary feature map.

[0015] Furthermore, the Weighted_Sum_Attention layer performs attention feature fusion by weighted addition of feature maps through the following formula:

[0016]

[0017] Wherein, ε is a preset small constant; n represents the number of input feature maps, and the input feature maps include input bionic visual feature maps and convolution feature maps; Finput i represents the i-th feature map of the input; w i represents the weight of the i-th input feature map, w i Preset to the training variable of the model, preset w i ≥0; SE() means feature extraction based on SE attention module; CBAM() means feature extraction based on CBAM attention module.

[0018] Furthermore, the target detection model includes a backbone network and a head network;

[0019] The backbone network includes m sequentially connected feature map compression and extraction modules, which are used to sequentially perform m feature map size compression and feature extraction on the primary feature map to obtain m backbone feature maps and feature maps of interest of different sizes;

[0020] The head network is used to perform attention fusion and feature extraction of weighted channel splicing based on the m feature maps of interest and the backbone feature maps, respectively, to obtain m target feature maps.

[0021] Furthermore, the feature map compression and extraction module includes a Conv layer, a C3 (CSP Bottleneck with 3 Convolutions) layer and a CA attention layer; wherein the Conv layer is used to compress the input feature map; the C3 (CSP Bottleneck with 3 Convolutions) layer is used to extract features of the feature map after the size compression to obtain a backbone feature map; the CA attention layer is used to extract features of interest from the backbone feature map to obtain a feature map of interest;

[0022] The feature map obtained after feature extraction of the primary feature map through the CA attention layer is used as the input of the first feature map compression and extraction module to obtain the first backbone feature map and the feature map of interest; the feature map of interest obtained by the Ith feature map compression and extraction module is used as the input of the I+1th feature map compression and extraction module, and after feature map size compression and feature extraction, the I+1th backbone feature map and the I+1th feature map of interest are obtained, where I is an integer of 1≤I≤m-1.

[0023] Further, the head network includes m FPN modules and m weighted channel aggregation modules;

[0024] The FPN module is used to upsample and cross-layer fuse the image based on the backbone feature map and the feature map of interest output by the backbone network to obtain a feature map with bottom-level visual information and high-level semantic information;

[0025] The weighted channel aggregation module is used to perform attention fusion based on the feature map weighted channel splicing based on the feature map output by the corresponding FPN module to obtain a feature map with enhanced features, and then obtain the corresponding target feature map after feature extraction;

[0026] Among them, the I+1th FPN module performs weighted channel splicing attention fusion based on the backbone feature map and the feature map of interest output by the Ith feature map compression and extraction module, and the feature map obtained after the fusion is subjected to feature extraction by the C3 (CSPBottleneck with 3Convolutions) layer to obtain the feature map output by the I+1th FPN module; the output of the Conv layer of the Ith FPN module is subjected to weighted channel splicing attention fusion processing through the Ith weighted channel aggregation module to obtain the Ith target feature map, wherein I is an integer of 1≤I≤m-1.

[0027] Further, the FPN module includes a Conv layer, an Upsamle layer, a first Weighted_Concat_Attention layer, and a C3 (CSP Bottleneck with 3 Convolutions) layer connected in sequence;

[0028] Among them, the output of the C3 layer of the (I + 1)-th FPN module is the input of the Conv layer of the I-th FPN module;

[0029] The input images of the first Weighted_Concat_Attention layer included in the k-th FPN module include: the backbone feature map and the region of interest feature map obtained by compressing and extracting the (k - 1)-th feature map extraction module of the backbone network, and the feature map obtained by upsampling the (k + 1)-th FPN module, where k is an integer satisfying 1 < k ≤ m;

[0030] The weighted channel aggregation module includes a second Weighted_Concat_Attention layer; the input images of the l-th weighted channel aggregation module are: the feature map obtained by compressing the size of the (l - 1)-th target feature map through a Conv layer, the feature map output by the Conv layer of the l-th FPN module, the feature map obtained by compressing the channels of the image output by the (l + 1)-th FPN module and then upsampling, the l-th region of interest feature map output by the backbone network, and, where l is an integer satisfying 1 ≤ l < m.

[0031] Further, the region of interest feature map obtained by extracting features from the primary feature map through the CA attention layer and the feature map output by the Upsample layer of the first FPN module are input into the Weighted_Concat_Attention layer of the first FPN module to obtain the output of the first FPN module;

[0032] The region of interest feature map output by the m-th feature map extraction module and the feature map obtained by fusing the spatial information of the feature map and compressing the number of channels through the SPPF layer and the Conv layer for the backbone feature map output by the m-th feature map extraction module are input into the m-th weighted channel aggregation module to obtain the m-th target feature map.

[0033] Further, the Weighted_Concat_Attention layer performs attention fusion based on weighted channel splicing of feature maps through the following formula:

[0034]

[0035] where ε = 0.0001; n represents the number of input feature maps; Finput i represents the i-th input feature map; wi represents the weight of the i-th input feature map, w i The training variables preset for the model can be obtained through the training and learning process of the model. i ≥0; Concat() indicates channel concatenation operation; SE() indicates feature extraction based on SE attention module; CBAM() indicates feature extraction based on CBAM attention module.

[0036] On the other hand, there is also provided an electronic device, comprising at least one processor, and at least one memory communicatively connected to the processor;

[0037] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the aforementioned small target detection method.

[0038] Beneficial effects of this technical solution:

[0039] 1. The present invention adopts an improved imitation primary visual cortex model. First, the VOneBlock layer is connected in parallel with a Conv layer to make the model more concise; then a Weighted_Sum_Attention layer is added to perform attention fusion based on weighted addition of feature maps on the pixel values ​​of the feature maps output by the VOneBlock layer and the Conv layer. By performing weighted addition fusion on the pixel values ​​of different input feature maps, it plays an effect of strengthening important features and weakening noise in the image preprocessing stage, thereby improving the accuracy of small target detection.

[0040] 2. The present invention adopts multiple Weighted_Concat_Attention layers to perform attention fusion on different feature maps based on weighted channel splicing of feature maps. By assigning weights to different input feature maps and performing channel splicing, it can play a good role in strengthening important features and weakening unimportant features, greatly improving the expression ability of important features and improving small target detection performance.

[0041] 3. The model structure of the present invention adopts multiple scale features for target detection. The maximum size of the target feature map is 1 / 2 of the original input image, and the minimum size of the target feature map is 1 / 32 of the original input image, so as to improve the feature extraction capability of small targets, improve the problem of missed detection of small targets in the prior art, and improve the detection performance of small targets.

[0042] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0044] Figure 1 4 is a flow chart of a small target detection method according to an embodiment of the present invention.

[0045] Figure 2 It is a structural diagram of the simulated primary visual cortex model of the small target detection method according to an embodiment of the present invention.

[0046] Figure 3 It is a structural diagram of a target detection model of a small target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the implementation cases of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0048] Small targets refer to targets with very few visual features in an image. Currently, targets smaller than 32 pixels × 32 pixels are usually defined as small targets. Small target detection has long been a difficult point in target detection.

[0049] An embodiment of the present invention provides a small target detection method based on feature map weighted attention fusion, such as Figure 1 As shown, the following steps are included:

[0050] Step S1, obtaining an image to be detected;

[0051] The image to be detected may be an image taken in any manner including at least one small target to be detected, such as an image taken by a drone including ground vehicles, personnel, mobile equipment, and infrastructure small targets to be detected.

[0052] Step S2, preprocessing the image to be detected by using an improved simulated primary visual cortex model to obtain a primary feature map;

[0053] Preferably, Figure 2As shown, the improved simulated primary visual cortex adds a Weighted_Sum_Attention layer on the basis of the existing simulated primary visual cortex model; the improved simulated primary visual cortex model of this embodiment includes: a VoneBlock layer, a Conv layer and a Weighted_Sum_Attention layer; the VoneBlock layer is used to extract bionic visual features of the image to be detected, and the bionic visual feature is to construct a neural network layer according to the primary visual cortex of primates, simulate the information processing mechanism of the human visual perception cortex to extract features of the input image, and obtain a feature map that is closer to the features after visual processing of the human brain; the Conv layer is used to extract convolution features of the image to be detected; the Weighted_Sum_Attention layer is used to perform attention fusion by weighted addition of pixel values ​​of the bionic visual feature map and the convolution feature map to obtain the primary feature map.

[0054] The Weighted_Sum_Attention layer performs attention feature fusion by weighted addition of feature maps through the following formula:

[0055]

[0056] Wherein, ε is a preset small constant, and in this embodiment, the value ε=0.0001; n represents the number of input feature maps; Finput i represents the i-th input feature map; w i represents the weight of the i-th input feature map, w i The preset training variables of the model can be obtained through the training and learning process of the model. i ≥0; SE() means feature extraction based on SE attention module; CBAM() means feature extraction based on CBAM attention module.

[0057] Among them, the SE attention module is an attention calculation module used to extract attention features from feature maps in the channel direction; the CBAM attention module is an attention calculation module used to extract attention features from feature maps in the channel direction and spatial position direction.

[0058] Specifically, a real-time image with a size of H*W of an image to be detected is input, feature extraction and feature fusion are performed through an improved simulated primary visual cortex model, and a primary feature map with a size of H*W is output.

[0059] This embodiment adds a Weighted_Sum_Attention layer to perform attention fusion based on weighted addition of feature maps on the pixel values ​​of the feature maps output by the VOneBlock layer and the Conv layer. By weighting and adding pixel values ​​of different input feature maps, the important features of the primary feature map are strengthened, and the noise is weakened, providing a good input image for subsequent processing.

[0060] Step S3: extracting features from the primary feature map through a preset small target detection model to obtain m target feature maps; including: performing feature map size compression and feature extraction on the primary feature map m times in sequence to obtain m backbone feature maps and feature maps of interest of different sizes; performing attention fusion and feature extraction of weighted channel splicing based on the m backbone feature maps and feature maps of interest to obtain m target feature maps, where m is an integer greater than 1;

[0061] Feature fusion is performed on the m target feature maps to obtain a target detection result.

[0062] Among them, the preset small target detection model includes a backbone network and a head network;

[0063] The backbone network includes m feature map compression and extraction modules connected in sequence; used to perform m feature map size compression and feature extraction on the primary feature map, to obtain m backbone feature maps and feature maps of interest of different sizes; the feature map compression and extraction module includes a Conv layer, a C3 (CSP Bottleneck with 3Convolutions) layer and a CA attention layer; wherein the Conv layer performs size compression on the input feature map; the C3 (CSP Bottleneck with 3Convolutions) layer performs backbone feature extraction on the feature map after size compression to obtain a backbone feature map; the CA attention layer performs feature extraction on the backbone feature map to obtain a feature map of interest;

[0064] Among them, the Conv layer is a basic convolution unit, which is used to perform two-dimensional convolution, regularization and activation operations on the input feature map in sequence. The C3 (CSP Bottleneck with 3Convolutions) layer is composed of several Bottleneck modules. Bottleneck is a classic residual result. After the input passes through two layers of convolution, the ADD operation is performed with the original value to complete the residual feature transfer without increasing the output depth; the CA attention layer is an attention calculation unit, which is used to extract attention features from the feature map in two directions of the spatial position to improve the expression ability of the features of interest.

[0065] The head network is used to perform attention fusion and feature extraction of weighted channel splicing based on the m feature maps of interest and the backbone feature maps, respectively, to obtain m target feature maps.

[0066] Specifically, the feature map obtained after feature extraction of the primary feature map through the CA attention layer is used as the input of the first feature map compression and extraction module to obtain the first backbone feature map and the feature map of interest; the feature map of interest obtained by the Ith feature map compression and extraction module is used as the input of the I+1th feature map compression and extraction module, and after feature map size compression and feature extraction, the I+1th backbone feature map and the I+1th feature map of interest are obtained, where I is an integer of 1≤I≤m-1.

[0067] As a specific example, Figure 3 As shown, the backbone network of the present invention adopts 5 feature map compression and extraction modules connected in sequence, performs image compression and feature extraction on the primary feature map 5 times, and obtains 5 feature maps of different sizes, which are respectively obtained through attention fusion and feature extraction of weighted channel splicing through the subsequent head network to obtain 5 target feature maps of different sizes.

[0068] Specifically, for the primary feature map of size 1280*1280 output by the improved imitation primary visual cortex model, the CA attention layer is used to extract the features of interest to obtain a primary feature map of interest; the primary feature map of interest is input into the first feature map compression and extraction module, and the image is compressed through the Conv layer to obtain a first feature map of 640*640, and the first feature map is input into the C3 (CSP Bottleneck with 3Convolutions) layer for feature extraction to obtain a first backbone feature map of size 640*640, and the first backbone feature map is input into the CA attention layer for feature extraction of interest to obtain a first feature map of size 640*640; the first feature map of interest is sequentially input into the second to fourth first feature map compression and extraction modules to obtain a second backbone feature map and a second feature map of interest of size 320*320, a third backbone feature map and a third feature map of interest of size 160*160, a fourth backbone feature map and a fourth feature map of interest of size 80*80, and a fifth backbone feature map and a fifth feature map of interest of size 40*40.

[0069] The five sizes of feature maps of interest and backbone feature maps are processed through the head network respectively to obtain the corresponding five target feature maps of different sizes.

[0070] Furthermore, the head network includes m FPN modules and m weighted channel aggregation modules;

[0071] The FPN module is used to perform upsampling and cross-layer fusion on an image based on the backbone feature map and the region of interest feature map output by the backbone network to obtain a feature map with low-level visual information and high-level semantic information;

[0072] The weighted channel aggregation module is used to perform attention fusion based on weighted channel splicing of the feature map output by the corresponding FPN module to obtain a feature-strengthened feature map, and then perform feature extraction to obtain the corresponding target feature map;

[0073] Among them, the (I + 1)-th FPN module performs attention fusion based on weighted channel splicing of the backbone feature map and the region of interest feature map output by the I-th feature map compression and extraction module. The feature map obtained after fusion is subjected to feature extraction through a C3 (CSP Bottleneck with 3 Convolutions) layer to obtain the feature map output by the (I + 1)-th FPN module; the I-th target feature map is obtained by performing attention fusion processing based on weighted channel splicing on the output of the Conv layer of the I-th FPN module through the I-th weighted channel aggregation module, where I is an integer satisfying 1 ≤ I ≤ m - 1.

[0074] The FPN module includes a Conv layer, an Upsamle layer, a first Weighted_Concat_Attention layer, and a C3 (CSP Bottleneck with 3 Convolutions) layer connected in sequence;

[0075] Among them, the output of the C3 layer of the (I + 1)-th FPN module is the input of the Conv layer of the I-th FPN module;

[0076] The input image of the first Weighted_Concat_Attention layer included in the k-th FPN module includes: the backbone feature map and the region of interest feature map obtained by the (k - 1)-th feature map compression and extraction module of the backbone network, and the feature map output by the Upsample layer of the k-th FPN module, where k is an integer satisfying 1 < k ≤ m;

[0077] The weighted channel aggregation module includes a second Weighted_Concat_Attention layer; the input image of the l-th weighted channel aggregation module includes: the feature map obtained by compressing the size of the (l - 1)-th target feature map through a Conv layer, the feature map output by the Conv layer of the l-th FPN module, the feature map output by the Upsample layer of the (l + 1)-th FPN module, and the l-th region of interest feature map output by the backbone network, where l is an integer satisfying 1 ≤ l < m.

[0078] The feature map of interest obtained after feature extraction of the primary feature map through the CA attention layer and the feature map output by the Upsample layer of the first FPN module are input into the Weighted_Concat_Attention layer of the first FPN module for weighted channel splicing and attention fusion to obtain the output of the first FPN module.

[0079] The feature map of interest output by the mth feature map compression and extraction module and the backbone feature map output by the mth feature map compression and extraction module are input into the mth weighted channel aggregation module after the feature map spatial information fusion and channel number compression are performed by the SPPF layer and the Conv layer to obtain the feature map to obtain the mth target feature map.

[0080] As a specific example, Figure 3 As shown, 5 FPN modules and corresponding 5 weighted channel aggregation modules are used for image fusion and feature extraction;

[0081] Among them, the output of the C3 (CSP Bottleneck with 3Convolutions) layer of the fifth FPN module is the input of the Conv layer of the fourth FPN module; the output of the C3 (CSP Bottleneck with 3Convolutions) layer of the fourth FPN module is the input of the Conv layer of the third FPN module; the output of the C3 (CSP Bottleneck with3Convolutions) layer of the third FPN module is the input of the Conv layer of the second FPN module; the output of the C3 (CSP Bottleneck with 3Convolutions) layer of the second FPN module is the input of the Conv layer of the first FPN module.

[0082] As a specific embodiment, first, the primary feature map of interest with a size of 1280*1280 and the feature map output by the Upsample layer of the first FPN module are input into the Weighted_Concat_Attention layer for attention fusion based on weighted channel concatenation of the feature maps to obtain a feature map of the first FPN module with a size of 1280*1280;

[0083] The first backbone feature map with a size of 640*640 output by the backbone network, the first feature map of interest, and the feature map output by the Upsample layer of the second FPN module are input into the Weighted_Concat_Attention layer of the second FPN module, and feature extraction is performed through the C3 (CSP Bottleneck with 3Convolutions) layer to obtain a feature map of the second FPN module with a size of 640*640;

[0084] The second backbone feature map with a size of 320*320 output by the backbone network, the second feature map of interest, and the feature map output by the Upsample layer of the third FPN module are input into the Weighted_Concat_Attention layer of the third FPN module, and feature extraction is performed through the C3 (CSP Bottleneck with 3Convolutions) layer to obtain the feature map of the third FPN module with a size of 320*320;

[0085] The third backbone feature map with a size of 160*160 output by the backbone network, the third feature map of interest, and the feature map output by the Upsample layer of the fourth FPN module are input into the Weighted_Concat_Attention layer of the fourth FPN module, and feature extraction is performed through the C3 (CSP Bottleneck with 3Convolutions) layer to obtain the feature map of the fourth FPN module with a size of 160*160;

[0086] The fourth backbone feature map with a size of 80*80 output by the backbone network, the fourth feature map of interest, and the feature map output by the Upsample layer of the fifth FPN module are input into the Weighted_Concat_Attention layer of the fifth FPN module, and feature extraction is performed through the C3 (CSP Bottleneck with 3Convolutions) layer to obtain the feature map of the fifth FPN module with a size of 80*80;

[0087] Furthermore, the feature map output by the Conv layer of the first FPN module, the feature map output by the Weighted_Concat_Attention layer of the first FPN module, the feature map obtained by size compression through the Conv layer, the feature map output by the Upsample layer of the second FPN module, and the first feature map of interest output by the backbone network are input into the first weighted channel aggregation module, and the feature map output by the first weighted channel aggregation module is subjected to feature extraction through the C3 (CSP Bottleneck with 3 Convolutions) layer to obtain a first target feature map with a size of 640*640.

[0088] The feature map obtained by size compression of the first target feature map, the feature map output by the Conv layer of the second FPN module, the feature map output by the Upsample layer of the third FPN module, and the second feature map of interest output by the backbone network are input into the second weighted channel aggregation module, and the feature map output by the second weighted channel aggregation module is subjected to feature extraction through the C3 (CSP Bottleneck with 3 Convolutions) layer to obtain a second target feature map with a size of 320*320;

[0089] The feature map obtained by size compression of the second target feature map, the feature map output by the Conv layer of the third FPN module, the feature map output by the Upsample layer of the fourth FPN module, and the third feature map of interest output by the backbone network are input into the third weighted channel aggregation module, and the feature map output by the third weighted channel aggregation module is subjected to feature extraction through the C3 (CSP Bottleneck with 3 Convolutions) layer to obtain a third target feature map with a size of 160*160;

[0090] The feature map obtained by size compression of the third target feature map, the feature map output by the Conv layer of the fourth FPN module, the feature map output by the Upsample layer of the fifth FPN module, and the fourth feature map of interest output by the backbone network are input into the fourth weighted channel aggregation module, and the feature map output by the fourth weighted channel aggregation module is subjected to feature extraction through the C3 (CSP Bottleneck with 3 Convolutions) layer to obtain a fourth target feature map with a size of 80*80;

[0091] The feature map obtained by size compression of the fourth target feature map, the feature map output by the Conv layer of the fifth FPN module, and the fifth feature map of interest output by the backbone network are input into the fifth weighted channel aggregation module, and the feature map output by the fifth weighted channel aggregation module is subjected to feature extraction through the C3 (CSP Bottleneck with 3Convolutions) layer to obtain the fifth target feature map with a size of 40*40.

[0092] The five target feature maps are fused through the Detect layer to achieve target classification and coordinate positioning. This embodiment uses the Detect layer of the existing YOLOv5 model to perform target classification and coordinate positioning to obtain small target detection results.

[0093] Among them, the Conv layer, SPPF layer, C3 layer, Upsample layer, and Detect are consistent with the existing YOLOv5 model.

[0094] Preferably, the Weighted_Concat_Attention layer performs attention fusion based on weighted channel concatenation of feature maps through the following formula:

[0095]

[0096] Where ε is a preset small constant, and in this embodiment, the value ε=0.0001; n represents the number of input feature maps; Finput i represents the i-th input feature map; w i represents the weight of the i-th input feature map, w i The preset training variables of the model can be obtained through the training and learning process of the model. i ≥0; Concat() indicates channel concatenation operation; SE() indicates feature extraction based on SE attention module; CBAM() indicates feature extraction based on CBAM attention module.

[0097] As a specific example, Figure 3 In the "38.Weighted_Concat_Attention layer", the different input feature maps are the feature maps output by the "37.Conv layer", "34.Conv layer", "31.Upsample layer", and "4.CA attention layer". The number of input feature maps n=4, and the formula can be changed to the following specific formula,

[0098]

[0099] Among them, Finput1, Finput2, Finput3, and Finput4 are the feature maps output by the four layers "37.Conv layer", "34.Conv layer", "31.Upsample layer", and "4.CA attention layer", respectively. The SE attention module is used to extract attention features from these four feature maps to obtain SE(Finput1), SE(Finput2), SE(Finput3), and SE(Finput4); then different weights are assigned to the obtained feature maps to obtain:

[0100]

[0101] Among them, w1, w2, w3, and w4 are the corresponding weights, which can be obtained through the training and learning process of the model; then the Concat module is used to splice the channels. The Concat module is consistent with the existing YOLOv5 model, and we can get:

[0102]

[0103] The CBAM attention module is further used to extract attention features from the feature map after channel splicing in the channel direction and spatial position direction, and obtain the feature map after attention fusion based on the weighted channel splicing of the feature map.

[0104] The present invention uses an improved imitation of the primary visual cortex model to weight different input feature maps and add and fuse pixel values, which has the effect of strengthening important features and weakening noise in the image preprocessing stage, thereby improving the accuracy of small target detection. Using multiple Weighted_Concat_Attention layers, attention fusion based on weighted channel splicing of feature maps is performed on different feature maps. By assigning weights to different input feature maps and performing channel splicing, it can have a good effect of strengthening important features and weakening unimportant features, greatly improving the expression ability of important features of small targets. In addition, by generating multiple target feature maps of different sizes for small target detection, the feature extraction capability of small targets is effectively improved, and the small target detection performance is improved.

[0105] A second embodiment of the present invention further discloses an electronic device, comprising at least one processor, and at least one memory communicatively connected to the processor;

[0106] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the small target detection method in the above embodiment.

[0107] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0108] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A small target detection method based on feature map weighted attention fusion, characterized in that: The following steps are involved: Acquire the image to be detected; Preprocessing the image to be detected by using an improved simulated primary visual cortex model to obtain a primary feature map; The improved simulated primary visual cortex model includes: a VoneBlock layer, a Conv layer and a Weighted_Sum_Attention layer; the VoneBlock layer is used to extract bionic visual features from the image to be detected to obtain a bionic visual feature map; the Conv layer is used to extract convolutional features from the image to be detected to obtain a convolutional feature map; the Weighted_Sum_Attention layer is used to perform attention fusion by weighted addition of pixel values ​​of the bionic visual feature map and the convolutional feature map to obtain the primary feature map; the Weighted_Sum_Attention layer performs attention feature fusion by weighted addition of feature maps through the following formula: Wherein, ε is a preset small constant; n represents the number of input feature maps, and the input feature maps include input bionic visual feature maps and convolution feature maps; Finput i represents the i-th feature map of the input; w i represents the weight of the i-th input feature map, w i Preset to the training variable of the model, preset w i ≥0; SE() means feature extraction based on SE attention module; CBAM() means feature extraction based on CBAM attention module; The primary feature map is subjected to feature extraction through a preset small target detection model to obtain m target feature maps, including: performing feature map size compression and feature extraction on the primary feature map m times in sequence to obtain m backbone feature maps and feature maps of interest of different sizes; performing attention fusion and feature extraction based on weighted channel splicing of the m backbone feature maps and feature maps of interest to obtain m target feature maps, where m is an integer greater than 1; Feature fusion is performed on the m target feature maps to obtain a target detection result.

2. The small target detection method according to claim 1, characterized in that: The target detection model includes a backbone network and a head network; The backbone network includes m sequentially connected feature map compression and extraction modules, which are used to sequentially perform m feature map size compression and feature extraction on the primary feature map to obtain m backbone feature maps and feature maps of interest of different sizes; The head network is used to perform attention fusion and feature extraction of weighted channel splicing based on the m feature maps of interest and the backbone feature maps, respectively, to obtain m target feature maps.

3. The small target detection method according to claim 2, characterized in that: The feature map compression and extraction module includes a Conv layer, a C3 layer and a CA attention layer; wherein the Conv layer performs size compression on the input feature map; the C3 layer performs feature extraction on the feature map after size compression to obtain a backbone feature map; the CA attention layer performs feature extraction on the backbone feature map to obtain an interesting feature map; The feature map obtained after feature extraction of the primary feature map through the CA attention layer is used as the input of the first feature map compression and extraction module to obtain the first backbone feature map and the feature map of interest; the feature map of interest obtained by the Ith feature map compression and extraction module is used as the input of the I+1th feature map compression and extraction module, and after feature map size compression and feature extraction, the I+1th backbone feature map and the I+1th feature map of interest are obtained, where I is an integer of 1≤I≤m-1.

4. The small target detection method according to claim 2 or 3, characterized in that: The head network includes m FPN modules and m weighted channel aggregation modules; The FPN module is used to perform upsampling and cross-layer fusion on an image based on the backbone feature map and the region of interest (RoI) feature map output by the backbone network to obtain a feature map with low-level visual information and high-level semantic information; The weighted channel aggregation module is used to perform attention fusion based on weighted channel splicing of the feature map output by the corresponding FPN module to obtain a feature map with enhanced features, and then perform feature extraction to obtain the corresponding target feature map; Among them, the (I + 1)-th FPN module performs attention fusion based on weighted channel splicing of the backbone feature map and the RoI feature map output by the I-th feature map compression and extraction module. The feature map obtained after fusion is subjected to feature extraction through a C3 (CSP Bottleneck with 3 Convolutions) layer to obtain the feature map output by the (I + 1)-th FPN module. The I-th target feature map is obtained by performing attention fusion based on weighted channel splicing on the output of the Conv layer of the I-th FPN module through the I-th weighted channel aggregation module, where I is an integer satisfying 1 ≤ I ≤ m - 1.

5. The small target detection method according to claim 4, characterized in that: The FPN module includes a Conv layer, an Upsamle layer, a first Weighted_Concat_Attention layer, and a C3 layer connected in sequence; Among them, the output of the C3 layer of the (I + 1)-th FPN module is the input of the Conv layer of the I-th FPN module; The input image of the first Weighted_Concat_Attention layer included in the k-th FPN module includes: the backbone feature map and the RoI feature map obtained by the (k - 1)-th feature map compression and extraction module of the backbone network, and the feature map output by the Upsample layer of the k-th FPN module, where k is an integer satisfying 1 < k ≤ m; The weighted channel aggregation module includes a second Weighted_Concat_Attention layer. The input image of the I-th weighted channel aggregation module includes: the feature map obtained by compressing the size of the (l - 1)-th target feature map through a Conv layer, the feature map output by the Conv layer of the l-th FPN module, the feature map output by the Upsample layer of the (l + 1)-th FPN module, and the l-th RoI feature map output by the backbone network, where l is an integer satisfying 1 ≤ l < m.

6. The small target detection method according to claim 5, wherein The RoI feature map obtained by performing feature extraction on the primary feature map through a CA attention layer and the feature map output by the Upsample layer of the first FPN module are input into the Weighted_Concat_Attention layer of the first FPN module for attention fusion based on weighted channel splicing to obtain the output of the first FPN module; The feature map of interest output by the mth feature map compression and extraction module and the backbone feature map output by the mth feature map compression and extraction module are input into the mth weighted channel aggregation module after the feature map spatial information fusion and channel number compression are performed by the SPPF layer and the Conv layer to obtain the feature map to obtain the mth target feature map.

7. The small target detection method according to claim 6, characterized in that: The Weighted_Concat_Attention layer performs attention fusion based on weighted channel concatenation of feature maps through the following formula: Where ε = 0.0001; n represents the number of input feature maps; Finput i represents the i-th input feature map; w i represents the weight of the i-th input feature map, w i Preset to the training variable of the model, preset w i ≥0; Concat() indicates channel concatenation operation; SE() indicates feature extraction based on SE attention module; CBAM() means feature extraction based on the CBAM attention module.

8. An electronic device, characterized in that: comprising at least one processor, and at least one memory communicatively connected to the processor; The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the small target detection method according to any one of claims 1 to 7.

Images