An Adaptive Small Target Detection Method Based on Attention Mechanism

Abstract

This invention discloses an adaptive small object detection method based on an attention mechanism, belonging to the field of computer vision technology. The method includes: acquiring a dataset; constructing a feature enhancement module; constructing an adaptive sparse convolutional detection head; constructing an adaptive small object detection model based on an attention mechanism, adding the feature enhancement module to the backbone network, and using the adaptive sparse convolutional detection head to replace the original detection head; and determining the final small object detection model. This invention constructs an adaptive sparse convolutional head, dynamically adjusting the use of the detection head corresponding to different FPN layers through a channel attention mechanism, effectively balancing model complexity and detection accuracy, improving the accuracy of small object detection while maintaining computational efficiency; and constructs a feature enhancement module, utilizing global average pooling to fuse global contextual information, enhancing the model's ability to represent the features of small objects, and improving the model's detection performance for objects of different sizes in complex environments.

Description

Technical Field

[0001] This invention relates to an adaptive small target detection method based on an attention mechanism, belonging to the field of computer vision technology. Background Technology

[0002] With the rapid development of computer vision technology, the detection of small targets has become increasingly important in many fields, such as autonomous driving, aerial surveillance, and remote sensing image analysis. In these applications, accurately and quickly detecting small targets in images or videos is crucial. However, because small targets occupy a limited number of pixels in an image, they are often easily affected by background noise, lighting changes, occlusion, and other factors, making it difficult for traditional computer vision algorithms to accurately locate and identify these targets.

[0003] Traditional multi-scale feature-based methods, exemplified by Feature Pyramid Networks (FPNs), improve the detection capability for objects of varying sizes by integrating deep and shallow features. For instance, some advanced object detection algorithms utilize FPN structures to extract information from feature maps at different levels, enabling the model to simultaneously consider both the overall semantic information and local details of the object. However, as the number of network layers increases and the need for multi-scale feature processing grows, the model's complexity also rises, significantly increasing the demand for computational resources. This makes it difficult to meet the real-time detection requirements of devices with limited computing resources, such as embedded devices and mobile devices.

[0004] To address the challenge of limited computing resources, many studies have attempted to employ sparse convolution techniques. Sparse convolution focuses on computing important features by performing convolution operations only in key regions of an image, thereby reducing computational load and memory usage. He S, Jiang C, Dong D, and others proposed a sparse dynamic convolution technique in their paper "Sd-conv: Towards the parameter-efficiency of dynamic convolution" (Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision. 2023.). The authors first designed a binary mask that prunes static convolution kernels based on a learnable threshold, removing redundancy from the large network to obtain a lightweight network, thus significantly reducing parameters and computational cost. Du B, Huang Y, Chen J, and Huang D proposed a Global Context Enhancement (CEASC) adaptive sparse convolutional network in their paper "Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images" (Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2023). This method saves computational resources by using sparse convolutions in the detection head. The CESC module in this network uses point-wise convolutions to generate global feature information. Furthermore, the authors use the mean and variance of this global feature information to perform group normalization on the feature map, thereby compensating for missing contextual information and improving detection accuracy. However, when sparse convolutions are applied to the detection heads corresponding to different layers of the FPN, new problems arise. In the lower layers of FPN, the image has high spatial resolution and rich detail information, which is suitable for capturing small objects. At this time, using sparse convolution can effectively reduce the amount of computation and retain key details. However, in the higher layers of FPN, after multiple downsampling, the space occupied by small objects in the image is limited, and the detailed information has been greatly reduced. Applying sparse convolution may further ignore this limited information, resulting in excessively sparse high-level feature maps, thereby losing contextual information and ultimately affecting the detection accuracy of small objects.Therefore, how to dynamically adjust the use of sparse convolution and supplement effective contextual feature information based on the feature characteristics of different FPN levels is a key issue in optimizing the performance of small object detection.

[0005] In summary, while existing micro-object detection technologies improve detection performance, they face numerous challenges such as limited computing resources, insufficient feature representation, and difficulty in balancing detection efficiency. There is an urgent need for a new technical solution to address these issues and meet the growing demands of practical applications. Summary of the Invention

[0006] The present invention aims to provide an adaptive small target detection method based on an attention mechanism, and aims to solve the technical problems existing in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an adaptive small target detection method based on an attention mechanism, comprising the following steps:

[0008] S1, Obtain the dataset: Collect images of small targets using an image acquisition device, label the images, and divide the dataset into a training set and a validation set;

[0009] S2, Construct the feature enhancement module;

[0010] S3, Construct an adaptive sparse convolutional detection head;

[0011] S4, Construct an adaptive small target detection model based on attention mechanism: including backbone network, neck network, and detection head, add feature enhancement module to backbone network, and use adaptive sparse convolution detection head to replace the original detection head;

[0012] S5, Determine the final small object detection model: Train an attention-based adaptive small object detection model using the training set, and evaluate the small object detection model using the validation set.

[0013] Furthermore, in S1, various types of drones are used to capture images of pedestrians, vehicles, and bicycles at different densities under different environments as the target dataset.

[0014] Furthermore, based on the RetinaNet framework, optimizations were made to the characteristics of each layer of FPN to improve the performance of small object detection.

[0015] Further, in step S2, the feature enhancement module obtains a global feature vector by performing global average pooling on the C5 layer feature map, adjusts the number of channels of this vector using 1×1 convolution, and then adds and fuses it with the C5 layer lateral connection feature map after bilinear interpolation upsampling, thereby improving the feature map representation of the FPNP5 layer. Specific steps include:

[0016] S21, Global average pooling. This operation performs global average pooling on the highest-level feature map C5, that is, calculates the average of the feature values for each channel, obtaining a global feature vector with the same number of channels and a spatial dimension of 1×1. This vector represents the global context information of the entire image.

[0017] S22, 1×1 convolution adjustment: Adjust the number of channels of the global feature vector obtained by global average pooling through a 1×1 convolutional layer to match the number of channels of the original highest-level feature map for subsequent fusion operations. S23, Bilinear interpolation upsampling. Upsample the global feature vector after 1×1 convolution adjustment through bilinear interpolation to make its spatial size match that of the C5 layer feature map.

[0018] S24, Feature fusion. Additively fuse the global feature map after bilinear interpolation upsampling with the horizontally connected feature map of the bilinear interpolation layer.

[0019] Furthermore, in S3, the adaptive sparse convolution detection head uses a channel attention mechanism to quantify the importance of each feature channel of the FPN layer, obtaining weights, and dynamically determines whether to use the sparse convolution detection head or the ordinary detection head according to these weights.

[0020] The ordinary detection head is a detection head that does not use sparse convolution and adopts a standard convolutional neural network structure, including a series of convolutional layers for feature extraction and object classification and localization. They perform convolutional operations at each position of the feature map without considering the importance of spatial positions.

[0021] The sparse convolution detection head focuses on key regions in the image by using sparse convolution and utilizes global context information, effectively enhancing the recognition ability for small targets while significantly reducing the consumption of computing resources in extensive invalid background regions. By applying a binary mask, the sparse convolution detection head can determine which feature units participate in the convolution process, thereby reducing the computational amount in the invalid background regions.

[0022] The adaptive process of the adaptive sparse convolution detection head includes the following steps:

[0023] S31, Use the channel attention mechanism to quantify the importance of each feature channel of the FPN layer, obtaining weights. Define the output of the FPN as a list of feature maps F, where each value f ,

[0025] ,

[0024] ,

[0023] , i , ,

[0022] ,<​​​​​​The feature map list F is processed using a channel attention network A to obtain an attention score list S, where S... i For the attention score output corresponding to the i-th layer, S can be expressed as:

[0026] S=A(F)=[s3,s4,s5,s6,s7];

[0027] S32, Set a threshold. Compare the attention score from step S31 with the threshold to determine whether to use a detection head with sparse convolution at a specific FPN level. The threshold is denoted as t. If s i If the condition is t, proceed to step S33; otherwise, proceed to step S34.

[0028] S33, the i-th layer of FPN uses a detection head H1 containing sparse convolutions for detection, and outputs the detection result r, which can be expressed as:

[0029] r = H1(f i );

[0030] S34, using a standard detection head H2 for detection, the detection result r is:

[0031] r = H2(f i ).

[0032] Further, in step S31, the channel attention network consists of a global average pooling layer and two fully connected layers connected in series. The first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the Sigmoid activation function. Specific steps include:

[0033] Step 1: Given a feature map of size C×H×W, perform global average pooling on the feature map, compressing its size to C×1×1 using the following formula:

[0034]

[0035] Where z c Let x represent the global average value of the C-th channel, which represents the feature map compressed from C×H×W to C×1×1, where C represents the number of channels, H represents the height of the feature map, W represents the width of the feature map, and x represents the width of the feature map. c (i,j) represents the element of the input feature map at channel C and spatial position (i,j).

[0036] Step 2: Put z c The input is processed through two fully connected layers. The first fully connected layer is used for feature dimensionality reduction, and the second fully connected layer further transforms the feature vector into a single scalar output.

[0037] For the first fully connected layer, use the following formula:

[0038] u c =ReLU(w·z) c +b)

[0039] Where u c It is the result after processing by the first fully connected layer and the ReLU activation function. w and b are the weights and biases of the first fully connected layer, respectively, and ReLU is the activation function.

[0040] For the second fully connected layer, use the following formula:

[0041] v c =σ(w′·u c +b′)

[0042] Where w′ and b′ are the weights and biases of the second fully connected layer, σ is the Sigmoid activation function, and v c For the final output, where v c It is a value between 0 and 1, representing the attention weight.

[0043] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the aforementioned adaptive small target detection method based on an attention mechanism.

[0044] A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned adaptive small target detection method based on an attention mechanism.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] 1. This invention employs an adaptive sparse convolution head that dynamically adjusts the use of sparse convolution in the detection heads corresponding to different FPN layers through a channel attention mechanism. Specifically, a channel attention module is inserted after the output of each FPN layer. This module quantifies the importance of feature channels at each layer to derive weights and makes decisions based on set thresholds to determine whether the detection head at each FPN layer should use sparse convolution. This method effectively reduces unnecessary computational overhead while retaining accurate detection capabilities for small targets at critical feature layers.

[0047] 2. This invention uses a channel attention mechanism to calculate weights. The channel attention mechanism is relatively simple to design and has high computational efficiency. It mainly consists of global average pooling and two fully connected layers, avoiding complex spatial operations and a large increase in parameters. This allows the channel attention mechanism to be effectively integrated into various network architectures without significantly increasing the computational burden.

[0048] 3. This invention employs a feature enhancement module that utilizes global average pooling to fuse global contextual information. Global average pooling is an operation with very few or no parameters, which does not significantly increase the number of parameters and computational cost of the model. Compared with complex contextual modeling methods, GAP provides a global perspective through simple operations, avoiding over-reliance on contextual information. This allows for the introduction of global information while not only improving the overall detection accuracy, but also keeping the model's resource consumption within a controllable range. By adding a feature enhancement module, the model's ability to represent features of small objects is enhanced, improving the model's detection performance for objects of different sizes in complex environments. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the basic process of the adaptive small target detection method in this invention;

[0050] Figure 2 This is a basic flowchart of the feature enhancement module in this invention;

[0051] Figure 3 This is a schematic diagram of the basic process of channel attention in this invention;

[0052] Figure 4 This is a diagram of the adaptive small target detection model based on the attention mechanism in this invention. Detailed Implementation

[0053] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0054] Example: Figure 1 As shown, this invention discloses an adaptive small target detection method based on an attention mechanism, comprising the following steps:

[0055] S1, Obtain the dataset: Collect images of small targets using an image acquisition device, label the images, and divide the dataset into a training set and a validation set;

[0056] In S1, various types of drones are used to capture images of pedestrians, vehicles, and bicycles at different densities in different environments as the target dataset.

[0057] S2, Construct the feature enhancement module;

[0058] In S2, the feature enhancement module obtains a global feature vector by performing global average pooling on the feature map of layer C5, adjusts the number of channels of the vector using 1×1 convolution, and then adds and fuses it with the lateral connection feature map of layer C5 after bilinear interpolation upsampling, thereby improving the feature map representation of layer FPNP5.

[0059] like Figure 2 As shown, the operation process of the feature enhancement module includes the following steps:

[0060] S21, Global Average Pooling: This operation performs global average pooling on the highest-level feature map C5, that is, it averages the feature values ​​of each channel to obtain a global feature vector with the same number of channels and a spatial dimension of 1×1. This vector represents the global context information of the entire image.

[0061] S22, 1×1 Convolution Adjustment: The number of channels in the global feature vector obtained by global average pooling is adjusted through a 1×1 convolutional layer. The purpose of this step is to match the number of channels in the original highest-level feature map so as to facilitate subsequent fusion operations.

[0062] S23, Bilinear Interpolation Upsampling, performs bilinear interpolation upsampling on the global feature vector adjusted by 1×1 convolution to match its spatial size with the feature map of layer C5.

[0063] S24, Feature Fusion, which adds and fuses the global feature map after bilinear interpolation upsampling with the feature map of the lateral connection of the bilinear interpolation layer.

[0064] Introducing a feature enhancement module can improve the object detection network's ability to detect objects at different scales, thereby improving the overall detection performance.

[0065] S3, Construct an adaptive sparse convolutional detection head;

[0066] In S3, the adaptive sparse convolutional detection head is used. This structure uses different detection heads for each layer of FPN. It uses a channel attention mechanism to quantify the importance of the feature channels of each FPN layer to obtain weights. Based on these weights, it dynamically decides whether to use a sparse convolutional detection head or a normal detection head.

[0067] The standard detection head does not use sparse convolution; it employs a standard convolutional neural network structure, including a series of convolutional layers for feature extraction and target classification and localization. These layers perform convolution operations at every location in the feature map, without considering the importance of spatial location.

[0068] The sparse convolution detection head effectively enhances the recognition of small targets by focusing on key regions in an image using sparse convolution and leveraging global contextual information, while significantly reducing computational resource consumption in extensive invalid background regions. By applying a binary mask, the sparse convolution detection head can determine which feature units participate in the convolution process, thereby reducing the computational load in invalid background regions.

[0069] The adaptive process of the adaptive sparse convolutional detection head includes the following steps:

[0070] S31, the importance of feature channels in each FPN layer is quantified using a channel attention mechanism to obtain weights. The output of the FPN is defined as a list of feature maps F, where each value f in the list... i The feature map output corresponds to a specific layer of FPN, where i represents the i-th layer of FPN, and 3 << i << 7. F can be expressed as:

[0071] F = [f3, f4, f5, f6, f7];

[0072] The feature map list F is processed using a channel attention network A to obtain an attention score list S, where S... i For the attention score output corresponding to the i-th layer, S can be expressed as:

[0073] S=A(F)=[s3,s4,s5,s6,s7];

[0074] In S31, such as Figure 3 As shown, the channel attention network consists of a global average pooling layer and two fully connected layers connected in sequence. The first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the Sigmoid activation function.

[0075] The specific steps include:

[0076] Step 1: Given a feature map of size C×H×W, perform global average pooling on the feature map, compressing its size to C×1×1 using the following formula:

[0077]

[0078] Where z c Let x represent the global average value of the C-th channel, which represents the feature map compressed from C×H×W to C×1×1, where C represents the number of channels, H represents the height of the feature map, W represents the width of the feature map, and x represents the width of the feature map. c (i,j) represents the element of the input feature map at channel c and spatial position (i,j).

[0079] Step 2: Put z cThe input is processed through two fully connected layers. The first fully connected layer is used for feature dimensionality reduction, and the second fully connected layer further transforms the feature vector into a single scalar output.

[0080] For the first fully connected layer, use the following formula:

[0081] u c =ReLU(w·z) c +b)

[0082] Where u c It is the result after processing by the first fully connected layer and the ReLU activation function. w and b are the weights and biases of the first fully connected layer, respectively, and ReLU is the activation function.

[0083] For the second fully connected layer, use the following formula:

[0084] v c =σ(w′·u c +b′)

[0085] Where w′ and b′ are the weights and biases of the second fully connected layer, σ is the Sigmoid activation function, and v c For the final output, where v c It is a value between 0 and 1, representing the attention weight.

[0086] S32, Set a threshold. Compare the attention score from step S31 with the threshold to determine whether to use a detection head with sparse convolution at a specific FPN level. The threshold is denoted as t. If s i If the condition is t, proceed to step S33; otherwise, proceed to step S34.

[0087] S33, the i-th layer of FPN uses a detection head H1 containing sparse convolutions for detection, and outputs the detection result r, which can be expressed as:

[0088] r = H1(f i );

[0089] S34, using a standard detection head H2 for detection, the detection result r is:

[0090] r = H2(f i );

[0091] The channel attention network processes the feature maps input from each FPN layer separately, outputting a weight value between 0 and 1. We set a threshold of 0.5 to determine whether to apply a sparse convolutional detection head at that layer. If the weight value exceeds the threshold, the sparse convolutional detection head is selected; otherwise, a regular detection head is selected. This method allows the model to adaptively select the most suitable detection head based on the specific context information of each layer, thereby optimizing overall performance and computational efficiency.

[0092] S4, Construct an adaptive small target detection model based on an attention mechanism: including a backbone network, a neck network, and a detection head, such as... Figure 4 As shown, the feature enhancement module is added to the backbone network, and an adaptive sparse convolutional detection head is used to replace the original detection head;

[0093] In S4, the model is built on the RetinaNet framework and optimized for the characteristics of each layer of FPN to improve the performance of small object detection.

[0094] S5, Determine the final small object detection model: Train an attention-based adaptive small object detection model using the training set, and evaluate the small object detection model using the validation set.

[0095] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. An adaptive small target detection method based on an attention mechanism, characterized in that: The method includes the following steps: S1, Obtain the dataset: Acquire images of small targets using image acquisition equipment, label the images, and divide the dataset into training and validation sets. S2, Construct the feature enhancement module S3, Construct an adaptive sparse convolution detection head S4. Construct an adaptive small target detection model based on an attention mechanism: including a backbone network, a neck network, and a detection head. A feature enhancement module is added to the backbone network, and an adaptive sparse convolutional detection head is used instead of the original detection head. S5, Determine the final small object detection model: Train an attention-based adaptive small object detection model using the training set, and evaluate the small object detection model using the validation set; Wherein: in S2, the feature enhancement module enhances the feature by... Global average pooling is performed on the layer feature maps to obtain global feature vectors, which are then used... Convolution adjusts the number of channels in the vector, and after bilinear interpolation upsampling, it is then compared with... Layer feature maps are added and fused to improve FPN. The feature map representation of the layer includes the following steps: S21, Global Average Pooling, for the highest layer feature map Perform global average pooling, which involves averaging the feature values ​​of each channel to obtain a pool with the same number of channels and a spatial dimension of 1. The global feature vector, which represents the global context information of the entire image. S22, Convolution adjustment: via The convolutional layers adjust the number of channels in the global feature vector obtained by global average pooling to match the number of channels in the original highest-level feature map, thus facilitating subsequent fusion operations. S23, bilinear interpolation upsampling, will be processed by... The global feature vector after convolution adjustment is upsampled using bilinear interpolation to make its spatial size equal to... Layer feature maps are matched. S24, Feature Fusion: The global feature map after bilinear interpolation upsampling is added and fused with the feature map of the lateral connection of the bilinear interpolation layer; In step S3, the adaptive sparse convolutional detection head uses a channel attention mechanism to quantify the importance of feature channels in each FPN layer, obtaining weights. Based on these weights, it dynamically determines whether to use a sparse convolutional detection head or a regular detection head. The adaptive process of the adaptive sparse convolutional detection head includes the following steps: S3, Construct an adaptive sparse convolution detection head S31, the importance of feature channels in each FPN layer is quantified using a channel attention mechanism to obtain weights, and the output of the FPN is defined as a list of feature maps. , where each value in the list The feature map output corresponds to a specific level of FPN. The first representing FPN Layer, in which , Represented as: ； Using channel attention network For the feature map list The process is performed to obtain a list of attention scores. ,in Corresponding to FPN Layer attention score output, Represented as: ； S32, Set a threshold: Compare the attention score from step S31 with the threshold to determine whether to use a detection head with sparse convolution at a specific FPN level. The threshold is expressed as... ,if If yes, proceed to step S33; otherwise, proceed to step S34. S33, FPN's The layer uses a detection head containing sparse convolution. Perform the test and output the test results. , Represented as: ； S34, using a standard detection head Tests were conducted, and the test results were obtained. for: 。 2. The adaptive small target detection method based on attention mechanism according to claim 1, characterized in that: In S1, various types of drones are used to capture images of pedestrians, vehicles, and bicycles at different densities under different environments as the target dataset.

3. The adaptive small target detection method based on attention mechanism according to claim 2, characterized in that: In step S4, based on the RetinaNet framework, optimizations are made to the characteristics of each layer of FPN to improve the performance of small object detection.

4. The adaptive small target detection method based on attention mechanism according to claim 3, characterized in that: In step S31, the channel attention network consists of a global average pooling layer and two fully connected layers connected in sequence. The first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the Sigmoid activation function. The specific steps include: Step 1: Given a feature map of size , Global average pooling is performed on the feature map, and the size of the feature map is compressed using the following formula. : in Indicates the first The global average value of each channel, representing the value from... Compressed to The feature map, where C represents the number of channels. Indicates the height of the feature map. Indicates the width of the feature map. This indicates that the input feature map is in the channel and spatial location elements, Step Two: Put The input is processed through two fully connected layers. The first fully connected layer is used for feature dimensionality reduction, and the second fully connected layer further transforms the feature vector into a single scalar output. For the first fully connected layer, use the following formula: in It is the result after processing through the first fully connected layer and the ReLU activation function. and These are the weights and biases of the first fully connected layer, and ReLU is the activation function. For the second fully connected layer, use the following formula: in, and These are the weights and biases of the second fully connected layer. It is the Sigmoid activation function, where It is a value between 0 and 1, representing the attention weight.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements an adaptive small target detection method based on an attention mechanism as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing computer instructions thereon, characterized in that: When executed by a processor, the computer instructions implement an adaptive small target detection method based on an attention mechanism as described in any one of claims 1-4.

Images