Target detection method based on multi-scale flexible attention and electronic equipment
By combining ConvNeXt and FPN with a multi-scale flexible attention mechanism, the problems of boundary ambiguity and positional offset in small object detection are solved, improving detection accuracy and robustness, and achieving faster training convergence and higher detection efficiency.
Patent Information
- Application Number
- CN202511097365.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-06
- Publication Date
- 2025-11-18
AI Technical Summary
Existing end-to-end target detection methods suffer from problems such as blurred boundaries, positional offset, and high false negative rate in small target detection. They also lack the ability to perceive local salient regions in a fine manner, making it difficult to effectively capture discriminative features in complex backgrounds, resulting in unstable detection performance.
A multi-scale flexible attention mechanism is adopted, which combines ConvNeXt and FPN for feature extraction to generate a multi-scale fused feature map. Multi-scale flexible attention computation is introduced during the decoding process, and attention aggregation is performed using local receptive window regions to enhance the perception of small targets.
It improves the accuracy and recall of small object detection, enhances the training efficiency and convergence speed of the model, strengthens the model's adaptability to complex scenes, and reduces the consumption of computing resources and training complexity.
Smart Images
Figure CN120976756A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image processing technology, specifically to a target detection method and electronic device based on multi-scale flexible attention. Background Technology
[0002] Existing end-to-end object detection methods, such as DETR (Detection Transformer), have made significant progress in eliminating traditional anchor settings and post-processing steps. However, because these methods use global feature maps of fixed resolution for object modeling, they struggle to obtain sufficient spatial resolution for small objects in images, leading to problems such as blurred boundaries, severe positional shifts, and high false negative rates. Furthermore, small objects are often accompanied by complex background factors such as occlusion, dense distribution, and low texture information. Existing methods lack the ability to finely perceive local salient regions, making it difficult to effectively capture their discriminative features and severely impacting overall detection performance. Therefore, improving the accuracy and stability of object detection models in small object scenarios has become an important research direction in this field.
[0003] DETR-based object detection methods generally rely on large-scale data training and long training iterations to achieve stable performance. This is primarily because these methods employ a decoding structure entirely based on a global self-attention mechanism, making it difficult to quickly establish the semantic correspondence between queries and targets during the early training phase. Furthermore, the lack of a multi-scale explicit feature modeling mechanism results in uneven perception capabilities of the model across different scales, further exacerbating the problems of unstable matching and optimization difficulties during training. Therefore, how to accelerate the model's convergence process by introducing structure-guided attention strategies and multi-scale semantic fusion mechanisms has become a significant technical challenge in improving the practicality of end-to-end object detection methods. Summary of the Invention
[0004] The purpose of this invention is to provide a target detection method and electronic device based on multi-scale flexible attention that can improve detection accuracy and model training and detection efficiency.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A target detection method based on multi-scale flexible attention includes the following steps:
[0007] S1. Preliminary Model Establishment: A preliminary model is established based on the DETR model;
[0008] S2. Establish a feature extraction module: Use ConvNeXt and FPN to extract features and obtain four feature maps with different resolutions, and generate a multi-scale fused feature map;
[0009] S3. Model Optimization: In the process of Decoder cross-attention calculation, multi-scale flexible attention calculation is used to improve the detection performance of the model.
[0010] S4. Image Input: Input the image to be detected into the multi-scale flexible attention target detection model obtained in step S1, and output the target detection result.
[0011] Preferably, the specific process of establishing the feature extraction module in step S2 includes:
[0012] The input image is sequentially fed into multiple stages of the ConvNeXt backbone network, and four feature maps of different resolutions are extracted respectively. The four feature maps are then input into the Feature Pyramid Network (FPN) for multi-scale fusion, and the channel dimensions are unified to generate a multi-scale fused feature map.
[0013] Preferably, the ConvNeXt backbone network includes: a set of convolutional modules for initial image processing, each convolutional module being connected to four sequentially connected stages, each stage consisting of a stack of several ConvNeXt modules.
[0014] Preferably, the ConvNeXt module includes: a depthwise separable convolutional layer, a normalization layer, an activation function, and a residual connection structure; the four stages output feature maps with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively, and feature extraction at different scales is achieved through a downsampling operation with a stride of 2.
[0015] Preferably, the structure of the Feature Pyramid Network (FPN) includes: multiple lateral connection modules and a top-down feature fusion path; the lateral connection modules are used to perform 1×1 convolution on the feature maps output from different stages to unify the channel dimension; the feature fusion path transmits the semantic information of the high-level feature maps to the low-level feature maps through upsampling operations, and performs element-wise summation with the corresponding lateral feature maps to generate multi-scale fused feature maps for use by the subsequent object detection module.
[0016] Preferably, the multi-scale flexible attention in the optimization model in step S3 is obtained based on cross-attention improvement, and its specific process includes:
[0017] For each decoded query vector, on the fused feature map of multiple resolutions obtained in step S2, a receptive window region of fixed size is defined with the spatial location indicated by the decoded query vector as the center. Within this window region, based on the degree of matching between the query content and local features, several locations with high responsivity are adaptively selected for attention calculation and feature aggregation, thereby enhancing selective attention to salient target regions while maintaining multi-scale context awareness.
[0018] Preferably, the method for generating the spatial reference position of the decoding query vector is as follows:
[0019] During model initialization, a two-dimensional positional code is assigned to each decoded query vector. The positional code is obtained by uniform sampling in the input image space, representing its initial spatial position. During training, the saliency response information in the fused feature map output by the multi-scale feature extraction module is combined to calculate the response intensity of each query vector in the feature maps at different scales. Based on the output result of the current decoder layer at the previous time step and the location of the local response maximum on the feature map, the initial positional code is updated and corrected to obtain accurate spatial reference position coordinates. After multiple iterations, the final spatial reference position coordinates are obtained. These coordinates are used to define the local receptive window region in each scale feature map and serve as the attention sampling center in the multi-scale flexible attention mechanism to achieve focusing on the potential target region.
[0020] Preferably, the output features after inputting the image in step S4 are:
[0021]
[0022] in, Indicates that ultimately by The output result is obtained by weighted summation of the value vectors of the positions; Top- This represents the set of the K largest positions with the highest matching degree, where K is a hyperparameter that controls the sparsity of the local attention. Indicates query q i A fixed-size local receptive window region is defined in the feature map, centered on the spatial location of α. ij The matching degree s ij exist Attention weights obtained within the region after softmax normalization; v j This represents the actual image feature value extracted at that location; s ij Indicates query q i With feature k at position j in the window j The matching score.
[0023] An electronic device includes a memory and a processor, characterized in that the memory stores a computer program, and the processor, when executing the computer program, implements a target detection method such as any of the above-described multi-scale flexible attention methods.
[0024] The beneficial effects of this invention are:
[0025] This invention presents a target detection model based on multi-scale flexible attention. It employs a feature extraction module combining ConvNeXt and N, enabling the extraction of rich semantic feature maps at different spatial resolutions, achieving effective fusion from layer-by-layer structural information to deep semantic information. The input image first passes through four stages of convolutional modules and a ConvNeXt backbone network, generating multi-scale feature maps at four different resolutions. Each stage consists of multiple ConvNeXt modules containing depthwise separable convolutions, normalization, activation functions, and residual connections. The extracted multi-scale features are then input into a Feature Pyramid Network (FPN) for cross-layer information fusion, unifying the channel dimensions to obtain a fused feature map with rich semantic and spatial information. This module not only enhances the model's ability to perceive targets at different scales, especially small targets, but also provides a high-quality feature foundation for subsequent flexible attention mechanisms, significantly improving detection accuracy and robustness.
[0026] This invention presents a target detection model based on multi-scale flexible attention. By introducing a flexible attention mechanism onto a multi-scale fused feature map, for each decoded query vector, a fixed-size receptive window region is first defined on the multi-scale fused feature map, centered on the spatial location indicated by the query. Then, within this window region, the responsiveness of each location is evaluated based on the similarity or semantic relevance between the query vector and local features. Several key locations with high responsiveness are adaptively selected for attention computation and feature aggregation. This mechanism avoids the significant redundant computation problems inherent in traditional global attention mechanisms by focusing on locally salient regions, while simultaneously improving the model's adaptability to changes in spatial structure. Compared to existing global attention or fixed sampling methods, the flexible attention method of this invention can more accurately capture weak but crucial image information such as edges, textures, and shapes of small targets, thereby effectively improving the recall and accuracy of small target detection. Furthermore, since aggregation operations are performed only at local key locations, this mechanism also improves the model's training efficiency and convergence speed.
[0027] This invention introduces a spatially guided query location generation strategy based on a multi-scale flexible attention-based object detection model to efficiently determine the region center for attention computation. Specifically, during model initialization, the system assigns a two-dimensional location code to each decoded query vector. This code is obtained through uniform sampling in the input image space and represents the initial spatial reference location. During training, the system dynamically evaluates the attention focus of each query vector at different scales by combining the saliency response intensity of each location in the multi-scale fusion feature map extracted by ConvNeXt and FPN. It also iteratively updates and corrects the initial location code by referencing the historical output of the previous layer of the decoder and the local region with the largest response value in the current feature map, ultimately determining accurate spatial reference coordinates. These spatial reference coordinates are used to construct a local receptive window on the feature map at each scale and serve as the sampling center for the multi-scale flexible attention mechanism, focusing attention computation on key regions with high semantic responses. In this way, this invention effectively limits the computational scope of the attention mechanism, avoids the high-dimensional redundant computation caused by traditional full-image fully connected layers, significantly reduces resource consumption and training complexity, thereby accelerating model convergence and improving optimization stability and detection efficiency.
[0028] This invention constructs a test sample set based on a multi-scale flexible attention target detection model combined with various data augmentation strategies, including image stitching, occlusion simulation, and blur perturbation, which can test the model's ability to adapt to changes in target shape in complex scenes. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the detection model structure in Embodiment 1 of the present invention;
[0030] Figure 2 This is a schematic diagram of the specific structure of ConvNeXt and FPN in Embodiment 1 of the present invention;
[0031] Figure 3 This is a schematic diagram of the specific architecture of the multi-scale flexible attention in Embodiment 1 of the present invention;
[0032] Figure 4 This is a schematic diagram of the specific architecture for generating the reference position of the decoding query vector space in Embodiment 1 of the present invention;
[0033] Figure 5 This is a diagram showing the model comparison experiment results in a small target scenario in Embodiment 3 of the present invention;
[0034] Figure 6 This is a diagram showing the experimental results of model comparison in an occluded scene in Embodiment 4 of the present invention;
[0035] Figure 7 This is a diagram showing the results of a model comparison experiment in a low-visibility scene in Embodiment 5 of the present invention. Detailed Implementation
[0036] Example 1
[0037] The following is a further explanation of the present invention in conjunction with specific embodiments, such as... Figure 1 As shown, this embodiment is a target detection method based on multi-scale flexible attention, including the following steps:
[0038] S1. Initial Model Establishment: The initial model is established based on the DETR model.
[0039] S2. Establish a feature extraction module: Use ConvNeXt and FPN to extract features and obtain four feature maps with different resolutions, and generate a multi-scale fused feature map;
[0040] like Figure 2 As shown, the input image is sequentially fed into the four stages (Stage 1-Stage 4) of the ConvNeXt backbone network, and four feature maps (C1-C4) of different resolutions are extracted respectively. The four feature maps are then input into the Feature Pyramid Network (FPN) for multi-scale fusion, and the channel dimensions are unified to generate multi-scale fused feature maps.
[0041] The ConvNeXt backbone network includes: a set of convolutional modules for initial image processing, each convolutional module being connected to four sequentially connected stages, each stage consisting of several stacked ConvNeXt modules.
[0042] The ConvNeXt module includes: depthwise separable convolutional layers, normalization layers, activation functions, and residual connection structures; the four stages output feature maps with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively, and feature extraction at different scales is achieved through downsampling operations with a stride of 2.
[0043] The Feature Pyramid Network (FPN) consists of multiple lateral connection modules and a top-down feature fusion path. The lateral connection modules are used to perform 1×1 convolutions on the feature maps output from different stages to unify the channel dimension. The feature fusion path passes the semantic information of the high-level feature maps to the low-level feature maps through upsampling operations and adds them element-wise with the corresponding lateral feature maps to generate multi-scale fused feature maps for use by the subsequent object detection module.
[0044] S3. Model Optimization: In the process of Decoder cross-attention calculation, multi-scale flexible attention calculation is used to improve the detection performance of the model.
[0045] like Figure 3 As shown, multi-scale flexible attention is obtained based on the improvement of cross-attention, and its specific process includes:
[0046] For each decoded query vector, on the fused feature map of multiple resolutions obtained in step S2, a receptive window region of fixed size is defined with the spatial location indicated by the decoded query vector as the center. Within this window region, based on the degree of matching between the query content and local features, several locations with high responsivity are adaptively selected for attention calculation and feature aggregation, thereby enhancing selective attention to salient target regions while maintaining multi-scale context awareness.
[0047] like Figure 4 As shown, the method for generating the spatial reference position of the decoding query vector is as follows:
[0048] During model initialization, a two-dimensional positional code is assigned to each decoded query vector. The positional code is obtained by uniform sampling in the input image space, representing its initial spatial position. During training, the saliency response information in the fused feature map output by the multi-scale feature extraction module is combined to calculate the response intensity of each query vector in the feature maps at different scales. Based on the output result of the current decoder layer at the previous time step and the location of the local response maximum on the feature map, the initial positional code is updated and corrected to obtain accurate spatial reference position coordinates. After multiple iterations, the final spatial reference position coordinates are obtained. These coordinates are used to define the local receptive window region in each scale feature map and serve as the attention sampling center in the multi-scale flexible attention mechanism to achieve focusing on the potential target region.
[0049] S4. Image Input: Input the image to be detected into the multi-scale flexible attention target detection model obtained in step S1, and output the target detection result.
[0050] The output features are:
[0051]
[0052] in, Indicates that ultimately by The output result is obtained by weighted summation of the value vectors of positions; This represents the set of the K largest positions with the highest matching degree, where K is a hyperparameter that controls the sparsity of the local attention. Indicates query q i A fixed-size local receptive window region is defined in the feature map, centered on the spatial location of α. ij The matching degree s ij exist Attention weights obtained within the region after softmax normalization; v j This represents the actual image feature value extracted at that location; s ijIndicates query q i With feature k at position j in the window j The matching score.
[0053] Example 2
[0054] This embodiment presents a method for comparing the performance of target detection models, including the following steps:
[0055] S1: Perform various data enhancement processes on the original image data, specifically including:
[0056] Mosaic data augmentation technology is used to stitch multiple images into one to simulate the distribution of multiple targets in complex scenes; random region erasure technology is used to occlude some target areas in the image to simulate an occluded environment; and Gaussian noise and Gaussian blur processing technology is used to blur the image to simulate a low visibility scene.
[0057] S2: Input the image enhanced by S1 into multiple target detection models to be compared, and obtain the output detection result image of each model; compare and analyze the detection results of each model on the enhanced image to evaluate the detection completeness and robustness of different models in complex scenarios such as small target detection, occluded target detection and low-quality image detection.
[0058] Example 3
[0059] This embodiment compares the performance of this application with other existing target detection models according to the target detection model performance comparison method in Embodiment 2. Figure 5 As shown, in this embodiment, an array of small target scenes is used as the original image. By comparing the method in Embodiment 1 of this application with DETR, it can be seen that the speed at which this application detects targets in the image is significantly improved compared to the existing DETR method.
[0060] Example 4
[0061] This embodiment compares the performance of this application with other existing target detection models according to the target detection model performance comparison method in Embodiment 2. Figure 6 As shown, in this embodiment, the image under the array occlusion scene is used as the original image. By comparing the method in Embodiment 1 of this application with the DETR method, it can be seen that the speed at which the target in the image is detected by this application is significantly improved compared with the existing DETR method, and it can detect the target in the image that DETR cannot detect more accurately.
[0062] Example 5
[0063] This embodiment compares the performance of this application with other existing target detection models according to the target detection model performance comparison method in Embodiment 2. Figure 7 As shown, in this embodiment, the three images are processed with Gaussian blur and Gaussian noise respectively, and then the target detection is compared with the method in Embodiment 1 of this application and the DETR method respectively.
[0064] Comparing the detection results of this application and the DETR method under different degrees of image degradation (such as Gaussian blur kernel sizes of 3×3 and 5×5, or Gaussian noise standard deviations of 0.01 and 0.05), we found that:
[0065] 1. In Gaussian blur scenarios, the DETR method is prone to false detection or missed detection due to blurred target edges, while the method in this application can still maintain high target localization accuracy, especially when the boundaries of small targets are severely blurred, it still has clear segmentation capabilities.
[0066] 2. In Gaussian noise scenarios, the detection box offset and confidence level of DETR decrease more significantly, while the method in this application can still output a detection box with higher confidence under noise interference, demonstrating stronger robustness to noise disturbance.
[0067] 3. From the perspective of detection integrity, when processing multi-target images, the method of this application can still detect more effective targets when the target is small or occluded, and the false negative rate is significantly lower than that of DETR.
[0068] 4. From the perspective of robustness analysis, the method of this application maintains low performance fluctuation under test conditions where the degree of image quality degradation is constantly increasing, and its detection results are more stable than DETR.
[0069] In summary, this application not only outperforms existing DETR methods in terms of recognition speed, but also demonstrates higher detection integrity and stronger robustness in complex degradation scenarios, further verifying the effectiveness and practicality of the performance comparison method proposed in Example 2.
[0070] The above description is merely a further explanation of the present invention in conjunction with specific embodiments. All descriptions made do not imply any limitation on the scope of protection of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A target detection method based on multi-scale flexible attention, characterized in that, Includes the following steps: S1. Preliminary Model Establishment: A preliminary model is established based on the DETR model; S2. Establish a feature extraction module: Use ConvNeXt and FPN to extract features and obtain four feature maps with different resolutions, and generate a multi-scale fused feature map; S3. Model Optimization: In the process of Decoder cross-attention calculation, multi-scale flexible attention calculation is used to improve the detection performance of the model; S4. Image Input: Input the image to be detected into the multi-scale flexible attention target detection model obtained in step S1, and output the target detection result.
2. The target detection method based on multi-scale flexible attention according to claim 1, characterized in that: The steps The specific process of establishing the feature extraction module in S2 includes: The input image is sequentially fed into multiple stages of the ConvNeXt backbone network, and four feature maps of different resolutions are extracted respectively. The four feature maps are then input into the Feature Pyramid Network (FPN) for multi-scale fusion, and the channel dimensions are unified to generate a multi-scale fused feature map.
3. The target detection method based on multi-scale flexible attention according to claim 3, characterized in that: The ConvNeXt backbone network includes: a set of convolutional modules for initial image processing, each convolutional module being connected to four sequentially connected stages, each stage consisting of several stacked ConvNeXt modules.
4. The target detection method based on multi-scale flexible attention according to claim 3, characterized in that: The ConvNeXt module includes: depthwise separable convolutional layers, normalization layers, activation functions, and residual connection structures; it outputs feature maps with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 in four stages, and achieves feature extraction at different scales through downsampling operations with a stride of 2.
5. The target detection method based on multi-scale flexible attention according to claim 2, characterized in that: The structure of the Feature Pyramid Network (FPN) includes: multiple lateral connection modules and a top-down feature fusion path; the lateral connection modules are used to perform 1×1 convolution on the feature maps output at different stages to unify the channel dimension; the feature fusion path transmits the semantic information of the high-level feature maps to the low-level feature maps through upsampling operations, and performs element-wise summation with the corresponding lateral feature maps to generate multi-scale fused feature maps for use by the subsequent object detection module.
6. The target detection method based on multi-scale flexible attention according to claim 2, characterized in that: The multi-scale flexible attention in the optimization model in step S3 is obtained based on the improvement of cross-attention, and its specific process includes: For each decoded query vector, on the fused feature map of multiple resolutions obtained in step S2, a receptive window region of fixed size is defined with the spatial location indicated by the decoded query vector as the center. Within this window region, based on the degree of matching between the query content and local features, several locations with high responsivity are adaptively selected for attention calculation and feature aggregation, thereby enhancing selective attention to salient target regions while maintaining multi-scale context awareness.
7. The target detection method based on multi-scale flexible attention according to claim 6, characterized in that: The method for generating the spatial reference position of the decoding query vector is as follows: During the model initialization phase, a two-dimensional position code is assigned to each decoded query vector. The position code is obtained by uniform sampling in the input image space and represents its initial spatial position. During training, the saliency response information in the fused feature map output by the multi-scale feature extraction module is combined to calculate the response intensity of each query vector in the feature maps at different scales. Based on the output result of the current decoder layer at the previous moment and the location of the local response maximum value on the feature map, the initial position encoding is updated and corrected to obtain the accurate spatial reference position coordinates. After multiple rounds of iteration, the final spatial reference position coordinates are obtained. These coordinates are used to define the local receptive window region in each scale feature map and serve as the attention sampling center in the multi-scale flexible attention mechanism to achieve focusing on the potential target region.
8. The target detection method based on multi-scale flexible attention according to claim 7, characterized in that: The output features of the input image in step S4 are: in, Indicates that ultimately by The output result is obtained by weighted summation of the value vectors of positions; This represents the set of the K largest positions with the highest matching degree, where K is a hyperparameter that controls the sparsity of the local attention. Indicates query q i A fixed-size local receptive window region is defined in the feature map, centered on the spatial location of α. ij The matching degree s ij exist Attention weights obtained within the region after softmax normalization; v j This represents the actual image feature value extracted at that location; s ij Indicates query q i With feature k at position j in the window j The matching score.
9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the multi-scale flexible attention target detection method as described in any one of claims 1-8.