A small target detection method based on semantic compression enhancement

Abstract

This invention belongs to the field of object detection technology and relates to a small object detection method based on semantic compression enhancement. The method includes the following steps: acquiring test image data containing small objects and scaling the image to obtain scaled image data; using a backbone network to extract multi-scale features from the scaled image data to obtain multi-scale features; using a semantic compression enhancement encoder to perform multi-scale semantic compression and global feature enhancement on the multi-scale features, and fusing the enhanced global features with the multi-scale features to obtain encoder output features; and inputting the encoder output features into a decoder to obtain the object classification result and regression box. This small object detection method significantly reduces the number of parameters and computational cost, and improves the average accuracy of small object detection.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a small target detection method based on semantic compression enhancement. Background Technology

[0002] In practical applications such as drone aerial photography and remote sensing image analysis, small target detection plays a crucial role. Small targets in these scenarios typically have significant characteristics such as low pixel ratio, sparse feature information, and lack of detailed texture. They are also easily affected by factors such as complex background noise, changes in lighting, and dense target distribution, making detection far more difficult than for targets of regular size.

[0003] With the rapid development of deep neural networks, the DETR (Detection Transformer) series of object detection methods have achieved breakthroughs on general object detection datasets such as COCO (Common Objects in Context) thanks to their end-to-end detection framework and global feature modeling capabilities. However, when applied to typical small object detection datasets such as AITOD (Aerial Image TinyObject Detection) and VisDrone (Vision Meets Drones), existing DETR methods reveal shortcomings: DETR methods rely on multi-scale feature inputs from the backbone network. However, in the process of acquiring high-level semantic features through layer-by-layer downsampling, the backbone network inevitably suffers from severe loss of detailed information about small objects, making it difficult for the encoder to capture effective features sufficient to distinguish the target from the background, resulting in a significant increase in the false negative rate. To compensate for the lack of features in small objects, introducing shallow features into the encoder for computation has become an effective solution. However, shallow features contain a large amount of redundant background information, which not only causes computational redundancy in the feature interaction process, leading to a sharp increase in the number of parameters and floating-point operations, but also exacerbates the risk of small object features being overwhelmed by background noise, while also introducing significant inference latency, making it difficult to meet the needs of real-time detection scenarios.

[0004] Furthermore, traditional self-attention mechanisms lack specificity when processing global features, are insufficient in distinguishing between foreground and background features of small targets, and cannot efficiently focus on key information in the region where small targets are located, further restricting the improvement of small target detection accuracy.

[0005] In summary, how to enhance the DETR method's ability to extract and strengthen features of small targets while ensuring computational efficiency, reduce information loss during feature interaction, and achieve accurate detection of small targets has become a core issue that urgently needs to be addressed in the application of the DETR method in the field of small target detection. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides a small target detection method based on semantic compression enhancement. The technical problem to be solved by this invention is achieved through the following technical solution:

[0007] This invention provides a small target detection method based on semantic compression enhancement, comprising the following steps:

[0008] Acquire the image data of the test image containing small targets and perform image scaling to obtain scaled image data;

[0009] The scaled image data is used to extract multi-scale features using a backbone network to obtain multi-scale features.

[0010] The multi-scale features are subjected to multi-scale semantic compression and global feature enhancement using a semantic compression enhancement encoder. The enhanced global features are then fused with the multi-scale features to obtain the encoder output features.

[0011] The encoder output features are input into the decoder to obtain the target classification result and regression box.

[0012] In one embodiment of the present invention, the scaled image data has a shape of The tensor of, where, and These are the length and width of the scaled image data.

[0013] In one embodiment of the present invention, the backbone network includes a ResNet neural network;

[0014] The multi-scale features include a first-scale feature, a second-scale feature, and a third-scale feature, wherein the first-scale feature is a shape of The tensor of the second scale feature is of shape as The tensor of the third scale feature is of shape as tensor, and These are the length and width of the scaled image data.

[0015] In one embodiment of the present invention, the semantic compression enhancement encoder includes a multi-scale semantic compression module, a foreground sampling enhancement module, and a spatial information recovery module, wherein,

[0016] The multi-scale semantic compression module is used to compress and fuse the multi-scale features to obtain global features;

[0017] The foreground sampling and enhancement module is used to sample and enhance the foreground information of the global features to obtain enhanced global features;

[0018] The spatial information recovery module is used to fuse the enhanced global features and the multi-scale features to obtain the encoder output features.

[0019] In one embodiment of the present invention, the calculation method in the multi-scale semantic compression module includes:

[0020] The first downsampling module is used to downsample the features at the first scale to obtain the first downsampled features; the first convolutional block attention module is used to aggregate the features of the first downsampled features to obtain the first target concept information.

[0021] ;

[0022] in, Indicates the first-scale feature. This indicates the first downsampling module. This represents the attention module of the first convolutional block. This indicates the primary objective concept information;

[0023] The second convolutional block attention module is used to aggregate features at the second scale to obtain the second target concept information:

[0024] ;

[0025] in, This represents the second-scale feature. This represents the attention module of the second convolutional block. This indicates the concept information of the second objective;

[0026] The first upsampling module is used to upsample the third-scale features to obtain high-level features:

[0027] ;

[0028] in, Indicates the third-scale feature. Indicates the first upsampling module. Indicates high-level characteristics;

[0029] A gating module is used to convert the semantic knowledge within the high-level features into gate signals:

[0030] ;

[0031] in, Indicates the gate control module. Indicates the gate open signal;

[0032] The first complementary information enhancement module multiplies the gate signal with the first target concept information, and adds the multiplication result back to the first target concept information to obtain the first low-level feature:

[0033] ;

[0034] in, This represents the first low-level feature;

[0035] The second complementary information enhancement module multiplies the gate signal with the second target concept information, and adds the multiplication result to the second target concept information to obtain the second low-level feature:

[0036] ;

[0037] in, Indicates the second low-level feature;

[0038] By concatenating the first low-level feature, the second low-level feature, and the high-level feature, a multi-layer feature is obtained:

[0039] ;

[0040] in, Represents multi-layer features, Indicates the splicing module;

[0041] The first convolutional fusion module is used to map the multi-layer features to the global features:

[0042] ;

[0043] in, Represents global features. This indicates the first convolutional fusion module.

[0044] In one embodiment of the present invention, the calculation formula of the gating module is:

[0045] ;

[0046] in, This represents a convolutional module with C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the activation function.

[0047] In one embodiment of the present invention, the calculation method in the first convolutional fusion module includes:

[0048] The first convolutional module is used to process the multi-layer features to obtain the first convolutional features: ;

[0049] The second convolutional module is used to process the multi-layer features to obtain the second convolutional features: ;

[0050] The first convolutional feature is processed by sequentially connected first, second, and third sub-units to obtain the output feature of the repeating unit. The first, second, and third sub-units have the same structure, each including a third convolutional module, a fourth convolutional module, a fifth convolutional module, and an addition module. The output feature of the repeating unit is: ;

[0051] The second convolutional feature and the output feature of the repeating unit are added together to obtain the global feature: ;

[0052] in, This represents the first convolutional module. Represents the first convolutional feature. This indicates the second convolutional module. Indicates the second convolution feature. and The input channels are all 3×C, the output channels are all C, and the convolution kernels are all 1×1. This indicates the third convolutional module. This indicates the fourth convolutional module. This represents the fifth convolutional module. and The input channels are all C, the output channels are all C, and the convolution kernels are all 1×1. The input channel is C, the output channel is C, the convolution kernel is 3×3, the groups parameter is C, and the padding is 1; This indicates the output feature of the first sub-unit. Indicates the output characteristics of repeating units; Indicates will Repeat three times.

[0053] In one embodiment of the present invention, the calculation method in the foreground sampling enhancement module includes:

[0054] The global features are processed using a sixth convolutional module, a seventh convolutional module, and an activation function to output a score:

[0055] ;

[0056] in, To score points, As a global feature, This is the sixth convolutional module with C input channels, C output channels, and a 1×1 kernel. This is the seventh convolutional module with C input channels, the number of categories in the current dataset as the output channel, and a 1×1 kernel. For activation functions;

[0057] Using the Topk operation, the index corresponding to a preset proportion of the data is selected based on the score:

[0058] ;

[0059] in, Indicates an index. This indicates that the preset ratio is 0.25;

[0060] Using the Gather operation, features at corresponding positions in the global features are selected as foreground features based on the index:

[0061] ;

[0062] in, Foreground features;

[0063] Enhanced foreground features are obtained from the foreground features and the global features using a deformable attention module. ;

[0064] By using the Scatter operation, the value of the enhanced foreground feature is filled into the corresponding position in the global feature according to the index, thus obtaining the enhanced global feature:

[0065] ;

[0066] in, To enhance global features.

[0067] In one embodiment of the present invention, the calculation method in the spatial information recovery module includes:

[0068] The enhanced global features are upsampled using a second upsampling module to obtain second upsampled features:

[0069] ;

[0070] in, Indicates the second upsampling feature. This indicates the second upsampling module. Indicates enhanced global features;

[0071] The enhanced global features are copied to obtain the copied features:

[0072] ;

[0073] in, Characteristics indicating replication;

[0074] The enhanced global features are downsampled using a second downsampling module to obtain second downsampled features:

[0075] ;

[0076] in, This indicates the second downsampling feature. This indicates the second downsampling module;

[0077] After concatenating the third-scale feature with the second downsampled feature from the multi-scale features, the concatenated features are processed by the eighth convolution module to obtain the first encoder output features:

[0078] ;

[0079] in, Indicates the third-scale feature. Indicates splicing, This represents the eighth convolutional module with 2×C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the output feature of the first encoder;

[0080] The output features of the first encoder are upsampled using a third upsampling module, and the third upsampled features are concatenated with the second scale features in the multi-scale features and the copied features to obtain the first concatenated features:

[0081] ;

[0082] in, This represents the second-scale feature. Indicates splicing, Indicates the first splicing feature. This indicates the third upsampling module;

[0083] The first concatenated features are processed using the second convolutional fusion module to obtain the output features of the second encoder:

[0084] ;

[0085] in, This indicates the second convolutional fusion module. This indicates the output characteristics of the second encoder;

[0086] The output features of the second encoder are upsampled using a fourth upsampling module, and then concatenated with the first-scale feature and the second upsampled feature from the multi-scale features to obtain the second concatenated feature:

[0087] ;

[0088] in, This indicates the second splicing feature. This indicates the fourth upsampling module. Indicates the first-scale feature;

[0089] The second concatenated feature is processed using the third convolutional fusion module to obtain the output feature of the third encoder:

[0090] ;

[0091] in, This indicates the third convolutional fusion module. This indicates the output characteristics of the third encoder.

[0092] In one embodiment of the present invention, the first convolutional fusion module, the second convolutional fusion module, and the third convolutional fusion module have the same structure.

[0093] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0094] 1. The small target detection method of the present invention performs multi-scale semantic compression and global feature enhancement in a semantic compression enhancement encoder, and fuses the enhanced global features and multi-scale features to achieve spatial information recovery. Through multi-scale semantic compression, global feature enhancement and feature fusion of the semantic compression enhancement encoder, the loss caused by upsampling and downsampling in feature interaction can be reduced, and key features can be strengthened, thereby obtaining better feature fusion effect. This makes the small target detection method significantly reduce the number of parameters and computation, and improve the average accuracy of small target detection.

[0095] 2. The semantic compression enhancement encoder of the present invention includes a multi-scale semantic compression module, a foreground sampling enhancement module, and a spatial information recovery module. The multi-scale semantic compression module first unifies features at different scales to the same resolution, then uses a content-aware gating mechanism to eliminate semantic differences between features at different levels, while compressing multi-level feature mappings to reduce redundancy in semantic information. The foreground sampling enhancement module filters foreground features and uses a deformable attention module to focus on the foreground region with the highest information density, effectively reducing interference from background information. Compared to traditional self-attention modules, this module has significantly reduced computational overhead. The spatial information recovery module obtains the encoder output features by fusing the enhanced global features with the multi-scale features output by the backbone network. The use of a second and third convolutional fusion module significantly reduces computational and parameter requirements. Attached Figure Description

[0096] Figure 1 A flowchart illustrating a small target detection method based on semantic compression enhancement provided in an embodiment of the present invention;

[0097] Figure 2 A flowchart illustrating another small target detection method based on semantic compression enhancement provided in an embodiment of the present invention;

[0098] Figure 3 This is a schematic diagram of the overall structure of a small target detection method based on semantic compression enhancement provided in an embodiment of the present invention;

[0099] Figure 4 A schematic diagram of a multi-scale semantic compression module provided in an embodiment of the present invention;

[0100] Figure 5 A schematic diagram of the first convolutional fusion module provided in an embodiment of the present invention;

[0101] Figure 6 A schematic diagram of the foreground sampling enhancement module provided in an embodiment of the present invention;

[0102] Figure 7 This is a schematic diagram of a spatial information recovery module provided in an embodiment of the present invention. Detailed Implementation

[0103] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0104] Example 1

[0105] This embodiment uses RT-DETR (Real-Time Detection Transformer) as its foundation and provides a small target detection method based on semantic compression enhancement. This method achieves more efficient small target detection by designing a semantic compression enhancement encoder.

[0106] Please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating a small target detection method based on semantic compression enhancement provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another small target detection method based on semantic compression enhancement provided in an embodiment of the present invention.

[0107] The small target detection method based on semantic compression enhancement in this embodiment includes the following steps:

[0108] S100: Acquire the image data to be tested containing small targets and perform image scaling to obtain scaled image data.

[0109] Specifically, the scaled image data has a shape of The tensor, in which and These are the length and width of the scaled image data. For example, the length and width of the scaled image data are 640×640.

[0110] S200. Use a backbone network to extract multi-scale features from the scaled image data to obtain multi-scale features.

[0111] Specifically, the backbone network includes ResNet neural networks, such as ResNet50 or ResNet18. The multi-scale features extracted by the backbone network include first-scale features. Second-scale features and third-scale features First-scale features It is shaped as tensor, second-scale feature It is shaped as tensor, third-scale feature It is shaped as tensor, and These are the length and width of the scaled image data, with C set to 192.

[0112] S300: Use a semantic compression enhancement encoder to perform multi-scale semantic compression and global feature enhancement on multi-scale features, and fuse the enhanced global features with the multi-scale features to obtain the encoder output features.

[0113] Please see Figure 3 , Figure 3 This is a schematic diagram of the overall structure of a small target detection method based on semantic compression enhancement provided in an embodiment of the present invention. The semantic compression enhancement encoder includes a multi-scale semantic compression module, a foreground sampling enhancement module, and a spatial information recovery module; wherein, the multi-scale semantic compression module is used to compress and fuse multi-scale features to obtain global features; the foreground sampling enhancement module is used to sample and enhance foreground information of the global features to obtain enhanced global features; the spatial information recovery module is used to fuse the enhanced global features and multi-scale features to obtain the encoder output features.

[0114] Please see Figure 4 , Figure 4 This is a schematic diagram of a multi-scale semantic compression module provided in an embodiment of the present invention. To balance computational cost and the feature representation of small targets, the multi-scale semantic compression module first processes the features of the first scale... and third-scale features Projected onto second-scale features The same size. Considering that the low-level features are limited to the local receptive field and lack semantic concepts, the Convolutional Block Attention Module (CBAM) is applied to the downsampled first-scale features. Second-scale features The CBAM module's channel and spatial branches utilize pooling to aggregate global information and initially obtain conceptual information about small targets, thereby enhancing the semantic expressive power of low-level features. This enables low-level features to convey the initial conceptual information of small targets and reduces the semantic gap between different levels. The merged global feature mapping also faces semantic gaps between different levels; therefore, a gated unit is introduced to integrate high-level features... The semantic knowledge within is converted into a gate signal. Then send a signal to the door. Features of the CBAM module output and Multiplying them allows us to filter out complementary semantic information, resulting in... and ,in and It is related to the characteristics of high-level personnel Low-level features enhanced by complementary information. After semantic unification and de-obfuscation, the first convolutional fusion module merges the multi-layer feature maps into a single global feature. .

[0115] like Figure 4 As shown, the multi-scale semantic compression module includes a first downsampling module, a first convolutional block attention module, a first complementary information enhancement module, a second convolutional block attention module, a second complementary information enhancement module, a first upsampling module, a gating module, a concatenation module, and a first convolutional fusion module. Its computation method specifically includes:

[0116] S311. The first downsampling module is used to downsample the first-scale features to obtain the first downsampled features. The first convolutional block attention module is used to aggregate the first downsampled features to obtain the first target concept information.

[0117] ;

[0118] in, Indicates the first-scale feature. This indicates the first downsampling module. This represents the attention module of the first convolutional block. This represents the first target concept information. The first downsampling module uses an average pooling method with a kernel size of 2 and a stride of 2 for downsampling.

[0119] S312. The second convolutional block attention module is used to aggregate the features at the second scale to obtain the second target concept information:

[0120] ;

[0121] in, This represents the second-scale feature. This represents the attention module of the second convolutional block. This indicates information about the second objective concept.

[0122] S313. The third-scale features are upsampled using the first upsampling module to obtain high-level features:

[0123] ;

[0124] in, Indicates the third-scale feature. Indicates the first upsampling module. This represents high-level features. The first upsampling module uses the nearest neighbor interpolation method with a coefficient of 2 for upsampling.

[0125] S314. A gating module is used to convert the semantic knowledge within high-level features into gated signals:

[0126] ;

[0127] in, Indicates the gate control module. This indicates a gate open signal.

[0128] S315. The first complementary information enhancement module multiplies the gate signal with the first target concept information, and adds the multiplication result to the first target concept information to obtain the first low-level feature:

[0129] .

[0130] in, This represents the first low-level feature.

[0131] S316. The second complementary information enhancement module multiplies the gate signal with the second target concept information, and adds the multiplication result to the second target concept information to obtain the second low-level feature:

[0132] ;

[0133] in, This represents the second lowest level feature.

[0134] S317. Concatenate the first low-level features, the second low-level features, and the high-level features to obtain multi-layer features:

[0135] ;

[0136] in, Represents multi-layer features, This indicates the splicing module.

[0137] S318. The first convolutional fusion module is used to map multi-layer features into global features:

[0138] ;

[0139] in, Represents global features. This indicates the first convolutional fusion module.

[0140] In one specific embodiment, the calculation formula for the gating module is:

[0141] ;

[0142] in, This represents a convolutional module with C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the activation function.

[0143] Please see Figure 5 , Figure 5 This is a schematic diagram of a first convolutional fusion module provided in an embodiment of the present invention. The calculation method in the first convolutional fusion module includes:

[0144] The first convolutional module is used to process the multi-layer features to obtain the first convolutional features: ;

[0145] The second convolutional module is used to process the multi-layer features to obtain the second convolutional features: ;

[0146] The first convolutional feature is processed by sequentially connected first, second, and third sub-units to obtain the output feature of the repeating unit. The first, second, and third sub-units have the same structure, each including a third convolutional module, a fourth convolutional module, a fifth convolutional module, and an addition module. The output feature of the repeating unit is: ;

[0147] The global features are obtained by adding the features from the second convolution and the output features from the repeating units: ;

[0148] in, This represents the first convolutional module. Represents the first convolutional feature. This indicates the second convolutional module. Indicates the second convolution feature. and The input channels are all 3×C, the output channels are all C, and the convolution kernels are all 1×1. This indicates the third convolutional module. This indicates the fourth convolutional module. This represents the fifth convolutional module. and The input channels are all C, the output channels are all C, and the convolution kernels are all 1×1. The input channel is C, the output channel is C, the convolution kernel is 3×3, the groups parameter is C, and the padding is 1; This indicates the output feature of the first sub-unit. Indicates the output characteristics of repeating units; Indicates will Repeat three times.

[0149] In this embodiment, the complex ordinary convolution is split into a 1×1 convolution and a 3×3 depthwise convolution in the first convolution fusion module, which significantly reduces the computational complexity and the number of parameters.

[0150] Please see Figure 6 , Figure 6 This is a schematic diagram of the foreground sampling enhancement module provided in an embodiment of the present invention. To reduce information redundancy in global features and promote explicit foreground prediction by the encoder, this embodiment establishes a foreground sampling enhancement module based on sparse attention to extract semantically relevant foreground information. This module first uses a binary classification branch to predict global features. The information density map is used to simply distinguish the foreground region, which is denoted as the foreground feature. Then, a deformable attention module is used to enhance the features of the foreground region, and finally the enhanced foreground features are embedded into the original global features to obtain the enhanced global features.

[0151] like Figure 6 As shown, the calculation methods in the foreground sampling enhancement module include:

[0152] S321. The global features are processed using the sixth and seventh convolutional modules and activation functions, and the output score is obtained.

[0153] ;

[0154] in, To score points, As a global feature, This is the sixth convolutional module with C input channels, C output channels, and a 1×1 kernel. This is the seventh convolutional module with C input channels, num_classes (the number of classes in the current dataset) output channels, and a 1×1 kernel. The activation function is used. The sixth and seventh convolutional modules and the activation function constitute the score filtering module to filter global features. Screening foreground features .

[0155] S322. Using the Topk operation, select the index corresponding to the data of a preset proportion based on the score:

[0156] ;

[0157] in, Indicates an index. This indicates that the preset ratio is 0.25.

[0158] S323. Using the Gather operation, select the feature at the corresponding position in the global features based on the index as the foreground feature:

[0159] ;

[0160] in, Foreground features.

[0161] S324. Use a deformable attention module to obtain enhanced foreground features from foreground and global features. .

[0162] S325. Using the Scatter operation, fill in the enhanced foreground feature value according to the corresponding position of the index in the global feature to obtain the enhanced global feature:

[0163] ;

[0164] in, To enhance global features.

[0165] Please see Figure 7 , Figure 7 This is a schematic diagram of the spatial information recovery module provided in an embodiment of the present invention. To better utilize multi-scale information, multi-scale feature interaction is performed between enhanced global features and multi-scale features from the backbone network. The enhanced global features are used to guide multi-scale information interaction, reducing feature loss between information interactions and achieving better feature interaction.

[0166] like Figure 7 As shown, the calculation methods in the spatial information recovery module include:

[0167] S331. The enhanced global features are upsampled using the second upsampling module to obtain the second upsampled features:

[0168] ;

[0169] in, Indicates the second upsampling feature. This indicates the second upsampling module. This indicates an enhanced global feature.

[0170] S332. Copy the enhanced global features to obtain the copied features:

[0171] ;

[0172] in, Characteristics that indicate replication.

[0173] S333. The enhanced global features are downsampled using a second downsampling module to obtain the second downsampled features:

[0174] ;

[0175] in, This indicates the second downsampling feature. This indicates the second downsampling module.

[0176] S334. After concatenating the third-scale feature and the second downsampled feature from the multi-scale features, the concatenated features are processed by the eighth convolution module to obtain the output features of the first encoder:

[0177] ;

[0178] in, Indicates the third-scale feature. Indicates splicing, This represents the eighth convolutional module with 2×C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the output feature of the first encoder.

[0179] S335. The third upsampling module is used to upsample the output features of the first encoder, and the third upsampled features are concatenated with the second scale features and the copied features in the multi-scale features to obtain the first concatenated features:

[0180] ;

[0181] in, This represents the second-scale feature. Indicates splicing, Indicates the first splicing feature. This indicates the third upsampling module.

[0182] S336. The first concatenated features are processed using the second convolutional fusion module to obtain the output features of the second encoder:

[0183] ;

[0184] in, This indicates the second convolutional fusion module. This indicates the output characteristics of the second encoder.

[0185] S337. The fourth upsampling module is used to upsample the output features of the second encoder, and the fourth upsampled features are concatenated with the first scale features and the second upsampled features in the multi-scale features to obtain the second concatenated features:

[0186] ;

[0187] in, This indicates the second splicing feature. This indicates the fourth upsampling module. This represents the first-scale feature.

[0188] S338. The second concatenated features are processed using the third convolutional fusion module to obtain the output features of the third encoder:

[0189] ;

[0190] in, This indicates the third convolutional fusion module. This indicates the output characteristics of the third encoder.

[0191] In the spatial information recovery module, the second, third, and fourth upsampling modules use a nearest neighbor interpolation method with a coefficient of 2 for upsampling. The second downsampling module uses an average pooling method with a kernel size of 2 and a stride of 2 for downsampling. The structures of the second and third convolutional fusion modules are the same as those of the first convolutional fusion module.

[0192] S400: Input the encoder output features into the decoder to obtain the target classification result and regression box.

[0193] In one specific embodiment, the decoder structure is consistent with the decoder structure in RT-DETR, wherein the hidden_dim parameter is set to 192 and the num_layers parameter is set to 4.

[0194] The semantic compression enhancement encoder in this embodiment includes a multi-scale semantic compression module, a foreground sampling enhancement module, and a spatial information recovery module. The multi-scale semantic compression module first unifies features at different scales to the same resolution, then uses a content-aware gating mechanism to eliminate semantic differences between features at different levels, while compressing multi-level feature maps to reduce semantic redundancy. The foreground sampling enhancement module filters foreground features and uses a deformable attention module to focus on the foreground region with the highest information density, effectively reducing interference from background information. Compared to traditional self-attention modules, this module has significantly lower computational overhead. The spatial information recovery module obtains the encoder output features by fusing the enhanced global features with the multi-scale features output by the backbone network. The use of a second and third convolutional fusion module significantly reduces computation and parameter count, resulting in better operating efficiency when processing high-resolution features.

[0195] The small target detection method in this embodiment performs multi-scale semantic compression and global feature enhancement in a semantic compression enhancement encoder, and fuses the enhanced global features and multi-scale features to achieve spatial information recovery. Through multi-scale semantic compression, global feature enhancement and feature fusion in the semantic compression enhancement encoder, the loss caused by upsampling and downsampling in feature interaction can be reduced, and key features can be strengthened, thereby obtaining a better feature fusion effect. This makes the small target detection method significantly reduce the number of parameters and computational load, and improve the average accuracy of small target detection.

[0196] The effectiveness of this invention can be further illustrated by the following experimental data.

[0197] The method of the present invention is adopted as well as the existing Deformable DETR method (Zhu Detection[C] / / Proceedings of the AAAI Conference on ArtificialIntelligence. 2025, 39(8): 8673-8681.), RT-DETR method (Zhao Y, Lv W, Xu S, et al.DETRs Beat YOLOs on Real-time Object Detection[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2024: Comparative testing experiments were conducted from 16965 to 16974.

[0198] The comparative testing experiments were conducted on the VisDrone2019-DET dataset validation set (Du D, Zhu P, Wen L, et al. VisDrone-DET2019: The vision meets drone object detection in image challenge results[C] / / Proceedings of the IEEE / CVF international conference on computervision workshops. 2019: 0-0.). The VisDrone2019-DET dataset is a large-scale aerial image dataset, widely used in research on small object detection from the perspective of drones, and is an important validation object for evaluating small object detection algorithms in the international academic community.

[0199] Then, the average accuracy (mAP), parameter count, number of floating-point operations (FLOPs), and latency provided by COCO are used to quantitatively evaluate the performance of the proposed small target detection method based on semantic compression enhancement and existing methods.

[0200] This invention uses RT-DETR as the baseline method and designs a semantic compression enhancement encoder to replace the encoder of the RT-DETR method. To ensure the fairness of the experiment, the algorithm of this invention and its baseline method (RT-DETR method) were trained for 120 rounds on the VisDrone2019-DET dataset. The input image size was 640×640, and the backbone network used was ResNet18, with B2, B3, and B4 output features. Latency tests were conducted on a GPU (specifically an NVIDIA GeForce RTX 3090). The experimental results of the comparative tests are shown in Table 1.

[0201] Table 1. Comparison of Quantitative Parameters in the Comparative Test Results

[0202]

[0203] As shown in Table 1, the mean accuracy (mAP) of the proposed method is 0.5 higher than that of the RT-DETR method, and significantly higher than that of the Deformable DETR and FBRT-YOLO-X methods, demonstrating the effectiveness of the proposed method. In terms of parameter count and floating-point operations (FLOPs), the proposed method has fewer parameters than the other three methods. In terms of latency, the proposed method is lower than both the Deformable DETR and RT-DETR methods.

[0204] In summary, this invention proposes a semantic compression-enhanced encoder to achieve small target detection, which improves the accuracy by 0.5 compared with the RT-DETR method. Compared with other methods, the small target detection method based on semantic compression enhancement proposed in this invention has a higher average accuracy while having a smaller number of parameters, floating-point operations, and latency.

[0205] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A small target detection method based on semantic compression enhancement, characterized in that, Including the following steps: Acquire the image data of the test image containing small targets and perform image scaling to obtain scaled image data; The scaled image data is used to extract multi-scale features using a backbone network to obtain multi-scale features. The multi-scale features are subjected to multi-scale semantic compression and global feature enhancement using a semantic compression enhancement encoder. The enhanced global features are then fused with the multi-scale features to obtain the encoder output features. The encoder output features are input into the decoder to obtain the target classification result and regression box; The semantic compression enhancement encoder includes a multi-scale semantic compression module, a foreground sampling enhancement module, and a spatial information recovery module; the multi-scale semantic compression module is used to compress and fuse the multi-scale features to obtain global features. The foreground sampling enhancement module is used to sample and enhance the foreground information of the global features to obtain enhanced global features; the spatial information recovery module is used to fuse the enhanced global features and the multi-scale features to obtain the encoder output features. The computation method in the multi-scale semantic compression module includes: The first downsampling module is used to downsample the features at the first scale to obtain the first downsampled features; the first convolutional block attention module is used to aggregate the features of the first downsampled features to obtain the first target concept information. ； in, Indicates the first-scale feature. This indicates the first downsampling module. This represents the attention module of the first convolutional block. This represents the primary target concept information; The second convolutional block attention module is used to aggregate features at the second scale to obtain the second target concept information: ； in, This represents the second-scale feature. This represents the attention module of the second convolutional block. This indicates the concept information of the second objective; The first upsampling module is used to upsample the third-scale features to obtain high-level features: ； in, Indicates the third-scale feature. Indicates the first upsampling module. Indicates high-level characteristics; A gating module is used to convert the semantic knowledge within the high-level features into gate signals: ； in, Indicates the gate control module. Indicates the gate open signal; The first complementary information enhancement module multiplies the gate signal with the first target concept information, and adds the multiplication result back to the first target concept information to obtain the first low-level feature: ； in, This represents the first low-level feature; The second complementary information enhancement module multiplies the gate signal with the second target concept information, and adds the multiplication result to the second target concept information to obtain the second low-level feature: ； in, Indicates the second low-level feature; By concatenating the first low-level feature, the second low-level feature, and the high-level feature, a multi-layer feature is obtained: ； in, Represents multi-layer features, Indicates the splicing module; The first convolutional fusion module is used to map the multi-layer features to the global features: ； in, Represents global features. This indicates the first convolutional fusion module.

2. The small target detection method based on semantic compression enhancement according to claim 1, characterized in that, The scaled image data has a shape of The tensor of, where, and These are the length and width of the scaled image data.

3. The small target detection method based on semantic compression enhancement according to claim 1, characterized in that, The backbone network includes a ResNet neural network; The multi-scale features include a first-scale feature, a second-scale feature, and a third-scale feature, wherein the first-scale feature is a shape of The tensor of the second scale feature is of shape as The tensor of the third scale feature is of shape as tensor, and These are the length and width of the scaled image data.

4. The small target detection method based on semantic compression enhancement according to claim 1, characterized in that, The calculation formula for the gate control module is as follows: ； in, This represents a convolutional module with C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the activation function.

5. The small target detection method based on semantic compression enhancement according to claim 1, characterized in that, The calculation method in the first convolutional fusion module includes: The first convolutional module is used to process the multi-layer features to obtain the first convolutional features: ; The second convolutional module is used to process the multi-layer features to obtain the second convolutional features: ; The first convolutional feature is processed by sequentially connected first, second, and third sub-units to obtain the output feature of the repeating unit. The first, second, and third sub-units have the same structure, each including a third convolutional module, a fourth convolutional module, a fifth convolutional module, and an addition module. The output feature of the repeating unit is: ; The second convolutional feature and the output feature of the repeating unit are added together to obtain the global feature: ; in, This represents the first convolutional module. Represents the first convolutional feature. This indicates the second convolutional module. Indicates the second convolution feature. and The input channels are all 3×C, the output channels are all C, and the convolution kernels are all 1×1. This indicates the third convolutional module. This indicates the fourth convolutional module. This represents the fifth convolutional module. and The input channels are all C, the output channels are all C, and the convolution kernels are all 1×1. The input channel is C, the output channel is C, the convolution kernel is 3×3, the groups parameter is C, and the padding is 1; This indicates the output feature of the first sub-unit. Indicates the output characteristics of repeating units; Indicates will Repeat three times.

6. The small target detection method based on semantic compression enhancement according to claim 1, characterized in that, The calculation method in the foreground sampling enhancement module includes: The global features are processed using a sixth convolutional module, a seventh convolutional module, and an activation function to output a score: ； in, To score points, As a global feature, This is the sixth convolutional module with C input channels, C output channels, and a 1×1 kernel. This is the seventh convolutional module with C input channels, the number of categories in the current dataset as the output channel, and a 1×1 kernel. For activation functions; Using the Topk operation, the index corresponding to a preset proportion of the data is selected based on the score: ； in, Indicates an index. This indicates that the preset ratio is 0.25; Using the Gather operation, features at corresponding positions in the global features are selected as foreground features based on the index: ； in, Foreground features; Enhanced foreground features are obtained from the foreground features and the global features using a deformable attention module. ; By using the Scatter operation, the value of the enhanced foreground feature is filled into the corresponding position in the global feature according to the index, thus obtaining the enhanced global feature: ； in, To enhance global features.

7. The small target detection method based on semantic compression enhancement according to claim 6, characterized in that, The calculation method in the spatial information recovery module includes: The enhanced global features are upsampled using a second upsampling module to obtain second upsampled features: ； in, Indicates the second upsampling feature. This indicates the second upsampling module. Indicates enhanced global features; The enhanced global features are copied to obtain the copied features: ； in, Characteristics indicating replication; The enhanced global features are downsampled using a second downsampling module to obtain second downsampled features: ； in, This indicates the second downsampling feature. This indicates the second downsampling module; After concatenating the third-scale feature with the second downsampled feature from the multi-scale features, the concatenated features are processed by the eighth convolution module to obtain the first encoder output features: ； in, Indicates the third-scale feature. Indicates splicing, This represents the eighth convolutional module with 2×C input channels, C output channels, a stride of 1, and a kernel size of 1×1. This represents the output feature of the first encoder; The output features of the first encoder are upsampled using a third upsampling module, and the third upsampled features are concatenated with the second scale features in the multi-scale features and the copied features to obtain the first concatenated features: ； in, This represents the second-scale feature. Indicates splicing, Indicates the first splicing feature. This indicates the third upsampling module; The first concatenated features are processed using the second convolutional fusion module to obtain the output features of the second encoder: ； in, This indicates the second convolutional fusion module. This indicates the output characteristics of the second encoder; The output features of the second encoder are upsampled using a fourth upsampling module, and then concatenated with the first-scale feature and the second upsampled feature from the multi-scale features to obtain the second concatenated feature: ； in, This indicates the second splicing feature. This indicates the fourth upsampling module. Indicates the first-scale feature; The second concatenated feature is processed using the third convolutional fusion module to obtain the output feature of the third encoder: ； in, This indicates the third convolutional fusion module. This indicates the output characteristics of the third encoder.

8. The small target detection method based on semantic compression enhancement according to claim 7, characterized in that, The first convolutional fusion module, the second convolutional fusion module, and the third convolutional fusion module have the same structure.

Images