Small target end-to-end detection method, system and device and storage medium

Abstract

The invention relates to the field of image processing, and relates to a small target end-to-end detection method, which comprises the following steps: acquiring an image to be detected; extracting initial features of the image; inputting the initial features into a multi-scale feature extraction module, processing the initial features by the multi-scale feature extraction module according to a preset rule, and outputting multi-scale features; fusing and reconstructing the multi-scale features according to a preset context guidance mechanism to obtain enhanced fusion features; and determining a target recognition result of the image according to the enhanced fusion feature. The invention further provides a small target end-to-end detection system and device and a storage medium. The method can improve the recognition precision of small targets in the image and the adaptability to complex scenes.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a small target end-to-end detection method, system, device and storage medium. Background Art

[0002] With the continuous development of computer vision technology, target detection has become a core component of modern intelligent systems and is widely used in many fields such as drone inspection, intelligent driving and security monitoring. In these applications, the accuracy and real-time performance of target detection are crucial to the overall performance of the system, especially in complex environments and small target recognition.

[0003] Taking drone inspection as an example, when drones are used to perform low-altitude aerial photography tasks, small target detection faces many technical challenges due to the small size of the targets in the images taken by drones, complex and changeable backgrounds, lighting changes, and occlusion. Traditional target detection methods (such as feature extraction-based algorithms and conventional convolutional neural networks) often fail to meet actual needs in small target detection tasks, resulting in a large number of small targets being ignored or misdetected. Therefore, it is particularly important to improve the accuracy of small target detection algorithms, especially their effectiveness in complex environments. In order to solve this problem, researchers have gradually turned to more advanced deep learning technologies, using single-stage detection networks (such as YOLO, which stands for You Only Look Once, a target detection algorithm based on deep learning) and Transformer networks to improve detection performance, showing the great potential of deep learning in the field of small target detection.

[0004] However, although the existing technology has achieved significant improvements in detection accuracy, there are still many challenges in specific application scenarios such as low-altitude aerial photography. Especially in the detection of small targets, it faces higher difficulties. Under low-altitude aerial photography conditions, small targets usually occupy less pixel area in the image and are easily obscured by complex backgrounds. In addition, due to changes in shooting angles and flight speeds, the targets may be deformed or blurred. Especially under complex weather conditions, such as haze weather, the contrast and clarity of the image will drop significantly, which not only weakens the characteristic signal of the target, but also makes it more difficult to distinguish between the background and the target. In addition, lighting changes, dynamic occlusion, and scene interference (such as buildings, trees, vehicles, etc.) will further increase the difficulty of small target detection. Therefore, traditional detection models often find it difficult to maintain sufficient robustness, which can easily lead to problems of missed detection and false detection, seriously affecting the accuracy and real-time performance of detection. Summary of the invention

[0005] In order to enhance the model's ability to detect small targets in complex background images, improve the recognition accuracy of small targets in images and improve the adaptability to complex scenes, a small target end-to-end detection method, system, device and storage medium are provided.

[0006] In order to achieve the above-mentioned objective of the present invention, the present invention provides a small target end-to-end detection method.

[0007] The present invention provides a small target end-to-end detection method, comprising:

[0008] Acquire an image to be detected;

[0009] Extract the initial features of the image;

[0010] The initial features are input into a multi-scale feature extraction module, and the multi-scale feature extraction module processes the initial features according to preset rules and outputs multi-scale features;

[0011] According to the preset context-guided mechanism, multi-scale features are fused and reconstructed to obtain enhanced fusion features;

[0012] The target recognition result of the image is determined based on the enhanced fusion features.

[0013] Optionally, the multi-scale feature extraction module includes multiple multi-scale convolution units connected in sequence, and the feature of the input multi-scale convolution unit is recorded as the first input feature. The multi-scale convolution unit captures different levels of detail information of the first input feature through different convolution kernels to obtain a multi-level detail feature map, and the multi-level detail feature maps output by multiple multi-scale convolution units are integrated to obtain multi-scale features.

[0014] Optionally, the step of processing the first input feature by the multi-scale convolution unit includes:

[0015] Segmenting the first input feature into a first segmentation feature and a second segmentation feature;

[0016] Input the first segmentation feature into the multi-scale convolution unit to obtain detail feature maps at different levels;

[0017] Detail features at different levels are concatenated to obtain a fused detail feature map, and the fused detail feature map is connected with the second segmentation feature to obtain a multi-level detail feature map.

[0018] Optionally, fusing and reconstructing the multi-scale features according to a preset context guidance mechanism to obtain enhanced fused features includes:

[0019] Process the channel dimension of the multi-level detail feature map to obtain the channel attention feature;

[0020] Use the spatial attention mechanism to process multi-level detail feature maps to obtain spatial attention features;

[0021] The first fused detail feature is obtained by fusing the channel attention feature and the spatial attention feature;

[0022] Capturing the detail differences between the first fused detail features of adjacent layers, and fusing the difference characteristics of the first fused features of adjacent layers to obtain the second fused detail features;

[0023] Integrate multiple second fusion detail features to obtain enhanced fusion features.

[0024] Optionally, the method further comprises:

[0025] Divide the multi-scale features into several windows;

[0026] The feature vector of the window is processed by cascading application of grouped attention mechanism to obtain refined multi-scale features.

[0027] Optionally, the cascaded application grouped attention mechanism is: using multiple groups of attention heads to process feature vectors, the multiple groups of attention heads are used to focus on different channel subsets in the feature map, multiple attention heads in the same group are connected in sequence, and the output features of the attention heads are used as input features of the next attention head in the same group.

[0028] Optionally, the feature of the input attention head is recorded as the second input feature, and the processing steps of the attention head on the second input feature are:

[0029] Inputting the second input feature into the query generator to obtain a query vector, calculating the similarity between the second input feature and the query vector, and obtaining a similarity score;

[0030] Assign attention weights to the second input feature input based on the similarity score;

[0031] Performing weighted summation on the second input feature based on the attention weight to obtain an output feature of the attention head;

[0032] Integrate the output features of multiple groups of attention heads to obtain refined multi-scale features.

[0033] In order to solve the above problems, the present invention also provides a small target end-to-end detection system, the system comprising:

[0034] An acquisition module, used for acquiring an image to be detected;

[0035] Feature extraction module, used to extract initial features of the image;

[0036] A first processing module, used for inputting the initial features into a multi-scale feature extraction module, and the multi-scale feature extraction module processes the initial features according to preset rules and outputs multi-scale features;

[0037] The second processing module is used to fuse and reconstruct the multi-scale features according to a preset context guidance mechanism to obtain enhanced fusion features;

[0038] The output module is used to determine the target recognition result of the image based on the enhanced fusion features.

[0039] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0040] at least one processor; and,

[0041] a memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the small target end-to-end detection method described above.

[0043] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned small target end-to-end detection method.

[0044] In summary, this application includes the following beneficial technical effects:

[0045] The present application sets up a multi-scale feature extraction module, and processes the initial features according to preset rules to realize multi-scale feature fusion, so as to more effectively extract key features in the image and improve the accuracy of small target detection; in addition, through the context-guided spatial channel feature pyramid fusion reconstruction, the hierarchy and richness of feature fusion are enhanced, which can further enhance the detection effect of small targets in complex background images, improve the recognition accuracy of small targets in images and adaptability to complex scenes, and help promote the widespread application of intelligent systems in dynamic and complex environments;

[0046] The first input feature is split into a first segmentation feature and a second segmentation feature; the first segmentation feature is input into a multi-scale convolution unit to obtain detail feature maps at different levels, and the detail feature map is directly concatenated with the second segmentation feature. By introducing partial convolution technology, this module effectively reduces the amount of calculation while retaining more original feature information, thereby improving the efficiency of feature expression. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a process flow of a small target end-to-end detection method provided by an embodiment of the present invention;

[0048] Figure 2A schematic diagram of the structure of a small target end-to-end detection model corresponding to a small target end-to-end detection method provided in one embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the structure of a backbone network feature extraction PCMSF module provided in one embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the structure of a window cascade grouping attention module provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of the structure of a neck network feature fusion CG-SCFPR module provided in one embodiment of the present invention;

[0052] Figure 6 A schematic diagram of the structure of an RSCM module provided in one embodiment of the present invention and a schematic diagram of the process of processing features through the RSCA mechanism;

[0053] Figure 7 A first comparison diagram of target recognition results of a small target end-to-end detection method provided by an embodiment of the present invention and a traditional small target detection method in a complex scene;

[0054] Figure 8 A second comparison diagram of target recognition results of a small target end-to-end detection method provided by an embodiment of the present invention and a traditional small target detection method in a complex scene;

[0055] Fig. 9 A schematic diagram of the structure of an electronic device for implementing the small target end-to-end detection method provided by an embodiment of the present invention.

[0056] Figure numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface; 21, multi-scale feature extraction module; 211, multi-scale convolution unit; 22, window cascade grouping attention module; 23, feature fusion reconstruction module; 24, generation module.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0059] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0060] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0061] Reference Figure 1 and Figure 2 In this embodiment, the small target end-to-end detection method includes:

[0062] S1. Obtain an image to be detected.

[0063] Specifically, the image to be detected can be collected by a drone or a security monitoring system.

[0064] The camera installed in the drone or security monitoring system can be controlled to take a number of pictures at a preset time period; the monitoring video taken by the camera can also be directly obtained, and a number of images can be obtained by extracting frames from the monitoring video;

[0065] The image to be detected can be imported through an external device (such as a USB flash drive, a removable hard disk, etc.), or read through a memory or a database, and this application does not impose any restrictions.

[0066] S2. Extract the initial features of the image.

[0067] The initial features of the image can be extracted by a convolutional block (Conv for short). In this embodiment, the convolutional block used to extract the initial features is referred to as Figure 2 The module corresponding to Conv P1 in the figure; ResNet (Residual Network), VGG (Visual Geometry Group) and other networks can be used as the backbone network (Backbone) to extract the initial features of the image, or other network structures can be used to extract the initial features, which is not limited in this application.

[0068] S3. Input the initial features into the multi-scale feature extraction module 21. The multi-scale feature extraction module 21 processes the initial features according to preset rules and outputs multi-scale features.

[0069] Reference Figure 2 and Figure 3 The multi-scale feature extraction module 21 includes a plurality of multi-scale convolution units 211 connected in sequence (refer to Figure 3 The corresponding C1 module, C2 module and C3 module), the feature of the input multi-scale convolution unit 211 is recorded as the first input feature, the multi-scale convolution unit 211 captures the detail information of different levels of the first input feature through different convolution kernels, and obtains a multi-level detail feature map, and integrates the multi-level detail feature maps output by multiple multi-scale convolution units 211 to obtain a multi-scale feature.

[0070] The convolution kernels of the multiple multi-scale convolution units 211 have different sizes. Figure 3 In the figure, modules C1 and C2 are 5×5 convolutions, and module C3 is 7×7 convolution; convolution kernels of different sizes are used to capture multi-scale information. Each convolution layer further performs group convolution operations on features, making more efficient use of computing resources. And using group convolution not only reduces computing overhead, but also improves expression ability when extracting detailed features at different levels, which is suitable for the needs of subtle features in small target detection.

[0071] The multi-scale convolution unit 211 processes the first input feature in the following steps:

[0072] S31, segmenting the first input feature into a first segmentation feature and a second segmentation feature.

[0073] In this embodiment, the first input feature is processed by the CSP network, and the first input feature is divided into a first segmentation feature and a second segmentation feature. The full name of CSP is Cross Stage Partial. The CSP network can split the input feature into two parts according to the channel, one part is used for further multi-scale convolution operation, and the other part is directly retained for subsequent feature fusion. This partial convolution operation effectively reduces the amount of calculation while ensuring feature transfer.

[0074] S32. Input the first segmentation feature into a multi-scale convolution unit to obtain detail feature maps at different levels.

[0075] S33, concatenating detail features at different levels to obtain a fused detail feature map, and connecting the fused detail feature map and the second segmentation feature to obtain a multi-level detail feature map.

[0076] Specifically, the detail feature maps of different levels output by the multi-scale convolution units 211 at each level are concatenated together to form a multi-scale feature fusion. This fusion method ensures the model's sensitivity to targets of different scales in small target detection, especially in complex scenes, it can better distinguish targets and backgrounds; the fused features are further processed by a 1×1 convolution block, so that the feature dimension of the fused detail feature map is compressed and information is integrated at the same time; finally, the residual connection method is used to add the fused detail feature map to the feature map of the second segmentation feature. This residual structure not only retains the original feature information, but also enhances the gradient flow, which helps Figure 2 The stability and convergence speed of the corresponding model during training.

[0077] By introducing partial convolution technology, our multi-scale feature extraction module 21 effectively reduces the amount of calculation while retaining more original feature information, thereby improving the efficiency of feature expression. Combined with the CSP structure, the model is more accurate in extracting small target features and enhances the ability to perceive targets in complex scenes.

[0078] In a preferred embodiment of the present embodiment, a multi-level feature extraction network is used to process an image to obtain multi-level multi-scale features, thereby obtaining low-level edge structure information and high-level spatial detail features of the image; the multi-level feature extraction network includes: a plurality of multi-scale feature extraction modules 21 connected in sequence, the output features of the current multi-scale feature extraction module 21 are used as the input features of the next multi-scale feature extraction module 21, and in the present embodiment, the number of layers of the multi-level feature extraction network is 4, and 4 levels of multi-scale features (i.e. Figure 2 P2, P3, P4 and P5 as shown in FIG.

[0079] In a preferred implementation of this embodiment, the small target end-to-end detection method further includes:

[0080] S301, dividing the multi-scale features into a number of windows;

[0081] S302, processing the feature vector of the window according to the cascade application group attention mechanism to obtain refined multi-scale features.

[0082] Reference Figure 2 and Figure 4, the multi-scale features are input into the AIFI-WCGAF module. The full name of AIFI-WCGAF is Attention-based Intra-scale Feature-Window Cascaded Group Attention Fusion, which means the window cascaded group attention module based on the intra-scale feature of attention. The AIFI-WCGAF module uses the cascaded local attention mechanism to enable the network to show stronger capabilities in multi-scale feature extraction21. The adaptive adjustment of attention within the local window can capture detailed features more accurately, significantly improve the accuracy of target positioning and recognition, and enhance the detection effect, especially in dense target and complex background environments.

[0083] Figure 4 (a) is the schematic diagram of the improved AIFI module. Figure 4 (b) is a schematic diagram of the WCGAF module. Figure 4 (c) is the processing flow chart of the cascade group attention mechanism;

[0084] Figure 4 The Add&Norm module in (a) is for addition and normalization, which is used to add and normalize the features to enhance the stability of the model; Figure 4 The FFN module (Feed Forward Network) in (a) is a feed-forward neural network. FFN is a fully connected layer used to further process features. Figure 4 The Window Cascaded Group Attention Fusion (WCGAF module) in (a) is used for feature fusion, which improves the feature extraction capability by grouping the attention mechanism within a local window.

[0085] Figure 4 The Cascaded Group Attention in (b) is a cascaded group attention module, which divides the attention mechanism into multiple groups and refines the features layer by layer to better capture the target details. Figure 4 The Padding&Reshaping&Window Partition in (b) is a padding, reshaping and window partitioning module, which is used to format the image features to adapt to the window-level grouping processing. Figure 4 The Reconstruction&Crop in (b) is a reconstruction and cropping module, which is used to recombine the features after window grouping into a complete feature map.

[0086] Figure 4 The Concat&Projection in (c) is a connection and projection module used to connect multiple groups of features and map them to the specified feature space.

[0087] The cascaded application grouped attention mechanism is: use multiple groups of attention heads to process feature vectors, multiple groups of attention heads are used to focus on different channel subsets in the feature map, multiple attention heads in the same group are connected in sequence, and the output features of the attention heads are used as the input features of the next attention head in the same group.

[0088] The feature of the input attention head is recorded as the second input feature. The processing steps of the attention head for the second input feature are:

[0089] S310: Input the second input feature into the query generator to obtain a query vector, calculate the similarity between the second input feature and the query vector, and obtain a similarity score.

[0090] S320. Assign an attention weight to the second input feature input according to the similarity score.

[0091] S330, performing weighted summation on the second input feature based on the attention weight to obtain the output feature of the attention head.

[0092] S340, integrating the output features of multiple groups of attention heads to obtain refined multi-scale features.

[0093] Compared with the traditional multi-head self-attention (MHSA), the cascaded application grouped attention mechanism divides the attention heads into multiple groups before generating queries (Q), keys (K), and values ​​(V). Each group focuses on a specific subset of channels in the feature map. This grouping method not only improves the diversity of feature capture of the model, but also enables it to extract channel-specific information more effectively. In addition, the cascaded grouped attention adopts a layer-by-layer refinement structure. The output of each attention head will be used as the input of the next attention head in the same group, thereby gradually refining the feature expression; this cascaded progressive processing mode helps the model gradually learn richer and more detailed feature representations in the hierarchy, significantly improving the network's discriminative ability. Finally, the outputs of the attention heads in each group are spliced ​​and integrated into the final feature output through a linear layer to form a complete module output.

[0094] The AIFI-WCGAF module improved by using the WCGAF module can enable the model to focus more on local details, especially in capturing the boundaries and shapes of small targets. By refining the hierarchical processing of features, the model can show higher accuracy and robustness in small target detection tasks. This design shows significant advantages in complex backgrounds and dense small target scenes.

[0095] In this embodiment, the multi-scale feature P5 is processed by the AIFI-WCGAF module. The AIFI-WCGAF module first pulls the feature map of the two-dimensional multi-scale feature P5 into a feature vector, then processes the above feature vector according to the cascade application group attention mechanism, and then restores the processed feature vector to two dimensions again to obtain a refined multi-scale feature F5.

[0096] The reason why only the AIFI-WCGAF module is used to process the final multi-scale feature P5 is due to the following considerations: Traditional DETR models (Detection Transformer, an end-to-end target detection model), such as Deformable DETR, usually flatten feature maps from multiple scales into a very long vector to facilitate the interaction between features at different scales; but this will lead to huge computational complexity and computational time, which is also the main reason why the traditional DETR model is slow; and the multi-scale feature P5 is a high-level feature. Compared with the shallower multi-scale features P3 and P4, the multi-scale feature P5 has deeper, more advanced and richer semantic information. This semantic information can help the Transformer model distinguish the features of different objects; therefore, only using the AIFI-WCGAF module to process the final multi-scale feature P5 can significantly reduce the amount of computation and improve the computational speed while obtaining useful semantic information, without causing significant damage to the performance of the model, and can achieve better results.

[0097] S4. According to the preset context-guided mechanism, the multi-scale features are fused and reconstructed to obtain enhanced fused features.

[0098] Reference Figure 2 ,In order to better fuse and reconstruct the feature layers to cope with the scenes with complex background and many small targets in aerial photography, the CG-SCFPR module is used as the feature fusion module of the neck network of the feature fusion reconstruction module 23 to enhance context perception. To process multi-level multi-scale features, the full name of CG-SCFPR is Context-Guided SpatialChannel Feature Pyramid Reconstruction, and the Chinese interpretation is context-guided spatial channel feature pyramid reconstruction module; CG-SCFPR is a context-guided spatial channel feature pyramid reconstruction module, which is mainly used for hierarchical fusion and reconstruction of features, enhancing the context information and expression ability of features, and is particularly suitable for processing small target detection tasks in complex backgrounds.

[0099] Figure 5This is a structural diagram of the neck network feature fusion CG-SCFPR module. SOPFF (Small Object Pyramid Feature Fusion) is used to unify the features of multi-scale features (P2, P3, P4 and F5) from different levels to the same scale through adaptive pooling, and then perform splicing and multi-level feature fusion to enhance the model's perception of small targets. By effectively integrating features at different levels, the model can not only capture global and local contextual information, but also improve the expressiveness of features, enhance the distinction between targets and backgrounds, and ensure accurate detection in complex scenarios.

[0100] exist Figure 5 In the , SOPFF (Small Object Pyramid Feature Fusion) module is a small object pyramid feature fusion module. SOPFF unifies feature maps from different layers to the same spatial scale through adaptive pooling operations, and then splices them, and then further processes them through a feature fusion module (three-time RSCM). RSCM is used to fuse features of different scales to enhance the model's perception of small objects.

[0101] RepC3 (Reparameterization Convolution) is a reparameterization convolution. RepC3 is used to process the fused multi-scale feature map to provide richer feature information for the final detection head (Detect Head); DetectHead is the detection head. Detect Head receives the feature map from the feature extraction network and generates the final prediction output. The prediction output includes information such as the location and category of the target.

[0102] interpolate is interpolation. The interpolate module is used to scale the feature map in the spatial dimension to match the size of feature maps at different levels. FBM (Fuse Block Multi) is a multi-feature fusion block. FBM receives input from different feature layers and fuses these multi-scale features together, so that the model can better capture multi-scale information, especially subtle feature details, highlight key areas and detail features, and help improve detection accuracy. The DIF (Dynamic Interpolation Fusion) module is used to capture the detail differences between features at different levels and further enrich the fusion features.

[0103] Reference Figure 5First, the feature maps from P2, P3, P4 and P5 are input into the SOPFF module, and the features of each level are unified to the same spatial scale through the Adaptive Average Pooling operation. The pooled feature maps are spliced ​​in the channel dimension, and then further processed by RSCM (Rectangular Spatial-Channel Module) to generate a fused multi-scale feature map, which provides rich small target feature information for subsequent steps.

[0104] After that, the multi-scale features are processed by the SOPFF module and then input into the RSCM module for further enhancement. The RSCM module uses RSCA (Rectangular Spatial-Channel Attention) to enhance attention in the rectangular space and channel dimensions, highlighting the spatial and channel information of the target area, thereby strengthening the contextual information in the feature map. The structural diagram of the RSCM module and the flow diagram of feature processing through the RSCA mechanism are shown in Figure 1. Figure 6 shown.

[0105] refer to Figure 6 Specifically, the RSCM module uses RSCA to fuse and reconstruct multi-scale features to obtain enhanced fusion features, including:

[0106] S41, processing the channel dimension of the multi-level detail feature map to obtain a channel attention feature;

[0107] S42, using the spatial attention mechanism to process the multi-level detail feature map to obtain the spatial attention feature;

[0108] S43, fusing the channel attention feature and the spatial attention feature to obtain a first fused detail feature;

[0109] S44, capturing detail differences between first fused detail features of adjacent layers, and performing difference characteristic fusion on the first fused features of adjacent layers to obtain second fused detail features;

[0110] The feature maps of adjacent layers are fused by DIF. The DIF module first combines interpolation and convolution processing through addition operation (Add) to capture the detailed differences between features at different levels and further enrich the fused features. Then, the FBM module performs element-by-element multiplication (Mul) operation on the fused features and applies h-sigmoid activation to enhance the correlation between features, thereby improving the model's ability to detect small targets.

[0111] S45. Integrate multiple second fusion detail features to obtain enhanced fusion features.

[0112] Reference Figure 5 ,The feature maps processed by the DIF and FBM modules are sequentially concatenated and input into the RepC3 module to further integrate multi-level information. The fused feature maps are compressed by the RepC3 module and then passed to the Detect Head to achieve the final target detection.

[0113] This design enables the CG-SCFPR module to efficiently integrate multi-scale features, and through the joint processing of SOPFF, RSCM, DIF and FBM modules, it significantly enhances the small target detection performance of the model in complex scenes.

[0114] S5. Determine the target recognition result of the image according to the enhanced fusion features.

[0115] The target recognition result includes the target category and location information. Figure 2 , the generation module 24 is used to process the fused features to obtain the target recognition result; specifically, the small target end-to-end detection model corresponding to the small target end-to-end detection method uses the IoU-aware Query Selection module to screen and select the most relevant candidate boxes, and then uses the Decoder&Head module to restore the feature map in the candidate box to a higher resolution feature to obtain the target recognition result of the small target.

[0116] Specifically, IoU stands for Intersection over Union, which is an indicator to measure the degree of overlap between the predicted box and the true box. Specifically, IoU is the ratio of the intersection area to the union area of ​​the predicted box and the true box. In the IoU-aware Query Selection module, IoU information is used to screen and select the most relevant candidate boxes to improve the detection precision and accuracy of the model.

[0117] The Decoder&Head module is the output part of the model. The Decoder is the decoder and the Head is the detection head. The decoder can restore the feature map to higher-resolution features, and then generate the final target recognition result through the detection head.

[0118] In the preferred implementation of this embodiment, the small target end-to-end detection method has made improvements to the alignment IoU, and this application introduces an improved loss function PIoU (Powerful IoU). The traditional IoU mainly focuses on the overall overlapping area of ​​the predicted box and the real box, and has limited capture of details in small targets and complex backgrounds, which can easily lead to background mis-overlap or target omission. PIoU, on the other hand, effectively improves the model's sensitivity to small target boundary details by finely calculating the boundary difference and proportional error between the predicted box and the target box, allowing it to detect small targets more accurately in complex scenes.

[0119] PIoU performs well in small object detection, mainly because it introduces the difference in width and height between the predicted box and the target box and the normalized proportional error. It is shown as follows:

[0120]

[0121] Where |Δw| represents the boundary difference between the prediction box and the target box in width, |Δh| represents the boundary difference between the prediction box and the target box in height, and w gt Indicates the width of the target box, w ht Indicates the height of the target box; specifically, the two values ​​|Δw| and |Δh| measure the degree of alignment between the predicted box and the target box on the left and right boundaries and the upper and lower boundaries; through the refined calculation of this boundary difference, PIoU can more effectively capture the edge details of the box and avoid the false overlap of the background and the target box, thereby providing more accurate gradient information for small target detection. This multi-dimensional evaluation method based on boundary differences makes the model more stable and robust in complex backgrounds, and improves the accuracy and adaptability of small target detection.

[0122] By combining the advantages of PIoU, the overall performance of the CSP-PCMSF, WCGAF and CG-SCFPR modules is further improved. The present invention significantly enhances the model's ability to detect small targets and improves its adaptability in changing environments. The present invention not only provides new ideas for the optimization of target detection technology and promotes the application of drones in complex scenarios, but also injects new impetus into the development of future intelligent monitoring systems.

[0123] Figure 7 and Figure 8 These are comparison charts of target recognition results in complex scenarios between the small target end-to-end detection method provided by this application and the traditional small target detection method RT-DETR; the full name of RT-DETR is Real-Time Detection Transformer. It is an innovative model architecture that combines Transformer and real-time target detection, aiming to solve the trade-off between speed and accuracy of existing target detection models. By introducing efficient Transformer modules and optimized detection heads, RT-DETR improves the real-time performance and accuracy of the model and can be directly used for end-to-end target detection.

[0124] Figure 7 The heat map and visualization of the small target end-to-end detection model and RT-DETR network in the case of misdetection corresponding to the small target end-to-end method of this application, where: Figure 7 The two images on the left are the target recognition results and heat maps of the small target end-to-end detection model of this application in complex scenes. Figure 7The two images on the right are the target recognition results and heat maps of the traditional RT-DETR network in complex scenes. In comparison, the original RT-DETR has more misjudgments in some complex background areas, such as mistakenly marking some background objects that have no detection significance as target objects. The small target end-to-end detection model of the present application can more accurately distinguish between the target and the background through an improved detection mechanism, significantly reducing false detections in similar situations and improving the accuracy of detection. The small target end-to-end detection model of the present application shows stronger anti-interference capabilities in complex scenes.

[0125] Figure 8 The heat map and visualization of the small target end-to-end detection model and RT-DETR network in the case of missed detection corresponding to the small target end-to-end method of this application, where: Figure 8 The two images on the left are the target recognition results and heat maps of the small target end-to-end detection model of this application in complex scenes. Figure 8 The two images on the right are the target recognition results and heat maps of the traditional RT-DETR network in complex scenes. The traditional RT-DETR has certain missed detection phenomena on some smaller or difficult-to-identify targets. The small target end-to-end detection model of this application effectively reduces the missed detection of small targets by enhancing feature extraction and multi-scale target detection capabilities, further improving the comprehensiveness and reliability of detection.

[0126] In summary, the improvements of SOD-DETR in the problems of false detection and missed detection have significantly enhanced the adaptability and robustness of the model in complex scenarios, providing a better solution for high-precision target detection.

[0127] Based on the same inventive concept, an embodiment of the present invention provides a small target end-to-end detection system.

[0128] The small target end-to-end detection system described in the present invention can be loaded in an electronic device. According to the functions to be implemented, the small target end-to-end detection system includes an acquisition module, a feature extraction module, a first processing module, a second processing module and an output module, wherein: the acquisition module can acquire the image to be detected; the feature extraction module can extract the initial features of the image; the first processing module can input the initial features into the multi-scale feature extraction module 21, and the multi-scale feature extraction module 21 processes the initial features according to preset rules and outputs multi-scale features; the second processing module can fuse and reconstruct the multi-scale features according to the preset context guidance mechanism to obtain enhanced fusion features; the output module can determine the target recognition result of the image according to the enhanced fusion features. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0129] The various variations and specific examples of the small target end-to-end detection method provided in the above embodiments are also applicable to the small target end-to-end detection system of the present embodiment. Through the above detailed description of the small target end-to-end detection method, those skilled in the art can clearly know the implementation method of the small target end-to-end detection system of the present embodiment. For the sake of brevity of the specification, it will not be described in detail here.

[0130] The present application also discloses an electronic device, such as Fig. 9 , which is a schematic diagram of the structure of an electronic device for a method for end-to-end detection of small targets provided by an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 connected to the at least one processor for communication, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a method program for end-to-end detection of small targets.

[0131] Among them, the processor 10 can be composed of an integrated circuit in some embodiments, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as a method for performing end-to-end detection of small targets, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0132] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the method program for end-to-end detection of small targets, but also can be used to temporarily store data that has been output or is to be output.

[0133] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0134] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0135] Fig. 9 Only an electronic device with components is shown, and those skilled in the art will understand that Fig. 9The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently. For example, although not shown, the electronic device may also include a power supply (such as a battery) for supplying power to each component. Preferably, the power supply may be logically connected to at least one processor 10 through a power management device, so that functions such as charging management, discharging management, and power consumption management are implemented through the power management device. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0136] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0137] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0138] The present application embodiment provides a computer-readable storage medium, for example, including: any entity or device, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM) that can carry the computer program code. The computer-readable storage medium stores a computer program that can be loaded by a processor and execute the small target end-to-end detection method of the above embodiment.

[0139] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0140] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A small target end-to-end detection method, characterized in that: Acquire an image to be detected; Extract the initial features of the image; The initial features are input into a multi-scale feature extraction module, and the multi-scale feature extraction module processes the initial features according to preset rules and outputs multi-scale features; According to the preset context-guided mechanism, multi-scale features are fused and reconstructed to obtain enhanced fusion features; The target recognition result of the image is determined based on the enhanced fusion features.

2. The small target end-to-end detection method according to claim 1, characterized in that: The multi-scale feature extraction module includes multiple multi-scale convolution units connected in sequence. The feature of the input multi-scale convolution unit is recorded as the first input feature. The multi-scale convolution unit captures the detail information of different levels of the first input feature through different convolution kernels to obtain a multi-level detail feature map. The multi-level detail feature maps output by multiple multi-scale convolution units are integrated to obtain multi-scale features.

3. The small target end-to-end detection method according to claim 2, characterized in that: The multi-scale convolution unit processes the first input feature in the following steps: Segmenting the first input feature into a first segmentation feature and a second segmentation feature; Input the first segmentation feature into the multi-scale convolution unit to obtain detail feature maps at different levels; Detail features at different levels are concatenated to obtain a fused detail feature map, and the fused detail feature map is connected with the second segmentation feature to obtain a multi-level detail feature map.

4. The small target end-to-end detection method according to claim 2 or 3, characterized in that: The multi-scale features are fused and reconstructed according to a preset context guidance mechanism to obtain enhanced fusion features, including: Process the channel dimension of the multi-level detail feature map to obtain the channel attention feature; Use the spatial attention mechanism to process multi-level detail feature maps to obtain spatial attention features; The first fused detail feature is obtained by fusing the channel attention feature and the spatial attention feature; Capturing the detail differences between the first fused detail features of adjacent layers, and fusing the difference characteristics of the first fused features of adjacent layers to obtain the second fused detail features; Integrate multiple second fusion detail features to obtain enhanced fusion features.

5. The small target end-to-end detection method according to claim 1, 2 or 3, characterized in that: The method further comprises: Divide the multi-scale features into several windows; The feature vector of the window is processed by cascading application of grouped attention mechanism to obtain refined multi-scale features.

6. The small target end-to-end detection method according to claim 5, characterized in that: The cascaded application grouped attention mechanism is: use multiple groups of attention heads to process feature vectors, multiple groups of attention heads are used to focus on different channel subsets in the feature map, multiple attention heads in the same group are connected in sequence, and the output features of the attention heads are used as the input features of the next attention head in the same group.

7. The small target end-to-end detection method according to claim 6, characterized in that: The feature of the input attention head is recorded as the second input feature. The processing steps of the attention head for the second input feature are: Inputting the second input feature into the query generator to obtain a query vector, calculating the similarity between the second input feature and the query vector, and obtaining a similarity score; Assign attention weights to the second input feature input based on the similarity score; Performing weighted summation on the second input feature based on the attention weight to obtain an output feature of the attention head; Integrate the output features of multiple groups of attention heads to obtain refined multi-scale features.

8. A small target end-to-end detection system, used to implement the small target end-to-end detection method according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used for acquiring an image to be detected; Feature extraction module, used to extract initial features of the image; A first processing module, used for inputting the initial features into a multi-scale feature extraction module, and the multi-scale feature extraction module processes the initial features according to preset rules and outputs multi-scale features; The second processing module is used to fuse and reconstruct the multi-scale features according to a preset context guidance mechanism to obtain enhanced fusion features; The output module is used to determine the target recognition result of the image based on the enhanced fusion features.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor (10); and, a memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10), and the computer program is executed by the at least one processor (10) so that the at least one processor (10) can execute the small target end-to-end detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program; when the computer program is executed by the processor, the small target end-to-end detection method as described in any one of claims 1 to 7 is implemented.

Images