Intensive small target detection method and device for remote sensing image under complex background, equipment and medium

Abstract

The invention discloses an intensive small target detection method and device for a remote sensing image under a complex background, equipment and a medium, and the method comprises the steps: obtaining a to-be-detected remote sensing image, enabling the input image to pass through a series of convolution layers, activation functions and pooling layers, and gradually extracting the features from a low layer to a high layer; a target enhancement mechanism ORM is added behind each RepNCSPELAN4 module, an EMA attention mechanism is introduced, and feature representation is optimized through multi-scale interaction; enhancing feature representation of the output feature map through an up-sampling module, a down-sampling module and a feature pyramid network module, and preparing for detection; wherein a DSConv operator is adopted to perform down-sampling on a feature map, a context region perception fusion mechanism CRAFM is designed in front of a detection head, and global context information is utilized to represent a cross-space relationship between pixels; and finally detecting the output feature map. According to the invention, multi-scale information can be effectively extracted, and the detection capability of small objects is improved; the complex background is well inhibited, the object feature information is enhanced, and the detection capability of dense objects is improved.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image target detection, and specifically, relates to a method, device, equipment and medium for intensive small target detection of remote sensing images under complex backgrounds. Background Art

[0002] Remote sensing image target detection is a key research field in remote sensing image processing. Its core purpose is to accurately locate and identify specific targets in complex remote sensing images. With the increasing advancement of remote sensing technology, remote sensing images have shown broad application prospects in many fields such as autonomous driving, urban planning, environmental monitoring, disaster warning, and agriculture. However, the characteristics of remote sensing images make target detection particularly complex and difficult. These images often have complex backgrounds, unique perspectives, large variations in target scales, and targets may be densely distributed. At the same time, due to differences in weather, imaging technology, seasonal changes, and equipment performance, the imaging quality of remote sensing images is also uneven. These characteristics pose great challenges to target detection in remote sensing images. At present, remote sensing image target detection faces three main challenges:

[0003] (1) Small target detection problem. Due to the long shooting distance and wide coverage area, the number of pixels of small targets in remote sensing images is relatively small, resulting in weak feature expression. This makes it easy for small targets to be ignored or misjudged during the detection process, thus increasing the difficulty of detection.

[0004] (2) Detection problems in complex backgrounds. Unlike traditional natural images, the backgrounds of remote sensing images are usually more complex and changeable. The uniqueness of the shooting angle leads to significant differences in the way the target is presented. At the same time, affected by a variety of objective factors, the imaging quality of remote sensing images is often unsatisfactory. These factors have brought considerable difficulties to the accurate detection of targets in complex backgrounds.

[0005] (3) Dense target detection problem. Since remote sensing images are usually captured in the air, the ground object information in the image is very rich and dense. In this case, the targets may overlap or be closely arranged, making it difficult for the detection algorithm to accurately distinguish and locate each target. Therefore, the dense target detection problem is an important test for remote sensing image target detection algorithms.

[0006] To solve the above problems in remote sensing image target detection, researchers have conducted extensive and in-depth experiments. Most of the current detectors are based on a two-stage framework, for example, Zhang J, Xie C, Xu X, et al. A contextual bidirectional enhancement method for remote sensing image object detection [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 4518-4531. The contextual bidirectional enhancement method enhances the target features, reduces the background noise information, and effectively improves the accuracy of remote sensing image target detection. In order to further enhance the feature extraction capability and overcome the difficulty of remote sensing image feature extraction, Zhu D, Xia S, Zhao J, et al. Spatial hierarchy perception and hard samples metric learning for high-resolution remote sensing image object detection [J]. Applied Intelligence, 2022, 52 (3): 3193-3208. A spatial hierarchical perception component (SHPC) is designed to search for features in multiple spatial layers. At the same time, the hard sample metric learning (HSML) method is used to reduce false detections and improve detection performance. Drawing on the idea of ​​feature pyramid network (FPN), Guo C, Fan B, Zhang Q, et al. Augfpn: Improving multi-scale feature learning for object detection [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 12595-12604. Multi-scale object proposal networks are also studied. By generating candidate regions with different intermediate layer features and combining them with multi-scale object detection networks for prediction, the object detection accuracy of remote sensing images is further improved.Yue C, Yan J, Zhang Y, et al. SCFNet: Semantic correction and focus network for remote sensing image object detection [J]. Expert Systems with Applications, 2023, 224: 119980. A semantic correction and focus network is designed to effectively solve the problem of the complex relationship between the target and the background of the remote sensing image, significantly improving the detection accuracy. In order to reduce the complexity of the model, Xu X, Feng Z, Cao C, et al. An improved swin transformer-based model for remote sensing object detection and instance segmentation [J]. Remote Sensing, 2021, 13 (23): 4779. A local perception backbone based on Swin Transformer is proposed for remote sensing image target detection and instance segmentation, and the performance of the backbone under different detection frameworks is studied.

[0007] Although the above methods have solved the problem of remote sensing image target detection to a certain extent and improved the detection accuracy. First, although these methods have processed complex background and target features, there is still an incomplete separation between the background and the target, which leads to missed detection or false detection. Secondly, the above algorithms fail to effectively deal with the impact of scale changes, which is particularly prominent in remote sensing images, and may lead to poor detection performance for targets with large scale changes. Finally, the two-stage algorithm has a high computational complexity, which leads to a significant increase in the computational time cost. Therefore, further research and improvement of remote sensing image target detection algorithms are still needed to overcome these challenges. Summary of the invention

[0008] The purpose of the present invention is to provide a method, device, equipment and medium for intensive small target detection of remote sensing images under complex backgrounds, which can effectively solve the problem of intensive small target detection under complex backgrounds.

[0009] The technical solution to achieve the purpose of the present invention is: a method for intensive small target detection of remote sensing images under complex backgrounds, comprising:

[0010] Step S1, obtain the remote sensing image to be detected, and pass the image through a series of convolutional layers, feature extraction fusion module RepNCSPELAN4, target reinforcement mechanism module ORM under the YOLOv9 framework, and finally through the attention mechanism module EMA to gradually extract features from low-level to high-level;

[0011] Step S2, the output feature map is processed through upsampling, downsampling and feature pyramid network modules to enhance feature representation and prepare for detection; wherein, the DSConv operator is used to downsample the feature map, and a context region aware fusion mechanism CRAFM is designed before the detection head to use global context information to represent the cross-spatial relationship between pixels;

[0012] Step S3, perform a final test on the output feature map. Further, in step S1, a target reinforcement mechanism module ORM is added after each RepNCSPELAN4 module, and the specific implementation of the target reinforcement mechanism ORM includes:

[0013] (1) Feature adjustment: ORM consists of four branches, each of which performs a 1×1 convolution operation on the input feature map;

[0014] (2) Residual structure: Select any branch to adopt the residual structure, that is, add a "short-circuit connection" in the branch so that the network can learn the residual between the input and output; the other three branches perform cascaded standard convolution operations, which use convolution kernels of different sizes;

[0015] (3) Atrous convolutional layer: Add atrous convolutional layer on any two branches.

[0016] Furthermore, the specific implementation of the attention mechanism module EMA is as follows:

[0017] (1) Feature grouping: The input feature map is divided into multiple sub-features along the channel dimension to generate a grouped feature map;

[0018] (2) Parallel subnetwork: extract the attention weight descriptor from the group feature map through three parallel branches; two 1×1 branches perform one-dimensional global average pooling GAP in the horizontal and vertical directions respectively to encode the intra-channel information of the two branches, as shown in formulas (1) and (2); the encoded features are connected and input into 1×1 convolution, and then the dual-channel attention map is generated using the Sigmoid function; the 3×3 branch performs 3×3 convolution on the input features of the group feature map to expand the feature space; and then average pooling is performed to generate a spatial attention map

[0019]

[0020] Among them, F represents the output, H and W represent the height and width of the input respectively, C represents the number of input channels, and f k Represents the input features of the kth channel;

[0021] (3) Cross-space learning: According to the definition of formula (3), the spatial attention feature maps generated on the 1×1 and 3×3 branches are subjected to two-dimensional adaptive GAP operation and reshaped; then, the data is normalized using the SoftMax function; then, the results obtained on the two branches are element-wise multiplied to generate a spatial attention map that retains the complete spatial position information;

[0022]

[0023] (4) Attention mechanism: The sigmoid function is used to derive the weights of the above attention map, and then these weights are incorporated into the final feature map.

[0024] Furthermore, the specific implementation of the contextual region aware fusion mechanism CRAFM is as follows:

[0025] It contains three main branches. The first branch uses global average pooling GAP and global maximum pooling GMP to integrate global information. These two operations are to achieve dimensionality reduction and spatial information integration through feature mapping while retaining key information, so that CRAFM can fully understand the contextual information of the channel dimension. The second branch uses 1×1 convolution to obtain the linear transformation result of the feature map; the third branch also uses 1×1 convolution to simplify the multiples of the query (Q) and key (K); then, the outputs of the initial and third branches are multiplied by the matrix of the second branch to obtain two results, which represent the content detail extraction across channels and across spaces respectively; finally, the two results are multiplied element by element to obtain the final output of CRAFM; the equation is as follows:

[0026] In the formula, and Represent the input and output of the qth pixel in the pth level feature map, N p Indicates the total number of pixels; μ Value and μ QK are the linear transformation matrices used to project feature maps, which are simplified by 1×1 convolution; Avg(·) performs global average pooling (GAP) and Max(·) performs global maximum pooling (GMP).

[0027] A dense small target detection device for remote sensing images under complex backgrounds, comprising:

[0028] The feature extraction network module obtains the remote sensing image to be detected, passes the image through a series of convolutional layers, feature extraction fusion module RepNCSPELAN4, target reinforcement mechanism module ORM under the YOLOv9 framework, and finally extracts features from low-level to high-level through the attention mechanism module EMA;

[0029] The feature enhancement module processes the output feature map through upsampling, downsampling and feature pyramid network modules to enhance the feature representation and prepare it for detection. The DSConv operator is used to downsample the feature map, and the context region awareness fusion mechanism module CRAFM is designed before the detection head to use global context information to represent the cross-spatial relationship between pixels.

[0030] The detection module is used to perform final detection on the output feature map.

[0031] Furthermore, the target reinforcement mechanism module ORM includes:

[0032] Feature Reshaping Module: ORM consists of four branches, each of which performs a 1×1 convolution operation on the input feature map;

[0033] Residual structure module: select any branch to adopt the residual structure, add a "short-circuit connection" in the branch, so that the network can learn the residual between the input and output; the other three branches perform cascaded standard convolution operations, which use convolution kernels of different sizes;

[0034] Atrous convolutional layer module: Add atrous convolutional layers on any two branches.

[0035] Furthermore, the attention mechanism module EMA includes:

[0036] Feature grouping module: used to divide the input feature map into multiple sub-features along the channel dimension and generate a grouped feature map;

[0037] Parallel subnetwork module; extract attention weight descriptors from group feature maps through three parallel branches; two 1×1 branches perform one-dimensional global average pooling GAP in the horizontal and vertical directions respectively to encode the intra-channel information of the two branches, as shown in formulas (1) and (2); the encoded features are connected and input into 1×1 convolution, and then the dual-channel attention map is generated using the Sigmoid function; the 3×3 branch performs 3×3 convolution on the input features of the group feature map to expand the feature space; and then average pooling is performed to generate a spatial attention map

[0038]

[0039] Among them, F represents the output, H and W represent the height and width of the input respectively, C represents the number of input channels, and f k Represents the input features of the kth channel;

[0040] Cross-spatial learning module: According to the definition of formula (3), the spatial attention feature maps generated on the 1×1 and 3×3 branches are subjected to two-dimensional adaptive GAP operation and reshaped; then, the data is normalized using the SoftMax function; then, the results obtained on the two branches are element-wise multiplied to generate a spatial attention map that retains the complete spatial position information;

[0041]

[0042] Attention mechanism module: Use the Sigmoid function to derive the weights of the above attention map, and then incorporate these weights into the final feature map.

[0043] Furthermore, the context region aware fusion mechanism CRAFM module includes: three main branches, the first branch uses global average pooling GAP and global maximum pooling GMP to integrate global information; the second branch uses 1×1 convolution to obtain the linear transformation result of the feature map; the third branch also uses 1×1 convolution to simplify the multiples of the query (Q) and the key (K); then, the outputs of the initial and third branches are multiplied by the matrix of the second branch to obtain two results, which represent the content detail extraction across channels and across spaces respectively; finally, the two results are multiplied element by element to obtain the final output of CRAFM; the equation is as follows:

[0044] In the formula, and Represent the input and output of the qth pixel in the pth level feature map, N p Indicates the total number of pixels; μ Value and μ QK are the linear transformation matrices used to project feature maps, which are simplified by 1×1 convolution; Avg(·) performs global average pooling GAP, and Max(·) performs global maximum pooling GMP.

[0045] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0046] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0047] Compared with the prior art, the present invention has the following significant advantages: (1) The ORM mechanism is innovatively proposed, which significantly improves the local perception ability of the network and can be applied to other networks as a general mechanism to optimize feature representation and reduce the interference of complex background. (2) The CRAFM mechanism is designed to effectively enhance the global correlation ability across space, improve the multi-scale feature fusion effect, and achieve accurate separation and detection of dense targets. This mechanism is also universal and can be easily integrated into other networks. (3) By introducing EMA and DSConv networks, the multi-scale feature representation ability and model efficiency are improved respectively, providing strong support for practical applications. (4) In comparison experiments with six algorithms on multiple data sets such as DOTA, DIOR and VisDrone2019, the EOCD-YOLOv9 algorithm adopted by the present invention achieved the best detection effect, showing great potential in future airborne real-time applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is the overall framework diagram of the target detection model of the present invention.

[0049] Figure 2 It is the EMA structure diagram of the present invention.

[0050] Figure 3 This is the ORM structure diagram of the present invention.

[0051] Figure 4 This is a structural diagram of the CRAFM of the present invention.

[0052] Figure 5 This is the label distribution diagram of DOTA, DIOR and VisDrone2019.

[0053] Figure 6 This is a graph showing the results of the algorithm proposed in this invention on the DOTA dataset.

[0054] Figure 7 It is a bar chart of the comparative experiment of DOTA dataset of the present invention.

[0055] Figure 8 This is the result diagram of the algorithm proposed in the present invention on the DIOR dataset.

[0056] Fig. 9 This is a bar chart comparing the DIOR dataset experiments of the present invention.

[0057] Fig.10 This is a graph showing the results of the algorithm proposed in this invention on the VisDrone2019 dataset.

[0058] Fig.11 This is a bar chart of the comparative experiment of the VisDrone2019 dataset of the present invention.

[0059] Fig.12 The present invention is a flow chart of the intensive small target detection method of remote sensing images under complex background. DETAILED DESCRIPTION

[0060] The present invention selects the YOLO algorithm as the basic framework and designs a special enhancement mechanism for the detection requirements of dense small targets in complex backgrounds. Through such improvements, the method of the present invention can better adapt to remote sensing tasks, thereby further improving the accuracy and efficiency of target detection.

[0061] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0062] Combination Fig.12 The present invention provides a method for detecting small dense targets in remote sensing images under complex backgrounds. The method adopts the basic architecture of YOLOv9 and proposes an improved target detection model based on it. The model further improves the feature extraction capability, especially the detection capability of small targets and dense targets under complex backgrounds, by introducing ORM, CRAFM, ​​EMA and adopting a new downsampling network structure. Figure 1 The framework of this improved model is clearly presented and emphasized in the improved parts.

[0063] Furthermore, the present invention proposes an EMA attention mechanism module.

[0064] The EMA module is an advanced attention mechanism based on the principle of coordinated attention (CA), which excels at improving image feature representation by complexly modeling the interactions between channels and spatial dimensions. By being good at capturing the similarities between global and local features, the EMA module can quickly and comprehensively examine image features at multiple scales. This enhancement ultimately helps to gain a deeper understanding and accurately depict the complex structures contained in the image. Figure 2 As shown, the main process of this module is as follows:

[0065] (1) Feature grouping. The first step in this process is to split the input feature map into many sub-features along the channel dimension to generate a grouped feature map; comprehensively consider the forward-looking channel correlation and difference, thereby effectively enhancing the model's ability to express complex details.

[0066] (2) Parallel sub-networks. Subsequently, the attention weight descriptor is extracted from the group feature map through three parallel paths. The two 1×1 branches perform one-dimensional global average pooling (GAP) in the horizontal and vertical directions respectively to encode the intra-channel information, as shown in formulas (1) and (2). The encoded features are then concatenated and input into a 1×1 convolution, and then a two-channel attention map is generated using a sigmoid function. The 3×3 branch technique performs a 3×3 convolution on the input features of the subgroup to expand the feature space. Average pooling is then performed to generate a spatial attention map.

[0067]

[0068] Where F represents the output, H and W represent the height and width of the input respectively, C represents the number of input channels, and f k Represents the input feature of the kth channel.

[0069] (3) Cross-space learning. First, according to the definition of formula (3), the spatial attention feature maps generated on the 1×1 and 3×3 branches are subjected to two-dimensional adaptive GAP operations and reshaped. Subsequently, the data is normalized using the SoftMax function. Subsequently, the results obtained on the two branches are element-wise multiplied to generate a spatial attention map that retains the complete spatial position information. This method integrates information from different spatial dimensions, enhances the feature representation capability, and improves the accuracy and performance of object detection.

[0070]

[0071] Where F represents the output, H and W represent the height and width of the input respectively, C represents the number of input channels, and fx represents the input feature of the kth channel.

[0072] (4) Attention mechanism. Finally, the sigmoid function is used to obtain the weights of the above attention map, and then these weights are incorporated into the final feature map.

[0073] The above steps show that the EMA module can significantly improve the accuracy of the model in identifying small objects by integrating global and local information and capturing spatial background details. In addition, this module clearly has a considerable advantage in detecting small objects in RSI.

[0074] Furthermore, the present invention proposes an ORM (Object Recognition Module) mechanism, which strengthens the features of small objects from two perspectives. On the one hand, by adopting a multi-branch convolution structure, multiple discriminative semantic information can be extracted, thereby increasing the richness of the features. On the other hand, Alex convolution is applied to expand the receptive field and obtain richer local context information.

[0075] The object reinforcement mechanism (ORM) is embedded into the backbone network, and its overall structure is as follows: Figure 3 As shown. Next, we will introduce the working principle of this structure in detail:

[0076] (1) Feature adjustment: First, each branch of the ORM performs a 1×1 convolution operation on the input feature map. The main purpose of this operation is to preliminarily adjust the number of channels required for subsequent processing so that the feature map can more effectively transmit information in the subsequent convolution operation.

[0077] (2) Residual structure: The first branch adopts the residual structure. The core idea of ​​the residual structure is to add a "shortcut connection" so that the network can learn the residual between the input and output, rather than directly learning the mapping from input to output. This structure helps solve the gradient vanishing or gradient exploding problems that may occur in the training process of deep neural networks, while retaining the key feature information of small targets and preventing it from being lost during deep convolution.

[0078] (3) Cascaded standard convolution: Except for the residual structure branch, the other three branches perform cascaded standard convolution operations. These convolution operations use convolution kernels of different sizes (1×3, 3×1, and 3×3) to extract features from different angles. This design can increase the receptive field of the network and enable the network to capture richer spatial information.

[0079] (4) Atrous convolutional layers: In particular, additional atrous convolutional layers are added to the middle two branches. This is done to further extract and retain more contextual information. This design draws on the idea of ​​RFB-s (Receptive Field Block-small), which helps to improve the network's sensitivity to local details and contextual information.

[0080] The mathematical expression of ORM is:

[0081] Y 1 =Conv 3×3 [Conv 1×1 (X)] (4)

[0082]

[0083]

[0084] Among them, Conv 1×3 、Conv 3×1 、Conv 3×3 and Conv 1×1 They represent standard convolution operations with kernel sizes of 1×3, 3×1, 3×3, and 1×1, respectively. represents a secondary convolution operation with a dilation rate of 5. Concat(·) is a feature map concatenation operation. represents the element-wise addition operation of feature maps. X is the input feature map. Y 1 , Y 2 , Y 3 It represents the output feature map of the first three branches after standard convolution and At convolution. Y is the output feature map of ORM.

[0085] Furthermore, after the application of ORM and EMA, the feature map has integrated multi-scale feature information and effectively suppressed the interference of background information by strengthening the target information, thereby achieving a good representation of small targets and densely arranged targets. In order to further model the complex background and the global relationship of the target, the backbone network part was improved again, and a context region-aware fusion mechanism (CRAFM) was designed before the detection head.

[0086] This mechanism makes full use of global contextual information to represent the cross-spatial relationship between pixels, which helps to suppress useless background and significantly enhance the ability to distinguish between targets and complex backgrounds. CRAFM consists of three branches. The first branch uses global average pooling (GAP) and global maximum pooling (GMP) to integrate global information. GAP and GMP can help feature maps reduce dimensions and integrate spatial information while retaining important information, so that CRAFM can fully understand the contextual information of the channel size. The second branch uses 1×1 convolution to generate a linear transformation result of the feature map, which is Figure 4 The third branch also uses a 1×1 convolution to simplify the query and key multiples. This convolution is in Figure 4 It is called QK in the paper. Then, the outputs of the first branch and the third branch are multiplied by the matrix of the second branch respectively, and the two branches obtained represent the cross-channel and spatial context information respectively. Finally, the final output of CRAFM is obtained on these two branches using the broadcast Hadamard product. The structure of CRAFM is as follows Figure 4 In each layer, the pixel space context can be effectively represented and fused, which further enhances the model's ability to understand and model the global relationship between complex scenes and objects. The expression is:

[0087]

[0088] In the formula, and Represent the input and output of the qth pixel in the pth level feature map, N p Indicates the total number of pixels. Value and μ QKare the linear transformation matrices used to project the feature maps, which are simplified by 1×1 convolutions. Avg(·) performs global average pooling (GAP) and Max(·) performs global maximum pooling (GMP).

[0089] Furthermore, the present invention uses the DSConv operator to downsample the feature map. DSConv is used to replace the four standard downsampling convolutional networks originally used in backbone and Neck. This replacement aims to maintain the performance of the model while reducing the computational complexity, making the model more suitable for real-time applications.

[0090] The effect of the present invention will be further described below in conjunction with experiments.

[0091] To evaluate the performance of the proposed method, the DOTA, DIOR and VisDrone2019 datasets were used for training and validation, and the proposed algorithm was compared with the current advanced and classic algorithms, including Casade-RCNN, mSODANet, Cascade-DETR, SGMFNet, SP-YOLOv8s

[32] , and YOLOv7-sea. All experiments were conducted under the same hardware and software environment, and the configuration is shown in Table 1. The network of the present invention was implemented using PyTorch on the Ubuntu operating system, using Nvidia GeForce RTX 3090 GPU and CUDA12.1. The stochastic gradient descent (SGD) optimizer was used for parameter learning, with the initial learning rate set to 0.01, the momentum set to 0.937, and the weight decay set to 0.005. The batch size during training was set to 8 and the image size was 640.

[0092] Table 1 Experimental environment configuration

[0093]

[0094]

[0095] Experimental Dataset

[0096] Three public datasets were selected for experimental learning and verification, namely DOTA, DIOR, and VisDrone2019 datasets. These three datasets are widely used in the field of RSI and are very suitable for our experimental background. Table 2 summarizes the basic characteristics of each dataset. Figure 5 The label distribution, number of instances and other relevant details of the DOTA, DIOR and VisDrone2019 datasets are described.

[0097] Table 2 Dataset overview

[0098]

[0099] Experimental evaluation indicators

[0100] To verify the accuracy of the experiment, five indicators, mAP@50, mAP@0.5:0.95, Precision, Recall, and Parameters (Para.), were selected as tools to qualitatively measure the experimental results. Among them, mAP@50 refers to the average precision when the IoU (Intersection over Union) threshold is set to 0.5, which measures the accuracy of the model in detecting targets. The higher the value, the more accurate it is. mAP@0.5:0.95 calculates the average precision of all categories in the IoU threshold range from 0.5 to 0.95. The higher the value, the more accurate it is. Precision refers to the proportion of samples classified as positive that are truly positive. High Precision means that there are fewer false positives in the target objects predicted by the model. Recall measures the percentage of correctly detected positive samples to all positive samples. High Recall means that the model can detect more real target objects and reduce missed detections. The number of parameters refers to the number of parameters contained in the model. The number of parameters is an important indicator for measuring the complexity of the model and the required computing resources.

[0101] Comparative experiment

[0102] In order to verify the detection effect of the proposed algorithm on dense small targets in complex environments, the proposed algorithm is compared with six advanced or classic algorithms on the DOTA, DIOR and VisDrone2019 datasets, which will be described in detail below.

[0103] Comparative experiments on the DOTA dataset

[0104] Figure 6 The detection results of the proposed algorithm on the DOTA dataset are shown. It can be seen that the proposed algorithm can accurately detect densely arranged targets and targets of different directions and shapes in these images, and also has good detection performance for some overlapping targets. In addition, Table 3 shows the comparison results of the proposed algorithm with the other six algorithms in mAP@50, mAP@0.5:0.95, Precision, and Recall. It can be seen that the proposed algorithm performs best in these indicators. The mAP@50 value is 12.1% higher than the second highest Cascade-DETR algorithm. It can be seen that the proposed EOCD-YOLOv9 algorithm has a good detection effect on this dataset. In order to more intuitively see the comparison results of different indicators of each algorithm, Fig. 9 The comparative experimental results are presented in a bar graph.

[0105] Table 3 Comparative experiments on the DOTA dataset

[0106]

[0107] Comparative experiments on the DIOR dataset

[0108] Figure 8 The detection results of the proposed algorithm on the DIOR dataset are shown. It can be seen that the proposed algorithm not only has a good detection effect on conventional targets, but also has good detection performance for small targets. At the same time, Table 4 shows the comparison results of the proposed algorithm with the other six algorithms in mAP@50, mAP@0.5:0.95, Precision and Recall. It can be seen that the proposed algorithm performs best in these indicators, among which the mAP@50 value is 9.2% higher than the second highest Cascade-DETR algorithm, and the average detection accuracy reaches 87.5%. In order to more intuitively see the comparison results of different indicators of each algorithm, Fig. 9 The comparative experimental results are presented in a bar graph.

[0109] Table 4 Comparative experiments on the DIOR dataset

[0110]

[0111] Comparative experiments on the VisDrone2019 dataset

[0112] In order to further verify the performance of the proposed algorithm, the VisDrone2019 dataset was also selected for experiments. The purpose is to better test the proposed algorithm's detection ability for dense small targets and drone data, so as to realize airborne applications in the future. Fig.10 The detection results of the proposed algorithm on the dataset are shown in Table 5. The comparison results of the proposed algorithm with the other six algorithms in mAP@50, mAP@0.5:0.95, Precision and Recall are shown in Table 5. Comprehensive analysis shows that the proposed algorithm can still accurately detect remote sensing images from the perspective of drones, and still has high detection accuracy for dense small targets. The proposed algorithm performs best in these indicators, and the mAP@50 value is 6.8% higher than the second highest Cascade-DETR algorithm. In order to more intuitively see the comparison results of different indicators of each algorithm, Fig.11 The comparative experimental results are presented in a bar graph.

[0113] Table 5 Comparative experiments on the VisDrone2019 dataset

[0114]

[0115] Ablation experiment

[0116] In order to analyze the importance of each component in the proposed algorithm, EMA, ORM, CRAFM and DSConv are gradually applied in the baseline, and ablation experiments are performed on the DOTA dataset to verify their effectiveness. Table 6 shows the impact of adding or reducing each module on the evaluation indicators, where √ indicates that the module is used and × indicates that the module is not used.

[0117] (1) EMA: As can be seen from Table 6, the added EMA attention mechanism can effectively improve the values ​​of various indicators, and its mAP@50 is increased by 8.3%, indicating that this mechanism can promote the target detection capability of remote sensing images.

[0118] (2) ORM: As shown in Table 6, adding ORM can significantly improve all evaluation indicators, including accuracy (from 75.6 to 76.1) and map, which confirms that the mechanism is more likely to distinguish small objects from the background. This is because ORM enriches the local background features and the network shows good suppression effect under complex backgrounds.

[0119] (3) CRAFM: Table 6 shows the performance improvement after adding CRAFM, ​​which can improve all evaluation indicators. This shows that this mechanism further enhances the feature representation of small objects and suppresses the background.

[0120] (4) DSConv: The proposed downsampling network can make the model lighter and more real-time. From the Para. value in the table, we can see that after replacing the original downsampling network with this network, the number of parameters is reduced by 2.9%, and other index values ​​are slightly improved. Therefore, it can be shown that this network not only makes the model lighter while maintaining the detection accuracy, but also improves the detection accuracy of the model.

[0121] Table 6 Ablation experiments of improved mechanisms

[0122]

[0123] Through the above ablation experiment analysis, it can be concluded that the proposed mechanisms all stably improve the performance of the proposed algorithm without any conflict.

[0124] In summary, due to the complex background, close arrangement of objects and the presence of small objects in remote sensing images such as drones, the task of object detection in remote sensing images such as drones is particularly difficult. Therefore, the present invention proposes a new model to overcome the above difficulties.

[0125] The present invention introduces EMA, which can effectively extract multi-scale information and improve the detection ability of small objects. The ORM method is proposed, which obtains different receptive fields by using a multi-branch structure, integrates the regional background details of small objects, and well suppresses the complex background, thereby enhancing the object feature information. CRAFM adopts global pooling technology to promote the acquisition of global context information, learn the correlation between channels, reconstruct the correlation between pixels, and obtain cross-channel and cross-space global context information, thereby improving the detection ability of dense objects. The introduced DSConv network improves the training efficiency and reduces the number of parameters. The verification results on three datasets and the comparison results with six state-of-the-art algorithms show that the proposed algorithm shows the best performance in this task and is significantly better than the comparison algorithms in all indicators. In addition, qualitative results show that the proposed algorithm can accurately detect small and dense objects.

Claims

1. A method for dense small target detection in remote sensing images under complex backgrounds, characterized in that: include: Step S1, obtain the remote sensing image to be detected, and pass the image through a series of convolutional layers, feature extraction fusion module RepNCSPELAN4, target reinforcement mechanism module ORM under the YOLOv9 framework, and finally through the attention mechanism module EMA to gradually extract features from low-level to high-level; Step S2, the output feature map is processed through upsampling, downsampling and feature pyramid network modules to enhance feature representation and prepare for detection; wherein, the DSConv operator is used to downsample the feature map, and a context region aware fusion mechanism CRAFM is designed before the detection head to use global context information to represent the cross-spatial relationship between pixels; Step S3, performing final detection on the output feature map.

2. The method for dense small target detection in remote sensing images under complex background according to claim 1, characterized in that: In step S1, a target reinforcement mechanism module ORM is added after each RepNCSPELAN4 module. The specific implementation of the target reinforcement mechanism ORM includes: (1) Feature adjustment: ORM consists of four branches, each of which performs a 1×1 convolution operation on the input feature map; (2) Residual structure: Select any branch to adopt the residual structure, that is, add a "short-circuit connection" in the branch so that the network can learn the residual between the input and output; the other three branches perform cascaded standard convolution operations, which use convolution kernels of different sizes; (3) Atrous convolutional layer: Add atrous convolutional layer on any two branches.

3. The method for dense small target detection in remote sensing images under complex background according to claim 1, characterized in that: The specific implementation of the attention mechanism module EMA is: (1) Feature grouping: The input feature map is divided into multiple sub-features along the channel dimension to generate a grouped feature map; (2) Parallel subnetwork: extract the attention weight descriptor from the group feature map through three parallel branches; two 1×1 branches perform one-dimensional global average pooling GAP in the horizontal and vertical directions respectively to encode the intra-channel information of the two branches, as shown in formulas (1) and (2); the encoded features are connected and input into 1×1 convolution, and then the dual-channel attention map is generated using the Sigmoid function; the 3×3 branch performs 3×3 convolution on the input features of the group feature map to expand the feature space; and then average pooling is performed to generate a spatial attention map Where F represents the output, H and W represent the height and width of the input, c represents the number of input channels, and f k Represents the input features of the kth channel; (3) Cross-space learning: According to the definition of formula (3), the spatial attention feature maps generated on the 1×1 and 3×3 branches are subjected to two-dimensional adaptive GAP operation and reshaped; then, the data is normalized using the SoftMax function; then, the results obtained on the two branches are element-wise multiplied to generate a spatial attention map that retains the complete spatial position information; (4) Attention mechanism: The sigmoid function is used to derive the weights of the above attention map, and then these weights are incorporated into the final feature map.

4. The method for intensive small target detection in remote sensing images under complex background according to claim 1, characterized in that: The specific implementation of the contextual region aware fusion mechanism CRAFM is as follows: It contains three main branches. The first branch uses global average pooling GAP and global maximum pooling GMP to integrate global information. The second branch uses 1×1 convolution to obtain the linear transformation result of the feature map. The third branch also uses 1×1 convolution to simplify the multiples of the query (Q) and key (K). Subsequently, the outputs of the initial and third branches are multiplied by the matrix of the second branch to obtain two results, which represent the content detail extraction across channels and across spaces respectively. Finally, the two results are multiplied element by element to obtain the final output of CRAFM. The equation is as follows: In the formula, and Represent the input and output of the qth pixel in the pth level feature map, N p Indicates the total number of pixels; μ Value and μ QK are the linear transformation matrices used to project feature maps, which are simplified by 1×1 convolution; Avg(·) performs global average pooling (GAP) and Max(·) performs global maximum pooling (GMP).

5. A dense small target detection device for remote sensing images under complex backgrounds, characterized in that: include: The feature extraction network module obtains the remote sensing image to be detected, passes the image through a series of convolutional layers, feature extraction fusion module RepNCSPELAN4, target reinforcement mechanism module ORM under the YOLOv9 framework, and finally extracts features from low-level to high-level through the attention mechanism module EMA; The feature enhancement module processes the output feature map through upsampling, downsampling and feature pyramid network modules to enhance the feature representation and prepare it for detection. The DSConv operator is used to downsample the feature map, and the context region awareness fusion mechanism module CRAFM is designed before the detection head to use global context information to represent the cross-spatial relationship between pixels. The detection module is used to perform final detection on the output feature map.

6. The device for detecting small targets in a remote sensing image under a complex background according to claim 5, characterized in that: The target reinforcement mechanism module ORM includes: Feature Reshaping Module: ORM consists of four branches, each of which performs a 1×1 convolution operation on the input feature map; Residual structure module: select any branch to adopt the residual structure, add a "short-circuit connection" in the branch, so that the network can learn the residual between the input and output; the other three branches perform cascaded standard convolution operations, which use convolution kernels of different sizes; Atrous convolutional layer module: Add atrous convolutional layers on any two branches.

7. The device for detecting small targets of remote sensing images under complex background according to claim 5, characterized in that: The attention mechanism module EMA includes: Feature grouping module: used to divide the input feature map into multiple sub-features along the channel dimension and generate a grouped feature map; Parallel subnetwork module; extract attention weight descriptors from group feature maps through three parallel branches; two 1×1 branches perform one-dimensional global average pooling GAP in the horizontal and vertical directions respectively to encode the intra-channel information of the two branches, as shown in formulas (1) and (2); the encoded features are connected and input into 1×1 convolution, and then the dual-channel attention map is generated using the Sigmoid function; the 3×3 branch performs 3×3 convolution on the input features of the group feature map to expand the feature space; and then average pooling is performed to generate a spatial attention map Among them, F represents the output, H and W represent the height and width of the input respectively, C represents the number of input channels, and f k Represents the input features of the kth channel; Cross-spatial learning module: According to the definition of formula (3), the spatial attention feature maps generated on the 1×1 and 3×3 branches are subjected to two-dimensional adaptive GAP operation and reshaped; then, the data is normalized using the SoftMax function; then, the results obtained on the two branches are element-wise multiplied to generate a spatial attention map that retains the complete spatial position information; Attention mechanism module: Use the Sigmoid function to derive the weights of the above attention map, and then incorporate these weights into the final feature map.

8. The device for detecting small targets in remote sensing images under complex background according to claim 5, characterized in that: The context region aware fusion mechanism CRAFM module includes: three main branches, the first branch uses global average pooling GAP and global maximum pooling GMP to integrate global information; the second branch uses 1×1 convolution to obtain the linear transformation result of the feature map; the third branch also uses 1×1 convolution to simplify the multiples of the query (Q) and the key (K); then, the outputs of the initial and third branches are multiplied by the matrix of the second branch to obtain two results, which represent the content detail extraction across channels and across spaces respectively; finally, the two results are multiplied element by element to obtain the final output of CRAFM; the equation is as follows: In the formula, and Represent the input and output of the qth pixel in the pth level feature map, N p Indicates the total number of pixels; μ Value and μ QK are the linear transformation matrices used to project feature maps, which are simplified by 1×1 convolution; Avg(·) performs global average pooling GAP, and Max(·) performs global maximum pooling GMP.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images