Remote sensing image small-scale target detection method based on edge attention mechanism

Abstract

The invention relates to a remote sensing image small-scale target detection method based on an edge attention mechanism, and belongs to the technical field of target detection. According to the method, an edge attention mechanism and an edge perception enhancement module are designed in a detection head network, so that the feature extraction capability and the boundary perception capability of a target detection network on a small-scale target are enhanced by establishing global relevance between row elements and column elements in a remote sensing image feature map; therefore, high-precision small-scale target detection of the remote sensing image is realized, and the problem of difficulty in small-scale target recognition and positioning caused by fuzzy features and small positioning error tolerance is solved. By adjusting the output characteristic parameters of the detection head network, the method can be conveniently transplanted to detection tasks of various target scenes and data sets. The method is suitable for various complex scene target detection tasks such as ocean safety, intelligent traffic and battlefield intelligence guarantee.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a small-scale target detection method for remote sensing images based on an edge attention mechanism. Background Art

[0002] Small-scale target detection is a classic challenge in remote sensing intelligent interpretation and has shown great application potential in many fields. Generally, the difficulty of target detection tasks is related to the feature richness and size of the target. Compared with standard-sized targets, the appearance features of small-scale targets are more blurred and more easily submerged by complex background environments. In addition, small-scale targets have a lower tolerance for positioning errors. Positioning errors that can be tolerated in standard-sized target detection may lead to failure in small target detection. Figure 1 As shown in Figure 3, as the target size decreases, the intersection over union (IoU) caused by edge positioning error changes more dramatically.

[0003] The attention mechanism is a machine learning data processing method with widespread applications in tasks such as natural language processing, image processing, and speech recognition. Self-attention (see Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need [J]. arXiv, 2017. DOI: 10.48550 / arXiv.1706.03762) is one of the most widely used attention mechanisms. By assisting object detection models in focusing on key regions of feature maps, the self-attention mechanism captures key target features, reduces background interference, and improves the accuracy and robustness of object detection.

[0004] To effectively address the challenges posed by small object detection tasks, such as poor feature representation and sensitivity to positioning errors, researchers have conducted extensive research and achieved numerous innovative results. However, existing research has focused less on improving positioning accuracy. Therefore, there is still significant room for improvement in research on small-scale object detection methods. Summary of the Invention

[0005] (1) Technical issues to be solved

[0006] The technical problem to be solved by the present invention is: an efficient and convenient small-scale target detection method for remote sensing images, which is used for small-scale target feature extraction and positioning error optimization of remote sensing images, so as to solve the problems of poor small-scale target feature extraction ability and inaccurate target positioning of current mainstream target detection models.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention provides a method for small-scale target detection in remote sensing images based on an edge attention mechanism, comprising the following steps:

[0009] Step 1: Select a suitable small-scale object detection dataset for model parameter tuning based on task requirements;

[0010] Step 2: Construct a small-scale target detection model for remote sensing images. The small-scale target detection model for remote sensing images includes three parts: a backbone network, a neck network, and a detection head network. The backbone network is used to extract global and local features of the input image. The neck network is located between the backbone network and the detection head network, and is used to further process and integrate the features extracted by the backbone network, fuse the multi-scale features extracted by the backbone network to obtain a fused feature map, and pass it to the detection head network to perform the final detection or classification task. The edge-aware enhancement module is integrated in the detection head network to map the feature map information fused by the neck network to the final output space after enhanced perception, and generate the network detection result. The edge-aware enhancement module is used in the P3 feature map branch of small-scale target detection. The edge-aware enhancement module enhances the feature extraction ability and boundary perception ability of the small-scale target detection model of remote sensing images for small-scale targets by establishing a global correlation between row elements and column elements in the remote sensing image feature map. The output parameters of the detection head network are determined according to the target type and positioning frame type in the small-scale target detection dataset.

[0011] Step 3: Use the training set in the small-scale target detection dataset to train the remote sensing image small-scale target detection model. Input the images in the training set into the remote sensing image small-scale target detection model constructed in step 2, obtain the detection result output and calculate the loss function. Select a suitable optimizer to adjust the parameters of the remote sensing image small-scale target detection model according to the loss function.

[0012] Step 4: Use the test set in the small-scale object detection dataset to test and verify the detection performance of the trained remote sensing image small-scale object detection model;

[0013] Step 5: Select the remote sensing image small-scale target detection model with the best test results and deploy it to obtain the final target detection results.

[0014] The present invention also provides a system for implementing the method.

[0015] The present invention also provides an application of the method in complex scene target detection tasks for marine safety, smart transportation or battlefield intelligence support.

[0016] (3) Beneficial effects

[0017] The present invention proposes a novel method for small-scale target detection in remote sensing images. This method designs an edge attention mechanism and an edge perception enhancement module in the detection head network, and realizes that by establishing a global correlation between row elements and column elements in the remote sensing image feature map, the target detection network's feature extraction capability and boundary perception capability for small-scale targets are enhanced, thereby achieving high-precision small-scale target detection in remote sensing images and solving the problem of difficulty in identifying and locating small-scale targets due to feature ambiguity and small positioning error tolerance. By adjusting the output feature parameters of the detection head network, the present invention can be easily transplanted to detection tasks of various target scenes and data sets. The present invention is suitable for target detection tasks in various complex scenarios such as marine safety, smart transportation, and battlefield intelligence support. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is the scale target detection error curve;

[0019] Figure 2 This is a structural diagram of a remote sensing image small-scale target detection model designed in a remote sensing small-scale target detection method based on an edge attention mechanism of the present invention;

[0020] Figure 3 This is the overall structure diagram of the edge perception enhancement module;

[0021] Figure 4 This is a flow chart of a remote sensing small-scale target detection method based on edge attention mechanism of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0023] This paper provides an efficient and convenient method for detecting small-scale targets in remote sensing images. It can be used for feature extraction and positioning error optimization of small-scale targets (targets with a pixel size smaller than 32×32) in remote sensing images. This method addresses the current problems of mainstream target detection models, which suffer from poor feature extraction and inaccurate target positioning for small-scale targets. The paper proposes an edge attention mechanism and an edge perception enhancement module. By establishing global correlations between row and column elements in the remote sensing image feature map, this module strengthens the target detection network's feature extraction and boundary perception capabilities for small-scale targets, achieving high-precision small-scale target detection in remote sensing images.

[0024] This paper follows the YOLOX architecture of the anchor-free target detection model. The detailed framework is as follows Figure 2 Among them, the edge perception enhancement module is integrated into the detection head network, mainly used in the P3 feature map branch for detecting small-scale targets. The specific details of each part of the YOLOX architecture are as follows:

[0025] a) Backbone network: This network is used to extract features from the input data. The backbone network typically contains multiple convolutional layers and other feature extraction layers, primarily used to extract high-level features from the input data. In this method, the backbone network follows the DarkNet-53 network architecture and is primarily used to extract global and local features of the input image, such as edges, shapes, textures, and other implicit feature representations.

[0026] b) Neck network: This network is used to further process and integrate the features extracted by the backbone network. Located between the backbone network and the detection head network, the neck network helps fuse the multi-scale features extracted by the backbone network to generate a fused feature map, which is then passed to the detection head network for final detection or classification. In this method, the neck network adopts an "FPN+PAN" structure.

[0027] c) The detailed structure of the edge perception enhancement module is as follows Figure 3 As shown in the figure, the edge-aware enhancement module is based on the edge attention mechanism. It analyzes the importance of each element (row element and column element) of the feature map to the target boundary positioning from a global level. The edge-aware enhancement module performs self-attention calculation on the target edge information in the feature map after multi-scale feature fusion to achieve enhanced perception. The input of this module is the feature map after multi-scale feature fusion, that is, the feature map transmitted from the neck network to the detection head network.

[0028] The output is a weighted feature map. Edge attention includes two types: horizontal edge attention and vertical edge attention. The calculation formula for horizontal edge attention is:

[0029]

[0030] in, is the feature map of the input image (the feature map after multi-scale feature fusion), H, W, C are the length, width, and height of the feature map of the input image respectively, E H (X) Flatten the feature map of the input image into a feature sequence of length C×H and depth W (E H (X) is a C×H by W-dimensional matrix); W q 、W k and W v All of them are learnable feature matrices, that is, feature matrices with specific parameters can be obtained through model training (parameter optimization). These three feature matrices are linear calculation matrices, with initial random parameters and the final parameters are obtained through model training; d k =C×H.

[0031] The calculation formula for vertical edge attention is:

[0032]

[0033] Among them, through E H ′(X) flattens the feature map of the input image into a feature sequence of length C×W and depth H, d k ′=C×W;

[0034] d) Detection Head Network: This outputs target recognition and localization parameters. The detection head network is the top layer of the entire object detection network. Based on the edge-aware enhancement module, it maps the feature map information fused by the neck network to the final output space after enhanced perception, generating the network's detection results (the detection head network outputs the localization box).

[0035] The target detection method of the present invention is as follows: Figure 4 Each step is described as follows:

[0036] Step 1: Select a suitable small-scale object detection dataset for model parameter tuning based on task requirements;

[0037] Step 2: Construct a small-scale target detection model for remote sensing images. The model consists of a backbone network, a neck network, and a detection head network. The edge-aware enhancement module is integrated into the detection head network and is primarily used in the P3 feature map branch for small target detection. The output parameters of the detection head network are determined based on the target type and positioning frame type in the small-scale target detection dataset.

[0038] Step 3: Use the training set in the small-scale target detection dataset to train the remote sensing image small-scale target detection model. Input the images in the training set into the remote sensing image small-scale target detection model constructed in step 2, obtain the detection result output and calculate the loss function, select a suitable optimizer to adjust the parameters of the remote sensing image small-scale target detection model according to the loss function. The detection results obtained during the training process are used to calculate the difference between the model output and the true value, and then adjust the model parameters according to the difference.

[0039] Step 4: Use the test set in the small-scale object detection dataset to test and verify the detection performance of the trained remote sensing image small-scale object detection model;

[0040] Step 5: Select the remote sensing image small-scale target detection model with the best test results and deploy it to obtain the final target detection results.

[0041] This example uses the Remote Sensing Image Small-Scale Object Detection (AI-TOD) dataset for experimental verification. The AI-TOD dataset contains 700,621 target instances in 8 categories. AP is the most commonly used target detection performance evaluation metric. The closer the evaluation result is to 1, the better the detection performance of the model. vt 、AP t、AP s 、AP m It is the AP index distinguished by target size, where AP vt Indicates size less than or equal to 8 2 The target AP, AP t Indicates size between (8 2 , 16 2 ] target AP, AP s Indicates a size between (16 2 , 32 2 ] target AP, AP m Indicates that the size is greater than 32 2 The experimental results are shown in Table 1, and the optimal indicators are marked in bold.

[0042] Table 1 Experimental results of different target detection models on AI-TOD

[0043]

[0044] First, this method improves the detection performance of the baseline model YOLOX for small-scale targets, AP vt Improved AP by 2.6% t Secondly, compared with some small target detection methods that have achieved the best performance on the AI-TOD dataset, the performance of this method is also more outstanding, AP 50 The accuracy index reached 56.5%, and other indicators also showed significant improvements. Compared with the current mainstream object detection models, this method has a more obvious advantage in detecting small objects. In summary, the effectiveness and advancement of this method have been fully verified.

[0045] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A small-scale target detection method for remote sensing images based on edge attention mechanism, characterized in that: The following steps are involved: Step 1: Select a suitable small-scale object detection dataset for model parameter tuning based on task requirements; Step 2: Construct a small-scale target detection model for remote sensing images. The small-scale target detection model for remote sensing images includes three parts: a backbone network, a neck network, and a detection head network. The backbone network is used to extract global and local features of the input image. The neck network is located between the backbone network and the detection head network, and is used to further process and integrate the features extracted by the backbone network, fuse the multi-scale features extracted by the backbone network to obtain a fused feature map, and pass it to the detection head network to perform the final detection or classification task. The edge-aware enhancement module is integrated in the detection head network to map the feature map information fused by the neck network to the final output space after enhanced perception, and generate the network detection result. The edge-aware enhancement module is used in the P3 feature map branch of small-scale target detection. The edge-aware enhancement module enhances the feature extraction ability and boundary perception ability of the small-scale target detection model of remote sensing images for small-scale targets by establishing a global correlation between row elements and column elements in the remote sensing image feature map. The output parameters of the detection head network are determined according to the target type and positioning frame type in the small-scale target detection dataset. Step 3: Use the training set in the small-scale target detection dataset to train the remote sensing image small-scale target detection model. Input the images in the training set into the remote sensing image small-scale target detection model constructed in step 2, obtain the detection result output and calculate the loss function. Select a suitable optimizer to adjust the parameters of the remote sensing image small-scale target detection model according to the loss function. Step 4: Use the test set in the small-scale object detection dataset to test and verify the detection performance of the trained remote sensing image small-scale object detection model; Step 5: Select the remote sensing image small-scale target detection model with the best test results and deploy it to obtain the final target detection results.

2. The method according to claim 1, wherein The edge perception enhancement module is based on the edge attention mechanism and analyzes the importance of each element of the feature map to the target boundary positioning from a global level. The edge perception enhancement module performs self-attention calculation on the target edge information in the feature map after multi-scale feature fusion to achieve enhanced perception. The input of the edge perception enhancement module is the feature map after multi-scale feature fusion, that is, the feature map transmitted from the neck network to the detection head network; the output is the weighted feature map; the edge attention calculated by the edge perception enhancement module includes two types: horizontal edge attention and vertical edge attention. The calculation formula of horizontal edge attention is: in, is the feature map of the input image, H, W, and C are the length, width, and height of the feature map of the input image, respectively. H (X) Flatten the feature map of the input image into a feature sequence of length C×H and depth W; W q 、W k and W v are all learnable feature matrices; d k =C×H; The calculation formula for vertical edge attention is: Among them, through E H ′(X) flattens the feature map of the input image into a feature sequence of length C×W and depth H, d k ′=C×W.

3. The method according to claim 1, wherein The remote sensing image small-scale target detection model follows the YOLOX architecture of the anchor-free target detection model.

4. The method according to claim 1, wherein The backbone network follows the DarkNet-53 network architecture.

5. The method according to claim 1, wherein The features extracted by the backbone network include edge, shape and texture features.

6. The method according to claim 1, wherein W q 、W k and W v It is the feature matrix of specific parameters obtained through model training. These three feature matrices are linear calculation matrices. The initial parameters are random and the final parameters are obtained through model training.

7. The method according to claim 1, wherein The small-scale target detection dataset is the AI-TOD dataset.

8. The method according to claim 1, wherein The small-scale target is a target with a pixel size smaller than 32×32.

9. A system for implementing the method according to any one of claims 1 to 9.

10. Application of the method according to any one of claims 1 to 9 in complex scene target detection tasks for marine safety, smart transportation or battlefield intelligence support.

Images