A method for directional target detection in remote sensing images based on multi-feature aggregation and interaction

Abstract

The present invention discloses a method for directional target detection in remote sensing images based on multi-feature aggregation and interaction, the method comprising: obtaining a high-resolution remote sensing image, cutting it into a small image and sending it into a network. After feature extraction, multi-feature aggregation and interaction are performed. First, a shallow feature fusion module and an attention module are used to obtain local features and global information. Then, a multi-scale attention feature map is obtained using a bidirectional fusion feature pyramid network and a convolutional attention mechanism module. Finally, the feature map containing important information obtained by the cross-scale fusion module is sent to the detection head to obtain the target category and the directional bounding box. The detection result of the small image is mapped back to the original image and invalid detection results are filtered. This method can solve the problems that the target direction in the remote sensing image is arbitrary, the change amplitude is large, the amount of information is small, and it is easily disturbed by complex background. This method has superior performance in directional target detection in remote sensing images.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology based on deep learning, and in particular relates to a remote sensing image directional target detection method based on multi-feature aggregation and interaction. Background Art

[0002] Remote sensing images can collect a large amount of ground information efficiently and economically. Automatic interpretation of remote sensing images can screen important information in a short time. In these analyses, multi-category target detection plays an important role. Multi-category target detection can quickly locate and classify various objects in images (such as ships, aircraft, vehicles, buildings, etc.). It plays an important role in practical applications such as geological exploration, infrastructure inspection, urban planning, national defense construction, and maritime search and rescue. In recent years, the target detection task of remote sensing images has attracted more and more research interests.

[0003] Unlike natural images, which are usually taken from a horizontal angle, remote sensing images are usually taken from a bird's-eye view, which means that objects are often distributed in different directions. If horizontal box target detection is used, the detected horizontal bounding box will not match the actual target closely. Therefore, the directional surrounding rotating box target detection algorithm is the preferred method for capturing remote sensing targets. At present, the rotating box target detection algorithm is mainly based on the anchor box detector. First, the anchor box is densely distributed on the feature map, and then the parameter offset between the target box and the anchor box is regressed. This training method is easy for the network to converge. However, the parameters of the anchor box need to be manually pre-set, and different scenes need to be adjusted, which depends on certain prior knowledge. Although the anchor box can be automatically determined by methods such as K-means clustering, a few objects with rare aspect ratios are difficult to match with the anchor box, resulting in missed detection of such objects. In addition, a picture may have only a small number of objects, but a large number of anchor boxes need to be preset, which is easy to cause an extreme imbalance in the number of positive and negative anchor boxes, resulting in poor training results.

[0004] Directed detectors based on key points are another method for detecting rotated box objects. This type of method does not require the introduction of anchor boxes and can directly predict the corners or center points of the object. Then the width w and height h of the bounding box are regressed. Finally, a new angle parameter θ is added to perform rotated box regression. Detectors based on key point detection usually need to jointly learn w and h for each arbitrary direction target in the Cartesian coordinate system, which is difficult to train. The BBAVector algorithm provides a solution to obtain the bounding box of the rotated target by learning the box boundary perception vector. The excellent multi-head detection network design enables the model to have good detection speed and accuracy. However, the potential of this algorithm has not been fully explored, and the simple local feature fusion network cannot achieve good interaction and aggregation of shallow location information and deep semantic information. Since remote sensing images cover a large area, there is usually a lot of complex ground object information. Various types of targets are mixed in complex information, which is likely to cause serious interference to classification and regression. Summary of the invention

[0005] In view of the problems existing in the prior art methods, the present invention proposes a remote sensing image directional target detection method based on multi-feature aggregation and interaction. The remote sensing image directional target detection method of the present invention is a recognition system combining a convolutional neural network and a Transformer.

[0006] The technical solutions adopted are:

[0007] A remote sensing image directional target detection method, comprising:

[0008] Acquire high-resolution remote sensing images;

[0009] Crop the remote sensing image into multiple small images of preset sizes;

[0010] Inputting the cropped small image into a directional target detection model to obtain an output directional target detection result; wherein the directional target detection model includes a feature extraction module, multi-feature aggregation and interaction (shallow feature fusion module, self-attention module, bidirectional interactive feature pyramid network module, convolutional attention mechanism module, cross-scale fusion module) and a detection head;

[0011] The feature extraction module is configured to: extract features from the cropped small image to obtain five layers of feature maps of different scales, where the scales of the feature maps are as follows from large to small: feature Figure 1 ,feature Figure 2 ,feature Figure 3 ,feature Figure 4 ,feature Figure 5 ;

[0012] The shallow feature fusion module is configured to: Figure 1Input shallow feature fusion module to process and get shallow feature 1 as output. Figure 2 The shallow feature fusion map is obtained by fusion;

[0013] The self-attention module is configured to: Figure 5 Input the self-attention module to get the output attention feature map;

[0014] The bidirectional interactive feature pyramid network module is configured to: fusion graph of shallow features, feature Figure 3 ,feature Figure 4 The attention feature map is input into the bidirectional interactive feature pyramid network to perform multi-scale feature interaction to obtain a multi-scale interactive feature map;

[0015] The convolutional attention mechanism module is configured to: input the multi-scale interaction feature map into the convolutional attention mechanism module to extract the attention area to obtain the multi-scale attention feature map;

[0016] The cross-scale fusion module is configured to: input the multi-scale attention feature map into the cross-scale fusion module for cross-scale interaction to obtain a cross-scale fused feature map;

[0017] The detection head is configured to: input the cross-scale fused feature map into the detection head for directional detection, and obtain the output target category and directional bounding box, that is, the detection result of the small image; in response to the completion of the detection of all cropped small images, the detection result of the small image is mapped back to the original remote sensing image, and through non-maximum suppression, the target with the same position, the same category and the highest confidence is retained as the final detection result.

[0018] In some embodiments, the remote sensing image is cropped into a plurality of small images of preset sizes, including:

[0019] The remote sensing image is cropped with an overlap ratio, and the scale of the cropped image is controlled by the overlap ratio coefficient to achieve multi-scale cropping.

[0020] In some embodiments, the feature extraction module is implemented by a ResNet network, and five layers of feature maps of different scales are obtained through the ResNet network.

[0021] Furthermore, the feature extraction module uses ResNet101, ResNet 18, ResNet34 or ResNet50 network as the feature extraction network.

[0022] In some embodiments, the shallow feature fusion module fuses the feature maps of the two largest scales. The shallow feature fusion first acts on the shallowest features, expands the receptive field by connecting convolution kernels of different sizes in parallel, extracts features from different scales through spatial pyramid pooling, and then aggregates the feature information. The aggregated feature map is fused with the feature map of the previous layer to obtain a shallow fusion feature map.

[0023] In some embodiments, the self-attention module includes multiple self-attention and fully connected layers, and each layer is connected by a residual block; the self-attention module processes the deepest feature map to construct a global relationship between the feature maps.

[0024] In some embodiments, the specific processing steps of the bidirectional interactive feature pyramid network module include: first, from top to bottom, the deep features output by the shallow feature fusion module are fused with the upper layer information layer by layer, and then from bottom to top, the shallowest layer information is passed back to the deep layer features; a short circuit connection is established between the input and output of the same layer.

[0025] In some embodiments, the detection head is divided into four parts, namely: Heatmap responsible for classification and center point positioning; Offset for calculating bias loss; Box Param for capturing directional regression box parameters; Orientation for divide-and-conquer angle problems.

[0026] Furthermore, the Heatmap uses the Heatmap value as the detection confidence score, obtains the Heatmap through a 3×3 maximum pooling, and uses NMS on the Heatmap to obtain the K center points with the largest values.

[0027] In some embodiments, the data set of the directional target detection model is divided into a training set, a validation set and a test set; the training set and the validation set contain images and corresponding label files; the label file contains the coordinates and categories of the four corner points corresponding to each target in the image.

[0028] The detection process of this method is: input high-resolution remote sensing images, image cropping, feature extraction, feature fusion, directional detection head detection, image stitching and output detection results.

[0029] The input high-resolution remote sensing images have a large size span. Taking the DOTA dataset as an example, the dataset contains 2806 remote sensing images, and the image size spans from 400×400 to 12000×5000. If the high-resolution image is directly adjusted to a preset small-scale length and width (for example, 608×608) without cropping and sent to the network for detection, most small and medium-sized targets will be missed. The present invention will crop the high-resolution remote sensing images with overlap (see the specific implementation method for details) before inputting them into the network for training and testing.

[0030] The ResNet101 network used for feature extraction is widely used in mainstream target detection backbone networks. The present invention uses ResNet101 for feature extraction and outputs five layers of feature maps of different scales.

[0031] The present invention focuses on improving feature fusion. By designing a multi-feature aggregation and interaction module, the shallow position information and deep semantic information of the feature map are fully integrated. The overall design is: using the shallow feature fusion module to fuse shallow information, so that the network can learn more features. Generally speaking, most feature fusion schemes establish feature pyramid models with more than three layers. Fusion of shallower features means that more shallow information can be retained, and the probability of capturing important information is greater. The detection results of this method are often significantly improved, but the computational cost is also greatly increased. In order to minimize the amount of calculation while retaining shallower features and improving the accuracy of target detection, we use a shallow feature fusion method, which allows the network to learn more features. This means that more sufficient local information can be obtained, alleviating the problems of information imbalance and feature dislocation. The self-attention module is used to process the deep feature map output by the feature extraction module. The feature map obtained at this position has the smallest size and rich semantic information. The multi-head self-attention module can capture global relationships and rich contextual information, which is very important for understanding the overall information of the image. On the other hand, the computational cost of the multi-head self-attention module increases sharply with the increase in the size of the input feature map, so the amount of calculation processed on the low-resolution feature map is small. Adding a multi-head self-attention module at this position can quickly build a global model with low computational cost. A bidirectional interactive feature pyramid network is designed to perform multi-scale feature interaction on the feature maps output by the feature extraction module, the shallow feature fusion module and the multi-head self-attention module. First, from top to bottom, the deep features output by the shallow feature fusion module are fused with the upper layer information layer by layer, and then from bottom to top, the shallowest layer information is passed back to the deep features. In order to better retain the information transferred, a short-circuit connection is established between the input and output of the same layer. The generated feature map contains more significant differential features and improves the description ability of various objects. In each layer of the fused feature pyramid network, a simple and powerful lightweight convolutional attention mechanism module is added to find the attention area with dense target scenes. The feature map obtained by the output of the fused feature pyramid network has rich information. However, this information also contains more confusion areas. The convolutional attention mechanism module can further extract the attention area to assist the network to better filter the complex background information, highlight the important target objects, and generate the attention feature map. The cross-scale fusion module can realize cross-scale interaction on the multi-scale attention feature map to obtain more obvious and balanced features.

[0032] The directional detection head uses BBAVector. It is divided into four parts: Heatmap, Offset, Box Param, and Orientation. Heatmap is responsible for the center point of the positioning box and the classification function. It can detect the center point of the oriented target and calculate the heatmap value as the confidence of the target detection. Offset is the bias loss of the target, which is used to deal with the accuracy loss of the downsampled feature map and the feature map mapped back to the normal scale. In order to capture the bounding box of the oriented target, the traditional method is to detect the width w, height h, angle θ and center point of the target. However, joint learning of so many parameters has caused great difficulties for the network. Therefore, four box boundary perception vectors are set up in the four quadrants of the Cartesian coordinate system, namely the upper left t, upper right r, lower right b and lower left l, and the rotation box is described by these four vectors belonging to different coordinate systems. Since all arbitrarily oriented targets have the same coordinate system, information transmission is relatively easy and the difficulty of model learning is reduced. In this way, the parameters of the BoxParam used for the bounding box are {r, t, l, b, w, h}, where w and h are the width and height of the target bounding box, respectively. Due to the division of four quadrants, when an object is surrounded by a horizontal box in the image, the box boundary perception vector will become a coordinate axis distribution, making it impossible for the detector to make a direct prediction. Orientation can solve this problem well. The role of Orientation is to divide the target into two categories, HBB and RBB. Among them, RBB is all directional bounding boxes except horizontal bounding boxes. The triggering method of Orientation is:

[0033]

[0034] IoU is the intersection of the directional bounding box (OBB) and the horizontal bounding box (HBB). α is the coefficient that determines whether to use RBB or HBB. In this way, when encountering a horizontal box target, it can be converted to the way of processing the horizontal box HBB.

[0035] After the detection is completed, all sub-images are mapped back to the corresponding positions of the original image. At this time, there are repeated detection targets in the overlapping area, and non-maximum suppression operations are required to retain the targets with the same position, the same category, and the highest confidence.

[0036] The method of the invention can solve the problems that the target direction in the remote sensing image is arbitrary, the change range is large, the amount of information is small, and it is easily disturbed by the complex background. The method has excellent performance in the detection of directional targets in remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a directional target detection model diagram of the present invention;

[0038] Figure 2 It is a schematic diagram of the shallow feature fusion module;

[0039] Figure 3 It is a bidirectional interactive feature pyramid network graph;

[0040] Figure 4 It is a comparison chart of input and output of each module;

[0041] Figure 5 This is a graph of experimental test results. DETAILED DESCRIPTION

[0042] The model and operation flow of the present invention are further described below in conjunction with the accompanying drawings and implementation examples.

[0043] A remote sensing image directional target detection method, comprising:

[0044] Acquire high-resolution remote sensing images;

[0045] Crop the remote sensing image into multiple small images of preset sizes;

[0046] Inputting the cropped small image into a directional target detection model to obtain an output directional target detection result; wherein the directional target detection model includes a feature extraction module, multi-feature aggregation and interaction (shallow feature fusion module, self-attention module, bidirectional interactive feature pyramid network module, convolutional attention mechanism module, cross-scale fusion module) and a detection head;

[0047] The feature extraction module is configured to: extract features from the cropped small image to obtain five layers of feature maps of different scales, where the scales of the feature maps are as follows from large to small: feature Figure 1 ,feature Figure 2 ,feature Figure 3 ,feature Figure 4 ,feature Figure 5 ;

[0048] The shallow feature fusion module is configured to: Figure 1 Input shallow feature fusion module to process and get shallow feature 1 as output. Figure 2 The shallow feature fusion map is obtained by fusion;

[0049] The self-attention module is configured to: Figure 5 Input the self-attention module to get the output attention feature map;

[0050] The bidirectional interactive feature pyramid network module is configured to: fusion graph of shallow features, feature Figure 3 ,feature Figure 4 The attention feature map is input into the bidirectional interactive feature pyramid network to perform multi-scale feature interaction to obtain a multi-scale interactive feature map;

[0051] The convolutional attention mechanism module is configured to: input the multi-scale interaction feature map into the convolutional attention mechanism module to extract the attention area to obtain the multi-scale attention feature map;

[0052] The cross-scale fusion module is configured to: input the multi-scale attention feature map into the cross-scale fusion module for cross-scale interaction to obtain a cross-scale fused feature map;

[0053] The detection head is configured to: input the cross-scale fused feature map into the detection head for directional detection, and obtain the output target category and directional bounding box, that is, the detection result of the small image; in response to the completion of the detection of all cropped small images, the detection result of the small image is mapped back to the original remote sensing image, and through non-maximum suppression, the target with the same position, the same category and the highest confidence is retained as the final detection result.

[0054] In an example of the present invention, the data set is divided into a training set, a validation set, and a test set. The training set and the validation set contain images and corresponding label files. The label file corresponding to each image records the corner point coordinates and categories corresponding to each type of target in the image. The corner point coordinates here refer to the x-axis and y-axis coordinates of each corner point of the directional target box, that is, each target contains 8 coordinate information and one category information. This information is used to train multiple batches in the training set to obtain a weight file. The evaluation index is obtained by testing the validation set and comparing it with the real label. The present invention uses the average accuracy as the evaluation index, and the higher the average accuracy, the better the model performance. Select the weight file that performs best in the validation set and test it in the test set.

[0055] Before the high-resolution remote sensing images are input into the training network, overlapping cropping is performed. Specifically, the original images of the data set are cropped into small images, each of which is 600×600 in size and has an overlapping area of ​​100. The scale of the cropped images is controlled by coefficients 1 and 0.5 to obtain the training set, validation set, and test set, respectively. The training set and validation set are used to feed the network for training to obtain the weight file and the test set detection results. While the training set and validation set images are cropped, the label file also updates the coordinate position of the target. Each label file will mark the relative position of the corresponding cropped small image in the original high-resolution image, so as to map the detection results back to the corresponding area of ​​the original image.

[0056] like Figure 1 The figure is a model diagram for directional target detection. During the training and testing phases, the input image with a width and height of w×h is resized to 608×608 and then fed into the network. The network uses stochastic gradient descent (SGD) to optimize the loss function, with an initial learning rate of 1.25×10 -4The backbone network of the network uses ResNet101 to extract features and obtain five layers of feature maps of different sizes. Figure 1 will be sent to the shallow feature fusion module for processing. The processing flow is as follows Figure 2 Dimensions are Features Figure 1 It is fed into a parallel convolution operation, where the parallel convolution kernel sizes are 5×5 and 7×7 respectively. Then, the results obtained from the parallel structure are fused. Finally, the fused results are fed into four parallel processes, namely, the maximum pooling with window sizes of 5, 9, and 13, and the residual block that does not perform any operation. The result of shallow feature fusion will be combined with the feature Figure 2 Fusion Figure 3 The shallow feature fusion diagram shown. The smallest feature Figure 5 It will be sent to the multi-head self-attention module to calculate the spatial relationship of the image in the average distance. When data x is input, M weighted feature matrices will be obtained through M different self-attention structures. Then, these feature matrices are concatenated to obtain matrix Z. Finally, matrix Z passes through a fully connected layer to obtain the self-attention feature map of the output result.

[0057] By fusing shallow features, features Figure 3 ,feature Figure 4 The multi-scale feature map composed of the attention feature map will be fed into the bidirectional interactive feature pyramid network, such as Figure 3 The generated multi-scale interactive feature maps are each fed into the convolutional attention mechanism module, which i The channel and spatial attention modules are applied sequentially to obtain weighted attention maps from the channel and spatial dimensions respectively. Then, the attention map is combined with P i Multiplication achieves adaptive optimization. The process of the convolutional attention mechanism module is:

[0058]

[0059]

[0060] P i It is the feature map of the output of the i-th layer of the fusion feature pyramid network and the input feature map of the convolutional attention mechanism module in the same layer. c It is the process of calculating channel attention. s It is the inference process of spatial attention. i is the i-th feature map output by the convolutional attention mechanism module. The convolutional attention mechanism module can further extract the attention area to assist the network to better filter complex background information and highlight important target objects.

[0061] After passing through the convolutional attention mechanism module, the feature maps of four layers at different scales have a lot of important information. However, this information is not balanced. The cross-scale fusion scheme can obtain more obvious and balanced features. 3 It corresponds to the smallest feature map, which is upsampled eight times to be adjusted to A 0 The same size. Similarly, A 2 Upsampling four times, A 1 Finally, all upsampling results are combined with A 4 Feature map fusion generates a cross-scale fused feature map P.

[0062] The process of the cross-scale fusion module is as follows:

[0063]

[0064] A i is the i-th feature map output by the convolutional attention mechanism, Upsample is upsampling, and the upsampling multiple is 2 i OK, Concat is to connect the multi-scale upsampling results.

[0065] Figure 4 The figure shows a picture of the DOTA dataset as input (the target is marked with a red frame), and the visualization results of the input and output feature maps of the shallow feature fusion module, the self-attention module, the bidirectional interactive feature pyramid, the convolutional attention mechanism module, and the cross-scale fusion module. It should be noted that the input and output features of each module are separated by a dotted line. The upper half of the dotted line is the input feature map, and the lower half is the output feature map. These feature maps reflect the confidence scores of the target and the background respectively. The higher the confidence, the brighter the corresponding position of the feature map. As can be seen from the changes in the feature map, the shallow feature fusion module successfully filters the background information such as the lawn and lane lines, and retains the potential important information. The self-attention module establishes a global relationship, and the information of the feature map becomes abstract. The bidirectional interactive feature pyramid further filters the noise information. It can be observed that most of the bright spots in the image are located at the position of the real target, but the spots are still small. After passing through the convolutional attention mechanism module, by further extracting the attention area, the position corresponding to the important information is gradually enlarged, and basically accurately covers the positions corresponding to all real targets. Finally, after cross-scale fusion, all bright spots in the image are the positions corresponding to the real targets. Figure 5 Contains the detection results of this image.

[0066] The feature map P will be sent to the detection head for directional detection. The four parts of the detection head, Heatmap is responsible for locating the center point of the target and calculating the confidence of the target category. Offset calculates the precision loss of downsampling and mapping. Box Param sets up four box boundary perception vectors in the four quadrants of the Cartesian coordinate system to help information transmission and reduce the difficulty of model learning. Orientation is responsible for processing the horizontal box and the directional box respectively.

[0067] After all the cropped small images are detected, the detection results are mapped back to the original image. Through non-maximum suppression, the targets with the same position, the same category, and the highest confidence are retained as the final detection results.

[0068] In order to verify the effectiveness of the application example, the present invention conducted a comparative experiment using the DOTA and HRSC2016 datasets as examples.

[0069] The experiment is based on the pytorch1.8.1 deep learning framework, and the device operating system is Red Hat 8.4.1-1. The hardware configuration is Intel(R)Xeon(R)Gold 5218R CPU and two NVIDIA GeForce RTX3090 GPUs (24G video memory).

[0070] The commonly used evaluation indicators in the field of target detection are the average precision (AP) of a single category target and the average precision (mAP) of all categories. The higher the AP value and mAP value, the better the model performance.

[0071] The DOTA dataset has many types of targets and dense small targets. The present invention is an end-to-end single-stage algorithm. Without the use of test data augmentation (TTA) and other data preprocessing data augmentation schemes, the detection results achieved a mAP score of 75.19% in the evaluation results of the DOTA official server. While ensuring high accuracy, the present invention also has a fast detection speed. On the RTX3090 graphics card, the image inference speed of the algorithm reaches 23.22fps, which is twice that of ROI Trans. The experimental results verify the efficiency and accuracy of the algorithm.

[0072] Table 1 shows the detection comparison results of the method in the embodiment of the present application and other methods on the DOTA dataset test set.

[0073]

[0074] The HRSC2016 dataset is a remote sensing dataset specially collected for ship detection, containing 1061 ship images. The resolutions of these images range from 400×400 to 1500×900.

[0075] Table 2 shows the detection comparison results of the method of the embodiment of the present application and other methods on the HRSC2016 dataset test set.

[0076] method BL2 <![CDATA[R 2 PN]]> RRD ROI Trans Gliding Vertex BBAVectors Ours mAP(%) 69.6 79.6 84.3 86.2 88.20 88.39 89.50

[0077] Table 3 shows the ablation experiment performed on the HRSC2016 dataset by the method of the embodiment of the present application, and the evaluation index is mAP when the intersection over union (IoU) is 0.5.

[0078] experiment Bidirectional interactive feature pyramid Convolutional Attention + Self-Attention Shallow feature fusion mAP0.5 A - - - 88.39 B √ - - 89.17 C √ √ - 89.38 D √ √ √ 89.50

[0079] After adding the bidirectional interactive feature pyramid, the model's mAP increased by 0.78%, convolutional attention and self-attention increased the model's AP index by 0.21%, and shallow feature fusion increased the model's AP index by 0.12%.

[0080] Since the same detection head is used as the BBAVector algorithm, the method in this application example and BBAVector also compare the AP index at the same IoU detection results. As shown in the following table:

[0081] method mAP0.5 mAP0.6 mAP0.7 mAP0.8 mAP0.9 BBAVector 88.39 78.87 64.62 32.61 9.09 OURS 89.50 88.55 75.76 39.92 9.09

[0082] Through the description of the above implementation method, those skilled in the art can clearly understand the technical process of the present invention. The above implementation method relies on software and necessary general hardware platforms for implementation. The present invention does not use pre-processing and post-processing techniques. If pre-processing schemes such as mosaic data enhancement, Mixup data enhancement or TTA data enhancement post-processing methods are adopted, the detection effect of the present invention will be further improved. For the hardware platform, the experiment of this implementation method uses RTX3090. If Quadro RTX6000GPUs or better graphics cards are used, the detection results of the present invention will be further improved.

[0083] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0084] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0087] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting directional targets in remote sensing images, characterized in that: include: Acquire high-resolution remote sensing images; Crop the remote sensing image into multiple small images of preset sizes; Inputting the cropped small image into a directional target detection model to obtain an output directional target detection result; wherein the directional target detection model includes a feature extraction module, a shallow feature fusion module, a self-attention module, a bidirectional interactive feature pyramid network module, a convolutional attention mechanism module, a cross-scale fusion module and a detection head; The feature extraction module is configured to: extract features from the cropped small image to obtain five layers of feature maps of different scales, wherein the scales of the feature maps are from large to small: feature map 1, feature map 2, feature map 3, feature map 4, and feature map 5; The shallow feature fusion module is configured to: input feature map 1 into the shallow feature fusion module for processing to obtain output shallow feature 1, and fuse shallow feature 1 with feature map 2 to obtain a shallow feature fusion map; The self-attention module is configured to: input the feature map 5 into the self-attention module to obtain an output attention feature map; The bidirectional interactive feature pyramid network module is configured to: input the shallow feature fusion map, feature map 3, feature map 4 and attention feature map into the bidirectional interactive feature pyramid network to perform multi-scale feature interaction to obtain a multi-scale interactive feature map; The convolutional attention mechanism module is configured to: input the multi-scale interaction feature map into the convolutional attention mechanism module to extract the attention area to obtain the multi-scale attention feature map; The cross-scale fusion module is configured to: input the multi-scale attention feature map into the cross-scale fusion module for cross-scale interaction to obtain a cross-scale fused feature map; The detection head is configured to: input the cross-scale fused feature map into the detection head for directional detection, and obtain the output target category and directional bounding box, that is, the detection result of the small image; in response to the completion of the detection of all cropped small images, the detection result of the small image is mapped back to the original remote sensing image, and through non-maximum suppression, the target with the same position, the same category and the highest confidence is retained as the final detection result.

2. The method according to claim 1, characterized in that Crop remote sensing images into multiple small images of preset sizes, including: The remote sensing image is cropped with an overlap ratio, and the scale of the cropped image is controlled by the overlap ratio coefficient to achieve multi-scale cropping.

3. The method according to claim 1, characterized in that The feature extraction module is implemented by the ResNet network, through which five layers of feature maps of different scales are obtained.

4. The method according to claim 3, characterized in that The feature extraction module uses ResNet101, ResNet18, ResNet34 or ResNet50 network as the feature extraction network.

5. The method according to claim 1, characterized in that The shallow feature fusion module fuses the feature maps of the two largest scales. The shallow feature fusion first acts on the shallowest features, expands the receptive field by connecting convolution kernels of different sizes in parallel, extracts features from different scales through spatial pyramid pooling, and then aggregates the feature information. The aggregated feature map is fused with the feature map of the previous layer to obtain a shallow fusion feature map.

6. The method according to claim 1, characterized in that The self-attention module includes multi-head self-attention and fully connected layers, and each layer is connected by residual blocks; the self-attention module processes the deepest feature map to construct the global relationship of the feature map.

7. The method according to claim 1, characterized in that The specific processing steps of the bidirectional interactive feature pyramid network module include: first, from top to bottom, the deep features output by the shallow feature fusion module are fused with the upper layer information layer by layer, and then from bottom to top, the shallowest layer information is transmitted back to the deep layer features; a short circuit connection is established between the input and output of the same layer.

8. The method according to claim 1, characterized in that: The detection head is divided into four parts: Heatmap for classification and center point positioning; Offset for calculating bias loss; Box Param for capturing directional regression box parameters; and Orientation for divide-and-conquer angle problems.

9. The method according to claim 8, characterized in that The Heatmap uses the Heatmap value as the detection confidence score, obtains the Heatmap through a 3×3 maximum pooling, and uses NMS on the Heatmap to obtain the K center points with the largest values.

10. The method according to claim 1, characterized in that The dataset of the directional target detection model is divided into training set, validation set and test set; the training set and validation set contain images and corresponding label files; the label file contains the coordinates and categories of the four corner points corresponding to each target in the image.

Images