A rotating small target detection system and method in remote sensing scenes

Abstract

The present invention discloses a system and method for detecting rotating small targets in remote sensing scenarios. The system comprises an image acquisition module and a target detection module. The image acquisition module is used to acquire images of the remote sensing scene to be detected. The target detection module is used to improve the classification and regression detection head structure and construct a target detection network based on the improved classification and regression detection head structure. The target detection module is also used to detect rotating small targets in the remote sensing scene based on the target detection network. The target detection algorithm provided by the present invention is applicable to both single-stage and dual-stage target detection, significantly improving the detection head's classification performance and target regression accuracy while slightly increasing computational complexity.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a rotating small target detection system and method in a remote sensing scene. Background Art

[0002] Remote sensing scenarios often contain objects of varying scales and shapes, requiring the detector head to capture global features to locate their complex boundaries. Common detection heads typically consist of a shared convolutional branch, consisting of multiple stacked convolutional blocks, used to learn shared information within features. This branch is followed by two classification and regression branches, also stacked with multiple convolutional blocks and a multi-layer perceptron (MLP), to perform bounding box position coordinate regression and target classification. However, common parallel heads are limited by the characteristics of convolution and their parallel structure: convolution samples the input feature map at a fixed sampling location, while the parallel structure independently performs the two tasks, lacking cross-task perception and cross-task interaction. Consequently, they have limitations in deformation and global perception. Summary of the Invention

[0003] The present invention aims to address the deficiencies of the existing technology and proposes a rotating small target detection system and method in remote sensing scenarios, which is used to solve the problem that traditional detection head modules lack deformation perception, global perception and self-attention perception.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A rotating small target detection system in a remote sensing scene includes: an image acquisition module and a target detection module;

[0006] The image acquisition module is used to acquire the remote sensing scene image to be detected;

[0007] The target detection module is used to improve the classification and regression detection head structure and build a target detection network based on the improved classification and regression detection head structure;

[0008] The target detection module is further configured to detect a rotating small target in the remote sensing scene based on the target detection network.

[0009] Preferably, the target detection network includes: a backbone network submodule, a feature pyramid submodule, a region proposal network submodule and an improved classification regression detection head structure;

[0010] The backbone network submodule is used to extract image features from the remote sensing scene image;

[0011] The feature pyramid submodule is used to fuse the image features to obtain a fused feature image;

[0012] The region proposal network submodule is used to filter out the region of interest from the fused feature image;

[0013] The improved classification regression detection head structure is used to detect the rotating small target in the region of interest.

[0014] Preferably, the improved classification regression detection head structure includes: a deformable convolutional v3 network, a cascaded serial attention network and a cross attention network;

[0015] The deformable convolutional V3 network is used to enhance the scale information of the region of interest to obtain a feature map;

[0016] The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of the branch vectors respectively, obtain three corresponding feature vectors, and fuse the three feature vectors to obtain a global enhancement vector;

[0017] The cross attention network is used to perform feature compression on the global enhancement vector to obtain a classification feature vector and a positioning feature vector, and combine the classification feature vector and the positioning feature vector in a cross attention form to obtain the type and position of the rotated small target.

[0018] The present invention also provides a method for detecting a rotating small target in a remote sensing scenario, which is applied to any of the above-mentioned detection systems and includes the following steps:

[0019] Acquire a remote sensing scene image to be detected;

[0020] Improve the classification and regression detection head structure, and build the target detection network based on the improved classification and regression detection head structure;

[0021] The rotating small target in the remote sensing scene is detected based on the target detection network.

[0022] Preferably, the target detection network includes: a backbone network, a feature pyramid, a region proposal network and an improved classification regression detection head structure;

[0023] The backbone network is used to extract image features from the remote sensing scene image;

[0024] The feature pyramid is used to fuse the image features to obtain a fused feature image;

[0025] The region proposal network is used to screen out a region of interest from the fused feature image;

[0026] The improved classification regression detection head structure is used to detect the rotating small target in the region of interest.

[0027] Preferably, the improved classification regression detection head structure includes: a deformable convolutional v3 network, a cascaded serial attention network and a cross attention network;

[0028] The deformable convolutional V3 network is used to enhance the scale information of the region of interest to obtain a feature map;

[0029] The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of the branch vectors respectively, obtain three corresponding feature vectors, and fuse the three feature vectors to obtain a global enhancement vector;

[0030] The cross attention network is used to perform feature compression on the global enhancement vector to obtain a classification feature vector and a positioning feature vector, and combine the classification feature vector and the positioning feature vector in a cross attention form to obtain the type and position of the rotated small target.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) The present invention combines deformation perception, global perception, and cross-task perception to fuse deformable information, global information, and local information to improve the detection head in the object detection network;

[0033] (2) With the help of deformable convolution, the model of the present invention can better capture long-range semantic information. Even if the position or shape of the target object in the image changes, the network can still accurately identify and locate the target, which helps to reduce background interference and enables the network to focus more on the salient feature information of the target object, thereby improving the accuracy and robustness of target detection;

[0034] (3) The cascaded serial attention mechanism of the present invention is used in the global perception module, the purpose of which is to fuse the feature information of multiple channels and the global feature information while ensuring the computational complexity; the improved model can not only utilize the feature information of each channel, but also integrate the global context information, so as to more comprehensively understand the image content. By focusing on the key position coordinates of each channel, the global perception module can help the model locate the target or key feature points more accurately, thereby improving the regression accuracy of the model;

[0035] (4) The present invention improves the cross-attention mechanism of the model, so that the model can combine the focus on target positioning in actual scenarios with the accurate classification of corresponding detection frames; the improved model can not only learn the positioning information such as the position and shape of the target in the image, but also pay attention to the category information to which the target belongs, thereby achieving more comprehensive and accurate target detection and classification.

[0036] (5) The present invention provides an algorithm for target detection that is suitable for both single-stage and dual-stage detection, which greatly improves the classification effect and target regression accuracy of the detection head while slightly increasing the computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 Schematic diagram of the system structure of an embodiment of the present invention;

[0039] Figure 2 Schematic diagrams of the classification and regression detection head structures before and after improvement according to an embodiment of the present invention, wherein a is the classification and regression detection head structure before improvement, and b is the classification and regression detection head structure after improvement;

[0040] Figure 3 Schematic diagram of the cascaded serial attention network structure of an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the cross-attention network structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] In this embodiment, if Figure 1 As shown, a rotating small target detection system in a remote sensing scene includes: an image acquisition module and a target detection module.

[0046] The image acquisition module is used to acquire the remote sensing scene image to be detected.

[0047] The target detection module is used to improve the classification and regression detection head structure, build a target detection network based on the improved classification and regression detection head structure, and detect small rotating targets in remote sensing scenes based on the target detection network.

[0048] The target detection network includes: backbone network sub-module, feature pyramid sub-module, region proposal network sub-module and improved classification regression detection head structure.

[0049] The backbone network submodule is used to extract image features from remote sensing scene images. The feature pyramid submodule is used to fuse image features to produce a fused feature image. The region proposal network submodule is used to filter regions of interest from the fused feature image. The improved classification and regression detection head structure is used to detect small rotated objects in the region of interest.

[0050] like Figure 2 As shown, Figure 2 (a) is a general detection head, and (b) is an improved detection head in this embodiment. The improved classification and regression detection head structure includes: a deformable convolutional v3 network, a cascaded serial attention network, and a cross attention network.

[0051] The deformable convolutional v3 network is used to enhance the scale information of the region of interest and obtain the feature map.

[0052] The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of the branch vectors respectively, obtain three corresponding feature vectors, and fuse the three feature vectors to obtain the global enhancement vector.

[0053] Due to the serious redundancy problem in the multi-head attention (MHSA) mechanism, the computational efficiency is low. Inspired by the group convolution in the efficient CNN, a cascaded serial attention network is proposed in this embodiment to model long-distance dependencies, thereby enhancing the global perception ability of the detector. The network divides the complete feature into different parts and provides them to each attention head separately. Such a cascade design has two advantages: first, providing different feature segmentations to each attention head can improve the diversity of the attention map (including channel information and global information), and the cascaded attention reduces the QKV layer of the input and output channels through multiple independent detection heads, thereby reducing the model calculation amount and parameter amount. Secondly, the cascaded attention head allows the network depth to be increased without introducing any additional parameters, thereby further improving the model capacity.

[0054] Specifically, such as Figure 3 As shown, the feature map of the cascaded serial attention network input is F, with a size of N×7×7. The split operation along the channel direction obtains three features F with a size of (N / 3)×7×7. i ′(i∈(1, 2, 3)). Fi After the pattern embedding operation, the aggregated feature F is obtained by using the jump connection method. i ", at this time F i ″=F′ i +Attention(F′ i ), and then a similar skip connection is used to combine the MLP network to obtain the first cascade partial feature output Since each cascaded partial output feature only contains the information of the corresponding 1 / 3 channel, in order to fuse the global feature information, this embodiment adopts the cascade connection method to combine the output features of the previous stage and the input F of the next stage i Perform channel information fusion. At this time, the fusion feature We send the fused features to the multi-head attention and FFN modules of the corresponding stage to obtain the final feature output Finally, the outputs of the three different stages are merged in the channel direction to obtain the final global enhancement vector

[0055] The cross attention network is used to perform feature compression on the global enhancement vector to obtain a classification feature vector and a positioning feature vector, and then combine the classification feature vector and the positioning feature vector in the form of cross attention to obtain the type and position of the rotated small target.

[0056] The detection of small rotating targets is an integration of classification and localization. The detection head needs to use information from these two tasks to make consistent predictions rather than perform these two tasks independently. That is, the detector needs to output a box with precise location and high classification confidence. Therefore, this embodiment uses channel-level cross attention to perform the interaction between classification and localization tasks. Channel-level attention can benefit the model in two ways: (1) reducing the complexity of attention to be linearly related to the image size; (2) enhancing the channel-level global perception ability of the model, complementing the cascaded serial attention that focuses on global spatial perception. Cross attention enables the model to use the features of one task to guide the feature learning process of another task, achieving the ultimate goal of cross-task learning. As Figure 4 As shown, the input feature is the classification feature X after scale perception and global perception enhancement. cls and regression feature X reg , the cross-task attention module can be expressed as:

[0057]

[0058]

[0059] Q=Cat(X cls , X reg), K=Cat(K cls , K reg )

[0060]

[0061]

[0062]

[0063] in, and Represents the input features X cls and X reg The projection matrix mapped to the key-value pairs in self-attention, and Represents the input features X cls and X reg The projection matrix that maps to the query value.

[0064] As shown in the above formula, cross-task attention uses reg The cross-task supervision information is used as a query to guide the classification feature X cls Representation learning (similarly using cls The cross-task supervision information is used as a query to guide the classification feature X reg Representation learning). In addition, it concatenates the features of the two tasks to generate keys and values, thereby further fusing cross-task information. To further improve the expressiveness of local information in the cross-task detection head, we introduced a local enhancement module consisting of a stack of depthwise separable convolutions after the cross-task attention module. It consists of a 1×1 DW input convolution, a 3×3 DW convolution, and a 1×1 DW output convolution. While introducing as few parameters as possible, it enhances the fusion of local information in the classification and regression branches. Finally, the prediction values for classification and regression are obtained through the FFN network structure. At this time:

[0065] X′ reg =FFN reg (DW 1x1 (DW 3x3 (DW 1x1 (X reg )))

[0066] X′ cls =FFN cls (DW 1x1 (DW 3x3 (DW 1x1 (X cls )))

[0067] Among them, X′ cls and X′ regThey represent the output vectors of the classification branch and the regression branch, as well as the classification feature vector and the positioning feature vector, respectively. FFN represents the MLP feedforward network, and DW represents the depth-wise separable convolution module. Finally, the classification feature vector and the positioning feature vector are combined in the form of cross attention to obtain the type and position of the rotated small target.

[0068] Example 2

[0069] In this embodiment, the workflow of the present invention will be introduced.

[0070] In this embodiment, the target detection module consists of four parts: backbone network (Backbone), feature pyramid (FPN), region proposal network (RPN) and improved classification regression detection head structure. Assume that the input image is a three-channel image with a size of (1024, 1024, 3). The input vector of the improved classification regression detection head structure is the region of interest RoI after the region extraction network RPN network is filtered, with a size of (7, 7, 255), denoted as F. F is Figure 2 The deformable convolution v3 network structure in (b) enhances the scale information and obtains the feature vector F of the same size. solid 。 solid Send in Figure 3 In the cascaded serial attention network, it is divided into three parallel branches, and the input of each branch is (7,7,85), which are respectively denoted as and Will The vector is flattened into a one-dimensional feature vector of (1, 7×7×85) through vector embedding and sent to the multi-head attention mechanism to calculate the global similarity information of the vector. After FFN and norm operations, the corresponding feature vector is obtained. At this time due to It only contains 1 / 3 of the channel information, so After being restored to the size of (7,7,85) The vectors are element-by-element added and combined, and the combined vector is flattened into a one-dimensional feature vector of (1, 7×7×85) through vector embedding, and sent to the multi-head attention mechanism to calculate the global similarity information of the vector. After FFN and norm operations, the corresponding feature vector is obtained. Similarly, we can get as well as Vector. After channel concatenation of the three vectors, a joint vector F of size (7,7,255) is obtained. mixer . F mixer Vector input Figure 4 In the cross attention network shown in the figure, the feature compression is first performed through two layers of MLP to obtain F reg (1, 7×7×255) and F cls(1, 7×7×255). There is similar semantic information between two different features, so F reg Guidance F cls and F cls Guidance F reg The two feature vectors are combined in the form of cross attention, which improves the cross-task expression ability of the classification and regression feature vectors and achieves accurate positioning of small rotating targets under the premise of high confidence classification. reg and F cls They are mapped to the feature spaces of (1, number of categories + 1) and (1, 5) respectively to complete the accurate positioning of the candidate frame and the detailed classification of the targets in the candidate frame.

[0071] Example 3

[0072] In this embodiment, a method for detecting a rotating small target in a remote sensing scenario includes the following steps:

[0073] S1. Obtain a remote sensing scene image to be detected.

[0074] S2. Improve the classification and regression detection head structure, and build a target detection network based on the improved classification and regression detection head structure. Detect small rotating targets in remote sensing scenes based on the target detection network.

[0075] The target detection network includes: backbone network, feature pyramid, region proposal network and improved classification regression detection head structure.

[0076] The backbone network is used to extract image features from remote sensing scene images. The feature pyramid is used to fuse image features to produce a fused feature image. The region proposal network is used to filter regions of interest from the fused feature image. The improved classification and regression detection head structure is used to detect small rotated objects in the region of interest.

[0077] The improved classification and regression detection head structure includes a deformable convolutional v3 network, a cascaded serial attention network, and a cross-attention network. The deformable convolutional v3 network is used to enhance the scale information of the region of interest to obtain a feature map. The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of each branch vector, and obtain three corresponding feature vectors. These three feature vectors are then fused to obtain a global enhancement vector. The cross-attention network is used to compress the features of the global enhancement vector to obtain a classification feature vector and a localization feature vector. The classification feature vector and the localization feature vector are then combined in a cross-attention manner to determine the type and position of the rotated small target.

[0078] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A rotating small target detection system in a remote sensing scenario, characterized in that: include: Image acquisition module and target detection module; The image acquisition module is used to acquire the remote sensing scene image to be detected; The target detection module is used to improve the classification and regression detection head structure and build a target detection network based on the improved classification and regression detection head structure; The target detection module is further configured to detect a rotating small target in the remote sensing scene based on the target detection network; The target detection network includes: a backbone network submodule, a feature pyramid submodule, a region proposal network submodule and an improved classification regression detection head structure; The backbone network submodule is used to extract image features from the remote sensing scene image; The feature pyramid submodule is used to fuse the image features to obtain a fused feature image; The region proposal network submodule is used to filter out the region of interest from the fused feature image; The improved classification regression detection head structure is used to detect the rotating small target in the region of interest; The improved classification regression detection head structure includes: deformable convolution v3 network, cascaded serial attention network and cross attention network; The deformable convolutional V3 network is used to enhance the scale information of the region of interest to obtain a feature map; The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of the branch vectors respectively, obtain three corresponding feature vectors, and fuse the three feature vectors to obtain a global enhancement vector; The cross attention network is used to perform feature compression on the global enhancement vector to obtain a classification feature vector and a positioning feature vector, and combine the classification feature vector and the positioning feature vector in a cross attention form to obtain the type and position of the rotated small target.

2. A method for detecting rotating small targets in remote sensing scenarios, characterized in that: The following steps are involved: Acquire a remote sensing scene image to be detected; Improve the classification and regression detection head structure, and build the target detection network based on the improved classification and regression detection head structure; Detecting a rotating small target in the remote sensing scene based on the target detection network; The target detection network includes: a backbone network, a feature pyramid, a region proposal network and an improved classification regression detection head structure; The backbone network is used to extract image features from the remote sensing scene image; The feature pyramid is used to fuse the image features to obtain a fused feature image; The region proposal network is used to screen out a region of interest from the fused feature image; The improved classification regression detection head structure is used to detect the rotating small target in the region of interest; The improved classification regression detection head structure includes: deformable convolution v3 network, cascaded serial attention network and cross attention network; The deformable convolutional V3 network is used to enhance the scale information of the region of interest to obtain a feature map; The cascaded serial attention network is used to split the feature map into three parallel branch vectors, calculate the global similarity information of the branch vectors respectively, obtain three corresponding feature vectors, and fuse the three feature vectors to obtain a global enhancement vector; The cross attention network is used to perform feature compression on the global enhancement vector to obtain a classification feature vector and a positioning feature vector, and combine the classification feature vector and the positioning feature vector in a cross attention form to obtain the type and position of the rotated small target.

Images