Remote sensing rotating small target detection method and system based on deformable convolution improved FPN

Abstract

This invention discloses a method and system for detecting small rotating remote sensing targets based on deformable convolution-modified FPN. The method comprises: acquiring a small rotating remote sensing target image; constructing a two-stage rotating target detection model; and inputting the small rotating remote sensing target image into the two-stage rotating target detection model to perform positioning and classification of the small rotating remote sensing target. This invention facilitates the specific classification and accurate coordinate positioning of small targets in remote sensing scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a remote sensing rotating small target detection method and system based on deformable convolution improved FPN. Background Art

[0002] The Feature Pyramid Network (FPN) is a deep neural network architecture for object detection, designed to address the challenge of detecting objects of varying scales. The core idea of ​​FPN is to leverage multi-scale image features to improve detection performance, particularly for small objects. This architecture was originally proposed in a paper from Tesla AI Research and has been widely adopted in Mask R-CNN. Figure 1 It is a traditional FPN network structure.

[0003] The steps of feature extraction using Feature Pyramid Network (FPN) mainly include the following stages:

[0004] (1) Basic feature extraction:

[0005] First, select a pre-trained convolutional neural network (such as ResNet, VGG, or MobileNet) and use it as the basic feature extraction network.

[0006] After the input image passes through the basic feature extraction network, a series of feature maps with different levels are obtained. The resolution of these feature maps gradually decreases, but the semantic information gradually increases.

[0007] (2) Constructing a feature pyramid:

[0008] A feature pyramid is constructed between feature maps at different levels. Typically, a baseline feature map is selected, such as a feature map with a higher resolution, and then a series of feature maps with different resolutions are generated through upsampling and downsampling operations.

[0009] Upsampling can be achieved through interpolation methods, such as bilinear interpolation or transposed convolution, to upsample the low-resolution feature map to the same resolution as the baseline feature map.

[0010] Downsampling is achieved through methods such as pooling, which downsamples the high-resolution feature map to the same resolution as the baseline feature map.

[0011] (3) Top-down and lateral connections:

[0012] After the feature pyramid is constructed, a top-down path is used to gradually restore the resolution of the feature map through upsampling operations starting from high-level semantic features.

[0013] At the same time, lateral connections are used to connect low-level high-resolution features with high-level low-resolution features to achieve information fusion.

[0014] Through a top-down process, abstract semantic information can be gradually transferred to low-level feature maps, thereby generating multi-scale feature maps with rich semantic information.

[0015] (4) Multi-scale feature fusion:

[0016] In the process of top-down and lateral connections, features of different scales are fused to produce multi-scale feature maps with rich semantic information.

[0017] This multi-scale feature representation can better adapt to the detection requirements of targets of different sizes and improve detection performance.

[0018] The shortcomings of using Feature Pyramid Network (FPN) for feature extraction are:

[0019] (1) Information transfer loss: In FPN, information loss may occur during the top-down and lateral connection process. Due to the different resolutions and semantic information of feature maps at different levels, information transfer may cause some important feature information to be lost or blurred, affecting the final detection performance.

[0020] (2) Feature duplication and redundancy: The feature pyramid structure in FPN may lead to repeated and redundant information in the feature map. During the top-down information transmission process, the information in the high-level feature map may be copied and transmitted to the low-level feature map multiple times, resulting in feature redundancy and waste.

[0021] (3) Not suitable for small-scale targets: The traditional FPN structure may not perform well when processing small-scale targets. Due to the low resolution of high-level feature maps in the top-down information transmission process, the detection accuracy of small targets may be low.

[0022] (4) Insufficient processing of targets with blurred boundaries: The FPN structure may have certain difficulties in processing targets with blurred boundaries, because the information transmission from top to bottom and horizontal connections may blur the boundary information, affecting the accuracy of target detection. Summary of the Invention

[0023] In response to the shortcomings of the existing technology, the present invention proposes a remote sensing rotating small target detection method and system based on deformable convolution to improve FPN, which helps to complete specific classification and accurate coordinate positioning of small targets in remote sensing scenes.

[0024] To achieve the above object, the present invention provides the following solutions:

[0025] The remote sensing rotating small target detection method based on deformable convolution improved FPN includes the following steps:

[0026] Acquire remote sensing rotating small target images;

[0027] Build a two-stage rotation target detection model;

[0028] The remote sensing rotating small target image is input into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target.

[0029] Preferably, the two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network and a classification regression detection head;

[0030] The backbone network is used to extract the feature maps of each layer of the remote sensing rotating small target;

[0031] The deformable convolution FPN structure is used to extract semantic and position information on each layer of feature maps of remote sensing rotating small targets;

[0032] The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets;

[0033] The classification and regression detection head is used to complete the final remote sensing rotating small target positioning and classification tasks based on the initially extracted candidate frames and classification information.

[0034] Preferably, the deformable convolution FPN structure includes: a global-local information fusion module, a feature fusion modulation module and a feature alignment module;

[0035] The global-local information fusion module is used to extract semantic information of high-level feature maps of remote sensing rotating small targets;

[0036] The feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps of different levels;

[0037] The feature alignment module is used to align the upsampled high-level feature map and the low-level feature map using the Carafe operator in combination with deformable convolution in a self-attention-like manner.

[0038] Preferably, the global-local information fusion module includes three parts: an MLP module, a DW module and a channel attention module;

[0039] The MLP module is used to obtain global features based on high-level features of remotely sensed rotating small targets;

[0040] The DW module is used to obtain local features based on the high-level features of the remotely sensed rotating small target;

[0041] The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

[0042] Preferably, the feature fusion modulation module includes: a feature fusion module and a feature modulation module;

[0043] The feature fusion module is used to introduce a full bidirectional feature fusion operation to perform bidirectional fusion on feature maps at different levels;

[0044] The feature modulation module is used to perform weighted processing on feature maps of different levels using a feature modulation method.

[0045] The present invention also provides a remote sensing rotating small target detection system based on deformable convolution improved FPN, comprising: an acquisition module, a construction module and a detection module;

[0046] The acquisition module is used to acquire remote sensing rotating small target images;

[0047] The building module is used to build a two-stage rotation target detection model;

[0048] The detection module is used to input the remote sensing rotating small target image into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target.

[0049] Preferably, the two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network and a classification regression detection head;

[0050] The backbone network is used to extract the feature maps of each layer of the remote sensing rotating small target;

[0051] The deformable convolution FPN structure is used to extract semantic and position information on each layer of feature maps of remote sensing rotating small targets;

[0052] The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets;

[0053] The classification and regression detection head is used to complete the final remote sensing rotating small target positioning and classification tasks based on the initially extracted candidate frames and classification information.

[0054] Preferably, the deformable convolution FPN structure includes: a global-local information fusion module, a feature fusion modulation module and a feature alignment module;

[0055] The global-local information fusion module is used to extract semantic information of high-level feature maps of remote sensing rotating small targets;

[0056] The feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps of different levels;

[0057] The feature alignment module is used to align the upsampled high-level feature map and the low-level feature map using the Carafe operator in combination with deformable convolution in a self-attention-like manner.

[0058] Preferably, the global-local information fusion module includes three parts: an MLP module, a DW module and a channel attention module;

[0059] The MLP module is used to obtain global features based on high-level features of remotely sensed rotating small targets;

[0060] The DW module is used to obtain local features based on the high-level features of the remotely sensed rotating small target;

[0061] The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

[0062] Preferably, the feature fusion modulation module includes: a feature fusion module and a feature modulation module;

[0063] The feature fusion module is used to introduce a full bidirectional feature fusion operation to perform bidirectional fusion on feature maps at different levels;

[0064] The feature modulation module is used to perform weighted processing on feature maps of different levels using a feature modulation method.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] 1. The global-local information fusion module enhances the semantic information of high-level features through the MLP module and the DW module respectively, weakens the noise in the background information, and helps small targets in remote sensing scenes to complete specific classification and accurate coordinate positioning.

[0067] 2. The feature fusion modulation module performs feature fusion not only in the upsampling path but also in the downsampling path to ensure that features at different levels can fully exchange information with each other. This effectively compensates for the information imbalance caused by one-way fusion and improves the global consistency of features.

[0068] 3. The feature alignment module uses the Carafe operator instead of the traditional upsampling algorithm, and combines it with deformable convolution to align the upsampled high-level feature maps and low-level feature maps in a self-attention-like manner, solving the artifacts and accuracy loss problems caused by the interpolation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 This is a diagram of a traditional FPN network structure according to an embodiment of the present invention;

[0071] Figure 2 This is a diagram of the deformable convolution improved FPN structure according to an embodiment of the present invention;

[0072] Figure 3 This is a structural diagram of the feature fusion MLP module according to an embodiment of the present invention;

[0073] Figure 4 This is a structural diagram of a feature fusion DW module according to an embodiment of the present invention;

[0074] Figure 5 This is a channel attention structure diagram of an embodiment of the present invention;

[0075] Figure 6 This is a structural diagram of a feature fusion modulation module according to an embodiment of the present invention;

[0076] Figure 7 This is a structural diagram of a feature alignment module according to an embodiment of the present invention;

[0077] Figure 8 This is a framework diagram of a two-stage rotating target detection algorithm according to an embodiment of the present invention;

[0078] Figure 9 This is a flow chart of a remote sensing rotating small target detection method based on deformable convolution improved FPN according to an embodiment of the present invention;

[0079] Figure 10 This is a schematic diagram of an embodiment of the present invention in which the input image is a three-channel image;

[0080] Figure 11 Outputting a heat map of features of multiple aircraft targets in a dense aircraft scene according to an embodiment of the present invention;

[0081] Figure 12 Schematic diagram of a rotating small target detection frame according to an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0083] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0084] Example 1

[0085] like Figure 9 As shown, the present invention provides a remote sensing rotating small target detection method based on deformable convolution to improve FPN, comprising the following steps:

[0086] Acquire remote sensing rotating small target images;

[0087] Build a two-stage rotation target detection model;

[0088] The remote sensing rotating small target image is input into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target.

[0089] In this embodiment, the two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network, and a classification regression detection head;

[0090] The backbone network is used to extract the feature maps of each layer of remote sensing rotating small targets;

[0091] The deformable convolutional FPN structure is used to extract the semantic and position information of each layer of feature maps of remote sensing rotating small targets;

[0092] The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets;

[0093] The classification and regression detection head is used to complete the final positioning and classification tasks of remote sensing rotating small targets based on the initially extracted candidate boxes and classification information.

[0094] In this embodiment, if Figure 2 As shown in the figure, the deformable convolution FPN structure includes: global-local information fusion module, feature fusion modulation module and feature alignment module;

[0095] The global-local information fusion module is used to extract semantic information from high-level feature maps of small rotating remote sensing targets. However, there is a lack of information interaction between the shallow multi-layer information. Therefore, the feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps at different levels. Through the cross-fusion of features at different levels and the redistribution of weights, semantic information adapted to features at different levels is obtained. At this point, there is still a lack of information interaction between high-level features and shallow features, and the salient semantic information fails to provide guidance for target positioning in shallow large-scale feature maps. Therefore, the feature alignment module is used to align the upsampled high-level feature maps with the low-level feature maps using the Carafe operator and deformable convolution in a self-attention-like manner. This allows the salient semantic information in the deep-level features to gradually guide the target coordinate positioning and regression tasks of the shallow features in a top-down manner.

[0096] Specifically, (1) Global-local information fusion module: In target detection, the feature information extracted by the backbone network at different stages contains semantic information at different levels. Low-level features have a larger scale and are usually suitable for rough positioning of targets, especially in dense small target scenes. This is because low-level feature maps usually contain complete small target contours and part of the background information, which helps to roughly locate small targets. However, for the specific classification and accurate coordinate positioning of small targets, high-level information is usually needed. High-level features usually weaken the background information part and strengthen the characteristics of specific significant small targets, such as texture, color, key point information, etc., which are crucial for accurate positioning and contour segmentation in dense small target scenes.

[0097] (2) Feature fusion modulation module: The traditional feature pyramid structure usually only adopts a unidirectional upsampling operation, which leads to uneven fusion of feature information at different levels, thus affecting the detection accuracy in subsequent steps. To solve this problem, the feature fusion modulation module introduces a full bidirectional feature fusion operation to perform bidirectional fusion on feature maps at different levels. Specifically, the feature fusion modulation module not only performs feature fusion in the upsampling path, but also in the downsampling path to ensure that features at different levels can fully exchange information with each other. This can effectively make up for the information imbalance problem caused by unidirectional fusion and improve the global consistency of features. In order to further reduce the problem of feature information imbalance, the feature fusion modulation module adopts the idea of ​​feature modulation to perform weighted processing on feature maps at different levels. This means that when performing feature fusion, the module will weight the features according to their importance to ensure that the contributions of features at each level can be balanced and fully utilized.

[0098] (3) Feature alignment module: The traditional feature pyramid structure usually uses a bilinear interpolation algorithm for feature upsampling alignment so as to upsample the low-resolution feature map to the same size as the high-resolution feature map. However, the bilinear interpolation algorithm may produce some artifacts, such as jagged edges, when resizing or resampling the image. This is because the algorithm assumes that the function is continuous and there is no obvious discontinuity between data points. The present invention designs a feature alignment module, adopts the Carafe operator instead of the traditional upsampling algorithm, and combines deformable convolution to align the upsampled high-level feature map and the low-level feature map in a self-attention-like manner.

[0099] In this embodiment, the global-local information enhancement module includes three parts: a global feature extraction MLP module, a DW module, and a channel attention module;

[0100] The global feature extraction MLP module is used to obtain global features based on the high-level features of remotely sensed rotating small targets;

[0101] The DW module is used to obtain local features based on the high-level features of small targets rotated by remote sensing;

[0102] The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

[0103] Specifically, such as Figure 3 As shown, the input to the global feature extraction MLP module is the output of the fourth stage of the backbone network, denoted as c5, (B, C, H, W). The MLP module uses skip connections for feature learning. After GroupNormailization, feature c5′ is normalized to c5′, which still has the size (B, C, H, W). c5′ undergoes a 3x3 DW convolution with a stride of 1 and C channels to obtain the summarized feature c5.

[0104] The MLP module introduces a learnable scaling factor τ, multiplies c5" by τ to obtain the scaled feature and performs a dropout operation to obtain C5'", which is denoted as:

[0105] C5′′=C5+Dropout(DW 3x3 (GN(C5)×τ)).

[0106] C″′5 is passed through a similar skip connection structure, and then goes through a Group Normailization layer, a nonlinear mapping function layer (implemented by a three-layer perceptron MLP, with SiLU as the activation function), a channel modulation layer, and a Dropout layer to obtain C global .

[0107] C global =C5"′+Dropout(SiLU(MLP(GN(C5′")))×τ2).

[0108] like Figure 4 As shown in the figure, the DW module introduces dilation 2, 4, and 6 dilated convolutions to learn feature information of different receptive field sizes, perform deep fusion of features of different receptive fields, and learn local information on high-level feature maps. After the C5 feature undergoes a 3x3 depth-separable convolution, it obtains the feature C of the same size. normal , C normal , after splitting along the channel direction, we can get After three parallel convolution branches with different dilations extract information and perform feature splicing on the channel, the final output C is obtained. local .

[0109] C normal =DW 3x3 (C5),

[0110]

[0111] C local =Conv 3x3 (C local ).

[0112] Global feature C global and local features C local After splicing in the channel direction, the fused feature F is obtained. Since the fused feature contains local information and global information on the high-level feature map, but lacks focus on significant small targets, the channel self-attention structure is introduced, such as Figure 5 As shown. For the input features (the above fusion features) F(F∈(B,C,H,W)), the spatial information of the feature map is aggregated by the maximum pooling and average pooling operations respectively, and two different spatial context feature vectors are generated respectively, which are recorded as and and The two features are forwarded to a multi-layer perceptron MLP structure, the output of which is a channel attention map M c ∈(C,1,1), the vectors output by the maximum pooling feature and the average pooling feature after passing through the three-layer MLP structure are merged to obtain a channel weight feature map F with the same output dimension as the number of channels of the input feature map channel Then, the fusion feature and the channel weight feature map are multiplied by the elements in the channel direction to obtain the final output vector C out (Cout The size remains unchanged).

[0113] In this embodiment, the feature fusion modulation module includes: a feature fusion module and a feature modulation module;

[0114] The feature fusion module is used to introduce a full bidirectional feature fusion operation and perform bidirectional fusion of feature maps at different levels;

[0115] The feature modulation module is used to perform weighted processing on feature maps of different levels using the feature modulation method.

[0116] Specifically, during the feature fusion process, information from different objects at the same location may interfere with each other. In the feature fusion module proposed in the present invention, this limitation is particularly obvious because more rounds of up-sampling and down-sampling feature fusion are required. Therefore, the present invention adopts an adaptive spatial fusion method to effectively fuse features at different levels. Figure 6 As shown, the input of the feature modulation module is the four outputs of the feature fusion network, namely C′2, C′3, C′4, and C′5, whose sizes are (2048, 32, 32), (1024, 64, 64), (512, 128, 128), and (256, 256, 256). After 1x1 convolution and downsampling operations, the four levels of the pyramid structure obtain compressed features C with exactly the same size. compress (Use the subscript i to distinguish the levels), the size of the compressed feature is (256, 32, 32), C compress After channel compression through 3x3 convolution, we get four different levels of weight compression vectors C″2, C″3, C″4, and C″5, all of which are (8, 32, 32). Then, we fuse the four levels of weight compression vectors in the channel direction to get the weight fusion vector C channelMixer , the size is (32,32,32), and then the channel is compressed to be consistent with the number of levels, that is, (4,32,32), and the weighted vector of (4,32,32) size is split into four weighted vectors of the same size according to the channel direction, and respectively compared with the compressed features of the four levels. Perform element-by-element dot product operation to obtain the final weighted feature vectors P2, P3, P4, and P5.

[0117] In this embodiment, with the help of the recombination characteristics of the Carafe operator, the feature alignment module reduces the probability of spatial misalignment to a certain extent, while enhancing the aggregation ability of local position information. In order to further enrich the semantic information of small targets on the feature maps of each layer, we propose a new deformable convolution alignment method to perform feature alignment operations from top to bottom, such as Figure 7As shown in Figure 2, between adjacent features, deep features are upsampled using the Carafe operator to produce a fused feature vector. After concatenating the fused feature vector with the shallow features, an offset is learned. This offset is then strided and accumulated element-by-element with the shallow features to produce the final output vector.

[0118] Example 2

[0119] The present invention also provides a remote sensing rotating small target detection system based on deformable convolution improved FPN, comprising: an acquisition module, a construction module and a detection module;

[0120] The acquisition module is used to acquire remote sensing rotating small target images;

[0121] The construction module is used to build a two-stage rotation target detection model;

[0122] The detection module is used to input the remote sensing rotating small target image into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target.

[0123] In this embodiment, the two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network, and a classification regression detection head;

[0124] The backbone network is used to extract the feature maps of each layer of the remote sensing rotating small target;

[0125] The deformable convolutional FPN structure is used to extract the semantic and position information of each layer of feature maps of remote sensing rotating small targets;

[0126] The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets;

[0127] The classification and regression detection head is used to complete the final positioning and classification tasks of remote sensing rotating small targets based on the initially extracted candidate boxes and classification information.

[0128] In this embodiment, the deformable convolution FPN structure includes: a global-local information fusion module, a feature fusion modulation module and a feature alignment module;

[0129] The global-local information fusion module is used to extract the semantic information of the high-level feature map of the remote sensing rotating small target;

[0130] The feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps at different levels;

[0131] The feature alignment module is used to align the upsampled high-level feature maps and low-level feature maps using the Carafe operator and deformable convolution in a self-attention-like manner.

[0132] In this embodiment, the global-local information fusion module includes three parts: an MLP module, a DW module, and a channel attention module;

[0133] The MLP module is used to obtain global features based on the high-level features of remotely sensed rotating small targets;

[0134] The DW module is used to obtain local features based on the high-level features of small targets rotated by remote sensing;

[0135] The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

[0136] In this embodiment, the feature fusion modulation module includes: a feature fusion module and a feature modulation module;

[0137] The feature fusion module is used to introduce a full bidirectional feature fusion operation and perform bidirectional fusion of feature maps at different levels;

[0138] The feature modulation module is used to perform weighted processing on feature maps of different levels using the feature modulation method.

[0139] Example 3

[0140] like Figure 8 As shown in , the two-stage rotation target detection model consists of four parts: backbone network (Backbone), feature pyramid (FPN), region proposal network (RPN) and classification regression detection head. Assume that the input image is a three-channel image with a size of (1024,1024,3), as shown in Figure 10As shown. The deformable convolutional FPN structure proposed in this invention replaces the feature pyramid structure in the figure. The input and output vectors are all four-layer vectors, and the output vectors are denoted as C′5, C′4, C′3, and C′2. The vectors output by the four stages of the backbone network are C5, C4, C3, and C2, with sizes of (32, 32, 256), (64, 64, 256), (128, 128, 256), and (256, 256, 256), respectively. They are respectively input into the deformable convolutional FPN network proposed in this article. The C5 feature, as the highest-level feature, contains rich semantic information and is used to extract the saliency information of the rotated small target. Therefore, after downsampling the C5 feature, C6 (16, 16, 256) is obtained and sent to the global-local information enhancement module mentioned above to extract semantic information. At the same time, the channel information interaction of the feature vector is enhanced through the channel attention mechanism. The features of each layer of C5, C4, C3, and C2 are fully connected to interact with each other to enhance the semantic richness of each layer of features. At the same time, the feature allocation module mentioned above is used to allocate different degrees of feature information to features at different levels to improve the specificity between features at each layer. Finally, starting from C6, the Carafe operator is used for upsampling (therefore, the sizes of features between the upper and lower layers are different and need to be aligned), and deformable feature alignment is performed with C5, C4, C3, and C2 respectively. The semantics and position information on the feature maps of each layer of the rotated small target are highlighted in a self-attention-like manner to obtain the corresponding C′5, C′4, C′3, and C′2. Figure 11 As shown in the figure, the feature output heat map of multiple aircraft targets in a dense aircraft scene is shown. It can be seen that all aircraft targets are surrounded by significant information and the background information is greatly weakened. The four feature vectors output are fed into Figure 8 The RPN region proposal network on the right initially extracts candidate boxes and classifications (distinguishing foreground and background, belonging to binary classification), and then sends them to the final classification regression detection head to complete the final target positioning and classification tasks, as shown below: Figure 12 The rotated small target detection box shown.

[0141] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A remote sensing rotating small target detection method based on deformable convolution to improve FPN is characterized by: The following steps are involved: Acquire remote sensing rotating small target images; Build a two-stage rotation target detection model; Inputting the remote sensing rotating small target image into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target; The two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network, and a classification regression detection head; The backbone network is used to extract the feature maps of each layer of the remote sensing rotating small target; The deformable convolution FPN structure is used to extract semantic and position information on each layer of feature maps of remote sensing rotating small targets; The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets; The classification and regression detection head is used to complete the final remote sensing rotating small target positioning and classification tasks based on the initially extracted candidate frames and classification information; The deformable convolution FPN structure includes: a global-local information fusion module, a feature fusion modulation module and a feature alignment module; The global-local information fusion module is used to extract semantic information of high-level feature maps of remote sensing rotating small targets; The feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps of different levels; The feature alignment module is used to align the upsampled high-level feature map and the low-level feature map using the Carafe operator in combination with deformable convolution in a self-attention-like manner.

2. The remote sensing rotating small target detection method based on deformable convolution improved FPN according to claim 1 is characterized in that: The global-local information fusion module includes three parts: an MLP module, a DW module and a channel attention module; The MLP module is used to obtain global features based on high-level features of remotely sensed rotating small targets; The DW module is used to obtain local features based on the high-level features of the remotely sensed rotating small target; The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

3. The remote sensing rotating small target detection method based on deformable convolution improved FPN according to claim 1 is characterized in that: The feature fusion modulation module includes: a feature fusion module and a feature modulation module; The feature fusion module is used to introduce a full bidirectional feature fusion operation to perform bidirectional fusion on feature maps at different levels; The feature modulation module is used to perform weighted processing on feature maps of different levels using a feature modulation method.

4. A remote sensing rotating small target detection system based on deformable convolution improved FPN is characterized by: include: Acquire modules, build modules, and detect modules; The acquisition module is used to acquire remote sensing rotating small target images; The building module is used to build a two-stage rotation target detection model; The detection module is used to input the remote sensing rotating small target image into the two-stage rotating target detection model to complete the positioning and classification of the remote sensing rotating small target; The two-stage rotation target detection model includes: a backbone network, a deformable convolutional FPN structure, a region proposal network, and a classification regression detection head; The backbone network is used to extract the feature maps of each layer of the remote sensing rotating small target; The deformable convolution FPN structure is used to extract semantic and position information on each layer of feature maps of remote sensing rotating small targets; The region proposal network is used to preliminarily extract candidate boxes and classification information based on the semantic and position information of each layer of feature maps of remote sensing rotation small targets; The classification and regression detection head is used to complete the final remote sensing rotating small target positioning and classification tasks based on the initially extracted candidate frames and classification information; The deformable convolution FPN structure includes: a global-local information fusion module, a feature fusion modulation module and a feature alignment module; The global-local information fusion module is used to extract semantic information of high-level feature maps of remote sensing rotating small targets; The feature fusion modulation module is used to perform bidirectional fusion and weighted processing on feature maps of different levels; The feature alignment module is used to align the upsampled high-level feature map and the low-level feature map using the Carafe operator in combination with deformable convolution in a self-attention-like manner.

5. The remote sensing rotating small target detection system based on deformable convolution improved FPN according to claim 4 is characterized in that: The global-local information fusion module includes three parts: an MLP module, a DW module and a channel attention module; The MLP module is used to obtain global features based on high-level features of remotely sensed rotating small targets; The DW module is used to obtain local features based on the high-level features of the remotely sensed rotating small target; The channel attention module is used to obtain a channel weight feature map whose output dimension is the same as the number of channels of the input feature map based on the global features and the local features.

6. The remote sensing rotating small target detection system based on deformable convolution improved FPN according to claim 4 is characterized in that: The feature fusion modulation module includes: a feature fusion module and a feature modulation module; The feature fusion module is used to introduce a full bidirectional feature fusion operation to perform bidirectional fusion on feature maps at different levels; The feature modulation module is used to perform weighted processing on feature maps of different levels using a feature modulation method.

Images