A remote sensing image rotating target detection method based on double-path feature enhancement

Abstract

The application provides a remote sensing image rotating target detection method based on double-path feature enhancement, and relates to the technical fields of computer vision and remote sensing image processing. The method comprises the following steps: constructing a double-path feature enhancement remote sensing image rotating target detection network comprising a texture enhancement path and a direction modeling path; in the texture enhancement path, an adaptive wavelet reconstruction module is used to enhance the texture details and edge features in the input feature map; in the direction modeling path, a multi-scale angle guided deformable encoder is used to perform spatial alignment and direction consistency modeling on the input feature map, and extract features containing structure and direction information; the features output by the two paths are fused; a joint loss function is constructed, and the network is trained end-to-end; and the trained network is used to detect and locate the rotating targets in the remote sensing image. The application can effectively improve the detection accuracy and robustness of multi-direction, multi-scale, densely distributed and small-size targets in the remote sensing image.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and remote sensing image processing technology, and in particular to a method for detecting rotating targets in remote sensing images based on dual-path feature enhancement. Background Technology

[0002] With the rapid development of satellite remote sensing and UAV imaging technologies, remote sensing images are widely used in urban planning, disaster monitoring, and resource management. Rotating target detection in these images has become a key technical task. Compared to target detection in natural scenes, targets in remote sensing images typically exhibit large scale differences, wide directional distribution, and dense arrangement, which places higher demands on the accuracy and robustness of detection algorithms.

[0003] With the continuous improvement of remote sensing image resolution, applications are increasingly demanding higher accuracy in target localization and stronger feature representation capabilities. Spatially, remote sensing images, limited by top-down imaging, often contain both small targets and large features simultaneously, requiring detection models to consider both fine-grained texture features and global semantic representation. However, shallow details are easily overwhelmed by deeper semantics during cross-layer feature fusion, leading to a gradual attenuation of information about small targets (small-sized targets) and reduced detection performance.

[0004] In terms of geometric scale, targets such as ships and aircraft may appear in images from any direction due to factors such as target shape and imaging angle, and they are densely distributed in scenes such as ports, airports, and parking lots, posing challenges to the spatial alignment and orientation perception capabilities of features. In addition, remote sensing images are often affected by complex environments such as changes in lighting and cloud cover, further increasing the uncertainty of detection and placing higher demands on the robustness and adaptability of algorithms.

[0005] While existing methods have alleviated the aforementioned problems to some extent, they still have significant shortcomings in small target detail modeling, orientation-aware construction, and dense region detection, making it difficult to meet the complex requirements of rotating target detection in remote sensing images. Therefore, there is an urgent need to propose a novel feature enhancement mechanism and rotation-aware detection framework to improve the overall detection accuracy and the generalization ability of the detection model. Summary of the Invention

[0006] To address the technical problem that existing technologies struggle to meet the complex requirements of detecting rotating targets in remote sensing images, this invention provides a method for detecting rotating targets in remote sensing images based on dual-path feature enhancement. The technical solution is as follows:

[0007] On the one hand, a method for detecting rotating targets in remote sensing images based on dual-path feature enhancement is provided, including:

[0008] S1. Construct a dual-path feature enhancement network for remote sensing image rotation target detection, which includes texture enhancement path and orientation modeling path; whereby,

[0009] In the texture enhancement path, an adaptive wavelet reconstruction module is used to enhance the texture details and edge features in the input feature map; where the feature map is extracted from remote sensing images.

[0010] In the orientation modeling path, a multi-scale angle-guided deformable encoder is used to perform spatial alignment and orientation consistency modeling on the input feature map, and extract features containing structural and orientation information.

[0011] By fusing the features output from the two paths through a cross-attention mechanism, multi-scale features with enhanced discriminative capabilities are generated.

[0012] S2. Construct a joint loss function, and train a dual-path feature-enhanced remote sensing image rotation target detection network end-to-end based on the generated multi-scale features with enhanced discriminative ability and the joint loss function; wherein, the joint loss function includes: classification loss, rotation-aware detection box regression loss and angle regression loss;

[0013] S3. The trained dual-path feature enhancement network for remote sensing image rotation target detection is used to detect and locate rotating targets in remote sensing images.

[0014] Furthermore, the dual-path feature enhancement remote sensing image rotation target detection network includes: a backbone network, a feature pyramid network, a dual-path feature enhancement framework, and a target detection head; the dual-path feature enhancement framework includes a texture enhancement path and a orientation modeling path; wherein,

[0015] The backbone network is used to extract multi-scale features from the input remote sensing images;

[0016] Feature pyramid network is used to enhance the semantic expressive power of features at different scales;

[0017] A dual-path feature enhancement framework is used to process the features output by the backbone network and the feature pyramid network to enhance texture details and edge features, as well as model spatial alignment and orientation consistency.

[0018] The target detection head is used to process the features output by the dual-path feature enhancement framework to complete the classification and regression of rotating targets in remote sensing images.

[0019] Furthermore, the adaptive wavelet reconstruction module includes:

[0020] The Discrete Wavelet Transform submodule is used to perform multi-level discrete wavelet transform on the input shallow high-resolution feature map to extract high-frequency features containing texture and edge information as well as low-frequency features containing semantic structure.

[0021] An adaptive weighted fusion submodule is used to dynamically fuse the high-frequency features and low-frequency features based on the response intensity, so as to improve the structural integrity of the features;

[0022] The Selective Residual Attention submodule is used to enhance the structural representation of the target region, suppress background redundancy, and output a feature map with enhanced details based on fused features.

[0023] Furthermore, for the feature map P3 at level P3 output by the feature pyramid network, the fused feature representation is as follows:

[0024]

[0025] α L =σ(ReLU(GN(Conv) 1×1 (L1))))

[0026] α H =σ(ReLU(GN(Conv) 1×1 (H1))))

[0027]

[0028] in, Represents fused features; ⊙ represents element-wise multiplication; σ(·) represents the sigmoid activation function; ε is a parameter to prevent the denominator from being zero; ReLU represents the activation function; GN represents group normalization; Conv represents the convolution operation; α L α H These represent the weights of the low-frequency and high-frequency components, respectively. L1 and H1 represent the weights of the low-frequency and high-frequency components after normalization, respectively; L1 and H1 represent the low-frequency and high-frequency components after the first-order wavelet transform, respectively.

[0029] Furthermore, the selective residual attention submodule is specifically used to calculate the fused features. Residual between the original input feature P3 and the original input feature:

[0030]

[0031] Where R3 represents the residual, and P3 represents the feature map at level P3 output by the feature pyramid network;

[0032] Spatial-channel joint attention map A3 is generated using the convolutional block attention module:

[0033] A3 = CBAM(P3)

[0034] Wherein, CBAM() represents the convolutional block attention module;

[0035] Using A3 to selectively enhance the residual R3, we obtain:

[0036]

[0037] in, This represents the feature map after selective enhancement.

[0038] Controlled using residual weighting factor λ The injection intensity is then fused with the original input feature P3. Information is integrated through convolutional blocks to obtain the final detailed feature map, represented as:

[0039]

[0040] in, This represents the feature map after detail enhancement.

[0041] Furthermore, the multi-scale angle-guided deformable encoder includes:

[0042] The lightweight angle prediction module is used to predict pixel-level angle information from the input feature map and generate an angle guidance map to provide target orientation or direction cues in spatial location.

[0043] A rotation-sensitive visual encoder is used to co-model the angle guide map and the input feature map, extract orientation-sensitive features, and generate a rotation-sensitive visual feature map.

[0044] The visual-angle cross-attention module is used to achieve spatial alignment of rotation-sensitive visual features and angle features within rotation-sensitive regions, generating multi-scale feature maps with enhanced directional consistency.

[0045] Furthermore, the lightweight angle prediction module is used to analyze the multi-scale feature map output from the feature pyramid network. Predict pixel-level angle information and generate corresponding angle guidance maps. To provide target orientation or direction cues for each spatial location; where,

[0046] Θ l =f pred (P l )

[0047] Among them, f pred (·) indicates the lightweight angle prediction module.

[0048] Furthermore, the rotation-sensitive visual encoder is specifically used for the angle-guided image Θ. l Perform channel expansion to make it compatible with P l Alignment along the channel dimension generates angular features, represented as follows:

[0049]

[0050] in, Indicates angular features, f enc (·) represents an angle encoder consisting of two layers of convolution, normalization, and nonlinear activation;

[0051] right Introducing spatial perception modeling f cxt (·), construct a more discriminative spatial context-enhanced feature map. Represented as:

[0052]

[0053] F θ With F cxt Element-wise summation is performed, and global average pooling is applied to extract the spatial mean of each channel as its global representation. Channel attention weights G are then generated through nonlinear mapping. l , represented as:

[0054]

[0055] Where σ(·) represents the sigmoid activation function, GAP(·) represents global average pooling, and MLP(·) represents channel-wise mapping;

[0056] The angle features are modulated using a weighting factor μ and then injected into the original input features P. l In this process, the final rotation-sensitive visual feature map is formed. Represented as:

[0057]

[0058] Furthermore, the vision-angle cross-attention module is specifically used to focus on rotation-sensitive visual features. Construct a query to encode the angular features. Used as Value, it enables dynamic sampling of local regions through reference points and learnable offsets, achieving spatial alignment between rotation-sensitive visual features and angular features, and generating multi-scale feature maps with enhanced orientation consistency.

[0059] Furthermore, the classification loss is a FocalLoss-based classification loss used to address the foreground-background imbalance problem;

[0060] The rotation sensing detection box regression loss is a Smooth L1 Loss-based rotation sensing detection box regression loss, which is used to optimize target position and scale prediction.

[0061] The angle regression loss is used to improve the accuracy of target direction prediction.

[0062] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for remote sensing image rotation target detection based on dual-path feature enhancement.

[0063] In summary, the beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0064] (1) By introducing an adaptive wavelet reconstruction module, the ability to perceive shallow texture details and boundaries is enhanced, and the performance of small target detection is improved;

[0065] (2) By using a multi-scale angle-guided deformable encoder to construct a directional modeling path, extract the directional and structural information of the target, and achieve spatial structure alignment with directional consistency, thereby improving the modeling effect and directional perception capability of rotating targets;

[0066] (3) By fusing the features of the two paths through the cross-attention mechanism, texture-direction information complementarity is achieved, thereby enhancing the overall feature expression capability;

[0067] (4) The dual-path feature-enhanced remote sensing image rotation target detection network based on the constructed joint loss function can improve the fitting accuracy of the rotation perception detection box.

[0068] (5) The method has a lightweight structure and end-to-end training optimization, making it suitable for various remote sensing image rotation target detection tasks. It can effectively improve the detection accuracy and robustness of multi-directional, multi-scale, densely distributed, and small-sized targets in remote sensing images, and has good versatility and practical deployment value. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart of a remote sensing image rotation target detection method based on dual-path feature enhancement provided by an embodiment of the present invention;

[0071] Figure 2 This is a schematic diagram of the overall process framework provided in the embodiments of the present invention;

[0072] Figure 3 This is a schematic diagram of the adaptive wavelet reconstruction module provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of a multi-scale angle-guided deformable encoder provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram illustrating the detection performance of the method provided in this embodiment of the invention on the DOTA-v1.0 dataset;

[0075] Figure 6 This is a schematic diagram illustrating the detection performance of the method provided in this embodiment of the invention on the DIOR-R dataset;

[0076] Figure 7 This is a schematic diagram illustrating the detection performance of the method provided in this embodiment of the invention on the UAV-ROD dataset. Detailed Implementation

[0077] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0078] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0079] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0080] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0081] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0082] This invention provides a method for detecting rotating targets in remote sensing images based on dual-path feature enhancement, such as... Figure 1 As shown, the processing flow of this method may include the following steps:

[0083] This invention provides a method for detecting rotating targets in remote sensing images based on dual-path feature enhancement, which may include the following steps:

[0084] S1. Construct a dual-path feature enhancement network for remote sensing image rotation target detection, including texture enhancement paths and orientation modeling paths, such as... Figure 2 As shown, the dual-path feature enhancement remote sensing image rotation target detection network includes: a backbone network, a feature pyramid network, a dual-path feature enhancement framework, and a target detection head; wherein,

[0085] The backbone network is used to extract multi-scale features from the input remote sensing images;

[0086] Feature pyramid network is used to enhance the semantic expressive power of features at different scales;

[0087] A dual-path feature enhancement framework is used to process the features output by the backbone network and the feature pyramid network to enhance texture details and edge features, as well as model spatial alignment and orientation consistency.

[0088] The target detection head is used to process the features output by the dual-path feature enhancement framework to complete the classification and regression of rotating targets in remote sensing images.

[0089] In this embodiment, the backbone network is constructed using a hierarchical structure, sequentially extracting feature representations at different semantic levels. The backbone network can employ network architectures such as ResNet50 or ResNet101, sequentially outputting feature maps in four stages (C2–C5), corresponding to the spatial scale changes from shallow to deep layers. The output configuration of each stage of the backbone network is shown in Table 1.

[0090] Table 1 Output Configuration of Each Stage of the Backbone Network

[0091]

[0092] In Table 1, H represents the height of the feature map.

[0093] In this embodiment, the feature pyramid network is used to enhance the semantic information transmission capability between features in different layers. Its core lies in combining a top-down structure with lateral connections to improve the semantic expressiveness of shallow features while preserving the spatial detail information of high-resolution features. The output configuration of each layer of the feature pyramid network is shown in Table 2.

[0094] Table 2 Output configuration of each layer of the feature pyramid network

[0095]

[0096] In Table 2, W represents the width of the feature map.

[0097] In this embodiment, the dual-path feature enhancement framework includes two complementary paths—a texture enhancement path and a direction modeling path—as well as a cross-attention mechanism fusion scheme; wherein,

[0098] 1) In the texture enhancement path, an adaptive wavelet reconstruction module is used to enhance the texture details and edge features in the input feature map.

[0099] In this embodiment, the adaptive wavelet reconstruction module includes:

[0100] The Discrete Wavelet Transform submodule is used to perform multi-level discrete wavelet transform on the input shallow high-resolution feature map to extract high-frequency features containing texture and edge information as well as low-frequency features containing semantic structure.

[0101] An adaptive weighted fusion submodule is used to dynamically fuse the high-frequency features and low-frequency features based on the response intensity, so as to improve the structural integrity of the features;

[0102] The Selective Residual Attention submodule is used to enhance the structural representation of the target region, suppress background redundancy, and output a feature map with enhanced details based on fused features.

[0103] In this embodiment, for the shallow high-resolution feature maps P3 and P4, a discrete wavelet transform submodule is used to perform a discrete wavelet transform on the feature map C2 extracted by the backbone network to obtain high-frequency and low-frequency components (also referred to as high-frequency and low-frequency features). The high-frequency components are used to capture edge and texture information, while the low-frequency components contain semantic structure information. Subsequently, an adaptive weighted fusion submodule dynamically fuses the high- and low-frequency features, and a selective residual attention submodule is introduced to enhance the texture structure of the target region while suppressing redundant responses in the background region, outputting a feature map with enhanced details. Figure 3 As shown, the specific steps may include:

[0104] B1: Perform two-stage discrete wavelet transform operations on the feature map C2 extracted from the backbone network to obtain high- and low-frequency components at different resolutions, preparing for the enhancement of layers P3 and P4:

[0105]

[0106] Where DWT(·) represents the discrete wavelet transform, This represents the low-frequency components after the first-order wavelet transform. This represents the high-frequency components after the first-order wavelet transform, with dimensions corresponding to P3; This represents the low-frequency component after the second-order wavelet transform. This represents the high-frequency components after the second-order wavelet transform, with dimensions corresponding to P4. Among them, Represents high-frequency information in the horizontal direction. Represents high-frequency information in the vertical direction. This represents high-frequency information in the diagonal direction, and C represents the number of channels. To simplify subsequent fusion operations, the high frequencies in each direction are summed to obtain:

[0107]

[0108] Where H1 and H2 represent the high-frequency components after the first and second wavelet transforms, respectively;

[0109] B2: Taking P3 as an example, L1 and H1 are guided by 1×1 convolution, GN, ReLU and sigmoid activation to automatically adjust the weights of different frequency components, achieving adaptive weighted fusion of high-frequency and low-frequency features, and then normalized to stabilize the fusion.

[0110] α L =σ(ReLU(GN(Conv) 1×1 (L1))))

[0111] α H =σ(ReLU(GN(Conv) 1×i (H1))))

[0112]

[0113] Where σ(·) represents the sigmoid activation function, ReLU represents the activation function, GN represents group normalization, Conv represents the convolution operation, ε is a parameter to prevent the denominator from being zero, ε = 1e-6, α L ,α H These represent the weights of the low-frequency and high-frequency components, respectively. These represent the weights of the low-frequency and high-frequency components after normalization, respectively.

[0114] B3: The fused feature combines high-frequency and low-frequency information, represented as:

[0115]

[0116] in, ⊙ indicates fusion feature, and ⊙ indicates element-wise multiplication.

[0117] B4: Computational Fusion Features Residual between the original input feature P3 and the original input feature:

[0118]

[0119] Where R3 represents the residual;

[0120] B5: Generate a spatial-channel joint attention map using the Convolutional Block Attention Module (CBAM). A3:

[0121] A3 = CBAM(P3)

[0122] in, Generate saliency guidance for each location in the input feature map.

[0123] B6: Using A3 to selectively enhance the residual R3, we obtain:

[0124]

[0125] in, This represents the feature map after detail enhancement;

[0126] B7: Controlled using a learnable residual weighting factor λ The injection intensity is then fused with the original input feature P3, and the information is further integrated through convolutional blocks to obtain the final feature map with enhanced details, represented as:

[0127]

[0128] in, This represents the feature map after detail enhancement.

[0129] In this embodiment, the adaptive wavelet reconstruction module effectively preserves the main feature structure and selectively enhances only the target region, thereby improving the ability of the dual-path feature-enhanced remote sensing image rotating target detection network to perceive small targets and boundary details, and avoiding the introduction of unnecessary noise interference.

[0130] For P4, this embodiment also adopts the above strategy, using L2 and H2 obtained from the second-level wavelet transform as input to complete the supplementation and fusion of structural information at the corresponding scale, and obtain the feature map after detail enhancement.

[0131] In summary, in the texture enhancement path, a discrete wavelet transform module is used to perform multi-level discrete wavelet transform on shallow high-resolution features to extract high-frequency and low-frequency information. Then, an adaptive weighted fusion submodule is used to dynamically weight and fuse the extracted high-frequency and low-frequency information. Finally, a selective residual attention module is used to enhance the structural representation of the target region and suppress background redundancy, thereby achieving precise enhancement of the target region structure and improving the perception capability of small targets.

[0132] 2) In the orientation modeling path, a multi-scale angle-guided deformable encoder is used to perform spatial alignment and orientation consistency modeling on the input feature map, and extract features containing structural and orientation information.

[0133] In this embodiment, the multi-scale angle-guided deformable encoder includes:

[0134] The lightweight angle prediction module is used to predict pixel-level angle information from the input feature map and generate an angle guidance map to provide target orientation or direction cues in spatial location.

[0135] A rotation-sensitive visual encoder is used to co-model the angle guide map and the input feature map, extract orientation-sensitive features, and generate a rotation-sensitive visual feature map.

[0136] The visual-angle cross-attention module is used to achieve spatial alignment of rotation-sensitive visual features and angle features within rotation-sensitive regions, generating multi-scale feature maps with enhanced directional consistency.

[0137] In this embodiment, to improve the ability of the dual-path feature-enhanced remote sensing image rotation target detection network to model target orientation information during the feature encoding stage, this embodiment proposes a multi-scale angle-guided deformable encoder. By explicitly introducing an angle guidance map, it guides features to focus on rotation-sensitive regions, achieving spatial alignment and orientation consistency modeling. Figure 4 As shown, the specific steps may include:

[0138] C1: Multi-scale feature map output by the feature pyramid network A lightweight angle prediction module generates the corresponding angle guidance map. in, This indicates the target's orientation or direction at each spatial location. The lightweight angle prediction module consists of two stacked convolutional units, each containing a convolution and a non-linear activation function. Each convolution maps features to a one-dimensional angle prediction.

[0139] Θ l =f pred (P l )

[0140] Among them, f pred (·) indicates the lightweight angle prediction module, with input channel C and output channel 1.

[0141] C2: In order to effectively incorporate angle information into the feature modeling process, the angle guidance graph Θ is first processed. l Perform channel expansion to make it compatible with P l Alignment along the channel dimension generates angular features, represented as follows:

[0142]

[0143] in, Indicates angular features, f enc (·) represents an angle encoder consisting of two layers of convolution, normalization, and nonlinear activation. It has 1 input channel and C output channel, and is capable of channel expansion and directional semantic extraction.

[0144] C3: To guide the dual-path feature-enhanced remote sensing image rotation target detection network to more fully perceive the structural information surrounding the target, the following steps are taken: Spatial perception modeling f was introduced cxt (·), used to construct more discriminative spatial context-enhanced feature maps:

[0145]

[0146] in, Represents spatial context-enhanced feature maps, This module extracts local spatial features through depthwise separable convolution and integrates information between channels using pointwise convolution, thereby effectively modeling local structure and spatial distribution information.

[0147] C4: F θ With F cxt Element-wise summation is performed, and global average pooling is applied to extract the spatial mean of each channel as its global representation. Channel attention weights are then generated through non-linear mapping, as follows:

[0148]

[0149] Among them, G l Indicates channel attention weights. GAP(·) represents global average pooling, and MLP(·) represents channel-by-channel mapping.

[0150] C5: Modulate the angle feature using a weighting factor μ and inject it into the original input feature P. l In this process, the final rotation-sensitive visual feature map is formed:

[0151]

[0152] in, This represents rotation-sensitive visual features, where μ is actually a learnable scaling parameter used to adjust the injection intensity of the enhancement signal.

[0153] C6: To further enhance the spatial alignment and orientation consistency modeling capabilities of rotating targets, this embodiment introduces a visual-angle cross-attention mechanism. Combining this with the spatial sampling advantages of MS-Deformable Attention (MS-Deformable Attention Scale), it achieves orientation-guided multi-scale dynamic sampling and feature aggregation in rotation-sensitive regions. This mechanism utilizes rotation-sensitive visual features... Construct a query to encode the angular features. Used as the Value, dynamic sampling of local regions is performed through reference points and learnable offsets to achieve spatial alignment between rotation-sensitive visual features and angular features, generating multi-scale feature maps with enhanced orientation consistency. Compared to traditional attention mechanisms that need to construct and match the Key across the entire feature map, MS-Deformable Attention only performs sparse sampling at a small number of key locations within the neighborhood of each reference point, effectively avoiding global computation, significantly reducing complexity, and improving spatial alignment. Specifically, it can include the following steps:

[0154] C61: For Positional encoding and learnable hierarchical embeddings are added to enhance spatial awareness and cross-layer discrimination capabilities, which are then flattened into a unified multi-scale token sequence as input for cross-attention.

[0155]

[0156]

[0157] Among them, Q l V represents the query feature of layer l. l The PE represents the Value feature of the l-th layer, and Flatten(·) represents flattening the two-dimensional feature map into a one-dimensional token sequence. l It is a positional encoding, LE l For learnable hierarchical embeddings, Concat(·) concatenates feature sequences from different levels along the token dimension. Q represents the concatenated features of the Queries from all levels, V represents the concatenated features of the Values ​​from all levels, N is the total number of feature points across all levels, and H... l W represents the height of the feature map at layer l. l This represents the width of the feature map at layer l.

[0158] C62: Subsequently, sampling points are generated using learnable offsets, and MS-Deformable Attention applies each query to its reference point P. ref K neighborhood samples are taken, and the features of the corresponding positions in Value are aggregated through attention weights to form Z:

[0159] Z = MSDeformAttn(Q,P) ref V)

[0160] C63: Then, the feature sequence Z ' Input the feedforward neural network and perform further modeling and optimization using layer normalization:

[0161] Z′=LN(Q+Z)

[0162] Z"=LN(Z′+FFN(Z′))

[0163] Where LN(·) represents layer normalization, Z ' FFN(·) represents an intermediate variable and FFN(·) represents a feedforward neural network. C64: Restore the stitched features to a multi-scale structure based on the original spatial dimensions of each layer.

[0164]

[0165] in, This represents the feature map enhanced by the multi-scale angle-guided deformable encoder. Reshape(·) represents the feature map in sequence form (token representation). Reorganized into a two-dimensional spatial structure

[0166] This mechanism can be viewed as a visual-angle-guided spatial sampling method. Visual features themselves lack clear directional information, while angular features are generated by a single-channel angle map Θ. l Encoded, it explicitly includes spatial orientation distribution. Using visual features as the query and angular features as the value, the visual features actively "retrieve" orientation-related context from the angular features, thus supplementing its orientation sensitivity. This process enables the dual-path feature-enhanced remote sensing image rotation target detection network to more effectively extract contextual regions with consistent or significantly changing orientations at the current spatial location.

[0167] In summary, the multi-scale angle-guided deformable encoder achieves explicit modeling and spatial alignment of the rotation target's orientation information. This module not only modulates the injection intensity of angle features in the channel dimension (as demonstrated in step C4), but also constructs the alignment relationship between vision and angle in the spatial dimension, effectively improving the structural expressiveness of features in rotation-sensitive regions. The resulting multi-scale feature map with enhanced orientation consistency provides a more stable and orientation-consistent feature foundation for subsequent target classification and rotation-aware bounding box regression.

[0168] Figure 4In this context, GELU (Gaussian Error Linear Unit) is the Gaussian Error Linear Unit (activation function), DWConv (Depthwise Convolution) is the depthwise convolution, and PWConv (Pointwise Convolution) is the pointwise convolution.

[0169] 3) Cross-attention mechanism fusion scheme: The features output from the two paths are input into the dual-path feature fusion module for fusion to generate multi-scale features with enhanced discriminative ability.

[0170] In this embodiment, the dual-path feature fusion module adopts a cross-attention mechanism to dynamically perceive the target region and calculate the cross-correlation response between the two paths, thereby achieving feature complementarity and directional collaborative modeling, which enhances the structural consistency and directional expression capability of the rotating target.

[0171] In this embodiment, the object detection head is used to output the final object detection result. The object detection head includes a class prediction branch and a rotation bounding box regression branch. The class prediction branch is used to predict the class of the rotated object, and the rotation bounding box regression branch is used to regress the five-parameter rotation bounding box of the object, including the center coordinates, width, height, and rotation angle of the object. The object detection head performs the classification and regression tasks of the rotated object by inputting enhanced features, and performs end-to-end training through a joint loss function.

[0172] S2. Construct a joint loss function, and train a dual-path feature-enhanced remote sensing image rotation target detection network end-to-end based on the generated multi-scale features with enhanced discriminative ability and the joint loss function.

[0173] In this embodiment, to effectively optimize the performance of rotating target detection, multiple loss branches are jointly optimized during the training phase, including three tasks: target classification, rotation-aware bounding box regression, and angle prediction. Therefore, the constructed joint loss function consists of three parts: a target classification loss based on Focal Loss to handle the foreground-background imbalance problem; a rotation-aware bounding box regression loss based on Smooth L1 Loss to optimize target position and scale prediction; and an angle regression loss to improve the accuracy of target orientation prediction, thereby optimizing the overall performance of rotating target detection. The training process may specifically include the following steps:

[0174] D1: For each positive sample anchor or predicted bounding box, this embodiment uses the Smooth L1 loss function to supervise the five-parameter rotated bounding box, including the target's center coordinates (x, y), width w, height h, and rotation angle θ.

[0175]

[0176] in, Let represent the rotation-aware bounding box regression loss of Smooth L1 Loss, where β is the smoothing factor, set to 0.11 in this embodiment. This represents the predicted rotated box parameters. For its corresponding actual annotation box, N pos This represents the number of positive samples.

[0177] D2: In remote sensing images, the proportion of foreground objects is usually much lower than that of the background area, resulting in a severe imbalance between positive and negative samples. To alleviate this problem, this embodiment employs Focal Loss in the classification branch. Its core idea is to focus the learning process on difficult samples by adjusting the loss weights of easily classified samples. The formula is as follows:

[0178]

[0179] in, This represents the Focal Loss classification loss, where N represents the number of candidate boxes involved in the classification loss calculation. α represents the probability that the i-th candidate box is predicted to be its true class, α is the class balance factor, which is set to 0.25 in this embodiment, and γ represents the degree of attention to hard samples, which is set to 2.0 in this embodiment.

[0180] D3: In addition, for the lightweight angle prediction module introduced in the multi-scale angle-guided deformable encoder module, this embodiment introduces an angle loss. The angle guidance map Θ generated by it is obtained using the Smooth L1 loss function. l By monitoring and constraining the error between the angle and the actual target, the accuracy and discriminative power of directional guidance information can be improved.

[0181] Combining the above three losses, the final total loss function for:

[0182]

[0183] Where, ω reg ω cls ω θ In this embodiment, the weight hyperparameters for the loss terms are all set to 1.0.

[0184] D4: End-to-end training is performed based on the multi-scale features with enhanced discriminative power output by the dual-path feature enhancement framework and the joint loss function. Training parameters include: stochastic gradient descent as the optimizer, an initial learning rate of 0.0025, momentum of 0.9, weight decay coefficient of 0.0001, batch size of 2, and a warm-up strategy of 500 iterations introduced in the early stages of training to alleviate training instability caused by sudden changes in the learning rate.

[0185] S3. The trained dual-path feature enhancement network for remote sensing image rotation target detection is used to detect and locate rotating targets in remote sensing images.

[0186] In this embodiment, the dual-path feature enhancement remote sensing image rotation target detection network, which has been trained, is suitable for remote sensing scenes with features such as multi-directional, multi-scale, small targets, and dense distribution in remote sensing images, including but not limited to ports, airports, wind farms, and urban building complexes. It can achieve high-precision target detection and rotation positioning under interference conditions such as rotation, occlusion, and low contrast.

[0187] In summary, the beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0188] (1) By introducing an adaptive wavelet reconstruction module, the ability to perceive shallow texture details and boundaries is enhanced, and the performance of small target detection is improved;

[0189] (2) By using a multi-scale angle-guided deformable encoder to construct a directional modeling path, extract the directional and structural information of the target, and achieve spatial structure alignment with directional consistency, thereby improving the modeling effect and directional perception capability of rotating targets;

[0190] (3) By fusing the features of the two paths through the cross-attention mechanism, texture-direction information complementarity is achieved, thereby enhancing the overall feature expression capability;

[0191] (4) The dual-path feature-enhanced remote sensing image rotation target detection network based on the constructed joint loss function can improve the fitting accuracy of the rotation perception detection box.

[0192] (5) The method has a lightweight structure and end-to-end training optimization, making it suitable for various remote sensing image rotation target detection tasks. It can effectively improve the detection accuracy and robustness of multi-directional, multi-scale, densely distributed, and small-sized targets in remote sensing images, and has good versatility and practical deployment value.

[0193] To verify the effectiveness of the remote sensing image rotation target detection method based on dual-path feature enhancement provided in this embodiment, the performance of the method is tested using three datasets: DOTA-v1.0, DIOR-R, and UAV-ROD. The detection results are illustrated in the diagram below. Figure 5 - As shown in Figure 7. Final performance is evaluated using the mean accuracy (mAP) metric: mAP is a commonly used performance metric in object detection, reflecting the comprehensive detection capability of a dual-path feature-enhanced remote sensing image rotation target detection network across all categories. Specifically, mAP is calculated by first calculating the mean accuracy (AP) for each category at a set IoU threshold, and then averaging the APs across all categories.

[0194] To verify the performance of the method described in this embodiment, this embodiment selected remote sensing image rotation target detection methods in recent years for comparison, and the experimental results are shown in Tables 3, 4 and 5 respectively.

[0195] Table 3 shows the detection results on the DOTAv1.0 dataset.

[0196]

[0197] In Table 3, * indicates that the method uses a multi-scale training and testing strategy.

[0198] Table 4 shows the detection results on the DIOR-R dataset.

[0199] method mAP RetinaNet-O[] 57.55 Faster RCNN-O[] 59.54 Gliding Vertex[] 60.06 Rol-Trans[] 63.87 O-RepPoints[] 66.71 LSKNet-S[] 65.90 PKINet-S[] 67.03 Ours (R50) 68.60 Ours (R101) 68.88

[0200] Table 5 shows the detection results on the UAV-ROD dataset.

[0201] method mAP R-RetinaNet[] 97.68 R-Faster RCNN[] 98.07 TS4Net[] 98.34 Ours (R50+GWD) 98.36 Ours (R101+GWD) 98.71

[0202] To verify the effectiveness of the two core modules (adaptive wavelet reconstruction module and multi-scale angle-guided deformable encoder) in the proposed dual-path feature enhancement framework, this embodiment designed four ablation experiments on the DOTA-v1.0 dataset to evaluate the independent effects and synergistic gains of the adaptive wavelet reconstruction module and the multi-scale angle-guided deformable encoder. All experiments were based on a unified training strategy, using ResNet50 as the backbone and the S2ANet detection framework as the baseline model. The final performance was evaluated using mean accuracy (mAP). The experiments employed single-scale input, with joint training on the training and validation sets, and testing on the official test set. The experimental configuration and results are shown in Table 6.

[0203] Table 6 Ablation Experiment

[0204] Experiment 1 Experiment 2 Experiment 3 Experiment 4 Adaptive wavelet reconstruction module √ √ Multi-scale angle-guided deformable encoder √ √ mAP 74.12 77.31 77.08 77.44

[0205] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0206] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0207] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0208] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0209] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0210] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0211] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0214] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0215] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting rotating targets in remote sensing images based on dual-path feature enhancement, characterized in that, The method includes: S1. Construct a dual-path feature enhancement network for remote sensing image rotation target detection, which includes texture enhancement path and orientation modeling path; whereby, In the texture enhancement path, an adaptive wavelet reconstruction module is used to enhance the texture details and edge features in the input feature map; where the feature map is extracted from remote sensing images. In the orientation modeling path, a multi-scale angle-guided deformable encoder is used to perform spatial alignment and orientation consistency modeling on the input feature map, and extract features containing structural and orientation information. By fusing the features output from the two paths through a cross-attention mechanism, multi-scale features with enhanced discriminative capabilities are generated. The multi-scale angle-guided deformable encoder includes: The lightweight angle prediction module is used to predict pixel-level angle information from the input feature map and generate an angle guidance map to provide target orientation or direction cues in spatial location. A rotation-sensitive visual encoder is used to co-model the angle guide map and the input feature map, extract orientation-sensitive features, and generate a rotation-sensitive visual feature map. The visual-angle cross-attention module is used to achieve spatial alignment of rotation-sensitive visual features and angular features within rotation-sensitive regions, generating multi-scale feature maps with enhanced orientation consistency. The lightweight angle prediction module is used to analyze the multi-scale feature map output from the feature pyramid network. Predict pixel-level angle information and generate corresponding angle guidance maps. To provide target orientation or direction cues for each spatial location; where, ； in, This indicates a lightweight angle prediction module; Specifically, the rotation-sensitive visual encoder is used for the angle-guided image. Perform channel expansion to make it compatible with Alignment along the channel dimension generates angular features, represented as follows: ； in, Indicates angular features, This represents an angle encoder consisting of two layers of convolution, normalization, and nonlinear activation. right Introducing spatial perception modeling Construct more discriminative spatial context-enhanced feature maps , represented as: ； Will and Element-wise summation is performed, and global average pooling is applied to extract the spatial mean of each channel as its global representation. Channel attention weights are then generated through non-linear mapping. , represented as: ； in, This represents the sigmoid activation function. Indicates global average pooling. Indicates a channel-by-channel mapping; Using weighting factors The angle features are modulated and injected into the original input features. In this process, the final rotation-sensitive visual feature map is formed. , represented as: ； Specifically, the vision-angle cross-attention module is used to focus on rotation-sensitive visual features. Construct a query to encode the angular features. Used as Value, it dynamically samples local regions through reference points and learnable offsets to achieve spatial alignment of rotation-sensitive visual features and angular features, generating multi-scale feature maps with enhanced orientation consistency; S2. Construct a joint loss function, and train a dual-path feature-enhanced remote sensing image rotation target detection network end-to-end based on the generated multi-scale features with enhanced discriminative ability and the joint loss function; wherein, the joint loss function includes: classification loss, rotation-aware detection box regression loss and angle regression loss; S3. The trained dual-path feature enhancement network for remote sensing image rotation target detection is used to detect and locate rotating targets in remote sensing images.

2. The remote sensing image rotation target detection method based on dual-path feature enhancement according to claim 1, characterized in that, The dual-path feature enhancement remote sensing image rotation target detection network includes: a backbone network, a feature pyramid network, a dual-path feature enhancement framework, and a target detection head; the dual-path feature enhancement framework includes a texture enhancement path and a orientation modeling path; wherein... The backbone network is used to extract multi-scale features from the input remote sensing images; Feature pyramid network is used to enhance the semantic expressive power of features at different scales; A dual-path feature enhancement framework is used to process the features output by the backbone network and the feature pyramid network to enhance texture details and edge features, as well as model spatial alignment and orientation consistency. The target detection head is used to process the features output by the dual-path feature enhancement framework to complete the classification and regression of rotating targets in remote sensing images.

3. The remote sensing image rotation target detection method based on dual-path feature enhancement according to claim 1, characterized in that, The adaptive wavelet reconstruction module includes: The Discrete Wavelet Transform submodule is used to perform multi-level discrete wavelet transform on the input shallow high-resolution feature map to extract high-frequency features containing texture and edge information as well as low-frequency features containing semantic structure. An adaptive weighted fusion submodule is used to dynamically fuse the high-frequency features and low-frequency features based on the response intensity, so as to improve the structural integrity of the features; The selective residual attention submodule is used to enhance the structural representation of the target region, suppress background redundancy, and output a feature map with enhanced details based on fused features.

4. The remote sensing image rotation target detection method based on dual-path feature enhancement according to claim 3, characterized in that, For the feature map of the P3 level output by the feature pyramid network The fusion features are represented as: ； ； ； ； ； in, Indicates fusion characteristics; This represents element-wise multiplication; This represents the sigmoid activation function; To prevent parameters with a denominator of 0; ReLU represents the activation function; GN represents group normalization; Conv represents the convolution operation; , These represent the weights of the low-frequency and high-frequency components, respectively. , These represent the weights of the low-frequency and high-frequency components after normalization, respectively. , These represent the low-frequency and high-frequency components after the first-order wavelet transform, respectively.

5. The remote sensing image rotation target detection method based on dual-path feature enhancement according to claim 4, characterized in that, The selective residual attention submodule is specifically used to calculate the fused features. Compared with the original input features The residuals between: ； ； in, Represents the residual. The feature map at level P3, representing the output of the feature pyramid network; Generate spatial-channel joint attention map using convolutional block attention modules : ； Wherein, CBAM() represents the convolutional block attention module; use For residuals By selectively enhancing the effect, we obtain: ； in, This represents the feature map after selective enhancement. Use residual weighting factor control Injection strength, and compared with original input features The information is fused through convolutional blocks to obtain the final feature map with enhanced details, represented as follows: ； in, This represents the feature map after detail enhancement.

6. The remote sensing image rotation target detection method based on dual-path feature enhancement according to claim 1, characterized in that, The classification loss is a Focal Loss-based classification loss used to address the foreground-background imbalance problem. The rotation sensing detection box regression loss is a Smooth L1 Loss-based rotation sensing detection box regression loss, which is used to optimize target position and scale prediction. The angle regression loss is used to improve the accuracy of target direction prediction.

Images