A remote sensing target detection method and system based on structure tensor guidance

Abstract

A remote sensing target detection method and system based on structural tensor guidance. The method comprises: acquiring a remote sensing image to be detected and extracting multi-scale deep semantic features; in the feature extraction stage, a residual learnable structural tensor extraction module is constructed, the local coherent energy is calculated by using gradient covariance and residual compensation, so that the geometric edge prior of low-dissipation rigid target is explicitly reserved; in the feature fusion stage, a dynamic feature refining module is constructed, the affine modulation parameters are predicted by using the structural prior, the target geometric boundary blurred due to network down-sampling is refined by pixel-level spatial modulation and global gating; finally, the network parameters are optimized and the rotation detection result with angle information is output. The present application constructs a cooperative processing closed loop of "physical prior extraction" and "semantic feature refining", effectively solves the problems of easy loss of remote sensing target geometric structure and serious interference of high-entropy natural background noise, and significantly improves the positioning and detection accuracy of dense rotating targets.

Description

Technical Field

[0001] This invention relates to a computer vision target detection technology based on deep learning and physical priors, belonging to the fields of remote sensing image processing and artificial intelligence applications, specifically relating to a remote sensing target detection method and system based on structure tensor guidance. Background Technology

[0002] With the development of aerospace technology, remote sensing images are increasingly widely used in urban planning, traffic monitoring, military reconnaissance, and other fields. Remote sensing images typically feature large image sizes, complex backgrounds, small targets, and arbitrary orientations. Conventional data-driven target detection methods based on convolutional neural networks or Transformers face significant challenges in complex scenarios.

[0003] From the perspective of physical systems and thermodynamics, man-made rigid targets (such as airplanes, ships, and buildings) typically have straight edges and low entropy, while natural backgrounds (such as vegetation and rivers) undergo long-term energy dissipation processes, exhibiting a smooth, curved, and isotropic high-entropy state. However, existing deep neural networks inevitably lead to feature diffusion and spatial ambiguity during the process of extracting high-level semantic features through layer-by-layer spatial downsampling, thus completely losing the crucial geometric priors that distinguish between "man-made rigid targets" and "natural dissipative backgrounds." This makes existing networks highly susceptible to false detections when facing high-frequency complex backgrounds and makes it difficult to achieve high-precision rotational bounding box localization.

[0004] Therefore, how to extract the low-dissipation geometric structure of a target within an end-to-end deep learning framework using prior knowledge of physics (such as structural tensors), and use this to guide the dynamic refinement of deep semantic features, thereby achieving high-precision remote sensing rotating target detection, has become a key problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing remote sensing image rotation target detection methods, such as the lack of explicit physical geometric constraints, the ambiguity of structural information caused by multiple downsampling, and the susceptibility to interference from natural high-entropy backgrounds. This invention proposes a remote sensing target detection method and system based on structural tensor guidance.

[0006] This invention, from the perspective of physical energy dissipation, constructs a collaborative processing mechanism of "residual learnable structure tensor" and "dynamic feature refinement":

[0007] On the one hand, this invention utilizes the property that classical structure tensors are mathematically equivalent to calculating the local "coherent energy" of an image to design a residual learnable structure tensor extraction module. This module directly extracts the physical structure prior from the input image and compensates for non-ideal deformations and noise through residual learning branches, thereby explicitly preserving the geometric edges of low-dissipation rigid targets.

[0008] On the other hand, this invention designs a dynamic feature refinement module. This module transforms the extracted structural tensor prior into spatial affine modulation parameters, performing pixel-level modulation on the multi-scale deep semantic features extracted by the backbone network. This mechanism of "physical structure guiding semantics" effectively filters out high-dissipation natural background noise and accurately restores the blurred geometric boundaries of the target.

[0009] The present invention is implemented using the following specific technical solutions.

[0010] A remote sensing target detection method based on structure tensor guidance includes the following steps:

[0011] Step 1: Obtain the remote sensing image to be detected as input data.

[0012] Obtain raw remote sensing images containing targets in arbitrary directions as the network input matrix. The image contains not only small, dense, rigid targets (such as low-dissipation areas like airplanes and ships), but also high-frequency natural background noise (such as high-entropy dissipation areas like ocean waves and woodlands).

[0013] Step 2: Multi-scale feature extraction.

[0014] The remote sensing image to be detected is fed into the backbone network, and high-level semantic features are extracted through layer-by-layer convolution and spatial downsampling. Then, multi-scale top-down and bottom-up fusion is performed through the neck network (such as the Feature Pyramid Network, FPN) to obtain a multi-scale deep semantic feature set containing different receptive fields. .

[0015] Step 3: Extract prior features of physical structure using the learnable structure tensor extraction module.

[0016] To compensate for the low-dissipation geometric edge features lost in step 2 due to irreversible downsampling, a physical feature extraction branch is constructed in parallel with the backbone network, which includes the following sub-steps:

[0017] Step 3.1: Gradient awareness.

[0018] Calculate the horizontal gradient of the input image using the Sobel operator. and vertical gradient .

[0019] Step 3.2: Tensor construction.

[0020] Calculate the pixel-level original structure tensor based on gradient information. Its definition is:

[0021]

[0022] Step 3.3: Residual reconstruction and pooling.

[0023] To accommodate the complex deformation of remotely sensed targets and compensate for the difference between the ideal physical tensor and the actual feature distribution, a parallel residual branch is introduced. Perform deterministic local integration. The final integral tensor The calculation formula is:

[0024]

[0025] in This is a 5×5 average pooling operation used to capture the ideal macroscopic topology; This is a lightweight convolutional network module containing residual connections, used to learn nonlinear deformation compensation.

[0026] Step 3.4: Mapping to generate structural features.

[0027] Through multilayer perceptron low-dimensional tensors Mapped to high-dimensional structural prior features ,Right now

[0028]

[0029] Step 3.5: Downsampling spatial alignment.

[0030] Prior features of structure Spatial downsampling is performed to match the dimensions of the multi-scale feature maps at different levels in step 2.

[0031] Step 4: Affine modulation parameter prediction.

[0032] In the dynamic feature refinement stage, the aligned structural features are utilized. A lightweight multilayer perceptron network is used to predict the spatial scaling factor for pixel-level feature modulation. and bias terms ,Right now:

[0033]

[0034] MLP stands for Multilayer Perceptron.

[0035] Step 5: Affine modulation.

[0036] The predicted physical prior parameters are applied to multi-scale deep features. To achieve feature fusion modulation and obtain modulation features :

[0037]

[0038] in This indicates element-wise multiplication. The sigmoid activation function is used to constrain the response of the scaling factor within a reasonable range to ensure training stability. This step forces the network's attention to be anchored to a region of artificial, rigid targets with high coherence energy.

[0039] Step 6: Refine feature decoding.

[0040] To capture the global context and further filter the background, modulation features Perform space refinement function Processing and combining with Global Average Pooling (GAP) to generate channel-gated weights. :

[0041]

[0042] In this context, MLP stands for Multilayer Perceptron, and GAP stands for Global Average Pooling. This is the Sigmoid activation function. The final output is a refined multi-scale feature fully guided by the structure tensor. :

[0043]

[0044] Step 7: Rotate the bounding box and predict the category.

[0045] The refined features are fed into the detection head, which outputs the target's class probability and a rotated bounding box with angle information. During the training phase, the model parameters are optimized by combining classification loss and rotated bounding box regression loss. During the inference phase, the trained end-to-end network directly outputs the target's class and rotation position in the remote sensing image.

[0046] Based on the above method, this invention further proposes a remote sensing target detection system guided by structural tensors, including a data preprocessing subsystem, a multi-scale feature extraction subsystem, a learnable structural tensor extraction subsystem, a dynamic feature refinement subsystem, and a rotating target detection subsystem.

[0047] The connections between the above components are as follows:

[0048] The output of the data preprocessing subsystem is connected to the input of the multi-scale feature extraction subsystem and the learnable structure tensor extraction subsystem, respectively; the outputs of the multi-scale feature extraction subsystem and the learnable structure tensor extraction subsystem are connected to the input of the dynamic feature refinement subsystem; the output of the dynamic feature refinement subsystem is connected to the input of the rotating target detection subsystem.

[0049] Beneficial effects

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] 1. Explicit Structure Preservation with Physics Interpretation: This invention breaks through the limitations of traditional data-driven networks. Based on the physics of energy dissipation, it introduces the classical structure tensor as a mathematical implementation of coherent energy into deep networks for the first time. This mechanism can accurately lock low-entropy artificial rigid targets (such as straight line edges and corners) and naturally exclude high-entropy natural random backgrounds.

[0052] 2. Robust Residual Learnable Design: Unlike traditional hard-coded operators, the learnable structure tensor extraction module of this invention decouples deterministic gradient integrals from learnable nonlinear residual compensation, which not only retains the strong physical inductive bias of measurement energy dissipation, but also adapts to complex object deformation and non-ideal lighting noise in remote sensing images.

[0053] 3. Precise Background Filtering and Dynamic Feature Refinement: This invention transforms the structural tensor prior into affine parameters, performs pixel-level modulation of deep features, and combines a channel gating mechanism to force the network to focus on high-coherence-energy regions. This design fundamentally solves the spatial blurring problem caused by downsampling in traditional networks, significantly improving the localization accuracy of densely sensed and rotating targets under strict IoU thresholds. Attached Figure Description

[0054] Figure 1 This is the overall flowchart of the method of the present invention.

[0055] Figure 2 This is a schematic diagram of the core algorithm model described in the method of this invention.

[0056] Figure 3 This is a block diagram of the components of the system of the present invention. Detailed Implementation

[0057] To better illustrate the purpose and advantages of the present invention, the method of the invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0058] Example

[0059] This invention proposes a remote sensing target detection method guided by structure tensors, aiming to address the problems of blurred target geometry and low positioning accuracy in complex, high-dissipation backgrounds encountered by existing methods. The core of this method lies in introducing a structure tensor based on physical coherence energy to explicitly extract the low-dissipation rigid edges of the target, and using these as priors to dynamically refine deep semantic features. Combined with... Figure 1 and Figure 3 The specific implementation process of this embodiment can be broken down into the following steps:

[0060] Step S1: Input the remote sensing image to be detected. Obtain a remote sensing image containing targets in any direction as network input. Such images typically contain not only tiny, densely packed rigid targets (such as airplanes on the tarmac or ships in the port), but also high-frequency natural background noise (such as ocean waves, complex woodlands, and other high-entropy dissipation areas).

[0061] Step S2: Multi-scale semantic feature extraction. The specific implementation is as follows:

[0062] Step S2.1: Input image The data is fed into a standard backbone network (such as ResNet or Swin Transformer) and subjected to layer-by-layer convolution and spatial downsampling to extract feature maps with high-level semantic information.

[0063] Step S2.2: Utilize a neck network (such as a Feature Pyramid Network, FPN) to perform multi-scale top-down and bottom-up fusion of the extracted features, outputting a multi-scale semantic feature set containing different receptive fields. Although these features possess rich semantics for distinguishing target categories, the feature diffusion effect caused by the irreversible downsampling process severely blurs the physical geometric boundaries and corners of the targets, making it difficult to support subsequent high-precision bounding box regression.

[0064] Step S3: Extract prior features of the physical structure using the residual learnable structure tensor extraction module. To compensate for the low-dissipation geometric edge information lost in step S2, it is directly extracted from the original image. The physical feature extraction branch is constructed in parallel with the backbone network, specifically including the following sub-steps:

[0065] Step S3.1: Gradient-aware calculation. The Sobel operator is applied to the input image in both the horizontal and vertical directions. Perform convolution filtering to obtain the horizontal gradient strength. and vertical gradient intensity Simultaneously, the quadratic term of the gradient is calculated. , and the cross-product term representing the consistency of gradient direction. .

[0066] Step S3.2: Construction of the original tensor. The three independent components calculated above are concatenated along the channel dimension to construct a pixel-level original structure tensor. This tensor matrix is ​​mathematically equivalent to measuring the local coherence energy of an image, and can sensitively capture the high coherence of the straight edges of rigid targets.

[0067] Step S3.3: Residual Reconstruction and Integration. Reconstruct the original structure tensor... The data is fed into a dual-branch integral network: the main branch uses average pooling layers to perform deterministic local integration within the spatial neighborhood to capture the ideal macroscopic topology; to adapt to the complex viewpoint deformation of remotely sensed targets, the auxiliary residual branch uses a lightweight convolutional network. Learn nonlinear deformation compensation. Then, sum the outputs of the two branches element-wise to obtain the final integral tensor. :

[0068]

[0069] Step S3.4: Structural Feature Mapping. The low-dimensional integral tensor is mapped using a multilayer perceptron. Mapping to a high-dimensional feature space generates learnable structural features rich in deep geometric context. .

[0070] Step S3.5: Downsampling spatial alignment. For high-dimensional structural features... Perform convolution or pooling operations with strides of different magnitudes to generate a multi-scale semantic feature set compared to the output of step S2. Structural feature sets with strictly consistent spatial resolution .

[0071] Step S4: Predict affine modulation parameters. Proceed to the dynamic feature refinement module. Semantic features for each scale. and the corresponding downsampled and aligned structural features Structural features The input is independently fed into a lightweight multilayer perceptron. This network dynamically predicts a set of spatially dependent affine transformation parameters based on physical edge priors: scaling factors. and bias terms The calculation formula is: .

[0072] Step S5: Perform pixel-level affine modulation. Utilize the spatial scaling factor predicted in step S4. and bias terms For deep semantic features that are blurred due to downsampling Pixel-level spatial affine modulation calculations are performed to obtain preliminary fusion features. The calculation formula is:

[0073]

[0074] in, This indicates element-wise multiplication. The sigmoid activation function is used to limit the response of the scaling factor within a reasonable range to ensure numerical stability. This modulation step forces the network attention to be "anchored" to a rigid target region with high coherence energy, explicitly recovering the target contour of high-level semantic features.

[0075] Step S6: Refined Feature Decoding and Global Gating. To capture the global context and completely remove high-entropy background responses caused by lighting, waves, etc., the modulated features are... The space and passageway are refined in sequence:

[0076] In the spatial dimension refinement stage, features The input is fed into a bottleneck refining network consisting of dimensionality reduction, feature extraction, and dimensionality increase. After aggregating local contextual information, residual connections are established with the original features to obtain refined features. In the global channel gating stage, intermediate features are input into the global gating network. First, global average pooling is performed to extract the global scene vector, then it is processed by a multilayer perceptron, and finally activated by a sigmoid function to generate channel-wise attention gating weights.

[0077]

[0078] In this context, MLP stands for Multilayer Perceptron, and GAP stands for Global Average Pooling. Channel weights are then... With refining characteristics Element-wise multiplication and weighting are performed along the channel dimension to finally output a refined multi-scale feature that is denoised, sharpened, and enhanced. .

[0079] Step S7: Output rotated bounding boxes and class predictions. This involves combining fully refined multi-scale features. The input features are then fed into a general-purpose rotating detection head. Since the input features have been guided and cleaned by the structure tensor with clear physical meaning, the detection network can ignore background noise interference, tightly lock onto the low-dissipation manifold boundary of the target, and finally accurately output the classification confidence of various remote sensing tilted targets and the closely fitting rotating bounding box to complete the entire detection loop.

[0080] Figure 2 This is a diagram of the core algorithm model structure. It illustrates the complete data flow from the input image to gradient calculation and tensor reconstruction in the learnable structure tensor extraction module, then to the multi-scale feature output of the backbone network, and finally to spatial affine modulation and global gating weighting through the dynamic feature refinement module. These two parts work together, from the explicit preservation of low-dissipation physical structure at the feature extraction end to the dynamic recovery of deep geometric boundaries at the feature fusion end, systematically solving the problem of blurred and missing key geometric structure information caused by continuous downsampling of the network in remote sensing rotating target detection.

[0081] Figure 3 This is a block diagram of the system components of the present invention. The present invention provides a remote sensing target detection system based on structure tensor guidance. The system consists of five main subsystems: a data preprocessing subsystem, a multi-scale feature extraction subsystem, a learnable structure tensor extraction subsystem, a dynamic feature refinement subsystem, and a rotating target detection subsystem.

[0082] Specifically, the data preprocessing subsystem performs format conversion and standardization on the raw remote sensing images, providing unified basic input data for subsequent dual-stream feature extraction. The multi-scale feature extraction subsystem extracts multi-level deep semantic visual features from the preprocessed images through a backbone network, providing a basic set of feature maps for subsequent fusion processing. The learnable structure tensor extraction subsystem includes a gradient perception and tensor construction module and a residual reconstruction module. The gradient perception and tensor construction module calculates the local coherence energy distribution from the input image and generates the original structure tensor; the residual reconstruction module, based on the target's gradient covariance distribution, performs residual compensation for nonlinear deformation while preserving the deterministic integral macrostructure, outputting high-fidelity, low-dissipation rigid edge structure prior features. The dynamic feature refinement subsystem includes an affine space modulation module and a global gated refinement module. The affine space modulation module transforms the structural prior into spatial parameters for pixel-level preliminary modulation of semantic features; the global gated refinement module, through context aggregation of spatial and channel dimensions, combats feature diffusion and blurring caused by network downsampling, restoring the high-frequency geometric boundaries of the target features. The rotating target detection subsystem is used to calculate target classification, rotating bounding box regression loss, update network parameters, and output the final remote sensing target detection results.

[0083] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing target detection method based on structure tensor guidance, characterized in that, Includes the following steps: Step 1: Acquire the remote sensing image to be detected, and extract a multi-scale semantic feature map set through the backbone network and neck network; Step 2: Construct a residual learnable structure tensor extraction module to directly extract the physical structure prior from the input image, compensate for non-ideal deformation and noise through residual learning branches, and explicitly preserve the geometric edge features of low dissipation rigid targets; Step 3: Construct a dynamic feature refinement module to convert the physical structure prior into spatial affine modulation parameters, perform pixel-level adaptive modulation and global gating refinement on the multi-scale semantic feature map, filter high dissipation background noise and restore the target geometric boundary; Step 4: Calculate the total loss function to optimize the network parameters, and output the category of the remote sensing target and the rotated bounding box with angle information based on the trained model to complete the detection.

2. The method as described in claim 1, characterized in that, The residual learnable structure tensor extraction module in step 2 includes: Gradient-aware unit: Calculates the horizontal and vertical gradients of the input image using the Sobel operator; Tensor building unit: Based on the calculated gradient information, a pixel-level original structure tensor is constructed by concatenating quadratic terms and cross-product terms; Residual reconstruction and pooling mechanism: The original structure tensor is subjected to deterministic spatial local integration using average pooling operation, while nonlinear deformation compensation is learned using convolutional branches with residual connections. The two are combined to generate the final integral tensor. Feature mapping and alignment unit: The final integral tensor is mapped to high-dimensional structural prior features through a multilayer perceptron, and spatial downsampling is performed to strictly align it with the spatial dimensions of the multi-scale semantic feature map.

3. The method as described in claim 2, characterized in that, In the tensor construction unit, the original structure tensor The calculation formula is: in, The horizontal gradient of the input image is calculated using the Sobel operator. The vertical gradient is calculated using the Sobel operator on the input image. and These are the squared terms of the horizontal and vertical gradients, respectively. This is a cross-product term.

4. The method as described in claim 2, characterized in that, In the residual reconstruction and pooling mechanism, the final integral tensor The calculation formula is: in, This represents the average pooling operation, used to capture the local covariance of the image gradient to perform deterministic integration; The convolutional network branch with residual connections is used to extract task-driven dynamic bias compensation.

5. The method as described in claim 1, characterized in that, The dynamic feature refinement module in step 3 includes: Parameter prediction unit: Utilizes a lightweight multilayer perceptron network to predict the spatial scaling factor and bias term from downsampled and aligned structural prior features; Affine Modulation Unit: Utilizes the predicted spatial scaling factor and bias term to perform pixel-level spatial affine modulation on multi-scale deep semantic features, highlighting the target contour region; The refined feature decoding unit performs spatial dimension refinement calculations on the modulated features and combines global average pooling to generate channel-level gating weights. The channel gating weights are then multiplied with the spatially refined features to output the final refined multi-scale features after background noise has been filtered out.

6. The method as described in claim 5, characterized in that, In the affine modulation unit, the modulation features The calculation formula is: in, Multi-scale deep semantic features extracted for the backbone network The predicted spatial scaling factor. For the predicted bias term, This indicates element-wise multiplication. This indicates the Sigmoid activation function to ensure scaling stability.

7. The method as described in claim 5, characterized in that, In the refined feature decoding unit, the multi-scale features are finally refined. The calculation formula is: in, For global channel gating weights, For spatially refined convolution group operations containing residual connections, This is a global average pooling operation. It is a multilayer perceptron.

8. The method as described in claim 1, characterized in that, The total loss function in step 4 for: in, This is the classification loss function, used to measure the prediction error of the target category; This is the rotation bounding box regression loss function, used to measure the bounding box localization error that includes angular information; The weighting coefficients are used to balance the two losses.

9. A remote sensing target detection system based on structure tensor guidance, characterized in that, include: Multi-scale feature extraction subsystem: used to extract a basic multi-scale feature set containing multi-level semantic information from the input image through convolution and downsampling; Learnable structure tensor extraction subsystem: includes gradient awareness, tensor construction and residual pooling modules, used to calculate the image coherence energy distribution, extract and output structural prior features representing the low dissipation rigid edges of the target; The dynamic feature refinement subsystem includes affine parameter prediction, spatial modulation and global gating modules, which are used to perform pixel-level adaptive refinement of multi-scale semantic features using structural prior features, and output enhanced refined multi-scale features. Rotated Target Detection Subsystem: This subsystem receives refined multi-scale features, calculates classification and regression losses to update network parameters, and outputs the final remote sensing target category and rotated bounding box.

Images