Lightweight remote sensing image target detection method

Abstract

The invention discloses a lightweight remote sensing image target detection method, which comprises the following steps: acquiring a remote sensing image for preprocessing, and generating an initial feature token sequence with a position code; inputting the initial feature token sequence with the position code into a pre-trained S3ML-ViT model; a backbone network in the S3ML-ViT model extracts different scale features in stages according to an input initial feature token sequence to obtain a fusion feature vector retaining the different scale features of the image; inputting the fusion feature vector into a lightweight detection head in an S3ML-ViT model, and outputting a target detection result of the remote sensing image; the target detection result comprises a target category, a bounding box position, a size and a rotation angle. According to the method, selective sampling dimensionality reduction and multi-dimensional feature enhancement are performed through the attention module based on selective linear attention and multi-level feature enhancement, the complexity of attention calculation is greatly reduced under the condition that the detection effect is not reduced, and the richness of image extraction features is reserved.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to the field of a lightweight remote sensing image target detection method. Background Technology

[0002] With the rapid development of remote sensing technology towards higher resolution, multimodality, and real-time capabilities, the demand for refined detection of surface targets is becoming increasingly urgent. Urban management requires real-time identification of illegal buildings and road encroachments; agricultural monitoring requires locating pest and disease patches and irrigation facilities; and disaster relief requires rapidly delineating affected areas—all these scenarios demand that detection models simultaneously meet the dual requirements of high-precision global modeling and lightweight edge deployment.

[0003] Traditional deep learning methods face significant bottlenecks in remote sensing target detection: while CNN-based methods boast high inference speeds, they are limited by local receptive fields, making it difficult to capture long-range correlations among scattered targets in large remote sensing scenes, leading to missed and false detections; while Transformer-based methods achieve global feature modeling through self-attention mechanisms, exhibiting strong detection accuracy in high-resolution remote sensing images, but standard self-attention... Computational complexity ( The limited number of image block tokens is a fatal flaw. When processing 2048×2048 resolution images, the number of tokens already exceeds 4 million, the inference time for a single image exceeds 8 seconds, and the video memory usage reaches 32GB, making it completely unsuitable for lightweight scenarios such as drones and satellite terminals.

[0004] Existing lightweight Transformer optimization strategies have significant limitations in adapting to remote sensing scenarios: Linearization Approximation Loses Accuracy: Methods such as Linformer reduce complexity to low-rank approximation. However, ignoring the spectral characteristics of remote sensing targets leads to the loss of discriminative features in multispectral bands and the failure of fixed window sampling: SwingTransformer's window attention mechanism uses a fixed 16×16 division, which cannot adapt to the irregular shape and size of remote sensing targets. The feature breakage problem of targets at the window edge increases the false negative rate.

[0005] Poor adaptability to large models: Directly applying linearization methods from the LLM domain is not optimized for remote sensing tasks. When processing high-resolution images with 4 million tokens, the accuracy is severely lost, and large-scale retraining is required.

[0006] More importantly, the multi-scale distribution characteristics of remote sensing targets exacerbate the contradiction between lightweight design and accuracy: small targets rely on fine-grained features, while large targets require global context. Existing methods cannot dynamically adjust the attention scale, resulting in significant differences in detection accuracy for targets of different scales.

[0007] Therefore, a lightweight attention mechanism capable of balancing linear complexity, multi-scale adaptation and modal perception ability is provided to break through the deployment bottleneck of remote sensing target detection, which is a problem that needs to be solved by those skilled in the art. SUMMARY

[0008] In view of the above problems, the present application is proposed to provide a lightweight remote sensing image target detection method to overcome the above problems or at least partially solve the above problems, reduce the attention calculation complexity through a complexity linearized feature extraction network backbone, and identify and classify the targets in the input remote sensing image through a lightweight network, thereby completing the lightweight remote sensing image target detection.

[0009] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: The embodiment of the present application provides a lightweight remote sensing image target detection method, comprising: S1, obtaining a remote sensing image for preprocessing to generate an initial feature token sequence with position encoding; S2, inputting the initial feature token sequence with position encoding into a pre-trained S3ML-ViT model; the S3ML-ViT model comprises a backbone network constructed based on a selective linear attention and a multi-level feature enhancement attention module, and a lightweight detection head; S3, the backbone network extracts different scale features in stages according to the input initial feature token sequence with position encoding, to obtain a fusion feature vector retaining different scale features of the image; S4, inputting the fusion feature vector into the lightweight detection head to output a target detection result of the remote sensing image; the target detection result comprises a target category, a bounding box position, a size and a rotation angle.

[0010] Further, the S3ML-ViT model adopts the following loss function during training:

[0011] wherein, is a focal loss for calculating the classification prediction branch of the target detection task, is a rotated bounding box loss for calculating the distance between the predicted bounding box and the target true value label corresponding to the input remote sensing image, IoU is a loss function, and are the predicted target information result and the target true value label result, respectively, is the total loss of model training.

[0012] Further, in step S1, the preprocessing specifically includes: normalization processing, tokenization processing and position coding.

[0013] Further, in step S1, the preprocessing specifically includes: normalization processing, tokenization processing and position coding. S11, acquiring a remote sensing image and performing normalization processing; wherein the remote sensing image is a multispectral image or a synthetic aperture radar image; When the remote sensing image is a multispectral image, the normalization processing specifically includes: selecting red, green, blue and near-infrared bands of the multispectral image and performing band normalization; When the remote sensing image is a synthetic aperture radar image, the normalization processing specifically includes: performing Lee filtering denoising on the synthetic aperture radar image and then converting it into a decibel value; S12, performing tokenization processing on the image data after normalization processing, slicing to generate initial tokens with an overlap rate of 25%, and slicing with different slice windows according to target size and target density to obtain initial feature tokens; S13, performing position coding on the initial feature tokens by using a sine-cosine function that fuses spatial coordinates and band indexes to obtain an initial feature token sequence with position coding.

[0014] Further, in step S2, the attention module based on selective linear attention and multi-level feature enhancement includes: an input mapping layer, a selective sampling Softmax layer, an adaptive sampling layer, a linear attention layer, a multi-level feature enhancement layer and an output fusion layer; The input mapping layer projects input features to a query, key and value space to obtain mapped feature blocks; The selective sampling Softmax layer calculates spatial saliency scores and spectral discriminability scores of each feature block and fuses them into a comprehensive score vector by weighting; The adaptive sampling layer performs three-level sampling according to the comprehensive score vector; retains the top 30% high-score feature blocks, probabilistically samples the middle 40% feature blocks, and only retains a part of the last 30% feature blocks as background reference; The linear attention layer performs low-rank approximate attention calculation on the sampled feature matrix to obtain optimized sampled features; The multi-level feature enhancement layer sequentially performs spectral feature enhancement, spatial feature enhancement and multi-scale fusion on the sampled features to obtain enhanced features; The output fusion layer outputs fused features by connecting the enhanced features and input features through residual connection, layer normalization and Dropout regularization.

[0015] Further, in the linear attention layer, the low-rank approximate attention calculation is performed using the formula:

[0016] =

[0017] =

[0018]

[0019] wherein, is the final output attention result, is the input feature channel number, is the query matrix, is the transposed key matrix after sampling, is the value matrix after sampling, is the learnable output projection matrix, is the feature matrix after linear layer mapping of the input, is the feature matrix after sampling, , and are the learnable output projection q , k and v corresponding weight matrices.

[0020] Further, in the multi-level feature enhancement layer; 1) For multispectral images, the spectral feature enhancement generates channel weights through 1D convolution and Sigmoid activation; For synthetic aperture radar images, the spectral feature enhancement adopts a residual contraction network to suppress coherent speckle noise; 2) The spatial feature enhancement adopts parallel deep separable convolution for multi-receptive field feature extraction, and uses 1x1 convolution to fuse and reduce the dimension of the parallel convolution output; a local self-attention mechanism is executed to calculate the correlation between each feature block within the window and strengthen the neighborhood semantic association; 3) The multi-scale fusion adopts a feature pyramid structure to generate 1 / 2 and 1 / 4 scale features through down-sampling with a step of 2; and performs cross-layer enhancement by residual fusion of high-level features after up-sampling with low-level features to obtain enhanced features.

[0021] Further, in step S3, the different scale features are extracted in stages, specifically including: In the first stage, shallow feature extraction is performed on the initial feature token sequence with position encoding through standard convolution and deep separable convolution to obtain the first feature with a first reduction in channel number and size; In the second stage, shallow feature extraction is performed on the first feature through standard convolution and depth separable convolution to obtain a second feature with a second time reduction in channel number and size; In the third stage, deep feature extraction is performed on the second feature through an inverted linear bottleneck layer and the attention module based on selective linear attention and multi-level feature enhancement to obtain a third feature with a third time reduction in channel number and size; In the fourth stage, deep feature extraction is performed on the third feature through an inverted linear bottleneck layer and the attention module based on selective linear attention and multi-level feature enhancement to obtain a fourth feature with a fourth time reduction in channel number and size.

[0022] Further, in step S3, the fusion feature vector of the reserved image different scale features is obtained by 4 times upsampling of the output of the fourth stage, 2 times upsampling of the output of the third stage, and fusion of the output of the second stage, which is expressed by the formula as:

[0023] wherein, is the fusion feature vector, is the fourth feature, is the third feature, is the second feature, is the upsampling.

[0024] Further, in step S4, specifically comprising: using a regression branch of a lightweight detection head to perform multi-layer perception on the fusion feature vector to obtain the bounding box position, size, and rotation angle in the remote sensing image; using a classification branch of the lightweight detection head to perform depth separable convolution on the fusion feature vector to obtain the target class in the remote sensing image and the detection target corresponding to the class label with the highest confidence; which is expressed by the formula as:

[0025]

[0026]

[0027] wherein, c is the class label with the highest confidence, DSConv is the depth separable convolution, Conv is the standard convolution, Sigmoid is the activation function, argmax is the prediction probability operation, MLP is the multi-layer perception operation, is the target center coordinate, For the boundary box width and height, θ For the rotation angle.

[0028] Compared with the prior art, the light-weight remote sensing image target detection method provided by the present application has the following beneficial effects: The attention module based on selective linear attention and multi-level feature enhancement proposed by the present application can maintain the integrity of remote sensing target features while reducing the amount of calculation by dynamically screening key tokens; at the same time, the multi-level feature enhancement proposed by the present application fuses the features of the three stages after the fusion, solves the feature loss problem of linear attention, and realizes light-weight and efficient remote sensing image target detection, reduces the calculation consumption without reducing the detection effect.

[0029] At the same time, the attention module based on selective linear attention and multi-level feature enhancement as a plug-and-play architecture can directly replace the attention layer of the existing Transformer structure, reduce the migration cost, and is especially suitable for transformer backbone structure and can be used for various downstream remote sensing image processing tasks. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0031] Figure 1 The flow chart of the light-weight remote sensing image target detection method provided in the embodiments of the present application; Figure 2 The structural schematic diagram of the attention module based on selective linear attention and multi-level feature enhancement provided in the embodiments of the present application; Figure 3 The structural schematic diagram of the S3ML-ViT model provided in the embodiments of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0033] The embodiments of the present application disclose a light-weight remote sensing image target detection method, as shown in Figure 1 the figure. S1, obtaining a remote sensing image for preprocessing to generate an initial feature token sequence with position encoding; S2, inputting the initial feature token sequence with position encoding into a pre-trained S3ML-ViT model; the S3ML-ViT model comprises a backbone network constructed based on a selective linear attention and a multi-level feature enhancement attention module and a lightweight detection head; S3, the backbone network extracts different scale features in stages according to the input initial feature token sequence with position encoding to obtain a fusion feature vector of different scale features of the image; S4, inputting the fusion feature vector into the lightweight detection head to output a target detection result of the remote sensing image; the target detection result comprises a target category, a bounding box position, a size, and a rotation angle.

[0034] The embodiment is applied to urban management, focuses on urban land use / cover classification and change monitoring tasks, and specifically identifies five categories of ground objects: buildings, roads, water bodies, vegetation, and bare land. To solve the problem of high complexity of the existing remote sensing image processing process, the embodiment builds a selective linear attention and multi-level feature enhancement attention module (SelectiveSampling Softmax and Multi-Feature Enhancement Linear, S3ML for short), reduces the attention calculation complexity from to , and retains the richness of image extracted features; to realize target detection of remote sensing images, the embodiment builds a target detection model S3ML-ViT based on the S3ML module and Vision Transformer; finally, the target detection model S3ML-ViT is trained and tested for remote sensing image target detection tasks.

[0035] The implementation process of the embodiment will be described in detail below.

[0036] Referring to Figure 2 , the embodiment first builds a selective linear attention and multi-level feature enhancement attention module (S3ML).

[0037] The S3ML module of the embodiment takes selective sampling dimension reduction and multi-dimensional feature enhancement as dual cores, and the workflow is input mapping layer → selective sampling Softmax layer → adaptive sampling layer → linear attention layer → multi-level feature enhancement layer → output fusion layer, which realizes linear conversion of the complexity from to , while retaining the spectral-spatial fine-grained features of remote sensing targets.

[0038] The S3ML module of the embodiment comprises an input mapping layer, a selective sampling Softmax layer, an adaptive sampling layer, a linear attention layer, a multi-level feature enhancement layer, and an output fusion layer. The data processing procedures of the layers are as follows: (1) Input mapping layer: For the input remote sensing image vector, the input is mapped through a linear layer:

[0039] wherein, is the input of the attention module based on selective linear attention and multi-level feature enhancement, which is an encoded feature vector of a remote sensing image, with a dimension of [B, N, C], B is the batch size, N is the token number, and C is the original feature channel number; is a learnable linear transformation weight matrix, with a dimension of , is the mapped feature channel number; is a bias term, with a dimension of ; is the mapped output feature, with a dimension of , which is used to adapt to subsequent sampling and attention calculation.

[0040] (2) Selective sampling Softmax layer: The embodiment constructs a dynamic sampling mechanism to calculate double scores for input feature tokens, namely spatial saliency score and spectral discriminability score, aiming at the redundant calculation problem of traditional self-attention.

[0041] Spatial saliency score. The embodiment extracts the local gradient feature of the mapped output feature through a 3x3 convolution, calculates the response value, and uses the formula to represent it as:

[0042] wherein, is the spatial saliency score vector, with a dimension of , and the larger the value, the more important the token is in space (such as edges and texture regions); represents a 3x3 convolution operation for extracting local gradient features.

[0043] Spectral discriminability score. If the mapped output feature is multi-spectral data, the band variance in the spectral feature subset is calculated to obtain the spectral discriminability score. The formula is as follows:

[0044] wherein, is the spectral feature subset, with a dimension of , is the number of spectral correlation channels, such as multi-spectral bands; is a variance function, reflecting the numerical difference between different bands.

[0045] Finally, the comprehensive score vector is calculated, and the formula is:

[0046]

[0047] wherein, is the comprehensive score vector, with dimensions of [B, N, 1], which integrates spatial and spectral importance; is a balance coefficient, which adjusts the contribution proportion of spatial and spectral scores.

[0048] (3) Adaptive sampling layer: This embodiment performs three-level sampling based on the comprehensive score: The top 30% of tokens are reserved as key samples , so as to ensure that the target core features are not lost.

[0049] Set a dynamic candidate pool, and use probability sampling for the middle 40% of tokens, with the sampling probability calculation method being:

[0050] wherein, is the temperature coefficient.

[0051] Suppress the background of the picture, filter out the last 30% of low-score tokens, and only keep 10% as background reference samples , and the total amount of tokens after sampling is controlled to be:

[0052] wherein, is the number of tokens originally required for calculation, is the number of tokens required for calculation after sampling, and the embodiment can achieve a 40% reduction in calculation amount through the above calculation.

[0053] The sampled feature matrix obtained by the adaptive sampling layer of this embodiment is:

[0054] (4) Linear attention layer: This embodiment uses an optimization method combining low-rank approximation and sampling to perform attention calculation, and the calculation formula is:

[0055] =

[0056] =

[0057]

[0058] where, is the final output attention result, is the input feature channel number, is the query matrix, is the sampled key transpose matrix, is the sampled value matrix, is the learnable output projection matrix, is the input feature matrix after linear layer mapping, is the sampled feature matrix, , and are the learnable output projection q , k and v corresponding weight matrices, by selective sampling complexity is reduced to .

[0059] (5) Multi-level feature enhancement layer: This embodiment constructs a "spectrum-space-scale" three-dimensional enhancement structure to make up for the loss of features caused by sampling, including three steps of spectral feature enhancement, spatial feature enhancement and multi-scale fusion.

[0060] Spectral feature enhancement: This embodiment uses band attention gating for multispectral images and synthetic aperture radar images. For multispectral images, generate channel weights through 1D convolution (kernel size 3) and Sigmoid activation, represented by the formula as:

[0061] For synthetic aperture radar images, use residual contraction network to suppress speckle noise, represented by the formula as:

[0062] where, is the channel attention weight vector, is the feature tensor after suppressing speckle noise, is the original feature tensor of the synthetic aperture radar image, is the 5x5 convolution layer output of , and is the noise mask.

[0063] Spatial feature enhancement: This embodiment combines the local modeling advantages of CNN, performs multi-receptive field extraction, and performs parallel 3x3, 5x5 depth separable convolution to capture different scale spatial textures; among them, the 3x3 convolution kernel captures fine-grained local texture (such as road edge, roof outline), and the 5x5 convolution kernel captures larger range context structure (such as building group layout, farmland plot). Then, 1x1 convolution is performed on the parallel convolution output to fuse and reduce the channel; reduce the amount of subsequent calculation, and promote cross-branch information interaction. After compression, local self-attention is performed, and self-attention mechanism is performed within a 7x7 window; calculate the correlation between each token in the window, and strengthen the neighborhood semantic association; realize long-range local interaction across tokens, and make up for the defect of limited receptive field of CNN.

[0064] Multi-scale fusion: A feature pyramid structure is used to generate three-level feature maps: 1 / 2 and 1 / 4 scale features are generated by downsampling with a step of 2; and cross-layer enhancement is performed, the high-level feature is upsampled and fused with the low-level feature residual to obtain enhanced features; the formula is

[0065] Among them, is the high-level feature, is the low-level feature, is the scale weight.

[0066] (6) Output fusion layer: The enhanced features are fused with the features after linear attention through residual connection, and the layer is normalized and regularized by dropout=0.1 before output to obtain the fusion features, which is expressed by the formula:

[0067]

[0068] Among them, is the fusion feature, is the feature after linear attention.

[0069] Secondly, as shown in Figure 3 , the embodiment constructs a target detection model S3ML-ViT based on the improved S3ML module, and performs target detection on the remote sensing image.

[0070] According to step S1, the remote sensing image is acquired and preprocessed; specifically including: Taking the application in urban observation as an example, focusing on the urban land use / cover classification and change monitoring task, specifically identifying five categories of ground objects: buildings, roads, water bodies, vegetation and bare land. An original remote sensing image , the resolution of the embodiment is Wherein the remote sensing image includes two modalities of multispectral image or synthetic aperture radar image, through the data preprocessing module, the backbone network and the detection head, and finally the detection result is obtained.

[0071] The preprocessing module of the embodiment carries out standardization, tokenization and position encoding on the data , and adapts the subsequent backbone network.

[0072] Specifically, first, data standardization is carried out. For multispectral images: select red, green, blue and near-infrared key bands , normalize to [0, 1] according to the band, and use the formula to express:

[0073] Wherein, is the standardized data of a certain band, is the mapping of b-band data into a feature matrix, is one of the red, green, blue and near-infrared key bands.

[0074] For synthetic aperture radar images: first, Lee filter denoising is carried out to obtain , and then converted to decibel value , and use the formula to express:

[0075] Tokenization processing is carried out on the data processed by standardization, and according to the 25% overlap rate slicing, an initial token set is generated. The target size and target density use an 8x8 slicing window for slicing in small target dense areas; a 16x16 slicing window is used for slicing in large target areas; and finally is obtained: is the number of channels, 512 1024 is the spatial dimension of the token.

[0076] Position encoding, using sine-cosine encoding, adding band dimension information; the position encoding information is obtained using the formula:

[0077] Wherein, is the token spatial coordinate, is the band index, is the encoding dimension, is the total encoding dimension; and the final encoded token is .

[0078] According to step S2, the initial feature token sequence with position encoding The pre-trained S3ML-ViT model is inputted, and different scale features are extracted by the backbone network in stages according to step S3, specifically including: The input stream mapping is inputted, and the input is , and the dimension is (including red, green, blue, and near-infrared key bands), and the original input is mapped into a feature tensor in a feature space through standard convolution and depth separable convolution.

[0079]

[0080] wherein, represents standard convolution, the convolution kernel is 3x3, the step is 2, and the padding is 1, represents depth separable convolution, the convolution kernel is 3x3, the step is 1, and the padding is 1, and the output dimension of the output is .

[0081] Stage 1 is used to extract shallow features and focuses on small target details: the input size is ; shallow feature extraction is performed on the first feature through one inverted linear bottleneck layer, and the first feature of the embodiment mainly includes the outline features of large buildings, roads, water bodies, vegetation, and bare land, etc. large range targets, such as the overall continuous outline of high-density residential areas;

[0082] is an inverted linear bottleneck layer, and the dimension of the output is .

[0083] Stage 2 is also used to extract shallow features and focuses on small target details: the input size is ; shallow feature extraction is performed on the first feature through two inverted linear bottleneck layers to obtain the second feature , and the second feature of the embodiment mainly includes the outline features of small buildings, roads, water bodies, vegetation, and bare land, etc. small range, such as the concave-convex outline of the artificial lake shoreline;

[0084] wherein, is an inverted linear bottleneck layer, and the dimension of the output is . At the same time, after stage 1 and stage 2, the token number is reduced through 2x2 pooling to balance the accuracy and efficiency.

[0085] Stage 3 is used to extract deep features and focuses on global context information; the input size is The third feature is obtained by performing deep feature extraction on the second feature using an inverted linear bottleneck layer and an S3ML module. The third feature of this embodiment mainly includes large-scale texture features such as large buildings, roads, water bodies, vegetation and bare land;

[0086] in, It is an inverted linear bottleneck layer, and the S3ML module is the attention module constructed above. It is the third feature of the stage 3 output, dimension .

[0087] Stage 4, also used to extract deep features, focuses on global context information: deep feature extraction is performed on the third feature using an inverted linear bottleneck layer and the S3ML module to obtain the fourth feature. The fourth feature of this embodiment mainly includes the microscopic texture details of small features in buildings, roads, water bodies, vegetation, and bare land;

[0088] in, It is the fourth feature of the stage 4 output, dimension .

[0089] In this embodiment, the output of the backbone network is formed by fusing the output of stage 4 (upsampled four times), the output of stage 3 (upsampled twice), and the output of stage 2. This enriches the information at different scales, covers remote sensing targets of different sizes, and yields a fused feature vector. It contains all information about the extent, outline, and texture location of features, such as the extent, outline, and information of high-density residential areas.

[0090]

[0091] This embodiment, according to step S4, inputs the fused feature vector into the lightweight detection head and outputs the target detection result of the remote sensing image; specifically including: This embodiment predicts the target category and detection box in an image based on fused feature vectors. The input dimension is... The detection head is designed to be lightweight, balancing accuracy and speed.

[0092] The classification branch structure consists of depthwise separable convolution and standard convolution. The depthwise separable convolution has a kernel size of 3×3, a stride of 1, and padding of 1, and is used to extract local contextual information to enhance semantic discrimination capabilities. The standard convolution has a kernel size of 1×1 and is used to compress the number of channels to the number of categories K, such as five categories: buildings, roads, water bodies, vegetation, and bare land.

[0093] The Sigmoid activation function is used to output the probabilities (range [0, 1]) of five categories: buildings, roads, water bodies, vegetation, and bare land. It is suitable for multi-label or soft classification. argmax indicates that the category with the highest probability is taken as the prediction result, and the output category is... The predicted probability output for a detection box is [0.2, 0.1, 0.4, 0.15, 0.15]. This box has the highest probability of being a water body and is predicted as a water target.

[0094] The regression branch uses a three-layer MLP to output the target center coordinates (x, y), width and height (w, h), and rotation angle θ to adapt to the rotation characteristics of the remote sensing target, thereby obtaining the bounding box position, size, and rotation angle.

[0095] The formula is expressed as follows:

[0096]

[0097] Each detection target corresponds to a set of parameters. The category with the highest confidence level Center coordinates Width and height The rotation angle θ is adapted to the rotation characteristics of the remote sensing target.

[0098] Finally, this embodiment trains and tests the constructed target detection model S3ML-ViT for remote sensing image target detection tasks; specifically including: The network is trained for remote sensing image target detection using focal loss and rotated IoU loss, expressed by the following formula:

[0099] in, This represents the focus loss, used in the classification and prediction branch of remote sensing image target detection tasks. This represents the IoU loss function for the rotated bounding box, used to calculate the distance between the predicted bounding box and the ground truth label corresponding to the target in the input remote sensing image. and These represent the predicted target information result and the target ground truth labeling result, respectively. This represents the total loss used in model training. Once the model training continues until the loss no longer decreases, the model training has reached a stable state, the training process ends, and the trained object detection model is obtained.

[0100] Furthermore, the trained object detection model is tested on the remote sensing images to be tested, which can be formally represented as:

[0101] in, This represents the completed object detection model. and These represent the remote sensing image to be tested and the corresponding target detection result, respectively.

[0102] The selective sampling Softmax mechanism proposed in this invention can dynamically filter key tokens, maintaining the integrity of remote sensing target features while reducing computational load. Simultaneously, it proposes multi-level feature enhancement, fusing features from the last three stages to address the feature loss problem of linearized attention. By combining the ViT model and the S3ML module, the S3ML-ViT model is obtained, enabling lightweight and efficient remote sensing image target detection, reducing computational consumption without compromising detection performance.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0104] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for lightweight remote sensing image object detection, characterized in that, The method comprises the steps of: S1, obtaining a remote sensing image for preprocessing to generate an initial feature token sequence with position encoding; S2, inputting the initial feature token sequence with position encoding into a pre-trained S3ML-ViT model; The S3ML-ViT model comprises a backbone network constructed based on a selective linear attention and a multi-level feature enhancement attention module and a lightweight detection head; S3, the backbone network extracts different scale features in stages according to the input initial feature token sequence with position encoding to obtain a fusion feature vector retaining different scale features of the image; S4, inputting the fusion feature vector into the lightweight detection head to output a target detection result of the remote sensing image; the target detection result comprises a target category, a bounding box position, a size and a rotation angle.

2. The method of claim 1, wherein, Further comprising: The S3ML-ViT model adopts the following loss function during training: wherein, is a focal loss for computing a classification prediction branch of a target detection task, is a rotated bounding box loss for computing a distance between a predicted bounding box and a ground truth label of a target corresponding to an input remote sensing image, IoU is a loss function, and are a predicted target information result and a ground truth label result, respectively, is a total loss for model training.

3. The method of claim 1, wherein, In step S1, the preprocessing specifically comprises standardization processing, tokenization processing and position encoding.

4. The method of claim 1, wherein, In step S2, the attention module based on the selective linear attention and the multi-level feature enhancement comprises an input mapping layer, a selective sampling Softmax layer, an adaptive sampling layer, a linear attention layer, a multi-level feature enhancement layer and an output fusion layer; The input mapping layer projects the input features to the query, key and value spaces to obtain the mapped feature blocks; The selective sampling Softmax layer calculates the spatial significance score and the spectral discriminability score of each feature block and weights and fuses them into a comprehensive score vector; The adaptive sampling layer performs three-level sampling according to the comprehensive score vector; retains the top 30% high-score feature blocks, probabilistically samples the middle 40% feature blocks, and only retains a part of the last 30% feature blocks as background reference; The linear attention layer performs low-rank approximate attention calculation on the sampled feature matrix to obtain optimized sampled features; The multi-level feature enhancement layer sequentially performs spectral feature enhancement, spatial feature enhancement and multi-scale fusion on the sampled features to obtain enhanced features; The output fusion layer outputs the fusion features by residual connection, layer normalization and Dropout regularization of the enhanced features and the input features.

5. The method of claim 4, wherein, In the linear attention layer, the low-rank approximate attention calculation is represented by the formula: = = wherein, is the attention result of the final output, is the number of input feature channels, is the query matrix, is the sampled key transpose matrix, is the sampled value matrix, is the learnable output projection matrix, is the input feature matrix after linear layer mapping, is the sampled feature matrix, , and are the learnable output projection q , k and v corresponding weight matrices.

6. The method of claim 4, wherein, In the multi-level feature enhancement layer: 1) For a multispectral image, the spectral feature enhancement generates channel weights through 1D convolution and Sigmoid activation; For a synthetic aperture radar image, the spectral feature enhancement adopts a residual contraction network to suppress coherent speckle noise; 2) The spatial feature enhancement adopts parallel depth separable convolution for multi-receptive field feature extraction and uses 1x1 convolution to perform channel fusion and dimension reduction on the parallel convolution output; a local self-attention mechanism is executed to calculate the correlation between feature blocks within a window and strengthen the neighborhood semantic association; 3) The multi-scale fusion adopts a feature pyramid structure to generate 1 / 2 and 1 / 4 scale features through down-sampling with a step of 2; And cross-layer enhancement, The high-level features are up-sampled and fused with the low-level features to obtain enhanced features.

7. The method of claim 1, wherein, In step S3, the different scale features are extracted in stages, specifically including: In a first stage, shallow feature extraction is performed on the initial feature token sequence with position encoding by standard convolution and depth separable convolution to obtain first features with a first reduction in channel number and size; In a second stage, shallow feature extraction is performed on the first features by standard convolution and depth separable convolution to obtain second features with a second reduction in channel number and size; In a third stage, deep feature extraction is performed on the second features by an inverted linear bottleneck layer and the attention module based on selective linear attention and multi-level feature enhancement to obtain third features with a third reduction in channel number and size; In a fourth stage, deep feature extraction is performed on the third features by an inverted linear bottleneck layer and the attention module based on selective linear attention and multi-level feature enhancement to obtain fourth features with a fourth reduction in channel number and size.

8. The method of claim 7, wherein, In step S3, the fusion feature vector of the different scale features of the image is obtained by 4 times upsampling of the output of the fourth stage, 2 times upsampling of the output of the third stage, and fusion of the output of the second stage, which is represented by the formula: wherein, is a fused feature vector, is a fourth feature, is a third feature, is a second feature, is an up-sampling.

9. The method of claim 1, wherein, Step S4 specifically includes: A regression branch of a lightweight detection head is used to perform multi-layer perception on the fusion feature vector to obtain the position, size, and rotation angle of the bounding box in the remote sensing image; A classification branch of the lightweight detection head is used to perform depth separable convolution on the fusion feature vector to obtain the target class in the remote sensing image and the detection target corresponding to the class label with the highest confidence; which is represented by the formula: wherein, c is a class label of highest confidence, DSConv is a depthwise separable convolution, Conv is a standard convolution, Sigmoid is an activation function, argmax is a prediction probability operation, MLP is a multilayer perceptron operation, is a target center coordinate, is a bounding box width-height, θ is a rotation angle.

Images