Small target detection method and system based on state space dual model encoder

Abstract

The present application relates to the technical field of computer vision target detection, and particularly relates to a small target detection method and system based on a state space dual model encoder; the method acquires and pre-processes a target image, and inputs the target image into a pre-trained target detection neural network for detection; the encoder of the neural network comprises a cross-scale feature fusion module and a global-local multi-scale feature interaction module; the cross-scale feature fusion module realizes deep context fusion of adjacent scale features through a state space dual model and a depth separable convolution; the global-local multi-scale feature interaction module combines global and local attention mechanisms to form dual modeling capabilities of global perception and local focusing. The small target detection method proposed by the present application effectively solves the problems of insufficient feature fusion and global-local information imbalance of existing methods, significantly improves the small target detection precision and robustness, and is particularly suitable for micro target detection in complex scenes such as remote sensing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision target detection, and particularly relates to a small target detection method and system based on a state space dual model encoder. BACKGROUND

[0002] In the field of target detection, small targets generally refer to objects with extremely small pixel areas in images. Such targets have extremely limited visual information, and their feature expression ability and anti-interference ability are much lower than those of regular size targets, which leads to serious challenges in their detection. On the one hand, small targets have few effective pixel points available, making it difficult to extract features with strong discriminability. On the other hand, in complex and variable actual scenes, such as bare soil, forest, hilly or ocean background in remote sensing images, small targets are easily overwhelmed by background noise, further exacerbating the difficulty of accurate detection.

[0003] Deep learning-based target detection methods, especially those represented by convolutional neural network (CNN) and Transformer architecture, perform well in general target detection tasks. However, these methods still have obvious performance bottlenecks when directly applied to small target detection. The fundamental reason is that: first, small targets themselves contain very little effective pixel information, making it difficult to extract robust and discriminative features; second, the continuous decline in feature map resolution caused by the increase in network depth causes the fine spatial information of small targets to decay and even disappear in deep networks; finally, the existing feature pyramid network relies on simple upsampling and addition or splicing operations when performing multi-scale fusion, and cannot effectively bridge the semantic gap and spatial mismatch between different scale features, resulting in insufficient fusion of shallow details and deep semantics, and failing to provide sufficient and accurate context support for small target detection.

[0004] In view of the problem of small target detection, various improvement strategies have been proposed in the prior art, among which multi-scale feature enhancement is one of the mainstream directions. For example, DINO (DETR detector with improved denoising anchor box), Align-DETR (DETR detector with feature alignment), and other detectors based on Transformer, simply concatenate the feature maps of different levels extracted by the backbone network in the channel dimension to form a multi-scale feature sequence, and then input these multi-scale feature sequences into the Transformer encoder for global interaction. This way integrates multi-scale information to some extent, but still has obvious shortcomings: first, simple feature concatenation cannot realize deep context fusion of cross-scale features, and there is a lack of effective interaction mechanism between shallow details and deep semantics; second, although the Transformer module based on standard self-attention can model global dependencies, it has high computational complexity and limited ability to model local fine structures, which easily ignores the local key information crucial for small target detection; finally, there is a lack of mechanism for collaborative modeling of multi-scale features from the perspectives of global context and local space, resulting in insufficient representation of small targets in complex scenes, and the detection accuracy and robustness still need to be improved.

[0005] In view of this, the present application is proposed. SUMMARY

[0006] In view of the deficiencies in the prior art, the present application proposes a small target detection method based on a state space dual model encoder to solve the problems of existing methods in cross-scale feature fusion, global and local information collaboration, and computational efficiency, thereby effectively improving the accuracy and robustness of small target detection.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] The present application proposes a small target detection method based on a state space dual model encoder, comprising the following steps:

[0009] Obtain the target image to be detected, pre-process the target image to obtain an image tensor;

[0010] Input the image tensor into a pre-trained target detection neural network, perform small target detection on the target image through the target detection neural network, and output the class label and bounding box coordinates of all detected small targets; the small target is a target with a spatial pixel point less than a set threshold;

[0011] The target detection neural network comprises a backbone network, an encoder, a decoder and a prediction head connected in turn;

[0012] The backbone network is used to extract multi-scale features from the image tensor;

[0013] The encoder comprises a cross-scale feature fusion module and a global-local multi-scale feature interaction module, the cross-scale feature fusion module is used for receiving multi-scale features and performing context fusion on features of adjacent scales to obtain fused features; the global-local multi-scale feature interaction module is used for receiving the fused features and performing global context modeling and local space modeling on the fused features respectively to generate enhanced features;

[0014] The decoder receives the enhanced features and performs small target query and feature interaction based on the enhanced features to output a small target representation vector;

[0015] The prediction head receives the small target representation vector and generates the category label and the bounding box coordinates according to the small target representation vector.

[0016] Further, the cross-scale feature fusion module comprises a first fusion path and a second fusion path;

[0017] The first fusion path comprises a plurality of first fusion units connected in sequence, which are used to iteratively transmit and fuse deep semantic information to shallow layers, and the second fusion path comprises a plurality of second fusion units connected in sequence, which are used to iteratively transmit and fuse shallow detail information to deep layers; in the second fusion path, the input of the first second fusion unit includes the output of the first fusion unit corresponding to its spatial scale and the output of the last first fusion unit, the input of the last second fusion unit includes the output of the previous second fusion unit and the feature with the smallest spatial scale output by the backbone network, and the input of the remaining second fusion units includes the output of the first fusion unit corresponding to its spatial scale and the output of the previous second fusion unit; the output of the last first fusion unit and the output of all second fusion units are spliced to obtain the fused features.

[0018] Further, the first fusion unit and the second fusion unit have the same internal structure and are both realized by a state space dual model for feature fusion.

[0019] Further, the first fusion unit performs context fusion on features of adjacent scales in the following process:

[0020] The first feature and the second feature input are respectively adjusted in channels to obtain first channel-aligned features and second channel-aligned features;

[0021] The first channel-aligned features are linearly transformed to obtain first query features and second query features;

[0022] The first query features are processed by depth separable convolution to obtain key features, and the second query features are input into a first activation function to obtain activated features;

[0023] inputting the first query feature and the key feature into the first state space dual model to obtain a correlation feature;

[0024] element-wise multiplying the correlation feature and an activation feature to obtain a first intermediate feature;

[0025] linearly transforming the first intermediate feature to obtain a second intermediate feature;

[0026] performing residual connection on the first channel alignment feature and the second intermediate feature to obtain a third intermediate feature;

[0027] inputting the third intermediate feature into the first feedforward neural network to obtain a first feedforward enhanced feature;

[0028] performing residual connection on the first feedforward enhanced feature and the third intermediate feature to obtain a fourth intermediate feature;

[0029] performing residual connection on the fourth intermediate feature and the second channel alignment feature to obtain an output of the first fusion unit.

[0030] Further, the global-local multi-scale feature interaction module comprises a global-local attention mechanism and a second feedforward neural network, the global-local attention mechanism comprises a global attention mechanism and a local attention mechanism, and a process of the global-local multi-scale feature interaction module for generating an enhanced feature is as follows:

[0031] modeling a global context of the fusion feature through the global attention mechanism to obtain a global enhanced feature;

[0032] modeling a local space of the fusion feature through the local attention mechanism to output a local enhanced feature;

[0033] element-wise adding the global enhanced feature and the local enhanced feature, and linearly transforming a result after the addition to obtain a global-local feature;

[0034] element-wise adding the global-local feature and the fusion feature to obtain a primary enhanced feature;

[0035] inputting the primary enhanced feature into the second feedforward neural network to obtain a second feedforward enhanced feature;

[0036] element-wise adding the second feedforward enhanced feature and the primary enhanced feature to generate an enhanced feature.

[0037] Further, a process of the global attention mechanism for modeling a global context of the fusion feature is as follows:

[0038] linearly transforming the fusion feature to obtain a first global transformed feature and a second global transformed feature;

[0039] performing convolution processing on the first global transformed feature to obtain a convolution enhanced global feature;

[0040] inputting the convolution enhanced global feature and the first global transformed feature into a second state space dual model to obtain an intermediate global feature;

[0041] inputting the second global transformed feature into a second activation function to obtain an activated global feature;

[0042] performing element-wise multiplication on the activated global feature and the intermediate global feature to obtain the global enhanced feature.

[0043] Further, the process of the local attention mechanism for local space modeling of the fusion feature is as follows:

[0044] linearly transforming the fusion feature to obtain a first local transformed feature and a second local transformed feature;

[0045] performing convolution processing on the first local transformed feature to obtain a convolution enhanced local feature;

[0046] adding a first position encoding and a second position encoding to the convolution enhanced local feature, respectively, to obtain a first encoded local feature and a second encoded local feature, respectively;

[0047] linearly transforming the first encoded local feature to obtain a first modulation feature, performing dynamic local sampling on the first modulation feature, and obtaining a feature vector of each sampling point through bilinear interpolation;

[0048] assigning an attention weight to the feature vector of each sampling point and aggregating to obtain a local aggregated feature;

[0049] linearly transforming the second encoded local feature to obtain a second modulation feature, performing feature fusion on the second modulation feature and the local aggregated feature to obtain a modulated local feature;

[0050] inputting the second local transformed feature into a third activation function to obtain an activated local feature;

[0051] performing element-wise multiplication on the activated local feature and the modulated local feature to obtain the local enhanced feature.

[0052] Further, the target detection neural network is an end-to-end detector based on a Transformer architecture.

[0053] The application further provides a small target detection system based on a state space dual model encoder, which is used for implementing the small target detection method and comprises an image acquisition module and a detection processing module.

[0054] Compared with the prior art, the application has the following beneficial effects:

[0055] (1) The cross-scale feature fusion module is introduced into the encoder, the depth separable convolution is combined with the state space dual model through the bidirectional fusion path, the depth context interaction between adjacent scale features is realized, the semantic gap between the shallow detail features and the deep semantic features is effectively bridged, the long-range semantic association across scales is established, and more abundant and consistent context information support is provided for the small target.

[0056] (2) The global-local multi-scale feature interaction module is introduced into the encoder, the global attention mechanism is used to capture the overall context of the image, the local attention mechanism is used to focus on the key spatial structure around the small target, the modeling mechanism in which the global perception and the local focus are coordinated is formed, the model can understand the overall scene while not losing the fine capture of the small target details, and therefore, the high detection precision and the robustness can be maintained under the complex background interference.

[0057] (3) While improving the detection performance, the calculation efficiency is also considered, the depth separable convolution in the cross-scale fusion module greatly reduces the parameters and the calculation amount, and the state space dual model has the linear complexity advantage when processing the sequence dependence. Therefore, compared with the prior art, the method provided by the application significantly improves the inference speed. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 FIG. 1 is a schematic diagram of the overall structure of the target detection neural network in the embodiment of the application;

[0059] Figure 2 FIG. 5 is a schematic diagram of the structure of the first fusion unit in the encoder in the embodiment of the application;

[0060] Figure 3 FIG. 6 is a schematic diagram of the structure of the global-local attention mechanism in the encoder in the embodiment of the application. DETAILED DESCRIPTION

[0061] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of the present application.

[0062] Embodiments

[0063] The embodiment provides a small target detection method based on a state space dual model encoder, including the following steps:

[0064] Step 1: Obtain a target image to be detected by unmanned aerial vehicle aerial photography or radar shooting, and perform size standardization and normalization preprocessing on the target image to convert it into an image tensor meeting the input requirements of a target detection neural network.

[0065] Step 2: input the image tensor into a pre-trained target detection neural network, detect small targets in the target image through the target detection neural network, and output the class labels and bounding box coordinates of all detected small targets; in the embodiment, small targets refer to targets with a spatial pixel less than 1024 in the image, such as pedestrians, vehicles, buildings, ships, buoys, etc.

[0066] Reference Figure 1 The target detection neural network is based on a Transformer architecture and includes a backbone network, an encoder, a decoder and a prediction head connected in sequence.

[0067] The backbone network is used to extract multi-scale features from the image tensor; in the embodiment, the backbone network uses ResNet-50 (Residual Network-50) as a main feature extractor to extract four features with different sizes from the input image tensor:

[0068] C5: spatial size 25x25, channel number 2048, containing the richest semantic information;

[0069] C4: spatial size 50x50, channel number 1024;

[0070] C3: spatial size 100x100, channel number 512;

[0071] C2: spatial size 200x200, channel number 256.

[0072] The encoder comprises a cross-scale feature fusion module and a global-local multi-scale feature interaction module, the cross-scale feature fusion module is configured to receive multi-scale features and perform context fusion on features of adjacent scales to obtain fused features, and the global-local multi-scale feature interaction module is configured to receive the fused features and perform global context modeling and local space modeling on the fused features respectively to generate enhanced features.

[0073] The cross-scale feature fusion module comprises a first fusion path from top to bottom and a second fusion path from bottom to top.

[0074] The first fusion path comprises a plurality of first fusion units connected in sequence and configured to iteratively transmit and fuse deep semantic information to a shallow layer, and the second fusion path comprises a plurality of second fusion units connected in sequence and configured to iteratively transmit and fuse shallow detail information to a deep layer; the number of first fusion units is the same as the number of second fusion units. In the second fusion path, the input of the first second fusion unit comprises the output of the first fusion unit corresponding to the spatial scale thereof and the output of the last first fusion unit, the input of the last second fusion unit comprises the output of the previous second fusion unit and the feature with the smallest spatial size in the output of the backbone network, and the input of the remaining second fusion units comprises the output of the first fusion unit corresponding to the spatial scale thereof and the output of the previous second fusion unit.

[0075] The output of the last first fusion unit and the output of all second fusion units are spliced to obtain the fused features.

[0076] In this embodiment, four features with different sizes are extracted from the input image tensor, therefore, the first fusion path comprises three first fusion units connected in sequence, and the second fusion path comprises three second fusion units connected in sequence. The processing process of the cross-scale feature fusion module on the four features with different sizes is as follows:

[0077] The first first fusion unit receives the feature C5 and the feature C4, fuses C5 and C4, and outputs the primary fused feature F54, the intermediate first fusion unit receives the primary fused feature F54 and the feature C3, fuses F54 and C3, and obtains the primary fused feature F543, and the last first fusion unit receives the primary fused feature F543 and the feature C2, fuses F543 and C2, and obtains the primary fused feature F5432.

[0078] The first second fusion unit receives the primary fusion feature F5432 and the primary fusion feature F543, fuses F5432 and F543, and outputs a secondary fusion feature S1, the intermediate second fusion unit receives the secondary fusion feature S1 and the primary fusion feature F54, fuses S1 and F54, and obtains a secondary fusion feature S2, and the last second fusion unit receives the secondary fusion feature S2 and the feature C5, fuses S2 and C5, and obtains a secondary fusion feature S3.

[0079] The channel dimension splicing is performed on F5432, S1, S2 and S3, and a fusion feature R is output.

[0080] The internal structure of each first fusion unit is as shown in Figure 2 The fusion process of adjacent scale features is as follows:

[0081] The input two features are respectively subjected to 1x1 convolution channel alignment, and a first channel alignment feature from a deep layer and a second channel alignment feature from a shallow layer are obtained;

[0082] The first channel alignment feature is subjected to two independent linear transformations, and a first query feature and a second query feature are obtained;

[0083] The first query feature is input into a depth separable convolution layer, the depth separable convolution processing is performed on the first query feature, the local spatial structure is extracted, and a key feature is obtained, and the second query feature is input into a first activation function, and an activated feature is obtained;

[0084] The first query feature and the key feature are input into a first state space dual model, a global long-range dependency is established, and a correlation feature is obtained;

[0085] The correlation feature and the activated feature are element-wise multiplied, and a first intermediate feature is obtained;

[0086] The first intermediate feature is subjected to linear transformation, and a second intermediate feature is obtained;

[0087] The first channel alignment feature and the second intermediate feature are residual connected, and a third intermediate feature is obtained;

[0088] The third intermediate feature is input into a first feedforward neural network, and is subjected to nonlinear enhancement, and a first feedforward enhanced feature is obtained;

[0089] The first feedforward enhanced feature and the third intermediate feature are residual connected, and a fourth intermediate feature is obtained;

[0090] The fourth intermediate feature and the second channel alignment feature are residual connected, and the output of the first fusion unit is obtained.

[0091] The internal structure of the second fusion unit and its fusion process of the adjacent scale features are the same as those of the first fusion unit.

[0092] The structure of the global-local multi-scale feature interaction module is shown in Figure 3 FIG. 3, which includes a global-local attention mechanism and a second feedforward neural network, the global-local attention mechanism including a global attention mechanism and a local attention mechanism, and the process of the global-local multi-scale feature interaction module for generating an enhanced feature is as follows:

[0093] The global attention mechanism is used to model the global context of the fusion feature R to obtain a global enhanced feature Q, and the local attention mechanism is used to model the local space of the fusion feature R to output a local enhanced feature J, the global enhanced feature Q and the local enhanced feature J are added element by element, and the result is linearly transformed to obtain a global-local feature , and the global-local feature is added element by element with the fusion feature R to obtain a primary enhanced feature , the primary enhanced feature is input into the second feedforward neural network to obtain a second feedforward enhanced feature , and the second feedforward enhanced feature is added element by element with the primary enhanced feature to generate an enhanced feature T.

[0094] The process of the global attention mechanism for modeling the global context of the fusion feature R is as follows:

[0095] Two independent linear transformations are performed on the fusion feature R to obtain a first global transformed feature and a second global transformed feature .

[0096] The first global transformed feature is subjected to 1x1 convolution processing to obtain a convolution enhanced global feature .

[0097] The convolution enhanced global feature and the first global transformed feature are input into a second state space dual model to obtain an intermediate global feature Z.

[0098] The second global transformed feature is input into a second activation function to obtain an activated global feature .

[0099] The activated global feature and the intermediate global feature Z are multiplied element by element to obtain the global enhanced feature Q.

[0100] The local attention mechanism performs local spatial modeling on the fusion feature, and the process is as follows:

[0101] Two independent linear transformations are performed on the fusion feature R to obtain a first local transformed feature and a second local transformed feature .

[0102] The first local transformed feature is subjected to 1x1 convolution processing to obtain a convolution-enhanced local feature .

[0103] The convolution-enhanced local feature is respectively added with a first sinusoidal position encoding and a second sinusoidal position encoding to obtain a first encoded local feature and a second encoded local feature , wherein the formula for adding the sinusoidal position encoding is as follows:

[0104]

[0105]

[0106] In the formula, i is a position index, j is a dimension index, and D is the total dimension of the position encoding vector.

[0107] The first encoded local feature is subjected to linear transformation to obtain a first modulation feature, a learnable sampling offset is predicted based on the convolution-enhanced local feature, and the first modulation feature is dynamically locally sampled using the offset. Specifically, each encoded position in the first modulation feature is taken as a reference point, K neighborhood points are sampled for each reference point, and a feature vector of each sampling point is obtained through bilinear interpolation. In this embodiment, K is 8. The feature vectors of the sampling points are assigned attention weights and aggregated to obtain a local aggregation feature corresponding to the reference point, wherein the attention weight of the i-th sampling point is calculated by the following formula:

[0108] In the formula, q is a query vector of the current reference point, k is a key vector of the i-th sampling point, and D is the dimension of q and k.

[0109]

[0110] ​​​​​​​​performing linear transformation on the second encoded local feature to obtain a second modulation feature, performing feature fusion on the second modulation feature and the local aggregated feature corresponding to each reference point to obtain a modulation local feature corresponding to each reference point;

[0111] performing linear transformation on the second local feature inputting a third activation function to obtain an activated local feature corresponding to each reference point;

[0112] performing element-wise multiplication on the activated local feature corresponding to each reference point and the modulation local feature corresponding thereto to obtain a local feature corresponding to the reference point, and the local features of all reference points forming a local enhanced feature .

[0113] In this embodiment, the first activation function, the second activation function and the third activation function all adopt the SILU activation function.

[0114] The decoder adopts a Transformer decoder structure, receives the enhanced feature, and performs small target query and feature interaction based on the enhanced feature to output a small target representation vector.

[0115] The prediction head receives the small target representation vector and generates the category label and the bounding box coordinates according to the small target representation vector.

[0116] In this embodiment, the SODA-D (Small Object Detection Airborne-Dataset) is used to train and evaluate the target detection neural network. During the training process, the training objective function is the total loss function :

[0117]

[0118] wherein, is a classification loss, is a regression loss, is a generalized intersection over union loss, is a weight coefficient of the classification loss, is a weight coefficient of the regression loss, is a weight coefficient of the generalized intersection over union loss.

[0119] The training process aims to continuously update the network weights through the gradient descent algorithm to minimize the total loss function , so that the model can accurately predict the category and position of the small target at the same time.

[0120] ​​​​To verify the effectiveness of the small target detection method proposed in the above embodiment, the small target detection method proposed in the above embodiment was verified on SODA-D, which contains 24828 images and 278433 instances in total. The data set was divided into a training set, a validation set and a test set according to a ratio of 5:2:3.

[0121] DINO and Align-DETR were selected as baseline models, and improved DINO was constructed based on DINO, and improved Align-DETR was constructed based on Align-DETR. The difference between improved DINO and DINO is that the encoder in DINO is replaced by the encoder in the above embodiment; the difference between improved Align-DETR and Align-DETR is that the encoder in Align-DETR is replaced by the encoder in the above embodiment.

[0122] The evaluation indexes include:

[0123] The average precision (AP) of small targets with spatial pixels less than 1024, the average precision (AP eS ) of small targets with spatial pixels in the range of 0-144, and the average precision (AP rS ) of small targets with spatial pixels in the range of 144-400.

[0124] Under the same experimental environment and training strategy, the performance of each model on the above test set is shown in Table 1.

[0125] Table 1: Performance comparison of each model on SODA-D test set

[0126]

[0127] As can be seen from Table 1, the small target detection method proposed in the present application can effectively enhance the model's ability to capture context information of small targets by introducing a cross-scale feature fusion module and a global-local multi-scale feature interaction module, and improve the modeling effect of multi-scale features from the global and local dual perspectives, thereby significantly improving the small target detection precision.

[0128] The specific embodiments of the present application enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application.

[0129] It should be understood that the present application is not limited to the above described and that various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A small target detection method based on a state-space dual model encoder, characterized in that, Includes the following steps: The target image to be detected is acquired, and the target image is preprocessed to obtain an image tensor; The image tensor is input into a pre-trained target detection neural network, which performs small target detection on the target image and outputs the category labels and bounding box coordinates of all detected small targets; the small targets are targets with spatial pixels smaller than a set threshold. The target detection neural network includes a backbone network, an encoder, a decoder, and a prediction head connected in sequence. The backbone network is used to extract multi-scale features from the image tensor; The encoder includes a cross-scale feature fusion module and a global-local multi-scale feature interaction module. The cross-scale feature fusion module receives multi-scale features and performs contextual fusion on features of adjacent scales to obtain fused features. The global-local multi-scale feature interaction module receives the fused features and performs global contextual modeling and local spatial modeling on the fused features to generate enhanced features. The cross-scale feature fusion module achieves deep contextual fusion of features of adjacent scales through a state-space dual model and depthwise separable convolution. The global-local multi-scale feature interaction module combines global and local attention mechanisms to form a dual modeling capability of global perception and local focus. The cross-scale feature fusion module includes a first fusion path and a second fusion path. The first fusion path contains multiple sequentially connected first fusion units, used to iteratively transfer and fuse deep semantic information to shallow layers. The second fusion path contains multiple sequentially connected second fusion units, used to iteratively transfer and fuse shallow detail information to deep layers. In the second fusion path, the input of the first second fusion unit includes the output of the first fusion unit corresponding to its spatial scale and the output of the last first fusion unit. The input of the last second fusion unit includes the output of the previous second fusion unit and the feature with the smallest spatial scale output by the backbone network. The input of the remaining second fusion units includes the output of the first fusion unit corresponding to its spatial scale and the output of the previous second fusion unit. The output of the last first fusion unit and the outputs of all second fusion units are concatenated to obtain the fused feature. The first and second fusion units have the same internal structure and both implement feature fusion based on a state-space dual model. The decoder receives the enhanced features and performs small target query and feature interaction based on the enhanced features, and outputs a small target representation vector; The prediction head receives the small target representation vector and generates the category label and bounding box coordinates based on the small target representation vector.

2. The small target detection method based on a state-space dual model encoder according to claim 1, characterized in that, The first fusion unit performs contextual fusion of features at adjacent scales as follows: The first and second features of the input are adjusted by channels respectively to obtain the first channel aligned feature and the second channel aligned feature; A linear transformation is performed on the first channel alignment feature to obtain the first query feature and the second query feature; The first query feature is subjected to depthwise separable convolution to obtain the key feature, and the second query feature is input into the first activation function to obtain the activation feature; Input the first query feature and key feature into the first state space dual model to obtain the associated features; The association feature and the activation feature are multiplied element-wise to obtain the first intermediate feature; A linear transformation is performed on the first intermediate feature to obtain the second intermediate feature; The first channel alignment feature and the second intermediate feature are residually connected to obtain the third intermediate feature; The third intermediate feature is input into the first feedforward neural network to obtain the first feedforward enhanced feature; The first feedforward enhancement feature and the third intermediate feature are residually connected to obtain the fourth intermediate feature; The fourth intermediate feature is residually connected with the second channel alignment feature to obtain the output of the first fusion unit.

3. The small target detection method based on a state-space dual model encoder according to claim 1, characterized in that, The global-local multi-scale feature interaction module includes a global-local attention mechanism and a second feedforward neural network. The global-local attention mechanism includes a global attention mechanism and a local attention mechanism. The process by which the global-local multi-scale feature interaction module generates enhanced features is as follows: The global attention mechanism is used to model the global context of the fused features to obtain globally enhanced features; Local spatial modeling of fused features is performed using a local attention mechanism to output locally enhanced features; The global enhancement features and local enhancement features are added element-wise, and the result of the addition is linearly transformed to obtain the global-local features; After adding the global-local features and the fused features element by element, the primary enhanced features are obtained; The primary enhancement features are input into the second feedforward neural network to obtain the second feedforward enhancement features; The second feedforward enhancement feature is added element-wise to the primary enhancement feature to generate the enhancement feature.

4. The small target detection method based on a state-space dual model encoder according to claim 3, characterized in that, The global attention mechanism performs global context modeling of the fused features as follows: A linear transformation is performed on the fused features to obtain a first global transformation feature and a second global transformation feature; The first global transformation feature is convolved to obtain the convolution-enhanced global feature. The convolution-enhanced global features and the first global transformation features are input into the second state-space dual model to obtain intermediate global features. The second global transformation feature is input into the second activation function to obtain the activated global feature; The global enhanced feature is obtained by multiplying the activated global feature element-wise with the intermediate global feature.

5. The small target detection method based on a state-space dual model encoder according to claim 3, characterized in that, The process by which the local attention mechanism performs local spatial modeling of the fused features is as follows: A linear transformation is performed on the fused features to obtain a first local transformation feature and a second local transformation feature; The first local transformation feature is convolved to obtain convolutionally enhanced local features. The convolution-enhanced local features are respectively added with a first position code and a second position code to obtain the local features after the first encoding and the local features after the second encoding. The first encoded local features are linearly transformed to obtain the first modulation features. The first modulation features are dynamically locally sampled, and the feature vector of each sampling point is obtained by bilinear interpolation. Attention weights are assigned to the feature vectors of each sampling point and then aggregated to obtain local aggregated features. A linear transformation is performed on the second encoded local features to obtain the second modulation features. The second modulation features are then fused with the local aggregated features to obtain the modulated local features. The second local transformation feature is input into the third activation function to obtain the activated local feature; The activated local features are multiplied element-wise with the modulated local features to obtain the local enhanced features.

6. The small target detection method based on a state-space dual model encoder according to claim 1, characterized in that, The target detection neural network is an end-to-end detector based on the Transformer architecture.

7. A small target detection system based on a state-space dual model encoder, used to implement the small target detection method according to any one of claims 1 to 6, characterized in that, It includes an image acquisition module and a detection processing module. The image acquisition module is used to acquire the target image to be detected. The detection processing module includes a pre-trained target detection neural network, which is used to perform small target detection on the target image and output the detection result.

Images