Unmanned aerial vehicle small target detection method based on motion perception space-time fusion
By using the STFF module based on motion perception spatiotemporal fusion and the dual-feature perception attention module, the problems of accuracy and computational complexity in UAV small target detection under complex backgrounds are solved, achieving high-precision, real-time small target detection, which is suitable for deployment on edge devices.
Patent Information
- Application Number
- CN202510818318.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-18
- Publication Date
- 2025-11-14
AI Technical Summary
Existing UAV small target detection methods struggle to effectively distinguish small targets from background noise in multi-target and complex environments. They lack the ability to finely model the motion information of small targets and have high computational complexity, making them unsuitable for deployment on edge devices.
A motion-aware spatiotemporal fusion-based detection method is adopted. The STFF module deeply fuses the spatiotemporal information of adjacent frames at the feature level, introduces a dual feature-aware attention module to model spatiotemporal dependencies within a local window, and improves detection accuracy through multi-level feature fusion, supporting end-to-end training and inference.
It significantly improves the detection accuracy of small targets in complex backgrounds and occluded environments, reduces the false negative and false positive rates, improves the efficiency of motion information utilization, simplifies the model development and deployment process, and is suitable for deployment in edge computing devices.
Smart Images
Figure CN120953571A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to small target detection technology, specifically to a method for detecting small targets on unmanned aerial vehicles (UAVs) based on motion perception spatiotemporal fusion. Background Technology
[0002] With the rapid development of drone technology, the automatic detection and identification of aerial drones has become a core technological challenge in several key areas such as security monitoring, counter-terrorism, and airspace management. However, drone targets generally have characteristics such as small size, low image proportion, and inconspicuous appearance, and are often in motion, making them susceptible to motion blur, target occlusion, and background interference, severely limiting the performance of traditional detection methods. Traditional static image target detection methods (such as YOLO and Faster R-CNN) often suffer from limited performance in detecting small targets due to a lack of temporal information. To compensate for the shortcomings of traditional static methods in temporal modeling, researchers have recently proposed various spatiotemporal feature fusion methods based on video sequences, attempting to fully exploit the temporal and motion information between adjacent frames to improve the detection performance of small targets in dynamic scenes.
[0003] Zhu et al. proposed a feature alignment and fusion method based on optical flow. They estimated the pixel-level motion field between adjacent frames using an optical flow network, warped the feature map of the previous frame to the reference coordinate system of the current frame, and fused temporal features through a weighted mechanism, thereby improving detection performance. Bertasius et al. constructed a multi-frame feature fusion framework based on deformable convolution. This method uses ResNet-101 to extract features from the reference and support frames, dynamically samples high-quality feature points using a learnable spatial offset prediction mechanism, and aggregates multi-frame information through weighted fusion to enhance detection capabilities. Liu et al. extended the self-attention mechanism of Transformer to the spatiotemporal domain, achieving global modeling of multi-frame features. They divided the input video sequence into several spatial windows, independently performed self-attention calculation within each window to capture local spatiotemporal dependencies, and achieved cross-window information interaction through a window shifting mechanism. After multi-layer stacking, they fused local and global spatiotemporal features to improve detection performance.
[0004] Although the above methods have improved the detection capability of small targets to some extent, existing UAV small target detection methods still have the following shortcomings:
[0005] 1. Limited small target detection capability: In scenarios with multiple targets or complex backgrounds, existing methods struggle to effectively distinguish small targets from background noise, resulting in high rates of missed detections and false detections.
[0006] 2. It lacks the ability to model the motion information of small targets in a detailed way, making it difficult to capture complex spatiotemporal dependencies at the same time, and its utilization of motion information is limited;
[0007] 3. While global modeling based on Transformer or multi-layer deformable convolutional structures offer improved accuracy, their computational complexity is high and the model structure is large, making them unsuitable for deployment on edge devices or in real-time detection scenarios, thus limiting their practicality and scalability.
[0008] Therefore, there is an urgent need for an innovative detection method that has stronger motion perception capabilities, high efficiency in spatiotemporal feature fusion, and takes into account computational resource constraints, in order to achieve high-precision, real-time detection of small target UAVs in motion. Summary of the Invention
[0009] To overcome the aforementioned problems, the inventors conducted intensive research and designed a method for detecting small targets on UAVs based on motion-aware spatiotemporal fusion. This method introduces an innovative STFF module, which deeply fuses the spatiotemporal information of adjacent frames at the feature level, significantly improving the ability to distinguish moving targets from the background. Especially in complex backgrounds or occluded environments, it significantly enhances the detection accuracy of small targets on moving UAVs. Furthermore, this method takes into account the computational resource constraints of practical applications, supports end-to-end training and inference, and is suitable for deployment in resource-constrained scenarios such as edge computing devices and embedded systems, thus completing this invention.
[0010] Specifically, the purpose of this invention is to provide a method for detecting small targets on UAVs based on motion perception spatiotemporal fusion. In this method, the current frame image is acquired and the previous consecutive frame images are retrieved. Multi-level feature extraction is performed on the two frames through the backbone network to obtain the fused features. The fused features are further optimized and enhanced in the Neck stage, and the target detection result is finally output in the Head stage.
[0011] The backbone network includes spatiotemporal feature fusion modules in multiple core layers.
[0012] The spatiotemporal feature fusion module operates through the following steps:
[0013] Step 1: Input the features of the current frame and the features of the previous frame, and align the sizes of the two.
[0014] Step 2: Introduce a dual-feature perception attention module to obtain motion and appearance features between the current frame and the previous frame;
[0015] Step 3: Generate gate weights based on the extracted motion features, and perform weighted and spatial deformation operations on the features of the previous frame to obtain finely aligned features of the previous frame.
[0016] Step 4, dynamic feature fusion, involves concatenating the current frame features, appearance features, and aligned features from the previous frame, generating three adaptive fusion weights through convolution and Softmax operations; then using these three adaptive fusion weights to perform weighted fusion of the three types of features to obtain rich intermediate fusion features;
[0017] Step 5, residual enhancement fusion, which involves performing residual weighted fusion of the intermediate fused features and the original current frame features to obtain the fused features.
[0018] The backbone network includes at least three consecutive core layers, each containing a spatiotemporal feature fusion module. Each core layer performs feature extraction on two frames of images at one level.
[0019] In the second and subsequent core-level spatiotemporal feature fusion modules, the fused features output by the previous spatiotemporal feature fusion module are used as the feature input for the current frame.
[0020] Specifically, in the dual-feature perception attention module described in step 2, the process of obtaining motion features and appearance features includes the following sub-steps:
[0021] Sub-step 1: Input the feature tensors of the current frame and the previous frame;
[0022] Sub-step 2: Rearrange the features of each frame from [B,N,C] format to [B,C,H,W] for window division;
[0023] Sub-step 3: Divide the rearranged two-dimensional feature map into several non-overlapping local windows;
[0024] Sub-step 4: Flatten the window and obtain the position code:
[0025] Sub-step 5: Apply a linear mapping to the feature sequence of the current frame window to obtain the query vector Q; generate the key K and value vector V for the feature sequence of the previous frame window;
[0026] Sub-step 6: Execute a multi-head self-attention mechanism within each window and calculate the attention weights;
[0027] Sub-step 7: Use attention weights to perform a weighted summation of the value vector V to obtain the appearance features within the window;
[0028] The position encoding Pos is weighted and summed using the same attention weights, and the original coordinates are subtracted to obtain the difference representing the motion offset. The motion features are then obtained through linear projection.
[0029] Sub-step 8 restores the processed features in each window to their original spatial layout, reconstructing a complete global feature map.
[0030] In sub-step 1, the shape of the feature tensor is [B,N,C], where B represents the batch size, i.e. the number of samples, N = H × W represents the number of spatial locations, C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.
[0031] In sub-step 3, the size of the local window is set to (s,s), and the shape of each local window after division is [B×nH×nW,C,s,s], where nH represents the number of local windows in the vertical direction and nW represents the number of local windows in the horizontal direction.
[0032] In sub-step 4, each window is flattened to [B×nH×nW,s] 2 The sequence is in the form of [,C], and normalized two-dimensional coordinates in the range of [-1,1] are generated for each spatial location. These coordinates are then mapped to motion_dim-dimensional position embeddings through a fully connected layer to obtain the position encoding Pos.
[0033] In sub-step 6, the attention weights are obtained using the following equation (a):
[0034]
[0035] Where Attn represents the attention weight;
[0036] This represents the scaling factor.
[0037] The beneficial effects of this invention include:
[0038] (1) According to the motion-sensory spatiotemporal fusion-based UAV small target detection method provided by the present invention, the method significantly enhances the detection capability of small targets by fusing multi-scale and multi-level spatiotemporal features through the STFF module. Especially in complex backgrounds and occlusion situations, it can effectively distinguish targets from background noise and reduce the rate of missed detection and false detection.
[0039] (2) According to the motion-sensory spatiotemporal fusion-based UAV small target detection method provided by the present invention, the method performs self-attention calculation in a local window through a dual-feature perception attention module, which can efficiently capture the spatiotemporal dependency between consecutive frames and improve the efficiency of motion information utilization.
[0040] (3) According to the motion-sensory spatiotemporal fusion-based UAV small target detection method provided by the present invention, all modules in the method support end-to-end training and inference, without the need for additional optical flow networks or complex multi-stage optimization processes, which simplifies the model development and deployment process. Attached Figure Description
[0041] Figure 1A logic diagram illustrating the working process of the detection method of this application is shown;
[0042] Figure 2 This diagram illustrates the working process logic of the spatiotemporal feature fusion module.
[0043] Figure 3 The working process logic diagram of the dual-feature perception attention module is shown;
[0044] Figure 4 The map50 performance metrics in the embodiment are shown;
[0045] Figure 5 The map50-95 performance metrics are shown in the example.
[0046] Figure 6 The PR curve diagram in the embodiment is shown. Detailed Implementation
[0047] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Through these descriptions, the features and advantages of the present invention will become clearer and more apparent.
[0048] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.
[0049] This invention provides a method for detecting small targets on unmanned aerial vehicles (UAVs) based on motion-sensory fusion, wherein, as shown in the invention... Figure 1 As shown, the current frame image is acquired and the previous consecutive frame images are retrieved. Multi-level feature extraction is performed on the two frames through the backbone network to obtain the fused features. The fused features are further optimized and enhanced in the Neck stage, and the target detection results are finally output by the Head stage.
[0050] The backbone network includes multiple core layers that incorporate a spatiotemporal feature fusion module, STFF. It also includes an SPPF module and a C2PSA module. The SPPF module is a variant or faster version of Spatial Pyramid Pooling (SPP), designed to improve processing speed and efficiency. The C2PSA module is a high-level module in YOLOv11 used to enhance feature extraction. The fused features output by the STFF module in the last core layer of this application are processed by the SPPF and C2PSA modules before being passed to the Neck stage.
[0051] In a preferred embodiment, such as Figure 2As shown, the working process of the spatiotemporal feature fusion module includes the following steps:
[0052] Step 1, Feature Input and Alignment: Input the features of the current frame and the features of the previous frame, and align the two in size to ensure consistency in subsequent fusion operations;
[0053] Step 2, Spatiotemporal Dependency Modeling: A dual-feature perception attention module is introduced to simultaneously model the spatial and temporal dependencies within a local window; the motion and appearance features between the current frame and the previous frame are obtained through this dual-feature perception attention module, thereby enhancing the ability to model dynamic targets.
[0054] Step 3, fine motion alignment: Based on the extracted motion features, gating weights are generated, and weighted and spatial deformation (i.e., warping) operations are performed on the features of the previous frame to achieve fine-grained alignment of the features of the previous frame, improve the consistency of features across frames, and obtain finely aligned features of the previous frame.
[0055] Step 4, dynamic feature fusion, involves concatenating the current frame features, appearance features, and finely aligned features from the previous frame, generating three adaptive fusion weights through convolution and Softmax operations; then using these three adaptive fusion weights to perform weighted fusion of the three types of features to obtain richly expressive intermediate fusion features;
[0056] Step 5, residual enhancement fusion, involves performing residual weighted fusion of intermediate fused features and original current frame features to effectively retain key information of the current frame while introducing cross-frame dynamic information to obtain fused features, which are used to support the high-precision execution of downstream detection tasks.
[0057] Preferably, the backbone network has at least three consecutive core layers, each core layer includes a spatiotemporal feature fusion module, and each core layer performs feature extraction on two frames of images at one level.
[0058] Preferably, in the second and subsequent core-level spatiotemporal feature fusion modules, the fused features output by the previous spatiotemporal feature fusion module are used as the feature input for the current frame.
[0059] In a preferred embodiment, the dual-feature perception attention module described in step 2 innovatively models feature dependencies in both spatial and temporal dimensions simultaneously within each local window through a dual-feature self-attention mechanism. It also extracts appearance and motion features from the current and previous frames, effectively capturing the dynamic changes of moving targets, while significantly reducing computational overhead through local operations. This dual-feature perception attention module combines high expressiveness and high efficiency, making it a key unit for achieving high-performance spatiotemporal modeling.
[0060] Specifically, such as Figure 3 As shown, the process of obtaining motion features and appearance features in this dual-feature perception attention module includes the following sub-steps:
[0061] Sub-step 1, Feature Input: Input the feature tensors of the current frame and the previous frame. The shape of the feature tensor is [B,N,C], where B represents the batch size, i.e. the number of samples, N = H × W represents the number of spatial locations, C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.
[0062] Sub-step 2, Feature rearrangement: rearrange the features of each frame from [B,N,C] format to [B,C,H,W] in order to perform window division;
[0063] Sub-step 3, window partitioning: Divide the rearranged 2D feature map into several non-overlapping local windows; the size of each local window is set to (s, s), and the shape of each local window after partitioning is [B×nH×nW, C, s, s], where nH represents the number of local windows in the vertical direction, and nW represents the number of local windows in the horizontal direction. The value of s is 4-16, preferably 8.
[0064] Sub-step 4, flatten the window and obtain the position code: Specifically, flatten each window into [B×nH×nW,s] 2 The sequence is in the form of [,C], and normalized two-dimensional coordinates in the range of [-1,1] are generated for each spatial location. These coordinates are then mapped to motion_dim-dimensional position embeddings through a fully connected layer to obtain the position encoding Pos.
[0065] Sub-step 5, Query-Key-Value Generation: Apply a linear mapping to the feature sequence of the current frame window to obtain the query vector Q; generate the key K and value vector V for the feature sequence of the previous frame window;
[0066] Sub-step 6, In-window attention calculation: Execute a multi-head self-attention mechanism within each window to calculate attention weights;
[0067] Specifically, attention weights are obtained through the following formula (i):
[0068]
[0069] Where Attn represents the attention weight;
[0070] This represents the scaling factor used for stabilizing training. In practical applications, this... The value ranges from 8 to 16;
[0071] Sub-step 7, Feature and location fusion: Use attention weights to perform a weighted summation of the value vector V to obtain the appearance features within the window;
[0072] The position encoding Pos is weighted and summed using the same attention weights, and the original coordinates are subtracted to obtain the difference representing the motion offset. The motion features are then obtained through linear projection.
[0073] Sub-step 8, Window Restoration: Restore the processed features in each window to their original spatial layout and reconstruct the complete global feature map.
[0074] Sub-step 9, Result Output: The final output includes appearance features and motion features, which are used for visual modeling and motion perception in subsequent modules, respectively.
[0075] The STFF (Spatiotemporal Feature Fusion Module) in this application needs to be trained and flushed with data before use. The initial learning rate is 0.01 and the number of training rounds is 200. After training, the module can dynamically generate query vector Q, key K, and value vector V based on the input features and the learned parameters.
[0076] Example
[0077] A drone image dataset was retrieved, consisting of 8 consecutive video sequences totaling 5496 frames. This dataset was divided into training and validation sets in a 7:3 ratio. An additional drone image dataset, including 8 consecutive sequences not used in training (totaling 5690 frames), was retrieved as a test set for independent evaluation of detection performance. Simulations were performed on a high-performance computing platform equipped with dual NVIDIA GeForce RTX 3090 GPUs.
[0078] The key parameters for training are set as follows: image input resolution is 640×640, the optimization algorithm is stochastic gradient descent (SGD), the momentum factor is set to 0.937, the initial learning rate is 0.01, the number of training epochs is 200, and the batch size is 32.
[0079] The models used in the training were the YOLO11S model, the YOLO11-P2 model, and the improved YOLO11-P2 model. The improved YOLO11-P2 model is the YOLO11-P2-motion model, which incorporates the STFF module.
[0080] The YOLOv11 S model and YOLOv11-P2 model are selected from the models described in the literature Khanam R, Hussain M. YOLOv11: An overview of the key architectural enhancements[J]. arXiv preprint arXiv:2410.17725,2024;
[0081] The working process of the YOLO11-P2-Motion model is as follows:
[0082] The current frame image is acquired and the previous consecutive frame images are retrieved. Multi-level feature extraction is performed on the two frames through the backbone network to obtain the fused features. The fused features are further optimized and enhanced in the Neck stage, and the target detection results are finally output by the Head stage.
[0083] The backbone network includes spatiotemporal feature fusion modules in multiple core layers.
[0084] The working process of the spatiotemporal feature fusion module includes the following steps:
[0085] Step 1: Input the features of the current frame and the features of the previous frame, and align the sizes of the two.
[0086] Step 2: Introduce a dual-feature perception attention module to obtain motion and appearance features between the current frame and the previous frame;
[0087] Step 3: Generate gate weights based on the extracted motion features, and perform weighted and spatial deformation operations on the features of the previous frame to obtain finely aligned features of the previous frame.
[0088] Step 4, dynamic feature fusion, involves concatenating the current frame features, appearance features, and aligned features from the previous frame, generating three adaptive fusion weights through convolution and Softmax operations; then using these three adaptive fusion weights to perform weighted fusion of the three types of features to obtain rich intermediate fusion features;
[0089] Step 5, residual enhancement fusion, which involves performing residual weighted fusion of the intermediate fused features and the original current frame features to obtain the fused features.
[0090] In the dual-feature perception attention module described in step 2, the process of obtaining motion features and appearance features includes the following sub-steps:
[0091] Sub-step 1, Feature Input: Input the feature tensors of the current frame and the previous frame. The shape of the feature tensor is [B,N,C], where B represents the batch size, i.e. the number of samples, N = H × W represents the number of spatial locations, C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.
[0092] Sub-step 2, Feature rearrangement: rearrange the features of each frame from [B,N,C] format to [B,C,H,W] in order to perform window division;
[0093] Sub-step 3, window partitioning: Divide the rearranged 2D feature map into several non-overlapping local windows; the size of each local window is set to (s, s), and the shape of each local window after partitioning is [B×nH×nW, C, s, s], where nH represents the number of local windows in the vertical direction, and nW represents the number of local windows in the horizontal direction. s takes the value of 8;
[0094] Sub-step 4, flatten the window and obtain the position code: Specifically, flatten each window into [B×nH×nW,s] 2 The sequence is in the form of [,C], and a normalized two-dimensional coordinate in the range of [-1,1] is generated for each spatial location. This coordinate is then mapped to a motion_dim-dimensional position embedding through a fully connected layer to obtain the position code Pos.
[0095] Sub-step 5, Query-Key-Value Generation: Apply a linear mapping to the feature sequence of the current frame window to obtain the query vector Q; generate the key K and value vector V for the feature sequence of the previous frame window;
[0096] Sub-step 6, In-window attention calculation: Execute a multi-head self-attention mechanism within each window to calculate attention weights;
[0097] Specifically, attention weights are obtained through the following formula (i):
[0098]
[0099] Where Attn represents the attention weight;
[0100] This represents the scaling factor used for stabilizing training. In practical applications, this... The value range is [8, 16];
[0101] Sub-step 7, Feature and location fusion: Use attention weights to perform a weighted summation of the value vector V to obtain the appearance features within the window;
[0102] The position encoding Pos is weighted and summed using the same attention weights, and the original coordinates are subtracted to obtain the difference representing the motion offset. The motion features are then obtained through linear projection.
[0103] Sub-step 8, Window Restoration: Restore the processed features in each window to their original spatial layout and reconstruct the complete global feature map.
[0104] Sub-step 9, Result Output: The final output includes appearance features and motion features, which are used for visual modeling and motion perception in subsequent modules, respectively.
[0105] The performance metrics of map50 obtained from the training results are as follows: Figure 4 As shown, the performance metrics of map50-95 are as follows: Figure 5 As shown.
[0106] The plotted PR (Precision-Recall) curve is as follows: Figure 6 As shown, the horizontal axis represents recall, and the vertical axis represents precision. Comparing the three methods yolol1s, yolol1-p2, and yolol1-p2-motion, it can be observed that the yolol1-p2-motion proposed in this invention exhibits higher detection precision in most recall ranges.
[0107] The trained model was tested on the test set, and the results are shown in Table 1 below. It can be seen that the yolol1-p2-motion detection results with the STFF module are significantly improved, indicating that this method effectively utilizes motion perception to achieve feature fusion, effectively improving detection performance and fully demonstrating the technical feasibility and practical application value of the proposed method.
[0108] Table 1 Comparison of test results for different models
[0109] Model Precision Recall mAP50 mAP50-95 Yolo11s 0.6 0.257 0.293 0.115 Yolo11-p2 0.64 0.357 0.368 0.144 Yolo11-p2-motion 0.755 0.362 0.504 0.226
[0110] The present invention has been described above with reference to preferred embodiments; however, these embodiments are merely exemplary and illustrative. Various substitutions and modifications can be made to the present invention based on these embodiments, all of which fall within the scope of protection of the present invention.
Claims
1. A method for detecting small targets on unmanned aerial vehicles (UAVs) based on motion perception spatiotemporal fusion, characterized in that, In this method, the current frame image is acquired and the previous consecutive frame images are retrieved. Multi-level feature extraction is performed on the two frames through the backbone network to obtain the fused features. The fused features are further optimized and enhanced in the Neck stage, and the target detection results are finally output by the Head stage. The backbone network includes spatiotemporal feature fusion modules in multiple core layers.
2. The method for detecting small targets on unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 1, characterized in that, The working process of the spatiotemporal feature fusion module includes the following steps: Step 1: Input the features of the current frame and the features of the previous frame, and align the sizes of the two. Step 2: Introduce a dual-feature perception attention module to obtain motion and appearance features between the current frame and the previous frame; Step 3: Generate gate weights based on the extracted motion features, and perform weighted and spatial deformation operations on the features of the previous frame to obtain finely aligned features of the previous frame. Step 4, dynamic feature fusion, involves concatenating the current frame features, appearance features, and aligned features from the previous frame, generating three adaptive fusion weights through convolution and Softmax operations; then using these three adaptive fusion weights to perform weighted fusion of the three types of features to obtain rich intermediate fusion features; Step 5, residual enhancement fusion, which involves performing residual weighted fusion of the intermediate fused features and the original current frame features to obtain the fused features.
3. The method for detecting small targets on unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 2, characterized in that, The backbone network includes at least three consecutive core layers, each containing a spatiotemporal feature fusion module. Each core layer performs a feature extraction at one level on two frames of images.
4. The UAV small target detection method based on motion perception spatiotemporal fusion according to claim 3, characterized in that, In the second and subsequent core-level spatiotemporal feature fusion modules, the fused features output by the previous spatiotemporal feature fusion module are used as the feature input for the current frame.
5. The method for detecting small targets on unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 2, characterized in that, In the dual-feature perception attention module described in step 2, the process of obtaining motion features and appearance features includes the following sub-steps: Sub-step 1: Input the feature tensors of the current frame and the previous frame; Sub-step 2: Rearrange the features of each frame from [B,N,C] format to [B,C,H,W] for window division; Sub-step 3: Divide the rearranged two-dimensional feature map into several non-overlapping local windows; Sub-step 4: Flatten the window and obtain the position code: Sub-step 5: Apply a linear mapping to the feature sequence of the current frame window to obtain the query vector Q; generate the key K and value vector V for the feature sequence of the previous frame window; Sub-step 6: Execute a multi-head self-attention mechanism within each window and calculate the attention weights; Sub-step 7: Use attention weights to perform a weighted summation of the value vector V to obtain the appearance features within the window; The position encoding Pos is weighted and summed using the same attention weights, and the original coordinates are subtracted to obtain the difference representing the motion offset. The motion features are then obtained through linear projection. Sub-step 8 restores the processed features in each window to their original spatial layout, reconstructing a complete global feature map.
6. The method for detecting small targets on unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 5, characterized in that, In sub-step 1, the shape of the feature tensor is [B,N,C], where B represents the batch size, i.e. the number of samples, N = H × W represents the number of spatial locations, C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.
7. The method for detecting small targets on unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 5, characterized in that, In sub-step 3, the size of the local window is set to (s,s), and the shape of each local window after division is [B×nH×nW,C,s,s], where nH represents the number of local windows in the vertical direction and nW represents the number of local windows in the horizontal direction.
8. The method for detecting small targets in unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 5, characterized in that, In sub-step 4, each window is flattened to [B×nH×nW,s] 2 The sequence is in the form of [,C], and normalized two-dimensional coordinates in the range of [-1,1] are generated for each spatial location. These coordinates are then mapped to motion_dim-dimensional position embeddings through a fully connected layer to obtain the position encoding Pos.
9. The method for detecting small targets in unmanned aerial vehicles based on motion perception spatiotemporal fusion according to claim 5, characterized in that, In sub-step 6, the attention weights are obtained using the following equation (a): Where Attn represents the attention weight; This represents the scaling factor.