Aerial traffic small target detection method based on structure prior and space-frequency gaussian cooperation

Abstract

The application provides a structure prior and space-frequency Gaussian cooperative aerial traffic small target detection method, relates to the technical field of unmanned aerial vehicle aerial complex traffic scene and image acquisition, and comprises the following steps: firstly, constructing an unmanned aerial vehicle aerial image into a network input tensor, inputting the network input tensor into a main semantic detection branch and a structure prior auxiliary branch for multi-scale feature extraction and feature scale alignment; introducing an improved structure guided Gaussian space-frequency enhancement block into the main semantic detection branch for feature enhancement processing, and simultaneously reconstructing and fusing Gaussian channel weights and multi-peak Gaussian space weights to obtain enhanced main branch features; then, performing same-scale adaptive deep fusion on the enhanced main branch features and structure prior features provided by the structure prior auxiliary branch; and finally, outputting the position and category results of small targets through a high-resolution detection neck and a decoupled detection head. The application can solve the problems of unclear small targets and unstable outlines, and realize stable detection of unmanned aerial vehicle aerial small targets in a complex traffic scene.

Description

Technical Field

[0001] This invention relates to the field of drone aerial photography of complex traffic scenes and image acquisition technology, and in particular to a method for detecting small traffic targets in aerial photography using a combination of structural prior and spatial frequency Gaussian collaboration. Background Technology

[0002] With the widespread application of drone technology in intelligent transportation, target detection based on drone aerial imagery has become a core component in applications such as traffic flow statistics, intersection operation monitoring, congestion analysis, and emergency patrols. Currently, mainstream detection methods include two-stage detectors based on convolutional neural networks, single-stage detectors, and end-to-end detectors based on Transformers. Researchers have also made numerous improvements for drone scenarios in areas such as data augmentation, multi-scale feature fusion, and lightweight network design, which have enhanced detection performance to some extent.

[0003] However, existing technologies still have significant shortcomings in small target detection tasks for complex traffic scenes captured by drones. Due to the high-altitude overhead imaging of drones, targets such as vehicles, pedestrians, and non-motorized vehicles occupy only a few pixels in the image. After multiple downsampling by the network, their weak feature responses are severely diluted, resulting in "small targets being unclear." At the same time, targets in traffic scenes are densely distributed and mutually occluded. Coupled with strong interference from complex backgrounds such as road textures, shadows, and building edges, it is difficult to distinguish targets from the background, resulting in serious feature confusion. In addition, existing methods are unstable in regressing the boundaries of small targets, with large fluctuations in the position and size of the detection box and unclear contour boundaries, making it difficult to meet the positioning accuracy and stability requirements of traffic monitoring. Furthermore, it is difficult to balance the high-precision model with the real-time requirements of embedded platforms.

[0004] Therefore, there is an urgent need for an aerial traffic target detection method that can solve the problems of "small targets being unclear and having unstable outlines" under conditions of high-altitude perspective, small target size, dense distribution, severe occlusion, and complex background. Summary of the Invention

[0005] To address the aforementioned technical problems in related technologies, this invention proposes an aerial traffic small target detection method based on structural prior and spatial frequency Gaussian coordination, comprising the following steps:

[0006] S1. After preprocessing the original UAV aerial images, construct the network input tensors in batches.

[0007] S2. The network input tensor is simultaneously input into the main semantic detection branch and the structural prior auxiliary branch for multi-scale feature extraction, and then the feature scale is aligned to obtain the main branch features and structural prior features.

[0008] S3. Input the main branch features and structural prior features into the improved structure-guided Gaussian space-frequency enhancement block to obtain mixed-scale spatial modeling features, structure-guided Gaussian channel spatial enhancement features and frequency domain enhancement features. Then, perform residual aggregation at each scale to obtain the enhanced main branch features. The improved structure-guided Gaussian space-frequency enhancement block includes a mixed-scale spatial modeling branch, a structure-guided Gaussian channel spatial enhancement branch and a frequency domain enhancement branch.

[0009] S4. Perform same-scale gated deep fusion and calibration on the enhanced main branch features and structural prior features to obtain deep fused features;

[0010] S5. After lateral mapping of each scale of the deep fusion features, the features are fused from top to bottom and then input into the decoupled detection head for prediction. The prediction results are then processed by candidate box decoding, threshold filtering and non-maximum suppression to obtain the aerial small target detection results in complex traffic scenarios.

[0011] Specifically, step S1 includes the following steps:

[0012] S11. Uniform Size Resampling and Channel Normalization: Resample the original UAV aerial image I to a fixed size and perform RGB per-channel normalization to obtain a standardized image. ;

[0013] S12, Overlapping Cut: Uses preset window size Sliding window method with preset step size for standardized images Overlapping and slicing results in a set of slices. ;in, To cut into pieces; This refers to the number of pieces; This is the height of the cut piece; Where n is the slice width and n is the slice index;

[0014] S13. Network Input Tensor Generation: Transforming the Segment Set The network input tensors are obtained by organizing and stacking them in batches while maintaining the channel order and scale information. ;in, This represents the batch size; 3 indicates RGB three-channel.

[0015] Specifically, step S2 includes the following steps:

[0016] S21. In the main semantic detection branch, the network input tensor X0 is input into the hierarchical visual backbone network, and the main branch scale features of each scale are extracted according to the four-level stage. To obtain the main branch features The hierarchical visual backbone network includes an input embedding layer, four stages, inter-stage downsampling layers, a normalization layer, and a multi-scale output layer; the four stages are cascaded.

[0017] S22. Simultaneously, in the structural prior auxiliary branch, the network input tensor X0 is input into the SAM image encoder to extract the original structural prior sub-features at four scales. Constructing the prior features of the original structure ;

[0018] S23. Prior features of the original structure The original structural prior features at each scale are subjected to size transformation and channel mapping, and then to main branch features. Corresponding scale alignment yields prior structural features. .

[0019] Specifically, step S3 includes the following steps:

[0020] S31, Characteristics of Main Branches Main branch scale features at each scale Perform layer normalization to obtain normalized main branch scale features This leads to the normalized principal branch features. ;

[0021] S32. The three-layer mixed-scale space modeling constructed in the mixed-scale space modeling branch normalizes the main branch features. After multi-scale feature extraction and optimization recovery, the feature is fused with the gated signal generated by linear mapping and activation function to obtain mixed-scale spatial modeling features. ;

[0022] S33. Based on the characteristics of the normalized principal branch and structural prior features In the structure-guided Gaussian channel space enhancement feature branch, Gaussian channel weights and multi-peak Gaussian space weights are constructed and coupled to normalize the main branch features. Normalized principal branch scale features Enhancement is performed to obtain the spatial enhancement features of the structure-guided Gaussian channel. ; Guide the Gaussian channel spatial enhancement sub-feature for the structure at the i-th scale;

[0023] S34. Based on Normalized Main Branch Features In the frequency domain enhancement branch, a fast Fourier transform is performed to obtain a complex spectrum, which is then decomposed into an amplitude spectrum and a phase spectrum. After differential convolution coding, the spectrum is fused and reconstructed, and then restored to the spatial domain and the normalized main branch features by an inverse fast Fourier transform. Residual connections are performed on the normalized principal branch scale features to obtain frequency domain enhanced features. ; For the frequency domain enhancer feature at the i-th scale;

[0024] S35, Model the hybrid scale space features Structurally guided Gaussian channel spatial enhancement features and frequency domain enhancement features Performing residual aggregation at each scale yields enhanced main branch features. ; Let i be the enhanced scale semantic sub-feature of the i-th scale.

[0025] Specifically, step S4 includes the following steps:

[0026] S41. Each structural prior sub-feature in the structural prior features is mapped to a projected structural prior sub-feature using a 1×1 convolutional layer. :

[0027] S42, the prior sub-features of the projection structure Augmented scale semantic sub-features at the same scale as the augmented main branch features The data is stitched together along the channel dimension and then fed into a convolutional layer to generate scale fusion gating for each scale.

[0028] S43. Based on the scale fusion gating and enhanced scale semantic sub-features Calculate the weighted sub-features of the main branch at each scale. Simultaneously, based on scale fusion gating and projection structure prior sub-features Calculate the structural prior weighted sub-features at each scale :

[0029] ,

[0030] ,

[0031] In the formula, This represents element-wise multiplication; This represents the weighted sub-features of the main branch after gating and weighting; This represents the structural prior weighted sub-features after complementary gating weighting; This represents a structural prior sub-gating that is complementary to scale-fusion gating. This represents a 1×1 convolutional layer; This represents the scale fusion gating at the i-th scale;

[0032] S44. Weight the sub-features of the main branch. Structural prior weighted sub-features Residual fusion is performed with the enhanced scale semantic sub-features to obtain the initial fused features for each scale. ;

[0033] S45, Based on the initial fusion features Convolutional calibration and residual preservation are performed to obtain the deep fusion sub-features at each scale, as shown in the following equation:

[0034] ,

[0035] In the formula, The initial fusion sub-features at the i-th scale; Represents the deep fusion sub-feature at the i-th scale; This represents a 3×3 convolutional layer.

[0036] Specifically, step S5 includes the following steps:

[0037] S51, The deep fusion feature Lateral mapping is performed at each scale, and the data is then fused from top to bottom to obtain the pyramid detection sub-features at each scale.

[0038] S52. Input the pyramid detection sub-features of each scale into the classification head, regression head and target head of the decoupled detection head for prediction, and output the multi-scale prediction sub-results of each scale; wherein, the pyramid detection sub-features of the first scale are used as a high-resolution detection layer for small target prediction.

[0039] S53. Based on the multi-scale prediction sub-results at each scale, candidate boxes are decoded to obtain all candidate detection boxes and their category confidence scores. Then, threshold filtering and non-maximum suppression are performed to obtain the aerial small target detection results in complex traffic scenarios.

[0040] Specifically, step S33 includes the following steps:

[0041] S331. Calculate the scale feature of each normalized main branch in the normalized main branch features using deep convolutional layers. Preprocessing features Then, it is divided into the first sub-feature of the i-th scale along the channel. With the second sub-feature Constructing channel description Where GAP represents global average pooling; GMP represents global max pooling; and Concat represents concatenation operation.

[0042] S332, According to the channel description r i确 Determine the maximum response channel index The standard deviation of the Gaussian weights for each channel is determined based on the number of channels for each main branch scale feature. ; where argmax() represents the index function that retrieves the maximum value in the array; Represents the scale feature of the i-th main branch The number of channels;

[0043] S333. Using the maximum response channel index as the mean and the standard deviation of the channel Gaussian weights as the standard deviation, establish Gaussian channel weights. As shown in the following formula:

[0044] ,

[0045] In the formula, represents the Gaussian channel weight value in the u-th channel at the i-th scale; ch represents the channel domain; exp() is the natural exponential function;

[0046] S334. Calculate the scale feature of each normalized main branch in the normalized main branch feature based on the Sobel edge operator. Edge enhancement map Then, combined with structural prior features Constructing the spatial saliency map of the edge enhancement map :

[0047] ,

[0048] In the formula, This indicates that the maximum value is taken along the channel dimension; This indicates averaging along the channel dimension; Representation and normalized principal branch scale features Aligned structural prior features;

[0049] S335, From the spatial saliency map Selected from Each significant peak is identified based on its neighborhood local structure tensor. Calculate the covariance matrix ;in, A positive Gaussian scaling adjustment coefficient; It is a relatively small positive number; It is the identity matrix;

[0050] S336. Establishing Multimodal Gaussian Spatial Weights Based on Covariance Matrix As shown in the following formula:

[0051] ,

[0052] in, This represents the spatial weights of the multimodal Gaussian system at the i-th scale in spatial coordinates. The weight value at the location, where the superscript sp indicates the spatial domain; This represents the spatial coordinates of the m-th significant peak on the feature map; The weight of the m-th significant peak is represented by ; T represents the transpose. Representing spatial coordinates Two-dimensional coordinate vector;

[0053] S337. Couple the Gaussian channel weights with the multi-peak Gaussian spatial weights to obtain the structure-guided Gaussian channel spatial attention map A. i Then, based on the structure, the Gaussian channel spatial attention map A is used for guidance. i For each normalized principal branch scale feature in the normalized principal branch features Enhancement is performed to obtain the spatial enhancer feature of the structure-guided Gaussian channel. Ultimately, the spatial enhancement features of the structure-guided Gaussian channel were obtained. ;

[0054] The structure guides the spatial attention diagram of the Gaussian channel. (A) i As shown in the following formula:

[0055] ,

[0056] In the formula, , , These are the first, second, and third fusion coefficients of the channel-space attention coupling, respectively. This represents element-wise multiplication; This represents the Sigmoid function.

[0057] Specifically, step S335 involves analyzing the spatial saliency map... Selected from Each significant peak is identified based on its neighborhood local structure tensor. Specifically, it includes:

[0058] Firstly, in the spatial saliency map Select K from p We identify several significant peaks and select one of them, represented by coordinates (x, y). Then, we determine the neighborhood window based on this significant peak. Then, the first-order partial derivatives in the x and y directions of the image within the neighborhood window are calculated, the image gradient is calculated, and the structure tensor matrix of each pixel in the image within the window is constructed. Finally, the neighborhood weights are combined. Gaussian weighting of the structure tensor matrix yields the neighborhood local structure tensor. :

[0059] ,

[0060] in, The gradient in the x-direction of the image; The gradient in the y-direction of the image.

[0061] Specifically, step S32 includes the following steps:

[0062] S321. For each normalized principal branch scale feature in the normalized principal branch features... The feature map is extended along the channel dimension by linear mapping, then spatial features are extracted by deep convolutional layers, and then multi-scale feature extraction is performed by downsampling layers to obtain multi-mixed scale feature maps for each scale.

[0063] S322. The multi-scale feature map is optimized by using a two-dimensional selective scanning state space module, and the size of the multi-scale feature map is restored by using an upsampling layer to obtain the multi-scale optimized feature map.

[0064] S323. Gated signals for normalized main branch features at each scale are generated using linear mapping and activation functions:

[0065] S324. Optimize the first mixed-scale sub-feature map and the gated signal at each scale using element-wise multiplication. After fusion and linear mapping, the features are then connected to the normalized main branch features via a jump connection to obtain mixed-scale spatial modeling features.

[0066] Specifically, step S34 includes the following steps:

[0067] S341. Use Fast Fourier Transform to extract the normalized principal branch scale features from each normalized principal branch feature. Transforming from the spatial domain to the frequency domain yields the complex spectrum. ;

[0068] S342, The complex spectrum Decomposed into amplitude spectrum and phase spectrum For amplitude spectrum and phase spectrum Differential convolutional coding is performed to obtain the encoded amplitude spectrum. With the encoded phase spectrum ;in, Representing the spectrum of complex numbers amplitude, Representing the spectrum of complex numbers The phase angle;

[0069] S343, Encode the amplitude spectrum With the encoded phase spectrum After fusion and reconstruction, the data is restored to the spatial domain via inverse fast Fourier transform and then compared with the normalized principal branch scale features. Perform residual connections to obtain frequency domain enhancer features at various scales. Finally, the frequency domain enhancement features are obtained. .

[0070] This invention provides a method for small target detection in aerial traffic images based on structure prior and spatial-frequency Gaussian collaboration. First, the aerial traffic image captured by a UAV is input into a dual-branch detection network that does not require text prompts. The main branch is responsible for extracting multi-scale semantic information, while the retained SAM auxiliary branch provides more stable prior features of edges, contours, and spatial structure, thereby improving the stability of small target detection. Subsequently, a structure-guided Gaussian spatial-frequency enhancement module, modified from the spatial-frequency hybrid enhancement block SFMB, is introduced into the main branch. On the one hand, it utilizes mixed-scale modeling and frequency domain enhancement to supplement the detailed representation of small targets under different scales and texture conditions; on the other hand, it integrates Gaussian channel weights with multi-peak Gaussian... Spatial weights are coupled to obtain a structure-guided Gaussian channel spatial attention map. Combined with prior structural information in traffic scenarios, this can more effectively highlight the areas where small targets such as vehicles and pedestrians are located, while suppressing interference from road textures, shadows, and surrounding noise. Then, the enhanced main branch features and the structural prior features provided by the SAM branch are subjected to same-scale adaptive deep fusion, so that semantic and structural information complement and synergize. Finally, the location and category results of small targets are output through the detection neck of the high-resolution feature pyramid network and the decoupled detection head. This achieves stable detection of small targets in UAV aerial photography in complex traffic scenarios, solving the technical problems of "small targets being unclear and their outlines being unstable".

[0071] Furthermore, this embodiment innovatively improves the CEB in SFMB by integrating it with Gaussian channel weights and multi-peak Gaussian spatial weights, and by combining it with structural priors in traffic scenarios, enabling the network to focus more intently on dense, small-scale target areas that are susceptible to background interference. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 This is a schematic diagram of an aerial traffic small target detection method based on structural prior and spatial frequency Gaussian coordination, provided as an embodiment of the present invention. Detailed Implementation

[0074] The present invention will be explained in detail through the following embodiments. The purpose of this invention is to protect all technical improvements within its scope. In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0075] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0076] refer to Figure 1 This embodiment provides a method for detecting small traffic targets in aerial photography based on structural priors and spatial frequency Gaussian coordination, including the following steps:

[0077] S1. After preprocessing the original UAV aerial images, construct the network input tensors in batches.

[0078] First, data is collected through drone aerial photography, and the original drone aerial images collected are denoted as I;

[0079] Secondly, the original UAV aerial image I is preprocessed, including: uniform size resampling, channel normalization, and overlapping slicing, to obtain a set of slices. To ensure that vehicles, pedestrians and non-motorized vehicles at long distances still have a recognizable proportion of small target pixels when entering the detection network;

[0080] Finally, the set of slices Organize the network into a unified network input tensor, resulting in network input tensor X0.

[0081] S11. Uniform Size Resampling and Channel Normalization: Resample the original UAV aerial image I to a fixed size. Then, perform RGB per-channel normalization to obtain a standardized image. Where H is a fixed height and W is a fixed width;

[0082] The RGB per-channel normalization is used to eliminate the differences in brightness and contrast between different original drone aerial images, adjusting the image pixels to a uniform range; this is existing technology and will not be described in detail here.

[0083] S12, Overlapping Cut: Uses preset window size and preset step size Sliding window method for standardized images Overlapping and slicing results in a set of slices. In the formula, To cut into pieces; This refers to the number of pieces; This is the height of the cut piece; This is the width of the cut piece;

[0084] S13. Generation of network input tensor: Organize and stack the sliced ​​set in batches, and keep the channel order and scale information consistent to obtain the network input tensor;

[0085] Constructing the network input tensor: dividing the set of chunks By batch size The input tensor is stacked, and each slice is an RGB three-channel image with a size of [size missing]. Using a channel-first format, construct the network input tensor X0 for a single batch:

[0086] ;

[0087] The 3 in the upper right corner indicates the RGB three-channel designation.

[0088] S2. The network input tensor is simultaneously input into the main semantic detection branch and the structural prior auxiliary branch for multi-scale feature extraction, and then the feature scale is aligned to obtain the main branch features and structural prior features.

[0089] S21. In the main semantic detection branch, the network input tensor X0 is input into the hierarchical visual backbone network, and the main branch scale features of each scale are extracted according to the four-level stage. To obtain the main branch features ;

[0090] The hierarchical visual backbone network includes an input embedding layer, four stages, an inter-stage downsampling layer (channel expansion layer), a normalization layer, and a multi-scale output layer; the four stages are cascaded.

[0091] The four-stage cascade connection is shown below:

[0092] The four stages sequentially output the main branch scale features corresponding to the i-th scale. The spatial resolution decreases progressively while the number of channels increases progressively, for example, forming four main branch scale features of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, which are used for subsequent feature enhancement and pyramid detection; among them... This represents the height of the main branch scale feature at the i-th scale. Represents the width of the scale feature of the i-th main branch;

[0093] Specifically, the first stage, Stage 1, outputs the scale features of the first main branch. The second stage, Stage 2, outputs the scale features of the first main branch. Stage 3 outputs the scale features of the third main branch. Stage 4 outputs the scale features of the fourth main branch. The output of the fourth stage constitutes the main branch characteristics. ;

[0094] Down1, Down2, Down3, and Down4 all represent staged downsampling layers, used to reduce the sampling level and further deepen feature extraction;

[0095] The four-level hierarchical backbone, stage downsampling, and multi-scale output of the hierarchical visual backbone network are all mature existing technologies and are only used as a basic feature extraction framework, not as a separate innovation of this invention.

[0096] S22. Simultaneously, in the structural prior auxiliary branch, the network input tensor X0 is input into the SAM image encoder to extract the original structural prior sub-features at four scales. Constructing the prior features of the original structure ;

[0097] The structural prior auxiliary branch consists of a frozen or semi-frozen SAM image encoder, which extracts original structural prior sub-features at four different scales from the network input tensor X0 through the frozen or semi-frozen SAM image encoder. Thus, the prior features of the original structure are obtained. ;

[0098] The structural prior auxiliary branch only accepts image input;

[0099] The frozen or semi-frozen SAM image encoding is a borrowed pre-trained structure prior extractor, whose structure is usually fixed. It obtains four scale structure prior sub-features from different levels or through projection / resampling by a SAM auxiliary branch. The frozen means that the parameters do not participate in the training, and the semi-frozen means that only the adaptation layer, projection layer or a small number of back layers are trained. This is a mature existing technology and will not be described in detail here.

[0100] S23. Prior features of the original structure The original structural prior features at each scale are subjected to size transformation and channel mapping, and then to main branch features. Corresponding scale alignment yields prior structural features. .

[0101] The original structural prior features Original structural prior features at each scale Performing size transformation and channel mapping yields features related to the main branch. Structural prior features aligned to the corresponding scale The process is shown in the following formula:

[0102]

[0103] Here, Resize() represents a size transformation, used to transform the original prior features at the i-th scale from the original structural prior features. Adjust to the i-th main branch scale feature in the main branch features Same space size ; Indicates ordinary Convolution is used for channel mapping and channel fusion, ultimately yielding aligned structural prior features for each scale. , making the structural prior features With main branch scale features It has consistent spatial resolution and fusionable channel representation;

[0104] Finally, the structural prior features are obtained. ;

[0105] S3. Input the main branch features and structural prior features into the improved structure-guided Gaussian space-frequency enhancement block to obtain mixed-scale spatial modeling features, structure-guided Gaussian channel spatial enhancement features and frequency domain enhancement features. Then, perform residual aggregation at each scale to obtain the enhanced main branch features. The improved structure-guided Gaussian space-frequency enhancement block includes a mixed-scale spatial modeling branch, a structure-guided Gaussian channel spatial enhancement branch and a frequency domain enhancement branch.

[0106] The improved structure guides the Gaussian space-frequency enhancement block to borrow the structure of the space-frequency enhancement Mamba block (SFMB); the SFMB includes a three-way skeleton consisting of a mixed-scale Mamba block (MMB) for spatial perception in the spatial dimension, a channel enhancement block (CEB) for spatial perception in the channel dimension, and a frequency enhancement block (FEB) for frequency perception.

[0107] The improved structure-guided Gaussian spatial frequency enhancement block in this example includes a mixed-scale spatial modeling branch, a structure-guided Gaussian channel spatial enhancement branch, and a frequency domain enhancement branch; the mixed-scale spatial modeling branch is similar in structure to the mixed-scale Mamba block (MMB); the frequency domain enhancement branch is similar in structure to the frequency enhancement block (FEB); in this embodiment, the structure-guided Gaussian channel spatial enhancement branch is constructed to replace the channel enhancement block (CEB).

[0108] The structure-guided Gaussian channel spatial enhancement branch utilizes the SAM structure prior to construct edge and salient information, generates anisotropic spatial Gaussians with multi-peak positions and local structure tensors, and then couples them with channel Gaussians through learnable coefficients to replace the original CEB's GAP / GMP dual-path convolutional attention; this approach also differs from the original MGAM's single-peak modeling and direct merging of channel and spatial attention.

[0109] S31, Characteristics of Main Branches Main branch scale features at each scale Perform layer normalization to obtain normalized main branch scale features This leads to the normalized principal branch features. ;in:

[0110] ,

[0111] In the formula, LN represents Layer Normalization, which is used to normalize the scale features of each main branch feature. Normalization is performed to obtain the normalized principal branch scale features at the i-th scale. This is to stabilize subsequent spatial modeling, attention computation, and frequency domain processing;

[0112] S32. The three-layer mixed-scale space modeling constructed in the mixed-scale space modeling branch normalizes the main branch features. After multi-scale feature extraction and optimization recovery, the feature is fused with the gated signal generated by linear mapping and activation function to obtain mixed-scale spatial modeling features. ;

[0113] The hybrid-scale spatial modeling branch references the hybrid-scale Mamba block (MMB) of the spatial-frequency enhanced Mamba block.

[0114] Specifically, step S32 includes:

[0115] S321. For each normalized principal branch scale feature in the normalized principal branch features... The feature map is expanded along the channel dimension via linear mapping, then spatial features are extracted by a deep convolutional layer, and then multi-scale feature extraction is performed sequentially by a downsampling layer to obtain a multi-mixed-scale feature map for each scale; the multi-mixed-scale feature map includes :

[0116]

[0117]

[0118]

[0119] In the formula, This represents a 3×3 depthwise convolution; Indicates downsampling; SiLU activation function is represented by `SiLU()`; `Linear()` represents a linear mapping, which can be understood as a learnable projection onto the channel dimension, and is usually equivalent to this in feature map implementation. Convolutional or fully connected projection; where, This is the first mixed-scale sub-feature map at the i-th scale; This represents the second mixed-scale sub-feature map at the i-th scale; This is the third mixed-scale sub-feature map at the i-th scale;

[0120] S322. The multi-scale feature maps are optimized using a two-dimensional selective scanning state space module, and an upsampling layer is used to restore the size of the multi-scale feature maps to obtain multi-scale optimized feature maps; the multi-scale optimized feature maps include :

[0121]

[0122]

[0123]

[0124] In the formula, Indicates upsampling; This represents a two-dimensional selective scan state space module; Let SiLU represent the activation function, and LN represent the layer normalization; where, The first mixed-scale optimized sub-feature map for the i-th scale; The second hybrid scale optimized sub-feature map for the i-th scale; The third mixed-scale optimized sub-feature map is the i-th scale.

[0125] The 2D selective scanning state space module SS2D can be used as an existing 2D selective scanning state space operator. Its implementation steps can be supplemented as follows: expand the 2D features into several 1D sequences along the horizontal, vertical and reverse directions; perform selective state space recursion on each sequence to capture long-distance dependencies; and then restore the outputs of each direction to the 2D feature map and fuse them. In this scheme, it is used for long-range space modeling of the hybrid scale space modeling branch.

[0126] S323. Gated signals for normalized main branch features at each scale are generated using linear mapping and activation functions:

[0127]

[0128] In the formula, Linear() represents a linear mapping. This represents the SiLU activation function. This represents the i-th scale-gated signal;

[0129] S324. Optimize the first mixed-scale sub-feature map and the gated signal at each scale using element-wise multiplication. After fusion and linear mapping, the features are then skipped and connected with the normalized main branch features to obtain the mixed-scale spatial modeling features, as shown in the following equation:

[0130] ,

[0131] In the formula, Linear() represents a linear mapping, and ⊙ represents element-wise multiplication;

[0132] S33. Based on the characteristics of the normalized principal branch and structural prior features In the structure-guided Gaussian channel space enhancement feature branch, Gaussian channel weights and multi-peak Gaussian space weights are constructed and coupled to normalize the main branch features. Normalized principal branch scale features Enhancement is performed to obtain the spatial enhancement features of the structure-guided Gaussian channel. ;

[0133] Specifically, step S33 includes:

[0134] S331. Calculate the scale feature of each normalized main branch in the normalized main branch features using deep convolutional layers. Preprocessing features V i And after bisecting the channel, construct the channel description r. i ;

[0135] First, the normalized main branch scale feature in the normalized main branch feature is calculated using a 3×3 convolutional layer. Preprocessing features V i :

[0136] ,

[0137] in, This represents a 3×3 convolutional layer;

[0138] Then, the preprocessed features are bisected along the channel dimension to obtain the first sub-feature at the i-th scale. With the second sub-feature ;

[0139] Finally, based on the first sub-feature With the second sub-feature Construct a channel description, represented as:

[0140] ,

[0141] In the formula, GAP is global average pooling, which averages the channel description vectors along the spatial dimension; GMP is global max pooling, which takes the maximum value for each channel along the spatial dimension. Concat is a concatenation operation, which concatenates the channel description vectors obtained from GAP and GMP along the channel dimension.

[0142] S332, According to the channel description r i Determine the maximum response channel index and determine the channel Gaussian weight standard deviation based on the number of channels for each main branch scale feature;

[0143] The maximum response channel index As shown in the following formula:

[0144] ,

[0145] In the formula, argmax() represents the index function that retrieves the maximum value in the array;

[0146] The channel Gaussian weight standard deviation As shown in the following formula:

[0147] ,

[0148] In the formula, Represents the scale feature of the i-th main branch The number of channels;

[0149] S333. Using the maximum response channel index as the mean and the standard deviation of the channel Gaussian weights as the standard deviation, establish Gaussian channel weights. As shown in the following formula:

[0150] ,

[0151] In the formula, This represents the weight value of the Gaussian channel weight in the u-th channel at the i-th scale; ch is an abbreviation for channel, representing the channel domain; This represents the standard deviation of the Gaussian weights of the channels. This represents the index of the maximum response channel; exp() is the natural exponential function.

[0152] S334. Calculate the scale feature of each normalized main branch in the normalized main branch feature based on the Sobel edge operator. Edge enhancement map Then, combined with structural prior features Constructing the spatial saliency map of the edge enhancement map ;

[0153] The edge enhancement map is shown in the following formula:

[0154] ,

[0155] In the formula, c represents the channel dimension. Sobel() represents the Sobel edge operator, which calculates the horizontal and vertical gradients to obtain the edge enhancement map.

[0156] Constructing the spatial saliency map of the edge enhancement map The process is shown in the following formula:

[0157] ,

[0158] In the formula, This indicates taking the maximum value along the channel dimension, that is, taking the maximum response of all channels at each spatial location, which is used to form part of the spatial saliency map; This indicates averaging along the channel dimension. Represents an edge enhancement map. Representation and normalized principal branch scale features Aligned structural prior features;

[0159] S335, From the spatial saliency map Selected from Each significant peak is identified based on its neighborhood local structure tensor. Calculate the covariance matrix ;

[0160] From the spatial saliency map Selected from A significant peak is identified by using a neighborhood window of the significant peak. Calculate gradient Construct a structure tensor matrix and weight it to obtain the neighborhood local structure tensor of significant peaks;

[0161] Specifically, firstly, K is selected in the spatial saliency map. p We identify several significant peaks and select one of them, represented by coordinates (x, y). Then, we determine the neighborhood window based on this significant peak. Then, the first-order partial derivatives in the x and y directions of the image within the neighborhood window are calculated to determine the image gradient. Then, the structure tensor matrix of each pixel in the image within the window is constructed, and finally, the neighborhood weights are combined. Gaussian weighting of the structure tensor matrix yields the neighborhood local structure tensor. ;

[0162] ,

[0163] in, Neighborhood weights; The gradient in the x-direction of the image; The gradient in the y-direction of the image;

[0164] Then, based on the neighborhood local structure tensor of significant peaks... Calculate the covariance matrix As shown in the following formula:

[0165] ,

[0166] In the formula, Used for numerical stability is a positive Gaussian scaling factor used to control the spread range of the spatial Gaussian distribution; is a small positive constant used to improve the numerical stability of matrix inversion and avoid matrix singularities; for Identity matrix.

[0167] In this embodiment, the covariance matrix determines the direction, shape, and diffusion range of the spatial Gaussian;

[0168] S336. Establishing Multimodal Gaussian Spatial Weights Based on Covariance Matrix As shown in the following formula:

[0169] ,

[0170] In the formula, This represents the spatial weights of the multimodal Gaussian system at the i-th scale in spatial coordinates. The weight value at the location, which is determined by The summation of the Gaussian components corresponding to the significant peaks is used to reflect the relative importance of the current location belonging to the significant small target region. Indicates spatial saliency map The number of significant peaks selected; the superscript sp indicates spatial, i.e., the spatial domain; This represents the spatial coordinates of the m-th significant peak on the feature map. This represents the covariance matrix corresponding to the neighborhood of the m-th significant peak. The weight representing the m-th significant peak can be obtained from the peak response through normalization or softmax, and satisfies the following conditions: x, y are the i-th normalized principal branch scale features. The spatial coordinates on the graph correspond to the width and height directions, respectively; T indicates transpose. Representing spatial coordinates If a two-dimensional coordinate vector needs to be mapped back to the original coordinates, it can be multiplied by the step size corresponding to that scale; exp() is the natural exponential function.

[0171] The number of significant peaks The value of can be set as a hyperparameter. Preferably, in this embodiment, Choosing 5 or 8, in practice, one can first perform local maxima filtering, and then select the highest response. One peak; if the effective peak is insufficient Then, the actual peak quantity will be used;

[0172] Therefore, steps S331-S333 above are for calculating Gaussian channel weights, and steps S334-S336 are for calculating multi-peak Gaussian spatial weights. Both are used to calculate the subsequent structure-guided Gaussian channel spatial attention map.

[0173] S337. Couple the Gaussian channel weights with the multi-peak Gaussian spatial weights to obtain the structure-guided Gaussian channel spatial attention map A. i Then, based on the structure, the Gaussian channel spatial attention map A is used for guidance. i For each normalized principal branch scale feature in the normalized principal branch features Enhancement is performed to obtain the spatial enhancer feature of the structure-guided Gaussian channel. Ultimately, the spatial enhancement features of the structure-guided Gaussian channel were obtained. ;

[0174] The structure guides the spatial attention diagram of the Gaussian channel. (A) i As shown in the following formula:

[0175] ,

[0176] In the formula, , , These are the first, second, and third fusion coefficients of the channel-space attention coupling, which can be set as learnable parameters or initialized with equal weights. This represents element-wise multiplication and uses a broadcast mechanism to match the dimensions of channel weights and spatial weights. This represents the Sigmoid function. Indicates the Gaussian channel weights. Indicates the weights in a multimodal Gaussian space;

[0177] The structure guides the Gaussian channel spatial enhancer feature. As shown in the following formula:

[0178] ,

[0179] In the formula, Indicates element-wise multiplication; A i Spatial attention map of Gaussian channel for structure guidance. This represents the normalized main branch characteristics.

[0180] S34. Based on Normalized Main Branch Features In the frequency domain enhancement branch, a fast Fourier transform is performed to obtain a complex spectrum, which is then decomposed into an amplitude spectrum and a phase spectrum. After differential convolution coding, the spectrum is fused and reconstructed, and then restored to the spatial domain and the normalized main branch features by an inverse fast Fourier transform. Residual connections are performed on the normalized principal branch scale features to obtain frequency domain enhanced features. ;

[0181] Specifically, step S34 includes:

[0182] S341. Use Fast Fourier Transform to extract the normalized principal branch scale features from each normalized principal branch feature. Transforming from the spatial domain to the frequency domain yields the complex spectrum. As shown in the following formula:

[0183]

[0184] In the formula, FFT represents Fast Fourier Transform;

[0185] S342, The complex spectrum Decomposed into amplitude spectrum and phase spectrum For amplitude spectrum and phase spectrum Differential convolutional coding is performed to obtain the encoded amplitude spectrum. With the encoded phase spectrum ;

[0186] The amplitude spectrum is encoded using two 1×1 convolutional layers with activation functions to obtain the encoded amplitude spectrum. The phase spectrum is encoded using two 3×3 convolutional layers with activation functions to obtain the encoded phase spectrum. ;

[0187] ,

[0188] ,

[0189] in, Representing the spectrum of complex numbers amplitude, Representing the spectrum of complex numbers The phase angle; Represents the ReLU activation function; This represents a 1×1 convolutional layer. This represents a 3×3 convolutional layer; The differential convolutional encoding includes two paths: the amplitude spectrum passes through two layers of 1×1 convolution, with each convolution followed by a ReLU activation function for channel information combination and amplitude response adjustment; the phase spectrum passes through two layers of 3×3 convolution, with each convolution followed by a ReLU activation function for joint encoding of phase information at adjacent frequency positions; the two outputs reconstruct the complex spectrum, recover it to the spatial domain through inverse fast Fourier transform, and connect it with the original feature residual.

[0190] S343, Encode the amplitude spectrum With the encoded phase spectrum After fusion and reconstruction, the data is restored to the spatial domain via inverse fast Fourier transform and then compared with the normalized principal branch scale features. Perform residual connections to obtain frequency domain enhancer features at various scales. , is represented as:

[0191]

[0192] In the formula, IFFT stands for Inverse Fast Fourier Transform, used to restore the enhanced frequency domain representation to spatial domain features; freq is an abbreviation for frequency, representing the frequency domain; finally, the enhanced frequency domain features are obtained. ;

[0193] S35, Model the hybrid scale space features Structurally guided Gaussian channel spatial enhancement features and frequency domain enhancement features Performing residual aggregation at each scale yields enhanced main branch features. ;

[0194] Modeling features of the mixed-scale space Structurally guided Gaussian channel spatial enhancement features and frequency domain enhancement features Residual aggregation is performed at each scale to obtain the enhanced scale semantic sub-features for each scale. As shown in the following formula:

[0195] .

[0196] Ultimately, the enhanced main branch feature is obtained. It enhances the main branch features while possessing mixed-scale spatial representation, structural prior guidance capabilities, and frequency domain compensation capabilities.

[0197] Understandably, in this example, i is the index of the scale. That is, the enhanced scale semantic sub-feature of the i-th size, and so on. This means that the frequency domain enhancer feature of the i-th size is represented. Other variables also follow this rule, which will not be elaborated here. If i is the same, it means that the same size is represented.

[0198] S4. Perform same-scale gated deep fusion and calibration on the enhanced main branch features and structural prior features to obtain deep fused features;

[0199] Specifically, step S4 includes:

[0200] S41. Each structural prior sub-feature in the structural prior features is mapped to a projected structural prior sub-feature using a 1×1 convolutional layer. :

[0201]

[0202] in, This represents a 1×1 convolutional layer;

[0203] S42, the prior sub-features of the projection structure Augmented scale semantic sub-features at the same scale as the augmented main branch features The data is stitched together along the channel dimension and then fed into a convolutional layer to generate a scale fusion gating for each scale, as shown in the following equation:

[0204]

[0205] In the formula, This represents the scale fusion gating at the i-th scale; This indicates feature concatenation, specifically the enhanced main branch features. With the projected structural prior features The layers are stitched together along the channel dimension, and then a 3×3 convolutional layer is input to generate a fusion gate.

[0206] S43. Based on the scale fusion gating and enhanced scale semantic sub-features Calculate the weighted sub-features of the main branch at each scale. Simultaneously, based on scale fusion gating and projection structure prior sub-features Calculate the structural prior weighted sub-features at each scale :

[0207] ,

[0208] ,

[0209] In the formula, This represents element-wise multiplication; This represents the weighted sub-features of the main branch after gating and weighting; This represents the structural prior weighted sub-features after complementary gating weighting; This represents a structural prior sub-gating that is complementary to scale-fusion gating. This represents a 1×1 convolutional layer;

[0210] S44. Weight the sub-features of the main branch. Structural prior weighted sub-features Residual fusion is performed with the enhanced scale semantic sub-features to obtain the initial fused features for each scale. As shown in the following formula:

[0211] ;

[0212] S45, Based on the initial fusion features Convolutional calibration and residual preservation are performed to obtain the deep fusion sub-features at each scale, as shown in the following equation:

[0213] ,

[0214] In the formula, The initial fusion sub-features at the i-th scale; Represents the deep fusion sub-feature at the i-th scale; This represents a 3×3 convolutional layer;

[0215] The convolution calibration and residual preservation are used to suppress fusion spurious responses and enhance boundary continuity.

[0216] Ultimately, deep fusion features are obtained. ;

[0217] In this step, the features at each scale of the enhanced main branch features and structural prior features are subjected to same-scale gating fusion to obtain the main branch weighted sub-features and structural prior weighted sub-features. This allows the structural prior features extracted by the SAM image encoder to supplement boundary and geometric information without overly covering the semantic information of the main branch. This step inherits the ADF adaptive deep fusion idea of ​​MCD-Net and transforms it into a pure vision structural prior deep fusion process.

[0218] S5. After lateral mapping of each scale of the deep fusion features, the features are fused from top to bottom and then input into the decoupled detection head for prediction. The prediction results are then processed by candidate box decoding, threshold filtering and non-maximum suppression to obtain the aerial small target detection results in complex traffic scenarios.

[0219] Specifically, step S5 includes:

[0220] S51, The deep fusion feature Lateral mapping is performed at each scale, and the data is then fused from top to bottom to obtain the pyramid detection sub-features at each scale.

[0221] First, the number of channels in the deep fusion feature is compressed / aligned using 1×1 convolution through lateral mapping, so that the number of channels in the feature map of the previous layer is the same as that in the upsampled feature map of the next layer, resulting in the lateral mapping feature at the i-th scale, as shown in the following formula:

[0222]

[0223] In the formula, This represents the deep fusion sub-feature at the i-th scale. Represents the lateral mapping feature at the i-th scale; This represents a 1×1 convolutional layer;

[0224] Understandably, step S51 is the specific construction process of the high-resolution feature pyramid for neck detection: it first performs channel alignment on the deeply fused features at each scale through convolution while maintaining the original spatial resolution, then upsamples from top to bottom to pass high-level semantic information to the high-resolution layer, fuses it with lateral features of the same resolution, and finally suppresses upsampling aliasing through convolution and outputs the result. .

[0225] The lateral mapping of this high-resolution feature pyramid network preserves detailed information (edges, textures, and positions) without changing the spatial dimensions. It also uses lateral features of the same resolution to compensate for insufficient edge, texture, and position information during the top-down upsampling process, so that shallow features of the same resolution and high localization accuracy in the bottom-up path can be directly introduced into the fusion process in the subsequent top-down fusion operation.

[0226] Secondly, the lateral mapping features are fused from top to bottom to obtain pyramid detection sub-features at each scale. As shown in the following formula:

[0227]

[0228]

[0229] Among them, U p () represents nearest neighbor interpolation or bilinear interpolation; This represents a 3×3 convolutional layer; For the pyramid detection sub-features at the i-th scale;

[0230] Thus, the pyramid detection sub-features are obtained. And retain the first scale pyramid detection sub-features. It is used as a high-resolution layer to enhance the position-preserving capability of small targets;

[0231] It is the detection feature with the highest spatial resolution in the pyramid, and belongs to one of the multi-scale features of the input decoupled detection head in step S52. It is mainly used to preserve the details and positioning information of small targets.

[0232] In the formula, 3×3 convolution is used to eliminate aliasing; This indicates that top-down upsampling is performed on the deep fusion features so that the upsampled feature map has the same size as the feature map of the next layer, i.e., Q... i+1 Spatial resolution magnified to match lateral mapping features Therefore, high-level features are enhanced, which is beneficial for constructing multi-scale detection features for small targets, thus obtaining pyramid detection features. .

[0233] S52, Detect sub-features of the pyramid at each scale ( Input the classification head of the decoupling detection head respectively. , return to head and target head Perform predictions and output multi-scale prediction sub-results for each scale; among which, the first scale pyramid detection sub-features. It can be used as a high-resolution detection layer for small target prediction;

[0234] The multi-scale prediction results include: category, location, and targetability results, corresponding to the classification head output, regression head output, and targetability head output, respectively, as shown in the following formula:

[0235]

[0236]

[0237]

[0238] In the formula, This represents the output of the classification head at the i-th scale, used to predict the category; This represents the regression head output at the i-th scale, used to predict the candidate box location parameters; The target-oriented head output at the i-th scale is used to predict whether a target exists at the current location; these three components together constitute the multi-scale prediction sub-result of the decoupled detection head, thus yielding the multi-scale prediction result. ;

[0239] First-scale pyramid detection of sub-features To achieve the highest spatial resolution detection features, we fuse lateral features of the same resolution with high-level semantic features passed down from top to bottom, preserving the details and location information of small targets to enhance their detection. Other scale detection features provide supplementary predictions to adapt to the scale variations of aerial traffic targets.

[0240] S53. Based on the multi-scale prediction sub-results at each scale, candidate boxes are decoded to obtain all candidate detection boxes and their category confidence scores. Then, threshold filtering and non-maximum suppression are performed to obtain the aerial small target detection result D in complex traffic scenarios.

[0241] Specifically, step S53 includes:

[0242] According to the Each scale regression head output By combining the grid center and step size or prior box parameters at this scale, the predicted location parameters are decoded into candidate detection boxes in image coordinates. ;

[0243] The offset is determined directly from the pyramid detection sub-features by the trainable regression head. The predictions are derived, not predetermined manually. Specifically, the features at each grid location are processed by the convolutional and output layers of the regression head to form four-dimensional location parameters. The scheme employs end-to-end learning through the localization regression loss corresponding to the labeled bounding boxes. The preferred approach is to use an anchorless bounding box, where the four-dimensional parameters are interpreted as the distances from the grid center to the left, top, right, and bottom edges of the candidate bounding box. Multiplying these distances by the scale step size allows the bounding boxes to be decoded into candidate bounding boxes in the image coordinates.

[0244] Then, based on the classification head output in the multi-scale prediction results, and target head output Calculate the class confidence score of candidate detection box B. As shown in the following formula:

[0245] ,

[0246] In the formula, Softmax represents the normalization exponential function, used to normalize the output of the classification head. Convert the results into probabilities for each category to determine the category of the outcome. Used to calculate the target score;

[0247] Therefore, the category confidence score can be obtained by combining the category probability and the targetability score;

[0248] Finally, threshold filtering and non-maximum suppression are applied to all candidate detection boxes at all scales to obtain the small target detection result D in the complex traffic scene, denoted as:

[0249] ,

[0250] In the formula, For the q-th target box, For the target category, Score for each category. To output the target quantity;

[0251] Candidate boxes are obtained by decoding the regression head output combined with grid centers and scale steps; target category The category is determined by the maximum confidence score among all categories of the candidate box. The category score is the confidence score corresponding to that category, i.e. ; This represents the number of detection boxes that are ultimately retained after confidence threshold filtering and non-maximum suppression by category.

[0252] Finally, the multi-scale prediction results of all scales are summarized. , and It always inherits from the same pre-selection candidate and outputs in the form of a one-to-one corresponding triplet, finally obtaining the aerial small target detection result D in complex traffic scenarios;

[0253] The preset threshold Intersection over Union (IOU) threshold Select according to actual needs; preferably, in this embodiment, a preset threshold is used. and preset crossover ratio threshold All are available at The threshold value is adjusted within the range based on detection accuracy, recall rate, and target density; preferably, in this embodiment, a preset threshold value is used. Set to 0.25; preset intersection-union threshold. Take 0.45.

[0254] The Intersection over Union (IoU) ratio is an indicator that measures the degree of overlap between two rectangles, defined as the area of ​​the intersection divided by the area of ​​the union.

[0255] This embodiment provides a method for small target detection in aerial traffic images based on structure prior and spatial frequency Gaussian collaboration. First, the drone-captured traffic image is input into a dual-branch detection network that requires no text prompts. The main branch extracts multi-scale semantic information, while the retained SAM auxiliary branch provides more stable edge, contour, and spatial structure prior features, thereby improving the stability of small target detection. Subsequently, a structure-guided Gaussian spatial frequency enhancement module, modified from the spatial frequency hybrid enhancement block SFMB, is introduced into the main branch. On the one hand, it utilizes mixed-scale modeling and frequency domain enhancement to supplement the detailed representation of small targets under different scales and texture conditions; on the other hand, it integrates Gaussian channel weights with multi-peak Gaussian... Spatial weights are coupled to obtain a structure-guided Gaussian channel spatial attention map. Combined with prior structural information in traffic scenarios, this can more effectively highlight the areas where small targets such as vehicles and pedestrians are located, while suppressing interference from road textures, shadows, and surrounding noise. Then, the enhanced main branch features and the structural prior features provided by the SAM branch are subjected to same-scale adaptive deep fusion, so that semantic and structural information complement and synergize. Finally, the location and category results of small targets are output through the detection neck of the high-resolution feature pyramid network and the decoupled detection head. This achieves stable detection of small targets in UAV aerial photography in complex traffic scenarios, solving the technical problems of "small targets being unclear and their outlines being unstable".

[0256] This embodiment primarily addresses the issues of feature attenuation, background interference, and unstable positioning for small targets in aerial traffic imaging. The SAM auxiliary branch provides structural priors, the frequency domain branch supplements spectral representation, and Gaussian weights highlight salient target regions; each branch complements the others. Computationally, the four-scale structural priors from the same input are extracted and reused by a single SAM image encoder (freezing SAM parameters), eliminating the need for re-encoding at each scale and subsequent fusion. Frequency domain enhancement acts on the multi-scale feature maps of the backbone output. Gaussian spatial weights are calculated using a finite number of salient peaks and a 2×2 local matrix, avoiding the construction of a large relational matrix between all spatial locations. The three components operate on different objects and at different levels (structural priors, spectral representation, and spatial attention), complementing each other rather than redundantly stacking, forming a holistic solution of "one-time encoding, multiplexing, and lightweight fusion." While improving the accuracy of small target detection, the computational overhead is kept within an acceptable range through structural reuse and computational constraint design, effectively solving the problem of balancing high-precision models with the real-time requirements of embedded platforms.

[0257] Furthermore, this embodiment innovatively improves the CEB in SFMB by integrating it with Gaussian channel weights and multi-peak Gaussian spatial weights, and by combining it with structural priors in traffic scenarios, enabling the network to focus more intently on dense, small-scale target areas that are susceptible to background interference.

[0258] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 A process, multiple processes, and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0259] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0260] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0261] The parts of this invention not described in detail are prior art. It will be apparent to those skilled in the art that this invention is not limited to the details of the above exemplary embodiments, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and are intended to encompass all changes falling within the meaning and scope of equivalents within this invention.

Claims

1. A method for detecting small traffic targets in aerial photography based on structural priors and spatial frequency Gaussian coordination, characterized in that, Includes the following steps: S1. After preprocessing the original UAV aerial images, construct the network input tensors in batches. S2. The network input tensor is simultaneously input into the main semantic detection branch and the structural prior auxiliary branch for multi-scale feature extraction, and then the feature scale is aligned to obtain the main branch features and structural prior features. S3. Input the main branch features and structural prior features into the improved structure-guided Gaussian space-frequency enhancement block to obtain mixed-scale spatial modeling features, structure-guided Gaussian channel spatial enhancement features and frequency domain enhancement features. Then, perform residual aggregation at each scale to obtain the enhanced main branch features. The improved structure-guided Gaussian space-frequency enhancement block includes a mixed-scale spatial modeling branch, a structure-guided Gaussian channel spatial enhancement branch and a frequency domain enhancement branch. S4. Perform same-scale gated deep fusion and calibration on the enhanced main branch features and structural prior features to obtain deep fused features; S5. After lateral mapping of each scale of the deep fusion features, the features are fused from top to bottom and then input into the decoupled detection head for prediction. The prediction results are then processed by candidate box decoding, threshold filtering and non-maximum suppression to obtain the aerial small target detection results in complex traffic scenarios.

2. The method according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Uniform Size Resampling and Channel Normalization: Resample the original UAV aerial image I to a fixed size and perform RGB per-channel normalization to obtain a standardized image. ; S12, Overlapping Cut: Uses preset window size Sliding window method with preset step size for standardized images Overlapping and slicing results in a set of slices. ;in, To cut into pieces; This refers to the number of pieces; This is the height of the cut piece; Where n is the slice width and n is the slice index; S13. Network Input Tensor Generation: Transforming the Segment Set The network input tensors are obtained by organizing and stacking them in batches while maintaining the channel order and scale information. ;in, This represents the batch size; 3 indicates RGB three-channel.

3. The method according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21. In the main semantic detection branch, the network input tensor X0 is input into the hierarchical visual backbone network, and the main branch scale features of each scale are extracted according to the four-level stage. To obtain the main branch features The hierarchical visual backbone network includes an input embedding layer, four stages, inter-stage downsampling layers, a normalization layer, and a multi-scale output layer; the four stages are cascaded. S22. Simultaneously, in the structural prior auxiliary branch, the network input tensor X0 is input into the SAM image encoder to extract the original structural prior sub-features at four scales. Constructing the prior features of the original structure ; S23. Prior features of the original structure The original structural prior features at each scale are subjected to size transformation and channel mapping, and then to main branch features. Corresponding scale alignment yields prior structural features. .

4. The method according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31, Characteristics of Main Branches Main branch scale features at each scale Perform layer normalization to obtain normalized main branch scale features This leads to the normalized principal branch features. ; S32. The three-layer mixed-scale space modeling constructed in the mixed-scale space modeling branch normalizes the main branch features. After multi-scale feature extraction and optimization recovery, the feature is fused with the gated signal generated by linear mapping and activation function to obtain mixed-scale spatial modeling features. ; S33. Based on the characteristics of the normalized principal branch and structural prior features In the structure-guided Gaussian channel space enhancement feature branch, Gaussian channel weights and multi-peak Gaussian space weights are constructed and coupled to normalize the main branch features. Normalized principal branch scale features Enhancement is performed to obtain the spatial enhancement features of the structure-guided Gaussian channel. ; Guide the Gaussian channel spatial enhancement sub-feature for the structure at the i-th scale; S34. Based on Normalized Main Branch Features In the frequency domain enhancement branch, a fast Fourier transform is performed to obtain a complex spectrum, which is then decomposed into an amplitude spectrum and a phase spectrum. After differential convolution coding, the spectrum is fused and reconstructed, and then restored to the spatial domain and the normalized main branch features by an inverse fast Fourier transform. Residual connections are performed on the normalized principal branch scale features to obtain frequency domain enhanced features. ; For the frequency domain enhancer feature at the i-th scale; S35, Model the hybrid scale space features Structurally guided Gaussian channel spatial enhancement features and frequency domain enhancement features Performing residual aggregation at each scale yields enhanced main branch features. ; Let i be the enhanced scale semantic sub-feature of the i-th scale.

5. The method according to claim 1, characterized in that, Step S4 specifically includes the following steps: S41. Each structural prior sub-feature in the structural prior features is mapped to a projected structural prior sub-feature using a 1×1 convolutional layer. : S42, the prior sub-features of the projection structure Augmented scale semantic sub-features at the same scale as the augmented main branch features The data is stitched together along the channel dimension and then fed into a convolutional layer to generate scale fusion gating for each scale. S43. Based on the scale fusion gating and enhanced scale semantic sub-features Calculate the weighted sub-features of the main branch at each scale. Simultaneously, based on scale fusion gating and projection structure prior sub-features Calculate the structural prior weighted sub-features at each scale : ， ， In the formula, This represents element-wise multiplication; This represents the weighted sub-features of the main branch after gating and weighting; This represents the structural prior weighted sub-features after complementary gating weighting; This represents a structural prior sub-gating that is complementary to scale-fusion gating. This represents a 1×1 convolutional layer; This represents the scale fusion gating at the i-th scale; S44. Weight the sub-features of the main branch. Structural prior weighted sub-features Residual fusion is performed with the enhanced scale semantic sub-features to obtain the initial fused features for each scale. ; S45, Based on the initial fusion features Convolutional calibration and residual preservation are performed to obtain the deep fusion sub-features at each scale, as shown in the following equation: ， In the formula, The initial fusion sub-features at the i-th scale; Represents the deep fusion sub-feature at the i-th scale; This represents a 3×3 convolutional layer.

6. The method according to claim 1, characterized in that, Step S5 specifically includes the following steps: S51, The deep fusion feature Lateral mapping is performed at each scale, and the data is then fused from top to bottom to obtain the pyramid detection sub-features at each scale. S52. Input the pyramid detection sub-features of each scale into the classification head, regression head and target head of the decoupled detection head for prediction, and output the multi-scale prediction sub-results of each scale; wherein, the pyramid detection sub-features of the first scale are used as a high-resolution detection layer for small target prediction. S53. Based on the multi-scale prediction sub-results at each scale, candidate boxes are decoded to obtain all candidate detection boxes and their category confidence scores. Then, threshold filtering and non-maximum suppression are performed to obtain the aerial small target detection results in complex traffic scenarios.

7. The method according to claim 4, characterized in that, Step S33 specifically includes the following steps: S331. Calculate the scale feature of each normalized main branch in the normalized main branch features using deep convolutional layers. Preprocessing features Then, it is divided into the first sub-feature of the i-th scale along the channel. With the second sub-feature Constructing channel description Where GAP represents global average pooling; GMP represents global max pooling; and Concat represents concatenation operation. S332, According to the channel description r i Determine the maximum response channel index The standard deviation of the Gaussian weights for each channel is determined based on the number of channels for each main branch scale feature. ; where argmax() represents the index function that retrieves the maximum value in the array; Represents the scale feature of the i-th main branch The number of channels; S333. Using the maximum response channel index as the mean and the standard deviation of the channel Gaussian weights as the standard deviation, establish Gaussian channel weights. As shown in the following formula: ， In the formula, represents the Gaussian channel weight value in the u-th channel at the i-th scale; ch represents the channel domain; exp() is the natural exponential function; S334. Calculate the scale feature of each normalized main branch in the normalized main branch feature based on the Sobel edge operator. Edge enhancement map Then, combined with structural prior features Constructing the spatial saliency map of the edge enhancement map : ， In the formula, This indicates that the maximum value is taken along the channel dimension; This indicates averaging along the channel dimension; Representation and normalized principal branch scale features Aligned structural prior features; S335, From the spatial saliency map Selected from Each significant peak is identified based on its neighborhood local structure tensor. Calculate the covariance matrix ;in, A positive Gaussian scaling adjustment coefficient; It is a relatively small positive number; It is the identity matrix; S336. Establishing Multimodal Gaussian Spatial Weights Based on Covariance Matrix As shown in the following formula: ， in, This represents the spatial weights of the multimodal Gaussian system at the i-th scale in spatial coordinates. The weight value at the location, where the superscript sp indicates the spatial domain; This represents the spatial coordinates of the m-th significant peak on the feature map; The weight of the m-th significant peak is represented by ; T represents the transpose. Representing spatial coordinates Two-dimensional coordinate vector; S337. Couple the Gaussian channel weights with the multi-peak Gaussian spatial weights to obtain the structure-guided Gaussian channel spatial attention map A. i Then, based on the structure, the Gaussian channel spatial attention map A is used for guidance. i For each normalized principal branch scale feature in the normalized principal branch features Enhancement is performed to obtain the spatial enhancer feature of the structure-guided Gaussian channel. Ultimately, the spatial enhancement features of the structure-guided Gaussian channel were obtained. ; The structure guides the spatial attention diagram of the Gaussian channel. (A) i As shown in the following formula: ， In the formula, , , These are the first, second, and third fusion coefficients of the channel-space attention coupling, respectively. This represents element-wise multiplication; This represents the Sigmoid function.

8. The method according to claim 7, characterized in that, The step S335 described in the spatial saliency map Selected from Each significant peak is identified based on its neighborhood local structure tensor. Specifically, it includes: Firstly, in the spatial saliency map Select K from p We identify several significant peaks and select one of them, represented by coordinates (x, y). Then, we determine the neighborhood window based on this significant peak. Then, the first-order partial derivatives in the x and y directions of the image within the neighborhood window are calculated, the image gradient is calculated, and the structure tensor matrix of each pixel in the image within the window is constructed. Finally, the neighborhood weights are combined. Gaussian weighting of the structure tensor matrix yields the neighborhood local structure tensor. : ， in, The gradient in the x-direction of the image; The gradient in the y-direction of the image.

9. The method according to claim 4, characterized in that, Step S32 specifically includes the following steps: S321. For each normalized principal branch scale feature in the normalized principal branch features... The feature map is extended along the channel dimension by linear mapping, then spatial features are extracted by deep convolutional layers, and then multi-scale feature extraction is performed by downsampling layers to obtain multi-mixed scale feature maps for each scale. S322. The multi-scale feature map is optimized by using a two-dimensional selective scanning state space module, and the size of the multi-scale feature map is restored by using an upsampling layer to obtain the multi-scale optimized feature map. S323. Gated signals for normalized main branch features at each scale are generated using linear mapping and activation functions: S324. Optimize the first mixed-scale sub-feature map and the gated signal at each scale using element-wise multiplication. After fusion and linear mapping, the features are then connected to the normalized main branch features via a jump connection to obtain mixed-scale spatial modeling features.

10. The method according to claim 4, characterized in that, Step S34 specifically includes the following steps: S341. Use Fast Fourier Transform to extract the normalized principal branch scale features from each normalized principal branch feature. Transforming from the spatial domain to the frequency domain yields the complex spectrum. ; S342, The complex spectrum Decomposed into amplitude spectrum and phase spectrum For amplitude spectrum and phase spectrum Differential convolutional coding is performed to obtain the encoded amplitude spectrum. With the encoded phase spectrum ;in, Representing the spectrum of complex numbers amplitude, Representing the spectrum of complex numbers The phase angle; S343, Encode the amplitude spectrum With the encoded phase spectrum After fusion and reconstruction, the data is restored to the spatial domain via inverse fast Fourier transform and then compared with the normalized principal branch scale features. Perform residual connections to obtain frequency domain enhancer features at various scales. Finally, the frequency domain enhancement features are obtained. .

Images