Visual unmanned aerial vehicle detection method and device, computer equipment and storage medium

Abstract

The invention relates to a visual unmanned aerial vehicle detection method and device, computer equipment and a storage medium. The method comprises the steps of determining a target prediction frame corresponding to an unmanned aerial vehicle in a to-be-detected image based on a target detection model; the target detection model comprises multiple layers of decoders; the method comprises the following steps: performing spatial channel collaborative enhancement processing on an input feature corresponding to a to-be-detected image based on a first-layer decoder to obtain an enhanced feature, and determining an initial prediction frame of the to-be-detected image and an initial probability distribution set corresponding to an initial edge distance set according to the enhanced feature; optimizing the initial probability distribution set layer by layer based on a multi-layer decoder to obtain a target probability distribution set; determining an optimized side distance set according to the target probability distribution set, the weighting function and the initial side distance set, and determining a target prediction frame of the unmanned aerial vehicle based on the optimized side distance set; the weighting function represents a weighted value of each initial edge distance in the initial edge distance set at each candidate offset position. By adopting the method, the positioning precision of target detection can be improved.

Description

Visual UAV detection methods, devices, computer equipment, and storage media Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a visual unmanned aerial vehicle (UAV) detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Deep learning-based visual target detection refers to a technology that identifies and locates key targets from aerial images, playing a crucial role in fields such as military reconnaissance, disaster monitoring, and logistics delivery. Deep learning-based UAV small target detection algorithms are cost-effective and adaptable to complex and ever-changing aerial environments. In particular, Transformer-based detectors (such as DETR), due to their global modeling capabilities, are suitable for detecting small UAV targets with variable flight attitudes and significant visual feature differences. However, the core challenge of deep learning-based UAV small target detection algorithms lies in achieving accurate localization of tiny targets in complex environments. Compared to traditional target detection tasks, UAV targets exhibit significant unique characteristics such as small size and diverse shapes. These unique characteristics present UAV target detection with more severe technical challenges than general target detection, particularly demonstrating a significant performance bottleneck in accurate bounding box localization.

[0003] In traditional techniques, target detection models treat the target bounding box coordinates as deterministic values ​​and predict them directly. Essentially, this simplifies the complex spatial localization problem into Dirac distribution modeling. This deterministic modeling lacks effective integration and utilization of feature information at different levels of the network, which limits the localization accuracy of target detection. Summary of the Invention

[0004] Therefore, it is necessary to provide a visual UAV detection method, device, computer equipment, and storage medium that can improve the positioning accuracy of target detection, addressing the aforementioned technical problems.

[0005] In a first aspect, this application provides a visual unmanned aerial vehicle (UAV) detection method, including:

[0006] The target detection model determines the target prediction box corresponding to the UAV target in the image to be tested based on the pre-trained target detection model; the target detection model includes a multi-layer decoder.

[0007] The process of determining the target prediction box corresponding to the UAV target based on the target detection model includes:

[0008] The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ;

[0009] Based on the initial probability distribution set of the multi-layer decoder By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder;

[0010] Based on the target probability distribution set Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0011] In one embodiment, the weighting function The calculation formula is In the formula, and These are the hyperparameters that control the upper bound and curvature, respectively.

[0012] In one embodiment, enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image to be tested based on the first-layer decoder. ,include:

[0013] The image to be tested is input into the backbone network of the object detection model for processing to obtain intermediate features. These intermediate features are then input into the encoder layer of the object detection model to obtain the input features corresponding to the image to be tested. ;

[0014] Based on the spatial channel collaborative enhancement layer in the first-layer decoder, the input features are enhanced. The input features are obtained by decomposing along the height and width dimensions and then performing global average pooling. Corresponding height dimension features and width dimension features ;

[0015] High-dimensional features and width dimension features Divided into Each set of height and width sub-features of equal size is given. Each height and width sub-feature is then subjected to depthwise separable convolution. Lightweight convolution is then used to align the resulting height and width sub-features, yielding multiple aligned height and width sub-features. The aligned height sub-features... The calculation formula is Aligned width dimension sub-features The calculation formula is , For application to the The size of the convolution kernel for each sub-feature The number of channels for each height dimension sub-feature and each width dimension sub-feature;

[0016] Multiple aligned height-dimensional sub-features and multiple aligned width-dimensional sub-features are concatenated and grouped for normalization to generate height-dimensional attention weights. Attention weights in width dimension ;in, , , This represents the Sigmoid activation function. Indicates the process along the height dimension Group normalization operation, Indicates operation along the width dimension Group normalization operation;

[0017] High-dimensional attention weights Attention weights in width dimension Input features Element-wise multiplication yields spatial attention enhancement features. .

[0018] In one embodiment, the method further includes:

[0019] Spatial attention enhancement features Spatial compression is performed to obtain compressed features. , This indicates spatial attention enhancement features. Spatial resolution from Compress to ;

[0020] Compressed features Perform linear mapping to generate a query matrix Q, a key matrix K, and a value matrix V, where, , , A constant representing the number of input feature channels;

[0021] The self-attention features are determined based on the query matrix Q, the key matrix K, and the value matrix V. ;

[0022] Features of self-attention Global pooling is performed, and channel attention weights are generated using a sigmoid activation function. These channel attention weights are then applied to the spatial attention enhancement feature. Enhanced features are obtained .

[0023] In one embodiment, based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ,include:

[0024] Based on enhanced features The initial prediction box of the image to be tested is determined as follows: ,in, Indicates the center coordinates of the initial prediction box. This indicates the width and height of the initial prediction box;

[0025] Initial prediction box Convert to center coordinates and the initial edge distance set ;in, The distance from the center coordinates of the initial predicted bounding box to the first side of the top edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the second side of the bottom edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the third side of the left edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the fourth side of the right edge box;

[0026] Based on the initial probability distributions corresponding to the distances of each side, the set of initial probability distributions is determined as follows: , These are the initial probability distributions corresponding to the distances of the first, second, third, and fourth sides, respectively. The probability distributions represent the probability of each side distance at one of the N candidate offset positions.

[0027] In one embodiment, the pre-training process of the object detection model includes:

[0028] The training images are processed based on the pre-acquired initial detection model to obtain the initial training prediction boxes and target training prediction boxes corresponding to the training images; the initial detection model includes a multi-layer decoder.

[0029] Based on the width and height of the ground truth bounding boxes corresponding to the initial training prediction boxes, and the width and height of the target training prediction boxes, the size difference term of the target training prediction boxes is determined as follows: , and For the width and height of the actual bounding box, and Train the width and height of the predicted bounding box for the target. These are the normalized weight parameters;

[0030] Based on the width and height of the ground truth bounding box, determine the first shape weight coefficient of the predicted bounding box. Second shape weighting coefficient Among them, the first shape weighting coefficient Second shape weighting coefficient , Scale factor;

[0031] Based on the first shape weight coefficient Second shape weighting coefficient The shape-aware distance of the target training prediction box is determined by the center point coordinates of the target training prediction box and the ground truth box. , Train the center point coordinates of the predicted bounding box for the target. The coordinates of the center point of the true bounding box;

[0032] Shape-perceived distance using an exponential function Normalization is performed to obtain auxiliary measurement parameters. And based on auxiliary metric parameters The loss function is determined as follows ;in, A constant representing the number of input feature channels. Train the intersection-union ratio (IU) of the predicted bounding boxes and the ground truth bounding boxes for the target. Train a shape penalty term for the predicted bounding box to the target. , For predefined shape parameters, These are predefined angle parameters;

[0033] Based on the loss function, the initial detection model is optimized to obtain the target detection model.

[0034] Secondly, this application also provides a visual unmanned aerial vehicle (UAV) detection device, comprising:

[0035] The detection module is used to determine the target prediction box corresponding to the UAV target in the image to be tested based on the pre-trained target detection model; the target detection model includes a multi-layer decoder;

[0036] The process of determining the target prediction box corresponding to the UAV target based on the target detection model includes:

[0037] The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ;

[0038] Based on the initial probability distribution set of the multi-layer decoder By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder;

[0039] Based on the target probability distribution set Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0040] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0041] The target detection model determines the target prediction box corresponding to the UAV target in the image to be tested based on the pre-trained target detection model; the target detection model includes a multi-layer decoder.

[0042] The process of determining the target prediction box corresponding to the UAV target based on the target detection model includes:

[0043] The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ;

[0044] Based on the initial probability distribution set of the multi-layer decoder By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder;

[0045] Based on the target probability distribution set Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0046] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0047] The target detection model determines the target prediction box corresponding to the UAV target in the image to be tested based on the pre-trained target detection model; the target detection model includes a multi-layer decoder.

[0048] The process of determining the target prediction box corresponding to the UAV target based on the target detection model includes:

[0049] The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ;

[0050] Based on the initial probability distribution set of the multi-layer decoder By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder;

[0051] Based on the target probability distribution set Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0052] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0053] The target detection model determines the target prediction box corresponding to the UAV target in the image to be tested based on the pre-trained target detection model; the target detection model includes a multi-layer decoder.

[0054] The process of determining the target prediction box corresponding to the UAV target based on the target detection model includes:

[0055] The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ;

[0056] Based on the initial probability distribution set of the multi-layer decoder By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder;

[0057] Based on the target probability distribution set Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0058] In this invention, spatial channel collaborative enhancement processing is first applied to the input features corresponding to the training images. This optimizes the spatial semantic information and channel feature representation of the input features, enhancing their representational power and providing rich semantic prior knowledge for the detection model, thereby improving the accuracy of target detection. Secondly, addressing the problem in existing methods where the Dirac distribution is used to model the position of the target prediction box, treating it as a fixed value, significantly increases positional deviation when detecting small targets, leading to missed detections of key targets, this invention transforms the bounding box localization problem of the target to be detected into a probability distribution problem. By modeling the probability density distribution of the bounding box coordinates of the target to be detected, the localization uncertainty is represented. A residual-based layer-by-layer optimization mechanism is used for iterative optimization at different levels of the decoder, effectively alleviating the uncertainty problem in small target localization and improving the localization accuracy of target detection. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 is a flowchart illustrating the process of determining a target prediction box based on a target detection model in one embodiment;

[0061] Figure 2 is a schematic diagram of the visual drone detection network process in one embodiment;

[0062] Figure 3 is a flowchart of the spatial channel collaborative enhancement processing layer in one embodiment;

[0063] Figure 4 is a schematic diagram of the fine-grained distribution optimization mechanism in one embodiment;

[0064] Figure 5 is a qualitative comparative analysis of visual UAV detection methods in one embodiment;

[0065] Figure 6 is a structural block diagram of a visual unmanned aerial vehicle (UAV) detection device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] Unmanned aerial vehicles (UAVs) possess significant unique characteristics, such as small size and diverse shapes. These characteristics present UAV target detection with more severe technical challenges than general target detection, particularly exhibiting a clear performance bottleneck in accurate bounding box localization. Traditional methods optimize target detection models by improving feature extraction modules, designing new attention mechanisms, and refining loss functions. However, these methods often overlook the shortcomings of existing detectors in bounding box regression. Existing detectors treat target bounding box coordinates as deterministic values ​​and directly predict them, essentially simplifying the complex spatial localization problem to Dirac distribution modeling. This deterministic modeling has two key problems: first, it lacks effective integration and utilization of feature information from different levels of the network; second, small targets are easily affected by environmental factors such as darkness and blur during localization, resulting in uncertainty in their true bounding box positions. This rigid modeling approach based on the Dirac distribution cannot describe the localization uncertainty caused by blur or occlusion, leading to a significant increase in the detection position deviation of small targets, resulting in missed detection of key targets and reducing the localization accuracy of target detection.

[0068] Furthermore, the multi-semantic characteristics of UAV targets themselves pose additional challenges to their accurate localization. The feature representations of UAV targets are often scattered across different spatial regions and channel dimensions. Different spatial locations may correspond to different parts of the target (such as the fuselage, propellers, landing gear, etc.), while different channels may capture different visual features. Effective integration of this multi-semantic information is crucial for improving the quality of feature representations, which in turn directly affects the accuracy of subsequent bounding box regression. However, existing RT-DETR methods have significant shortcomings in feature extraction and attention mechanisms when dealing with these multi-semantic characteristics: on the one hand, traditional attention methods often employ global pooling or simple spatial channel separation strategies, failing to fully explore and utilize the semantic differences and similarities between different spatial regions; on the other hand, the lack of in-depth modeling of the interaction relationships between multi-semantic features prevents the effective utilization of the synergistic effects between different semantic information, limiting the effectiveness and discriminative power of feature fusion. This leads to a decline in the target detection model's ability to perceive UAV features, severely impacting the detection accuracy of complex UAV targets.

[0069] In an exemplary embodiment, a visual unmanned aerial vehicle (UAV) detection method is provided. The method includes determining a target prediction box corresponding to a UAV target in a test image based on a pre-trained target detection model. The target detection model includes a multi-layer decoder. As shown in Figure 1, the process of determining the target prediction box based on the target detection model includes steps 102 to 106.

[0070] Step 102: Based on the input features corresponding to the image under test, perform spatial channel collaborative enhancement processing to obtain enhanced features. Determine the initial prediction box of the image under test and the initial edge distance set corresponding to the initial prediction box based on the enhanced features, and determine the initial probability distribution set corresponding to the initial edge distance set.

[0071] The initial detection model is used during the training phase to perform preliminary target prediction on the input training images, generating an initial probability distribution set of the initial prediction boxes and their corresponding initial edge distance sets as optimization benchmarks. The initial detection model may include, but is not limited to, a backbone network, encoder layer, and decoder. Training images refer to sample images used to train the initial detection model, and are images including targets (such as drones). The input features corresponding to the training images are obtained by processing the training images through the backbone network and encoder in the initial detection model. Spatial channel collaborative enhancement processing enhances the feature response of small target regions through the synergistic effect of Shareable Multi-Semantic Spatial Attention (SMSA) and Progressive Channel-wise Self-Attention (PCSA), further strengthening the multi-semantic features of the target to be detected and providing rich semantic prior knowledge for the detection model. The initial prediction box is the bounding box prediction result of the target to be detected. The initial edge distance set corresponding to the initial prediction box includes the distance values ​​from the center coordinates of the initial prediction box to the top, bottom, left and right edge boxes of the initial prediction box. The initial probability distribution set includes the probability distribution of the candidate offset positions of the edges from the center coordinates of the initial prediction box to the top, bottom, left and right edge boxes of the initial prediction box.

[0072] The enhanced features are obtained by processing the input features based on the first-layer decoder in the initial detection model. Based on the enhanced features An initial predicted bounding box for the target object in the image to be tested is determined; this initial predicted bounding box serves as a reference for subsequent optimization and is converted into an initial edge distance set. Meanwhile, based on the initial edge distance set The distance to each side in the matrix determines the probability distribution of its corresponding candidate offset position, thus forming an initial set of probability distributions. , is used to characterize the uncertainty of the position of each side corresponding to the initial prediction box.

[0073] Step 104: Optimize the initial probability distribution set layer by layer based on the multi-layer decoder to obtain the target probability distribution set.

[0074] The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder.

[0075] , Characterization in the The layer decoder has four independent probability distributions, each corresponding to the probability of an edge of the initial prediction box at one of the N candidate offset positions.

[0076] In the process of implementing layer-by-layer optimization, the probability distribution is updated using the residuals expressed in the following expression:

[0077]

[0078] In the formula, Indicates the first The confidence level of each edge distance of the layer decoder at each candidate offset position. Indicates the first The prediction residual of the layer decoder Indicates the first The confidence level of each edge distance of the layer decoder at each candidate offset position. (The remaining text appears to be incomplete and requires further context.) Confidence of edge distances of each layer in the layer decoder at each candidate offset position Layer and First prediction residuals of layer decoders The summation and softmax processing yield a new probability distribution. This update mechanism allows each layer to make incremental improvements based on the previous layer.

[0079] Through the aforementioned iterative mechanism of layer-by-layer optimization, each layer of decoder can make incremental improvements based on the results of the previous layer of decoder, and gradually improve the bounding box localization accuracy of the target to be detected through multiple iterations.

[0080] Step 106: Determine the optimized edge distance set based on the target probability distribution set, weight function, and initial edge distance set, and determine the target prediction box corresponding to the UAV target based on the optimized edge distance set.

[0081] Among them, the optimized set of edge distances is Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0082] For example, the weight function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted values ​​of each candidate offset position are obtained by using a weight function. Set of standard probability distributions The edge offset is obtained by performing a weighted summation. Further, it is calculated according to the size of the initial predicted bounding box. Scale the initial predicted bounding box and then compare the scaled size with the initial edge distance set. The summation process yields the optimized set of edge distances. This probabilistic representation allows the model detection to no longer output a single, definitive value, but rather express the confidence level for different offset probabilities, thus better handling the localization uncertainty of the target to be detected.

[0083] In order to achieve precise and flexible adjustment, the weight function... The calculation formula for using nonlinear design is as follows:

[0084]

[0085] In the above formula, and These are the hyperparameters controlling the upper bound and curvature, respectively. This weighting function... It features a gentle curve in the middle and steep curves at both ends: when the initial predicted bounding box is close to accurate, the small curvature in the middle region supports fine-tuning of the boundary position; when the prediction deviation is large, the large curvature at both ends provides sufficient correction range, ensuring that the detection model has the ability to adjust to different prediction accuracy scenarios. In this embodiment, by using probability distribution modeling combined with the design of a non-uniform weight function, more accurate and robust bounding box regression performance of the target object can be achieved while maintaining optimization stability.

[0086] In the aforementioned visual UAV detection method, firstly, spatial channel collaborative enhancement processing is performed on the input features corresponding to the training images. This optimizes the spatial semantic information and channel feature representation of the input features, enhancing the representational capability of the features and providing rich semantic prior knowledge for the detection model, thereby improving the accuracy of target detection. Secondly, addressing the problem in existing methods that use the Dirac distribution to model the position of the target prediction box, treating the target prediction box as a deterministic value, which significantly increases the positional deviation when detecting small targets, leading to missed detection of key targets, this invention transforms the bounding box localization problem of the target to be detected into a probability distribution problem. By modeling the probability density distribution of the bounding box coordinates of the target to be detected to represent the localization uncertainty, a residual-based layer-by-layer optimization mechanism is used to iteratively optimize at different levels of the decoder, effectively alleviating the uncertainty problem of small target localization and improving the localization accuracy of target detection.

[0087] In an exemplary embodiment, enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image to be tested based on the first-layer decoder. This includes: inputting the image to be tested into the backbone network of the target detection model for processing to obtain intermediate features, and inputting the intermediate features into the encoder layer of the target detection model to obtain the input features corresponding to the image to be tested. Based on the spatial channel collaborative enhancement layer in the first-layer decoder, the input features are enhanced. The input features are obtained by decomposing along the height and width dimensions and then performing global average pooling. Corresponding height dimension features and width dimension features ; High-dimensional features and width dimension features Divided into Each set of height and width sub-features of equal size is given. Each height and width sub-feature is then subjected to depthwise separable convolution. Lightweight convolution is then used to align the resulting height and width sub-features, yielding multiple aligned height and width sub-features. The aligned height sub-features... The calculation formula is Aligned width dimension sub-features The calculation formula is , For application to the The size of the convolution kernel for each sub-feature The number of channels for each height dimension sub-feature and each width dimension sub-feature; multiple aligned height dimension sub-features and multiple aligned width dimension sub-features are concatenated and grouped normalized to generate height dimension attention weights respectively. Attention weights in width dimension ;in, , , This represents the Sigmoid activation function. Indicates the process along the height dimension Group normalization operation, Indicates operation along the width dimension Group normalization operation; adjusting high-dimensional attention weights Attention weights in width dimension Input features Element-wise multiplication yields spatial attention enhancement features. .

[0088] Please refer to Figure 2, which is a schematic diagram of the visual UAV detection network process. This network extracts input features from the input training image or the image to be detected, enhances the input features using a Spatial Channel Collaborative Enhancement (SCSA) layer, and then the first decoder layer generates an initial predicted bounding box based on the enhanced input features. Layers one through three of the decoder layer attach probabilities to the coordinate representation of the initial predicted bounding box, thus modeling the uncertainty of the target to be detected. Optionally, CascadeSpikeNet in Figure 2 is the backbone network of the detection model, used to process the input training image or the image to be detected to obtain intermediate features, and PHE is the encoder layer of the detection model, used to process the intermediate input features to obtain input features. Decoder Layer 1 is the first decoder layer of the detection model, which includes the Spatial Channel Co-enhancing Processing Layer (SCSA), the D-FINE Head (a refined bounding box prediction head based on distribution modeling), the Initial BBox (the initial predicted bounding box), and the Refined BBox (the target predicted bounding box). The Spatial Channel Co-enhancing Processing Layer (SCSA) combines the Shareable Multi-Semantic Spatial Attention (SMSA) mechanism with the Progressive Channel-wise Self-Attention (PCSA) mechanism. A flowchart of the Spatial Channel Co-enhancing Processing Layer is shown in Figure 3.

[0089] For example, since small targets of UAVs have complex spatial distribution and sparse semantic information, traditional spatial attention mechanisms are difficult to effectively capture multi-scale semantic features. This invention achieves feature decomposition through the SMSA mechanism and extracts rich spatial semantic information using a multi-kernel convolution strategy.

[0090] SMSA first processes the input features corresponding to the training images. The feature map is decomposed along the height and width dimensions, and then transformed into two unidirectional one-dimensional sequence structures through global average pooling. These are the height dimension features. and width dimension features This approach can reduce computational complexity while preserving key information about spatial dimensions. To learn different spatial distribution patterns and contextual relationships, high-dimensional features are... and width dimension features Divided into We obtain high-dimensional sub-features by having 10 independent sub-features of equal size. and width dimension sub-features The number of channels for each sub-feature is The expression for the sub-feature partitioning process is:

[0091]

[0092]

[0093] In the above formula, Represents the i-th height dimension sub-feature. Represents the i-th width dimension sub-feature; optionally, when When the value is 4, .

[0094] High-dimensional features and width dimension features Divided into After identifying independent sub-features of equal size, the spatial structural differences between the sub-features are then addressed. Each sub-feature is applied with depthwise separable convolutions of kernel sizes 3, 5, 7, and 9. The multi-kernel strategy effectively captures spatial semantic structures at different scales. Small kernels are suitable for extracting local detail features, while large kernels can model a wider range of contextual relationships, thus better adapting to the diverse feature representations of small objects. However, the decomposition of sub-features in the height and width dimensions, along with the independent processing of one-dimensional convolutions, leads to a limited receptive field; that is, each convolution operation can only capture spatial relationships in a single direction. To address this issue, we employ lightweight shared convolutions to align the height and width sub-features after depthwise separable convolution processing. The expression is as follows:

[0095]

[0096]

[0097] In the above formula, This represents the height dimension sub-feature after alignment. This represents the aligned width dimension sub-feature. For application to the The method uses the same convolution kernel size for each sub-feature, forcing both dimensions to learn consistent feature representations by using the same convolution kernel parameters on both the height and width dimensions. This implicitly models the dependencies between dimensions, effectively overcoming the limitations of single-dimensional convolution and ensuring the integrity and consistency of the feature space.

[0098] After obtaining sub-features with different semantics, i.e., determining the aligned height dimension sub-features. and aligned width dimension sub-features Next, a spatial attention map needs to be constructed to further enhance the feature representation. Specifically, this is achieved by integrating the features through a concatenation operation. Sub-feature information with different semantics, and using Grouping normalization is used for standardization. This method independently standardizes each sub-feature without introducing batch statistical noise, effectively reducing semantic interference between sub-features and preventing attention dilution. The spatial attention map is generated through the following process: Attention weights are generated along the height dimension, as shown in the following equation:

[0099]

[0100] In the formula, This represents the Sigmoid activation function. Indicates the process along the height dimension Group normalization is performed. Simultaneously, attention weights are generated along the width dimension, as shown in the following equation:

[0101]

[0102] in, Indicates operation along the width dimension Group normalization operation. This involves adjusting the high-dimensional attention weights. Attention weights in width dimension Input features Element-wise multiplication yields spatial attention enhancement features. .

[0103] In one exemplary embodiment, spatial attention enhancement features Spatial compression is performed to obtain compressed features. , This indicates spatial attention enhancement features. Spatial resolution from Compress to ; compressed features Perform linear mapping to generate a query matrix Q, a key matrix K, and a value matrix V, where, , C represents a constant representing the number of input feature channels; the self-attention features are determined based on the query matrix Q, the key matrix K, and the value matrix V. ; Characteristics of self-attention Global pooling is performed, and channel attention weights are generated using a sigmoid activation function. These channel attention weights are then applied to the spatial attention enhancement feature. Enhanced features are obtained

[0104] Existing channel attention mechanisms primarily model inter-channel dependencies based on convolutional operations, but there is still room for improvement when handling complex channel similarity. This invention utilizes a progressive channel self-attention (PCSA) mechanism, combining self-attention with spatial prior information provided by shared multi-semantic spatial attention (SMSA), to achieve channel similarity modeling. To preserve the multi-semantic spatial information extracted by SMSA and reduce the computational overhead of self-attention, PCSA employs a progressive compression method based on average pooling. As shown in the following equation, the spatial attention enhancement features output by SMSA are... Perform space compression processing on 7×7 pooling kernels to reduce the spatial resolution from Compress to .

[0105]

[0106] Using expressions Compressed features A linear mapping process is performed, and after the mapping, a query matrix Q, a key matrix K, and a value matrix V are generated, where... , C represents a constant representing the number of input feature channels. The self-attention features are determined based on the query matrix Q, the key matrix K, and the value matrix V. By analyzing the characteristics of self-attention Global pooling is performed, and channel attention weights are generated using a sigmoid activation function. These channel attention weights are then applied to the spatial attention enhancement feature. Enhanced features are obtained .

[0107] In this embodiment, in the Spatial Channel Co-enhancement Processing (SCSA) layer, SCSA first extracts multi-semantic spatial information from the input features corresponding to the image under test, providing accurate spatial prior knowledge for subsequent PCSA. PCSA then utilizes this spatial prior information to model semantic associations along the channel dimension, achieving synergistic optimization of spatial and channel dimensions. Compared with traditional methods, SCSA avoids channel compression strategies, effectively preventing the loss of key feature information. By applying this collaborative attention mechanism, the complementary advantages of spatial attention and channel attention can be fully leveraged, providing more accurate and richer feature representations for small target detection tasks, thereby enhancing the feature extraction and representation capabilities for small targets.

[0108] In one exemplary embodiment, based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set This includes: based on enhanced features The initial prediction box of the image to be tested is determined as follows: ,in, Indicates the center coordinates of the initial prediction box. This indicates the width and height of the initial prediction box; the initial prediction box... Convert to center coordinates and the initial edge distance set ;in, The distance from the center coordinates of the initial predicted bounding box to the first side of the top edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the second side of the bottom edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the third side of the left edge bounding box. Let the distance from the center coordinates of the initial prediction bounding box to the fourth side of the right edge box be denoted as ; based on the initial probability distributions corresponding to each side distance, determine the set of initial probability distributions as follows: , These are the initial probability distributions corresponding to the distances of the first, second, third, and fourth sides, respectively. The probability distributions represent the probability of each side distance at one of the N candidate offset positions.

[0109] Traditional bounding box regression methods are based on the Dirac delta distribution, treating the bounding box position as a precise, fixed value. However, small targets like UAVs may be subject to interference from factors such as blurring and occlusion, resulting in uncertainty regarding the true boundary position, which mainstream detectors often overlook. While existing distributed methods incorporate probability distribution representations, they suffer from reliance on anchor boxes, lack of iterative optimization, and coarse localization. To address this, this paper employs a Fine-grained Distribution Refinement (FDR) mechanism. This mechanism abandons the deterministic modeling approach of directly predicting fixed coordinates in traditional methods. Instead, it comprehensively considers multiple potential positions of the bounding box edges and iteratively optimizes the probability distribution, thereby achieving progressively more precise localization of the bounding box. The FDR mechanism transforms the bounding box regression problem into a probability distribution optimization problem. The offset space of the four sides of the predicted box is discretized into a finite number of candidate positions, each corresponding to a probability value. This probability value represents the likelihood that the center point of the predicted box will reach the true boundary of the object within a distance of that candidate position. This design transforms deterministic regression into probabilistic modeling, progressively improving localization accuracy by iteratively refining the offset probability distribution of the four sides.

[0110] Figure 4 illustrates the fine-grained distribution optimization mechanism. In the initial stage, the first decoder layer includes an SCSA layer and a D-FINE header that generates the initial probability distribution. Each bounding box is associated with four independent probability distributions, predicting the probability of candidate offsets from the center coordinates of the initial predicted box to the first edge of the top bounding box, the second edge of the bottom bounding box, the third edge of the left bounding box, and the fourth edge of the right bounding box. Using the initial predicted box as a reference, subsequent decoder layers no longer directly predict new initial predicted box coordinates but instead focus on gradually optimizing the accuracy of the initial predicted box by adjusting the probability distribution through residuals.

[0111] Based on enhanced features The initial prediction box of the image to be tested is determined as follows: ,in, Indicates the center coordinates of the initial prediction box. This indicates the width and height of the initial prediction box. The initial prediction box... Convert to center coordinates and the initial edge distance set , The distance from the center coordinates of the initial predicted bounding box to the first side of the top edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the second side of the bottom edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the third side of the left edge bounding box. This represents the distance from the center coordinates of the initial predicted bounding box to the fourth side of the right edge box. Based on the augmented features... Determine the initial edge distance set corresponding to the initial prediction box. And determine the initial edge distance set. The corresponding initial probability distribution set , These are the initial probability distributions corresponding to the distances of the first, second, third, and fourth sides, respectively. The initial probability distribution set is based on a multi-layer decoder. By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder.

[0112] In this embodiment, unlike the deterministic regression approach used in traditional detection models, the present invention transforms the bounding box localization problem into a probability distribution problem. It uses a fine-grained distribution optimization strategy to model the probability density distribution of bounding box coordinates to characterize localization uncertainty, and employs a residual progressive optimization mechanism to iteratively optimize at different network levels to improve localization accuracy.

[0113] In an exemplary embodiment, the pre-training process of the object detection model includes: processing training images based on a pre-acquired initial detection model to obtain initial training prediction boxes and target training prediction boxes corresponding to the training images; the initial detection model includes a multi-layer decoder; and determining the size difference term of the target training prediction box based on the width and height of the ground truth boxes corresponding to the initial training prediction boxes and the width and height of the target training prediction boxes. , and For the width and height of the actual bounding box, and Train the width and height of the predicted bounding box for the target. The normalized weight parameters are used to determine the first shape weight coefficients of the predicted bounding box based on the width and height of the ground truth bounding box. Second shape weighting coefficient Among them, the first shape weighting coefficient Second shape weighting coefficient Based on the first shape weighting coefficient Second shape weighting coefficient The shape-aware distance of the target training prediction box is determined by the center point coordinates of the target training prediction box and the ground truth box. , Train the center point coordinates of the predicted bounding box for the target. The coordinates of the center point of the true bounding box; the size difference term is expressed using an exponential function. and shape perception distance Normalization is performed to obtain auxiliary measurement parameters. And based on auxiliary metric parameters The loss function is determined as follows ;in, Train the intersection-union ratio (IU) of the predicted bounding boxes and the ground truth bounding boxes for the target. Train a shape penalty term for the predicted bounding box to the target. , For predefined shape parameters, The angle parameters are predefined; the target detection model is obtained by optimizing the initial detection model based on the loss function.

[0114] The target detection model is obtained by optimizing the model parameters of the initial detection model using the acquired loss function, and is used to detect target objects in the image to be tested. The target object can be a drone.

[0115] In traditional object detection tasks, the Intersection over Union (IoU) measures the proximity between the predicted and ground truth bounding boxes by calculating the ratio of their overlapping area to their total area. However, traditional methods have limitations when the target is small or micro-targets: due to the extremely small size of the target, even slight shifts in the predicted bounding box can cause drastic fluctuations in the IoU value. Traditional IoU and its improved versions fail to effectively model the shape sensitivity and scale dependence of the target, ultimately limiting the improvement of detection accuracy. To address these issues, this invention introduces Shape-IoU (Intersection over Union considering target shape information) and NWD (Normalized Wasserstein Distance) to calculate the loss function, fully considering the characteristics of small target detection and providing a more accurate bounding box regression loss function, thereby significantly improving the detection sensitivity of small targets.

[0116] Specifically, to better adapt to changes in object scale and the impact of prediction offset, this invention assigns different weights to regression biases in different directions based on the shape features (width-to-height ratio of the object) of the true bounding box in the label. Specifically, a first shape weight coefficient is defined. Second shape weighting coefficient for:

[0117]

[0118]

[0119] in, and For the width and height of the actual bounding box, The scale factor is the first shape weight coefficient. Second shape weighting coefficient The sum of the two values ​​must be 2, and their value directly reflects the shape characteristics of the true bounding box: when the true bounding box is a square, both values ​​are 1; when the true bounding box is a rectangle, the weight coefficient corresponding to the shorter side is larger. This is based on the first shape weight coefficient. Second shape weighting coefficient The shape-aware distance of the target training prediction box is constructed using the center point coordinates of the target training prediction box and the ground truth box. , Train the center point coordinates of the predicted bounding box for the target. These are the coordinates of the center point of the true bounding box.

[0120] Define the size difference term of the target training prediction box as , and For the width and height of the actual bounding box, and Train the width and height of the predicted bounding box for the target. For normalized weight parameters; in this embodiment, it can be... The value is 2. Size Difference Item Used to quantify the size difference between the predicted bounding box and the true bounding box. Further, in this invention, a shape penalty term is defined as... , For predefined shape parameters, These are predefined angle parameters.

[0121] NWD (Normalized Wasserstein Distance) is a metric specifically designed for small object detection, exhibiting good numerical stability and convergence properties. This invention uses an exponential function to measure shape-aware distance. Normalization is performed to obtain auxiliary measurement parameters. And based on auxiliary metric parameters The loss function is determined as follows ;in, A constant representing the number of input feature channels. The intersection-union ratio (IUU) of the predicted bounding boxes and the ground truth bounding boxes is trained for the target. Based on the loss function, the initial detection model is optimized to obtain the target detection model.

[0122] In this embodiment, the loss function is designed to fully consider the characteristics of small target detection. It achieves adaptive processing of targets with different shapes through shape weight coefficients and through size difference terms. To enhance sensitivity to changes in the size of small targets, an exponential function is used for normalization to ensure numerical stability and training convergence. This method effectively combines shape perception capabilities with the advantages of small target detection, providing a more accurate bounding box regression loss function for small target detection tasks, thereby enhancing the robustness and generalization ability of the target detection model in complex scenes.

[0123] In an exemplary embodiment, to verify the superiority of the proposed visual UAV detection method, a qualitative comparative analysis was performed on the Det-Fly dataset. This qualitative comparative analysis primarily evaluates the detection performance of small targets on UAVs using the human visual system. To verify the effectiveness of the proposed method, it was compared with five classic methods (ResNet, YOLOv10, YOLOv5, YOLOv8, and YOLOv9) under eight different motion backgrounds. The results are shown in Figure 5. The comparison shows that under challenging conditions such as low light and motion blur, this method can stably capture fine-grained edges and structures related to the UAV's shape, with higher overlap between the localized bounding boxes and the ground truth boxes, smaller deviations in box size and center point, and significantly reduced false negatives and false positives.

[0124] A quantitative comparative analysis of the proposed method was conducted on the Det-Fly dataset. This quantitative comparative analysis refers to evaluating the quality of the proposed algorithm using objective evaluation metrics. In this invention, mAP50, parameters, and GFLOPs are the main evaluation metrics. mAP50, as a performance evaluation metric, reflects the average detection accuracy of the model at an IoU threshold of 50%, directly reflecting the detection performance of UAV targets. Parameters is an indicator of the number of model parameters, used to assess the model's complexity and resource consumption. GFLOPs is an indicator of the model's computational complexity, used to compare the model's real-time performance in practical applications. These three metrics together form the basis for a comprehensive performance evaluation of the target detection model in this invention.

[0125] In the table below, the first column represents the target detection models in the prior art and the target detection model proposed in this invention. Bold text indicates the best results among various methods, and underlined text indicates the second-best results. As can be seen from the table, the method proposed in this invention achieves 97.5% mAP on the Det-Fly dataset with 28.4M parameters, which is 1.4 percentage points higher than the MHAF-YOLO method (a prior art target detection model). Compared to the ADMNET method for detecting UAV targets and the Gold-YOLO method with an improved FPN fusion strategy, the mAP is improved by 2.7 percentage points and 2.9 percentage points, respectively. Although some YOLO series models, such as YOLOv5-M and YOLOv8-M, are close to or even slightly better than the proposed method in terms of parameter count and computational cost, their detection accuracy is significantly lower. The number of parameters in YOLOv8-M is similar to that of the method proposed in this chapter, but its mAP50 is only 94.4%, which is 3.1 percentage points lower than the method in this paper. The number of parameters and computational complexity of the YOLOv5-M model are both better than those in this paper, but its mAP50 is 3.4 percentage points lower. The method proposed in this invention significantly surpasses the benchmark model RT-DETR and other comparative methods of the YOLO series in terms of detection accuracy.

[0126]

[0127] As shown in the table below, the proposed method achieves an mAP50 value of 45.1% on the ARD-MAV dataset, which is 2.0 percentage points higher than the benchmark RT-DETR method and significantly outperforms other classic models. The number of parameters and computational cost are also significantly lower than RT-DETR. In the table, bold text indicates the best result among various methods, and underlined text indicates the second-best result. Compared with the YOLO series methods, the proposed method achieves a good balance between detection accuracy and resource consumption, effectively improving the performance of UAV target detection.

[0128]

[0129] In this invention, firstly, a spatial channel collaborative enhancement layer provides rich semantic prior knowledge for the target detection model, thereby assisting in the generation of more accurate prediction boxes. Secondly, the traditional direct coordinate regression prediction is replaced with a probabilistic modeling process. A probability density representation is established for the four coordinate components of the target's bounding box. Coarse-grained initial probability estimates are generated at a shallow layer, and fine-grained distribution fine-tuning is performed at a deeper layer through residual connections, ultimately forming a convergent target probability distribution, which improves the robustness and accuracy of small target localization. Finally, the design of the loss function fully considers the characteristics of small target detection, achieving adaptive processing for targets of different shapes through shape weight coefficients and using size difference terms. By enhancing sensitivity to changes in the size of small targets and normalizing through an exponential function to ensure numerical stability and training convergence, a more accurate bounding box regression loss function is provided for small target detection tasks, thereby enhancing the robustness and generalization ability of the target detection model in complex scenes.

[0130] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0131] Based on the same inventive concept, this application also provides a visual drone detection device for implementing the visual drone detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the visual drone detection device provided below can be found in the limitations of the visual drone detection method described above, and will not be repeated here.

[0132] In an exemplary embodiment, as shown in FIG6, a visual unmanned aerial vehicle (UAV) detection device is provided, comprising: a detection module 602, wherein:

[0133] Detection module 602 is used to determine the target prediction box corresponding to the UAV target in the image to be tested based on a pre-trained target detection model; the target detection model includes a multi-layer decoder; wherein, the process of determining the target prediction box corresponding to the UAV target based on the target detection model includes: performing spatial channel co-enhancement processing on the input features corresponding to the image to be tested based on the first layer decoder to obtain enhanced features. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ; Based on the multi-layer decoder for the initial probability distribution set By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for layer-by-layer optimization is: , For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder; based on the set of target probability distributions. Weighting function and the initial edge distance set Determine the optimized back distance set, and determine the target prediction box corresponding to the UAV target based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

[0134] In one exemplary embodiment, the weighting function The calculation formula is In the formula, and These are the hyperparameters that control the upper bound and curvature, respectively.

[0135] In an exemplary embodiment, the detection module 602 is further configured to input the image to be tested into the backbone network of the target detection model for processing to obtain intermediate features, and input the intermediate features into the encoder layer of the target detection model to obtain the input features corresponding to the image to be tested. Based on the spatial channel collaborative enhancement layer in the first-layer decoder, the input features are enhanced. The input features are obtained by decomposing along the height and width dimensions and then performing global average pooling. Corresponding height dimension features and width dimension features ; High-dimensional features and width dimension features Divided into Each set of height and width sub-features of equal size is given. Each height and width sub-feature is then subjected to depthwise separable convolution. Lightweight convolution is then used to align the resulting height and width sub-features, yielding multiple aligned height and width sub-features. The aligned height sub-features... The calculation formula is Aligned width dimension sub-features The calculation formula is , For application to the The size of the convolution kernel for each sub-feature The number of channels for each height dimension sub-feature and each width dimension sub-feature; multiple aligned height dimension sub-features and multiple aligned width dimension sub-features are concatenated and grouped normalized to generate height dimension attention weights respectively. Attention weights in width dimension ;in, , , This represents the Sigmoid activation function. Indicates the process along the height dimension Group normalization operation, Indicates operation along the width dimension Group normalization operation; adjusting high-dimensional attention weights Attention weights in width dimension Input features Element-wise multiplication yields spatial attention enhancement features. .

[0136] In one exemplary embodiment, the detection module 602 is further configured to detect spatial attention-enhanced features. Spatial compression is performed to obtain compressed features. , This indicates spatial attention enhancement features. Spatial resolution from Compress to ; compressed features Perform linear mapping to generate a query matrix Q, a key matrix K, and a value matrix V, where, , , The constant representing the number of input feature channels; the self-attention features are determined based on the query matrix Q, the key matrix K, and the value matrix V. ; Characteristics of self-attention Global pooling is performed, and channel attention weights are generated using a sigmoid activation function. These channel attention weights are then applied to the spatial attention enhancement feature. Enhanced features are obtained .

[0137] In one exemplary embodiment, the detection module 602 is further configured to, based on the enhanced features The initial prediction box of the image to be tested is determined as follows: ,in, Indicates the center coordinates of the initial prediction box. This indicates the width and height of the initial prediction box; the initial prediction box... Convert to center coordinates and the initial edge distance set ;in, The distance from the center coordinates of the initial predicted bounding box to the first side of the top edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the second side of the bottom edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the third side of the left edge bounding box. Let the distance from the center coordinates of the initial prediction bounding box to the fourth side of the right edge box be denoted as ; based on the initial probability distributions corresponding to each side distance, determine the set of initial probability distributions as follows: , These are the initial probability distributions corresponding to the distances of the first, second, third, and fourth sides, respectively. The probability distributions represent the probability of each side distance at one of the N candidate offset positions.

[0138] In one exemplary embodiment, the visual drone detection device further includes:

[0139] The pre-training module processes training images based on a pre-acquired initial detection model to obtain initial training prediction boxes and target training prediction boxes corresponding to the training images. The initial detection model includes a multi-layer decoder. Based on the width and height of the ground truth boxes corresponding to the initial training prediction boxes and the width and height of the target training prediction boxes, the size difference term of the target training prediction boxes is determined. , and For the width and height of the actual bounding box, and Train the width and height of the predicted bounding box for the target. The normalized weight parameters are used to determine the first shape weight coefficients of the predicted bounding box based on the width and height of the ground truth bounding box. Second shape weighting coefficient Among them, the first shape weighting coefficient Second shape weighting coefficient , The scaling factor; based on the first shape weight coefficient. Second shape weighting coefficient The shape-aware distance of the target training prediction box is determined by the center point coordinates of the target training prediction box and the ground truth box. , Train the center point coordinates of the predicted bounding box for the target. The coordinates of the center point of the true bounding box; the shape-perceived distance is calculated using an exponential function. Normalization is performed to obtain auxiliary measurement parameters. And based on auxiliary metric parameters The loss function is determined as follows ;in, A constant representing the number of input feature channels. Train the intersection-union ratio (IU) of the predicted bounding boxes and the ground truth bounding boxes for the target. Train a shape penalty term for the predicted bounding box to the target. , For predefined shape parameters, The angle parameters are predefined; the target detection model is obtained by optimizing the initial detection model based on the loss function.

[0140] Each module in the aforementioned visual UAV detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0142] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0143] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A visual unmanned aerial vehicle (UAV) detection method, characterized in that, The method includes: determining the target prediction box corresponding to the UAV target in the image to be tested based on a pre-trained target detection model; the target detection model includes a multi-layer decoder; wherein, the process of determining the target prediction box corresponding to the UAV target based on the target detection model includes: performing spatial channel co-enhancement processing on the input features corresponding to the image to be tested based on the first-layer decoder to obtain enhanced features. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ; Based on the multi-layer decoder for the initial probability distribution set By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for the layer-by-layer optimization is: ， For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder; based on the set of target probability distributions. Weighting function and the initial edge distance set An optimized back distance set is determined, and a target prediction box corresponding to the UAV target is determined based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

2. The method according to claim 1, characterized in that, The weighting function The calculation formula is In the formula, and These are the hyperparameters that control the upper bound and curvature, respectively.

3. The method according to claim 1, characterized in that, The enhanced features are obtained by performing spatial channel co-enhancement processing on the input features corresponding to the image under test based on the first-layer decoder. The process includes: inputting the image to be tested into the backbone network of the target detection model for processing to obtain intermediate features, and inputting the intermediate features into the encoder layer of the target detection model to obtain the input features corresponding to the image to be tested. Based on the spatial channel collaborative enhancement layer in the first layer decoder, the input features are... The input features are obtained by decomposing along the height and width dimensions and then performing a global average pooling operation. Corresponding height dimension features and width dimension features The height dimension features and width dimension features Divided into Each set of height and width sub-features of equal size is given. Each height and width sub-feature is then subjected to depthwise separable convolution. Lightweight convolution is then used to align the resulting height and width sub-features, yielding multiple aligned height and width sub-features. The aligned height sub-features... The calculation formula is Aligned width dimension sub-features The calculation formula is ， For application to the The size of the convolution kernel for each sub-feature The number of channels for each height dimension sub-feature and each width dimension sub-feature; multiple aligned height dimension sub-features and multiple aligned width dimension sub-features are concatenated and grouped normalized to generate height dimension attention weights respectively. Attention weights in width dimension ;in, ， ， This represents the Sigmoid activation function. Indicates the process along the height dimension Group normalization operation, Indicates operation along the width dimension Group normalization operation; adjusting high-dimensional attention weights Attention weights in width dimension With the input features Element-wise multiplication yields spatial attention enhancement features. 。 4. The method according to claim 3, characterized in that, The method further includes: spatial attention enhancement features. Spatial compression is performed to obtain compressed features. ， This indicates spatial attention enhancement features. Spatial resolution from Compress to ; compressed features Perform linear mapping to generate a query matrix Q, a key matrix K, and a value matrix V, where, ， ， The constant representing the number of input feature channels; the self-attention features are determined based on the query matrix Q, the key matrix K, and the value matrix V. ; Characteristics of self-attention Global pooling is performed, and channel attention weights are generated using the Sigmoid activation function. These channel attention weights are then applied to the spatial attention enhancement feature. Enhanced features are obtained 。 5. The method according to claim 1, characterized in that, The enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set This includes: based on enhanced features The initial prediction box of the image to be tested is determined as follows: ,in, Indicates the center coordinates of the initial prediction box. This indicates the width and height of the initial prediction box; the initial prediction box... Convert to center coordinates and the initial edge distance set ;in, The distance from the center coordinates of the initial predicted bounding box to the first side of the top edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the second side of the bottom edge bounding box. The distance from the center coordinates of the initial predicted bounding box to the third side of the left edge bounding box. Let the distance from the center coordinates of the initial prediction bounding box to the fourth side of the right edge box be denoted as ; based on the initial probability distributions corresponding to each side distance, determine the set of initial probability distributions as follows: ， These are the initial probability distributions corresponding to the distances of the first, second, third, and fourth sides, respectively. The probability distributions represent the probability of each side distance at one of the N candidate offset positions.

6. The method according to claim 1, characterized in that, The pre-training process of the object detection model includes: processing the training images based on a pre-acquired initial detection model to obtain initial training prediction boxes and target training prediction boxes corresponding to the training images; the initial detection model includes a multi-layer decoder; and determining the size difference term of the target training prediction box based on the width and height of the ground truth boxes corresponding to the initial training prediction boxes and the width and height of the target training prediction boxes. ， and The width and height of the actual bounding box. and Train the width and height of the predicted bounding box for the target. The normalized weight parameters are used to determine the first shape weight coefficient of the predicted box based on the width and height of the ground truth bounding box. Second shape weighting coefficient Among them, the first shape weighting coefficient Second shape weighting coefficient ， The scaling factor; based on the first shape weight coefficient. Second shape weighting coefficient The shape-aware distance of the target training prediction box is determined by the center point coordinates of the target training prediction box and the ground truth box. ， Train the center point coordinates of the predicted bounding box for the target. The coordinates of the center point of the true bounding box; the distance to the shape is perceived using an exponential function. Normalization is performed to obtain auxiliary measurement parameters. And based on auxiliary metric parameters The loss function is determined as follows ;in, A constant representing the number of input feature channels. Train the intersection-union ratio (IU) of the predicted bounding boxes and the ground truth bounding boxes for the target. Train a shape penalty term for the predicted bounding box to the target. ， For predefined shape parameters, The angle parameters are predefined; based on the loss function, the initial detection model is optimized to obtain the target detection model.

7. A visual unmanned aerial vehicle (UAV) detection device, characterized in that, The device includes: a detection module, used to determine the target prediction box corresponding to a UAV target in a test image based on a pre-trained target detection model; the target detection model includes a multi-layer decoder; wherein, the process of determining the target prediction box corresponding to the UAV target based on the target detection model includes: performing spatial channel co-enhancement processing on the input features corresponding to the test image based on the first-layer decoder to obtain enhanced features. Based on the enhanced features Determine the initial prediction bounding box of the image to be tested and the initial edge distance set corresponding to the initial prediction bounding box. And determine the initial edge distance set. The corresponding initial probability distribution set ; Based on the multi-layer decoder for the initial probability distribution set By performing layer-by-layer optimization, the set of target probability distributions is obtained. The expression for the layer-by-layer optimization is: ， For the first The set of probability distributions for the layer decoder. and They represent the first Layer decoder and the first The confidence level of each edge distance corresponding to the layer decoder at each candidate offset position. For the first The prediction residuals of the layer decoder; based on the set of target probability distributions. Weighting function and the initial edge distance set An optimized back distance set is determined, and a target prediction box corresponding to the UAV target is determined based on the optimized back distance set; wherein, the optimized back distance set is... Weighting function Represents the distance of each initial edge in the initial edge distance set at the th position. The weighted value of each candidate offset position. and These represent the width and height of the initial prediction box, respectively.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images