Robustness detection method for aerial photography target of unmanned aerial vehicle

Abstract

The invention belongs to the technical field of computer vision, and discloses a robustness detection method for an aerial target of an unmanned aerial vehicle. A pulse neural network architecture is adopted, LIF neurons serve as basic calculation units, and invalid calculation is reduced through an event-driven sparse pulse distribution mechanism. In the feature extraction stage, a pulse adversarial interactive distillation module is introduced, L2 normalized energy constraint is applied to synaptic current, amplification and propagation of adversarial disturbance in the network are inhibited, and feature stability and detection robustness of the model under the conditions of noise interference and hostile attack are enhanced. The pulse channel characteristic refining module is based on a pulse perception-analog modulation mechanism, carries out adaptive modulation on synaptic current gain by using channel and space context information on the premise of maintaining pulse binary distribution characteristics, thereby highlighting target related characteristics and inhibiting complex background interference, and improving the target tracking precision. And the accuracy and reliability of unmanned aerial vehicle aerial image target detection are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a robust detection method for aerial targets captured by unmanned aerial vehicles. Background Technology

[0002] With the rapid development of drone technology, its applications in security patrol, disaster relief, traffic monitoring, and military reconnaissance are becoming increasingly widespread. In these application scenarios, drones typically need to perform real-time detection of ground targets in complex environments, which places high demands on target detection algorithms in terms of detection accuracy, real-time performance, and system stability. However, although target detection algorithms have made significant progress on standard public datasets in recent years, most existing methods rely on high-performance computing hardware and ample computing resources, making them difficult to deploy directly on embedded drone platforms where computing power, storage space, and power consumption are limited. Furthermore, drone aerial photography scenarios are characterized by drastic changes in perspective, complex backgrounds, and small target scales. In practical applications, they may also face noise disturbances and adversarial interference. Traditional target detection methods based on artificial neural networks are prone to performance degradation, false detections, or missed detections in complex or disturbed environments, making it difficult to simultaneously meet real-time and energy efficiency constraints while maintaining adversarial robustness. Spiking neural networks, as a computational model inspired by biological nervous systems, possess advantages such as event-driven processing, sparse computation, and natural noise resistance. They demonstrate great potential in low-power, high-efficiency information processing, providing a new technical approach to solving the energy efficiency and robustness issues in UAV target detection. Therefore, there is an urgent need for a UAV aerial target detection method that combines high-efficiency computation with robustness to meet the practical application needs of UAVs in complex environments and under resource-constrained conditions.

[0003] (1) Target detection method for drone aerial photography;

[0004] Drone aerial images commonly suffer from problems such as small target scale, drastic viewpoint changes, and highly cluttered backgrounds. To improve target detection performance, an increasing number of studies have introduced attention mechanisms to enhance feature representation capabilities. Based on different modeling methods, existing feature enhancement methods can be broadly divided into two categories: one is based on channel attention, which filters key features by modeling the dependencies between feature channels; the other is based on spatial attention, which focuses on salient regions in the image using large receptive field convolutions or self-attention mechanisms. However, these methods typically emphasize enhancing the amplitude of feature responses, neglecting the fact that while enhancing effective features, they may also simultaneously amplify adversarial noise or background interference. In attack or strongly interfered scenarios, this can easily lead to overfitting of the model to false textures, thereby weakening overall robustness.

[0005] (2) Robust target detection methods;

[0006] To address security issues in complex environments, various studies have proposed different robust target detection methods.

[0007] Some methods employ adversarial training strategies, introducing adversarial examples during the training phase to allow the model to adapt to both clean and perturbed samples during parameter updates, thereby improving its stability and generalization ability under malicious noise or extreme interference conditions. Although adversarial training has achieved some success in improving model robustness, it often relies on multi-step iterative attacks or dual forward and backward propagation processes, significantly increasing training time and computational overhead.

[0008] Furthermore, some studies have focused on input preprocessing, employing methods such as image denoising, frequency domain filtering, or low-pass constraints to suppress high-frequency noise and anomalous texture responses before model inference, thereby mitigating the impact of adversarial perturbations or complex backgrounds on detection results. However, such methods typically rely on prior assumptions about the type of perturbation and are prone to losing detailed information about the target itself in complex scenes, thus affecting detection accuracy.

[0009] Other works introduce constraint mechanisms at the feature level, such as feature smoothing, regularization, or consistency constraints, to limit the excessive sensitivity of the feature space to small perturbations, enabling the model to learn more stable and discriminative representations. Simultaneously, by designing robust loss functions, the dominant role of outlier samples or noisy gradients in the model optimization process is reduced, thus mitigating the overfitting problem of the model to local spurious textures or background interference.

[0010] Most of the methods mentioned above rely on complex module designs and training processes or high computational costs, inevitably introducing significant time and resource overhead while improving robustness. They also neglect the actual requirements of UAV platforms for real-time performance, lightweight design, and low power consumption.

[0011] (3) Spiking Neural Network (SNN);

[0012] Spiking neural networks (SNNs), inspired by biological neural systems, are gradually becoming an important research direction for edge-side intelligent sensing due to their event-driven computation mechanism and ultra-low power consumption. SNNs transmit information using a temporal encoding method of pulse firing; their neurons only generate discrete pulse responses when the membrane potential reaches a threshold. This event-driven and threshold-triggered mechanism naturally possesses a certain degree of ability to suppress weak perturbations and random noise. Compared to continuously activated artificial neural networks (ANNs), the output of SNNs is less sensitive to small changes in input amplitude, emphasizing temporal consistency and accumulated evidence. This provides a potential advantage in improving the stability of models under noise interference and anomalous perturbation conditions. Furthermore, the information integration process of SNNs in the temporal dimension helps smooth transient anomalous responses, thereby reducing the model's dependence on local spurious textures or high-frequency noise.

[0013] However, existing SNN research largely focuses on basic visual tasks such as classification, action recognition, or static object detection. Its network architecture design and training objectives often prioritize accuracy and energy efficiency optimization, neglecting robust features and adversarial security issues in complex scenarios. In drone aerial photography scenarios, due to the small target scale, cluttered backgrounds, and drastic viewpoint changes, models not only need sophisticated spatiotemporal feature representation capabilities but also need to maintain stable detection performance under adversarial perturbations or strong noise interference. Directly applying existing SNN architectures to such tasks often fails to effectively distinguish between real targets and spurious responses introduced by perturbations, leading to a significant decline in detection performance. Summary of the Invention

[0014] This invention addresses the problems of high computational power consumption and weak anti-interference capability in UAV target detection technology by proposing a robust detection method for UAV aerial targets, which integrates adversarial interactive distillation and feature refinement.

[0015] The technical solution of this invention: A robust detection method for drone aerial photography targets, comprising the following steps:

[0016] Step 1: Data preprocessing and timing coding;

[0017] Aerial images captured by a drone are acquired and their dimensions are normalized. A direct encoding strategy is then employed to copy and expand the normalized static RGB image over time, generating a shape... The pulse input tensor, where For time step, Batch size;

[0018] Step 2: Construct a deep SNN feature extraction network;

[0019] A deep spiking neural network based on residual structure is used as the backbone network to extract multi-scale features of images. Furthermore, a spiking adversarial interaction distillation (AID) module and a spiking channel-wise feature refinement (CFR) module are integrated into the deep spiking neural network based on residual structure.

[0020] The basic unit in a deep spiking neural network based on residual structures adopts the Leaky Integrate-and-Fire (LIF) neuron model; the LIF neuron simulates the potential accumulation, leakage, and pulse firing processes of a biological neuron. At discrete time step t, the membrane potential of neuron i... The update follows the following difference dynamics equation:

[0021]

[0022]

[0023] in, This is the membrane potential time constant, used to control the leakage rate of the membrane potential; It is the synaptic input current at the current time step, which is obtained by weighted convolution and normalization of the pulse output of the previous layer; The pulse output from the previous moment; when the membrane potential exceeds the threshold... At that time, a binary pulse is emitted; It is a step function;

[0024] The multi-scale feature map output by the backbone of the residual-structured spiking neural network is sequentially connected to a spiking adversarial interactive distillation module and a spiking channel feature refinement module to obtain an enhanced spiking feature sequence. The spiking adversarial interactive distillation module is set at the mid-scale feature output end, and the spiking channel feature refinement module is set at the high-scale feature output end. Both of them contain LIF neurons and act on features at different scales.

[0025] Step 3: Temporal aggregation and target decoding;

[0026] The enhanced pulse feature sequence is used as input to the YOLO detector head and is aggregated in the time dimension to convert the discrete and sparse pulse firing information into a stable and continuous analog feature representation.

[0027] Adversarial interference typically manifests as abnormally large fluctuations in synaptic currents, causing LIF neurons to fire pulses at incorrect time steps. The pulse adversarial interactive distillation module is used to suppress abnormal pulse firing behavior caused by adversarial perturbations in deep spiking neural networks based on residual structures. It introduces L2 normalization constraints at the end of the residual path in the residual-structure-based deep spiking neural network, projecting the accumulated synaptic current onto a unit hypersphere, thereby smoothing high-frequency noise and suppressing perturbations.

[0028] For the original input pulse tensor at the current time step t: The processing flow is as follows;

[0029] The input pulse tensor is convolved through an intermediate layer to convert it into an analog synaptic current. And drive the intermediate LIF neurons to fire intermediate layer pulses. :

[0030]

[0031]

[0032] in, This represents the convolution operation. Indicates batch normalization. These are the kernel weights;

[0033] The intermediate layer pulses are passed through a second convolution layer to generate residual current characteristics. :

[0034]

[0035] in, Indicates batch normalization. These are the kernel weights;

[0036] Calculate the identity mapping current of the input pulse tensor :

[0037]

[0038] in, This is the projection bias vector. The projective bias matrix is ​​a learnable matrix.

[0039] Add the residual current to the constant current:

[0040]

[0041] exist Before driving the LIF neurons in the output layer, L2 normalization is performed on them to dynamically adjust the input gain of the LIF neurons, giving them a certain degree of defense against adversarial noise.

[0042]

[0043] in, Denotes the L2 norm along the channel dimension. To learnable affine transformation parameters, It is a stable term;

[0044] Finally, the normalized current drives the LIF neuron at the end of the AID module to obtain the denoised robust pulse features. :

[0045] .

[0046] To address the sparsity and discreteness of binary pulse information in SNNs, a pulse channel feature refinement module is designed. This module employs a hybrid attention mechanism of pulse sensing and analog modulation to adaptively enhance channels and spatial regions with discriminative information, improving target detection performance while preserving the binary nature of pulses. The pulse channel feature refinement module includes pulse feature to analog feature conversion, channel attention weight generation, spatial attention weight generation, analog attention weight generation, and pulse sensing gain modulation.

[0047] The conversion from pulse characteristics to analog characteristics is specifically as follows:

[0048] For the pulse input sequence from the previous layer Converted into corresponding synaptic current :

[0049]

[0050] in, This represents the initial pulse to analog current mapping, including convolution, BatchNorm, and activation layers;

[0051] The channel attention weight generation is specifically as follows:

[0052] First, global average pooling is performed on the accumulated synaptic current of the current pulse input to obtain channel statistics:

[0053]

[0054] in, This represents the cumulative response statistics for each channel; Indicates at time step , No. individual channels, spatial locations The cumulative synaptic current;

[0055] Introduce a lightweight affine transformation network to generate channel attention weights:

[0056]

[0057] in, To enhance channel attention weights, , For learnable weights, , For bias;

[0058] The spatial attention weight generation specifically involves:

[0059] For the same pulse input synaptic current Spatial-level attention mapping is achieved using two convolutions:

[0060]

[0061] This represents the importance weight of each spatial location;

[0062] The simulated attention weight generation specifically involves:

[0063] Channel attention weights Spatial attention weights Tensor reconstruction is performed to obtain simulated attention weights:

[0064]

[0065] The gain modulation of the pulse sensing is specifically as follows:

[0066] Simulate attention weights Used for gain modulation of synaptic current:

[0067]

[0068] The LIF neurons at the end of the CFR module then produce refined pulse features:

[0069] .

[0070] Step 3 specifically involves: analyzing the pulse features at each scale. Averaging over time yields the temporal aggregated feature map:

[0071]

[0072] in, For time step, Represents a continuous feature map at the corresponding scale;

[0073] Aggregate feature maps in the time domain Input the target detection decoding head, use 1×1 convolution to decode the feature map, and predict the bounding box regression parameters, target confidence and class probability of the target respectively;

[0074] For temporal aggregated feature maps from different scales, the above temporal aggregation and decoding processes are performed respectively, and the prediction results of each scale are summarized. The final target detection result is obtained by filtering through non-maximum suppression.

[0075] The beneficial effects of this invention are:

[0076] (1) It adopts a spiking neural network (SNN) architecture, with LIF neurons as the basic computing unit. It reduces invalid computation through an event-driven sparse pulse firing mechanism, thereby reducing overall computing power consumption. It is suitable for deployment on UAV airborne platforms with limited computing resources and power consumption.

[0077] (2) In the feature extraction stage, the present invention introduces the pulse adversarial interactive distillation module (AID), which effectively suppresses the amplification and propagation of adversarial disturbances in the network by applying L2 normalized energy constraints to the synaptic current, thereby enhancing the feature stability and detection robustness of the model under noise interference and malicious attack conditions.

[0078] (3) The pulse channel feature refinement module (CFR) proposed in this invention is based on the pulse sensing-analog modulation mechanism. Under the premise of maintaining the binary pulse emission characteristics, it uses channel and spatial context information to adaptively modulate the synaptic current gain, thereby highlighting the target-related features and suppressing complex background interference, and improving the accuracy and reliability of target detection in UAV aerial images. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0080] Figure 2 This is a schematic diagram of the internal structure of the Pulse Counteracting Interactive Distillation Module (AID).

[0081] Figure 3 This is a schematic diagram of the internal structure of the pulse channel feature refinement module (CFR).

[0082] Figure 4 This image shows the detection results of the present invention in an adversarial scenario on the VisDrone dataset.

[0083] Figure 5 This image shows the network's detection performance in adversarial scenarios on the VisDrone dataset. Detailed Implementation

[0084] The present invention will be further described in detail below with reference to specific embodiments, but the present invention is not limited to the specific embodiments.

[0085] Robust detection methods for drone aerial photography targets include network model construction, training, and performance verification.

[0086] (1) Network model construction;

[0087] A target detection network named EMS-ADV is constructed, which uses a spiking neural network (SNN) as the backbone structure and combines a spiking adversarial interactive distillation module and a spiking channel feature refinement module to improve detection robustness.

[0088] The backbone network employs an improved ResNet-18 architecture, replacing standard convolutional layers with spiking convolutional layers (Snn_Conv2d) and using LIF neurons as the basic computational units to implement event-driven spiking information processing. The network's time window length T is set to 3.

[0089] AID and CFR modules are inserted at the mid-scale feature output (P4) and high-scale feature output (P5) of the backbone network, respectively, to improve the robustness of features and target recognition capabilities.

[0090] The AID module performs residual modeling on the synaptic current corresponding to the input features, and applies L2 normalization constraints after adding the residuals to the identity mapping to limit the synaptic current and suppress amplification against disturbances. Its output serves as the input for subsequent pulse delivery.

[0091] The CFR module extracts channel context information and spatial context information of the input features, generates analog weights, and uses these weights to modulate the synaptic current gain. While maintaining the binary pulse firing characteristics, it highlights the target-related features and suppresses background interference, and finally outputs the refined pulse features.

[0092] (2) Model training;

[0093] The model training process includes two steps: adversarial example generation and model parameter updating.

[0094] In each training iteration, a decision is made based on a preset probability p whether to modify the original images of the current batch. Adversarial perturbations are implemented. Adversarial examples are generated using the Projected Gradient Descent (PGD) method. This is to simulate input scenarios under conditions of malicious disturbance.

[0095] The iterative process of a PGD attack is as follows: First, the original image is initialized with a perturbation.

[0096]

[0097] in The noise is randomly sampled within the perturbation constraint range; then, through multiple iterations, it is updated by gradient and projected onto the specified perturbation range to obtain the final adversarial example. .

[0098] The generated adversarial examples are temporally encoded and then input into the network. Spiking neurons fire according to LIF dynamics, and the target detection loss function is calculated. Including bounding box regression loss Target confidence loss and category loss :

[0099]

[0100] Since the pulse firing function is a non-differentiable step function, this embodiment uses a surrogate gradient method during training to maintain the step characteristics during the forward propagation stage and approximate its gradient using a differentiable function during the back propagation stage, thereby achieving end-to-end training.

[0101] During model training, the batch size was set to 64, and the number of training iterations was set to 300. The optimization process employed a stochastic gradient descent-based strategy, with an initial learning rate of 0.01. A piecewise decay strategy was used to adjust the learning rate, eventually reducing it to 0.1 times the initial value to ensure stable convergence in later training stages. The optimizer's momentum parameter was set to 0.937, and the weight decay coefficient was set to 0.0005 to suppress overfitting. For adversarial training, adversarial examples were introduced to participate in model optimization, and the gradient update magnitude of the adversarial perturbation was scaled with a scaling factor of 0.1. Adversarial training was gradually introduced after the model achieved initial convergence, starting after the 10th training iteration. Throughout the training process, the proportion of adversarial examples in the batch was dynamically controlled, with a maximum proportion not exceeding 0.5, to achieve a balance between improved robustness and the performance of clean samples.

[0102] (3) Effect verification;

[0103] This invention validates the effectiveness of the proposed method on the publicly available UAV aerial target detection dataset VisDrone2019-DET. The VisDrone2019-DET dataset consists of UAV aerial images from multiple scenes, including complex environments such as urban roads, residential areas, and squares. It is characterized by small target scale, high density, cluttered backgrounds, and drastic viewpoint changes, and is widely used to evaluate the performance and robustness of target detection algorithms under UAV aerial conditions, realistically reflecting the detection difficulty in actual UAV application scenarios.

[0104] Tests were performed on adversarial examples generated in the VisDrone2019-DET dataset, such as... Figure 4 and Figure 5 As shown, the original network without the technical solution of this invention exhibits a significant decrease in target detection performance under adversarial perturbation conditions, with a large number of missed detections. However, after adopting the network structure proposed in this invention, the model can still maintain effective target recognition in adversarial examples, the detection results are more stable, and the accuracy of target localization and category discrimination are significantly improved, verifying the robust detection effect of this invention under adversarial interference environments.

Claims

1. A robust detection method for drone aerial photography targets, characterized in that, The steps are as follows: Step 1: Data preprocessing and timing coding; Aerial images captured by a drone are acquired and their dimensions are normalized. A direct encoding strategy is then employed to copy and expand the normalized static RGB image over time, generating a shape... The pulse input tensor, where For time step, Batch size; Step 2: Construct a deep SNN feature extraction network; A deep spiking neural network based on residual structure is used as the backbone network to extract multi-scale features of images. A pulse adversarial interactive distillation module and a pulse channel feature refinement module are also connected to the deep spiking neural network based on residual structure. In deep spiking neural networks based on residual structures, the basic unit adopts the LIF neuron model; at discrete time step t, the membrane potential of neuron i is... u i l t The update follows the following difference dynamics equation: u i l t =τ⋅ u i l t-1 ⋅ 1- o i l t-1 + I i l t ； o i l t = Θ u i l t - V th ； in, This is the membrane potential time constant, used to control the leakage rate of the membrane potential; I i l t It is the synaptic input current at the current time step, which is obtained by weighted convolution and normalization of the pulse output of the previous layer; o i l t -1 The pulse output from the previous moment; when the membrane potential exceeds the threshold... At that time, a binary pulse is emitted; It is a step function; The multi-scale feature map output by the backbone of the residual-structured spiking neural network is sequentially connected to a spiking adversarial interactive distillation module and a spiking channel feature refinement module to obtain an enhanced spiking feature sequence. The spiking adversarial interactive distillation module is set at the mid-scale feature output end, and the spiking channel feature refinement module is set at the high-scale feature output end. Both of them contain LIF neurons and act on features at different scales. Step 3: Temporal aggregation and target decoding; The enhanced pulse feature sequence is used as input to the YOLO detector head and is aggregated in the time dimension to convert the discrete and sparse pulse firing information into a stable and continuous analog feature representation.

2. The robust detection method for drone aerial targets according to claim 1, characterized in that, The pulse adversarial interactive distillation module is used to suppress abnormal pulse firing behavior caused by adversarial perturbations in deep spiking neural networks based on residual structures; Introducing L2 normalization constraints at the end of the residual path in a deep spiking neural network based on residual structure projects the accumulated synaptic current onto a unit hypersphere, thereby smoothing high-frequency noise and suppressing disturbances. For the original input pulse tensor at the current time step t: S in t ∈ {0,1 } B × C × H × W The processing flow is as follows; The input pulse tensor is convolved through an intermediate layer to convert it into an analog synaptic current. I conv1 t And drive the intermediate LIF neurons to fire intermediate layer pulses. S mid t : I conv1 t = B 1 W 1 ⊛S in t ； S mid t =LIF I conv1 t ;τ, V th ； in, This represents the convolution operation. Indicates batch normalization. These are the kernel weights; The intermediate layer pulses are passed through a second convolution layer to generate residual current characteristics. I res t : I res t = B 2 W 2 ⊛ S mid t ； in, Indicates batch normalization. These are the kernel weights; Calculate the identity mapping current of the input pulse tensor I identity t : I identity t = B proj W proj ⊛ S in t ； in, This is the projection bias vector. The projective bias matrix is ​​a learnable matrix. Add the residual current to the constant current: I sum t = I res t + I identity t ； exist I sum t Before driving the LIF neurons in the output layer, L2 normalization is performed on them to dynamically adjust the input gain of the LIF neurons, giving them a certain degree of defense against adversarial noise. I ̂ AID t =γ⋅ I sum t I sum t 2 +ϵ ； in, Denotes the L2 norm along the channel dimension. To learnable affine transformation parameters, It is a stable term; Finally, the normalized current drives the LIF neuron at the end of the AID module to obtain the denoised robust pulse features. S out_AID t : S out_AID t =LIF I ̂ AID t ;τ, V th 。 3. The robust detection method for drone aerial targets according to claim 2, characterized in that, The pulse channel feature refinement module includes pulse feature to analog feature conversion, channel attention weight generation, spatial attention weight generation, analog attention weight generation, and pulse sensing gain modulation. The conversion from pulse characteristics to analog characteristics is specifically as follows: For the pulse input sequence from the previous layer S out_AID t Converted into corresponding synaptic current I t : I t = F base ( S out_AID t ) ； in, This represents the initial pulse to analog current mapping, including convolution, BatchNorm, and activation layers; The channel attention weight generation is specifically as follows: First, global average pooling is performed on the accumulated synaptic current of the current pulse input to obtain channel statistics: z c = 1 H × W ∑ i=1 H ∑ j=1 W I c,i,j t ； in, This represents the cumulative response statistics for each channel; I c,i,j t Indicates at time step , No. individual channels, spatial locations The cumulative synaptic current; Introduce a lightweight affine transformation network to generate channel attention weights: ； in, To enhance channel attention weights, , For learnable weights, , For bias; The spatial attention weight generation specifically involves: For the same pulse input synaptic current I t Spatial-level attention mapping is achieved using two convolutions: M spa =σ conv 2 δ conv 1 I t ； This represents the importance weight of each spatial location; The simulated attention weight generation specifically involves: Channel attention weights Spatial attention weights Tensor reconstruction is performed to obtain simulated attention weights: ； The gain modulation of the pulse sensing is specifically as follows: Simulate attention weights Used for gain modulation of synaptic current: I ̂ CFR t =M ⊙ I t ； The LIF neurons at the end of the CFR module then produce refined pulse features: S out_CFR t =LIF I ̂ CFR t ;τ, V th 。 4. The robust detection method for drone aerial targets according to claim 1, characterized in that, Step 3 specifically involves: analyzing the pulse features at each scale. S out_CFR t Averaging over time yields the temporal aggregated feature map: F= 1 T ∑ t=1 T S t ； in, For time step, Represents a continuous feature map at the corresponding scale; Aggregate feature maps in the time domain Input the target detection decoding head, use 1×1 convolution to decode the feature map, and predict the bounding box regression parameters, target confidence and class probability of the target respectively; For temporal aggregated feature maps from different scales, the above temporal aggregation and decoding processes are performed respectively, and the prediction results of each scale are summarized. The final target detection result is obtained by filtering through non-maximum suppression.

Images