A UAV Target Detection Method Based on Spiking Neural Networks

Abstract

This invention discloses a UAV target detection method based on a spiking neural network. It reduces latency by directly training the spiking neural network and enhances spatiotemporal feature extraction using a spiking multi-scale attention mechanism, thereby improving the accuracy of target detection. The method includes: acquiring a UAV image to be detected and performing pulse encoding; inputting the pulse signal into a spiking residual backbone network composed of pulse residual basic blocks for feature extraction, generating multi-scale and multi-level feature maps; processing the feature maps using a spiking multi-scale attention mechanism; and then inputting the processed feature maps into a detection head for target detection and outputting the results. This invention can extract richer image feature information, has the advantages of high computational performance and low energy consumption, and achieves low latency through a small time step.

Description

Technical Field

[0001] This invention belongs to the field of neural network technology and is a method for UAV target detection based on spiking neural networks. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have been widely used in both civilian and military fields due to their high cost-effectiveness, flexibility, and efficiency, thanks to declining manufacturing costs and advancements in flight control technology. UAV target detection has important applications in various fields such as power line inspection, plant growth disaster analysis, emergency rescue, public safety monitoring, and traffic management. However, because UAVs typically fly at high altitudes, the images they capture have a wide field of view and complex, variable backgrounds, increasing the difficulty of target detection.

[0003] Drones' limited energy is primarily used to maintain flight, thus significantly restricting their efficiency in image recognition and processing. While drone target detection methods based on traditional Artificial Neural Networks (ANNs) can optimize computation through model pruning and parameter quantization, they still rely heavily on floating-point operations. This not only consumes substantial computational resources but also significantly increases energy consumption, posing a limitation for resource-constrained drone platforms.

[0004] Spiking Neural Networks (SNNs), as the third generation of artificial neural networks, exhibit extremely high energy efficiency due to their event-driven mechanism and efficient computational characteristics. SNNs mimic the workings of a biological brain, using binary pulse signals (0 or 1) instead of traditional continuous real values ​​for computation. Sparse addition is triggered only when a pulse is emitted; otherwise, the network remains idle, significantly reducing computational complexity and energy consumption. However, in implementing this invention, it was discovered that existing spiking neural networks still suffer from insufficient accuracy in UAV target detection. Summary of the Invention

[0005] This invention provides a UAV target detection method based on spiking neural networks, aiming to solve the problem of low accuracy of current spiking neural networks in UAV target detection. This method significantly improves target detection performance while effectively reducing detection latency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for UAV target detection based on a spiking neural network includes the following steps:

[0008] Step 1: Acquire the image of the UAV to be inspected and perform pulse coding on it;

[0009] Step 2: Input the pulse signal into the pulse residual backbone network composed of pulse residual basic blocks, perform feature extraction, and obtain feature maps of different scales and levels;

[0010] Step 3: Input the feature map obtained in Step 2 into the pulse multi-scale attention mechanism for processing;

[0011] Step 4: Input the feature map processed in Step 3 into the detection head to perform target detection and output the detection results.

[0012] As a preferred embodiment, the pulsed residual basic block includes a residual path and a short-circuit path; the pulsed basic block includes LIF neurons, convolution (Conv), and batch normalization (BN);

[0013] The LIF neuron converts the floating-point inputs of the neuron at different time steps into binary pulse representations (0 or 1) through nonlinear operations;

[0014] Convolutional layers extract spatial features, while BN layers stabilize the training process by normalizing the output.

[0015] O block =BN(Conv(LIF(X) block )))

[0016] Where: X block Floating-point input represents the input, O block The output represents the pulse basic block; in the residual path, two pulse basic blocks are used consecutively, while in the shortcut path, a max pooling layer is applied for downsampling to reduce parameters; then, the pulse-based output from the shortcut path is connected to the output of the residual path after passing through another pulse basic block.

[0017] As a preferred embodiment, step one specifically comprises:

[0018] The input UAV image to be detected is pulse encoded to obtain an image pulse signal; the input image to be detected is a visible light image or an infrared image, and the format of the image to be detected is RGB.

[0019] As a preferred embodiment, step two specifically involves:

[0020] The pulse residual backbone network consists of pulse residual basic blocks, containing N layers of pulse basic blocks, where N is an integer greater than 2; feature maps of different scales and hierarchical structures are extracted from the image pulse signal; the time step of the pulse residual basic block is 2.

[0021] As a preferred embodiment, step three specifically comprises:

[0022] The feature maps obtained in step two are fed into the pulsed multi-scale attention mechanism for feature enhancement. The pulsed multi-scale attention mechanism uses feature grouping to divide the features into multiple sets across different resources, ensuring that each group is well distributed in space. Then, the feature maps of these groups are processed in parallel through 1×1 and 3×3 attention branches.

[0023] X = [X0, X1, X...] i ,…,X G ],X i ∈R T×(C / / G)×H×W

[0024] Where: the input feature tensor X is represented as X=[X0,X1,X2] i ,…,X G ], where: X i For the i-th subgroup feature, satisfying X i ∈R T×(C / / G)×H×W R is a matrix representing the dimension of the parameter tensor; T represents the time step, C is the total number of channels, G represents the number of groups, and H and W are the height and width of the feature map, respectively; by setting G << C, subgroup convolution can utilize the parallel computing capabilities of modern hardware to process across different groups simultaneously.

[0025] P=GN(σ(f PW (AvgPool H (X S )))⊙σ(f PW (AvgPool W (X S )))⊙X S )

[0026] Where: σ represents the Sigmoid function, ⊙ is the element-wise dot product, GN is the group normalization, and AvgPool is the pool normalization function. H AvgPool represents the global average pooling operation performed along the height dimension H. W P represents the global average pooling operation performed along the width dimension W; P is achieved by applying pointwise convolution f to the global average pooling results across the two spatial dimensions (H, W). PW The output is calculated from; where: AvgPool H (X S )∈R T×(C / / G)×1×W and AvgPool H (X S )∈R T×(C / / G)×H×1 These represent global average pooling operations along the height and width dimensions, respectively; X S ∈R T×(C / / G)×H×W Indicates the input feature map;

[0027] In the 1×1 attention branch Att 1×1In this process, 1×1 convolution operations do not require dimensionality reduction, thus enabling channel feature integration and spatial attention computation;

[0028]

[0029] in: Represents matrix multiplication, AvgPool(P)∈R T×(C / / G)×1×1 Global average pooling of P is used to produce a compact feature map that captures the global context; in the 3×3 attention branch Att 3×3 In the middle, a 3×3 convolutional layer f is stacked. 3×3 It is used to capture multi-scale feature representations and local contextual information:

[0030]

[0031] Where: AvgPool(f 3×3 (X S The first part is an intermediate feature map after 3×3 convolution and subsequent pooling; the second part is an attention map. 1×1 and Att 3×3 The shapes are all R T×1×H×W This allows them to effectively capture spatial dependencies while maintaining inter-channel interaction;

[0032] Att S =σ(Att) 1×1 +Att 3×3 )

[0033] Fusion of multi-scale attention maps Att S The final output feature map O is obtained by summing the outputs of the 1×1 and 3×3 branches, performing sigmoid activation, and then multiplying it element-wise with the input feature map X. att :

[0034] O att =X⊙Att S .

[0035] As a preferred embodiment, step four specifically comprises:

[0036] In the feature map obtained in step three, the last membrane potential of the neuron is used as input, resulting in a five-dimensional feature map T×B×C×H×W. By averaging the T dimension, a four-dimensional feature map B×C×H×W is obtained. Here, T represents the time step, B is the batch size, C is the total number of channels, and H and W represent the height and width of the feature map, respectively. Then, multiple anchor boxes of different scales are used for prediction. For each anchor box, the matching degree with the target is calculated, and the target's category and location are predicted. After predicting the category and location for each anchor box, non-maximum suppression (NMS) is used to remove overlapping detection results, retaining only the result with the highest confidence. Finally, the target's bounding box and category are derived based on the prediction results, and the detection results are output.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] This invention addresses the high energy consumption problem of current UAVs based on traditional neural networks by designing a UAV target detection algorithm based on direct training of a spiking neural network. It achieves high detection accuracy with low latency, using a time step of only 2. By introducing a spiking multi-scale attention mechanism that incorporates time and channel dimensions, it not only enhances the network's feature extraction capabilities but also significantly reduces the spiking rate of the detection head, thereby effectively reducing energy consumption and improving overall performance. Attached Figure Description

[0039] Figure 1 This is the pulse residual backbone network part of the present invention.

[0040] Figure 2 It is the pulse residual basic block structure in the model of this invention.

[0041] Figure 3 This is the structure of the pulse multiscale attention mechanism of the present invention.

[0042] Figure 4 This is a flowchart illustrating the detection method in this invention. Detailed Implementation

[0043] The present invention will be further described below with reference to embodiments.

[0044] This invention provides a UAV target detection method based on a spiking neural network, comprising spiking encoding of UAV images, feature extraction using a spiking residual backbone network, feature enhancement through a spiking multi-scale attention mechanism, and finally outputting the target detection result. The following is combined with... Figure 1-4 The examples and implementation details further illustrate this.

[0045] A method for UAV target detection based on a spiking neural network includes the following steps:

[0046] Step 1: Acquire the image of the UAV to be inspected and perform pulse coding on it;

[0047] Step one specifically involves:

[0048] The input UAV image to be detected is pulse encoded to obtain an image pulse signal; the input image to be detected is a visible light image or an infrared image, and the format of the image to be detected is RGB.

[0049] Step 2: Input the pulse signal into the pulse residual backbone network composed of pulse residual basic blocks, perform feature extraction, and obtain feature maps of different scales and levels;

[0050] The pulsed residual basic block includes residual paths and short-circuit paths; the pulsed basic block includes LIF neurons, convolution (Conv), and batch normalization (BN);

[0051] The LIF neuron converts the floating-point inputs of the neuron at different time steps into binary pulse representations (0 or 1) through nonlinear operations;

[0052] Convolutional layers extract spatial features, while BN layers stabilize the training process by normalizing the output.

[0053] O block =BN(Conv(LIF(X) block )))

[0054] Where: X block Floating-point input represents the input, O block The output represents the pulse basic block; in the residual path, two pulse basic blocks are used consecutively, while in the shortcut path, a max pooling layer is applied for downsampling to reduce parameters; then, the pulse-based output from the shortcut path is connected to the output of the residual path after passing through another pulse basic block.

[0055] Step two specifically involves:

[0056] The pulse residual backbone network consists of pulse residual basic blocks, containing N layers of pulse basic blocks, where N is greater than 2.

[0057] Integers; extract feature maps of different scales and hierarchical structures from image pulse signals; the time step of the pulse residual basic block is 2.

[0058] Step 3: Input the feature map obtained in Step 2 into the pulse multi-scale attention mechanism for processing;

[0059] Step three specifically involves:

[0060] The feature maps obtained in step two are fed into the pulsed multi-scale attention mechanism for feature enhancement. The pulsed multi-scale attention mechanism uses feature grouping to divide the features into multiple sets across different resources, ensuring that each group is well distributed in space. Then, the feature maps of these groups are processed in parallel through 1×1 and 3×3 attention branches.

[0061] X = [X0, X1, X...] i ,…,X G ],X i ∈R T×(C / / G)×H×W

[0062] Where: the input feature tensor X is represented as X=[X0,X1,X2] i ,…,X G ], where: X i For the i-th subgroup feature, satisfying X i ∈R T×(C / / G)×H×W R is a matrix representing the dimension of the parameter tensor; T represents the time step, C is the total number of channels, G represents the number of groups, and H and W are the height and width of the feature map, respectively; by setting G << C, subgroup convolution can utilize the parallel computing capabilities of modern hardware to process across different groups simultaneously.

[0063] P=GN(σ(f PW (AvgPool H (X S )))⊙σ(f PW (AvgPool W (X S )))⊙X S )

[0064] Where: σ represents the Sigmoid function, ⊙ is the element-wise dot product, GN is the group normalization, and AvgPool is the pool normalization function. H AvgPool represents the global average pooling operation performed along the height dimension H. W P represents the global average pooling operation performed along the width dimension W; P is achieved by applying pointwise convolution f to the global average pooling results across the two spatial dimensions (H, W). PW The output is calculated from; where: AvgPool H (X S )∈R T×(C / / G)×1×W and AvgPool H (X S )∈R T×(C / / G)×H×1 These represent global average pooling operations along the height and width dimensions, respectively; X S ∈R T×(C / / G)×H×W Indicates the input feature map;

[0065] In the 1×1 attention branch Att1×1 In this process, 1×1 convolution operations do not require dimensionality reduction, thus enabling channel feature integration and spatial attention computation;

[0066]

[0067] in: Represents matrix multiplication, AvgPool(P)∈R T×(C / / G)×1×1 Global average pooling of P is used to produce a compact feature map that captures the global context; in the 3×3 attention branch Att 3×3 In the middle, a 3×3 convolutional layer f is stacked. 3×3 It is used to capture multi-scale feature representations and local contextual information:

[0068]

[0069] Where: AvgPool(f 3×3 (X S The first part is an intermediate feature map after 3×3 convolution and subsequent pooling; the second part is an attention map. 1×1 and Att 3×3 The shapes are all R T×1×H×W This allows them to effectively capture spatial dependencies while maintaining inter-channel interaction;

[0070] Att S =σ(Att) 1×1 +Att 3×3 )

[0071] Fusion of multi-scale attention maps Att S The final output feature map O is obtained by summing the outputs of the 1×1 and 3×3 branches, performing sigmoid activation, and then multiplying it element-wise with the input feature map X. att :

[0072] O att =X⊙Att S .

[0073] Step 4: Input the feature map processed in Step 3 into the detection head to perform target detection and output the detection results.

[0074] Step four specifically involves: Using the last membrane potential of the neuron as input, a five-dimensional feature map T×B×C×H×W is obtained from the feature map obtained in step three. By averaging the T dimension, a four-dimensional feature map B×C×H×W is obtained. Here, T represents the time step, B is the batch size, C is the total number of channels, and H and W represent the height and width of the feature map, respectively. Then, multiple anchor boxes of different scales are used for prediction. For each anchor box, the matching degree with the target needs to be calculated, and the target's category and location need to be predicted. After predicting the category and location of each anchor box, a non-maximum suppression (NMS) algorithm is used to remove overlapping detection results, retaining only the result with the highest confidence. Finally, the bounding box and category of the target are derived based on the prediction results, and the detection results are output.

[0075] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for UAV target detection based on a spiking neural network, characterized in that, Includes the following steps: Step 1: Acquire the image of the UAV to be inspected and perform pulse coding on it; Step 2: Input the pulse signal into the pulse residual backbone network composed of pulse residual basic blocks, perform feature extraction, and obtain feature maps of different scales and levels; Step 3: Input the feature map obtained in Step 2 into the pulse multi-scale attention mechanism for processing; Step three specifically involves: The feature maps obtained in step two are fed into the pulsed multi-scale attention mechanism for feature enhancement. The pulsed multi-scale attention mechanism uses feature grouping to divide the features into multiple sets across different resources, ensuring that each group is well distributed in space. Then, the feature maps of these groups are processed in parallel through 1×1 and 3×3 attention branches. X=[X0,X1,X i ,…,X G ],X i ∈R T×(C / / G)×H×W Where: the input feature tensor X is represented as X=[X0,X1,X2] i ,…,X G ], where: X i For the i-th subgroup feature, satisfying X i ∈R T×(C / / G)×H×W R is a matrix representing the dimension of the parameter tensor; T represents the time step, C is the total number of channels, G represents the number of groups, and H and W are the height and width of the feature map, respectively; by setting G << C, subgroup convolution can utilize the parallel computing capabilities of modern hardware to process across different groups simultaneously. P=GN(σ(f PW (AvgPool H (X S )))⊙σ(f PW (AvgPool W (X S )))⊙X S ) Where: σ represents the Sigmoid function, ⊙ is the element-wise dot product, GN is the group normalization, and AvgPool is the pool normalization function. H AvgPool represents the global average pooling operation performed along the height dimension H. W P represents the global average pooling operation performed along the width dimension W; P is achieved by applying pointwise convolution f to the global average pooling results across the two spatial dimensions (H, W). PW The output is calculated from; where: AvgPool H (X S )∈R T×(C / / G)×1×W and AvgPool H (X S )∈R T×(C / / G)×H×1 These represent global average pooling operations along the height and width dimensions, respectively; X S ∈R T×(C / / G)×H×W Indicates the input feature map; In the 1×1 attention branch Att 1×1 In this process, 1×1 convolution operations do not require dimensionality reduction, thus enabling channel feature integration and spatial attention computation; in: Represents matrix multiplication, AvgPool(P)∈R T×(C / / G)×1×1 Global average pooling of P is used to produce a compact feature map that captures the global context; in the 3×3 attention branch Att 3×3 In the middle, a 3×3 convolutional layer f is stacked. 3 ×3 It is used to capture multi-scale feature representations and local contextual information: Where: AvgPool(f 3×3 (X S The first part is an intermediate feature map after 3×3 convolution and subsequent pooling; the second part is an attention map. 1×1 and Att 3×3 The shapes are all R T×1×H×W , To S =σ(To 1×1 +To 3×3 ) Fusion of multi-scale attention maps Att S The final output feature map O is obtained by summing the outputs of the 1×1 and 3×3 branches, performing sigmoid activation, and then multiplying it element-wise with the input feature map X. att : THE att =X⊙Att S ; Step 4: Input the feature map processed in Step 3 into the detection head to perform target detection and output the detection results.

2. The UAV target detection method based on spiking neural networks according to claim 1, characterized in that, The pulsed residual basic block includes a residual path and a short-circuit path; the pulsed basic block includes LIF neurons, convolutional Conv, and batch normalized BN. The LIF neuron converts the floating-point inputs of the neuron at different time steps into binary pulse representations of 0 or 1 through nonlinear operations; Convolutional layers extract spatial features, while BN layers stabilize the training process by normalizing the output. O block =BN(Conv(LIF(X block ))) Where: X block Floating-point input represents the input, O block The output represents the pulse basic block; in the residual path, two pulse basic blocks are used consecutively, while in the shortcut path, a max pooling layer is applied for downsampling to reduce parameters; then, the pulse-based output from the shortcut path is connected to the output of the residual path after passing through another pulse basic block.

3. The UAV target detection method based on spiking neural networks according to claim 1, characterized in that, Step one specifically involves: The input UAV image to be detected is pulse encoded to obtain an image pulse signal; the input image to be detected is a visible light image or an infrared image, and the format of the image to be detected is RGB.

4. The UAV target detection method based on spiking neural networks according to claim 1, characterized in that, Step two specifically involves: The pulse residual backbone network consists of pulse residual basic blocks, containing N layers of pulse basic blocks, where N is an integer greater than 2; feature maps of different scales and hierarchical structures are extracted from the image pulse signal; the time step of the pulse residual basic block is 2.

5. The UAV target detection method based on spiking neural networks according to claim 1, characterized in that, Step four specifically involves: In the feature map obtained in step three, the last membrane potential of the neuron is used as input, resulting in a five-dimensional feature map T×B×C×H×W. By averaging the T dimension, a four-dimensional feature map B×C×H×W is obtained; where: T represents the time step, B is the batch size, C is the total number of channels, and H and W represent the height and width of the feature map, respectively. Then, multiple anchor boxes of different scales are used for prediction. The matching degree between each anchor box and the target needs to be calculated, and the target's category and location are predicted. After predicting the category and location of each anchor box, the non-maximum suppression algorithm (NMS) is used to remove overlapping detection results, retaining only the results with the highest confidence. Finally, the bounding box and category of the target are obtained based on the prediction results, and the detection results are output.

Images