Single target short-range tracking method and device based on deep spiking neural network

Abstract

The present invention provides a single-target short-range tracking method and device based on a deep spiking neural network. The method obtains a monitoring image set from a monitoring device, preprocesses the monitoring image set, and inputs the preprocessed monitoring image set into an initialized deep spiking neural network. However, the spiking neuron function is non-differentiable and cannot be directly trained on the spiking neural network. Therefore, a differentiable softsign function is introduced during backpropagation to replace the spiking neuron function, train the deep spiking neural network, and use the trained deep spiking neural network to track the target. Compared with the existing technology, it can achieve a tracking effect similar to that of the original convolutional neural network, while allowing the network to exhibit an event-driven computing method, with the advantages of high computational efficiency and low power consumption. It also provides feasibility for subsequent model transplantation to edge devices such as artificial intelligence chips or neuromorphic hardware.

Description

Technical Field

[0001] The present invention belongs to the field of deep spiking neural networks, and specifically relates to a single-target short-range tracking method and device based on deep spiking neural networks. Background Art

[0002] In recent years, deep learning has developed rapidly, and some scholars have tried to introduce deep learning into target tracking. For example, the tracking method based on pre-trained deep features, the main idea of which is to use the high-performance Convolutional Neural Networks (CNN) to perform feature extraction. Another part of target tracking based on offline training features is to train features that match the target tracking task in an end-to-end manner to achieve better tracking performance. Another method is to directly design a lightweight network to learn the features of the tracking target, which has high accuracy but very slow speed. Later, some scholars applied the Twin Fully Convolutional Siamese Networks (SiamFC) to target tracking. Using the Twin network structure, a similarity model was trained offline on the Generic Object Tracking Benchmark over 10,000 Video Segments (GOT-10K) dataset, and used this model for target tracking.

[0003] Traditional target tracking methods based on convolutional neural networks (CNNs) suffer from high computational complexity, high resource usage, and reliance on parallel accelerated computing devices, making them difficult to apply to resource-constrained edge devices such as AI chips. Spiking neural networks, as a "third-generation neural network," offer advantages such as high computational efficiency, low energy consumption, minimal resource usage, and ease of hardware implementation. However, due to the complex dynamics of spiking neurons and the non-differentiable nature of their pulses, SNN training algorithms are immature and inefficient, making their practical application in real-world applications difficult. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a single target short-range tracking method and device based on a deep spiking neural network. The technical problems to be solved by the present invention are achieved through the following technical solutions:

[0005] In a first aspect, the present invention provides a single target short-range tracking method based on a deep spiking neural network, comprising:

[0006] Acquire a monitoring image set from a monitoring device;

[0007] The monitoring image set includes a plurality of monitoring images with target locations marked;

[0008] Initialize the constructed deep spiking neural network;

[0009] Preprocessing the monitoring image set and inputting the preprocessed monitoring image set into the initialized deep spiking neural network, introducing a differentiable softsign function to replace the step function, and adjusting the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and updating the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network;

[0010] The trained deep spiking neural network is used to track the target in the current monitoring image obtained from the monitoring device.

[0011] Optionally, the constructed deep spiking neural network includes: a SpikingAlexNet network and a prediction network, the SpikingAlexNet network includes a pulse encoder and a pulse feature extractor; the pulse encoder is composed of a convolution layer and a connected pulse neuron layer, the convolution layer of the pulse encoder is used to perform convolution calculations on the preprocessed monitoring images of the three channels, and encode the calculation results into square waves and input them into the pulse neuron layer, the pulse neuron layer is used to output a pulse sequence formed according to the square wave;

[0012] The pulse feature extractor is composed of four convolutional pulse neuron layers and two pooling layers, which are used to extract features from the pulse sequence output by the pulse encoder;

[0013] The prediction network consists of a convolutional layer and a tracker, and is used to predict the target position in the monitoring image based on the feature extraction results.

[0014] Optionally, the monitoring image set is preprocessed, and the preprocessed monitoring image set is input into the initialized deep spiking neural network, the spiking neuron replaces the activation function in the CNN, and a differentiable softsign function is introduced to replace the step function, and the parameters of the deep spiking neural network are adjusted at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and the parameter values are updated until the training cutoff condition is reached, and the trained deep spiking neural network is obtained, including:

[0015] Preprocessing the surveillance image set;

[0016] At the current iteration, the preprocessed monitoring image set is input into the initialized pulse encoder, and a differentiable softsign function is introduced to replace the step function, so that the pulse encoder outputs a pulse sequence to the pulse feature extractor. The pulse feature extractor extracts the pulse features of the pulse sequence and inputs the features into the prediction network to output the target position on each monitoring image.

[0017] For each surveillance image, the target position predicted by the surveillance image is compared with the marked target position and the loss function is calculated;

[0018] At each iteration, the optimization direction of the parameters of the deep spiking neural network is adjusted according to the direction of the decrease of the loss function to perform backpropagation training on the deep spiking neural network, and the parameter values are updated until the deep spiking neural network converges to obtain a trained deep spiking neural network.

[0019] Optionally, the using a trained deep spiking neural network to track a target in a current monitoring image obtained from a monitoring device includes:

[0020] Obtain the current monitoring image from the monitoring device;

[0021] Using the first frame of the current monitoring image as a template frame and the subsequent frames as the current frame;

[0022] The template frame and the current frame are respectively fed into the deep spiking neural network so that the SpikingAlexNet network performs pulse encoding and feature extraction to obtain feature matrices respectively;

[0023] The feature matrices of the template frame and the current frame are input into the prediction network, so that the convolution layer of the prediction network convolves the two to obtain a score matrix, which is then input into the tracker.

[0024] The tracker locates the target in the current frame according to the score matrix and completes target tracking.

[0025] Optionally, inputting the feature matrices of the template frame and the current frame into the prediction network so that the convolution layer of the prediction network convolves the two to obtain a score matrix, and inputting the score matrix into the tracker includes:

[0026] Input the feature matrices of the template frame and the current frame into the prediction network so that the prediction network retains the time dimension and performs convolution operation on the two to obtain the score pulse matrix;

[0027] The score pulse matrix is averaged in the time dimension and normalized to obtain a score matrix which is input into the tracker.

[0028] Optionally, the tracker locates the position of the target in the current frame according to the score matrix, and completing target tracking includes:

[0029] The tracker performs bicubic linear interpolation restoration on the score matrix to locate the position of the tracked target in the current frame and complete target tracking.

[0030] Optionally, the alternative softsign function is

[0031]

[0032] Among them, x is the independent variable, and α is the parameter that controls the smoothness of the gradient during back propagation.

[0033] Optionally, before using the trained deep spiking neural network to track the target on the current monitoring image obtained from the monitoring device, the single target short-range tracking method further includes:

[0034] Use the test set to test the trained deep spiking neural network and obtain the predicted box of the tested target;

[0035] Calculate the ratio of the intersection and union between the predicted box and the true labeled box of the labeled target location to confirm whether the trained deep spike neural network meets the conditions.

[0036] Optionally, the preprocessing includes center cropping or random cropping.

[0037] In a second aspect, the present invention provides a single target short-range tracking device based on a deep pulse neural network, comprising:

[0038] An acquisition module is used to acquire a monitoring image set from a monitoring device;

[0039] The monitoring image set includes a plurality of monitoring images with target locations marked;

[0040] Initialization module, used to initialize the constructed deep pulse neural network;

[0041] A training module is used to preprocess the monitoring image set and input the preprocessed monitoring image set into the initialized deep spiking neural network, introduce a differentiable softsign function to replace the step function, and adjust the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and update the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network;

[0042] The tracking module is used to track the target in the current monitoring image obtained from the monitoring device using the trained deep pulse neural network.

[0043] Beneficial effects of the present invention:

[0044] The present invention provides a single-target short-range tracking method and device based on a deep spiking neural network. The method obtains a monitoring image set from a monitoring device, preprocesses the monitoring image set, and inputs the preprocessed monitoring image set into an initialized deep spiking neural network. However, the spiking neuron function is non-differentiable and cannot be directly trained. Therefore, a differentiable softsign function is introduced to replace the spiking neuron function, and the deep spiking neural network is trained. The trained deep spiking neural network is then used to track the target. Compared to existing technologies, this method can achieve tracking effects similar to those of the original convolutional neural network, while allowing the network to exhibit an event-driven computing method, with the advantages of high computational efficiency and low power consumption. It also provides feasibility for subsequent model transplantation to edge devices such as artificial intelligence chips or neuromorphic hardware.

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a single target short-range tracking method based on a deep spiking neural network provided by an embodiment of the present invention;

[0047] Figure 2 This is a diagram of the SpikingAlexNet network structure provided by an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the step function input and output of a pulse neuron in the prior art;

[0049] Figure 4 is a schematic diagram of a deep spiking neural network provided by an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the positions of the prediction box and the annotation box provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0052] like Figure 1 As shown, the present invention provides a single target short-range tracking method based on a deep pulse neural network, which includes:

[0053] S1, obtains a monitoring image set from a monitoring device;

[0054] The monitoring image set includes a plurality of monitoring images with target locations marked;

[0055] S2, initialize the constructed deep spiking neural network;

[0056] The constructed deep pulse neural network includes: a SpikingAlexNet network and a prediction network. The SpikingAlexNet network includes a pulse encoder and a pulse feature extractor. The pulse encoder consists of a convolutional layer and a connected pulse neuron layer. The convolutional layer of the pulse encoder is used to perform convolution calculations on the preprocessed three-channel monitoring images, and encode the calculation results into square waves and input them into the pulse neuron layer. The pulse neuron layer is used to output a pulse sequence formed according to the square wave. The pulse feature extractor is composed of four convolutional pulse neuron layers and two pooling layers, which are used to extract features from the pulse sequence output by the pulse encoder. The prediction network is composed of a convolutional layer and a tracker, which is used to predict the target position in the monitoring image based on the feature extraction results.

[0057] refer to Figure 2 , Figure 2 The SpikingAlexNet network structure diagram. The left cube represents the input image, the first rectangle represents the convolution result, the second rectangle represents the LIF neuron, K represents the convolution kernel size, S represents the stride, Conv represents the convolution layer, and MaxPool represents the pooling layer.

[0058] It is worth noting that before initializing the deep spiking neural network, the present invention needs to initialize system parameters during actual operation. The initialization of parameters mainly includes the dataset path, device used, observation time, training cycle, and output path. The initialization process is as follows:

[0059] Set the input image path for the dataset. The training dataset used in this paper is GOT10K, and the test dataset is OTB100. If the host machine has a GPU, you can choose to use the GPU for training and testing. The default is to use the CPU for image target tracking testing. Set the observation time T (default T = 4) and the training period Epochs (default Epochs = 32). Set the save path for this test image result. The default is to save it to the logs folder under the local project.

[0060] S3, preprocessing the monitoring image set, and inputting the preprocessed monitoring image set into the initialized deep spiking neural network, introducing a differentiable softsign function to replace the step function, and adjusting the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and updating the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network;

[0061] The preprocessing includes center cropping or random cropping.

[0062] During forward propagation, spiking neurons use a step function, and the output value is discrete 0 or 1. However, during the backward propagation process, due to the non-differentiable nature of spiking neurons, it is impossible to use backpropagation to train spiking neural networks. The present invention uses the softsign function (Formula 1) instead of the step function, and uses the derivative of the softsign function (Formula 2) to complete the gradient calculation.

[0063] The softsign function is

[0064]

[0065]

[0066] Among them, x is the independent variable, and α is the parameter that controls the smoothness of the gradient during back propagation.

[0067] As an optional embodiment of the present invention, step S3 includes:

[0068] S31, preprocessing the monitoring image set;

[0069] S32, at the current iteration, input the preprocessed monitoring image set into the initialized pulse encoder, use pulse neurons to replace the activation function in the CNN, and introduce a differentiable softsign function to replace the step function, so that the pulse encoder outputs a pulse sequence to the pulse feature extractor. The pulse feature extractor extracts the pulse feature of the pulse sequence and inputs the feature into the prediction network to output the target position on each monitoring image;

[0070] During the training process, the training dataset is loaded and the defined image preprocessing method is used. It is then input into SpikingAlexNet to calculate the feature map and perform cross-correlation operations:

[0071]

[0072] in, represents the target image feature map, Represents the search image feature map, z and x represent the target image and search image respectively.

[0073] S33, for each monitoring image, comparing the target position predicted by the monitoring image with the marked target position, and calculating a loss function;

[0074] The loss function is calculated as:

[0075] L(y,v)=log(1+exp(-yv))

[0076] Among them, v is the score value of a sample candidate area, and y is the true label y∈{+1,-1} of this sample candidate area.

[0077] S34, adjusting the optimization direction of the parameters of the deep spiking neural network according to the direction of the decrease of the loss function at each iteration, so as to perform backpropagation training on the deep spiking neural network, and updating the parameter values until the deep spiking neural network converges to obtain a trained deep spiking neural network.

[0078] During each iteration, the gradient is reset to zero, the parameter gradient value is calculated through back propagation, and the parameters are updated; the weight of the current training cycle whose final loss function is less than the preset threshold is saved.

[0079] S4, using the trained deep pulse neural network, tracks the target in the current monitoring image obtained from the monitoring device.

[0080] As an optional embodiment of the present invention, the method of tracking a target in a current monitoring image obtained from a monitoring device using a trained deep spiking neural network includes:

[0081] S41, obtaining the current monitoring image from the monitoring device;

[0082] S42, using the first frame of the current monitoring image as a template frame and the subsequent frames as the current frame;

[0083] S43, respectively sending the template frame and the current frame into a deep spiking neural network, so that the SpikingAlexNet network performs pulse encoding and feature extraction, and obtains feature matrices respectively;

[0084] It's worth noting that the deep spiking neural network of the present invention adds a pulse feature extraction network to a CNN network, and designs spiking neurons to convert surveillance images into discrete binary sequences through encoding, which then transmits and calculates information. Compared to existing floating-point transmission, the present invention has faster computing speeds.

[0085] S44, inputting the feature matrices of the template frame and the current frame into the prediction network, so that the convolution layer of the prediction network convolves the two to obtain a score matrix, and inputs it into the tracker;

[0086] S45, the tracker locates the position of the target in the current frame according to the score matrix, and completes target tracking.

[0087] The tracker in the present invention performs bicubic linear interpolation restoration on the score matrix, locates the position of the tracking target in the current frame, and completes target tracking.

[0088] As an optional embodiment of the present invention, the step of inputting the feature matrices of the template frame and the current frame into the prediction network so that the convolution layer of the prediction network convolves the two to obtain a score matrix, and inputting the score matrix into the tracker includes:

[0089] S441, inputting the feature matrices of the template frame and the current frame into the prediction network, so that the prediction network retains the time dimension and performs a convolution operation on the two to obtain a score pulse matrix;

[0090] S442 , averaging the score pulse matrix in the time dimension and normalizing it to obtain a score matrix which is input into the tracker.

[0091] As an optional embodiment of the present invention, before using the trained deep spiking neural network to track the target in the current monitoring image obtained from the monitoring device, the single target short-range tracking method further includes:

[0092] Step a: Use the test set to test the trained deep spiking neural network and obtain the predicted box of the tested target;

[0093] Step b: Calculate the ratio of the intersection between the predicted box and the true labeled box of the labeled target location, and the union to confirm whether the trained deep spike neural network meets the conditions.

[0094] The present invention uses the trained SpikingAlexNet to extract features from the template frame and the current frame respectively, and then realizes target tracking through cross-correlation operation. The first frame of the image sequence of the test dataset is used as the template frame. The template frame of the image sequence and the current frame are respectively fed into the SpikingAlexNet network for pulse coding and feature extraction to obtain a feature matrix. The extracted feature matrix of the template frame and the current frame is convolved to obtain a score matrix. The groundtruth given in the first frame is the target position. The position of the tracked target in the current frame is located according to the score matrix to complete the target tracking test.

[0095] The present invention provides a single target short-range tracking device based on a deep pulse neural network, comprising:

[0096] An acquisition module is used to acquire a monitoring image set from a monitoring device;

[0097] The monitoring image set includes a plurality of monitoring images with target locations marked;

[0098] Initialization module, used to initialize the constructed deep pulse neural network;

[0099] A training module is used to preprocess the monitoring image set and input the preprocessed monitoring image set into the initialized deep spiking neural network, introduce a derivative alternative softsign function to replace the step function, and adjust the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and update the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network;

[0100] The tracking module is used to track the target in the current monitoring image obtained from the monitoring device using the trained deep pulse neural network.

[0101] The present invention provides a single-target short-range tracking method and device based on a deep spiking neural network. The method obtains a monitoring image set from a monitoring device, preprocesses the monitoring image set, and inputs the preprocessed monitoring image set into an initialized deep spiking neural network. However, spiking neurons are non-differentiable and cannot be directly trained on the spiking neural network. Therefore, a differentiable softsign function is introduced to replace the step function to train the deep spiking neural network, and the trained deep spiking neural network is used to track the target. Compared with the existing technology, it can achieve a tracking effect similar to that of the original convolutional neural network, while allowing the network to exhibit an event-driven computing method, with the advantages of high computational efficiency and low power consumption. It also provides feasibility for subsequent model transplantation to edge devices such as artificial intelligence chips or neuromorphic hardware.

[0102] The tracking process of the present invention is described below through actual simulation.

[0103] Step 1: Initialize the parameters. In the computer simulation, set the dataset path to / home / chaos / datasets / GOT10K, use device = GPU, observation time T = 6, training period epochs = 300, and output path to / home / chaos / spiking-siam-fc / logs / .

[0104] Step 2: Initialize the feature extraction network

[0105] Step 3: Train the Spiking AlexNet neural network. The specific process is as follows:

[0106] refer to Figure 3During forward propagation, spiking neurons use a step function, and the output value is discrete 0 or 1. However, during backward propagation, due to the non-differentiable nature of spiking neurons, it is impossible to use backpropagation to train spiking neural networks. The present invention introduces a softsign function to replace the spiking neuron function during backward propagation, and uses the derivative of the softsign function to complete the gradient calculation.

[0107] The deep spiking neural network is shown in Figure 4, and the parameters of each layer of SpikingAlexNet are shown in Table 1. After preprocessing, encoder and spiking feature extractor, the template box and the predicted box are then similarity calculated.

[0108] Table 1 Parameters of each layer of SpikingAlexNet.

[0109]

[0110]

[0111] The hardware configuration of the training network in this patent is as follows: the CPU is two Intel (R) Xeon (R) CPU E5-2620v4, running at a frequency of 2.10GHz, the memory capacity is 64GB, the frequency is 2400MHz, and the GPU uses two Nvidia RTX 2070 Super, with a video memory of 8GB x 2. The software environment of the experiment in this invention is implemented in Python language, interpreter version 3.9, and deep learning framework PyTorch, version 1.8.0.

[0112] Step 4: After training, use the saved weights to test on the OTB100 dataset, that is, use the trained SpikingAlexNet to extract features from the template frame and the current frame respectively, and then achieve target tracking through cross-correlation operation. The specific process is as follows:

[0113] 4.1 The template frame uses a 127×127 area as input, and the current frame candidate area uses a 255×255 area as input;

[0114] 4.2 Pulse coding is performed on the target area of the template frame and the candidate area of the current frame, using convolutional coding with learnable parameters;

[0115] 4.3 The encoded inputs are passed into the SpikingAlexNet network for feature extraction. The observation time is set to T milliseconds. For the target input in the template frame, a T×6×6×256 pulse feature map matrix is obtained. For the input of the candidate region of the current frame, a T×22×22×256 pulse feature map matrix is obtained.

[0116] 4.4 For the two pulse feature map matrices after feature extraction, retain the time dimension and perform convolution operation to obtain a T×17×17 score pulse matrix;

[0117] 4.5 Average the score pulse matrix in the time dimension, that is, calculate the firing rate of the pulse sequence, to obtain a 17 × 17 matrix. After normalizing this matrix, we can obtain the score matrix;

[0118] 4.6 Perform bicubic linear interpolation on the score matrix to restore the position of the tracked target in the current frame and complete target tracking. Using the Car24 sequence in the OTB100 dataset as an example, the tracking effect is shown in Figure 5, where the dark box is the visualization result of the network tracking box, and the light box is the visualization result of the ground truth position annotated in the dataset. The upper left corner shows the sequence number of the image in the video sequence, and the lower right corner shows the Intersection over Union (IoU) between the tracked box and the annotated box in the current frame. IoU is defined as: calculating the intersection between the predicted box and the true annotated box (the number of pixels in the intersection), and then dividing by the union of the two (the number of pixels in the union).

[0119] from Figure 5 It can be concluded that the prediction results of the present invention are very close to the actual annotation results, so efficient and accurate target tracking can be achieved.

Claims

1. A single target short-range tracking method based on deep spiking neural network, characterized in that: include: Acquire a monitoring image set from a monitoring device; The monitoring image set includes a plurality of monitoring images with target locations marked; Initialize the constructed deep spiking neural network; Preprocessing the monitoring image set and inputting the preprocessed monitoring image set into the initialized deep spiking neural network, introducing a differentiable softsign function to replace the step function, and adjusting the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and updating the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network; Use the trained deep pulse neural network to track the target in the current monitoring image obtained from the monitoring device; replace the softsign function with Among them, x is the independent variable, and α is the parameter that controls the smoothness of the gradient during back propagation.

2. The single target short-range tracking method based on deep pulse neural network according to claim 1 is characterized in that: The constructed deep spiking neural network includes: a SpikingAlexNet network and a prediction network, wherein the SpikingAlexNet network includes a pulse encoder and a pulse feature extractor; the pulse encoder is composed of a convolutional layer and a connected spiking neuron layer, the convolutional layer of the pulse encoder is used to perform convolution calculations on the preprocessed monitoring images of the three channels, and the calculation results are encoded into square waves and input into the spiking neuron layer, and the spiking neuron layer is used to output a pulse sequence formed according to the square wave; The pulse feature extractor is composed of four convolutional pulse neuron layers and two pooling layers, which are used to extract features from the pulse sequence output by the pulse encoder; The prediction network consists of a convolutional layer and a tracker, and is used to predict the target position in the monitoring image based on the feature extraction results.

3. The single target short-range tracking method based on deep pulse neural network according to claim 2 is characterized in that: The monitoring image set is preprocessed, and the preprocessed monitoring image set is input into the initialized deep spiking neural network, the spiking neuron replaces the activation function in the CNN, and a differentiable softsign function is introduced to replace the step function, and the parameters of the deep spiking neural network are adjusted at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and the parameter values are updated until the training cutoff condition is reached, and the trained deep spiking neural network is obtained, including: Preprocessing the surveillance image set; At the current iteration, the preprocessed monitoring image set is input into the initialized pulse encoder, and a differentiable softsign function is introduced to replace the step function, so that the pulse encoder outputs a pulse sequence to the pulse feature extractor. The pulse feature extractor extracts the pulse features of the pulse sequence and inputs the features into the prediction network to output the target position on each monitoring image. For each surveillance image, the target position predicted by the surveillance image is compared with the marked target position and the loss function is calculated; At each iteration, the optimization direction of the parameters of the deep spiking neural network is adjusted according to the direction of the decrease of the loss function to perform backpropagation training on the deep spiking neural network, and the parameter values are updated until the deep spiking neural network converges to obtain a trained deep spiking neural network.

4. The single target short-range tracking method based on deep pulse neural network according to claim 3 is characterized in that: The method of tracking a target on a current monitoring image obtained from a monitoring device by using a trained deep pulse neural network includes: Obtain the current monitoring image from the monitoring device; Using the first frame of the current monitoring image as a template frame and the subsequent frames as the current frame; The template frame and the current frame are respectively fed into the deep spiking neural network so that the SpikingAlexNet network performs pulse encoding and feature extraction to obtain feature matrices respectively; The feature matrices of the template frame and the current frame are input into the prediction network, so that the convolution layer of the prediction network convolves the two to obtain a score matrix, which is then input into the tracker. The tracker locates the target in the current frame according to the score matrix and completes target tracking.

5. The single target short-range tracking method based on deep pulse neural network according to claim 4 is characterized in that: The step of inputting the feature matrices of the template frame and the current frame into the prediction network so that the convolution layer of the prediction network convolves the two to obtain a score matrix, and inputting the score matrix into the tracker includes: Input the feature matrices of the template frame and the current frame into the prediction network so that the prediction network retains the time dimension and performs convolution operation on the two to obtain the score pulse matrix; The score pulse matrix is averaged in the time dimension and normalized to obtain a score matrix which is input into the tracker.

6. The single target short-range tracking method based on deep pulse neural network according to claim 4 is characterized in that: The tracker locates the position of the target in the current frame according to the score matrix, and completes target tracking including: The tracker performs bicubic linear interpolation restoration on the score matrix to locate the position of the tracked target in the current frame and complete target tracking.

7. The single target short-range tracking method based on deep spiking neural network according to claim 1, characterized in that: Before using the trained deep spiking neural network to track the target in the current monitoring image obtained from the monitoring device, the single target short-range tracking method further includes: Use the test set to test the trained deep spiking neural network and obtain the predicted box of the tested target; Calculate the ratio of the intersection and union between the predicted box and the true labeled box of the labeled target location to confirm whether the trained deep spike neural network meets the conditions.

8. The single target short-range tracking method based on deep spiking neural network according to claim 1, characterized in that: The pre-processing includes center cropping or random cropping.

9. A single target short-range tracking device based on deep pulse neural network, characterized in that: include: An acquisition module is used to acquire a monitoring image set from a monitoring device; The monitoring image set includes a plurality of monitoring images with target locations marked; Initialization module, used to initialize the constructed deep pulse neural network; A training module is used to preprocess the monitoring image set and input the preprocessed monitoring image set into the initialized deep spiking neural network, introduce a differentiable softsign function to replace the step function, and adjust the parameters of the deep spiking neural network at each iteration to perform backpropagation training on the deep spiking neural network in the direction of decreasing parameter gradient values, and update the parameter values until the training cutoff condition is reached, thereby obtaining a trained deep spiking neural network; The tracking module is used to track the target in the current monitoring image obtained from the monitoring device using the trained deep pulse neural network; the softsign function is replaced by Among them, x is the independent variable, and α is the parameter that controls the smoothness of the gradient during back propagation.

Images