Pulse neural network dynamic target detection method based on local-global learning

Abstract

The present disclosure provides a local-global learning-based spiking neural network dynamic target detection method, comprising: establishing a neuromorphic standard dataset and a neuromorphic small sample dataset; constructing a multi-scale attention feature fusion deep spiking neural network as a teacher network, and training the teacher network using the neuromorphic standard dataset, so that the teacher network learns global knowledge; constructing a shallow spiking neural network as a student network, and guiding the student network to train through knowledge distillation using the neuromorphic small sample dataset and the trained teacher network, so that the student network learns local knowledge; and using the trained student network to perform dynamic target detection. The present disclosure designs a knowledge distillation method suitable for deep spiking neural networks for small sample complex real environment, guides and optimizes the student network, and uses the student network as a model for dynamic target detection in a real environment to quickly identify dynamic targets.

Description

Technical Field

[0001] This disclosure relates to the field of target recognition technology, and in particular to a dynamic target detection method, apparatus, storage medium, and electronic device based on a spiking neural network with local-global learning. Background Technology

[0002] The ability of artificial intelligence (AI) to rapidly learn from limited data and adapt to constantly changing real-world environments is essential. The human brain can self-organize and coordinate different cognitive functions to flexibly adapt to complex and ever-changing realities, but current AI cannot fully simulate this ability. A major challenge in AI and computational neuroscience is integrating multi-scale biological principles to build biologically plausible brain-like intelligence models. Spike Neural Networks (SNNs), as third-generation neural networks, are more biologically accurate across multiple scales. Due to their strong temporal information processing capabilities, low power consumption, and high biological reliability, SNNs are naturally more suitable for modeling various cognitive functions of the brain and constructing brain-like intelligence models. With the development of AI technology, various neural simulators provide infrastructure support platforms for simulating neurodynamic models, large-scale neural networks, and deep spiking neural networks, making it possible to build brain-like AI based on spiking neural networks that can rapidly learn from limited data and flexibly adapt to constantly changing realities.

[0003] Traditional spiking neural networks (SNNs) lag behind artificial neural networks in visual recognition performance when faced with changing real-world environments and limited sample sizes. Inspired by the function of biological neurons, SNNs use discrete binary pulse sequences to transmit information between neurons. However, due to the non-differentiable nature of these pulse sequences, feedback propagation optimization is difficult to directly apply to training SNNs, significantly limiting their development. To address this issue, researchers have proposed various SNN optimization algorithms, including unsupervised learning, supervised learning, reward-penalty learning, and transformational learning methods. The method of converting artificial neural networks to spiking neural networks achieves nearly 10% lower accuracy on ImageNet-1k compared to artificial neural networks, with time steps reaching hundreds of steps. Directly training spiking convolutional neural networks achieves a 5%-8% accuracy loss within 10 time steps. Although spiking neural networks are more energy-efficient than artificial neural networks, their performance on visual datasets remains unsatisfactory.

[0004] Most deep learning models rely on large amounts of labeled data to tune parameters, which differs significantly from the rapid learning of the human brain, which relies on unsupervised local learning rules. Synaptic plasticity of neurons is the neurological basis of brain learning and memory. Spike Timing Dependent Plasticity (STDP) is a common learning rule in multiple brain regions, playing a crucial role in perception and learning. Many researchers have built spiking neural networks based on STDP training. Some researchers have experimented with a simplified unsupervised method of STDP to create a two-layer fully connected spiking neural network. Others have used an unsupervised STDP method with two layers of activation and inhibition, along with a hand-designed DoG filter to extract features and then trained convolutional layers with STDP. Some methods combine STDP with backpropagation for hybrid training, first performing STDP training to obtain weights with generalization capabilities, and then performing supervised backpropagation training for better performance. Spiking neural networks trained on STDP still perform poorly because, compared to the backpropagation algorithm, the local optimization rules lack global guidance errors, leading to a lack of coordination and self-organization within the model and between layers. Different parameter settings can easily lead to disordered pulses, making it difficult to transmit important information. Therefore, in order to enable the network to fully express the semantic features in the image, how to efficiently represent the discriminative features of dynamic target images under conditions of limited data samples has become an urgent research problem. Summary of the Invention

[0005] The purpose of this disclosure is to provide a dynamic target detection method, device, storage medium, and electronic device based on a spiking neural network using local-global learning, in order to solve the problem that the prior art cannot achieve efficient automatic monitoring of dynamic targets in real-world environments with a small number of samples.

[0006] The embodiments of this disclosure adopt the following technical solution: a dynamic target detection method based on local-global learning spiking neural networks, comprising: establishing a neuromorphic standard dataset and a neuromorphic few-shot dataset, wherein each type of data in the neuromorphic few-shot dataset is randomly selected from the corresponding type of data in the neuromorphic standard dataset according to a preset percentage; constructing a deep spiking neural network with multi-scale attention feature fusion as a teacher network, and training the teacher network using the neuromorphic standard dataset to enable the teacher network to learn global knowledge; constructing a shallow spiking neural network as a student network, and training the student network using the neuromorphic few-shot dataset and the trained teacher network through knowledge distillation to enable the student network to learn local knowledge; and using the trained student network for dynamic target detection.

[0007] Embodiments of this disclosure also provide a dynamic target detection device based on a spiking neural network using local-global learning, comprising: a dataset construction module for establishing a neuromorphic standard dataset and a neuromorphic few-shot dataset, wherein each type of data in the neuromorphic few-shot dataset is randomly selected from the corresponding type of data in the neuromorphic standard dataset according to a preset percentage; a teacher network training module for constructing a deep spiking neural network with multi-scale attention feature fusion as a teacher network, and training the teacher network using the neuromorphic standard dataset to enable the teacher network to learn global knowledge; a student network training module for constructing a shallow spiking neural network as a student network, and training the student network using the neuromorphic few-shot dataset and the trained teacher network through knowledge distillation to enable the student network to learn local knowledge; and a target detection module for performing dynamic target detection using the trained student model.

[0008] Embodiments of this disclosure also provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described dynamic target detection method based on local-global learning spiking neural networks.

[0009] Embodiments of this disclosure also provide an electronic device, including at least a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-described dynamic target detection method based on local-global learning spiking neural networks when executing the computer program in the memory.

[0010] The beneficial effects of this disclosure are as follows: a neuromorphic few-shot dataset based on a standard neuromorphic dataset is constructed, and a deep teacher spiking neural network and a shallow student spiking neural network based on multi-scale attention feature fusion are used as the feature extraction backbone networks. For complex real-world environments with few samples, a knowledge distillation method suitable for deep spiking neural networks is designed to guide and optimize the student network, enabling it to achieve fast and high-precision model training based on the few-shot dataset. Finally, the student network is used as a model for dynamic target detection in real-world environments to quickly identify dynamic targets. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is a flowchart of the dynamic target detection method based on local-global learning spiking neural network in the first embodiment of this disclosure;

[0013] Figure 2 This is a schematic diagram of the architecture of the local-global spiking neural network in the first embodiment of this disclosure;

[0014] Figure 3 This is a schematic diagram of the structure of the dynamic target detection device based on local-global learning spiking neural network in the second embodiment of this disclosure. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0016] To address the problems mentioned in the background art, the first embodiment of this disclosure provides a dynamic target detection method based on a spiking neural network using local-global learning, the flowchart of which is shown below. Figure 1 As shown, it mainly includes steps S10 to S40:

[0017] S10, establish a standard neuromorphic dataset and a small sample neuromorphic dataset.

[0018] Neuromorphic datasets are the primary data type for spiking neural network (SNLM) applications. Most existing neuromorphic datasets are transformed datasets generated from static images captured by dynamic vision sensors. Currently, standard neuromorphic datasets such as N-MNIST, N-Caltech101, DVS-Gesture, CIFAR10, CIFAR100, Tiny-ImageNet, and ImageNet-1K are widely used in SNLM classification and recognition tasks. Meanwhile, to simulate few-shot learning environments and constantly changing real-world conditions, constructing neuromorphic few-shot datasets based on standard datasets is of great significance for learning and optimizing local-global learning SNLMs.

[0019] In this embodiment, the data of each type in the neuromorphic few-sample dataset are randomly selected from the corresponding type of data in the neuromorphic standard dataset according to a preset percentage. The value of this preset percentage can be set by the network recognition accuracy obtained from network evaluation, and this setting method will be described in detail below. In addition, in some embodiments, a dynamic target dataset should also be constructed to verify the trained network. This dynamic target dataset is a self-built neuromorphic dataset formed by collecting dynamic targets in a real target detection environment through a dynamic vision sensor. The aforementioned real target detection environment usually refers to complex environments such as highly dynamic, low-light, and high-dynamic environments. That is, the dynamic target data collected in the aforementioned complex environments is used as a verification of network performance.

[0020] S20. A deep spiking neural network with multi-scale attention feature fusion is constructed as the teacher network, and the teacher network is trained using a standard neuromorphic dataset to enable the teacher network to learn global knowledge.

[0021] The teacher network constructed in this embodiment is a deep spiking neural network with multi-scale attention feature fusion, and its network architecture diagram is shown below. Figure 2 As shown, the system mainly consists of a teacher initial network, multiple attention spiking network blocks, and a fully connected teacher classifier. During teacher network training, neuromorphic data is passed through the teacher initial network to extract initial features of the dynamic target. Then, the initial features are fed into the attention spiking network blocks to extract global knowledge. Finally, the teacher classifier is used to predict and classify the feature labels.

[0022] Specifically, the teacher's initial network consists of a single convolutional layer and a max-pooling connection, and the attention spiking network block includes at least an attention module (such as...). Figure 2 (block pattern representation in the attention pulse network block) and pulse coding module (such as...) Figure 2 The circular pattern in the attention spiking network block represents the pulse coding module, which consists of N convolutional blocks connected to pooling layers. Each convolutional block contains n convolutional layers, normalization layers, and neurons. The attention module includes at least a temporal attention module, a multi-scale channel attention module, and a spatial attention module. By simultaneously employing three dimensions of attention modules, multi-dimensional and multi-scale attention feature fusion can be achieved, thereby improving the performance of the teacher network and better guiding the student network.

[0023] In its implementation, the temporal attention mechanism is designed in a manner similar to the Squeeze-and-Excitation (SE) module, with the temporal attention module outputting features... Represented as:

[0024]

[0025] Here, TA represents the time attention mechanism. Let n represent the intermediate feature of the nth teacher in the network. The feature space scale is represented by t, and time is represented by t. H represents the weights of the time attention mechanism. n ×W n C represents the feature map size. n Indicates the number of channels;

[0026] Inspired by the Convolutional Block Attention Module (CBAM), this embodiment introduces both channel attention and spatial attention mechanisms. To alleviate the weak recognition performance caused by scale variations and small objects, a multi-scale channel attention mechanism is implemented by adding local information from the feature map to the global information by changing the spatial pooling size. The multi-scale channel attention module selects point-wise convolution (PWConv) as the local channel information aggregator, utilizing only the point-like channel interactions at each spatial location. The features output by the multi-scale channel attention module... Represented as:

[0027]

[0028] Wherein, CA represents the multi-scale channel attention mechanism, g c (H t,n ) represents the weights for multi-scale channel attention. Represented as global channel information weight, This is represented as the local channel information weight, where σ is the sigmoid function;

[0029] Features output by the spatial attention module Represented as:

[0030]

[0031] in, This is represented as the weight of the spatial attention mechanism.

[0032] Furthermore, in this embodiment, the teacher network performs feedback optimization instead of gradient descent, achieving global knowledge transfer through data supervision. Specifically, instead of gradient descent, it uses derivatives to define the derivative of the threshold-triggered ignition mechanism, allowing the spiking neural network to perform feedback optimization using gradient descent like an artificial neural network. This alternative to gradient descent can be applied to the teacher network to achieve supervised global knowledge transfer and guide the feature representation of shallow student networks, compensating for the shortcomings of STDP during training.

[0033] When training with the alternative gradient descent algorithm, the derivative of the network with respect to the weights is the total number of peaks that change over time, as shown in Equation (4); the derivative of the output loss with respect to the weights of connection layers l and l+1 is determined by multiplying the transpose error gradient at l+1 with the input peaks from layer l, as shown in Equation (5); finally, the calculated partial derivatives of the loss function are used to update the respective weights using the learning rate, as shown in Equation (6):

[0034]

[0035] Among them, c l (t) represents the peak value of the input, Δw l The weight δ represents the derivative. l+1 Let η represent the transpose error gradient at position l+1, E represent the output loss, and η represent the transpose error gradient at position l+1. BP The learning rate is used. Therefore, by iteratively updating the weights of mini-batch input patterns, the network state will reach a local minimum, enabling the network to capture multi-level internal features of the data.

[0036] S30. A shallow spiking neural network is constructed as the student network. The student network is trained by knowledge distillation using a neuromorphic few-shot dataset and a pre-trained teacher network, so that the student network can learn local knowledge.

[0037] In this embodiment, the student network is a shallow spiking neural network. It does not include an attention mechanism and has significantly fewer convolutional layers than the teacher network, achieving a lightweight design. In some embodiments, the teacher and student networks are from the same source network, such as the teacher network being ResNet152 and the student network being ResNet18, or the teacher network being VGG19 and the student network being VGG11. Such source networks have the same number of modules, allowing for a one-to-one correspondence and facilitating subsequent knowledge distillation.

[0038] Figure 2 The diagram also shows the architecture of the student network, which mainly includes a student initial network, multiple spiking network blocks, and a student classifier. Similar to the teacher network, the student initial network is also used to extract the initial features of the dynamic target from the neuromorphic few-sample dataset. Then, the spiking network blocks are used to extract local knowledge. At the same time, the student network is guided by the teacher network through knowledge distillation to supervise and guide local knowledge with global knowledge, thereby achieving optimized training of the student network.

[0039] In some embodiments, such as Figure 2As shown, the local-global spiking neural network also includes an alignment network, which is usually a 1×1 convolutional adaptation layer. Since the teacher network and student network have different structures and different numbers of convolutional layers, during the knowledge distillation process, the size of the intermediate layer feature maps of the two networks is inconsistent when the teacher network features supervise the student network features, making it impossible to directly compare and calculate the loss function. Therefore, the alignment network is used to align the features of the teacher network and the student network, so that the feature maps of the two networks are the same size.

[0040] In practical student network optimization, this embodiment employs Impulse Time-Dependent Plasticity (STDP) forward propagation knowledge for optimization training of the student network, enabling the student network to learn local knowledge from the data. Specifically, the STDP method is an unsupervised local learning rule based on biological rationality, which can be expressed as:

[0041]

[0042] Where, Δu j U(Δt) is the correction factor for synapse j, U(Δt) is a function of pulse time-dependent plasticity, t is the pulse time, and A is the time-dependent plasticity. + and A - τ represents the correction degree for the time-dependent plasticity of the impulse. + and τ - This represents the time constant.

[0043] However, forward propagation training lacks a feedback loop. The student network executes sequentially from input to output without feedback, and the training parameters of the network are not adjusted during the training process. Therefore, it cannot learn the global knowledge of the dataset. Thus, this embodiment uses a teacher network to supervise and guide the student network based on knowledge distillation, enabling the student network to learn the global knowledge of the teacher network and improve the performance of the student network.

[0044] Specifically, this embodiment employs a combination of label-based and feature-based knowledge distillation to construct a local-global knowledge distillation network objective function to guide the training of the student network. Label-based knowledge distillation typically relies on the output of the last fully connected layer of the teacher network, i.e., the label result of the network output. While computationally simple, it cannot address the supervision problem from intermediate layers of the teacher network, which is crucial for representation learning in deep neural networks. Feature-based knowledge distillation is an extension of label-based knowledge distillation, applicable to both shallow and deep neural networks, effectively overcoming the limitations of label-based knowledge distillation.

[0045] Given the output z of the last fully connected layer of a network, label-based knowledge distillation can be expressed as:

[0046]

[0047] in, Let z represent the label-based knowledge distillation loss function. t and z s It is the output of the teacher network and the student network. In label-based knowledge distillation, the effectiveness of soft targets is similar to label smoothing or regularization. The soft target p(z) i The probability that the input (T) belongs to a category can be estimated using the maximum function:

[0048]

[0049] Among them, z i is the output of the i-th category, and a temperature coefficient T is introduced to control the importance of each soft target. Since soft targets contain dark knowledge from the teacher model, the label-based knowledge distillation loss function can be rewritten as:

[0050]

[0051] Wherein, p(z) t T) represents the soft target of the teacher network, p(z) s Z(t) represents the soft objective of the student network, where T represents the temperature coefficient. t It is the output of the teacher network, z s This is the output of the student network, where i represents the output of the i-th category. The loss function for label-based knowledge distillation is typically the KL divergence loss; the output z of the student network can be optimized using formula (9) or (11). s Continuously improve teachers' online output t This enables students to learn about teacher networks and improves student network performance.

[0052] Given an input sample image x, the feature-based knowledge distillation loss can be expressed as:

[0053]

[0054] Among them, L FD f represents the feature-based knowledge distillation loss. t (x) and f s (x) represent the feature maps of the intermediate layers of the teacher network and the student network, respectively. t (·) and Φ s (·) represents the transformation function when the feature maps of the intermediate layers of the teacher network and the student network are inconsistent, i.e., the function representation of the aligned network. This represents the similarity function used to match the feature maps of the teacher network and the student network. This similarity function can be implemented using L1 norm Manhattan distance, L2 norm Euclidean distance, cross-entropy loss function, and maximum mean difference loss function. This similarity function is also used to characterize the feature-based knowledge distillation loss and is used as the feature-based knowledge distillation loss function in subsequent calculations.

[0055] Based on the aforementioned label-based and feature-based knowledge distillation loss functions, a local-global knowledge distillation network objective function is constructed. as follows:

[0056]

[0057] in, This represents the loss function used to supervise the student network using truth values. The loss function for label-based knowledge distillation. Let β1 and β2 be the loss function for feature-based knowledge distillation. and The weighting coefficients.

[0058] This embodiment supervises the student network from two perspectives through the teacher network, combines the loss function trained by the student network itself, and uses three loss functions to constrain the student network simultaneously. With the help of the forward knowledge propagation method based on STDP, the student network learns the global knowledge of the teacher network, thereby improving network performance.

[0059] S40 utilizes a trained student network for dynamic target detection.

[0060] The trained student network possesses a lightweight size and performance similar to the teacher network, effectively enabling dynamic object detection. It is important to note that before practical application, the performance of the student network should be validated using a few-shot dataset and a dynamic object dataset. This can be achieved through exploratory research, experiments, quantitative analysis, and empirical summarization. The performance of the dynamic object recognition algorithm is verified by its recognition accuracy, while the energy consumption of the student network is evaluated using parameters, time step, floating-point operations per second, and network energy transfer. First, the effectiveness of the multi-scale attention feature fusion method is verified through ablation experiments and compared with existing attention-based spiking neural networks. Then, the effectiveness of the local-global learning method is verified through ablation experiments and compared with existing knowledge distillation-based spiking neural networks. Finally, by testing a self-built dynamic environment dataset captured by a dynamic visual sensor in a real-world scene, the differences between this embodiment and traditional dynamic object recognition methods in terms of feature extraction, recognition accuracy, and energy consumption are compared, verifying the recognition performance and environmental adaptability of this embodiment in few-shot real-world scenarios.

[0061] In some embodiments, the preset percentage selected when constructing a neuromorphic few-sample dataset can be determined based on the following:

[0062] First, a dynamic target dataset is collected from a real-world environment using a dynamic vision sensor, serving as sample data for subsequent verification of student network performance. Then, based on the standard neuromorphic dataset, multiple neuromorphic few-shot datasets are constructed at different percentages, for example, five datasets with percentages of 70%, 60%, 50%, 40%, and 30%. Next, with the teacher network trained and parameters fixed, multiple student networks are trained using knowledge distillation on each neuromorphic few-shot dataset and the trained teacher network, resulting in multiple trained student networks, each corresponding to a few-shot training set with a different percentage. After training, the recognition accuracy of each student network is calculated using the dynamic target dataset, serving as an evaluation criterion among the multiple student networks. Finally, the difference between the recognition accuracy and the recognition accuracy of the teacher network is greater than or equal to... Student networks with a preset threshold are deleted, and the lowest percentage from the remaining student networks corresponding to the neuromorphic few-shot dataset is determined as the preset percentage. For example, if the teacher network has a recognition accuracy of 95%, the student networks corresponding to the neuromorphic few-shot datasets constructed with percentages of 70%, 60%, 50%, 40%, and 30% have recognition accuracies of 93%, 90%, 87%, 84%, and 80%, respectively. With a preset threshold of 10%, student networks with recognition accuracies of 84% and 80% are deleted. The lowest percentage in the remaining three student networks corresponding to the neuromorphic few-shot datasets is 50%. Using 50% as the preset percentage indicates that the performance of the student network trained with 50% of the full sample size is not significantly different from that of the teacher network. Therefore, in subsequent student network training, a 50% few-shot dataset can be used to obtain better training and dynamic object detection results.

[0063] This embodiment constructs a neuromorphic few-shot dataset based on a standard neuromorphic dataset. It uses a deep teacher spiking neural network and a shallow student spiking neural network based on multi-scale attention feature fusion as the feature extraction backbone networks. For complex real-world environments with few samples, a knowledge distillation method suitable for deep spiking neural networks is designed to guide and optimize the student network, enabling it to achieve fast and high-precision model training based on the few-shot dataset. Finally, the student network is used as a model for dynamic target detection in real-world environments to quickly identify dynamic targets.

[0064] Based on the same inventive concept, the second embodiment of this disclosure provides a dynamic target detection device based on a spiking neural network using local-global learning, the schematic diagram of which is shown below. Figure 3 As shown, it mainly includes: a dataset construction module 10, used to establish a neuromorphic standard dataset and a neuromorphic few-shot dataset, wherein each type of data in the neuromorphic few-shot dataset is randomly selected from the corresponding type of data in the neuromorphic standard dataset according to a preset percentage; a teacher network training module 20, used to construct a deep spiking neural network with multi-scale attention feature fusion as the teacher network, and use the neuromorphic standard dataset to train the teacher network so that the teacher network learns global knowledge; a student network training module 30, used to construct a shallow spiking neural network as the student network, and use the neuromorphic few-shot dataset and the trained teacher network to guide the student network to train through knowledge distillation so that the student network learns local knowledge; and an object detection module 40, used to perform dynamic object detection using the trained student model.

[0065] In some embodiments, the teacher network and the student network are from the same source network.

[0066] In some embodiments, the teacher network includes at least: a teacher initial network, multiple attention spiking network blocks, and a teacher classifier. The attention spiking network blocks include at least an attention module and a spiking coding module. The attention module includes at least a temporal attention module, a multi-scale channel attention module, and a spatial attention module. The student network includes at least: a student initial network, multiple spiking network blocks, and a student classifier.

[0067] In some embodiments, the features output by the temporal attention module Represented as:

[0068]

[0069] Here, TA represents the time attention mechanism. Let n represent the intermediate feature of the nth teacher in the network. The feature space scale is represented by t, and time is represented by t. H represents the weights of the time attention mechanism. n ×W n C represents the feature map size. n Indicates the number of channels;

[0070] Features output by the multi-scale channel attention module Represented as:

[0071]

[0072] Wherein, CA represents the multi-scale channel attention mechanism, g c(H t,n ) represents the weights for multi-scale channel attention. Represented as global channel information weight, This is represented as the local channel information weight, where σ is the sigmoid function;

[0073] Features output by the spatial attention module Represented as:

[0074]

[0075] in, This is represented as the weight of the spatial attention mechanism.

[0076] In some embodiments, the student network training module 30 is specifically used to: construct a local-global knowledge distillation network objective function based on the trained teacher network through label-based knowledge distillation and feature-based knowledge distillation to guide the training of the student network;

[0077] The loss function for label-based knowledge distillation is:

[0078]

[0079] Wherein, p(z) t T) represents the soft target of the teacher network, p(z) s Z(t) represents the soft objective of the student network, where T represents the temperature coefficient. t It is the output of the teacher network, z s This is the output of the student network, where i represents the output of the i-th category. The loss function representing label-based knowledge distillation;

[0080] The loss function for feature-based knowledge distillation is:

[0081]

[0082] Among them, f t (x) and f s (x) represent the feature maps of the intermediate layers of the teacher network and the student network, respectively. Let Φ represent the similarity function used to match the feature maps of the teacher network and the student network. t (·) and Φ s (·) is the transition function when the feature maps of the intermediate layers of the teacher network and the student network are inconsistent;

[0083] Local-Global Knowledge Distillation Network Objective Function for:

[0084]

[0085] in, This represents the loss function used to supervise the student network using truth values. The loss function for label-based knowledge distillation. Let β1 and β2 be the loss function for feature-based knowledge distillation. and The weighting coefficients.

[0086] In some embodiments, the student network training module 30 is specifically used for: training using impulsive time-dependent plasticity forward propagation knowledge; wherein the learning rule for impulsive time-dependent plasticity is:

[0087]

[0088] Where, Δu j U(Δt) is the correction factor for synapse j, U(Δt) is a function of pulse time-dependent plasticity, t is the pulse time, and A is the time-dependent plasticity. + and A - τ represents the correction degree for the time-dependent plasticity of the impulse. + and τ - This represents the time constant.

[0089] In some embodiments, the dataset construction module 10 is further configured to collect dynamic target datasets in the actual environment using a dynamic visual sensor; construct multiple neuromorphic few-shot datasets according to different percentages; the student network training module 30 is further configured to guide multiple student networks to train using each neuromorphic few-shot dataset and the trained teacher network through knowledge distillation, thereby obtaining multiple trained student networks; the device also includes a verification module (not shown in the figure), configured to calculate the recognition accuracy of each student network using the dynamic target dataset; delete student networks whose recognition accuracy is greater than or equal to the recognition accuracy of the teacher network than a preset threshold, and determine the lowest percentage from the neuromorphic few-shot datasets corresponding to the remaining student networks as the preset percentage.

[0090] Based on the same inventive concept, the third embodiment of this disclosure provides a storage medium storing a computer program that can be installed in any electronic device with a processor. When the computer program is executed by the processor, it implements the steps of the dynamic target detection method based on local-global learning spiking neural network described in the first embodiment of this disclosure.

[0091] Based on the same inventive concept, the fourth embodiment of this disclosure provides an electronic device, including at least a memory and a processor, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program in the memory, implements the steps of the dynamic target detection method based on local-global learning spiking neural network of the first embodiment of this disclosure.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A dynamic target detection method based on a spiking neural network using local-global learning, characterized in that, include: Establish a standard neuromorphic dataset and a few-sample neuromorphic dataset, wherein each type of data in the few-sample neuromorphic dataset is randomly selected from the corresponding type of data in the standard neuromorphic dataset according to a preset percentage; A deep spiking neural network with multi-scale attention feature fusion is constructed as a teacher network, and the teacher network is trained using the neuromorphic standard dataset to enable the teacher network to learn global knowledge. A shallow spiking neural network is constructed as a student network, and the student network is trained by knowledge distillation using the neuromorphic few-shot dataset and the pre-trained teacher network, so that the student network learns local knowledge. Detecting dynamic targets in images using a pre-trained student network; The teacher network includes at least: a teacher initial network, multiple attention spiking network blocks, and a teacher classifier. The attention spiking network blocks include at least an attention module and a spiking coding module. The attention module includes at least a temporal attention module, a multi-scale channel attention module, and a spatial attention module. The student network includes at least: a student initial network, multiple spiking network blocks, and a student classifier. The features output by the time attention module Represented as: in, This represents the time attention mechanism. Indicates the teacher network Intermediate features of each, Indicates the feature space scale. Indicates time, The weights representing the temporal attention mechanism, Indicates the feature map size. Indicates the number of channels; The features output by the multi-scale channel attention module Represented as: in, This represents a multi-scale channel attention mechanism. For the weights of multi-scale channel attention, Represented as global channel information weight, Represented as local channel information weights, It is the sigmoid function; The features output by the spatial attention module Represented as: in, This is represented as the weight of the spatial attention mechanism.

2. The dynamic target detection method according to claim 1, characterized in that, The teacher network and the student network are from the same source network.

3. The dynamic target detection method according to claim 1, characterized in that, The step of using the neuromorphic few-shot dataset and the trained teacher network to guide the training of the student network through knowledge distillation includes: Based on the trained teacher network, a local-global knowledge distillation network objective function is constructed through label-based knowledge distillation and feature-based knowledge distillation to guide the training of the student network; The loss function for label-based knowledge distillation is: in, For the soft target of the teacher network, For the soft goals of student networks, T Indicates the temperature coefficient. It is the output of the teacher network. It is the output of the student network. Indicates the first Output of each category The loss function representing label-based knowledge distillation; Loss function based on feature-based knowledge distillation for: in, and These are feature maps of the intermediate layers of the teacher network and the student network, respectively. This represents the similarity function used to match the feature maps of the teacher and student networks. and For the transition function when the feature maps of the intermediate layers of the teacher network and the student network are inconsistent; The objective function of the local-global knowledge distillation network for: in, This represents the loss function used to supervise the student network using truth values. The loss function for label-based knowledge distillation. The loss function for feature-based knowledge distillation. and They are respectively and The weighting coefficients.

4. The dynamic target detection method according to claim 1, characterized in that, The step of using the neuromorphic few-shot dataset and the trained teacher network to guide the training of the student network through knowledge distillation includes: The student network is trained using impulsive time-dependent plasticity forward propagation knowledge; wherein the learning rule for impulsive time-dependent plasticity is: in, It is a synapse j The correction amount, It is a function of impulse time-dependent plasticity. t It is the pulse duration. and This indicates the degree of correction for pulse time-dependent plasticity. and This represents the time constant.

5. The dynamic target detection method according to any one of claims 1 to 4, characterized in that, Also includes: Dynamic target datasets are acquired in real-world environments using dynamic vision sensors. Multiple neuromorphic small sample datasets were constructed according to different percentages; Multiple student networks are trained by knowledge distillation using each of the neuromorphic few-sample datasets and the trained teacher network to guide the training of multiple student networks respectively. The recognition accuracy of each student network is calculated using the dynamic target dataset; Student networks whose difference between the recognition accuracy and the recognition accuracy of the teacher network is greater than or equal to a preset threshold are deleted, and the lowest percentage is determined from the neuromorphic few-sample dataset corresponding to the remaining student networks as the preset percentage.

6. A dynamic target detection device based on a spiking neural network using local-global learning, characterized in that, include: The dataset construction module is used to build a standard neuromorphic dataset and a few-sample neuromorphic dataset, wherein each type of data in the few-sample neuromorphic dataset is randomly selected from the corresponding type of data in the standard neuromorphic dataset according to a preset percentage; The teacher network training module is used to construct a deep spiking neural network with multi-scale attention feature fusion as the teacher network, and to train the teacher network using the neuromorphic standard dataset so that the teacher network learns global knowledge. The student network training module is used to construct a shallow spiking neural network as a student network, and to guide the student network to learn local knowledge by using the neuromorphic few-shot dataset and the trained teacher network through knowledge distillation. The object detection module is used to detect dynamic objects in images using a trained student model; The teacher network includes at least: a teacher initial network, multiple attention spiking network blocks, and a teacher classifier. The attention spiking network blocks include at least an attention module and a spiking coding module. The attention module includes at least a temporal attention module, a multi-scale channel attention module, and a spatial attention module. The student network includes at least: a student initial network, multiple spiking network blocks, and a student classifier. The features output by the time attention module Represented as: in, This represents the time attention mechanism. Indicates the teacher network Intermediate features of each, Indicates the feature space scale. Indicates time, The weights representing the temporal attention mechanism, Indicates the feature map size. Indicates the number of channels; The features output by the multi-scale channel attention module Represented as: in, This represents a multi-scale channel attention mechanism. For the weights of multi-scale channel attention, Represented as global channel information weight, Represented as local channel information weights, It is the sigmoid function; The features output by the spatial attention module Represented as: in, This is represented as the weight of the spatial attention mechanism.

7. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic target detection method based on local-global learning spiking neural network according to any one of claims 1 to 5.

8. An electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program on the memory, it implements the steps of the dynamic target detection method based on local-global learning spiking neural networks according to any one of claims 1 to 5.

Images