Multi-stage pulse neural network training method and device based on knowledge distillation

Abstract

The invention discloses a multi-stage spiking neural network training method and device based on knowledge distillation, and the method comprises the steps: mapping the intermediate output of a spiking neural network into a logs space in stages, and calculating the confidence-based loss and the feature-based inter-sample loss through the intermediate output and the final result of an artificial neural network respectively; and finally, integrating and calculating the confidence-based loss and the feature-based inter-sample loss of each stage to obtain a total loss function so as to train and update network parameters of the spiking neural network for executing an image classification task. According to the training method provided by the invention, the outputs of the artificial neural network and the spiking neural network are mapped to a unified logs space, and the joint loss between samples based on confidence and features is considered, so that a scheme capable of enabling the spiking neural network to extract knowledge from the artificial neural network more efficiently is formulated; therefore, the precision difference between the artificial neural network and the pulse neural network is further reduced.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a multi-stage pulse neural network training method and device based on knowledge distillation. Background Art

[0002] Spiking neural network (SNN) is an emerging neural network that transmits information through neuron pulse signals. It has unique characteristics close to biological nervous systems. Spiking neural network uses pulse discrete events representing neuron discharges to process and transmit information, and also has the advantages of efficient event-driven processing and low power consumption. In biological nervous systems, information transmission between neurons is carried out through discrete pulse signals. In contrast, traditional artificial neural networks (ANN) use continuous activation functions for information transmission, ignoring the temporal encoding method of pulses. The study of spiking neural networks can better simulate and understand the working principles of biological nervous systems, thereby providing a deep understanding of the operating mechanism of the brain. Therefore, how to effectively train spiking neural networks is currently a very worthy issue to study.

[0003] At present, there are three main methods for constructing spiking neural networks: the first is to use the spike time dependent plasticity (STDP) learning rule, which is a Hebbian-type learning rule related to the plasticity mechanism in the biological nervous system. It can be used to simulate the adjustment of synaptic weights between neurons, but does not involve the weight update method of the traditional neural network. Therefore, the lack of global guidance causes the spiking neural network trained with the STDP rule to be unable to be extended to a deep network with good performance, and is limited to a shallow structure for solving simple tasks. The second is to replace the actual gradient in the spiking neural network with a designed continuous function to approximate the discontinuous derivative of the spiking neuron. This approximation is sufficient for spiking neural networks with smaller network sizes, but it is difficult for spiking neural networks to use deep structures and networks that solve challenging tasks. In addition, spiking neural networks trained using the alternative gradient method cannot achieve performance comparable to that of their corresponding artificial neural networks. The third is the ANN-to-SNN conversion method, which converts the parameters of the pre-trained artificial neural network into a spiking neural network with the same structure. Although this approach can achieve almost equivalent representation even in large networks, the spiking neural network converted from ANN usually requires a large number of time steps, reaching hundreds or even thousands of time steps, which leads to a lot of power consumption, which runs counter to the original intention of spiking neural networks.

[0004] Therefore, it is a very worthy research issue to train pulse neural networks more efficiently and narrow the gap in accuracy between them and artificial neural networks. Therefore, a more innovative and effective training method is needed so that the trained model can not only have an accuracy closer to that of artificial neural networks, but also make the training process more efficient and low-cost. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-stage spiking neural network training method and device based on knowledge distillation in view of the deficiencies in the prior art.

[0006] The objective of the present invention is achieved through the following technical solutions: In a first aspect, an embodiment of the present invention provides a multi-stage pulse neural network training method based on knowledge distillation, comprising the following steps:

[0007] (1) Obtain known image data to construct a dataset;

[0008] (2) Selecting an artificial neural network as the teacher model in knowledge distillation, selecting a spiking neural network as the student model in knowledge distillation, and presetting the parameters of the LIF neuron in the spiking neural network;

[0009] (3) The spiking neural network as the student model and the artificial neural network as the teacher model are divided into k different stages according to the model structure;

[0010] (4) in the process of training the spiking neural network using the data set, calculating the feature-based inter-sample loss of the stage according to the features output by each stage of the artificial neural network and the features output by each stage of the spiking neural network;

[0011] (5) In the process of training the spiking neural network using the data set, the features output by each stage of the spiking neural network are mapped to the probability distribution space through the fully connected layer, and then converted into probability output through the softmax function. The probability of the final output of the artificial neural network is used as the basic knowledge to calculate the confidence-based loss of this stage;

[0012] (6) The total loss function is calculated based on the feature-based inter-sample loss and the confidence-based loss of the k stages. The minimization of the total loss function is taken as the optimization goal. The network parameters of the spiking neural network are updated by the back-propagation method to obtain the trained spiking neural network for performing image classification tasks.

[0013] Furthermore, the step (4) specifically includes: for the feature-based sample loss of the i-th stage, the calculation method thereof specifically includes:

[0014] First, assume that the feature F output by the artificial neural network in the i-th stage TMapped to a Gram matrix with a dimension of B*B, where B represents the batch size; then for the pulse neural network, at the output of the i-th stage, there are features at each time step Map it into a Gram matrix with a dimension of T*B*B, where T represents the time step; then map all time step features in the time dimension The corresponding Gram matrices are averaged to obtain the averaged Gram matrix; secondly, the feature F T The corresponding Gram matrix and the averaged Gram matrix are L2 regularized to obtain the regularized Gram matrices corresponding to the artificial neural network and the spiking neural network, respectively. Finally, the feature-based inter-sample loss L is calculated based on the regularized Gram matrices corresponding to the artificial neural network and the spiking neural network. inter .

[0015] Furthermore, the feature-based inter-sample loss L inter The calculation formula is:

[0016]

[0017] Among them, β is the adjustment factor, R t Represents the regularized Gram matrix corresponding to the artificial neural network, R s Represents the regularized Gram matrix corresponding to the pulse neural network, and its expressions are:

[0018] R t =exp|| rel (F T )||2

[0019]

[0020] Among them, exp represents the exponential function, ||·||2 represents L2 regularization, and f rel (F T ) represents the feature F T The corresponding Gram matrix, Represents the time step feature The corresponding Gram matrices are expressed as follows:

[0021] f rel (F T )=F T · T T

[0022]

[0023] Among them, F T TRepresentation feature F T The transpose of Represents the time step feature The transpose of .

[0024] Furthermore, the step (5) specifically includes: for the confidence-based loss L of the i-th stage conf , and its calculation method specifically includes:

[0025] First, the features output by the spiking neural network in the i-th stage are mapped to the probability distribution space through a fully connected layer, and then converted into the probability output p by a softmax function. sc , which is obtained by counting the pulse spikes in the specified time window and normalizing them; then, according to the probability p of the final output of the artificial neural network tc and the probability output p of the spiking neural network at the i-th stage sc Calculate the confidence-based loss L conf .

[0026] Furthermore, the probability output p sc The calculation formula is:

[0027]

[0028] Among them, p sc Indicates that the output of the pulse neural network is the probability output of category c, N c and N j Respectively represent the peak counts of category c and category j in time window T;

[0029] The confidence-based loss L conf The calculation formula is:

[0030]

[0031] Among them, p tc It means that the final output of the artificial neural network is the probability output of category c, C represents the total number of categories, and γ is an adjustable hyperparameter.

[0032] Furthermore, the calculation formula of the total loss function is:

[0033]

[0034] Among them, L total represents the total loss function, and λ is used to balance the feature-based sample loss L inter and the confidence-based loss L conf is a hyperparameter of , i represents the i-th stage.

[0035] A second aspect of an embodiment of the present invention provides a multi-stage pulse neural network training device based on knowledge distillation, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned multi-stage pulse neural network training method based on knowledge distillation.

[0036] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned multi-stage pulse neural network training method based on knowledge distillation.

[0037] The beneficial effects of the present invention are as follows: the present invention maps the intermediate output of the pulse neural network to the logits space in stages, calculates the confidence-based loss and the feature-based inter-sample loss respectively with the final result of the artificial neural network, and finally integrates the confidence-based loss and the feature-based inter-sample loss of each stage to calculate the total loss function, thereby training and updating the network parameters of the pulse neural network for performing image classification tasks; the training method described in the present invention maps the outputs of the artificial neural network and the pulse neural network to a unified logits space, and considers the joint loss between samples based on confidence and features, and formulates a scheme that enables the pulse neural network to more efficiently extract knowledge from the artificial neural network, thereby further narrowing the accuracy gap between the artificial neural network and the pulse neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the multi-stage spiking neural network training method based on knowledge distillation of the present invention;

[0039] Figure 2 The single-stage L inter Calculation diagram;

[0040] Figure 3 The single-stage L conf Calculation diagram;

[0041] Figure 4 It is a schematic diagram of the overall training process of the present invention;

[0042] Figure 5 It is a structural schematic diagram of a multi-stage pulse neural network training device based on knowledge distillation of the present invention. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the above general description and the detailed description that follows are merely exemplary and explanatory and cannot limit the present application.

[0044] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that, although the term first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of..." or "when..." or "in response to determination". Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also include other elements not clearly listed, or also include elements inherent to such process and method. In the absence of more restrictions, the elements limited by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0046] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0047] The multi-stage spiking neural network training method based on knowledge distillation of the present invention can map the outputs of artificial neural networks (ANN) and spiking neural networks (SNN) to a unified probability distribution (logits) space, and considers the joint loss between samples based on confidence and features, and formulates a scheme that enables SNN to extract knowledge from ANN more efficiently, thereby further narrowing the accuracy gap between artificial neural networks and spiking neural networks.

[0048] like Figure 1 As shown, the multi-stage pulse neural network training method based on knowledge distillation of the present invention specifically includes the following steps:

[0049] (1) Obtain known image data to construct a dataset.

[0050] It should be understood that obtaining known image data to construct a data set, then using the data set to train the spiking neural network, and using the trained spiking neural network to perform image classification tasks is conducive to improving image classification accuracy.

[0051] (2) An artificial neural network is selected as the teacher model in knowledge distillation, a spiking neural network is selected as the student model in knowledge distillation, and the parameters of the LIF neurons in the spiking neural network are preset.

[0052] (3) The pulse neural network as the student model and the artificial neural network as the teacher model are divided into k different stages according to the model structure.

[0053] It should be noted that this embodiment is an image classification task, and the pulse neural network and the artificial neural network are divided into k different stages according to their network structure. The value of k is usually a hyperparameter of the experiment, which can usually be determined according to the specific network structure and is set manually. For example, k=3 can be set to divide the SNN and ANN into three different stages: shallow feature extraction stage, deep feature extraction stage, and classification stage.

[0054] (4) In the process of training SNN using the data set, the feature-based sample loss L of each stage is calculated based on the features output by ANN at each stage and the features output by SNN at each stage. inter .

[0055] Specifically, Figure 2 As shown, for the feature-based inter-sample loss L at the i-th stage inter , and its calculation method specifically includes: first, assuming that the teacher model ANN outputs the feature F in the i-th stage T Mapped to a Gram matrix with a dimension of B*B, where B represents the batch size; then for the student model SNN, the output at the i-th stage has the features of each time step Map it into a Gram matrix with a dimension of T*B*B, where T represents the time step; then map all time step features in the time dimension The corresponding Gram matrix is ​​averaged to make it consistent with the dimension of the artificial neural network, and the averaged Gram matrix is ​​obtained; secondly, the feature F TThe corresponding Gram matrix and the averaged Gram matrix are L2 regularized to obtain the regularized Gram matrices corresponding to ANN and SNN respectively; finally, the feature-based inter-sample loss L is calculated based on the regularized Gram matrices corresponding to ANN and SNN. inter .

[0056] It should be understood that since the student model is an SNN, the features output at each stage include the time dimension. The application of L2 regularization can extend the features of artificial neural networks and spiking neural networks to the same value range.

[0057] Furthermore, feature F T The corresponding Gram matrix is ​​expressed as:

[0058] f rel (F T )=F T · T T

[0059] Among them, f rel (F T ) represents the feature F T The corresponding Gram matrix, F T T Representation feature F T The transpose of .

[0060] Furthermore, the time step feature The corresponding Gram matrix is ​​expressed as:

[0061]

[0062] in, Represents the time step feature The corresponding Gram matrix, Represents the time step feature The transpose of .

[0063] Furthermore, the regularized Gram matrices corresponding to ANN and SNN are expressed as:

[0064] R t =exp|| rel (F T )||2

[0065]

[0066] Among them, R t Represents the regularized Gram matrix corresponding to ANN, R srepresents the regularized Gram matrix corresponding to SNN, exp represents the exponential function, and ||·||2 represents L2 regularization.

[0067] Furthermore, the feature-based sample loss L inter The calculation formula is:

[0068]

[0069] Among them, β is the adjustment factor, and its range is [0,e].

[0070] (5) In the process of training SNN using the dataset, the features output by each stage of SNN are mapped to the logits space through the fully connected layer (FC), and then converted into probability output through the softmax function. The probability of the final output of ANN is used as the basic knowledge to calculate the confidence-based loss L at this stage. conf .

[0071] It should be understood that the features output by each stage of the SNN are mapped into the logits space, where the specific semantics brought by the model structure are eliminated, so that the feature semantics can be aligned between the ANN and the SNN. The probability of the final output of the ANN is used as the basic knowledge for calculating the confidence-based loss L during SNN training. conf Use, from the teacher model's side, its final output is directly used as a source of knowledge.

[0072] Specifically, Figure 3 As shown, for the confidence-based loss L at the i-th stage conf , and its calculation method specifically includes: firstly, the features output by the student model SNN in the i-th stage are mapped to the logits space through the fully connected layer, and then converted into the probability output p through the softmax function sc , since SNN does not produce continuous activation, the probability output p is calculated sc When the pulse spike is converted into a probability metric, it is done by counting the pulse spikes within a specified time window and normalizing them. The calculation formula is:

[0073]

[0074] Among them, p sc Indicates that the output of SNN is the probability output of category c, N c and N jThey represent the peak counts of category c and category j in the time window T. c represents a category in the image classification task result, and the denominator is for normalization so that the sum of the probability outputs of all categories is 1. Then, according to the probability p of the final output of the teacher model ANN tc and the probability output p of the i-th stage SNN sc Calculate the confidence-based loss L conf , and its calculation formula is:

[0075]

[0076] Among them, p tc It indicates that the final output of ANN is the probability output of category c, C represents the total number of categories, and γ is an adjustable hyperparameter that dynamically weights the importance of target information according to the confidence of the teacher model.

[0077] (6) According to the feature-based inter-sample loss L of k stages inter and the confidence-based loss L conf Calculate the total loss function L total , to minimize the total loss function L total In order to optimize the target, the back propagation method is used to update the network parameters of the SNN to obtain the trained SNN for performing image classification tasks.

[0078] Furthermore, the total loss function L total The calculation formula is:

[0079]

[0080] Among them, λ is used to balance the feature-based sample loss L inter and the confidence-based loss L conf is a hyperparameter of , i represents the i-th stage.

[0081] like Figure 4 As shown, it is the overall training method framework of the present invention. In this embodiment, k=3, that is, there are 3 stages, and the above L is calculated in each stage. inter and L conf After that, the total loss function L is calculated according to the above formula total . After each training batch, the total loss function L of the batch is calculated total Finally, the network parameters of the student model SNN are updated according to the back-propagation method, thereby completing the training of the pulse neural network model and obtaining the trained SNN, which is then used to perform image classification tasks.

[0082] In summary, the multi-stage pulse convolutional neural network training method based on knowledge distillation proposed in the present invention uses a new distillation framework to map the artificial neural network features and the pulse neural network features to the same space, namely the logits space, through the FC layer. In the logits space, redundant architecture-specific information is deleted, and the problem of inconsistency in the semantic space of the output of the artificial neural network and the pulse neural network is eliminated, thereby achieving feature alignment across any architecture. During the training process of the student model SNN, these additional branches are combined with the main body of the student model for optimization. In the reasoning of the student model, these branches are omitted to ensure that no additional reasoning overhead is incurred, so this does not violate the principle of deploying the pulse neural network model on a neuromorphic chip. Specifically, a fine-grained knowledge distillation method and a new total loss function L are introduced. total To facilitate this process. The new total loss function L total It consists of two parts: confidence-based loss L conf and feature-based inter-sample loss L inter In addition, the teacher model and the student model are divided into k stages to ensure that the student SNN thoroughly learns the compressed knowledge from the teacher ANN. The confidence-based loss L for these k stages is conf and feature-based inter-sample loss l inter Calculate and sum them separately to get the final total loss function L total , thereby updating the network parameters and obtaining a trained spiking neural network, thereby narrowing the gap in accuracy between spiking neural networks and artificial neural networks.

[0083] Corresponding to the aforementioned embodiment of the multi-stage spiking neural network training method based on knowledge distillation, the present invention also provides an embodiment of the multi-stage spiking neural network training device based on knowledge distillation.

[0084] See also Figure 5 An embodiment of the present invention provides a multi-stage pulse neural network training device based on knowledge distillation, including one or more processors and a memory, and the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the multi-stage pulse neural network training method based on knowledge distillation in the above embodiment.

[0085] The embodiment of the multi-stage pulse neural network training device based on knowledge distillation of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 5 As shown, it is a hardware structure diagram of any device with data processing capability where the multi-stage pulse neural network training device based on knowledge distillation of the present invention is located, except Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0086] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0087] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0088] An embodiment of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the multi-stage pulse neural network training method based on knowledge distillation in the above embodiment is implemented.

[0089] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-stage spiking neural network training method based on knowledge distillation, characterized in that: The following steps are involved: (1) Obtain known image data to construct a dataset; (2) Selecting an artificial neural network as the teacher model in knowledge distillation, selecting a spiking neural network as the student model in knowledge distillation, and presetting the parameters of the LIF neuron in the spiking neural network; (3) The spiking neural network as the student model and the artificial neural network as the teacher model are divided into k different stages according to the model structure; (4) in the process of training the spiking neural network using the data set, calculating the feature-based inter-sample loss of the stage according to the features output by each stage of the artificial neural network and the features output by each stage of the spiking neural network; (5) In the process of training the spiking neural network using the data set, the features output by each stage of the spiking neural network are mapped to the probability distribution space through the fully connected layer, and then converted into probability output through the softmax function. The probability of the final output of the artificial neural network is used as the basic knowledge to calculate the confidence-based loss of this stage; (6) The total loss function is calculated based on the feature-based inter-sample loss and the confidence-based loss of the k stages. The minimization of the total loss function is taken as the optimization goal. The network parameters of the spiking neural network are updated by the back-propagation method to obtain the trained spiking neural network for performing image classification tasks.

2. The multi-stage pulse neural network training method based on knowledge distillation according to claim 1 is characterized in that: The step (4) specifically includes: for the feature-based inter-sample loss of the i-th stage, the calculation method thereof specifically includes: First, assume that the feature F output by the artificial neural network in the i-th stage T Mapped to a Gram matrix with a dimension of B*B, where B represents the batch size; then for the pulse neural network, at the output of the i-th stage, there are features at each time step Map it into a Gram matrix with a dimension of T*B*B, where T represents the time step; then map all time step features in the time dimension The corresponding Gram matrices are averaged to obtain the averaged Gram matrix; secondly, the feature F T The corresponding Gram matrix and the averaged Gram matrix are L2 regularized to obtain the regularized Gram matrices corresponding to the artificial neural network and the spiking neural network, respectively. Finally, the feature-based inter-sample loss L is calculated based on the regularized Gram matrices corresponding to the artificial neural network and the spiking neural network. inter .

3. The multi-stage pulse neural network training method based on knowledge distillation according to claim 2 is characterized in that: The feature-based inter-sample loss L intet The calculation formula is: Among them, β is the adjustment factor, R y Represents the regularized Gram matrix corresponding to the artificial neural network, R s Represents the regularized Gram matrix corresponding to the pulse neural network, and its expressions are: R y =exp||f trl (F2)||2 Among them, exp represents the exponential function, ||·||2 represents L2 regularization, and f rel (F T ) represents the feature F T The corresponding Gram matrix, Represents the time step feature The corresponding Gram matrices are expressed as follows: f rel (F T )=F T ·F T T Among them, F T T Representation feature F T The transpose of Represents the time step feature The transpose of .

4. The multi-stage pulse neural network training method based on knowledge distillation according to claim 1 is characterized in that: The step (5) specifically includes: for the confidence-based loss L of the i-th stage conf , and its calculation method specifically includes: First, the features output by the spiking neural network in the i-th stage are mapped to the probability distribution space through a fully connected layer, and then converted into the probability output p by a softmax function. sc , which is obtained by counting the pulse spikes in the specified time window and normalizing them; then, according to the probability p of the final output of the artificial neural network tc and the probability output p of the spiking neural network at the i-th stage sc Calculate the confidence-based loss L conf .

5. The multi-stage pulse neural network training method based on knowledge distillation according to claim 4 is characterized in that: The probability output p sc The calculation formula is: Among them, p sc Indicates that the output of the pulse neural network is the probability output of category c, N c and N j Respectively represent the peak counts of category c and category j in time window T; The confidence-based loss L conf The calculation formula is: Among them, p tc It means that the final output of the artificial neural network is the probability output of category c, C represents the total number of categories, and γ is an adjustable hyperparameter.

6. The multi-stage pulse neural network training method based on knowledge distillation according to claim 1 is characterized in that: The calculation formula of the total loss function is: Among them, L total represents the total loss function, and λ is used to balance the feature-based sample loss L inter and the confidence-based loss L conf is a hyperparameter of , i represents the i-th stage.

7. A multi-stage spiking neural network training device based on knowledge distillation, comprising one or more processors and a memory, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the multi-stage pulse neural network training method based on knowledge distillation as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: A program is stored thereon, which, when executed by a processor, is used to implement the multi-stage pulse neural network training method based on knowledge distillation as described in any one of claims 1 to 6.

Images