Layered local mixed loss spiking neural network training algorithm with biological rationality

Abstract

The invention discloses a hierarchical local mixed loss spiking neural network training algorithm with biological rationality. The algorithm comprises the following steps: step 1, carrying out data preprocessing on a data set; 2, defining an LIF pulse neuron model based on soft reset and a pulse coding mode of direct coding; 3, constructing an auxiliary classifier for each layer of the pulse neural network, and calculating cross entropy prediction loss; 4, respectively calculating similarity matrixes of the hidden layer features and the corresponding labels, and further calculating a similarity matching error between the similarity matrixes; step 5, defining other types of loss functions, mainly comparing loss and the like; step 6, according to a set weighting coefficient, weighting and combining different types of loss to obtain a mixed local loss function; 7, calculating the gradient of the current layer by using the mixed loss, and carrying out local weight updating; and step 8, storing the trained model, evaluating the accuracy on classification tasks, and comparing to obtain an optimal mixed loss model.

Description

Technical Field

[0001] The present invention relates to a training algorithm for spiking neural networks, in particular to a hierarchical local hybrid loss spiking neural network training algorithm with biological rationality. Background Art

[0002] As the third-generation neural network model with biologically inspired features, the spiking neural network (SNN) is significantly different from the second-generation artificial neural network (ANN). Different from the continuous numerical transmission of the artificial neural network, the spiking neural network transmits discrete spike sequences by simulating the spike generation mechanism of biological neurons, which is closer to the working principle of the brain nervous system. The spiking neural network belongs to the cross-field of neuroscience and artificial intelligence, and is the research focus of the brain-inspired intelligence direction. Due to its biological interpretability and energy consumption saving characteristics, the spiking neural network is showing unique advantages in the fields of neuromorphic computing and low-power hardware deployment, which is particularly worthy of attention in the current situation where deep learning increasingly consumes computing resources.

[0003] Although the spiking neural network has good research value, it faces the problem of difficult training. This is because the spike generation function in the spiking neural network is the step Heaviside function, which is non-differentiable at the discontinuity and only has the generalized derivative Dirac Delta function. This generalized derivative is zero everywhere except at the discontinuity where it is positive infinity, which will cause difficulties in calculating the gradient when using the conventional backpropagation (BP) algorithm in ANN. To address the root cause of this problem, researchers have explored a variety of different feasible solutions, including three mainstream SNN training algorithms: spike-timing-dependent plasticity (STDP), surrogate gradient descent method, and the method of converting ANN to SNN.

[0004] These several mainstream SNN training algorithms have their own advantages and disadvantages: The STDP method has high biological interpretability, but it is an unsupervised local algorithm that does not use label information, and its performance on deep networks and complex tasks is very limited; The surrogate gradient descent method alleviates to a certain extent the problem of difficult gradient backpropagation in SNN due to the non-differentiability of the spike function, but inherits the biological irrationality and gradient disappearance problems of the BP algorithm. In addition, there are problems such as surrogate function approximation error accumulation and large computational resource overhead; The method of converting ANN to SNN has a high accuracy, close to that of an ANN with the same structure, but also has low biological rationality, requires a large number of time steps to reduce the conversion error, and is generally an offline conversion, making it difficult to be used for online inference. Therefore, there is currently no dominant general SNN training algorithm, and the improvement of existing training algorithms and the exploration of new training algorithms are both in progress.

[0005] The goal of the new training algorithm for spiking neural networks is to overcome the difficult-to-balance trade-offs in biological plausibility, computational energy consumption, accuracy, etc. of existing SNN training algorithms. Some existing attempts include the STDP method based on error modulation, the reinforcement learning method based on reward, the equilibrium propagation method based on implicit differentiation, the method based on feedback alignment, the method based on perturbation learning, and the method based on forward gradient, etc. These methods have certain commonalities and all belong to the non-global BP methods. For example, the STDP method with error modulation introduces an error supervision signal for local plasticity; the method based on feedback alignment directly connects the hidden layer through a random matrix to transmit error information, avoiding the problem of difficult-to-compute gradients caused by the error being transmitted layer by layer through the chain rule. Reference: REN M, KORNBLITH S, LIAO R, et al: Scaling forward gradient with local losses. In: The Eleventh International Conference on Learning Representations. 2022.

[0006] The research on the new training algorithm for spiking neural networks mentioned above verifies the feasibility of non-BP SNN algorithms. These methods cleverly avoid the problem of difficult backpropagation of gradients caused by the non-differentiability of the spiking function, avoid the criticism that backpropagation does not conform to the principles of the brain and thus has low biological plausibility, and can also obtain signals for guiding the change of network weights from labels and errors, and are superior to traditional local plasticity rules (such as STDP) in terms of model accuracy. However, these new algorithms also have obvious disadvantages. For example, the accuracy of the SNN algorithm based on direct feedback alignment (DFA) still has a certain gap compared with the SNN algorithm based on surrogate gradient descent; while the SNN algorithm based on perturbation learning requires a large number of parameter perturbation samplings to complete a single weight update, resulting in low training efficiency. Summary of the Invention

[0007] Objective of the Invention: The original intention of the present invention is to explore a new and better SNN training algorithm on the basis of the non-BP SNN training algorithm route, and implement an SNN hierarchical local hybrid loss learning algorithm that achieves a balance in biological plausibility, computational energy consumption, and model accuracy.

[0008] To solve the above technical problems, the present invention discloses a biologically plausible hierarchical local hybrid loss spiking neural network training algorithm, including the following steps:

[0009] Step 1, preprocess the dataset, including normalizing the pixel values to the interval [0, 1], and also performing data augmentation operations such as random cropping and random horizontal flipping;

[0010] Step 2, construct a discrete LIF spiking neuron model based on soft reset, and adopt direct coding for the spike coding method;

[0011] Step 3, add an Auxiliary Classifier to each hidden layer of the spiking neural network. Its function is to predict the final classification result using the output value of the current hidden layer, and calculate the cross-entropy prediction loss L of the auxiliary classifier predict ;

[0012] Step 4, calculate the similarity matrix of the feature representation of the current hidden layer and the similarity matrix of the true labels, and obtain the similarity matching error L between the two similarity matrices similarity

[0013] Step 5, calculate other types of optional loss functions, mainly referring to the contrastive loss L contrastive .

[0014] Step 6, design a loss combination scheme, and obtain the mixed loss L by weighted combination of different losses with certain coefficients hybrid ;

[0015] Step 7, calculate the single-layer gradient of the mixed loss for each time step, and update the weights W of the current layer online and in real time l ;

[0016] Step 8, after the training stage of the hierarchical local mixed loss algorithm ends, in the test stage, use the trained SNN to complete classification prediction on the task dataset (image classification), save the model, and record the experimental results.

[0017] In step 1 of the present invention, the original pictures of the image classification dataset are processed. The pixel values are normalized from the interval [0, 255] to the interval [0, 1], and then the data is standardized according to the global mean and standard deviation calculated for the entire training set, obtaining a data distribution with a mean of 0 and a variance of 1. For some datasets, data augmentation operations such as Random Crop and Random Horizontal Flip are also adopted.

[0018] In step 2 of the present invention, when constructing the spiking neural network model, we adopt the most common LIF neuron model:

[0019]

[0020] Among them, and respectively represent the membrane potential and the possible emitted spikes of neuron i in the l-th layer at time step t. τ is the membrane potential decay constant, is the weight connection from the j-th neuron in the previous layer to the i-th neuron in the current layer, v th is the firing threshold of the neuron, minus This term reflects the soft reset method of the membrane voltage. The pulse coding method is the Direct Coding method.

[0021] Step 3 of the present invention includes:

[0022] Step 3-1: Add an Auxiliary Classifier to each hidden layer of the spiking neural network. The Auxiliary Classifier takes the output spike train of the current layer as input and maps it to the prediction of the final label of the entire network, in the following form:

[0023]

[0024] where, y * (t) is the prediction of the final classification label made by the Auxiliary Classifier of the current layer at the current time step; Flatten(·): R B×C×H×W →R B×(C·H·W) is a flattening function (only effective for the spiking convolutional layer). B, C, H, and W respectively represent the batch size, the number of channels, the height and width of the feature. It can change the dimension of the feature map of a certain hidden layer to adapt to the input of the linear mapping; is a weight matrix, that is, the linear mapping;

[0025] Step 3-2: After obtaining the prediction of the Auxiliary Classifier, define the Cross-Entropy prediction loss L predict as follows:

[0026]

[0027] where, y represents the true label value, T represents the total number of time steps, and the Cross-Entropy prediction loss L predict measures the ability of the features of the current layer to judge the classification task.

[0028] Step 4 of the present invention includes:

[0029] Step 4-1: Calculate the similarity matrix of the feature representations of the current hidden layer and the similarity matrix of the true labels. Specifically, for the samples in a batch, its one-hot label matrix is denoted as Y = [y1,... yB] ∈ R N×B , where B and N respectively represent the batch size and the number of classes, then the label similarity matrix S(Y) ∈ R B×B can be obtained by the following formula:

[0030]

[0031] Similarly, a similarity matrix is also calculated for the feature representation of the current hidden layer. The feature representation of the current layer is H = [h1,..., h B , where h i is the feature representation of the i-th sample in the batch at the current layer. Through the learnable feature transformation layer E θ , more compact and effective features E θ (H) are extracted from the current feature H. Furthermore, the similarity matrix S(E θ (H)) ∈ R B×B is obtained, where S(·) represents the Cosine similarity calculation process. The element s θ in the similarity matrix s(E ij (H)) represents the cosine similarity between the features of samples i and j, and its formula is as follows:

[0032]

[0033] where, represents the more compact feature obtained by passing the feature h i of the i-th sample at the current layer through the feature transformation layer E θ ;

[0034] Step 4-2, after obtaining the similarity matrix of the labels and the feature representation of the current layer, the similarity matching loss can be defined (reference: A, EIDNES L H.Training neural networks with local error sig-nals[C] / / International conference on machine learning.2019:4839-4850.):

[0035]

[0036] where, ||·|| F represents the Frobenius norm, which is a matrix norm for measuring the size of a matrix. The similarity matching loss L similarity measures the degree of consistency between the similarity structure of different samples in a batch at the current layer in the sample space and the similarity structure of the corresponding samples in the target labels.

[0037] In step 5 of the present invention, other types of optional loss functions are defined, mainly the contrastive loss L contrastive . The contrastive loss function can bring closer similar samples and push away different samples, and its function expression is as follows:

[0038]

[0039] where y i,j ∈ {0, 1}, and y i，j = 1 indicates that a and j belong to the same class, and y i，j = 0 indicates that i and j belong to different classes; D i，j represents the Euclidean distance between sample i and sample j; m represents the margin, which is the minimum distance requirement between different-class samples. For positive sample pairs (y i，j = 1), optimizing L contrastive aims to minimize the distance between samples of the same class; for negative sample pairs (y i，j = 0), optimizing L contrastive aims to maximize the distance between different-class samples, at least reaching m.

[0040] In step 6 of the present invention, for the above three loss functions, a weighted combination is performed. The specific approach is to manually set the weight coefficients of different losses so that the scales of different loss components in the combined loss are approximately the same. The following several optional combined losses can be obtained:

[0041] L hybrid1 = αL predict + (1 - α)L similarity

[0042] L hybrid2 = βL predict + (1 - β)L contrastive

[0043] L hybrid3 = γL predict + δL similarity + (1 - γ - δ)L contrastive

[0044] In step 7 of the present invention, for the combined local loss obtained in step 6, the Adam optimizer is used for single-layer gradient update. The combined loss of each layer only affects the weight update of the current pulse layer and does not affect other layers, belonging to the category of hierarchical local learning. In particular, for L hybrid1 , which consists of two components summed in the time dimension, real-time online update can be performed for each moment. Its single-layer local weight update formula at time t is as follows:

[0045]

[0046] In step 8 of the present invention, after the stage of training the model with several mixed local losses in step 7 ends, the model is saved. In the testing stage, predictions are made on the downstream dataset (image classification) using several mixed local losses respectively, the experimental data is recorded, and the effects of several losses are compared to select the best model and weight coefficients.

[0047] Beneficial effects:

[0048] 1. The present invention designs a hierarchical local learning spiking neural network training algorithm, which overcomes to a certain extent the problem of difficult training of spiking neural networks due to the non-differentiability of the spike function, and performs well in terms of biological rationality, computational cost, and accuracy, providing a new idea and feasible solution for solving the training problem of neural networks.

[0049] 2. The present invention belongs to a non-BP spiking neural network training algorithm. Compared with the BP algorithm, it avoids a series of problems such as weight symmetry, update locking, memory occupation, gradient disappearance / explosion, and low biological interpretability, and has higher hardware friendliness and biological rationality.

[0050] 3. The present invention adopts a mixed loss combination scheme. Different types of losses provide optimization supervision signals for each layer of the spiking neural network from three complementary perspectives of "label supervision, sample structure, and sample pair distance", improving the convergence speed and final accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The following further specifically describes the present invention in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0052] Figure 1 It is a schematic flowchart of the present invention.

[0053] Figure 2 It is a schematic diagram of the algorithm principle of the mixed loss local learning spiking neural network algorithm proposed by the present invention.

[0054] Figure 3 It is a schematic diagram of the comparison experiment results of different loss combinations proposed by the present invention on the CIFAR-10 dataset.

[0055] Figure 4 It is a schematic diagram of the test results when the method proposed by the present invention is verified by examples on the MNIST, Fashion-MNIST, and CIFAR-10 datasets. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The following further describes the present invention in conjunction with the drawings and embodiments.

[0057] As Figure 1As shown in the figure, a biologically plausible hierarchical local hybrid loss pulse neural network training algorithm includes 8 steps:

[0058] In step 1, the original pictures of the image classification dataset are preprocessed. The pixel values are normalized from [0, 255] to the [0, 1] interval, and then the data in the [0, 1] interval is standardized according to the global mean and standard deviation obtained from the training set statistics, resulting in a data distribution with a mean of 0 and a variance of 1. For some more complex datasets, data augmentation operations such as random cropping (RandomCrop) and random horizontal flipping (RandomHorizontalFlip) are also used to enhance the diversity of samples.

[0059] In step 2, a pulse neural network model is constructed. We adopt the LIF neuron model based on the soft reset method:

[0060]

[0061] Among them, and respectively represent the membrane potential and the possible emitted pulse of neuron i in the l-th layer at time step t. τ is the membrane potential decay constant, is the weight connection from neuron j in the previous layer to neuron i in the current layer, v th is the pulse emission threshold of the neuron, and subtracting this term reflects the soft reset method of the membrane voltage. Compared with directly resetting the membrane potential to a fixed value in the hard reset, the soft reset retains more potential information of the part exceeding the threshold.

[0062] The pulse coding method is the direct coding (Direct Coding) method, that is, instead of explicitly encoding the preprocessed data (from real values to pulse values), it is directly input into the first layer of the network to retain as much original information of the data as possible.

[0063] Step 3 includes the following steps:

[0064] Step 3-1, as Figure 2 shown in the figure, an auxiliary classifier is added to each hidden layer (including the convolutional layer and the fully connected layer) of the pulse neural network. The input of the auxiliary classifier is the output pulse sequence of the current hidden layer, and the output of the auxiliary classifier is the predicted value of the final label of the entire network, and its form is as follows:

[0065]

[0066] Among them, y *(t) is the prediction made by the auxiliary classifier in the hidden layer at the current time step for the final classification label; Flatten(·): R B×C×H×W →R B×(C·H·W ) is a flattening function (only effective for the spiking convolutional layer, not required for the fully connected layer). B, C, H, and W represent the batch size, the number of channels, the height, and the width of the features respectively. It can change the dimension of the feature map of any shape in a certain hidden layer to adapt to the input of the linear mapping; is a weight matrix, whose height is equal to the number of categories N of the classification task, and whose width is equal to the feature dimension after the flattening operation of the output of the current layer.

[0067] Step 3-2, after obtaining the prediction of the auxiliary classifier, define the cross-entropy prediction loss L predict , and the expression is as follows:

[0068]

[0069] Among them, y represents the true label value, and the cross-entropy prediction loss L predict measures the discriminative ability of the features of the current layer for the classification task. Optimizing this loss helps to improve the accuracy of the SNN in the classification task.

[0070] Step 4 includes the following steps:

[0071] Step 4-1, as Figure 2 shown, calculate the similarity matrix of the feature representation of the current hidden layer and the similarity matrix of the true labels. The specific method is that for B samples in a batch, record their one-hot label matrix as Y = [y1,...,y B ∈R N×B , where N represents the number of categories, then the label similarity matrix S(Y)∈R B×B can be constructed in the following way:

[0072]

[0073] Similarly, calculate the similarity matrix for the feature representation of the current hidden layer. The feature representation of the current layer is H = [h1,...,h B , where h i is the feature representation of the i-th sample in the batch at the current layer. Through the learnable feature transformation layer E θ (θ represents the learnable parameter), extract more compact and effective features E θ (H) from the current feature H to reduce the computational complexity of the subsequent similarity matrix. If the current layer is a spiking convolutional layer, the transformation layer E θ is also in the form of a convolutional layer; if the current layer is a spiking fully connected layer, then the transformation layer Eθ It is also in the form of a fully connected layer. Then, the similarity matrix S(E θ (H)) ∈ R B ×B is obtained, where S(·) represents the Cosine similarity calculation process. The element s θ in the similarity matrix S(E ij (H)) represents the cosine similarity between the features of samples i and j, and its formula is as follows:

[0074]

[0075] where, represents the more compact feature obtained by transforming the feature h i of the i-th sample in the current layer through the feature transformation layer E θ .

[0076] Step 4-2, after obtaining the similarity matrix between the labels and the current layer feature representations, the similarity matching loss can be defined:

[0077]

[0078] where ||.|| F represents the Frobenius norm, which is a matrix norm for measuring the size of a matrix. The similarity matching loss L similarity measures the degree of consistency between the geometric structure of the sample space of different samples in a batch of the current layer and the geometric structure of the corresponding samples in the target labels, improving the consistency of the feature structures between layers. This loss optimizes the network from a perspective completely different from the cross-entropy prediction loss.

[0079] In Step 5, other types of optional loss functions are defined, namely the contrastive loss L contrastive . The contrastive loss function can bring similar samples closer and push different samples farther away. Its function expression is as follows:

[0080]

[0081] where, y i,j ∈ {0, 1}, y i,j = 1 indicates that i and j belong to the same class, and y i,j = 0 indicates that i and j belong to different classes; D i,j is the value of the i-th row and j-th column in the Euclidean distance matrix D between samples, representing the Euclidean distance between samples i and j; m represents the margin, indicating the minimum distance requirement between different-class samples. For positive sample pairs (y i，j = 1), the effect of optimizing L contrastive is to minimize the distance between similar samples; for negative sample pairs (yi，j = 0), optimize L contrastive The effect of contrastive is to maximize the distance between heterogeneous samples, at least reaching m. The contrastive loss measures the relative positional relationship between positive and negative sample pairs, making the feature space clearer at the class boundary.

[0082] In step 6, for the above-mentioned multiple loss functions, perform weighted combination, as Figure 2 shown. The specific approach is to manually set the weight coefficients of different losses based on the principle of consistent loss scale, so that the scales of different loss components in the mixed loss are roughly equivalent. The following several optional mixed losses can be obtained:

[0083] L hybrid1 = αL predict + (1 - α)L similarity

[0084] L hybrid2 = βL predict + (1 - β)L contrastive

[0085] L hybrid3 = γL predict + δL similarity + (1 - γ - δ)L contrastive

[0086] In step 7, for the mixed local loss obtained in step 6, use the Adam optimizer to perform single-layer gradient update. The mixed loss of each layer only affects the weight update of the current pulse layer and does not affect other layers. Therefore, it is called hierarchical local learning, which avoids the problems brought by the global gradient backpropagation layer by layer. It is worth mentioning that for the mixed loss L hybrid1 , it consists of two components summed in the time dimension. Therefore, real-time online update can be performed for each moment. The single-layer local weight update formula at time t is as follows:

[0087]

[0088] In step 8, after the stage of training the model separately with several mixed local losses in step 7, save the model. In the testing stage, use several mixed local losses to perform predictions on the downstream dataset (image classification), record the experimental data and compare the effects of several losses to determine the best model and weight coefficients.

[0089] Embodiment

[0090] To verify the effectiveness of the present invention, we conducted experimental verifications on three conventional static image classification datasets respectively, including the MNIST dataset, the Fashion-MNIST dataset, and the CIFAR-10 dataset. Among them, the MNIST dataset contains handwritten digits (0-9), a total of 10 categories, grayscale images with a size of 28x28 pixels, 60,000 images in the training set, and 10,000 images in the test set. It is a classic introductory dataset in the field of image classification; the Fashion-MNIST dataset contains pictures of various clothing, shoes, and hats, a total of 10 categories, including T-shirts, trousers, skirts, sports shoes, etc. Similar to MNIST, it is a grayscale image with a size of 28x28 pixels, 60,000 images in the training set, and 10,000 images in the test set; the CIFAR-10 dataset contains color images of 10 categories of objects, such as airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks. The images are color images with 3 RGB channels and a size of 32x32 pixels, 50,000 images in the training set, and 10,000 images in the test set.

[0091] To find the best loss combination scheme and proportional coefficient, we conducted experimental verifications on the CIFAR-10 dataset for a total of 7 cases, including the cases of training various losses separately and the cases of combining losses with each other. The experimental results are as Figure 3 shown. Among them, the cross-entropy prediction loss L predict and the similarity matching loss L similarity combined with a proportional coefficient of 0.99 and 0.01 can achieve the highest prediction accuracy and is the best model in our method.

[0092] To verify the effectiveness of the algorithm proposed in this invention for training SNN, we compared this algorithm with some existing SNN training algorithms, and the comparison metric was the test accuracy. The comparison methods included spike-timing-dependent plasticity (STDP), global surrogate gradient descent method (Reference: WU Y, DENG L, LI G, et al. Spatio-temporal backpropagation for training high-performance spiking neural networks[J]. Frontiers in neuroscience, 2018, 12: 331.), direct feedback alignment method (Reference: ZHAO D, ZENG Y, ZHANG T, et al. GLSNN: A multi-layer spiking neural network based on global feedback alignment and local STDP plasticity[J]. Frontiers in Computational Neuroscience, 2020, 14: 576841.), balanced propagation method, supervised local learning method (Reference: MAC, XU J, YU Q. Temporal dependent local learning for deep spiking neural networks[C] / / 2021 International joint conference on neural networks (IJCNN). 2021: 1-7.).

[0093] After testing, we obtained the best model L of the hybrid local loss hybrid1 and the comparison results with the above-mentioned several spiking neural network comparison training algorithms, as Figure 4 shown. Figure 4The name, technical principle category, network structure used, and accuracy rate on the test set are given for each method. First is the MNIST dataset, with the network structures being the fully connected network FCN and the convolutional network CNN. Our method achieved a test accuracy of 99.35% on the CNN, which is not only better than other local learning methods for spiking neural networks (DECOLLE and TSLL), but also exceeds one of the mainstream training algorithms for spiking neural networks, the global surrogate gradient descent method. Secondly, for the Fashion-MNIST dataset, the best model of our hybrid local loss method achieved a test accuracy of 93.46%, significantly better than several other methods in the control experiment. Finally, the results of the comparative experiment on the more complex dataset CIFAR-10 are presented. Our experiment on CIFAR-10 used a convolutional spiking neural network with the VGG structure, and several other works for comparison also used a similar network structure. The best model L of our hybrid local loss method hybrid1 could achieve an accuracy rate of up to 91.44% on CIFAR-10, which is better than most existing methods. The above experimental results show that our algorithm avoids the vanishing / exploding gradients and memory overhead caused by global direction propagation, as well as the approximation errors brought by the surrogate function with the increase in the number of layers and time steps, while also achieving better model accuracy. This means that our method, as a biologically plausible local learning algorithm, can perform well in terms of computational memory overhead, model accuracy, etc., achieving a good balance in several aspects.

[0094] The present invention provides a biologically plausible hierarchical local hybrid loss spiking neural network training algorithm. There are many methods and ways to specifically implement this technical solution. The above description is only the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented using existing technologies.

Claims

1. A biologically plausible hierarchical local hybrid loss pulse neural network training algorithm, characterized in that, It includes the following steps: Step 1: Preprocess the dataset, including normalizing the pixel values to the range [0, 1], and performing data augmentation operations such as random cropping and random horizontal flipping on some of the dataset; Step 2: Construct a discrete LIF spiking neuron model based on soft reset, and adopt Direct Coding for the spike coding method; Step 3: Add an auxiliary classifier to each hidden layer of the spiking neural network. The function is to predict the final classification result using the output value of the current hidden layer, and calculate the cross-entropy prediction loss L of the auxiliary classifier predict ; Step 4, calculate the similarity matrix of the feature representations of the current hidden layer and the similarity matrix of the true labels, and obtain the similarity matching error L between the two similarity matrices similarity ; Step 5, calculate other types of optional loss functions, mainly referring to the contrastive loss L contrastive . Step 6, design a loss combination scheme, and obtain the mixed loss L by weighted combination of different losses with a certain coefficient hybrid ; Step 7: Calculate the single-layer gradient of the hybrid loss for each time step and update the weights W of the current layer online in real time l ; Step 8: After the training phase of the hierarchical local mixed loss algorithm ends, use the trained SNN to complete classification prediction on the image classification task dataset during the test phase, save the model, and record the experimental results.

2. The biologically plausible hierarchical local mixed-loss spiking neural network training algorithm according to claim 1, wherein, In Step 1, process the original images of the image classification dataset, normalize the pixel values, and then standardize the data according to the global mean and standard deviation calculated from the entire training set to obtain a data distribution with a mean of 0 and a variance of 1. For some of the dataset, data augmentation operations such as RandomCrop and RandomHorizontalFlip are also adopted.

3. The method according to claim 2, wherein In Step 2, construct the LIF spiking neuron model: Among them, and respectively represent the membrane potential and the possible spike of neuron i in the l-th layer at time step t. τ is the membrane potential decay constant. is the weight connection from the j-th neuron in the previous layer to the i-th neuron in the current layer. v th is the spike firing threshold of the neuron, and v reset is the reset potential. The pulse coding method is the direct coding (DirectCoding) method.

4. The method according to claim 3, characterized in that, Step 3 includes: Step 3-1: Add an auxiliary classifier to each hidden layer of the spiking neural network. The auxiliary classifier takes the output spike train of the current layer as input and maps it to the prediction of the final label of the entire network, and its form is as follows: where y * (t) is the prediction made by the auxiliary classifier of the current layer at the current time step for the final classification label; Flatten(·): R B×C×H×W →R B×(C·H·W) represents the flattening function (only effective for the pulse convolutional layer), where B, C, H, and W represent the batch size, the number of channels, the height, and the width of the feature, respectively; is a weight matrix, i.e., a linear mapping; Step 3-2, define the cross-entropy prediction loss L predict , and the expression is as follows: Among them, y represents the true label value, and the cross-entropy prediction loss L predict measures the judgment ability of the current layer features for the classification task.

5. The method according to claim 4, wherein Step 4 includes: Step 4-1, calculate the similarity matrix of the feature representations of the current hidden layer and the similarity matrix of the true labels. For the samples in a batch, its one-hot label matrix is denoted as Y = [y1,..., y B ∈ R N×B , where B and N represent the batch size and the number of classes respectively, then the similarity matrix S(Y) ∈ R B×B can be obtained by the following formula: Calculate the similarity matrix for the feature representation of the current hidden layer. The feature representation of the current layer is H = [h1,...,h B , where h i is the feature representation of the i-th sample in the batch at the current layer. Through the learnable feature transformation layer E θ , obtain a more compact and effective feature E θ (H). Furthermore, obtain the similarity matrix S(E θ (H)) ∈ R B×B , where S(·) represents the Cosine similarity calculation process. The elements s θ in the similarity matrix S(E ij The formula is as follows: Among them, s ij represents the cosine similarity between the features of samples i and j, represents the feature h of the i-th sample in the current layer i after passing through the feature transformation layer E θ to obtain a more compact feature; Step 4-2: Calculate the similarity matching loss for the similarity matrix of the label and the current layer feature representation; Among them, ||·|| F represents the Frobenius matrix norm. The similarity matching loss L similarity measures the degree of consistency between the similarity structure of different samples in a batch in the sample space of the current layer and the similarity structure of the corresponding samples in the target labels.

6. The method according to claim 5, wherein In step 5, define the contrastive loss L contrastive , and its functional expression is as follows: where y i,j ∈ {0, 1}, y i,j = 1 indicates that i and j belong to the same class, and y i，j = 0 indicates that i and j belong to different classes; D i，j represents the Euclidean distance between sample i and sample j; m represents the margin, which is the minimum distance requirement between different-class samples.

7. The method according to claim 6, characterized in that, In Step 6, perform weighted combination for several loss functions. Set the weight coefficients of different losses so that the scales of different loss components in the mixed loss are roughly equivalent. The following several mixed loss functions can be obtained: L hybrid1 = αL predict + (1 - α)L similarity L hybrid2 = βL predict + (1 - β)L contrastive L hybrid3 = γL predict + δL similarity +(1 - γ - δ)L contrastive 。 8. The method according to claim 7, wherein In step 7, the Adam optimizer is used to perform separate gradient updates on the mixed local loss. The mixed loss of each layer only affects the weight update of the current pulse layer and does not affect other layers. In particular, for the mixed loss L hybrid1 , both of its two components are in the form of summation over the time dimension, enabling real-time online updates at each moment. The single-layer local weight update formula at time t is as follows: 。 9. The method according to claim 8, wherein, In Step 8, train the model with several mixed local losses, save the model after the training phase ends. During the test phase, make predictions on the task dataset respectively, record the experimental data and compare the effects of several loss combinations to determine the best loss combination model and weight coefficients.

Images