An image classification method and apparatus based on spiking neural networks

Abstract

This invention provides an image classification method and apparatus based on a spiking neural network. The method includes: acquiring image information of an image to be processed; inputting the image information into a neuron model of a target spiking neural network for processing to obtain a processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of a neuron model of an artificial neural network to a neuron model of a preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network; and classifying the image according to the processing result to obtain a classification result. The solution of this invention improves the performance of image classification at low time steps using the converted spiking neural network.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to an image classification method and apparatus based on a spiking neural network. Background Technology

[0002] Spike Neuron Networks (SNNs), as third-generation neural networks, are becoming increasingly well-known for their low power consumption and biological reliability. However, their discrete nature makes SNNs difficult to train directly. Researchers have proposed various methods to address this issue, but most require long time steps to achieve relatively high accuracy. The complexity of SNNs hinders their practical application to some extent. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an image classification method and apparatus based on spiking neural networks, which solves the problem of low accuracy of spiking neural networks in image processing at low time steps.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0005] An image classification method based on spiking neural networks includes:

[0006] Obtain image information of the image to be processed;

[0007] The image information is input into the neuron model of the target spiking neural network for processing to obtain the processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network.

[0008] Based on the processing results, the image is classified to obtain the classification result.

[0009] Optionally, the training process of the neuron model of the target spiking neural network includes:

[0010] Obtain the neuron model of the artificial neural network and the neuron model of the preset spiking neural network; wherein, the output of the l-th layer of the neuron model of the artificial neural network is: a l+1 =max(W l a l +b l ,0); where max(x,0) represents the activation function used in the neuron model of the artificial neural network, a l Let a represent the input of the l-th layer.l+1 W represents the input of the (l+1)th layer. l b represents the weight of the l-th layer of the neuron model of the artificial neural network. l This represents the bias of the l-th layer of the neuron model of the artificial neural network;

[0011] The equation for the change in the first membrane potential of the input at time t in the l-th layer of the preset spiking neural network is determined as follows: in, It is the temporary membrane potential of the l-th layer at time t, v l (t-1) is the membrane potential of the l-th layer at time (t-1), x l (t) is the pulse input at time t in the l-th layer;

[0012] Based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined;

[0013] Training is performed using a training set based on the aforementioned membrane potential change equation.

[0014] Optionally, based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined, including:

[0015] Based on the pulse firing mechanism, the equation for the change of the second membrane potential at time t in the neuron model of the pre-defined spiking neural network is determined as follows:

[0016] Among them, v l (t) represents the membrane potential of the neuron model in the pre-defined spiking neural network at time t in layer l, x l (t) represents the pulse input at time t of the l-th layer of the preset spiking neural network neuron model. θ represents the threshold of the neuron in the l-th layer of the predefined spiking neural network model. l (t) is a piecewise function used to determine whether to fire a pulse, described as follows: if The value is greater than Then θ l A value of 1 for θ indicates that a pulse is emitted; otherwise, θ l A value of 0 for (t) indicates that no pulse is emitted;

[0017] The membrane potential at time t of the neuron model of the preset spiking neural network is recursively processed to obtain the equation for the change of the third membrane potential at time T.

[0018] Among them, V l(0) represents the membrane potential at time 0 of the l-th layer of the preset spiking neural network neuron model, V l (T) represents the membrane potential at time T in the l-th layer of the preset spiking neural network neuron model. This represents the pulse sequence output of the l-th layer of the neuron model of the preset spiking neural network.

[0019] Optionally, training is performed using a training set based on the membrane potential change equation, including:

[0020] Based on the membrane potential change equation, determine the error term;

[0021] Based on the error term, determine the initial membrane potential of the neuron model of the preset spiking neural network;

[0022] Obtain the encoding compensation value of the neuron model of the preset spiking neural network;

[0023] Based on the encoded compensation value, the initial membrane potential of the neuron model of the preset spiking neural network is compensated to obtain the neuron model of the target spiking neural network after eliminating the conversion error.

[0024] Optionally, based on the membrane potential change equation, the error term is determined, including:

[0025] Based on the aforementioned equation for the change in the third membrane potential, a pulse rate formula for the l-th layer is introduced. And a formula is introduced based on threshold balancing technology. The formula for the pulse rate of layer l-1 is obtained. Wherein, the pulse rate formula of the l-th layer represents the rate at which pulses are emitted during the period from time 0 to time T of the l-th layer;

[0026] According to the pulse rate formula r of the (l-1)th layer l-1 And the pulse rate formula r of the l-th layer l Combining the third membrane potential change equation, the first pulse rate equation is obtained.

[0027] Based on the first pulse rate equation, the error term is determined as follows:

[0028] Optionally, determining the initial membrane potential of the neuron model of the preset spiking neural network based on the error term includes:

[0029] Using the weights of the artificial neural network, the sum of membrane potentials from time 0 to time T in each layer of the neuron model of the preset spiking neural network is determined by the formula sum_mem=ANN.weight[l]*T+ANN.bias[l]; where sum_mem is the sum of membrane potentials, ANN.weight[l] is the weight of the l-th layer of the neuron model of the artificial neural network, T is the length of the pulse sequence, and ANN.bias[l] is the bias of the l-th layer of the neuron model of the artificial neural network;

[0030] Through formula Determine the residual membrane potential; where mem is the residual membrane potential and % is the modulo operator. is the threshold of the neurons in layer l.

[0031] The initial membrane potential is determined by the formula init_mem[l] = mem.mean(), where init_mem[l] is the initial membrane potential of the l-th layer, and mean() is the function for calculating the average value.

[0032] Optionally, determining the initial membrane potential of the neuron model of the preset spiking neural network based on the error term includes:

[0033] The weight parameters and biases of the neuron model of the artificial neural network are converted into the neuron model of the preset spiking neural network;

[0034] Using the training set, the residual membrane potential is determined by the formula mem = l.mem.mean(); where mem is the average residual membrane potential of the training set in the l-th layer, and mean() is the function for calculating the average value.

[0035] The cumulative sum of the residual membrane potentials is determined by the formula mem_sum[l]=mem_sum[l]+mem; where mem_sum[l] is the cumulative sum of the average values ​​of the residual membrane potentials of the l-th layer obtained from each image in the training set;

[0036] The initial membrane potential of the neuron model of the preset spiking neural network is obtained by averaging the sum of the remaining membrane potentials in the l-th layer of the training set using the formula init_mem[l] = men_sum[l] / len(datasets); where init_mem[l] is the initial membrane potential of the neuron model in the l-th layer of the spiking neural network, and len(datasets) is the total number of images in the preset training set.

[0037] Optionally, obtaining the encoding compensation values ​​of the neuron model of the preset spiking neural network includes:

[0038] The second pulse rate equation is determined based on the initial membrane potential of the neuron model of the preset spiking neural network.

[0039] Introducing preset formulas The preset formula represents the decoding of the pulse input of the l-th layer of the preset spiking neural network neuron model. Substituting it into the second pulse rate equation, the output equation of the l-th layer is obtained as X. l =W l x l-1 +b l ;

[0040] Based on the output equation of the l-th layer, the error equation for the encoding method is determined as: Δa l =a l -X l =W l a l-1 +b l -(W l X l-1 +b l ) = W l (a l-1 -X l-1 );

[0041] Based on the rate coding type, the approximate values ​​before and after coding are obtained as follows: Among them, a l X represents the input of the l-th layer of a neuron model in an artificial neural network. l x represents the decoding of the pulse input of the l-th layer of the neuron model of the predefined spiking neural network. l (t) represents the pulse input at time t in the l-th layer;

[0042] Based on the approximation and the error equation, the coding compensation value is determined to be...

[0043] Optionally, the initial membrane potential of the neuron model of the preset spiking neural network is compensated according to the encoded compensation value, including:

[0044] The formula init_mem[l] = init_mem[l] + V is used. th [l] / T, the encoded compensation value is added to the initial membrane potential of the neuron model of the preset spiking neural network.

[0045] The present invention also provides an image classification device based on a spiking neural network, comprising:

[0046] The acquisition module is used to acquire image information of the image to be processed;

[0047] The processing module is used to input the image information into the neuron model of the target spiking neural network for processing to obtain the processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network. Based on the processing result, the image is classified to obtain the classification result.

[0048] The above-described solution of the present invention has at least the following beneficial effects:

[0049] The above-described solution of the present invention obtains image information of the image to be processed; inputs the image information into a neuron model of a target spiking neural network for processing to obtain a processing result; inputs the image information into a neuron model of a target spiking neural network for processing to obtain a processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of an artificial neural network to the neuron model of a preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network; and classifies the image according to the processing result to obtain a classification result; this solves the problem of low accuracy of spiking neural networks when processing images at low time steps and improves the performance of the converted spiking neural network in image classification at low time steps. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the image classification method based on a spiking neural network according to an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of a neuron model of an artificial neural network according to an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a neuron model of a preset spiking neural network according to an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the network architecture of the Artificial Neural Network (ANN) in a specific embodiment provided by the present invention;

[0054] Figure 5 This is a schematic diagram of the network architecture of the spiking neural network (SNN) in a specific embodiment provided by the present invention;

[0055] Figure 6 This is a schematic diagram of the process for determining the initial membrane potential and encoding compensation in a specific embodiment provided by the present invention;

[0056] Figure 7 This is a schematic diagram of the target spiking neural network in a specific embodiment provided by the present invention;

[0057] Figure 8 This is a schematic diagram illustrating the output accuracy of different neural networks at different step sizes in a specific embodiment provided by the present invention;

[0058] Figure 9 This is a schematic diagram showing the relationship between the time to first peak value and the number of layers in a specific embodiment of the present invention, for an SNN without initial membrane potential and an IC-SNN with initial membrane potential.

[0059] Figure 10 This is a schematic diagram of the output of the pixels of the image to be processed in the first layer of the ANN network in a specific embodiment provided by the present invention;

[0060] Figure 11 This is a schematic diagram of the output of the pixels of the image to be processed in the first layer of the IC-SNN network in a specific embodiment provided by the present invention;

[0061] Figure 12 This is a schematic diagram of the output of the pixels of the image to be processed in the first layer of the SNN network in a specific embodiment provided by the present invention;

[0062] Figure 13 This is a schematic diagram of the output of the pixels of the image to be processed in the third layer of the ANN network in a specific embodiment of the present invention;

[0063] Figure 14 This is a schematic diagram of the output of the pixels of the image to be processed in the third layer of the IC-SNN network in a specific embodiment of the present invention;

[0064] Figure 15 This is a schematic diagram of the output of the pixels of the image to be processed in the third layer of the SNN network in a specific embodiment of the present invention;

[0065] Figure 16 This is a schematic diagram of the structure of an image classification device based on a spiking neural network according to an embodiment of the present invention. Detailed Implementation

[0066] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0067] like Figure 1As shown, an embodiment of the present invention proposes an image classification method based on a spiking neural network, comprising:

[0068] Step 11: Obtain image information of the image to be processed;

[0069] Step 12: Input the image information into the neuron model of the target spiking neural network for processing to obtain the processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network.

[0070] Step 13: Based on the processing results, classify the image to obtain the classification results.

[0071] In this embodiment, the image information of the image to be processed is input into the neuron model of the target spiking neural network to obtain the processing result. The target spiking neural network is a spiking neural network with initial membrane potential and coding compensation (IC-SNN) obtained by combining a spiking neural network (SNN) and an artificial neural network (ANN). The ANN and SNN have the same network structure. The target spiking neural network IC-SNN fully considers the errors generated during the conversion process between ANN and SNN and the influence of the coding method on the error, achieving high-accuracy image processing within a short time step. Then, the image is classified based on the processing result to obtain the classification result. The image classification result obtained in this way eliminates the error term generated by low latency and compensates for the information loss during the coding process, which can effectively reduce the conversion error and improve the performance of low time step image classification.

[0072] It should be noted that the image to be processed is preferably an industrial product image obtained in an industrial scene. The industrial product image can be a grayscale image, a binary image, or other types of images, and this application does not limit it to these. Through the image classification method based on spiking neural networks in steps 11 to 13, the industrial product image can be processed so that the image can be classified according to the processing result, and a more accurate classification result can be obtained.

[0073] In an optional embodiment of the present invention, the training process of the neuron model of the target spiking neural network in step 12 includes:

[0074] Step 121: Obtain the neuron model of the artificial neural network and the neuron model of the preset spiking neural network; wherein, the output of the l-th layer of the neuron model of the artificial neural network is: a l+1 =max(W l a l +b l ,0); where max(x,0) represents the activation function used in the neuron model of the artificial neural network, a l Let a represent the input of the l-th layer. l+1 W represents the input of the (l+1)th layer. l b represents the weight of the l-th layer of the neuron model of the artificial neural network. l This represents the bias of the l-th layer of the neuron model of the artificial neural network;

[0075] Step 122, determine the first membrane potential change equation of the input at time t of the l-th layer of the preset spiking neural network as follows: in, It is the temporary membrane potential of the l-th layer at time t, v l (t-1) is the membrane potential of the l-th layer at time (t-1), x l (t) is the pulse input at time t in the l-th layer;

[0076] Step 123: Determine the membrane potential change equation of the neuron model of the preset spiking neural network according to the pulse firing mechanism;

[0077] Step 124: Train using the training set according to the membrane potential change equation.

[0078] In this embodiment, the neuron model of the target spiking neural network is obtained by transforming the weight parameters and biases of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network, and training it using a training set; the neuron model of the artificial neural network and the neuron model of the preset spiking neural network are obtained; wherein, the neuron model of the artificial neural network preferably uses ReLU (rectified linear function) as the activation function, and the output of the l-th layer of the neuron model of the artificial neural network after the activation function ReLU is a. l+1 =max(W l a l +b l ,0), where max(x,0) represents the ReLU activation function used, a l Let a represent the input of the l-th layer. l+1 b represents the input of the (l+1)th layer. l W represents the bias of the l-th layer in the neuron model of an artificial neural network.l This represents the weights of the l-th layer in the neuron model of the artificial neural network;

[0079] The preferred neuron model for the pre-defined spiking neural network is the IF (Integrated-And-Fire) neuron. It accumulates stimulation of the presynaptic neuron; if a neuronal threshold is reached, a pulse is fired in the postsynaptic neuron. The IF neuron model of the pre-defined spiking neural network exhibits good fitting performance. The neuron model of this pre-defined spiking neural network is V. temp (t)=V(t-1)+X(t), where the pulse firing mechanism is, Where V(t-1) is the membrane potential at the previous time step (time t-1), X(t) is the input at the current time step (time t), and V temp (t) is the temporary membrane potential at time t. This represents the neuron threshold of a predefined spiking neural network neuron model; when V temp (t) exceeds the neuron threshold V th At that time, the neuron will fire a pulse, and V(t) will be generated by V. temp (t) is obtained by soft reset; otherwise, the neuron will not fire a pulse, and V(t) is the V at the current time (time t). temp (t), where the soft reset method refers to subtracting the corresponding threshold value from the current membrane potential after exceeding the threshold, i.e., V temp (t)-V th .

[0080] The differences between the neuron model of the artificial neural network and the neuron model of the pre-defined spiking neural network are explained below:

[0081] Figure 2 It demonstrates a neuron model of an artificial neural network. Figure 3 This demonstrates a neuron model of a pre-defined spiking neural network, such as... Figure 2 and Figure 3 As shown, the input to the neuron model of the artificial neural network is the actual value. This represents the i-th input of the (l-1)-th layer, with a corresponding weight of W. ij Then, the sum of multiple inputs and weights of the (l-1)th layer is processed through an activation function. The input of the l-th layer is obtained as Compared with actual values ​​in artificial neural networks Unlike direct input, the spiking neural network (SNN) uses a pulse sequence as input. After neuron integration, the membrane potential changes and pulses are emitted. The emitted pulse sequence is then used as the output of that layer. The values ​​in the pulse sequence are only 0 and 1, where 0 indicates no pulse was emitted and 1 indicates a pulse was emitted. Figure 3 The U shown th This represents the neuron threshold.

[0082] The neuron model of the preset spiking neural network will undergo a membrane potential change when it receives the input at time t in the l-th layer. The first membrane potential change equation for the input at time t in the l-th layer is... in, It is the temporary membrane potential of layer l at time t, that is, the sum of the membrane potential at the previous time (time t-1) and the input received at the current time (time t), v l (t-1) is the membrane potential of the l-th layer at the previous time (time t-1), x l (t) represents the pulse input at the current time (time t) of the l-th layer, b l This represents the bias of the l-th layer.

[0083] It should be noted that the first membrane potential change equation is an intermediate equation in the mathematical derivation process. Furthermore, in an optional embodiment of the present invention, step 123 includes:

[0084] Based on the pulse firing mechanism, the equation for the change of the second membrane potential at time t in the neuron model of the pre-defined spiking neural network is determined as follows: Among them, v l (t) represents the membrane potential of the neuron model in the pre-defined spiking neural network at time t in layer l, x l (t) represents the pulse input at time t of the l-th layer of the preset spiking neural network neuron model. θ represents the neuron threshold of the l-th layer of a preset spiking neural network model. This neuron threshold is obtained using a threshold balancing technique, which trains the artificial neural network using a training set to obtain the neuron threshold for each layer. l (t) is a piecewise function used to determine whether to fire a pulse, described as follows: if The value is greater than Then θ l A value of 1 for θ indicates that a pulse is emitted; otherwise, θ l A value of 0 for (t) indicates that no pulse is emitted;

[0085] The membrane potential at time t of the neuron model of the preset spiking neural network is recursively processed to obtain the equation for the change of the third membrane potential at time T. Among them, V l (0) represents the membrane potential at time 0 of the l-th layer of the preset spiking neural network neuron model, V l (T) represents the membrane potential at time T in the l-th layer of the preset spiking neural network neuron model. This represents the pulse sequence output of the l-th layer of the neuron model of the preset spiking neural network. In this embodiment, the final membrane potential change equation is based on the formula... The obtained result is achieved by judging the temporary membrane potential to ensure whether the neuron has fired a pulse, and then performing a soft reset. The piecewise function θ(t) is used to determine whether a pulse has been fired. l (t) is the output pulse sequence of the l-th layer and also the input pulse sequence of the (l+1)-th layer.

[0086] According to the membrane potential change equation, the input sequence and the pulse firing sequence can be accumulated during the period t∈(0,T) to obtain the corresponding values; that is, the equation for the change of the third membrane potential at time T can be described as: Among them, v l (0) represents the membrane potential of the l-th layer at time 0, v l (T) represents the membrane potential of the l-th layer at time T, θ l It is the output of layer l (the output of layer l+1).

[0087] In an optional embodiment of the present invention, step 124 includes:

[0088] Step 1241: Determine the error term based on the membrane potential change equation;

[0089] Step 1242: Determine the initial membrane potential of the neuron model of the preset spiking neural network based on the error term;

[0090] Step 1243: Obtain the encoding compensation value of the neuron model of the preset spiking neural network;

[0091] Step 1244: Based on the encoded compensation value, compensate the initial membrane potential of the neuron model of the preset spiking neural network to obtain the neuron model of the target spiking neural network after eliminating the conversion error.

[0092] In this embodiment, based on the equation of the second membrane potential change Determine the error term in the conversion process; determine the residual membrane potential v based on the weights or training set of the artificial neural network. l (T), thereby determining the initial membrane potential v of the neuron model of the spiking neural network. l(0); Errors are eliminated by using the initial membrane potential, thereby enabling the network to achieve higher accuracy;

[0093] In addition, the preset spiking neural network needs to encode the real value and pass the pulse sequence as input into the network. By analyzing the encoding and decoding process, the approximate error value of the encoding is obtained, the encoding compensation value of the spiking neural network is determined, and the compensation value is combined with the initial membrane potential to obtain the neuron model of the target spiking neural network.

[0094] In an optional embodiment of the present invention, step 1241 includes:

[0095] Step 12411: Based on the third membrane potential change equation, the pulse rate formula for the l-th layer is introduced. And a formula is introduced based on threshold balancing technology. The formula for the pulse rate of layer l-1 is obtained. Wherein, the pulse rate formula of the l-th layer represents the rate at which pulses are emitted during the period from time 0 to time T of the l-th layer;

[0096] Step 12412, according to the pulse rate formula r of the (l-1)th layer l-1 And the pulse rate formula r of the l-th layer l Combining the third membrane potential change equation, the first pulse rate equation is obtained.

[0097] Step 12413, based on the first pulse rate equation, determine the error term as follows:

[0098] In this embodiment, the pulse rate formula r of the l-th layer is based on the threshold balancing technique used in the network conversion process. l After processing, the pulse rate formula r for the (l+1)th layer is obtained. l-1 Therefore, the new equation for the membrane potential change at time T can be obtained as follows: The equation for the membrane potential change at the new time T can be further modified to obtain the first pulse rate equation. in, This is the remainder term of the equation, and also the error in the transformation process; it is identified as an error term. To achieve higher accuracy within a shorter time step, this error term needs to be eliminated. Analysis of this error term reveals:

[0099] If we take v in the remainder term l (0) and v l (T) are processed simultaneously to obtain v l The approximation of (T) is used as the initial membrane potential v. l(0), then v can be eliminated simultaneously. l (0) and v l (T), thereby eliminating the error term.

[0100] In an optional embodiment of the present invention, the initial membrane potential can be determined by an artificial neural network, and step 1242 includes:

[0101] Step 12421: Using the weights of the artificial neural network, determine the sum of membrane potentials from time 0 to time T in each layer of the preset spiking neural network model using the formula sum_mem=ANN.weight[l]*T+ANN.bias[l]; where sum_mem is the sum of membrane potentials, ANN.weight[l] is the weight of the l-th layer of the artificial neural network neuron model, T is the length of the pulse sequence, and ANN.bias[l] is the bias of the l-th layer of the artificial neural network neuron model.

[0102] Step 12422, using the formula Determine the residual membrane potential; where mem is the residual membrane potential and % is the modulo operator. The threshold of the neurons in layer l;

[0103] Step 12423: Determine the initial membrane potential using the formula init_mem[l] = mem.mean(); where init_mem[l] is the initial membrane potential of the l-th layer, and mean() is the function for calculating the average value.

[0104] In this embodiment, all membrane potentials from time 0 to time T are accumulated to obtain the sum of membrane potentials, sum_mem. Here, it is assumed that the firing distribution of the pulse sequence is more extreme, that is, the neuron fires a pulse every moment. Therefore, the sum of membrane potentials, sum_mem, can be used as the weighted accumulation within the time period from time 0 to time T, and this sum of membrane potentials, sum_mem, is used as the divisor. The neuron threshold is then used. As a divisor, through the formula The remainder is obtained because the reset operation after pulse delivery uses a soft reset, which subtracts the threshold value from the membrane potential. Therefore, the remainder can be regarded as the remaining membrane potential mem (i.e., v). l (T)); By determining the residual membrane potential and using it as the initial membrane potential, errors in the conversion process between artificial neural networks and spiking neural networks can be effectively eliminated, thereby improving the accuracy of the network.

[0105] In an optional embodiment of the present invention, the initial membrane potential can also be determined using a training set, and step 1242 includes:

[0106] Step 1242a: Convert the weight parameters and biases of the neuron model of the artificial neural network to the neuron model of the preset spiking neural network;

[0107] Step 1242b: Using the training set, determine the residual membrane potential using the formula mem = l.mem.mean(); where mem is the average value of the residual membrane potential in the l-th layer of the training set, and mean() is the function for calculating the average value.

[0108] Step 1242c: Determine the sum of residual membrane potentials using the formula mem_sum[l]=mem_sum[l]+mem; where mem_sum[l] is the sum of the average residual membrane potentials of the l-th layer obtained from each image in the training set;

[0109] Step 1242d: The average value of the accumulated residual membrane potentials in the training set at layer l is calculated using the formula init_mem[l] = men_sum[l] / len(datasets) to obtain the initial membrane potential of the neuron model of the preset spiking neural network; where init_mem[l] is the initial membrane potential of the neuron model at layer l of the spiking neural network, and len(datasets) is the total number of images in the preset training set.

[0110] In this embodiment, the weight parameters and biases of the neuron model of the artificial neural network are converted into the neuron model of the spiking neural network. The threshold balancing technique is used to train the neuron model of the preset spiking neural network using a training set. The average value of the residual membrane potential of the training set at layer l is determined by the formula mem = l.mem.mean(). The average value of the residual membrane potential obtained from each image in the dataset is accumulated. Finally, the average value of the residual membrane potential of each layer obtained from the training set is calculated by the formula init_mem[l] = men_sum[l] / len(datasets), and it is used as the initial membrane potential of the neuron model of the spiking neural network. In this training process, it is only necessary to focus on the forward propagation process of each layer in the network and record the residual membrane potential at time T. Then, the initial membrane potential can be given based on the record.

[0111] In one specific embodiment, under assumed extreme spiking conditions, the residual membrane potential is calculated based on the weights and bias parameters of the artificial neural network, and used as the initial membrane potential of the neuron model of the target spiking neural network. Specifically, it is obtained through the processing of Algorithm 1 below (where the text after " / / " is explanatory):

[0112] Algorithm 1 Initialized membrane potential based on trained ANN.

[0113] Input:ANN that has been trained; Simulation duration T; threshold that has been threshold-balanced; / / Input: trained artificial neural network (ANN); simulation time T; neuron thresholds obtained using threshold balancing.

[0114] Output: Initialized membrane potential init_mem; / / Output: Initial membrane potential init_mem;

[0115] Step 1: The threshold is modulated based on the weights in the trained ANN;

[0116] 2: init_mem = [] / / Step 2: Set an initial membrane potential init_mem;

[0117] 3: for l in layers do / / Step 3: When in layer l, execute the loop of steps 4-6 below;

[0118] 4: sum_mem=ANN.weight[l]*T+ANN.bias[l] / / Step 4: Accumulate all membrane potentials from time 0 to time T, and add the bias of the l-th layer to obtain the sum of membrane potentials sum_mem;

[0119] 5: mem = sum_mem% / / Step 5: Set the neuron threshold The sum of membrane potentials, sum_mem, is used as the divisor, and the remaining membrane potential is obtained as the dividend.

[0120] 6: init_mem[l] = mem.mean() / / Step 6: Average the multiple remaining membrane potentials on the l-th layer to obtain the initial membrane potential init_mem;

[0121] 7: end for / / Step 7: End the loop of steps 4-6 above;

[0122] 8: return init_mem; / / Step 8: Return to the initial membrane potential, completing the training of the initial membrane potential;

[0123] Furthermore, since data typically plays a crucial role in the training process of neural networks, introducing a training set and obtaining the initial membrane potential of neurons through the training set while discarding extreme discharges of neurons not only avoids the limitations of pulse firing conditions but is also more universal and performs better in practical applications, enabling the network to achieve higher accuracy.

[0124] The initial membrane potential based on the training set is obtained through the following Algorithm 2: (where the text after " / / " is explanatory content):

[0125] Algorithm 2 Initialized membrane potential based on training set.

[0126] Input: ANN that has been trained; SNN ready for transformation; Simulation duration T; training datasets;

[0127] Output: Initialized membrane potential init_mem; / / Output: Initial membrane potential init_mem;

[0128] Step 1: The weights of the trained ANN are converted into the SNN, and then the SNN is used to train on the training set, and the value of the remaining membrane potential is recorded.

[0129] 2: init_mem = [] / / Step 2: Set an initial membrane potential init_mem;

[0130] 3: mem_sum = [] / / Step 3: Set a residual membrane potential mem_sum;

[0131] 4: for l in layers do / / Step 4: When in layer l, execute the loop of steps 5-6 below;

[0132] 5: SNN.weight[l] = ANN.weight[l] / / Step 5: Convert the weights of the artificial neural network (ANN) to the spiking neural network (SNN);

[0133] 6: SNN.bias[ / ] = ANN.bias[ / ] / / Step 6: Convert the bias of the artificial neural network ANN to that of the spiking neural network SNN;

[0134] 7: end for / / Step 7: End the loop of steps 5-6 above;

[0135] Step 8: For the image data in the training set, perform steps 9-16.

[0136] 9: for t in T do do / / Step 9: When t is within time period T, execute the loop of steps 10-15 below;

[0137] 10: for l in Layer do / / Step 10: When in the l-th layer, execute the loop of steps 11-14 below;

[0138] 11: if t = T then / / Step 11: When t = T, execute the loop of steps 12-13 below;

[0139] 12: mem = l.mem.mean() / / Step 12: Calculate the average residual membrane potential of the image data in the training set at layer l;

[0140] 13: mem_sum[l]=mem_sum[l]+mem / / Step 13: Accumulate the average value of the remaining membrane potential of the l-th layer;

[0141] 14: end if / / Step 14: End the loop of steps 12-13 above;

[0142] 15: end for / / Step 15: End the loop of steps 11-14 above;

[0143] 16: end for / / Step 16: End the loop of steps 10-15 above;

[0144] 17: end for / / Step 17: End of steps 9-16 above;

[0145] 18: for l in Layer do / / Step 18: When in the l-th layer, execute the loop of step 19 below;

[0146] 19: init_mem[l] = mem_sum[l] / len(datasets) / / Step 19: Calculate the sum of the remaining membrane potentials of the training set in layer l and the number of images used for training, as the initial membrane potentials based on the training set;

[0147] 20: end for / / Step 20: End the loop of step 19 above;

[0148] 21: return init_mem; / / Step 21: Return the initial membrane potential init_mem. At this point, the training of the initial membrane potential based on the training set is complete.

[0149] In an optional embodiment of the present invention, step 1243 includes:

[0150] Step 12431: Determine the second pulse rate formula based on the initial membrane potential of the neuron model of the preset spiking neural network.

[0151] Steps 12432: Introduce the preset formula The preset formula represents the decoding of the pulse input of the l-th layer of the neuron model of the preset spiking neural network. Substituting it into the second pulse rate equation, the output equation of the l-th layer is obtained as X. l =W l x l-1 +b l ;

[0152] Based on the output equation of the l-th layer, the error equation for the encoding method is determined as: Δa l =a l -X l =W l a l-1 +b l -(W l X l-1 +b l ) = W l (a l-1 -X l-1 );

[0153] Based on the rate coding type, the approximate values ​​before and after coding are obtained as follows: Among them, a l X represents the input of the l-th layer of a neuron model in an artificial neural network. l x represents the decoding of the pulse input of the l-th layer of the neuron model of the predefined spiking neural network. l (t) represents the pulse input at time t in the l-th layer;

[0154] Step 12435: Determine the encoding compensation value based on the approximate value and the error equation.

[0155] In this embodiment, during further analysis, the error equation of the encoding was determined. To avoid information loss caused by rate encoding, the encoding needs to be compensated, and this compensation value is combined with the initial membrane potential of the spiking neuron; based on the equation above... The remaining items Approximating to 0, we obtain the simplified equation as follows: Right now Introducing formulas Get X l =W l x l-1 +b, where x l-1 This represents the input of the (l-1)th layer of the neuron model of the pre-defined spiking neural network; combined with the neuron model a of the artificial neural network. l+1 =max(W l a l +b l From the given information, we can obtain the error between the output of the artificial neural network neuron model and the preset spiking neural network neuron model:

[0156] Δa l =a l -X l =W l a l-1 +b l -(W l X l-1 +b l ) = W l (a l-1 -X l-1 );

[0157] It is evident that each layer after the first layer of encoding incurs information loss during the cumulative encoding process. Considering that when using rate coding, the approximate value of the input to the neuron model of the artificial neural network at the lth layer is... This allows us to identify potential information loss items during the encoding process, i.e., the encoding compensation value.

[0158] In an optional embodiment of the present invention, step 1234 includes:

[0159] Steps 1, 2, 3, 4, 1, are performed using the formula init_mem[l] = init_mem[l] + V. th [l] / T, the encoded compensation value is added to the initial membrane potential of the neuron model of the spiking neural network.

[0160] In this embodiment, in each layer of the target spiking neural network, the formula init_mem[l]=init_mem[l]+V is used. th [l] / T implements corresponding encoding compensation based on neuron threshold and time step T.

[0161] In another specific embodiment, based on the initial membrane potential, encoding compensation is performed in each layer of the target spiking neural network using the following algorithm 3 (wherein the text after " / / " is explanatory):

[0162] Algorithm 3 Coding compensation. / / Algorithm 3 is used for coding compensation;

[0163] Input: SNN after initializing membrane potential; Simulation duration T; / / Input: SNN after initializing membrane potential; Simulation time T;

[0164] Output: Initialized membrane potential init_mem; / / Output: Initial membrane potential init_mem;

[0165] Step 1: A compensation value is added to each layer of the SNN network that has initialized the membrane potential.

[0166] 2: for l in layers do / / Step 2: When in layer l, execute the loop of steps 3-5 below;

[0167] 3: init_mem[l] = SNN.init_mem[l] / / Step 3: Use the initial membrane potential of the l-th layer of the preset spiking neural network as the initial membrane potential of the l-th layer of the target spiking neural network;

[0168] 4:V th [l] = SNN.V th [l] / / Step 4: Use the neuron threshold of the l-th layer of the preset spiking neural network as the neuron threshold of the l-th layer of the target spiking neural network;

[0169] 5: init_mem[l] = init_mem[l] + V th [l] / T / / Step 5: Combine the initial membrane potential of the target spiking neural network in layer l with the encoded compensation value V th Adding [l] / T together yields the initial membrane potential of the target spiking neural network in the l-th layer after compensation;

[0170] 6: end for / / Step 6: End the loop of steps 3-5 above;

[0171] 7: return init_mem; / / Step 7: Return the initial membrane potential init_mem. At this point, the encoding compensation is completed, reducing information loss during the encoding process.

[0172] like Figure 4-7 As shown, in another specific embodiment, a training instance is initialized as follows: Figure 4 The artificial neural network (ANN) shown transforms the trained weights into a spiking neural network (SNN) and performs a threshold balancing operation to obtain the following result: Figure 5 The spiking neural network (SNN) shown (this SNN and ANN have the same network structure) is illustrated; for example... Figure 6 As shown, training is performed on the residual membrane potential at time T to obtain a suitable initial membrane potential. Considering the data error brought about by the encoding process, the encoding error of the artificial neural network and the spiking neural network is determined first, and then the encoding compensation value is set. Then, the encoding compensation value (Codingerror) is combined with the initial membrane potential to complete the encoding compensation.

[0173] like Figure 7 As shown, a target spiking neural network with a suitable initial membrane potential is thus obtained. This target spiking neural network has high accuracy in image information processing at low time steps.

[0174] In another specific embodiment, two test sets were used. For the artificial neural network (ANN) used for training, the ReLU function was used as the activation function. The open-source machine learning library PyTorch was used as the deep learning framework. Only the threshold balancing technique was used throughout the entire conversion process.

[0175] The test set is preprocessed, and test set 1 and test set 2 are horizontally flipped and randomly cropped to prevent overfitting; the image size is set to 32×32 to fit the input size of the network, and the images in the test set are standardized to ensure that the mean is 0 and the variance is 1.

[0176] Hyperparameter settings were configured as follows: during the training of the target spiking neural network, the initial learning rate was set to 0.1, momentum to 0.9, and weight decay to 5e-4. The neural network used SGD (Stochastic Gradient Descent) as the optimizer, with a batch size of 128 and 300 iterations; the learning rate decayed by a factor of 5 at epochs 70, 130, 190, and 240. During the threshold balancing process between the artificial neural network and the preset spiking neural network, the p-quantile was set to [0.9-0.99].

[0177] Based on the aforementioned test sets 1 and 2, the following were processed using artificial neural networks (ANN), pre-defined spiking neural networks (SNN), SNNs with initialized membrane potential (I-SNN), SNNs with encoding compensation (C-SNN), and target spiking neural networks (IC-SNN) that simultaneously employ initialized membrane potential and encoding compensation, respectively, to obtain the following results: Figure 8 The diagram shows the accuracy at different step sizes.

[0178] like Figure 8 As shown, spiking neural networks (I-SNN, C-SNN, and IC-SNN) employing initial membrane potential and / or encoding compensation can achieve higher accuracy at lower time steps. The default spiking neural network (SNN) performs poorly under low latency conditions, with an accuracy of only 20%–30%. Spiking neural networks employing initial membrane potential and / or encoding compensation can easily achieve an accuracy of over 70%. As the time step gradually increases, although the improvement gradually decreases, the overall trend is still one of continuous improvement. The target spiking neural network (IC-SNN) employing both initial membrane potential and encoding compensation shows the best performance.

[0179] Spiking neurons typically need to receive the output of the previous layer for a long period of time, accumulating it until it reaches the neuron threshold, and then fire a pulse. The following analysis examines the firing time of spiking neurons and verifies the effect of initializing the membrane potential on the firing time.

[0180] At low time steps, neurons often cannot receive enough output from the previous layer to fire a pulse and remain active, severely hindering the firing of spiking neurons. A significant amount of information is lost during transmission, leading to a substantial decrease in the accuracy of the neural network. The solution in this embodiment essentially provides neurons with an initial membrane potential, enabling them to activate and fire pulses more quickly, transmitting information to the next layer. This, to some extent, avoids situations where neurons are not activated for extended periods or are underactivated. Figure 9 The diagram illustrates the relationship between the time to first peak value and the number of layers for an SNN without initial membrane potential and an IC-SNN with initial membrane potential.

[0181] like Figure 9 As shown, after adopting the initial membrane potential, the average firing time of neurons in each layer of IC-SNN is earlier than that of SNN. Especially in deeper networks, IC-SNN can significantly accelerate the firing time of pulses, which is beneficial for transmitting more information and improving the accuracy of the network.

[0182] Figure 10-12 The outputs of the IC-SNN, ANN, and SNN in the first layer of the image to be processed are shown at 8 time steps; as follows: Figure 10-12 As shown, at 8 time steps, IC-SNN is almost identical to artificial neural network (ANN), while there are still significant differences between SNN and artificial neural network (ANN).

[0183] Figure 13-15 The outputs of the IC-SNN, ANN, and SNN in the third layer of the image to be processed are shown at 8 time steps; as follows: Figure 13-15 As shown, as the number of network layers increases, conversion errors begin to accumulate; IC-SNN and ANN gradually diverge, but IC-SNN is still closer to the original values ​​than SNN. Therefore, IC-SNN with initial membrane potential and encoding compensation can effectively reduce conversion errors, making the output of each layer in SNN closer to ANN, thereby improving accuracy.

[0184] The embodiments of the present invention acquire image information of the image to be processed, input the image information into the neuron model of a target spiking neural network for processing, and obtain a processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of an artificial neural network to the neuron model of a preset spiking neural network, and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network; based on the processing result, the image is classified to obtain a classification result, thereby solving the problem of low accuracy when processing images through a network model at low time steps, and improving the performance of low time step image classification.

[0185] like Figure 16 As shown, embodiments of the present invention also provide an image classification device 160 based on a spiking neural network, comprising:

[0186] The acquisition module 161 is used to acquire image information of the image to be processed;

[0187] The processing module 162 is used to input the image information into the neuron model of the target spiking neural network for processing to obtain the processing result; the neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set; the preset spiking neural network has the same network structure as the artificial neural network; based on the processing result, the image is classified to obtain the classification result.

[0188] It should be noted that this device is the same as the method described above. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0189] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0190] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0191] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0194] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0195] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above-described series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve by using their basic programming skills after reading the description of the present invention.

[0196] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0197] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image classification method based on a spiking neural network, characterized in that, include: Obtain image information of the image to be processed; The image information is input into the neuron model of the target spiking neural network for processing to obtain the processing result; The neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set. The preset spiking neural network has the same network structure as the artificial neural network; Based on the processing results, the image is classified to obtain the classification results; The training process of the neuron model of the target spiking neural network includes: Obtain the neuron model of the artificial neural network and the neuron model of the preset spiking neural network; wherein, the output of the l-th layer of the neuron model of the artificial neural network is: ;in, This represents the activation function used in the neuron model of an artificial neural network. Indicates the first Layer input, Indicates the first Layer input, This represents the weights of the l-th layer of the neuron model of the artificial neural network. This represents the bias of the l-th layer of the neuron model of the artificial neural network; The equation for the change in the first membrane potential of the input at time t in the l-th layer of the preset spiking neural network is determined as follows: ;in, It is the temporary membrane potential of the l-th layer at time t. It is the membrane potential of the l-th layer at time (t-1). It is the pulse input at time t of the l-th layer; Based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined; Based on the membrane potential change equation, training is performed using a training set; Among them, based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined, including: Based on the pulse firing mechanism, the equation for the change of the second membrane potential at time t in the neuron model of the pre-defined spiking neural network is determined as follows: ; in, This represents the membrane potential of a neuron in the l-th layer at time t in a predefined spiking neural network model. This represents the pulse input at time t in the l-th layer of a predefined spiking neural network neuron model. This represents the threshold of the neurons in the l-th layer of a predefined spiking neural network model. It is a piecewise function used to determine whether to fire a pulse, described as follows: ;if The value is greater than ,but A value of 1 indicates that a pulse is emitted; otherwise, A value of 0 indicates that no pulse is emitted; The membrane potential at time t of the neuron model of the preset spiking neural network is recursively processed to obtain the equation for the change of the third membrane potential at time T. ; in, This represents the membrane potential at time 0 of the l-th layer of a predefined spiking neural network neuron model. This represents the membrane potential at time T in the l-th layer of a predefined spiking neural network neuron model. This represents the pulse sequence output of the l-th layer of the neuron model of the preset spiking neural network.

2. The image classification method based on a spiking neural network according to claim 1, characterized in that, Based on the aforementioned membrane potential change equation, training is performed using a training set, including: Based on the membrane potential change equation, determine the error term; Based on the error term, determine the initial membrane potential of the neuron model of the preset spiking neural network; Obtain the encoding compensation value of the neuron model of the preset spiking neural network; Based on the encoded compensation value, the initial membrane potential of the neuron model of the preset spiking neural network is compensated to obtain the neuron model of the target spiking neural network after eliminating the conversion error.

3. The image classification method based on a spiking neural network according to claim 2, characterized in that, Based on the membrane potential change equation, the error term is determined, including: Based on the aforementioned equation for the change in the third membrane potential, a pulse rate formula for the l-th layer is introduced. And based on the threshold balance technique, a formula is introduced. The pulse rate formula for the (l-1)th layer is obtained. The pulse rate formula for the l-th layer represents the rate at which pulses are emitted during the period from time 0 to time T in the l-th layer. According to the pulse rate formula of the (l-1)th layer and the pulse rate formula of the l-th layer Combining the third membrane potential change equation, the first pulse rate equation is obtained. ; Based on the first pulse rate equation, the error term is determined as follows: .

4. The image classification method based on a spiking neural network according to claim 2, characterized in that, Determining the initial membrane potential of the neuron model of the preset spiking neural network based on the error term includes: Using the weights of an artificial neural network, through the formula The sum of membrane potentials from time 0 to time T in each layer of a preset spiking neural network model is determined; where sum_mem is the sum of membrane potentials. Here, T represents the weights of the l-th layer in the neuron model of the artificial neural network, and T is the length of the pulse sequence. This represents the bias of the l-th layer in the neuron model of the artificial neural network. Through formula Determine the residual membrane potential; where mem is the residual membrane potential and % is the modulo operator. The threshold of the neurons in layer l; Through formula The initial membrane potential is determined; where init_mem[l] is the initial membrane potential of the l-th layer, and mean() is the function to calculate the average value.

5. The image classification method based on a spiking neural network according to claim 2, characterized in that, Determining the initial membrane potential of the neuron model of the preset spiking neural network based on the error term includes: The weight parameters and biases of the neuron model of the artificial neural network are converted into the neuron model of the preset spiking neural network; Using the training set, through the formula The residual membrane potential is determined; where mem is the average residual membrane potential of the training set in the l-th layer, and mean() is the function for calculating the average value. Through formula Determine the sum of the remaining membrane potentials; where, This is the sum of the average values ​​of the residual membrane potential of the l-th layer obtained for each image in the training set; Through formula The average value of the accumulated residual membrane potentials in the training set at layer l is calculated to obtain the initial membrane potential of the neuron model of the preset spiking neural network; wherein, This represents the initial membrane potential of a neuron in the l-th layer of a spiking neural network. This represents the total number of images in the preset training set.

6. The image classification method based on a spiking neural network according to claim 4 or 5, characterized in that, Obtaining the encoding compensation values ​​of the neuron model of the preset spiking neural network includes: The second pulse rate equation is determined based on the initial membrane potential of the neuron model of the preset spiking neural network. ; Introducing preset formulas The preset formula represents the decoding of the pulse input of the l-th layer of the neuron model of the preset spiking neural network. Substituting it into the second pulse rate equation, the output equation of the l-th layer is obtained as follows: ; Based on the output equation of the l-th layer, the error equation for the encoding method is determined as follows: ; Based on the rate coding type, the approximate values ​​before and after coding are obtained as follows: ;in, This represents the input of the l-th layer of a neuron model in an artificial neural network. This represents the decoding of the pulse input of the l-th layer of a predefined spiking neural network neuron model. This represents the pulse input at time t in the l-th layer; Based on the approximation and the error equation, the coding compensation value is determined to be... .

7. The image classification method based on a spiking neural network according to claim 2, characterized in that, The initial membrane potential of the neuron model of the preset spiking neural network is compensated according to the encoded compensation value, including: Through formula The encoded compensation value is added to the initial membrane potential of the neuron model of the preset spiking neural network.

8. An image classification device based on a spiking neural network, characterized in that, include: The acquisition module is used to acquire image information of the image to be processed; The processing module is used to input the image information into the neuron model of the target spiking neural network for processing and to obtain the processing result; The neuron model of the target spiking neural network is obtained by converting the weights and bias parameters of the neuron model of the artificial neural network into the neuron model of the preset spiking neural network and training it using a training set. The preset spiking neural network has the same network structure as the artificial neural network. Based on the processing result, the image is classified to obtain the classification result. The training process of the neuron model of the target spiking neural network includes: Obtain the neuron model of the artificial neural network and the neuron model of the preset spiking neural network; wherein, the output of the l-th layer of the neuron model of the artificial neural network is: ;in, This represents the activation function used in the neuron model of an artificial neural network. Indicates the first Layer input, Indicates the first Layer input, This represents the weights of the l-th layer of the neuron model of the artificial neural network. This represents the bias of the l-th layer of the neuron model of the artificial neural network; The equation for the change in the first membrane potential of the input at time t in the l-th layer of the preset spiking neural network is determined as follows: ;in, It is the temporary membrane potential of the l-th layer at time t. It is the membrane potential of the l-th layer at time (t-1). It is the pulse input at time t of the l-th layer; Based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined; Based on the membrane potential change equation, training is performed using a training set; Among them, based on the pulse firing mechanism, the membrane potential change equation of the neuron model of the preset spiking neural network is determined, including: Based on the pulse firing mechanism, the equation for the change of the second membrane potential at time t in the neuron model of the pre-defined spiking neural network is determined as follows: ; in, This represents the membrane potential of a neuron in the l-th layer at time t in a predefined spiking neural network model. This represents the pulse input at time t in the l-th layer of a predefined spiking neural network neuron model. This represents the threshold of the neurons in the l-th layer of a predefined spiking neural network model. It is a piecewise function used to determine whether to fire a pulse, described as follows: ;if The value is greater than ,but A value of 1 indicates that a pulse is emitted; otherwise, A value of 0 indicates that no pulse is emitted; The membrane potential at time t of the neuron model of the preset spiking neural network is recursively processed to obtain the equation for the change of the third membrane potential at time T. ; in, This represents the membrane potential at time 0 of the l-th layer of a predefined spiking neural network neuron model. This represents the membrane potential at time T in the l-th layer of a predefined spiking neural network neuron model. This represents the pulse sequence output of the l-th layer of the neuron model of the preset spiking neural network.

Images