Pulse neural network training method, device, electronic device and storage medium

Abstract

The present invention provides a pulse neural network training method, device, electronic device and storage medium, wherein the method includes: constructing a relationship between an input pulse sequence and an output pulse sequence; obtaining a stable pulse input stream, a stable pulse output stream and an average pulse power based on the relationship; a pulse encoder converting sample data into a sample pulse sequence code; inputting the sample pulse sequence code into a pulse neural network for forward propagation to obtain the input and output of each layer and the output of the forward propagation; determining a global error based on the output of the forward propagation and a true label; obtaining a gradient of the stable pulse output stream based on the gradient and the stable pulse output stream; determining a back propagation gradient based on the gradient of the stable pulse output stream; performing gradient back propagation until the global error is less than a preset threshold, thereby being able to skip the error back propagation process in the time dimension, thereby reducing training overhead, and obtaining a model with excellent performance while greatly reducing training overhead.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a pulse neural network training method, device, electronic device and storage medium. Background Art

[0002] Pulsing neural networks have strong biological interpretability, extremely high energy efficiency and the ability to process dynamic data, and are known as the third generation of artificial neural networks.

[0003] Currently, direct training of spiking neural networks is mainly done through the substitution gradient algorithm, which uses artificial gradients to replace the discrete pulse emission process of spiking neurons for error backpropagation and treats it as a recurrent neural network for training. This method can obtain a model with extremely high energy efficiency and good performance.

[0004] However, since the alternative gradient algorithm requires error backpropagation in the time step dimension, the training cost is very high, which greatly limits the development of pulse neural networks. Summary of the Invention

[0005] The present invention provides a multimodal concept knowledge alignment method for unpaired image-text matching, which is used to solve the problem in the prior art that the alternative gradient algorithm requires error backpropagation in the time dimension and the training cost is very high.

[0006] The present invention provides a pulse neural network training method, comprising:

[0007] Based on the spiking neural network, the relationship between the input pulse sequence and the output pulse sequence is constructed;

[0008] Based on the relationship between the input pulse sequence and the output pulse sequence, a stable pulse input flow, a stable pulse output flow and an average pulse power are obtained; the stable pulse input flow reflects the input of each time step, and the stable pulse output flow reflects the output of each time step;

[0009] Based on a pulse encoder, the sample data is converted into a sample pulse sequence code; the sample data is audio data, image data or video data;

[0010] Inputting the sample pulse sequence encoding into the spiking neural network and performing forward propagation to obtain the input and output of each layer in the process of forward propagation and the output of the forward propagation;

[0011] Determine a global error based on the output of the forward propagation and the true label, and when the global error is greater than a preset threshold, obtain the gradient of any neuron at each time step based on the input and output of each layer;

[0012] Based on the gradient of any neuron at each time step and the stable pulse output stream, obtaining the gradient of the stable pulse output stream;

[0013] Determining a gradient for back propagation based on the gradient of the stable pulse output stream;

[0014] Based on the back-propagated gradient, the gradient back-propagation is performed using the stochastic gradient descent method until the global error is less than the preset threshold.

[0015] According to a spiking neural network training method provided by the present invention, determining the gradient of back propagation based on the gradient of the stable pulse output stream includes:

[0016] constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power;

[0017] Obtaining a gradient of the stable pulse input flow based on a gradient of the stable pulse output flow and a mapping between the stable pulse output flow and the stable pulse input flow;

[0018] The gradient of the back propagation is determined based on the gradient of the stable pulse input stream.

[0019] According to a spiking neural network training method provided by the present invention, determining the gradient of the back propagation based on the gradient of the stable pulse input stream includes:

[0020] Correcting the gradient of the stable pulse input flow to obtain a corrected gradient of the stable pulse input flow;

[0021] Based on the modified gradient, a gradient of the back propagation is determined.

[0022] According to a spiking neural network training method provided by the present invention, the step of correcting the gradient of the stable pulse input stream to obtain the corrected gradient of the stable pulse input stream includes:

[0023] Based on the input and output of each layer, the average pulse power and the threshold, the gradient of the stable pulse input flow is corrected to obtain the corrected gradient of the stable pulse input flow.

[0024] According to a spiking neural network training method provided by the present invention, constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power includes:

[0025] A mapping between the stable pulse output stream and the stable pulse input stream is constructed based on the following formula:

[0026]

[0027] Among them, FO represents the stable pulse output flow, FI represents the stable pulse input flow, R represents the average pulse power, V th Indicates the threshold value.

[0028] According to a spiking neural network training method provided by the present invention, obtaining the gradient of the stable pulse input stream based on the gradient of the stable pulse output stream and the mapping between the stable pulse output stream and the stable pulse input stream includes:

[0029] The gradient of the stable pulse input flow is obtained based on the following formula:

[0030]

[0031] in, represents the gradient of the steady-state pulse input flow, represents the gradient of the stable pulse output flow, Represents a mapping between a stable pulse output stream and a stable pulse input stream.

[0032] According to a spiking neural network training method provided by the present invention, the relationship between the input pulse sequence and the output pulse sequence is constructed based on the spiking neural network, including:

[0033] The relationship between the input pulse sequence and the output pulse sequence is constructed based on the following formula:

[0034]

[0035] Where T is the total time step, λ is the decay coefficient of the spiking neuron, V th is the threshold, u[t] represents the membrane potential of the neuron at time step t, I[t] represents the sum of all inputs of the neuron at time step t, and o[t] is a Boolean value, which indicates whether the neuron emits a pulse at time step t.

[0036] The present invention also provides a pulse neural network training device, comprising:

[0037] A construction unit, used for constructing a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network;

[0038] a determination unit, configured to obtain a stable pulse input flow, a stable pulse output flow, and an average pulse power based on a relationship between the input pulse sequence and the output pulse sequence; the stable pulse input flow reflects the input at each time step, and the stable pulse output flow reflects the output at each time step;

[0039] An encoding unit, configured to convert sample data into a sample pulse sequence code based on a pulse encoder; the sample data is audio data, image data or video data;

[0040] A forward propagation unit, configured to encode the sample pulse sequence and input it into the spiking neural network and perform forward propagation, thereby obtaining the input and output of each layer in the forward propagation process and the output of the forward propagation;

[0041] A global error determination unit is configured to determine a global error based on the output of the forward propagation and the true label, and obtain the gradient of any neuron at each time step based on the input and output of each layer when the global error is greater than a preset threshold;

[0042] a stable pulse output flow gradient determination unit, configured to obtain a gradient of the stable pulse output flow based on the gradient of any neuron at each time step and the stable pulse output flow;

[0043] a back propagation gradient determination unit, configured to determine a back propagation gradient based on the gradient of the stable pulse output stream;

[0044] A gradient back propagation unit is used to perform gradient back propagation based on the back-propagated gradient and the stochastic gradient descent method until the global error is less than the preset threshold.

[0045] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the pulse neural network training method described above is implemented.

[0046] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the pulse neural network training methods described above.

[0047] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the pulse neural network training methods described above.

[0048] The pulse neural network training method, device, electronic device and storage medium provided by the present invention obtain a stable pulse input stream, a stable pulse output stream and an average pulse power based on the relationship between the input pulse sequence and the output pulse sequence. The stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step. By directly constructing the relationship between the input pulse sequence and the output pulse sequence, and the average pulse power, the effect of error is reduced while further weakening the influence of membrane potential. As a result, the error backpropagation process in the time dimension can be bypassed, thereby reducing training costs, and a model with excellent performance can be obtained while greatly reducing training costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 This is one of the flow charts of the pulse neural network training method provided by the present invention;

[0051] Figure 2 Flowchart 2 of the spiking neural network training method provided by the present invention;

[0052] Figure 3 is a schematic diagram of the existing error back propagation process;

[0053] Figure 4 Schematic diagram of the corrected error back propagation process provided by the present invention;

[0054] Figure 5 It is a structural diagram of the pulse neural network training device provided by the present invention;

[0055] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] The present invention provides a pulse neural network training method. Figure 1This is one of the flow charts of the pulse neural network training method provided by the present invention. Figure 2 The second flow chart of the pulse neural network training method provided by the present invention is as follows: Figure 1 、 Figure 2 As shown, the method includes:

[0058] Step 110: construct a relationship between the input pulse sequence and the output pulse sequence based on the pulse neural network.

[0059] Specifically, the relationship between the input pulse sequence and the output pulse sequence can be constructed based on a Spiking Neural Network (SNN).

[0060] Taking the LIF (Leaky Intergrate and fired model), a hard reset method that is currently widely used, as an example, the relationship between its input pulse sequence and output pulse sequence is:

[0061]

[0062] Where T is the total time step, λ is the decay coefficient of the spiking neuron, V th is the threshold, u[t] represents the membrane potential of the neuron at time step t, I[t] represents the sum of all inputs of the neuron at time step t, and o[t] is a Boolean value, which indicates whether the neuron emits a pulse at time step t.

[0063] The above relationship can be simply expressed as remaining power = total input power - total output power, which is commonly found in various types of pulse neural networks.

[0064] The pulse sequence is binary, for example, the pulse sequence may be 0101010101, where 1 represents a pulse and 0 represents no pulse is transmitted.

[0065] Step 120, based on the relationship between the input pulse sequence and the output pulse sequence, obtain a stable pulse input flow, a stable pulse output flow and an average pulse power; the stable pulse input flow reflects the input of each time step, and the stable pulse output flow reflects the output of each time step.

[0066] Specifically, after the relationship between the input pulse sequence and the output pulse sequence is obtained, the stable pulse input flow, the stable pulse output flow and the average pulse power can be obtained based on the relationship between the input pulse sequence and the output pulse sequence.

[0067] The average pulse charge here further weakens the influence of membrane potential while reducing the error, so that the method provided by the embodiment of the present invention can obtain a model with excellent performance while greatly reducing training overhead.

[0068] The formula for stabilizing the pulse input flow is: The formula for stabilizing the pulse output flow is: The formula for average pulse power is: The stable pulse input stream can reflect the input of each time step, and the stable pulse output stream reflects the output of each time step, thereby avoiding the cycle of the subsequent gradient back propagation process in the time dimension.

[0069] Step 130: Convert the sample data into a sample pulse sequence code based on a pulse encoder; the sample data is audio data, image data or video data.

[0070] Specifically, a pulse encoder can be used to pulse encode the sample data, encoding a non-pulse input signal (e.g., real number information) into a new pulse sequence in a certain distribution form, thereby obtaining a sample pulse sequence encoding corresponding to the sample data for subsequent processing by the spiking neuron. The sample data can be sequence data such as audio data, image data, or video data.

[0071] For example, when performing image recognition tasks, the pulse encoder can be an image encoder, such as a cascaded multi-layer convolutional neural network (CNN), deep neural network (DNN) and other models, and the sample pulse sequence encoding can be the image encoding features of each frame corresponding to the sample image sequence.

[0072] Furthermore, in image recognition tasks where the input is a static 2D image, a layer of IF neurons can be used to form an encoding layer. By fixing the input of this layer at each time step to the pixel value in the image, the encoding layer can map real values to a pulse sequence with a stable pulse frequency, i.e., a stable pulse stream.

[0073] For another example, when performing a speech recognition task, the pulse encoder may be a speech encoder, and the sample pulse sequence encoding may be the semantic encoding features of each frame of speech corresponding to the sample speech sequence.

[0074] For example, when performing a text recognition task, the pulse encoder can be a text encoder, such as a CNN, DNN or other model, and the sample pulse sequence encoding can be the text semantic features corresponding to the sample text.

[0075] Step 140: Encode the sample pulse sequence and input it into the pulse neural network and perform forward propagation to obtain the input and output of each layer in the process of forward propagation and the output of the forward propagation.

[0076] Specifically, after obtaining the sample pulse sequence code, the sample pulse sequence code can be input into the spiking neural network and forward propagated to obtain the input I[t] and output o[t] of each layer in the forward propagation process and the output of the forward propagation. The output of the forward propagation here is the output of the last layer of the forward propagation.

[0077] Step 150: Determine a global error based on the output of the forward propagation and the true label, and when the global error is greater than a preset threshold, obtain the gradient of any neuron at each time step based on the input and output of each layer.

[0078] Specifically, after obtaining the output of the forward propagation, a global error can be determined based on the forward propagation output and the true label. If the global error is greater than a preset threshold, the gradient of each neuron at each time step is obtained based on the input and output of each layer. In other words, the gradient is propagated back to the spiking layer using the backpropagation algorithm. The preset threshold here can be 0.02, 0.05, etc., and is not specifically limited in this embodiment of the present invention.

[0079] It can be understood that the greater the difference between the output of the forward propagation and the true label, the greater the global error; the smaller the difference between the output of the forward propagation and the true label, the smaller the global error.

[0080] Among them, taking the lth layer as an example, the gradient of any neuron i at each time step t is obtained

[0081] Step 160: Based on the gradient of any neuron at each time step and the stable pulse output flow, obtain the gradient of the stable pulse output flow.

[0082] Specifically, after obtaining the gradient of any neuron at each time step, the gradient of the stable pulse output flow can be obtained based on the gradient of any neuron at each time step and the stable pulse output flow. The formula for the gradient of the stable pulse flow is as follows:

[0083]

[0084] Step 170: Determine the gradient of the back propagation based on the gradient of the stable pulse output flow.

[0085] Step 180: Perform gradient back propagation based on the back propagated gradient and the stochastic gradient descent method until the global error is less than the preset threshold.

[0086] Specifically, after obtaining the gradient of the stable pulse flow, the gradient of the back propagation can be determined based on the gradient of the stable pulse output flow.

[0087] For example, the gradient of the stable pulse input flow can be determined based on the gradient of the stable pulse output flow, and the gradient of the back propagation can be determined based on the gradient of the stable pulse input flow.

[0088] Then, based on the back-propagation gradient, the gradient is back-propagated using the stochastic gradient descent method until the global error is less than the preset threshold.

[0089] The method provided by an embodiment of the present invention obtains a stable pulse input stream, a stable pulse output stream and an average pulse charge based on the relationship between the input pulse sequence and the output pulse sequence. The stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step. By directly constructing the relationship between the input pulse sequence and the output pulse sequence, and the average pulse charge, the effect of the error is reduced while further weakening the influence of the membrane potential. As a result, the error backpropagation process in the time dimension can be skipped, thereby reducing training overhead, and a model with excellent performance can be obtained while greatly reducing training overhead.

[0090] Based on the above embodiment, step 170 includes:

[0091] Step 171 , constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power;

[0092] Step 172: Obtaining the gradient of the stable pulse input flow based on the gradient of the stable pulse output flow and the mapping between the stable pulse output flow and the stable pulse input flow;

[0093] Step 173: Determine the gradient of the back propagation based on the gradient of the stable pulse input flow.

[0094] Specifically, ignoring the influence of the remaining power and considering the intrinsic characteristics of the spiking neuron, we can construct a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power:

[0095]

[0096] Among them, FO represents the stable pulse output flow, FI represents the stable pulse input flow, R represents the average pulse power, V th Indicates the threshold value.

[0097] Figure 3 is a schematic diagram of the existing error back propagation process, such as Figure 3 As shown, I l [T] represents the gradient of the lth input layer, o l [T] represents the gradient of the lth output layer, T represents the total time step length, u l [T-1] represents the gradient of the membrane potential at time T-1, I l-1[T] represents the gradient at time T passed to the next layer. Other parameters are similar and will not be repeated here.

[0098] Figure 4 is a schematic diagram of the corrected error back propagation process provided by the present invention, such as Figure 4 As shown, I l+1 [T] represents the gradient of the input at time T of the l+1 layer, that is, the gradient passed down from the upper layer, o l [T] represents the gradient of the output of the lth layer, T represents the total time step length, FO represents the stable pulse output flow, FI represents the stable pulse input flow, the pulse flow here is a stable pulse flow, I l [T] represents the gradient of the input of layer l, that is, the gradient at time T passed to the next layer. Other parameters are similar and will not be repeated here.

[0099] This relationship directly constructs a mapping between stable pulse output streams and stable pulse input streams, which can modify the error backpropagation process of traditional alternative gradient methods to avoid cycles in the time dimension.

[0100] Then, the gradient of the stable pulse input flow can be obtained based on the gradient of the stable pulse output flow and the mapping between the stable pulse output flow and the stable pulse input flow.

[0101] That is, since the mapping relationship between the stable pulse output flow and the stable pulse input flow is Therefore, the formula for the gradient of a stable pulsed input flow is:

[0102]

[0103] in, represents the gradient of the steady-state pulse input flow, represents the gradient of the stable pulse output flow, Represents a mapping between a stable pulse output stream and a stable pulse input stream.

[0104] Finally, the gradient of the backpropagation can be determined based on the gradient of the stable pulse input stream.

[0105] Based on the above embodiment, step 173 includes:

[0106] Step 1731, correcting the gradient of the stable pulse input flow to obtain a corrected gradient of the stable pulse input flow;

[0107] Step 1732: Determine the gradient of the back propagation based on the corrected gradient.

[0108] Specifically, since spiking neurons map a steady stream of spiking inputs to the range [0, 1], it is necessary to clip the neuron gradients whose inputs are outside this range.

[0109] Therefore, the gradient of the stable pulse input flow can be corrected to obtain the corrected gradient of the stable pulse input flow.

[0110] Then, based on the corrected gradient, the gradient of back propagation is determined.

[0111] Based on the above embodiment, step 1731 includes:

[0112] Based on the input and output of each layer, the average pulse power and the threshold, the gradient of the stable pulse input flow is corrected to obtain the corrected gradient of the stable pulse input flow.

[0113] Specifically, the stable pulse input stream is calculated based on the input of each layer and The gradient corresponding to the neuron is set to 0.

[0114] Considering the input In the forward derivation, the membrane potential has already gone through the decay process, so the gradient of the average pulse input flow to the input of each time step is 1, that is, Using the gradient of the stable pulse input flow and combining the formula The gradient of the stable pulse input flow is corrected to obtain a corrected gradient of the stable pulse input flow.

[0115] The pulse neural network training device provided by the present invention is described below. The pulse neural network training device described below and the pulse neural network training method described above can be referenced to each other.

[0116] Based on any of the above embodiments, the present invention provides a pulse neural network training device, Figure 5 This is a schematic diagram of the structure of the pulse neural network training device provided by the present invention. Figure 5 As shown, the device includes:

[0117] A construction unit 510 is configured to construct a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network;

[0118] a determining unit 520 configured to obtain a stable pulse input flow, a stable pulse output flow, and an average pulse power based on a relationship between the input pulse sequence and the output pulse sequence; wherein the stable pulse input flow reflects the input at each time step, and the stable pulse output flow reflects the output at each time step;

[0119] An encoding unit 530 is configured to convert sample data into a sample pulse sequence code based on a pulse encoder; the sample data is audio data, image data or video data;

[0120] A forward propagation unit 540 is configured to encode the sample pulse sequence and input it into the spiking neural network and perform forward propagation to obtain the input and output of each layer in the forward propagation process and the output of the forward propagation;

[0121] A global error determination unit 550 is configured to determine a global error based on the output of the forward propagation and the true label, and, if the global error is greater than a preset threshold, obtain the gradient of any neuron at each time step based on the input and output of each layer;

[0122] a stable pulse output flow gradient determination unit 560, configured to obtain a gradient of the stable pulse output flow based on the gradient of any neuron at each time step and the stable pulse output flow;

[0123] a back propagation gradient determination unit 570, configured to determine a back propagation gradient based on the gradient of the stable pulse output stream;

[0124] The gradient back propagation unit 580 is configured to perform gradient back propagation based on the back propagated gradient and the stochastic gradient descent method until the global error is less than the preset threshold.

[0125] The device provided by the embodiment of the present invention obtains a stable pulse input stream, a stable pulse output stream and an average pulse charge based on the relationship between the input pulse sequence and the output pulse sequence. The stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step. By directly constructing the relationship between the input pulse sequence and the output pulse sequence, and the average pulse charge, the effect of the error is reduced while further weakening the influence of the membrane potential. As a result, the error backpropagation process in the time dimension can be skipped, thereby reducing training overhead, and a model with excellent performance can be obtained while greatly reducing training overhead.

[0126] Based on any of the above embodiments, the back propagation gradient unit 570 is determined to be specifically configured to:

[0127] A mapping unit is constructed, configured to construct a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power;

[0128] a stable pulse input flow gradient determining unit, configured to obtain the gradient of the stable pulse input flow based on the gradient of the stable pulse output flow and a mapping between the stable pulse output flow and the stable pulse input flow;

[0129] The back propagation gradient determination subunit is configured to determine the back propagation gradient based on the gradient of the stable pulse input flow.

[0130] Based on any of the above embodiments, a back-propagation gradient subunit is determined, specifically for:

[0131] a correction unit, configured to correct the gradient of the stable pulse input flow to obtain a corrected gradient of the stable pulse input flow;

[0132] A back propagation unit is used to determine the gradient of the back propagation based on the modified gradient.

[0133] Based on any of the above embodiments, the correction unit is specifically configured to:

[0134] Based on the input and output of each layer, the average pulse power and the threshold, the gradient of the stable pulse input flow is corrected to obtain the corrected gradient of the stable pulse input flow.

[0135] Based on any of the above embodiments, a mapping unit is constructed, specifically configured to:

[0136] A mapping between the stable pulse output stream and the stable pulse input stream is constructed based on the following formula:

[0137]

[0138] Among them, FO represents the stable pulse output flow, FI represents the stable pulse input flow, R represents the average pulse power, V th Indicates the threshold value.

[0139] Based on any of the above embodiments, it is determined that the stable pulse input flow gradient unit is specifically used for:

[0140] The gradient of the stable pulse input flow is obtained based on the following formula:

[0141]

[0142] in, represents the gradient of the steady-state pulse input flow, represents the gradient of the stable pulse output flow, Represents a mapping between a stable pulse output stream and a stable pulse input stream.

[0143] Based on any of the above embodiments, the construction unit 510 is specifically configured to:

[0144] The relationship between the input pulse sequence and the output pulse sequence is constructed based on the following formula:

[0145]

[0146] Where T is the total time step, λ is the decay coefficient of the spiking neuron, V this the threshold, u[t] represents the membrane potential of the neuron at time step t, I[t] represents the sum of all inputs of the neuron at time step t, and o[t] is a Boolean value, which indicates whether the neuron emits a pulse at time step t.

[0147] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the pulse neural network training method, which includes: constructing the relationship between the input pulse sequence and the output pulse sequence based on the pulse neural network; based on the relationship between the input pulse sequence and the output pulse sequence, obtaining a stable pulse input stream, a stable pulse output stream and an average pulse power; the stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step; based on the pulse encoder, converting the sample data into a sample pulse sequence code; the sample data is audio data, image data or video data; the sample pulse sequence code is input into the pulse neural network The method comprises the following steps: a) performing a forward propagation on a neural network and performing forward propagation to obtain the input and output of each layer and the output of the forward propagation during the forward propagation process; b) determining a global error based on the output of the forward propagation and the true label, and when the global error is greater than a preset threshold, obtaining the gradient of any neuron at each time step based on the input and output of each layer; c) obtaining the gradient of the stable pulse output flow based on the gradient of any neuron at each time step and the stable pulse output flow; e) determining the gradient of the reverse propagation based on the gradient of the stable pulse output flow; and c) performing gradient back propagation based on the gradient of the reverse propagation and the stochastic gradient descent method until the global error is less than the preset threshold.

[0148] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pulse neural network training method provided by the above methods, which includes: constructing a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network; based on the relationship between the input pulse sequence and the output pulse sequence, obtaining a stable pulse input stream, a stable pulse output stream and an average pulse power; the stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step; based on a pulse encoder, converting sample data into a sample pulse sequence code; the sample data is audio Data, image data or video data; encoding the sample pulse sequence and inputting it into the pulse neural network and performing forward propagation to obtain the input and output of each layer in the process of forward propagation and the output of the forward propagation; determining the global error based on the output of the forward propagation and the true label, and when the global error is greater than a preset threshold, obtaining the gradient of any neuron at each time step based on the input and output of each layer; obtaining the gradient of the stable pulse output flow based on the gradient of any neuron at each time step and the stable pulse output flow; determining the gradient of back propagation based on the gradient of the stable pulse output flow; performing gradient back propagation based on the gradient of the back propagation and the stochastic gradient descent method until the global error is less than the preset threshold.

[0150] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the pulse neural network training method provided by the above methods, the method comprising: constructing a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network; obtaining a stable pulse input stream, a stable pulse output stream and an average pulse power based on the relationship between the input pulse sequence and the output pulse sequence; the stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step; converting sample data into a sample pulse sequence code based on a pulse encoder; the sample data is audio data, image data or video data; The sample pulse sequence encoding is input to the pulse neural network and forward propagated to obtain the input and output of each layer in the process of forward propagation and the output of the forward propagation; based on the output of the forward propagation and the true label, the global error is determined, and when the global error is greater than a preset threshold, based on the input and output of each layer, the gradient of any neuron at each time step is obtained; based on the gradient of any neuron at each time step and the stable pulse output stream, the gradient of the stable pulse output stream is obtained; based on the gradient of the stable pulse output stream, the gradient of the back propagation is determined; based on the gradient of the back propagation and the stochastic gradient descent method, gradient back propagation is performed until the global error is less than the preset threshold.

[0151] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A pulse neural network training method, characterized in that: include: Based on the spiking neural network, the relationship between the input pulse sequence and the output pulse sequence is constructed; Based on the relationship between the input pulse sequence and the output pulse sequence, a stable pulse input flow, a stable pulse output flow and an average pulse power are obtained; The stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step; Based on a pulse encoder, the sample data is converted into a sample pulse sequence code; the sample data is audio data, image data or video data; Inputting the sample pulse sequence encoding into the spiking neural network and performing forward propagation to obtain the input and output of each layer in the process of forward propagation and the output of the forward propagation; Determine a global error based on the output of the forward propagation and the true label, and when the global error is greater than a preset threshold, obtain the gradient of any neuron at each time step based on the input and output of each layer; Based on the gradient of any neuron at each time step and the stable pulse output stream, obtaining the gradient of the stable pulse output stream; Determining a gradient for back propagation based on the gradient of the stable pulse output stream; Performing gradient back propagation based on the back propagation gradient and the stochastic gradient descent method until the global error is less than the preset threshold; The step of determining the gradient of back propagation based on the gradient of the stable pulse output flow comprises: constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power; Obtaining a gradient of the stable pulse input flow based on a gradient of the stable pulse output flow and a mapping between the stable pulse output flow and the stable pulse input flow; The gradient of the back propagation is determined based on the gradient of the stable pulse input stream.

2. The pulse neural network training method according to claim 1, characterized in that: The step of determining the gradient of the back propagation based on the gradient of the stable pulse input flow comprises: Correcting the gradient of the stable pulse input flow to obtain a corrected gradient of the stable pulse input flow; Based on the modified gradient, a gradient of the back propagation is determined.

3. The pulse neural network training method according to claim 2, characterized in that: The step of correcting the gradient of the stable pulse input flow to obtain the corrected gradient of the stable pulse input flow includes: Based on the input and output of each layer, the average pulse power and the threshold, the gradient of the stable pulse input flow is corrected to obtain the corrected gradient of the stable pulse input flow.

4. The pulse neural network training method according to claim 1, characterized in that: The constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power includes: A mapping between the stable pulse output stream and the stable pulse input stream is constructed based on the following formula: ； in, represents a stable pulse output stream, represents a steady pulse input stream, Indicates the average pulse power, Indicates the threshold value.

5. The pulse neural network training method according to claim 1, characterized in that: The obtaining of the gradient of the stable pulse input flow based on the gradient of the stable pulse output flow and the mapping between the stable pulse output flow and the stable pulse input flow comprises: The gradient of the stable pulse input flow is obtained based on the following formula: ； in, represents the gradient of a steady pulse input flow, represents the gradient of the stable pulse output flow, Represents a mapping between a stable pulse output stream and a stable pulse input stream.

6. The pulse neural network training method according to any one of claims 1 to 5, characterized in that: The method of constructing a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network includes: The relationship between the input pulse sequence and the output pulse sequence is constructed based on the following formula: ； in, is the total time step, is the attenuation coefficient of the spiking neuron, Represents the time step The membrane potential of the neuron, Indicates that the neuron The sum of all inputs at a time step, is a Boolean value, Indicates that the neuron Whether to emit pulses in the time step.

7. A pulse neural network training device, characterized in that: include: A construction unit, used for constructing a relationship between an input pulse sequence and an output pulse sequence based on a pulse neural network; a determining unit, configured to obtain a stable pulse input flow, a stable pulse output flow, and an average pulse power based on a relationship between the input pulse sequence and the output pulse sequence; The stable pulse input stream reflects the input of each time step, and the stable pulse output stream reflects the output of each time step; An encoding unit, configured to convert sample data into a sample pulse sequence code based on a pulse encoder; the sample data is audio data, image data or video data; A forward propagation unit, configured to encode the sample pulse sequence and input it into the spiking neural network and perform forward propagation, thereby obtaining the input and output of each layer in the forward propagation process and the output of the forward propagation; A global error determination unit is configured to determine a global error based on the output of the forward propagation and the true label, and obtain the gradient of any neuron at each time step based on the input and output of each layer when the global error is greater than a preset threshold; a stable pulse output flow gradient determination unit, configured to obtain a gradient of the stable pulse output flow based on the gradient of any neuron at each time step and the stable pulse output flow; a back propagation gradient determination unit, configured to determine a back propagation gradient based on the gradient of the stable pulse output stream; A gradient back propagation unit, configured to perform gradient back propagation based on the back-propagated gradient and a stochastic gradient descent method until the global error is less than the preset threshold; The back propagation gradient unit is specifically used to: constructing a mapping between the stable pulse output flow and the stable pulse input flow based on the average pulse power; Obtaining a gradient of the stable pulse input flow based on a gradient of the stable pulse output flow and a mapping between the stable pulse output flow and the stable pulse input flow; The gradient of the back propagation is determined based on the gradient of the stable pulse input stream.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the pulse neural network training method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pulse neural network training method according to any one of claims 1 to 6 is implemented.

Images