Model training method, model reasoning method and equipment

Abstract

The invention provides a model training method, a model reasoning method and equipment, and the method comprises the steps: obtaining a pre-trained artificial neural network and a data set, and the data set comprises a sample image; setting corresponding upper bound parameters for each neuron in the artificial neural network; training an artificial neural network based on the data set, and updating upper bound parameters of each neuron to obtain a trained artificial neural network; obtaining a target upper bound parameter of each neuron in the trained artificial neural network; and based on the target upper bound parameter of each neuron, determining a pulse output threshold of each neuron in the pulse neural network. According to the embodiment of the invention, the artificial neural network is trained to obtain the pulse output threshold value of the pulse neural network, so that the trained pulse neural network is quickly obtained, and the training efficiency of the pulse neural network is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and particularly relates to a model training method, a model inference method, and a device. Background Art

[0002] In recent years, the rapid development of large-scale models has greatly promoted the development of deep learning technology, industry, and research. However, these models consume a large amount of energy during the inference and training processes, and face challenges in deployment on resource-constrained devices.

[0003] As the third-generation neural network, the Spiking Neural Network (SNN) is expected to become an alternative to low-energy operation of large-scale neural networks by simulating the working mode of biological neurons. The spiking neural network processes information through discrete spike signals, can efficiently process spatio-temporal information, and uses sparse representation to improve computational efficiency.

[0004] However, due to factors such as the complexity introduced by the time dimension, the increase in memory requirements, the gradient propagation problem, and the limitations of training algorithms in the spiking neural network, the training cost increases linearly with the number of time steps, resulting in low training efficiency. Summary of the Invention

[0005] This application proposes a model training method, a model inference method, and a device, which can solve the technical problem that the training cost of the spiking neural network increases linearly with the number of time steps and the training efficiency is low due to factors such as the complexity introduced by the time dimension, the increase in memory requirements, the gradient propagation problem, and the limitations of training algorithms.

[0006] The first aspect of the embodiments of this application proposes a model training method, including: Obtain a pre-trained artificial neural network and a data set, where the data set includes sample images; Set corresponding upper bound parameters for each neuron in the artificial neural network; Train the artificial neural network based on the data set, update the upper bound parameters of each neuron, and obtain a trained artificial neural network; Obtain the target upper bound parameters of each neuron in the trained artificial neural network; Based on the target upper bound parameters of each neuron, determine the pulse output threshold of each neuron in the spiking neural network.

[0007] The second aspect of the embodiments of this application proposes a model inference method, including: Optimize the spiking neural network to obtain a target spiking neural network, where the spiking neural network is trained by the method in the first aspect above; Input the target image into the target spiking neural network to obtain the model prediction value.

[0008] An embodiment of the third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor runs the computer program to implement the method described in the first aspect or the second aspect above.

[0009] The technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: The present application proposes a model training method, a model inference method, and a device, including: obtaining a pre-trained artificial neural network and a data set, where the data set includes sample images; setting corresponding upper bound parameters for each neuron in the artificial neural network; training the artificial neural network based on the data set, updating the upper bound parameters of each neuron to obtain a trained artificial neural network; obtaining the target upper bound parameters of each neuron in the trained artificial neural network; and determining the pulse output thresholds of each neuron in the spiking neural network based on the target upper bound parameters of each neuron. By training the artificial neural network in the embodiments of the present application, the pulse output thresholds of the spiking neural network are obtained to quickly obtain a trained spiking neural network, improving the training efficiency of the spiking neural network.

[0010] The additional aspects and advantages of the present application will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings: Figure 1 Shows a flowchart of a model training method provided by an embodiment of the present application; Figure 2 Shows another flowchart of a model training method provided by an embodiment of the present application; Figure 3 Shows a flowchart of a model inference method provided by an embodiment of the present application; Figure 4 Shows another flowchart of a model inference method provided by an embodiment of the present application; Figure 5 Shows a schematic structural diagram of a model training device provided by an embodiment of the present application; Figure 6The figure shows a schematic structural diagram of a model inference device provided by an embodiment of the present application; Figure 7 The figure shows a schematic structural diagram of an electronic device provided by an embodiment of the present application; Figure 8 The figure shows a schematic diagram of a storage medium provided by an embodiment of the present application. Detailed implementation manners

[0012] Hereinafter, the exemplary embodiments of the present application will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully conveyed to those skilled in the art.

[0013] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should have the ordinary meanings understood by those skilled in the art to which the present application belongs.

[0014] The model training method of the present application can be executed by a computing device. The computing device applies cloud computing and virtualization technologies. The computing device can be a server, such as a single server, multiple servers, a server cluster, a cloud computing platform, etc. Optionally, the computing device can also be a terminal device, such as a mobile phone, a tablet computer, a game console, a portable computer, a desktop computer, an advertising machine, an all-in-one computer, etc. The present application does not limit the device type and the number of devices of the computing device.

[0015] To solve the above problems, the present application proposes a model training method, a model inference method and a device, including: obtaining a pre-trained artificial neural network and a data set, where the data set includes sample images; setting corresponding upper bound parameters for each neuron in the artificial neural network; training the artificial neural network based on the data set, updating the upper bound parameters of each neuron, and obtaining a trained artificial neural network; obtaining the target upper bound parameters of each neuron in the trained artificial neural network; determining the pulse output thresholds of each neuron in the spiking neural network based on the target upper bound parameters of each neuron. By training the artificial neural network in the embodiment of the present application, the pulse output thresholds of the spiking neural network are obtained, so as to quickly obtain a trained spiking neural network and improve the training efficiency of the spiking neural network.

[0016] Next, a model training method proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0017] See Figure 1 , the method specifically includes the following steps: S101. Obtain a pre-trained artificial neural network and a data set.

[0018] The data set includes sample images.

[0019] In some embodiments, since both artificial neural networks and spiking neural networks can adopt similar network structures, such as convolutional layers, fully connected layers, etc. This structural similarity enables the training results of artificial neural networks to be partially transferred to spiking neural networks.

[0020] Therefore, during the process of training a spiking neural network, it is possible to preferentially obtain a pre-trained artificial neural network and a data set, so as to train the artificial neural network to obtain corresponding threshold parameters.

[0021] S102. Set corresponding upper bound parameters for each neuron in the artificial neural network.

[0022] Among them, the upper bound parameter is the upper bound parameter of the activation function corresponding to each neuron. That is, an activation function with an upper bound parameter can be set for each neuron.

[0023] Generally, each neuron in an artificial neural network is correspondingly set with an activation function. Therefore, this step is to replace the activation function with an activation function with an upper bound parameter.

[0024] It can be understood that in an artificial neural network, the role of the activation function is to convert the input signal of the neuron into an output signal. For example, the ReLU activation function can be regarded as a kind of "soft" threshold operation, where the output is 0 when the input is less than 0, and the output is the input value when the input is greater than 0. This operation can be analogized to the pulse firing mechanism of neurons in a spiking neural network: In a spiking neural network, a neuron fires a pulse when its membrane potential reaches or exceeds a threshold.

[0025] In an artificial neural network, the upper bound parameter of the activation function can be regarded as a "soft" threshold. When the input exceeds this upper bound, the output is limited to this upper bound value.

[0026] Therefore, the upper bound parameter in an artificial neural network can be analogized to the pulse output threshold in a spiking neural network, and they both play a role in restricting the output and controlling signal transmission.

[0027] In an artificial neural network, the activation function with an upper bound parameter can learn the optimal upper bound value through the training process. These upper bound values are obtained through optimization algorithms such as backpropagation and gradient descent, and can enable the network to achieve better performance on a given task. Using these upper bound parameters obtained through training directly as the pulse output threshold of the spiking neural network can utilize the optimal parameters learned during the training process of the artificial neural network, thereby improving the performance of the spiking neural network.

[0028] Therefore, by setting corresponding upper bound parameters for each neuron in the artificial neural network, the adjusted target upper bound parameters can be obtained during the training process as the pulse output threshold of each neuron in the spiking neural network.

[0029] S103: Training the artificial neural network based on the data set, updating the upper bound parameters of each neuron, and obtaining a trained artificial neural network.

[0030] Furthermore, the artificial neural network is trained based on the data set, and the upper bound parameters of each neuron are updated to obtain a trained artificial neural network.

[0031] S104: Obtain target upper bound parameters of each neuron in the trained artificial neural network.

[0032] Furthermore, the target upper bound parameters of each neuron and the target synaptic weight corresponding to each neuron can be obtained, so as to determine the pulse output threshold of the pulse neural network through the mapping relationship between the artificial neural network and the pulse neural network.

[0033] S105. Determine the pulse output threshold of each neuron in the spiking neural network based on the target upper bound parameter of each neuron.

[0034] In some embodiments, if the activation function used in the artificial neural network has similar characteristics to the pulse emission mechanism of the pulse neural network, or the network structure of the artificial neural network and the pulse neural network is consistent, the target upper limit parameter of each neuron can be directly determined as the pulse output threshold of each neuron in the fixed pulse neural network.

[0035] The network structure may include the number of network layers, the number of neurons in each layer, and the connection method between each layer.

[0036] In some embodiments, if the activation function used in the artificial neural network is not similar to the pulse emission mechanism of the spiking neural network, or the network structures of the artificial neural network and the spiking neural network are inconsistent, the target upper limit parameter of each neuron cannot be directly determined as the pulse output threshold of each neuron in the spiking neural network.

[0037] When the activation function used in an artificial neural network is dissimilar to the spiking mechanism of a spiking neural network, the activation values in the artificial neural network are normalized to the spiking threshold range of the spiking neural network. For example, if a nonlinear activation function is used in the artificial neural network, the output values can be normalized to the range [0, 1] and then used as the spiking threshold of the spiking neural network. The spiking threshold of the spiking neural network is dynamically adjusted using an optimization algorithm (such as grid search or gradient descent) to minimize the output difference between the artificial neural network and the spiking neural network.

[0038] When the network structures of the artificial neural network and the spiking neural network are inconsistent, adjust the network structure of the spiking neural network to make it as close as possible to the structure of the artificial neural network. For some layers (such as convolutional layers and fully connected layers), the weights and thresholds can be directly migrated; for other layers (such as Batch Normalization layers), they can be processed separately or omitted.

[0039] Of course, there are other cases where the upper bound parameter of each neuron can be directly determined as the spike output threshold of each neuron in the spiking neural network, or cases that require adaptive adjustment, which will not be elaborated here.

[0040] This application proposes a model training method, a model inference method and a device, including: obtaining a pre-trained artificial neural network and a data set, the data set including sample images; setting corresponding upper bound parameters for each neuron in the artificial neural network; training the artificial neural network based on the data set, updating the upper bound parameters of each neuron to obtain a trained artificial neural network; obtaining the target upper bound parameters of each neuron in the trained artificial neural network; determining the spike output threshold of each neuron in the spiking neural network based on the target upper bound parameters of each neuron. By training the artificial neural network in the embodiments of this application, the spike output threshold of the spiking neural network is obtained to quickly obtain a trained spiking neural network, improving the training efficiency of the spiking neural network.

[0041] In some embodiments, the artificial neural network includes multiple convolutional layers and multiple linear layers. Setting corresponding upper bound parameters for each neuron in the artificial neural network includes: setting corresponding upper bound parameters for each channel of the feature map output by each convolutional layer; or setting corresponding upper bound parameters for each element of the activation value of each linear layer.

[0042] In some embodiments, the artificial neural network includes multiple convolutional layers and multiple linear layers. Each network layer includes a weight part and an activation function part. Here, the activation function of the artificial neural network is the rectified linear unit. According to the order of the forward output of the artificial neural network, the input of the th activation function is , then the expression of the rectified linear unit is . This activation function is quite different from the spiking neuron model of the spiking neural network. Therefore, direct conversion is likely to produce large errors. Here, we use the high-fine-grained upper-bounded rectified linear unit as the activation function of the artificial neural network. Specifically, the upper-bounded rectified linear unit filters values less than zero or greater than the upper bound parameter. And, if the current feature is the output feature of the convolutional layer, an independent upper bound parameter is set for each channel, and if the current feature is the output feature of the linear layer, an independent upper bound parameter is set for each neuron. Assuming the current is the layer, the channel (or for a linear layer, the element) upper bound parameter is .

[0043] Among them, the convolutional layer slides the convolutional kernel on the input feature map and only processes a local area of the input feature map each time. One channel of the convolutional layer includes multiple neurons corresponding to a convolutional kernel. Therefore, the upper bound parameter of each channel is the upper bound parameter corresponding to the multiple neurons corresponding to this channel.

[0044] The linear layer (fully connected layer) is a common layer in a neural network, and its main function is to perform a linear combination of the input features. Specifically, each neuron in the linear layer will receive all the elements of the input feature vector and calculate the output through weighted summation. Each neuron in the linear layer calculates an output value, and these output values are the activation values of the linear layer. Therefore, the upper bound parameter corresponding to each element is the upper bound parameter of the neuron corresponding to this element.

[0045] In some embodiments, after setting the corresponding upper bound parameters for each neuron in the artificial neural network, all the upper bound parameters can be initialized, and all the upper bound parameters can be initialized to a preset upper bound parameter value, and this preset upper bound parameter value can be 0.

[0046] Furthermore, as Figure 2 shown, a schematic diagram of the training process of an artificial neural network provided by an embodiment of the present application is shown. Among them, the artificial neural network includes multiple convolutional layers and linear layers, and an activation layer is correspondingly set for each convolutional layer and each linear layer, and the activation function in the activation layer is replaced with an activation function with an upper bound parameter.

[0047] For the sample set of size , for each batch of image and label data obtained therein, input it into the modified artificial neural network for inference.

[0048] For each training process, the upper bound parameters are optimized layer by layer using the local learning algorithm during the inference process of the artificial neural network. During a single inference process, the activation function of the th layer of the artificial neural network model will obtain an input, denoted as , being the number of elements under this channel (if it is a linear layer, then ).

[0049] For each training process, the optimization process of the upper bound parameters includes: ; Among them, The upper bound parameter of the c-th channel in the l-th layer after optimization is the upper bound parameter of the c-th channel in the l-th layer before the optimization of this training is the step function is the learning rate is the number of neurons in the current channel is the input of this training

[0050] After the training is completed, the set of target upper bound parameters of each neuron in the artificial neural network is obtained, that is, the set of pulse output thresholds of each neuron in the spiking neural network

[0051] In some embodiments, after obtaining the trained spiking neural network, in order to improve the inference performance of the spiking neural network, the spiking neural network can also be optimized. A model inference method proposed according to an embodiment of the present application will be described below with reference to the accompanying drawings

[0052] For ease of understanding, the process of the above model training method is as follows Required parameters: learning rate , l-layer artificial neural network , training dataset .

[0053] Output results: converted spiking neural network , delay pulse counter parameter .

[0054] (1) Randomly sample a subset from the dataset , is the total number of data points sampled from the dataset

[0055] (2) Replace the rectified linear activation function in the artificial neural network with a rectified linear activation function with an upper bound

[0056] (3) Initialize the upper bound parameters of all rectified linear activation functions with an upper bound .

[0057] (4) Input into the network, calculate , and obtain the input of each layer .

[0058] (5) Let . (6) ,

[0060] (7) Calculate the number of channels in the current layer . ​

[0061] (8) Let . (9)

[0063] (10) be the number of neurons in the th layer and the th channel. For the corresponding upper bound parameter, calculate .

[0064] (11) If , then go to (12); otherwise, go back to (9).

[0065] (12) If , then go to (13); otherwise, go back to (6).

[0066] (13) R represents the rectified linear activation function with an upper bound. Calculate .

[0067] (14) Copy the weights of the artificial neural network to the spiking neural network .

[0068] (15) Set the threshold of each channel in each layer of the spiking neural network to .

[0069] (16) Return .

[0070] See Figure 3 . The method specifically includes the following steps: S301. Optimize the spiking neural network to obtain the target spiking neural network.

[0071] The spiking neural network is trained by the method described above Figure 1 .

[0072] Among them, the spiking neural network can be optimized through the structure of the spiking neural network, the initial membrane potential of each neuron, and the corresponding time step of the spiking neural network.

[0073] S302. Input the target image into the target spiking neural network to obtain the model prediction value.

[0074] Furthermore, input the target image into the optimized target spiking neural network to obtain the model prediction value.

[0075] In the embodiments of the present application, the trained spiking neural network is optimized in multiple aspects such as the structure of the spiking neural network, the initial membrane potential of each neuron, and the corresponding time step of the spiking neural network to improve the inference performance of the spiking neural network.

[0076] In some embodiments, the spiking neural network includes a convolutional layer and a neuron layer. The convolutional layer is placed before the neuron layer to optimize the spiking neural network. To obtain the target spiking neural network, the following steps are included: when the spiking neural network includes a max-pooling layer, place the max-pooling layer before the neuron layer and before the convolutional layer to obtain the target network structure; based on the spike output thresholds of the neurons in the trained spiking neural network, adjust the initial membrane potential of each neuron to a preset value to obtain the target initial membrane potential; based on the target network structure and the target initial membrane potential, optimize the spiking neural network to obtain the target spiking neural network.

[0077] In some embodiments, the spiking neural network may include a max-pooling layer. The max-pooling layer is a common downsampling method for artificial neural networks. Its main function is to reduce the size of the feature map while retaining the most important information, improve computational efficiency, and enhance the translational invariance of the model. The max-pooling layer slides a fixed-size window over the feature map and selects the maximum value within each window as the output. Suppose the input feature map is , the pooling window size is , the stride is , then the calculation method of a certain element of the output feature map is as follows:

[0078] where represents the index range of the pooling window in the input feature map.

[0079] The convolutional layer and the neuron layer have an upstream and downstream relationship in the data flow of the spiking neural network. The local features extracted by the convolutional layer are passed as input to the neuron layer, and the neuron layer further processes these features and generates output spike signals. For example, in a convolutional spiking neural network, the convolutional layer extracts the local features of an image, and the neuron layer processes these features and passes them to the next layer.

[0080] In artificial neural networks, the max-pooling layer is commonly placed after the neuron layer. However, in spiking neural networks, the output of neurons is represented by binary spikes. Therefore, during the max-pooling downsampling process, there are usually multiple neuron outputs in the pooling window, and the output of neurons in the spiking network is 0 or 1, that is, no spike or a spike, resulting in the inability to correctly select the maximum value as the target.

[0081] Therefore, a postsynaptic max-pooling layer is proposed, placing the pooling layer before the neuron layer and after the weight (synaptic) connection to ensure the effectiveness of the max-pooling layer.

[0082] That is, the target feature map corresponding to the target image is first subjected to maximum pooling processing, and the processed target feature map is further input into the neuron layer to obtain the corresponding pulse emission result.

[0083] In some embodiments, after obtaining the processed target feature map, the processed target feature map can be converted into an input sequence so that the convolution layer can obtain a corresponding convolution result based on the input sequence.

[0084] The preset value can be flexibly set based on actual conditions, for example, it can be half of the pulse output threshold, one third of the pulse output threshold, and so on.

[0085] In some embodiments, a spiking neuron layer refers to a layer composed of spiking neurons, and a set of equations can be used to describe the charging, discharging, and resetting behavior of spiking neurons. The first equation represents the charging process of neurons:

[0086] is the membrane potential after charging, yes Input at the moment, represents the voltage at time t, represents the initial membrane potential, is the membrane time constant. This differential equation describes the charging process of a spiking neuron in continuous time. In actual calculations, the charging equation is obtained by discretizing the continuous-time differential equation. The discretization method is to perform differentiation on the differential process.

[0087] From the above formula, we can see that the higher the initial membrane potential, The higher the initial membrane potential, the higher the membrane potential after charging. When the membrane potential is above the pulse output threshold, a pulse is output. Therefore, the higher the initial membrane potential, the more easily neurons are activated. Easily activated neurons may help the network converge faster because they are more sensitive to changes in gradients, thereby accelerating the weight update process. Neurons are sensitive to small changes in input, allowing the model to capture more subtle feature differences, thereby improving the model's ability to represent and distinguish complex data.

[0088] The embodiments of the present application optimize the pulse neural network based on the target network structure and the target initial membrane potential, thereby improving the effectiveness of the maximum pooling layer and making neurons more easily activated, thereby accelerating the speed of network convergence and the weight update process, and enabling the model to capture more subtle feature differences, thereby improving the model's ability to express and distinguish complex data.

[0089] After the optimization is completed, the target spiking neural network can be used to perform inference on the target image.

[0090] In some embodiments, inputting a target image into a target spiking neural network to obtain a model prediction value includes: obtaining an image sequence of the target image; obtaining a target activation function corresponding to each neuron target upper bound parameter in a trained artificial neural network, where the artificial neural network is trained by the method Figure 1 shown; calculating a delay time step of the spiking neural network based on the target activation function of each neuron, where the delay time step is an initial time step corresponding to a valid prediction value; and determining the model prediction value based on the image sequence, the delay time step, and the total time step of the target spiking neural network.

[0091] In some embodiments, the image features of the target image may be extracted first, the image features are converted into a pulse sequence, and further, the delay time step of the spiking neural network is calculated based on the target activation function of each neuron, where the delay time step is an initial time step corresponding to a valid prediction value.

[0092] Generally, in the process of outputting a model prediction value based on a pulse sequence, due to the pulse output threshold of neurons, neurons cannot output pulses in the first few time steps. Therefore, the prediction values output in the previous steps do not contribute much to the final model prediction value. Therefore, to improve the efficiency of the model prediction value, the delay time step of the spiking neural network can be calculated. The delay time step is an initial time step corresponding to a valid prediction value, that is, the prediction values in the time steps before the delay time step do not contribute much to the final model prediction value.

[0093] In some embodiments, in the process of updating the upper bound parameters of each neuron, each upper bound parameter corresponds to an activation function, and the activation function is also continuously updated. When obtaining the target upper bound parameters, the target activation functions corresponding to each neuron are also obtained.

[0094] In some embodiments, for the activation function of the th layer of the artificial neural network model, the input will be obtained. If the current input is , the expression of the activation function is .

[0095] In some embodiments, calculating the delay time step of the spiking neural network based on the target activation function of each neuron includes:

[0096] where is the delay time step, L is the number of layers of the artificial neural network, is the pulse output threshold of the th layer and the cth neuron, represents the number of elements in each layer, Represents the mean of the activation function with an upper bound parameter, Represents taking the maximum element.

[0097] In some embodiments, determining the model prediction value based on the delay time step and the total time step of the target spiking neural network includes: obtaining the prediction value of each time step from the delay time step to the termination time step of the total time step, and determining the model prediction value based on the prediction value of each time step.

[0098] Among them, each spiking neural network will set the total time step of the pulse sequence, that is, within each time step of the total time step, the same pulse sequence will be input, and each time step will output a corresponding prediction value.

[0099] Between the absence of delay time steps, the model prediction value will be determined based on the prediction values output by all time steps. However, the prediction values corresponding to the time steps before the delay time step do not contribute much to the model prediction value. Therefore, determining the model prediction value based on the prediction values output by each time step will result in inaccurate model prediction values.

[0100] After determining the delay time step, each time step from the delay time step to the termination time step of the total time step can be determined, and based on the image sequence, the prediction value of each time step is determined to determine the model prediction value. Since the prediction values corresponding to the time steps before the delay time step are removed, the accuracy of the model prediction value can be improved.

[0101] In some embodiments, determining the model prediction value based on the prediction value of each time step includes: taking the average of the prediction values of each time step to obtain the model prediction value.

[0102] Among them, the average of the prediction values of each time step can be taken to obtain the model prediction value. It is also possible to perform weighted summation on the prediction values of each time step to obtain the model prediction value. The weight coefficients corresponding to the prediction values of each time step can be flexibly set based on the actual situation and will not be elaborated here.

[0103] Figure 4 It is a schematic flow chart of a model inference method provided by an embodiment of the present application.

[0104] As Figure 4 shown, each spiking neural network includes multiple convolutional layers and multiple linear layers. Each convolutional layer and each linear layer are composed of multiple spiking neurons. In the embodiment of the present application, the initial membrane potential of the spiking neuron is set to half of the pulse output threshold, that is .

[0105] After setting the initial membrane potential, the samples to be tested are input at each time step, that is, the samples to be tested are output from t = 1 to t = T, and each time step from the acquisition delay time step to the termination time step of the total time step is based on; the model prediction value is determined based on the prediction value of each time step. That is, the model prediction value is determined based on the prediction values of the time steps from t = t0 to t = T.

[0106] In some embodiments, the above Figure 3 The process of the model inference method is as follows: Required parameters: Inference time step , spiking neural network , delayed pulse counter parameter input data .

[0107] Output: Model prediction value .

[0108] (1) Let t = 0.

[0109] (2) t = t + 1.

[0110] (3) Input x into the spiking neural network to obtain .

[0111] (4) If , then go to (5); otherwise, return to (2).

[0112] (5) Model prediction value .

[0113] (6) Return the model prediction value .

[0114] The embodiment of the present application also provides a model training device, which is used to execute the above Figure 1 The model training method provided by the embodiment. As Figure 5 shown, the device includes an acquisition module 501, a setting module 502, an update module 503, and a determination module 504.

[0115] The acquisition module 501 is used to acquire a pre-trained artificial neural network and a data set, and the data set includes sample images; The setting module 502 is used to set corresponding upper bound parameters for each neuron in the artificial neural network; The update module 503 is used to train the artificial neural network based on the data set, update the upper bound parameters of each neuron, and obtain a trained artificial neural network; The acquisition module 501 is used to acquire the target upper bound parameters of each neuron in the trained artificial neural network; A determination module 504, configured to determine the pulse output thresholds of the neurons in the spiking neural network based on the target upper bound parameters of the neurons.

[0116] This application provides a model training method, a model inference method, and a device, including: obtaining a pre-trained artificial neural network and a data set, where the data set includes sample images; setting corresponding upper bound parameters for each neuron in the artificial neural network; training the artificial neural network based on the data set, updating the upper bound parameters of each neuron to obtain a trained artificial neural network; obtaining the target upper bound parameters of each neuron in the trained artificial neural network; and determining the pulse output thresholds of the neurons in the spiking neural network based on the target upper bound parameters of the neurons. In the embodiments of this application, by training the artificial neural network, the pulse output thresholds of the spiking neural network are obtained, so as to quickly obtain a trained spiking neural network and improve the training efficiency of the spiking neural network.

[0117] In some embodiments, the setting module 502 is specifically configured to: Set corresponding upper bound parameters for each channel of the feature map output by each convolutional layer; Or set corresponding upper bound parameters for each element of the activation value of each linear layer.

[0118] In some embodiments, for each training process, the optimization process of the upper bound parameters includes: ; Wherein, The upper bound parameter of the c-th channel of the l-th layer after optimization, Is the upper bound parameter of the c-th channel of the l-th layer before optimization in this training, Is the step function, Is the learning rate, Is the number of neurons included in the current channel, Is the input of this training.

[0119] The embodiments of this application also provide a model inference device, which is used to execute the above Figure 3 The model training method provided by the embodiment. As Figure 6 Shown, the device includes an optimization module 601 and an inference module 602.

[0120] The optimization module 601 is configured to optimize the spiking neural network to obtain a target spiking neural network, and the spiking neural network is obtained by Figure 1 The method shown in the figure is trained; The inference module 602 is configured to input the target image into the target spiking neural network to obtain a model prediction value.

[0121] Embodiments of the present application optimize a trained spiking neural network in multiple aspects such as the structure of the spiking neural network, the initial membrane potential of each neuron, and the time steps corresponding to the spiking neural network, so as to improve the inference performance of the spiking neural network.

[0122] In some embodiments, an optimization module 601, the spiking neural network includes a convolutional layer and a neuron layer, and the convolutional layer is placed before the neuron layer, specifically for: In the case where the spiking neural network includes a max pooling layer, place the max pooling layer before the neuron layer and before the convolutional layer to obtain a target network structure; Based on the spike output thresholds of the neurons in the trained spiking neural network, adjust the initial membrane potential of each neuron to a preset value to obtain a target initial membrane potential; Based on the target network structure and the target initial membrane potential, optimize the spiking neural network to obtain a target spiking neural network.

[0123] In some embodiments, an inference module 602, specifically for: Obtain the image sequence of the target image; Obtain the target activation function corresponding to the target upper bound parameter of each neuron in the trained artificial neural network, and the artificial neural network is obtained by Figure 1 The method shown is trained; Based on the target activation function of each neuron, calculate the delay time step of the spiking neural network, and the delay time step is the initial time step corresponding to the valid prediction value; Based on the image sequence, the delay time step, and the total time step of the target spiking neural network, determine the model prediction value.

[0124] In some embodiments, the inference module 602 is further specifically for:

[0125] Wherein, Is the delay time step, L is the number of layers of the spiking neural network, Is the Spike output threshold of the c-th neuron in the layer, Represents the number of elements in each layer, Represents the mean value of the activation function with upper bound parameters, Represents taking the maximum element.

[0126] In some embodiments, the inference module 602 is further specifically for: Obtain the prediction value of each time step from the delay time step to the termination time step of the total time step; Determine the model prediction value based on the prediction values of each time step.

[0127] In some embodiments, the inference module 602 is further specifically configured to: Calculate the average value of the prediction values of each time step to obtain the model prediction value.

[0128] The embodiments of the present application further provide an electronic device to execute the above model training method. Please refer to Figure 7 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. As Figure 7 shown, the electronic device 7 includes: a processor 700, a memory 701, a bus 702, and a communication interface 703. The processor 700, the communication interface 703, and the memory 701 are connected through the bus 702. A computer program that can run on the processor 700 is stored in the memory 701. When the processor 700 runs the computer program, it executes the model training method or the model inference method provided by any one of the foregoing embodiments of the present application.

[0129] Among them, the memory 701 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 703 (which can be wired or wireless), a communication connection is established between this device network element and at least one other network element, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0130] The bus 702 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 701 is used to store the program. After receiving the execution instruction, the processor 700 executes the program. The model training method disclosed in any one of the foregoing embodiments of the present application can be applied to the processor 700 or implemented by the processor 700.

[0131] The processor 700 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method may be completed by the integrated logic circuit of the hardware in the processor 700 or the instructions in the form of software. The above-mentioned processor 700 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 701, and the processor 700 reads the information in the memory 701 and combines its hardware to complete the steps of the above method.

[0132] The electronic device provided in the embodiments of the present application and the model training method or model inference method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by them.

[0133] The embodiments of the present application also provide a computer-readable storage medium corresponding to the model training method provided in the foregoing embodiments. Please refer to Figure 8 , which shows that the computer-readable storage medium is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the model training method or model inference method provided in any of the foregoing embodiments.

[0134] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0135] The computer-readable storage medium provided by the above embodiments of the present application and the model training method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0136] It should be noted that: In the specification provided herein, a large number of specific details are set forth. However, it is understood that the embodiments of the present application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0137] In addition, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not others, the combination of features of different embodiments means within the scope of the present application and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0138] The above are only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A model training method, characterized in that, Including: Obtaining a pre-trained artificial neural network and a data set, the data set including sample images; Setting corresponding upper bound parameters for each neuron in the artificial neural network; Training the artificial neural network based on the data set, updating the upper bound parameters of each neuron, and obtaining a trained artificial neural network; Obtaining the target upper bound parameters of each neuron in the trained artificial neural network; Determining the pulse output thresholds of each neuron in the spiking neural network based on the target upper bound parameters of each neuron.

2. The method according to claim 1, characterized in that The artificial neural network includes a plurality of convolutional layers and a plurality of linear layers. The setting of corresponding upper bound parameters for each neuron in the artificial neural network includes: Setting corresponding upper bound parameters for each channel of the feature map output by each convolutional layer; Or setting corresponding upper bound parameters for each element of the activation value of each linear layer.

3. The method according to claim 1 or 2, wherein For each training process, the optimization process of the upper bound parameters includes: ； Among them, The upper bound parameter of the c-th channel in the l-th layer after optimization, Is the upper bound parameter of the c-th channel in the l-th layer before the optimization of this training, Is the step function, Is the learning rate, Is the number of neurons included in the current channel, Is the input of this training.

4. A model inference method, characterized in that, The method includes: Optimizing the spiking neural network to obtain a target spiking neural network, the spiking neural network being trained by the method according to any one of claims 1-3; Inputting a target image into the target spiking neural network to obtain a model prediction value.

5. The method according to claim 4, characterized in that, The spiking neural network includes a convolutional layer and a neuron layer, the convolutional layer being placed before the neuron layer. The optimizing the spiking neural network to obtain a target spiking neural network includes: In the case where the spiking neural network includes a max pooling layer, placing the max pooling layer before the neuron layer and before the convolutional layer to obtain a target network structure; Based on the pulse output thresholds of each neuron in the trained spiking neural network, adjusting the initial membrane potential of each neuron to a preset value to obtain a target initial membrane potential; Optimizing the spiking neural network based on the target network structure and the target initial membrane potential to obtain a target spiking neural network.

6. The method according to claim 4, wherein The inputting the target image into the target spiking neural network to obtain a model prediction value includes: Obtaining an image sequence of the target image; Obtaining the target activation function corresponding to the target upper bound parameters of each neuron in the trained artificial neural network, the artificial neural network being trained by the method according to any one of claims 1-3; Calculating the delay time steps of the spiking neural network based on the target activation functions of each neuron, the delay time steps being the initial time steps corresponding to valid prediction values; Determining the model prediction value based on the image sequence, the delay time steps, and the total time steps of the target spiking neural network.

7. The method according to claim 6, wherein The calculating the delay time steps of the spiking neural network based on the target activation functions of each neuron includes: Among them, is the delay time step, L is the number of layers of the artificial neural network, is the pulse output threshold of the c-th neuron in the l-th layer, represents the number of elements in each layer, represents the mean of the activation function with an upper bound parameter, represents taking the maximum element.

8. The method according to claim 6, characterized in that, The determining the model prediction value based on the image sequence, the delay time steps, and the total time steps of the target spiking neural network includes: Obtaining the prediction values at each time step from the delay time step to the termination time step of the total time steps; Determining the model prediction value based on the prediction values at each time step.

9. The method according to claim 8, wherein the determining the model prediction value based on the prediction values at each time step includes: The average value of the predicted values for each of the time steps is obtained to get the model prediction value.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the method according to any one of claims 1-3 or 4-9.

Images