Conversion method of artificial neural network model, storage medium, and program product

Abstract

The embodiment of the application provides a conversion method, a storage medium and a program product of an artificial neural network model, relates to the technical field of artificial intelligence, and the method comprises the following steps: after obtaining a pre-trained artificial neural network model, converting each nonlinear operator in the artificial neural network model into a corresponding pulse module, wherein the pulse module comprises a differential expectation compensation module, the differential expectation compensation module is used for calculating an output increment according to cumulative membrane potential, a differential pulse neuron is inserted in each pulse module, the differential pulse neuron updates an encoding activation value when a pulse is emitted, otherwise the encoding activation value remains unchanged, a bias term of a linear operator located in a previous layer of each nonlinear operator is removed, and an initial membrane potential of the differential pulse neuron inserted in the pulse module corresponding to the nonlinear operator is set as the bias term, and the encoding activation value is only updated when a pulse is emitted, so that the loss caused by updating the encoding activation value regardless of whether a pulse is emitted or not is avoided, and energy consumption is significantly reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for converting artificial neural network models, a storage medium, and a program product. Background Technology

[0002] Artificial neural networks (ANNs) are widely used in various fields such as image classification, speech recognition, and natural language processing due to their powerful data processing capabilities. However, the computational model of traditional ANNs relies on floating-point calculations, resulting in high energy consumption and high latency, which limits their application, especially in edge computing and mobile devices. Spiking neural networks (SNNs), as a novel computational model, are network models that use neuronal membrane potentials and pulses to remember and transmit information, exhibiting extremely low power consumption and high biological interpretability.

[0003] Converting a pre-trained ANN model to a SNN model is an effective method for quickly building high-performance, low-power SNNs. Existing ANN-to-SNN conversion methods mainly use rate coding, which requires a large number of pulses to transmit information, resulting in high energy consumption and significant inference latency. Summary of the Invention

[0004] This application provides a method for converting an artificial neural network model, a storage medium, and a program product to alleviate or solve one or more technical problems existing in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for converting an artificial neural network model, comprising:

[0006] Obtain the pre-trained artificial neural network model;

[0007] Each nonlinear operator in the artificial neural network model is converted into a corresponding pulse module. The pulse module includes a differential expectation compensation module, which is used to calculate the output increment based on the accumulated membrane potential.

[0008] Differential spiking neurons are inserted into each of the aforementioned pulse modules. The differential spiking neurons update the encoded activation value when firing a pulse; otherwise, the encoded activation value remains unchanged.

[0009] Remove the bias terms of the linear operators located in the layer preceding each nonlinear operator, and set the initial membrane potential of the differential spiking neuron inserted in the spiking module corresponding to the nonlinear operator as the bias term.

[0010] In some embodiments of this application, after obtaining the pre-trained artificial neural network model, the method further includes:

[0011] Calculate the statistical values ​​of the output values ​​of each layer in the artificial neural network model;

[0012] The threshold range is configured according to the statistical values, and multiple pulse firing thresholds of the differential spiking neuron are derived from the threshold range in an integer power of 2.

[0013] In some embodiments of this application, after deriving multiple pulse firing thresholds for the differential spiking neurons, the method further includes:

[0014] A threshold index is added to the output vector of the differential spiking neuron. The threshold index is used to index the threshold corresponding to the current pulse.

[0015] In some embodiments of this application, the method further includes:

[0016] Based on the threshold index, a bitwise shift operation is performed on the computational weights of the linear operators in the next layer of the nonlinear operator, wherein the number of bits in the bitwise shift operation is determined according to the threshold index.

[0017] In some embodiments of this application, at the time step No. The input current of the differential spiking neuron in the spiking module of the layer for: ,in, It is the first The output of the linear operator described in the layer, This refers to the encoding error term recorded by the differential spiking neuron, with an initial value of the first... The bias term of the linear operator described in the layer, , It is the first The output of the pulse module described in the layer.

[0018] In some embodiments of this application, the nonlinear operator includes at least one of the following nonlinear functions: Gaussian error linear unit (GELU), Sigmoid linear unit (SiLU), max pooling (MaxPool), normalization function (LayerNorm), and normalized exponential function (Softmax); the differential spiking neuron is inserted after the differential expectation compensation module; at time step... No. The nonlinear function described in the layer The corresponding output of the differential expectation compensation module The formula is as follows: ,in, It is the first The differential expectation compensation module of the layer is in the first... The cumulative membrane potential at each time step.

[0019] In some embodiments of this application, the nonlinear operator includes a bimatrix multiplication operator, and a differential spiking neuron is inserted after each of the two matrix inputs of the bimatrix multiplication operator; at time step No. The layer's two matrix multiplication operator The corresponding output of the differential expectation compensation module The formula is as follows: ,in, Input matrix The corresponding cumulative membrane potential, Input matrix The corresponding cumulative membrane potential.

[0020] In some embodiments of this application, at the time step No. The encoded activation value of the differential spiking neuron in the spiking module of the layer The update formula is as follows: ,in, It is the first The output of the pulse module described in the layer.

[0021] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0022] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0023] Fourthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements any of the methods described in the embodiments of this application.

[0024] Based on any of the above technical solutions, this application has at least the following beneficial effects or advantages:

[0025] After obtaining the pre-trained artificial neural network model, each nonlinear operator in the artificial neural network model is converted into a corresponding spiking module. The spiking module includes a differential expectation compensation module, which is used to calculate the output increment based on the accumulated membrane potential. Differential spiking neurons are inserted into each spiking module. The differential spiking neurons update the encoded activation value when firing a pulse; otherwise, the encoded activation value remains unchanged. The bias terms of the linear operators in the layer preceding each nonlinear operator are removed, and the initial membrane potential of the differential spiking neurons inserted in the spiking module corresponding to the nonlinear operator is set as the bias term. Based on the differential encoding mechanism that updates the encoded activation value only when firing a pulse, the loss caused by updating the encoded activation value regardless of whether a pulse is fired is avoided, significantly reducing energy consumption. The output of each layer can approximate a higher precision continuous value with a lower pulse density, reducing redundant pulse firing. Furthermore, by removing the bias terms in the linear layer and accumulating them in the membrane potential of the next layer, the addition operation of the bias terms can be avoided repeatedly at each time step.

[0026] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0027] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0028] Figure 1 A flowchart illustrating a method for converting an artificial neural network model according to an embodiment of this application is shown;

[0029] Figure 2 This illustration shows a schematic diagram of an example of a conversion method for an artificial neural network model provided in an embodiment of this application, in which a single-input nonlinear function is converted into an impulse module;

[0030] Figure 3 This illustration shows a schematic diagram of the pulse module after conversion by the dual-matrix input matrix multiplication operator, which is yet another example of a conversion method for an artificial neural network model provided in this application.

[0031] Figure 4 This illustration shows another flowchart of a method for converting an artificial neural network model according to an embodiment of this application;

[0032] Figure 5 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0033] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0034] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. It should be noted that the application scenarios or application examples provided in this application are for ease of understanding, and the embodiments of this application do not specifically limit the application of the technical solutions.

[0035] In related technologies, a traditional ANN to SNN conversion method is provided, which is a conversion method based on a rate coding mechanism. The rate coding mechanism represents the activation value through the average firing rate, but this method varies with time step... The increase in this will cause "encoded value decay", requiring more pulse compensation, which in turn increases energy consumption and latency.

[0036] When transmitting information based on rate coding, each time step No. Layer distribution rate This is used to represent the activation value of a neuron. The following formula gives the output firing rate. With output signal The relationship between them:

[0037] Formula 1

[0038] When the When a layer is composed of spiking neurons, its state can be represented as follows: ,in Indicates at time step Whether to issue a pulse, and This represents the firing threshold of a neuron. When a neuron is not firing a pulse, the update term for the firing rate is the firing rate from the previous time step. When a neuron fires a pulse, the firing rate update term becomes... .

[0039] However, rate coding methods have a problem: with time steps As the weight of the early input increases, the encoded value gradually "decays." This is due to the weight of the early input. along with As the pulses increase and decrease, their effects gradually weaken. Therefore, the system needs more pulses to compensate for this attenuation effect, thus increasing the number of pulses and energy consumption.

[0040] To address this issue, this application proposes a novel encoding mechanism, known as differential encoding. The core of differential encoding lies in updating the encoded value only when the neuron actually fires a pulse, thereby avoiding the losses caused by rate averaging in rate encoding mechanisms, reducing redundant pulse firing, and significantly lowering energy consumption.

[0041] In differential coding, let's denote... For the first The pulse output signal of the layer pulse module is the signal of the neuron at time step. The actual output (t is a positive integer greater than or equal to 1) when a pulse is emitted, The value is a multiple of the firing threshold; when no pulse is fired... The value is 0. (Definition) The encoded output value at this time step, at time step No. Encoded activation values ​​of differential spiking neurons in the layer spiking module Defined as from the first 1 Step to the first Step All The average value, through encoding activation value This indicates the degree of neuronal activation. The relationship between them is shown in the following formula:

[0042] Formula 2

[0043] Formula 3

[0044] in from 1 Start, initial value .

[0045] It should be noted that the above equation is only used to define the differential coding mechanism and does not involve the dynamics of neurons. In other words, the above equation explains a series of impulse outputs. How is it mapped to the actual output encoded value? of.

[0046] Comparing the encoding activation value formula 1 of the rate coding mechanism with the encoding activation value formula 2 of the differential coding mechanism proposed in the embodiments of this application, it can be seen that the main difference between the two is that the rate coding updates the encoding value at each time step (and introduces decay), while the differential coding only updates the encoding value when the neuron actually fires a pulse.

[0047] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0048] See Figure 1 The flowchart shown is a method for converting an artificial neural network model. The method specifically includes steps 101-104.

[0049] Step 101: Obtain the pre-trained artificial neural network model;

[0050] Step 102: Convert each nonlinear operator in the artificial neural network model into a corresponding pulse module. The pulse module includes a differential expectation compensation module, which is used to calculate the output increment based on the accumulated membrane potential.

[0051] Step 103: Insert differential spiking neurons into each spiking module. The differential spiking neurons update the encoded activation value when firing a pulse; otherwise, the encoded activation value remains unchanged.

[0052] Step 104: Remove the bias terms of the linear operators located in the layer preceding each nonlinear operator, and set the initial membrane potential of the differential spiking neurons inserted in the spiking module corresponding to the nonlinear operator as the bias term.

[0053] In different application scenarios, the artificial neural network model conversion method provided in this application embodiment can be configured into different forms such as services, applications, application plugins, and mini-programs.

[0054] This application does not specifically limit the type, structure, function, or parameters of the artificial neural network model. For example, the artificial neural network model includes multiple layers, each layer has multiple neurons, and the multiple neurons can be of the same type. According to the different neuron types, the multiple layers in the artificial neural network model can be divided into two categories: linear layers and nonlinear layers. Linear layers include multiple convolution operators, while nonlinear layers include nonlinear functions or two-input matrix multiplication operators, etc. Nonlinear functions include Gaussian error linear unit (GELU), Sigmoid linear unit (SiLU), max pooling (MaxPool), normalization function (LayerNorm), normalized exponential function (Softmax), etc.

[0055] After obtaining the artificial neural network model, each nonlinear operator in the artificial neural network model is converted into a corresponding spiking module. The spiking module includes a differential expectation compensation module, and differential spiking neurons are inserted into each spiking module.

[0056] refer to Figure 2In some embodiments of this application, the nonlinear operator includes at least one of the above nonlinear functions. For the pulse module used to replace the nonlinear function, the differential spiking neuron is inserted after the differential expectation compensation module. The output of the pulse module is the output of the differential spiking neuron.

[0057] refer to Figure 3 The nonlinear operator may also include a bimatrix multiplication operator. For the pulse module used to replace the bimatrix multiplication operator, a differential spiking neuron is inserted after each of the two matrix inputs of the bimatrix multiplication operator. That is, the pulse module corresponding to the bimatrix multiplication operator includes two differential spiking neurons, inserted before the differential expectation compensation module. The output of the pulse module is the output of the differential expectation compensation module.

[0058] Under the differential coding mechanism framework defined by Equations 2 and 3 proposed in this application, the embodiments of this application provide two differential expectation compensation modules, which are respectively used for differential expectation compensation modules corresponding to single-input nonlinear functions and differential expectation compensation modules corresponding to matrix multiplication operators with dual matrix inputs. Based on the general dynamic formula provided by theorems 2 and 3, the single-input and / or dual-input nonlinear layers are decomposed into differential forms in the time dimension, so that these complex modules can be losslessly mapped to time-series outputs without training.

[0059] It is understandable that artificial neural network models may not have matrix operations with two matrix inputs, or alternatively, certain nonlinear layers may be left unconverted. In such cases, it is not necessary to use the corresponding spiking module for conversion. The specific implementation depends on the application scenario and is not limited to converting all nonlinear layers.

[0060] For a single-input nonlinear function in a pre-trained ANN It has one and only one input. That is, the upper level (the first) The output of a linear operator connected in layers. A single-input nonlinear function. The module is converted to a differential expectation compensation module to calculate the output increment based on the accumulated membrane potential, which can be described by the following formula:

[0061] Formula 4

[0062] It is the current time step The accumulated membrane potential, or cumulative membrane potential, is used to represent the state of the neuron's membrane potential, transferring the value from the previous layer (the first layer) to the current layer. Bias terms of layered linear operators Set as the current layer (the first layer) The initial value of the membrane potential of the layer (i.e., the initial membrane potential) ), and the previous layer of linear operators bias term in After removing the bias terms from the linear operators in the previous layer (including each unit and / or convolution operator in the fully connected layer), the result is... , The weights of the preceding linear operator, after removing the bias terms of the linear layers (including fully connected layers and / or convolutional layers), do not require the linear layers to repeatedly perform the addition of bias terms at each time step. The bias terms are incorporated into the initial membrane potential of the subsequently inserted pulse module. This layer at time step... Output It will be used as input for the next layer. It is the previous layer at the time step The encoded activation value.

[0063] In some specific implementations, the cumulative membrane potential of each layer can be saved in a specified storage location. Each time step, the cumulative membrane potential value of the layer is updated, thereby reducing the amount of cumulative membrane potential stored.

[0064] At time step No. Layer nonlinear function The output of the corresponding differential expectation compensation module The formula is as follows:

[0065] Formula 5

[0066] in, It is the first The differential expectation compensation module of the layer is in the first layer. The cumulative membrane potential at each time step.

[0067] The differential expectation compensation module corresponding to a single-input nonlinear function can store two variables: cumulative membrane potential. and This is to avoid repetitive calculations at each time step and improve efficiency.

[0068] refer to Figure 3 In time step , No. Bimatrix multiplication operator with layer bimatrix input The output of the corresponding differential expectation compensation module The formula is as follows:

[0069] Formula 6

[0070] Formula 7

[0071] Formula 8

[0072] in, Input matrix The corresponding cumulative membrane potential, Input matrix The corresponding cumulative membrane potential. and These are the encoded activation values ​​corresponding to the two input channels, and the output is... It will be used as the input for the next layer.

[0073] The dual-input differential expectation compensation module can maintain two variables: and This is to record the accumulated input information.

[0074] Therefore, the function of the differential expectation compensation module is to calculate the increment of the expected output of the nonlinear operator in the current state at each time step, based on the input pulse current and accumulated membrane potential of the current layer. This increment is not a real floating-point information, but is approximated by pulse information.

[0075] Graded Units enable SNN networks to integrate variation information from nonlinear layers, thereby allowing for free conversion of training on complex neural network structures, including CNNs and Transformers.

[0076] Under the rate coding mechanism in related technologies, the output of the previous layer It is directly used as the input current of the current layer, that is: Under the differential coding mechanism proposed in this application embodiment, the input current... A bias term for linear layer removal needs to be added, that is, adjusted according to the following formula, to convert any spiking neuron, such as a spiking neuron in a rate coding mechanism, into a differential spiking neuron proposed in the embodiments of this application:

[0077] Formula 9

[0078] Formula 10

[0079] In time step No. The input current of the differential spiking neurons in the layer spiking module. It is the first The pulse output signal of the layer linear operator, For the encoding error term recorded by differential spiking neurons, It is the first Output of the layer pulse module.

[0080] refer to Figure 3 The encoding error terms of the differential spiking neurons corresponding to the two input channels are respectively and The update rule is based on formula 10.

[0081] If the linear operator connected in the previous layer has a bias Then the initial membrane potential Encoding error term in ,otherwise .

[0082] When processing linear layers (such as convolutional and fully connected layers), differential encoding methods incorporate bias terms into the initial membrane potential of subsequent layers, thereby avoiding repeated accumulation of bias at each time step. This reconstruction method significantly improves the running efficiency and hardware friendliness of SNNs while preserving the linear mapping relationships in ANNs.

[0083] Differential spiking neurons, as proposed in Formula 4, are inserted into each pulse module. The input current of the differential spiking neurons is adjusted based on Formulas 9 and 10. The encoded activation value is updated when a pulse is emitted; otherwise, the encoded activation value remains unchanged.

[0084] The differential spiking neuron proposed in this application improves the energy efficiency of spiking neural networks during inference. Within the differential coding framework, the input current depends not only on the current input pulse but also on the historical potential state. This mechanism allows neurons to adaptively adjust their output range, particularly suitable for neurons with multi-threshold structures, thereby improving representation accuracy and further compressing the number of pulses. Since most computations occur in fully connected layers, convolutional layers, and matrix multiplication layers, introducing a spiking neuron layer before these linear layers imbues the computation with event-driven characteristics, effectively reducing the network's energy consumption.

[0085] The artificial neural network model conversion method provided in this application, after obtaining the pre-trained artificial neural network model, converts each nonlinear operator in the artificial neural network model into a corresponding pulse module. The pulse module includes a differential expectation compensation module, which is used to calculate the output increment based on the accumulated membrane potential and insert differential spiking neurons into each pulse module. The differential spiking neurons update the encoded activation value when firing a pulse; otherwise, the encoded activation value remains unchanged. The bias term of the linear operator in the layer preceding each nonlinear operator is removed, and the initial membrane potential of the differential spiking neurons inserted in the pulse module corresponding to the nonlinear operator is set as the bias term. Based on the differential encoding mechanism that updates the encoded activation value only when firing a pulse, the loss caused by updating the encoded activation value regardless of whether a pulse is fired is avoided, significantly reducing energy consumption. The output of each layer can approximate a higher precision continuous value with a lower pulse density, reducing redundant pulse firing. Furthermore, by removing the bias term in the linear layer and accumulating it in the membrane potential of the next layer, the addition operation of the bias term can be avoided repeatedly at each time step.

[0086] In some embodiments of this application, after obtaining the pre-trained artificial neural network model, the following steps are further performed: calculating the statistical values ​​of the output values ​​of each layer in the artificial neural network model, configuring a threshold range according to the statistical values, and deriving multiple pulse firing thresholds of the differential spiking neurons based on the threshold range in an integer power of 2.

[0087] In the above embodiments, the differential spiking neuron is a multi-threshold (MT) neuron.

[0088] In related technologies, multi-threshold neurons are defined by several key parameters, including the base threshold. and based on the basic threshold The total derived from multiples of One issuance threshold, among which One is a positive threshold, The first is a negative threshold. Each threshold is in the... Using indexes in layers Indicated, denoted as Its values ​​are as follows:

[0089] Formula 11

[0090] Let the following variables represent the neuron at time step Status:

[0091] Input current, It is the first The layer weights are the result of normalizing the weights of the pre-trained ANN. It is the first Layer The first time step Pulse output on each threshold channel It is a binary value, 1 for issuing and 0 for not issuing. Total output signal, membrane potential before dispensing Membrane potential after discharge This is a multi-threshold pulse determination function; the input is the accumulated membrane potential before firing. and threshold index .

[0092] The dynamic behavior of MT neurons can be described by the following formula:

[0093] Formula 12

[0094] Formula 13

[0095] Formula 14

[0096] Formula 15

[0097] Formula 16

[0098] express The value is at the maximum positive threshold. With the smallest negative threshold Between, for example, in conjunction with the implementation of formula 11, It can be represented as: This neuron fires at most one pulse per time step, matching a threshold channel. When At that point, the model degenerates into an integral-fire (IF) neuron with a negative threshold.

[0099] In the multi-threshold-based differential spiking neuron provided in this application embodiment, the multiple spiking thresholds are designed to be generated proportionally in powers of 2. Since only one threshold can be activated at a time, therefore... Through the This is obtained by performing a shift operation; therefore, when implementing the MT neuron, a unified [mechanism / mechanism] can be calculated first. Then, through displacement operations, efficiently obtain This eliminates the need to store all weight levels individually, thus enabling efficient hardware implementation.

[0100] In specific implementations, the multi-threshold differential spiking neuron can be implemented in any of the following ways: The first implementation directly outputs the pulses of the differential spiking neuron to the next layer; the second implementation expands the output pulses to include... A vector with the nth element, only in the nth element The first bit is set to 1, and the rest are set to 0. At the same time, the weights of the next layer are also expanded into tensors containing multiple threshold dimensions, and the corresponding elements are the thresholds at each level.

[0101] To simplify hardware implementation, this application proposes a hardware implementation method that can be implemented on a GPU, based on the first direct pulse output implementation described above.

[0102] First, based on the weight normalization strategy, the parameters in the ANN are converted into the parameters in the SNN:

[0103] Formula 17

[0104] ANN in the th The weights of the layer, the SNN in the first layer Layer weights ANN in the th Layer bias, SNN in the first layer Layer bias. Set all base thresholds uniformly. , Option 1 is acceptable.

[0105] The base threshold is the threshold for multiple pulse firings. The reference amplitude, exemplarily, the first The firing threshold of 2n pulses for a layer differential spiking neuron can be expressed as: , Based on threshold scaling factor Depending on the differences, multiple pulse firing thresholds can be configured. .

[0106] Represented as:

[0107] Formula 18

[0108] It can statistically analyze the activation values ​​of neurons in each layer of an artificial neural network model, such as maximum value, mean, or quantile, and use these statistical values ​​to determine the firing thresholds for multiple pulses. Set an appropriate dynamic threshold range.

[0109] In some implementations, the threshold range can be divided into multiple pulse firing thresholds. There are several intervals when the cumulative membrane potential is within a certain range. If any interval is selected within a given interval, activate the upper threshold (or lower threshold; in the specific implementation, the activation threshold can be selected and fixed as either the upper or lower limit) of the corresponding interval, and fire a pulse; otherwise, if the accumulated membrane potential is not within the specified range... If the pulse is not emitted at any interval of the interval, but is outside the threshold range, then no pulse will be emitted at the current time step.

[0110] In other implementations, hardware-implemented rules can be used to quickly determine the pulse firing threshold for accumulating membrane potential activation. Specifically, the accumulated membrane potential... Multiply Then, it is encoded according to the IEEE 754 single-precision floating-point format as follows:

[0111] Formula 19

[0112] in: This is the sign bit (1 bit). For the exponent (8 bits). The mantissa (23 bits) is not involved in the threshold matching logic.

[0113] Cumulative membrane potential Multiply The purpose is to get closer to the midpoint of the firing thresholds of two adjacent pulses, since the firing thresholds of two adjacent pulses differ by a factor of two, or by a power of 2.

[0114] It is understandable that, given a value k, the value calculated based on value k is... With numerical values The median is: Therefore, the midpoint between two adjacent pulse firing thresholds is the smaller pulse firing threshold. times.

[0115] After processing the accumulated membrane potential according to Formula 19, it can be directly used. and To determine which pulse firing threshold should be activated at the current time, without executing... Multiple subtractions are used to find or perform multiple comparisons, utilizing... and The hardware time for determining the activation pulse firing threshold is shorter.

[0116] Multi-threshold differential spiking neuron pulse firing determination function The hardware implementation rules are represented as follows:

[0117] Formula 20

[0118] θ is the base threshold, and n is the number of positive thresholds. If the decision function outputs 1, it means a pulse is emitted in the i-th threshold channel; if the output for i is 0, it means no pulse is emitted in the i-th threshold channel. S is the sign bit encoded using Formula 19, where S=0 indicates a positive value (i<n) and S=1 indicates a negative value (i≥n). E is the exponent bit encoded using Formula 19. According to the mapping relationship 1-E, the exponent bit is associated with the threshold index i to achieve fast threshold matching. When the membrane potential is positive (S=0), only the first n positive threshold channels are matched, and the exponent bit E is mapped to the positive threshold index i through i=1-E; when the membrane potential is negative (S=1), only the last n negative threshold channels are matched, and the exponent bit E is mapped to the negative threshold index i through in=1-E.

[0119] The differential spiking neuron in this application utilizes the sign bit S and exponent bit E of the accumulated membrane potential to quickly map the corresponding threshold index at extremely low cost. Compared to the ANN-to-SNN conversion method in related technologies, the multi-threshold differential spiking neurons in this embodiment require an additional threshold index to be transmitted when transmitting information. This method is more hardware-friendly. Although it is no longer a pure binary encoding, it is functionally equivalent to the binary representation in the second implementation described above. Furthermore, the multi-threshold differential spiking neuron in this embodiment is also suitable for asynchronous computational neuromorphic chips because its output remains sparse events.

[0120] In some embodiments of this application, after deriving multiple pulse firing thresholds for the differential spiking neuron, a threshold index can be added to the output vector of the differential spiking neuron. The threshold index is used to index the threshold corresponding to the current firing pulse.

[0121] For example, in the Speck chip, the LIF neurons in the convolutional layers output... To the next layer; if using MT neurons, only the output needs to be expanded to That is, add an additional threshold index. That's all.

[0122] In some embodiments of this application, in the calculation of the next layer, the calculation weight of the linear operator of the next layer of the nonlinear operator can be performed by bit shifting according to the threshold index, and the number of bits in the bit shifting operation is determined according to the threshold index.

[0123] For example, when n = 4, 2n = 8, and the threshold index i = 1, 2, ..., 8, the threshold scaling factor... The weights are 1, 1 / 2, 1 / 4, 1 / 8, -1 / 8, -1 / 4, -1 / 2, and -1, respectively. For regions with positive signs (i ≤ n), shift right by i – 1; for regions with negative signs (i > n), shift right by i - n – 1. Only one set of weights needs to be stored in the hardware. The absolute weight is determined by shifting the value of the threshold index i based on its sign. The next layer's calculation only requires a single bit shift operation on the weights to complete the multiplication (since all thresholds are powers of 2), thus avoiding the overhead of a multiplier.

[0124] Compared with multi-threshold neurons in related technologies, the differential spiking neuron in this application only needs to maintain one additional membrane potential variable, namely the encoding error term. This is used to dynamically adjust the input current as described in Theorem 4. This modification incurs minimal storage overhead. Even after these changes, the entire computation process retains its asynchronous and event-driven characteristics.

[0125] refer to Figure 4 Here is a specific implementation process of an embodiment of this application, which includes the following steps:

[0126] Step 1: Load the pre-trained ANN model

[0127] Load a pre-trained ANN model from an existing framework (such as PyTorch or TensorFlow) and extract its network structure, weight parameters (including convolutional kernels, fully connected matrices, bias terms, etc.), and activation function type. This model will be used as the source network for transformation.

[0128] Step 2: Obtain the threshold parameters of the neuron

[0129] Based on the activation characteristics of different network layers, a firing threshold is constructed for each layer. Specific methods include, but are not limited to:

[0130] 1) Calculate the maximum, mean, or quantile of activation values ​​of neurons in each layer of the ANN, and set an appropriate dynamic range;

[0131] 2) Manually set according to the trade-off between power consumption and accuracy in actual applications;

[0132] 3) Use a uniform fixed threshold or exponentially decreasing multiple thresholds during hardware deployment (e.g., ).

[0133] Finally, the differential coding parameters (including thresholds, etc.) of each neuron are obtained, which prepares for subsequent replacement operations.

[0134] Step 3: Replace the nonlinear layer with a differential expectation compensation module

[0135] Equivalent reconstruction of nonlinear modules in ANNs (such as GELU, Silu, Softmax, LayerNorm, etc.):

[0136] (1) For a single-input function, a compensation unit is constructed using the first-order difference form, as follows: Figure 2 As shown;

[0137] (2) For dual-input modules (such as matrix multiplication and element-wise multiplication), construct a bivariate compensation structure, such as Figure 3 As shown.

[0138] The above modules achieve equivalent representation through time difference calculation, ensuring that the original nonlinear output can be approximated without training.

[0139] Step 4: Insert a differential spiking neuron layer to construct a differential multithreshold neuron structure.

[0140] A differentially encoded multi-threshold neuron layer is inserted before each linear layer or matrix multiplication module. This neuron structure uses dynamic membrane potential to adjust the input, firing at most one pulse per time step. Exponential bit matching is used to achieve fast threshold selection, improving hardware execution efficiency. The neuron is inserted at... Figure 2 , Figure 3 The image has already been displayed in China.

[0141] Step 5: Convert the linear layer to a difference form

[0142] Converting convolutional and fully connected layers in an ANN to differential form includes: (1) removing bias terms (2) Retain the weighted propagation structure: This eliminates the need for repeated biasing. This approach reduces time-step computational complexity and optimizes the event-driven execution efficiency of SNNs.

[0143] Step Six: The inference differential SNN performs the specified task.

[0144] The constructed differential SNN model is then applied to the target task. Due to its low impulse density and high representation accuracy, it is suitable for low-power, real-time edge scenarios and can be deployed on various platforms such as GPUs, FPGAs, or asynchronous neuromorphic chips.

[0145] Figure 5 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 5As shown, the electronic device includes a memory 501 and a processor 502. The memory 501 stores a computer program that can run on the processor 502. When the processor 502 executes the computer program, it implements the method described in the above embodiments. The number of memories 501 and processors 502 can be one or more. In a specific implementation, the electronic device may also include a communication interface 503 for communicating with external devices and exchanging data.

[0146] In practical implementation, if the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0147] Optionally, in a specific implementation, if the memory 501, processor 502 and communication interface 503 are integrated on a single chip, the memory 501, processor 502 and communication interface 503 can communicate with each other through an internal interface.

[0148] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0149] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0150] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0151] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0152] It should be understood that the aforementioned processor can be a CPU (Central Processing Unit), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0153] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0154] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0155] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0156] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0157] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0158] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0159] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes at least one or a combination of the steps of the method embodiments.

[0160] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0161] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for converting an artificial neural network model, characterized in that, include: Obtain the pre-trained artificial neural network model; Each nonlinear operator in the artificial neural network model is converted into a corresponding pulse module. The pulse module includes a differential expectation compensation module, which is used to calculate the output increment based on the accumulated membrane potential. Differential spiking neurons are inserted into each of the aforementioned pulse modules. The differential spiking neurons update the encoded activation value when firing a pulse; otherwise, the encoded activation value remains unchanged. Remove the bias terms of the linear operators located in the layer preceding each of the nonlinear operators, and set the initial membrane potential of the differential spiking neurons inserted in the spiking module corresponding to the nonlinear operator as the bias terms; After obtaining the pre-trained artificial neural network model, the method further includes: Calculate the statistical values ​​of the output values ​​of each layer in the artificial neural network model; Based on the statistical values, a threshold range is configured, and based on the threshold range, multiple pulse firing thresholds for the differential spiking neuron are derived in an integer power of 2-like ratio; the total number of derived pulse firing thresholds is... One, of which One is a positive threshold, One is a negative threshold; The pulse firing threshold activated by the accumulated membrane potential is determined using the following hardware implementation rules: The accumulated membrane potential Multiply The post-encoding is: ;in: For the sign bit, For the exponent, The last digit; according to and Determine the pulse firing threshold to be activated at the current time, and the firing decision function. The hardware implementation rules are represented as follows: θ is the base threshold. If the decision function outputs 1, it indicates that a pulse is emitted in the i-th threshold channel, where i is the index of the pulse emission threshold. If the output for i is 0, it indicates that no pulse is emitted in the i-th threshold channel. When the pulse firing threshold is a positive threshold, The pulse firing threshold is a negative threshold at that time.

2. The method according to claim 1, characterized in that, After deriving multiple pulse firing thresholds for the differential spiking neurons, the method further includes: A threshold index is added to the output vector of the differential spiking neuron. The threshold index is used to index the threshold corresponding to the current pulse.

3. The method according to claim 2, characterized in that, The method further includes: Based on the threshold index, a bitwise shift operation is performed on the computational weights of the linear operators in the next layer of the nonlinear operator, wherein the number of bits in the bitwise shift operation is determined according to the threshold index.

4. The method according to claim 1, characterized in that, At time step No. The input current of the differential spiking neuron in the spiking module of the layer for: ,in, It is the first The output of the linear operator described in the layer, This refers to the encoding error term recorded by the differential spiking neuron, with an initial value of the first... The bias term of the linear operator described in the layer, , It is the first The output of the pulse module described in the layer.

5. The method according to claim 1, characterized in that, The nonlinear operator includes at least one of the following nonlinear functions: Gaussian error linear unit (GELU), Sigmoid linear unit (SiLU), max pooling (MaxPool), normalization function (LayerNorm), and normalized exponential function (Softmax); the differential spiking neuron is inserted after the differential expectation compensation module; at time step... No. The nonlinear function described in the layer The corresponding output of the differential expectation compensation module The formula is as follows: ,in, It is the first The differential expectation compensation module of the layer is in the first... The cumulative membrane potential at each time step.

6. The method according to claim 1, characterized in that, The nonlinear operator includes a bimatrix multiplication operator, with a differential spiking neuron inserted after each of the two matrix inputs of the bimatrix multiplication operator; at time step No. The layer's two matrix multiplication operator The corresponding output of the differential expectation compensation module The formula is as follows: ,in, Input matrix The corresponding cumulative membrane potential, Input matrix The corresponding cumulative membrane potential.

7. The method according to claim 1, characterized in that, At time step No. The encoded activation value of the differential spiking neuron in the spiking module of the layer The update formula is as follows: ,in, It is the first The output of the pulse module described in the layer.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images