Optimization method and device of pulse neural network acceleration circuit, equipment and medium

Abstract

The invention relates to the technical field of neural networks, in particular to an optimization method and device of a pulse neural network acceleration circuit, equipment and a medium. The method comprises the following steps: acquiring at least one key neuron in a preset spiking neural network model; splitting the at least one key neuron to obtain at least one neuron group comprising a plurality of different thresholds; and estimating the optimal threshold configuration of at least one neuron group comprising a plurality of different thresholds by using a preset Monte Carlo sampling circuit, generating an optimized spiking neural network, and mapping the optimized spiking neural network to the target neuromorphic hardware. Therefore, more neurons are distributed for important activation values in the model, a traditional fixed mapping mode that one activation value corresponds to one neuron is broken, the problem that the spiking neural network representation precision is insufficient in finite time steps is solved, the time steps needed by reasoning are remarkably reduced on the premise that the model performance is kept, and the reasoning accuracy is improved. Therefore, system delay and power consumption are reduced.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to an optimization method, device, equipment and medium for a pulse neural network acceleration circuit. Background Art

[0002] With the rapid development of artificial intelligence (AI), traditional artificial neural networks (ANNs) have excelled in processing complex tasks, but their high energy consumption is increasingly becoming a limiting factor. This is particularly true in resource-constrained scenarios such as mobile devices, the Internet of Things (IoT), and edge computing. Against this backdrop, spiking neural networks (SNNs) have garnered widespread attention due to their bio-inspired design and low power consumption.

[0003] In the related art, in the frequency encoding of current pulse neural networks, each activation value of the pre-trained neural network is mapped to a single neuron of the neuromorphic chip, which generates 0 to T pulses in T time steps to represent different activation levels.

[0004] However, most existing neuromorphic hardware only supports single-bit neurons and fixed time step designs, resulting in significantly insufficient representation accuracy of spiking neural networks within a limited time step, which urgently needs to be addressed. Summary of the Invention

[0005] The present application provides a method, apparatus, device and medium for optimizing a pulse neural network acceleration circuit to solve the problem of insufficient representation accuracy of pulse neural networks within a finite time step, significantly reducing the time step required for inference while maintaining model performance, thereby reducing system latency and power consumption.

[0006] To achieve the above objectives, the first embodiment of the present application provides a method for optimizing a pulse neural network acceleration circuit, comprising the following steps:

[0007] Obtaining at least one key neuron in a preset spiking neural network model, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model;

[0008] Performing a splitting operation on the at least one key neuron to obtain at least one neuron group including a plurality of different thresholds;

[0009] The optimal threshold configuration of the at least one neuron group including multiple different thresholds is calculated using a preset Monte Carlo sampling circuit to generate an optimized spiking neural network, and the optimized spiking neural network is mapped to the target neuromorphic hardware.

[0010] According to one embodiment of the present application, obtaining at least one key neuron in a preset spiking neural network model includes:

[0011] Calculating the sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit;

[0012] The at least one key neuron is determined based on the sensitivity index of each neuron.

[0013] According to one embodiment of the present application, the step of calculating the sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit includes:

[0014] Calculating the residual potential of each neuron using the forward propagation unit in the preset sensitivity estimation circuit;

[0015] Utilizing the back propagation unit in the preset sensitivity estimation circuit, calculating the gradient of the output feature of each neuron with respect to the intermediate feature;

[0016] The sensitivity index of each neuron is calculated based on the residual potential of each neuron and the gradient of the output feature of the corresponding neuron to the intermediate feature.

[0017] According to one embodiment of the present application, the sensitivity index of each neuron is:

[0018]

[0019] Among them, ‖·‖_2 is the L2 norm, σ_fea is the sensitivity index of a single neuron, is the gradient of the output feature of the single neuron to the intermediate feature, is the output feature of the single neuron, is the intermediate feature of the single neuron, and v_fea is the residual potential of the single neuron.

[0020] According to one embodiment of the present application, the step of calculating the optimal threshold configuration of the at least one neuron group including multiple different thresholds using a preset Monte Carlo sampling circuit includes:

[0021] The residual potential of each neuron is used to construct the cumulative distribution function of the residual potential;

[0022] Obtaining a network search initial value, and based on the cumulative distribution function of the residual potential and the network search initial value, calculating the optimal threshold configuration of the at least one neuron group including multiple different thresholds using the preset Monte Carlo sampling circuit and Newton iteration strategy.

[0023] According to the optimization method for a spiking neural network acceleration circuit proposed in an embodiment of the present application, by obtaining at least one key neuron in a preset spiking neural network model and performing a splitting operation on the at least one key neuron, at least one neuron group containing multiple different thresholds can be obtained; the optimal threshold configuration of the at least one neuron group containing multiple different thresholds is estimated using a preset Monte Carlo sampling circuit, an optimized spiking neural network is generated, and the optimized spiking neural network is mapped to the target neuromorphic hardware. Thus, by allocating more neurons to important activation values ​​in the model, breaking the traditional fixed mapping mode of "one activation value corresponds to one neuron", the problem of insufficient representation accuracy of spiking neural networks within a limited time step is solved, and the time step required for inference is significantly reduced while maintaining model performance, thereby reducing system latency and power consumption.

[0024] To achieve the above objectives, a second embodiment of the present application provides an optimization device for a pulsed neural network acceleration circuit, comprising:

[0025] An acquisition module, configured to acquire at least one key neuron in a preset spiking neural network model, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model;

[0026] an acquisition module, configured to perform a splitting operation on the at least one key neuron to obtain at least one neuron group comprising a plurality of different thresholds;

[0027] A generation module is configured to calculate the optimal threshold configuration of the at least one neuron group comprising multiple neuron groups with different thresholds using a preset Monte Carlo sampling circuit, generate an optimized spiking neural network, and map the optimized spiking neural network to target neuromorphic hardware.

[0028] According to one embodiment of the present application, the acquisition module includes:

[0029] a calculation unit, configured to calculate a sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit;

[0030] A determination unit is configured to determine the at least one key neuron based on the sensitivity index of each neuron.

[0031] According to one embodiment of the present application, the computing unit is specifically configured to:

[0032] Calculating the residual potential of each neuron using the forward propagation unit in the preset sensitivity estimation circuit;

[0033] Utilizing the back propagation unit in the preset sensitivity estimation circuit, calculating the gradient of the output feature of each neuron with respect to the intermediate feature;

[0034] The sensitivity index of each neuron is calculated based on the residual potential of each neuron and the gradient of the output feature of the corresponding neuron to the intermediate feature.

[0035] According to one embodiment of the present application, the sensitivity index of each neuron is:

[0036]

[0037] Among them, ‖·‖_2 is the L2 norm, σ_fea is the sensitivity index of a single neuron, is the gradient of the output feature of the single neuron to the intermediate feature, is the output feature of the single neuron, is the intermediate feature of the single neuron, and v_fea is the residual potential of the single neuron.

[0038] According to one embodiment of the present application, the generating module is specifically configured to:

[0039] The residual potential of each neuron is used to construct the cumulative distribution function of the residual potential;

[0040] Obtaining a network search initial value, and based on the cumulative distribution function of the residual potential and the network search initial value, calculating the optimal threshold configuration of the at least one neuron group including multiple different thresholds using the preset Monte Carlo sampling circuit and Newton iteration strategy.

[0041] According to the optimization device for the spiking neural network acceleration circuit proposed in the embodiment of the present application, by obtaining at least one key neuron in a preset spiking neural network model and performing a splitting operation on the at least one key neuron, at least one neuron group containing multiple different thresholds can be obtained; the optimal threshold configuration of at least one neuron group containing multiple different thresholds is estimated using a preset Monte Carlo sampling circuit, an optimized spiking neural network is generated, and the optimized spiking neural network is mapped to the target neuromorphic hardware. Thus, by allocating more neurons to important activation values ​​in the model, breaking the traditional fixed mapping mode of "one activation value corresponds to one neuron", the problem of insufficient representation accuracy of spiking neural networks within a limited time step is solved, and the time step required for inference is significantly reduced while maintaining model performance, thereby reducing system latency and power consumption.

[0042] To achieve the above-mentioned objectives, the third aspect of the present application proposes an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the optimization method of the pulse neural network acceleration circuit as described in the above-mentioned embodiment.

[0043] To achieve the above-mentioned objectives, the fourth embodiment of the present application proposes a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the optimization method of the pulse neural network acceleration circuit as described in the above-mentioned embodiment.

[0044] To achieve the above-mentioned objectives, the fifth embodiment of the present application proposes a computer program product, which includes a computer program. When the computer program is executed by a processor, it is used to implement the optimization method of the pulse neural network acceleration circuit as described in the above embodiment.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0047] Figure 1 A flowchart of a method for optimizing a pulse neural network acceleration circuit according to an embodiment of the present application;

[0048] Figure 2 1 is a block diagram of an optimization system for a pulsed neural network acceleration circuit according to one embodiment of the present application;

[0049] Figure 3 1 is a schematic diagram of an implementation process of a sensitivity estimation circuit according to an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of pseudo code of an operation process of a sensitivity estimation circuit according to an embodiment of the present application;

[0051] Figure 5 is a block diagram of a compilation mapping circuit according to one embodiment of the present application;

[0052] Figure 6 A schematic diagram of an implementation process of a threshold splitting optimization circuit according to an embodiment of the present application;

[0053] Figure 7 A schematic diagram of pseudo code for an operation process of a threshold splitting optimization circuit according to an embodiment of the present application;

[0054] Figure 8 Schematic diagram of a block diagram of an optimization device for a pulsed neural network acceleration circuit according to an embodiment of the present application;

[0055] Figure 9 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0057] The following describes, with reference to the accompanying drawings, the optimization method, device, equipment, and medium for the pulse neural network acceleration circuit proposed in accordance with the embodiments of the present application.

[0058] Figure 1 This is a flow chart of a method for optimizing a pulse neural network acceleration circuit according to an embodiment of the present application.

[0059] Before introducing the optimization method of the pulse neural network acceleration circuit proposed in the embodiment of the present application, the optimization system of the pulse neural network acceleration circuit involved in the method is first introduced. Figure 2 As shown in Figure 1, the system includes the following components: a sensitivity estimation circuit, a threshold splitting optimization circuit, a compilation and mapping circuit, and an execution scheduling circuit. The sensitivity estimation circuit is used to identify key neurons in a neural network (i.e., a pre-trained deep neural network) and determine its splitting strategy; the threshold splitting optimization circuit is used to calculate the optimal threshold of the split neurons; the compilation and mapping circuit is used to map the optimized neural network to the target neuromorphic hardware; and the execution scheduling circuit is used to control the pulse transmission and computational flow during inference.

[0060] For example, Figure 1 As shown, the optimization method of the pulse neural network acceleration circuit includes the following steps:

[0061] In step S101, at least one key neuron in a preset spiking neural network model is obtained, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model.

[0062] It is understandable that a spiking neural network is composed of multiple spiking neurons, each of which can simulate the behavior of biological neurons and process information by receiving, integrating input signals and generating pulses. In a standard spiking neural network, the input signal is processed by a series of neurons and synapses to ultimately generate an output. In this process, the activation state of each neuron (i.e., whether a pulse is generated) will affect the activation of subsequent neurons, thereby affecting the output of the entire network. Key neurons refer to neurons that have a significant impact on the output of the network. By accurately identifying the key neurons in the preset spiking neural network model, key neurons can be given priority for resource allocation (such as dynamic neuron allocation) in the subsequent optimization process to improve the performance and efficiency of the network.

[0063] The preset pulse neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model, which means that the method of the embodiment of the present application can be directly applied to the pre-trained ANN-SNN conversion model or the pre-trained SNN model without the need for an additional special training process, thereby saving training costs and time.

[0064] The following details how to identify key neurons.

[0065] As a possible implementation method, in some embodiments, obtaining at least one key neuron in a preset pulse neural network model includes: using a preset sensitivity estimation circuit to calculate the sensitivity index of each neuron in the preset pulse neural network model; and determining at least one key neuron based on the sensitivity index of each neuron.

[0066] Specifically, the embodiment of the present application can use a preset sensitivity estimation circuit (ie Figure 2 The sensitivity estimation circuit in the circuit calculates the sensitivity index of each neuron in the preset spiking neural network model. This sensitivity index is used to assess the influence of each neuron on the network output, that is, to quantify the importance of each neuron. Neurons with higher sensitivity indexes have a greater impact on the network output. Based on the calculated sensitivity index of each neuron, the key neurons in the preset spiking neural network model can be identified.

[0067] Furthermore, in some embodiments, a preset sensitivity estimation circuit is used to calculate the sensitivity index of each neuron in a preset pulse neural network model, including: using the forward propagation unit in the preset sensitivity estimation circuit to calculate the residual potential of each neuron; using the back propagation unit in the preset sensitivity estimation circuit to calculate the gradient of the output feature of each neuron with respect to the intermediate feature; and calculating the sensitivity index of each neuron based on the residual potential of each neuron and the gradient of the output feature of the corresponding neuron with respect to the intermediate feature.

[0068] Optionally, in some embodiments, the sensitivity index of each neuron is:

[0069]

[0070] Among them, ‖·‖_2 is the L2 norm, σ_fea is the sensitivity index of a single neuron, is the gradient of the output feature of a single neuron to the intermediate feature, is the output feature of a single neuron, is the intermediate feature of a single neuron, and v_fea is the residual potential of a single neuron.

[0071] It can be understood that residual potential refers to the potential value remaining after a neuron in a spiking neural network fires a spike. The membrane potential of a neuron is a measure of its excitability. When the membrane potential reaches or exceeds a certain threshold, the neuron fires a spike. After the spike is fired, the membrane potential decreases by a corresponding value (usually minus the threshold), and the remaining value is the residual potential. The residual potential preserves the remaining excitation state of the neuron after the spike is fired and is crucial for the integration and response of subsequent input signals.

[0072] like Figure 3 As shown in the figure, the preset sensitivity estimation circuit includes a forward propagation unit and a backward propagation unit. Through two different modes of reasoning, the influence of the residual potential of each neuron on the output accuracy can be calculated to obtain the sensitivity index of each neuron, and then the most critical activation value (i.e., key neuron) in the network can be identified. Specifically, Figure 4 As shown in the figure, for each sample batch, first, the forward propagation unit is used to perform standard forward propagation to record the final residual potential v_fea(T) of each neuron (T is the time step); secondly, the pulse neuron is replaced by the ReLU (Rectified Linear Unit, activation function) function. In the process of back propagation using the back propagation unit, a forward propagation is first performed to generate the output, and then the back propagation is used to calculate the gradient of the output feature of each neuron to the intermediate feature. The purpose of this step is to determine the degree of influence of the activation of each neuron on the network output. By calculating the gradient, the contribution of each neuron to the final output can be quantified; finally, according to the residual potential of each neuron and the gradient of the output feature of the corresponding neuron to the intermediate feature, the sensitivity index of each neuron can be calculated.

[0073] In step S102, a splitting operation is performed on at least one key neuron to obtain at least one neuron group including multiple neuron groups with different thresholds.

[0074] It's understandable that in a spiking neural network, each spiking neuron has a primitive threshold θ (activation value). When the neuron's membrane potential reaches or exceeds this primitive threshold θ, the neuron fires a pulse. The primitive threshold θ is a key parameter of spiking neurons, determining their excitability—that is, when they become activated and transmit signals downstream.

[0075] That is, after determining multiple key neurons, a threshold splitting optimization circuit can be applied to each key neuron to split the single key neuron into groups of neurons with different thresholds. These thresholds serve as discrete quantized bit weights that collectively represent the original threshold. For example, a key neuron with an original threshold of θ can be split into neurons with thresholds of θ1 and θ2, respectively. The neurons with thresholds of θ1 and θ2 will form the neuron group after the key neuron is split.

[0076] It should be noted that in practical applications, the number of neuron splits can be flexibly adjusted according to the actual network requirements and optimization goals, and is not specifically limited here. Increasing the number of neurons after splitting can more finely represent the activation value of the original neuron, improving the network's representation ability and reasoning accuracy. However, increasing the number of splits will also increase computational complexity and resource consumption, so it is necessary to comprehensively consider the task requirements and hardware conditions. In addition, the neuron splitting process can be iterative. Through multiple iterations of the neuron splitting process, the representation accuracy of each activation value can be gradually improved. Each round of iteration adds an additional bit to each activation value to more finely represent its value. That is, initially, each activation value may be represented by one bit (0 or 1). After k rounds of iterations, the number of bits for each activation value can be increased to 1+k.

[0077] In step S103, a preset Monte Carlo sampling circuit is used to calculate the optimal threshold configuration of at least one neuron group including multiple different thresholds, generate an optimized spiking neural network, and map the optimized spiking neural network to the target neuromorphic hardware.

[0078] Specifically, to effectively improve the coding efficiency of spiking neural networks, after splitting each key neuron, a preset Monte Carlo sampling circuit is used to estimate the optimal threshold configuration for each group of neurons containing multiple different thresholds corresponding to each split key neuron, thereby obtaining an optimized spiking neural network. Using a compilation mapping circuit to map the optimized spiking neural network to the target neuromorphic hardware enables efficient hardware deployment. Simultaneously, a run-time scheduling circuit controls the pulse transmission and computation during the inference process, accelerating inference and reducing time steps.

[0079] It should be noted that the compilation mapping circuit can dynamically map a single logical activation value in the network to a different number of physical neurons according to the degree of influence of the activation value on the network output. This mechanism allows for higher representation accuracy for important activation values, thereby improving the performance and efficiency of the network. Figure 5 As shown, this compilation mapping circuit, based on a traditional spiking neural network, additionally incorporates a split mapping table and a split threshold memory. The split mapping table records the mapping relationship between the original neuron and the split neuron group. Specifically, when a key neuron is split into multiple physical neurons, the split mapping table records this mapping relationship. The split mapping table determines the distribution of membrane potential across the split neurons. When an input signal arrives, the split mapping table guides how the membrane potential is distributed across the multiple split neurons, ensuring that each split neuron is correctly assigned the corresponding potential value. The split threshold memory stores the thresholds of each split neuron. The split threshold memory is used to determine the pulse output threshold of each neuron. During inference, the system can read the corresponding threshold from the split threshold memory to determine whether the neuron's membrane potential meets the conditions for spiking. By using the split mapping table and split threshold memory, dynamic splitting of key neurons is achieved, improving accuracy.

[0080] As a possible implementation method, in some embodiments, a preset Monte Carlo sampling circuit is used to calculate the optimal threshold configuration of at least one neuron group including multiple different thresholds, including: using the residual potential of each neuron to construct a cumulative distribution function of the residual potential; obtaining an initial value of the network search, and based on the cumulative distribution function of the residual potential and the initial value of the network search, using a preset Monte Carlo sampling circuit and a Newton iteration strategy to calculate the optimal threshold configuration of at least one neuron group including multiple different thresholds.

[0081] Specifically, if Figure 6 and Figure 7 As shown, the sample data of the residual potential v of the original neuron is collected through the calibration sample set, where the residual potential refers to the membrane potential value of the neuron at the end of time step T. The collected residual potential sample data is used to construct the cumulative distribution function F(v) of the residual potential. This cumulative distribution function describes the distribution of the residual potential v and reflects the probability of occurrence of different potential values. Next, based on the original threshold θ_old of the key neuron to be split, the network search initial value θ_k (i.e., the new threshold after splitting) can be obtained. Assuming that the key neuron to be split is split into two neurons, then θ_k = θ_old / 2, and the Newton iteration strategy is applied to solve the following equation:

[0082] 2F(θ_k)-F(θ_old-θ_k)-1=0;

[0083] θ_k:=θ_k-[2F(θ_k)-F(θ_old-θ_k)-1] / [2p(θ_k)+p(θ_old-θ_k)];

[0084] where F(·) is the cumulative distribution function of the residual potential, and p(θ) is estimated using the interval [θ-δ,θ+δ].

[0085] According to the network search initial value θ_k obtained, the thresholds of the two neurons after splitting are set as follows:

[0086] θ_k+1=θ_old-θ_k;

[0087] In order to reduce the quantization error caused by the Heaviside function, a compensation amount can be added to the input of each time step:

[0088] q(t)+=0.5θ_k+1 / T;

[0089] Therefore, we can obtain the optimal threshold configuration for this splitting case as (θ_k,θ_k+1).

[0090] To facilitate those skilled in the art to further understand the optimization method of the pulse neural network acceleration circuit proposed in the embodiments of the present application, further explanation is given below in conjunction with specific embodiments.

[0091] In Example 1, taking the CIFAR-10 (Canadian Institute for Advanced Research 10-Class Dataset) image classification task as an example, the optimization method of the pulse neural network acceleration circuit proposed in the embodiment of the present application is specifically implemented on the ResNet-18 (Residual Network-18) network as follows:

[0092] Step 1, network preprocessing: convert the pre-trained ResNet-18 ANN model into an SNN model, and set the initial threshold to 1 / 64 of the maximum activation value of each layer;

[0093] Step 2: Sensitivity estimation: 1,000 calibration images were selected as input and sensitivity evaluation and threshold splitting were performed. Neuron sensitivity analysis results showed that about 15% of neurons had a significant impact on the output.

[0094] Step 3: Dynamic neuron allocation: Split the neurons ranked in the top 15% of sensitivity. Each split neuron is converted into two neurons, increasing the overall number of neurons by approximately 15%.

[0095] Step 4, threshold optimization: Apply the optimal threshold solution algorithm to the split neuron group to obtain the optimal threshold pairing of the split neurons. The threshold splitting method of each neuron is different, and the average value is (0.62θ_old, 0.38θ_old);

[0096] Step 5: Mapping to neuromorphic hardware: Deploy the optimized network to the Lynxi HP201 neuromorphic processor, reducing the time step setting from 64 to 16.

[0097] Step 6, performance evaluation: classification accuracy: original SNN 93.2%, optimized SNN 92.9%; inference latency: reduced by 75%; power consumption: reduced by 78%.

[0098] Example (II) takes the neuromorphic implementation of the Diffusion model as an example. The specific implementation steps are as follows:

[0099] Step 1, network preprocessing: obtain the U-Net SNN model based on SNN direct pre-training;

[0100] Step 2: Sensitivity estimation: Sensitivity analysis shows that in the U-Net structure, deep neurons in the downsampling and upsampling paths have the greatest impact on the generated quality;

[0101] Step 3, dynamic neuron allocation: apply up to 3 rounds of splitting to these key neurons, that is, one key neuron is split into at most 4 physical neurons;

[0102] Step 4, threshold optimization: a multi-level iterative scheme is used to ensure a refined activation value representation;

[0103] Step 5: Mapping to neuromorphic hardware: Deploy the optimized network to the Lynxi HP201 neuromorphic processor, reducing the time step setting from 128 to 24.

[0104] Step 6: Performance evaluation: With only a 25% increase in the total number of neurons, the inference time step was reduced from 128 to 24; the FID (Fréchet Inception Distance, a metric used to evaluate the effectiveness of generative models) of the generated image improved from 42.6 to 39.1, while energy consumption was reduced by 81%.

[0105] In summary, the optimization method of the pulse neural network acceleration circuit proposed in the embodiment of the present application has significant advantages over the traditional SNN:

[0106] (1) Reasoning time optimization: By transferring the encoding part of the time dimension to the spatial dimension, the required time steps can be reduced by 80%, significantly improving the reasoning speed;

[0107] (2) Reduced energy consumption: The required time steps are reduced, resulting in reduced overall energy consumption;

[0108] (3) Accuracy guarantee: Through theoretical proof and experimental verification, this method can maintain an accuracy level comparable to that of the original network;

[0109] (4) Flexible adaptability: can be seamlessly integrated into various pre-trained SNN models without the need for additional training;

[0110] (5) Hardware compatibility: The design is compatible with existing neuromorphic hardware architectures, without the need for special circuits;

[0111] The specific technical indicators are compared as shown in Table 1 (taking ResNet20 based on ANN-SNN conversion with a batch size of 32 as an example):

[0112] Table 1

[0113] Technical indicators Traditional SNN implementation This application method Change ratio Reasoning steps 64 12-16 Reduced by 75%-80% Power consumption 100％ 20％-25％ Reduce by 75%-80% Storage usage 2G <4.5G 2.25 times increase Loss of precision 0 <0.5％ Almost no loss

[0114] According to the optimization method for a spiking neural network acceleration circuit proposed in an embodiment of the present application, by obtaining at least one key neuron in a preset spiking neural network model and performing a splitting operation on the at least one key neuron, at least one neuron group containing multiple different thresholds can be obtained; the optimal threshold configuration of the at least one neuron group containing multiple different thresholds is estimated using a preset Monte Carlo sampling circuit, an optimized spiking neural network is generated, and the optimized spiking neural network is mapped to the target neuromorphic hardware. Thus, by allocating more neurons to important activation values ​​in the model, breaking the traditional fixed mapping mode of "one activation value corresponds to one neuron", the problem of insufficient representation accuracy of spiking neural networks within a limited time step is solved, and the time step required for inference is significantly reduced while maintaining model performance, thereby reducing system latency and power consumption.

[0115] Next, the optimization device for the pulse neural network acceleration circuit proposed in the embodiment of the present application is described with reference to the accompanying drawings.

[0116] Figure 8 It is a block diagram of an optimization device for a pulse neural network acceleration circuit according to an embodiment of the present application.

[0117] like Figure 8 As shown, the optimization device 10 for the pulse neural network acceleration circuit includes: an acquisition module 100, an acquisition module 200 and a generation module 300.

[0118] The acquisition module 100 is used to acquire at least one key neuron in a preset spiking neural network model, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model;

[0119] An acquisition module 200 is configured to perform a splitting operation on at least one key neuron to obtain at least one neuron group including a plurality of different thresholds;

[0120] The generation module 300 is used to calculate the optimal threshold configuration of at least one neuron group including multiple different thresholds using a preset Monte Carlo sampling circuit, generate an optimized spiking neural network, and map the optimized spiking neural network to the target neuromorphic hardware.

[0121] Optionally, in some embodiments, the acquisition module 100 includes:

[0122] a calculation unit, configured to calculate a sensitivity index of each neuron in a preset spiking neural network model using a preset sensitivity estimation circuit;

[0123] The determining unit is configured to determine at least one key neuron based on the sensitivity index of each neuron.

[0124] Optionally, in some embodiments, the computing unit is specifically configured to:

[0125] Calculate the residual potential of each neuron using the forward propagation unit in the preset sensitivity estimation circuit;

[0126] Using the back-propagation unit in the preset sensitivity estimation circuit, the gradient of the output feature of each neuron with respect to the intermediate feature is calculated;

[0127] The sensitivity index of each neuron is calculated based on the residual potential of each neuron and the gradient of the output feature of the corresponding neuron with respect to the intermediate feature.

[0128] Optionally, in some embodiments, the sensitivity index of each neuron is:

[0129]

[0130] Among them, ‖·‖_2 is the L2 norm, σ_fea is the sensitivity index of a single neuron, is the gradient of the output feature of a single neuron to the intermediate feature, is the output feature of a single neuron, is the intermediate feature of a single neuron, and v_fea is the residual potential of a single neuron.

[0131] Optionally, in some embodiments, the generating module 300 is specifically configured to:

[0132] The residual potential of each neuron is used to construct the cumulative distribution function of the residual potential;

[0133] Obtaining a network search initial value, and based on the cumulative distribution function of the residual potential and the network search initial value, using a preset Monte Carlo sampling circuit and a Newton iteration strategy to calculate the optimal threshold configuration of at least one neuron group containing multiple different thresholds.

[0134] It should be noted that the above explanation of the embodiment of the optimization method of the pulse neural network acceleration circuit is also applicable to the optimization device of the pulse neural network acceleration circuit of this embodiment, and will not be repeated here.

[0135] According to the optimization device for the spiking neural network acceleration circuit proposed in the embodiment of the present application, by obtaining at least one key neuron in a preset spiking neural network model and performing a splitting operation on the at least one key neuron, at least one neuron group containing multiple different thresholds can be obtained; the optimal threshold configuration of at least one neuron group containing multiple different thresholds is estimated using a preset Monte Carlo sampling circuit, an optimized spiking neural network is generated, and the optimized spiking neural network is mapped to the target neuromorphic hardware. Thus, by allocating more neurons to important activation values ​​in the model, breaking the traditional fixed mapping mode of "one activation value corresponds to one neuron", the problem of insufficient representation accuracy of spiking neural networks within a limited time step is solved, and the time step required for inference is significantly reduced while maintaining model performance, thereby reducing system latency and power consumption.

[0136] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0137] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0138] When the processor 902 executes the program, the optimization method of the pulse neural network acceleration circuit provided in the above embodiment is implemented.

[0139] Furthermore, the electronic device further includes:

[0140] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0141] The memory 901 is used to store computer programs that can be run on the processor 902 .

[0142] The memory 901 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0143] If the memory 901, the processor 902, and the communication interface 903 are implemented independently, the communication interface 903, the memory 901, and the processor 902 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0144] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0145] The processor 902 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0146] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for optimizing the pulse neural network acceleration circuit.

[0147] An embodiment of the present application also provides a computer program product, which includes a computer program, which, when executed by a processor, implements the above-mentioned optimization method for the pulse neural network acceleration circuit.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0150] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for optimizing a pulse neural network acceleration circuit, characterized in that: The following steps are involved: Obtaining at least one key neuron in a preset spiking neural network model, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model; Performing a splitting operation on the at least one key neuron to obtain at least one neuron group including a plurality of different thresholds; The optimal threshold configuration of the at least one neuron group including multiple different thresholds is calculated using a preset Monte Carlo sampling circuit to generate an optimized spiking neural network, and the optimized spiking neural network is mapped to the target neuromorphic hardware.

2. The method according to claim 1, characterized in that The obtaining of at least one key neuron in a preset spiking neural network model includes: Calculating the sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit; The at least one key neuron is determined based on the sensitivity index of each neuron.

3. The method according to claim 2, characterized in that The method of calculating the sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit includes: Calculating the residual potential of each neuron using the forward propagation unit in the preset sensitivity estimation circuit; Utilizing the back propagation unit in the preset sensitivity estimation circuit, calculating the gradient of the output feature of each neuron with respect to the intermediate feature; The sensitivity index of each neuron is calculated based on the residual potential of each neuron and the gradient of the output feature of the corresponding neuron to the intermediate feature.

4. The method according to claim 3, characterized in that The sensitivity index of each neuron is: Among them, ‖·‖_2 is the L2 norm, σ_fea is the sensitivity index of a single neuron, is the gradient of the output feature of the single neuron to the intermediate feature, is the output feature of the single neuron, is the intermediate feature of the single neuron, and v_fea is the residual potential of the single neuron.

5. The method according to claim 1, wherein The method of calculating the optimal threshold configuration of the at least one neuron group including a plurality of different thresholds by using a preset Monte Carlo sampling circuit includes: The residual potential of each neuron is used to construct the cumulative distribution function of the residual potential; Obtaining a network search initial value, and based on the cumulative distribution function of the residual potential and the network search initial value, calculating the optimal threshold configuration of the at least one neuron group including multiple different thresholds using the preset Monte Carlo sampling circuit and Newton iteration strategy.

6. An optimization device for a pulse neural network acceleration circuit, characterized in that: include: An acquisition module, configured to acquire at least one key neuron in a preset spiking neural network model, wherein the preset spiking neural network model is a pre-trained ANN-SNN conversion model or a pre-trained SNN model; an acquisition module, configured to perform a splitting operation on the at least one key neuron to obtain at least one neuron group comprising a plurality of different thresholds; A generation module is configured to calculate the optimal threshold configuration of the at least one neuron group comprising multiple neuron groups with different thresholds using a preset Monte Carlo sampling circuit, generate an optimized spiking neural network, and map the optimized spiking neural network to target neuromorphic hardware.

7. The device according to claim 6, characterized in that The acquisition module includes: a calculation unit, configured to calculate a sensitivity index of each neuron in the preset spiking neural network model using a preset sensitivity estimation circuit; A determination unit is configured to determine the at least one key neuron based on the sensitivity index of each neuron.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for optimizing the pulse neural network acceleration circuit as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the optimization method of the pulse neural network acceleration circuit as described in any one of claims 1 to 5.

10. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the optimization method of the pulse neural network acceleration circuit according to any one of claims 1 to 5.

Images