A method and system for converting a pulse neural network based on differentiable adaptive optimization

Abstract

The application relates to the technical field of pulse neural network, in particular to a pulse neural network conversion method and system based on a differentiable adaptive optimization, which comprises the following steps: constructing and pre-training a target ANN model; in the pre-trained ANN model, using a global differentiable numerical representation operator to simulate the activation value of each layer; based on a loss function, placing the original weight parameter in the target ANN model and the trainable firing threshold in the global differentiable numerical representation operator in the same optimization framework to jointly fine-tune, obtaining the optimal weight and the optimal threshold learned by each layer; and constructing an equivalent SNN model. The application can optimize the accurate correspondence between continuous activation and pulse sequences, is suitable for various pulse coding modes, maximally reduces quantization errors, can deeply adapt to the deployment requirements of neuromorphic hardware and edge intelligent computing platforms, and provides an efficient and universal model conversion solution for the neuromorphic computing industry.

Description

Technical Field

[0001] This invention relates to the field of spiking neural network technology, specifically to a spiking neural network conversion method and system based on differentiable adaptive optimization. Background Technology

[0002] Spiking neural networks, with their unique advantage of being based on pulse signals, offer a promising approach to high-energy-efficiency computing on neuromorphic hardware. Despite their enormous potential, the performance of spiking neural networks often lags behind that of artificial neural networks, mainly because it is difficult to effectively convert the activation values ​​of artificial neural networks into time-domain pulses.

[0003] In this invention, the activation threshold, pulse time window width, and scaling factor involved in the ANN→SNN conversion process are collectively referred to as conversion parameters. Conversion errors occur when mapping continuous activation values ​​to discrete pulses using these conversion parameters. Existing ANN-SNN conversion methods have the following problems: (1) The transformation parameters depend on manual setting and lack adaptability: Existing methods generally use statistics based on data distribution (such as the maximum activation value, 99.9% quantile, etc.) as fixed scaling factors. The parameter adjustment range is limited, or it depends on specific optimization algorithms. It cannot adaptively optimize the transformation parameters and model weights according to the network structure and task objectives, which leads to significant quantization errors in the transformation process. (2) Requires a long pulse time window and has high inference latency: In order to compensate for the accuracy loss caused by fixed parameters, an extremely long pulse time window is usually required to approximate the activation value of the original network. This results in high SNN inference latency and weakens the energy efficiency advantage, making it difficult to meet the requirements of real-time applications. (3) The conversion process is separated from the training process, and end-to-end optimization is not possible: Most existing methods are used as post-training processing steps. The conversion process is relatively separated from the ANN training process. Its optimization goal is disconnected from the main task loss function of the network, and it is impossible to achieve collaborative optimization of the model and conversion parameters. There is a lack of an end-to-end conversion parameter learning mechanism that is task performance oriented. (4) The transformation model or optimization process is often non-differentiable or locally differentiable, making it impossible to use gradient information for efficient updates. Summary of the Invention

[0004] To address the problems in the prior art, this invention provides a spiking neural network conversion method and system based on differentiable adaptive optimization, which solves the problems of conversion parameters relying on manual setting, high inference latency, and disconnect between training and conversion processes in the prior art.

[0005] This invention is achieved through the following technical solution: A spiking neural network conversion method based on differentiable adaptive optimization includes: Construct and pre-train the target ANN model to obtain the pre-trained ANN model; In the pre-trained ANN model, the activation values ​​of each layer are simulated using a globally differentiable numerical representation operator. Based on the loss function, the original weight parameters in the target ANN model and the trainable activation threshold in the globally differentiable numerical representation operator are placed under the same optimization framework for joint fine-tuning to obtain the optimal weights and the optimal thresholds learned by each layer. Construct an equivalent SNN model based on the optimal weights and the thresholds learned at each layer.

[0006] Preferably, in the pre-trained ANN model, the activation values ​​of each layer are simulated using a globally differentiable numerical representation operator, specifically as follows: For input Output a converted activation value. ;

[0007] in, For trainable scaling factors; In SNN, it represents the firing threshold of a neuron; T represents the pulse time window width, which is the maximum upper limit of the pulse count allowed by the neuron; The approximate function used in the ANN-SNN conversion process is constructed as follows: Input: x Parameter: f Calculation steps: (1) The constraint parameter f is in the range of 0 to 1: f = max(0, min(1, f)) (2) Calculate the intermediate functions g1(x) ~ g3(x): g1(x) = sin( (x - 1) * π ) g2(x) = arccos( f * g1(x) ) g3(x) = 1 - (2 * g2(x)) / π (3) Calculate the intermediate functions g4(x) ~ g6(x): g4(x) = sin( (x - 0.5) * π ) g5(x) = g4(x) / (1-f) g6(x) = arctan( g5(x) ) (4) Calculate roundSNN(x) roundSNN(x) = x - (g3(x) * g6(x)) / π in, The parameter, which controls the smoothness of the rounding operation, is between 0 and 1.

[0008] Preferably, the loss function is the mean squared error (MSE) of the last layer output data of the ANN model before and after the transformation.

[0009] The preferred optimization framework is backpropagation.

[0010] Preferably, the synaptic weights of the SNN model are the same as the weights of each layer of the ANN model.

[0011] Preferably, the SNN model is an integral-distribution model, the first... The activation threshold of a layer neuron is the learned optimal threshold. .

[0012] Preferably, during SNN model inference, neurons accumulate membrane potential based on input, and when the membrane potential exceeds an optimal threshold... A pulse is delivered and the membrane potential is reset.

[0013] A spiking neural network conversion system based on differentiable adaptive optimization, comprising the steps of the method, including: The first module is used to build and pre-train the target ANN model to obtain the pre-trained ANN model. The second module is used to simulate the activation values ​​of each layer in the pre-trained ANN model using a globally differentiable numerical representation operator. The third module is used to jointly fine-tune the original weight parameters in the target ANN model and the trainable activation threshold in the globally differentiable numerical representation operator under the same optimization framework based on the loss function, so as to obtain the optimal weights and the optimal threshold learned by each layer. The fourth module is used to construct an equivalent SNN model based on the optimal weights and the thresholds learned at each layer.

[0014] An electronic device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method.

[0015] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a spiking neural network conversion method based on differentiable adaptive optimization. By modeling the conversion process as a differentiable data representation operation, key parameters can be adaptively learned through gradient descent. This significantly reduces inference latency while ensuring high conversion accuracy, achieving efficient and high-fidelity ANN-SNN conversion.

[0017] On the one hand, this invention designs a globally differentiable numerical representation operator. This operator takes the continuous activation values ​​of the ANN as input and outputs numerical representation results corresponding to the discrete pulse sequence of the spiking neural network. The differentiability of this operator ensures that the gradient can be effectively transmitted to all transformation parameters during backpropagation, laying the foundation for the collaborative optimization of transformation parameters and model weights. This solves the problem of separation between transformation parameter updates and model training in the traditional transformation process. Through the backpropagation algorithm, transformation parameters and model weights can be automatically and adaptively collaboratively optimized according to the task loss function. The transformation process from ANN to SNN is clearly defined as a complete process that can automatically and collaboratively optimize model weights and transformation parameters.

[0018] On the other hand, this invention establishes a direct mapping relationship between conversion parameters and pulse firing characteristics, directly corresponding parameters such as the activation range and characterization accuracy to key characteristics in the pulse firing process, such as the excitation threshold and pulse time window width. Through this mapping mechanism, the optimization of conversion parameters can be directly equivalent to the precise adjustment of pulse firing characteristics.

[0019] This invention provides a complete implementation process: constructing and pre-training an artificial neural network, embedding a globally differentiable numerical representation operator, jointly optimizing the model weights and transformation parameters, and using the optimized parameters for the final spiking neural network transformation. Through discrete numerical representation, it realizes the process of efficiently representing continuous activation values ​​with a finite pulse sequence, thereby improving transformation efficiency and accuracy.

[0020] On the MNIST test set, this invention, through differentiable adaptive optimization during the training phase, enables the model to maintain accuracy similar to that of ANN while reducing the number of impulse steps by more than 40% (the accuracy after conversion is 98.6%, compared to 99.15% accuracy of the original model before conversion), verifying the advantage of this method in balancing accuracy and energy consumption. Attached Figure Description

[0021] Figure 1 This is a flowchart of a spiking neural network conversion method based on differentiable adaptive optimization according to the present invention; Figure 2 A schematic diagram of an apparatus for performing the method described in this invention. Detailed Implementation

[0022] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

[0023] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0024] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0025] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0026] SNN, as a neuromorphic model that simulates the spiking mechanism of biological neurons, possesses core advantages such as asynchronous event-driven operation, high energy efficiency, and strong temporal information representation capabilities. It holds significant application value in scenarios with strict constraints on energy consumption and response speed, such as low-latency edge computing and IoT smart terminals. However, current mainstream ANN-SNN conversion technologies have limitations; they cannot adapt to different network structures and data distributions, especially for complex models and high-precision tasks, making it difficult to balance conversion efficiency with performance loss.

[0027] Traditional ANN-SNN conversion methods typically rely on manually adjusted, fixed-policy activation function thresholds to calculate scaling factors, mapping continuous activation values ​​in an ANN to discrete impulse sequences in an SNN. This approach suffers from significant accuracy loss, long conversion latency, and sensitivity to network structure. The core of this invention lies in formally defining the traditional ANN-to-SNN conversion process as a learnable mapping process. It innovatively designs a globally differentiable numerical representation operator that can be seamlessly embedded into the ANN to be converted for end-to-end training. Within this framework, the key parameters determining the conversion effect are no longer generated according to a fixed policy, but rather serve as trainable parameters of the model. Through backpropagation, these scaling factors can adaptively and collaboratively optimize with model weights based on the task loss function, thereby finding the optimal conversion parameters and model weights best suited to the network structure and dataset, achieving high-fidelity, low-latency ANN-to-SNN conversion.

[0028] This invention discloses a spiking neural network conversion method based on differentiable adaptive optimization, referring to... Figure 1 ,include: Build and pre-train the target ANN model. Use standard methods (such as backpropagation based on gradient descent) to pre-train on the target dataset until the model converges to obtain the pre-trained ANN model. In the pre-trained ANN model, the activation values ​​of each layer are simulated using a globally differentiable numerical representation operator; specifically: For input Output a converted activation value. ;

[0029] in, For trainable scaling factors; In SNN, it represents the firing threshold of a neuron; T represents the pulse time window width, which is the maximum upper limit of the pulse count allowed by the neuron; The approximate function used in the ANN-SNN conversion process is constructed as follows: Input: x Parameter: f Calculation steps: (1) The constraint parameter f is in the range of 0 to 1: f = max(0, min(1, f)) (2) Calculate the intermediate functions g1(x) ~ g3(x): g1(x) = sin( (x - 1) * π ) g2(x) = arccos( f * g1(x) ) g3(x) = 1 - (2 * g2(x)) / π (3) Calculate the intermediate functions g4(x) ~ g6(x): g4(x) = sin( (x - 0.5) * π ) g5(x) = g4(x) / (1-f) g6(x) = arctan( g5(x) ) (4) Calculate roundSNN(x) roundSNN(x) = x - (g3(x) * g6(x)) / π in, The parameter that controls the smoothness of the rounding operation is between 0 and 1. The larger the value of f, the closer it is to the original approximate function.

[0030] Based on the loss function, the original weight parameters of the target ANN model and the trainable activation thresholds in the globally differentiable numerical representation operators are jointly fine-tuned within the same optimization framework to obtain the optimal weights and the optimal thresholds learned at each layer. Specifically, based on the loss function, the model weights and transformation parameters are collaboratively optimized through the same backpropagation process. This ensures that the model weights and transformation parameters can be synchronously and adaptively adjusted according to the final model performance target, which is the key to achieving high-precision, low-latency transformation, rather than the traditional staged optimization strategy.

[0031] The loss function is the mean squared error (MSE) of the last layer output data of the ANN model before and after the transformation.

[0032] Perform a deterministic SNN transformation, which does not require additional training of the SNN itself. The transformation rule is deterministic; specifically, an equivalent SNN model is constructed based on the optimal weights and the thresholds learned at each layer. The synaptic weights in the SNN model are the same as the weights in each layer of the ANN model. The SNN model is an integral-distribution model, and the... The activation threshold of a layer neuron is the learned optimal threshold. During SNN model inference, neurons accumulate membrane potential based on input, and when the membrane potential exceeds the optimal threshold... A pulse is delivered and the membrane potential is reset.

[0033] Compared with existing technologies, this invention provides a direct, end-to-end optimization method for spiking neural network (SNN) transformation based on differentiable adaptive optimization. By embedding a scaling factor into the ANN training graph, the task loss is directly optimized through backpropagation, achieving goal-driven optimization. The parameter optimization direction is highly consistent with the final application goal, leading to a globally optimal solution. A globally differentiable numerical representation operator is introduced, making the entire system (including parameters) globally differentiable, enabling automatic differentiation and optimization. Using the scaling factor as a key parameter in the ANN-to-SNN transformation process, its optimization process is combined with the network weight fine-tuning process, possessing the ability to adaptively adjust threshold and time window parameters based on task data, automatically adapting to the needs of different layers, architectures, and data distributions. It offers greater versatility and robustness, eliminating the need for tedious manual parameter tuning for each new network, achieving a high degree of integration between ANN training and SNN transformation parameter optimization.

[0034] Example The invention will be described in detail using a specific image classification example. The dataset is MNIST, and the model is a pulsed convolutional network based on an SNN structure (Quant-Lenet-SNN-MNIST). The model structure is: Input(28×28) Conv1(3×3×1×32) Pool(2×2) Conv2(3×3×32×64) Pool(2×2) FC1 (800×256) FC2 (256×10).

[0035] S1: Construct and pre-train an artificial neural network A standard Lenet network is built using the PyTorch framework and trained on the MNIST dataset.

[0036] S2: Embedded globally differentiable numerical representation operator To insert the globally differentiable numerical representation operator designed in this invention into the activation of each layer of the Lenet model, that is, to initialize a trainable activation threshold for each layer. (The initial value can be uniformly set to 1.0 or set according to the initial activation value distribution of the layer). The f parameter in the globally differentiable numerical representation operator can generally be set to 0.1.

[0037] S3: Adaptive optimization of model weights and scaling factors.

[0038] The activation threshold values ​​of all weights and differentiable operators in the model are used. Set it to trainable and fine-tune it using a small amount of the MNIST dataset.

[0039] S4: Perform deterministic SNN transformations.

[0040] After fine-tuning, save the optimal weights and the thresholds learned at each layer. .

[0041] Based on these parameters, an equivalent SNN is constructed. The synaptic weights of this SNN are the same as the weights of each layer in the ANN model. The neuron model uses an integral-fire (IF) model, and the threshold for each layer is [value missing]. The obtained SNN parameter configurations are shown in Table 1.

[0042] Table 1 SNN parameter configuration

[0043] In terms of encoding, the input layer employs Poisson rate coding based on a linear feedback shift register (LFSR), while the intermediate and output layers use pulse aggregation coding. Each input pixel fires a pulse according to its grayscale intensity probability within a time window; the output neuron fires a pulse and resets after its membrane potential reaches a threshold, achieving event-driven propagation. Pulse aggregation coding is an encoding method that accumulates or weights pulse firings within a limited time window. This method aggregates multiple pulse events into a single representation value to characterize the activation level of neurons, thereby maintaining information expression capacity while reducing the number of pulses, significantly improving inference efficiency and reducing power consumption.

[0044] For bit width selection, 4-bit fixed-point weights are used for convolutional layer weights, and 8-bit fixed-point weights are used for neuron inputs and outputs. Fully connected layers maintain the same configuration to balance resource consumption and accuracy. 24-bit precision is used in the intermediate storage area to ensure the numerical stability of membrane potential accumulation calculations.

[0045] This invention introduces a globally differentiable numerical representation operator to simulate the spiking behavior of a SNN in a pre-trained ANN model, based on a globally differentiable adaptive optimization method. The transformation parameters are used as trainable variables, and the optimal transformation parameters are adaptively learned through backpropagation with the task loss as the optimization objective. Furthermore, the model weights can be automatically fine-tuned to make the model more suitable for mapping to an SNN. The method can automatically and collaboratively optimize the model weights and transformation parameters, thereby improving the accuracy and efficiency of the transformation from traditional artificial neural networks (ANNs) to spiking neural networks (SNNs).

[0046] This invention achieves high inference accuracy with a low total number of pulse firing steps by jointly optimizing model weights and activation thresholds of each layer during the training phase. It supports completing computational tasks on inference hardware (such as ARM+FPGA heterogeneous acceleration platforms) in a short time window, improving real-time performance and reducing power consumption.

[0047] This invention reduces reliance on domain expert knowledge and manual parameter tuning through automated and adaptive parameter optimization, enabling more AI developers and researchers to easily use SNN technology and accelerating the process of SNN from research to application.

[0048] This invention also discloses a spiking neural network conversion system based on differentiable adaptive optimization, comprising the following steps for implementing the method: The first module is used to build and pre-train the target ANN model to obtain the pre-trained ANN model. The second module is used to simulate the activation values ​​of each layer in the pre-trained ANN model using a globally differentiable numerical representation operator. The third module is used to jointly fine-tune the original weight parameters in the target ANN model and the trainable activation threshold in the globally differentiable numerical representation operator under the same optimization framework based on the loss function, so as to obtain the optimal weights and the optimal threshold learned by each layer. The fourth module is used to construct an equivalent SNN model based on the optimal weights and the thresholds learned at each layer.

[0049] This invention presents a spiking neural network conversion method based on globally differentiable adaptive optimization. This method optimizes the precise correspondence between continuous activations and spiking sequences, is applicable to various spiking coding schemes, minimizes quantization errors, and can be deeply adapted to the deployment requirements of neuromorphic hardware and edge intelligent computing platforms. It provides an efficient and universal model conversion solution for the neuromorphic computing industry, reduces performance loss and latency during the ANN-SNN conversion process, eliminates the need for manual rule adjustments, reduces development and adaptation costs, and enables SNN models running on neuromorphic hardware to more closely approximate the performance of ANNs. This expands the application boundaries of SNNs in complex tasks such as high-resolution image recognition and natural language processing.

[0050] The present invention also discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described. (See also...) Figure 2 This device requires no manual parameter setting and can be widely applied to common neural network model architectures. It can be implemented via software modules or an ARM+FPGA hardware system. For ease of explanation, this section describes how to implement a differentiable adaptive spiking neural network (SNN) converter on an ARM+FPGA hardware system. The ARM processor runs the software framework, responsible for model loading, scaling factor configuration, and inference result reception and processing. The FPGA acceleration engine implements the core computational flow of the SNN.

[0051] On an ARM+FPGA hardware system, this device includes a transformation parameter loading module, an input impulse event construction module, a convolution module, a pooling module, a fully connected module, an on-chip cache module, and a runtime scheduling and performance monitoring module. Each unit interacts with the ARM processor via the AXI bus. These modules work collaboratively to achieve efficient real-time inference of the transformed spiking neural network model on the programmable hardware platform. Simultaneously, through a multi-level fixed-point mapping structure, event-driven convolution method, and double-buffered weight loading strategy, efficient and low-power inference of the spiking neural network is achieved on the hardware side. The device has a modular structure, good scalability and portability, and is suitable for widespread application in embedded intelligent computing platforms.

[0052] The conversion parameter loading module is used to load the learned thresholds from each layer. The pulse time window T and scaling factor are used to ensure that the pulse firing threshold and pulse time window width used in the hardware inference stage are consistent with those in the training stage, so as to achieve lossless reproduction of the software-side co-optimization results on the hardware side.

[0053] During the model deployment phase, the parameter allocation module performs fixed-point representation on the trained network. Each layer's weights can be represented in 16-bit, 8-bit, or 4-bit fixed-point format, meaning the numerical representation bit width N can be selected as 16-bit, 8-bit, or 4-bit, using a symmetric fixed-point format, with a corresponding pulse time window width T of (2^(N-1)-1). This module employs a "layer-slice" strategy, dividing the parameters into convolutional kernel mapping regions, fully connected mapping regions, and threshold regions. Register area. The scaling factor is packaged into an extended word-width structure (128-bit aligned) in IW format via a wide-bit bus packaging unit, and can be transferred to the FPGA's internal cache in burst mode over the AXI or PCIe interface. After on-chip loading is complete, the parameters are distributed to the on-chip weight cache array, and the control unit establishes an index table from convolutional layers to fully connected layers to enable subsequent fast addressing and inter-layer switching.

[0054] The input pulse event construction module is used to transform input image data into a temporal pulse event stream. This module includes an image linear rearrangement unit and a random firing generator. First, the image linear rearrangement unit maps the input data into a three-dimensional feature tensor [C×H×W] and performs local buffering. Subsequently, the random firing generator uses a pseudo-random pulse generation circuit based on a linear feedback shift register to generate multiple random comparisons for each pixel. If the grayscale value multiplied by the scaling factor is greater than the random number threshold, the number of pulse firings is accumulated at the corresponding pixel index. The pulse index and pulse count are written to the event index table and event intensity table, respectively, and the channel event pointer register is updated synchronously. Through this event compression, the static image is transformed into a sparse event sequence that only records the positions of valid pulses, thereby significantly reducing data transmission and computational redundancy.

[0055] The convolution module performs convolution calculations. Based on an event-driven strategy, it triggers computation only when a valid pulse input is detected. The spatial index of the input event is mapped to the output feature map region via the convolution kernel localization unit. The membrane potential accumulation array within the module performs parallel summation based on the convolution kernel weights and pulse counts. The membrane potential corresponding to each output channel is compared in real-time in the threshold monitoring and reset unit. When the membrane potential reaches a preset threshold... A new output pulse event is generated immediately, and the corresponding membrane potential is cleared to zero. This module uses a bidirectional pipeline strategy of "input mapping expansion and output aggregation" to enable the convolutional layer to achieve continuous data stream processing on the FPGA without waiting for a global synchronization clock.

[0056] The event pooling compression module receives pulse events output from the convolutional module and performs spatial downsampling. Internally, it includes a region event aggregation unit. This unit aggregates and accumulates events within the same region based on the pulse's two-dimensional position index (row and column coordinates). The pooling result is output to the input register array of the next convolutional layer, achieving inter-layer feature compression and noise suppression. The pooling method can be average pooling or max pooling, and the aggregation result is written to the pooling membrane potential buffer. This module employs a spatial offset alignment method, dynamically calculating pixel coordinate offsets to automatically align boundaries when different convolutional layers have inconsistent sizes.

[0057] The fully connected mapping module handles the sparse event stream output from convolutional layers. This module includes a dual-channel weight loading buffer and a fully connected accumulator array. Within each clock cycle, weight loading and multiply-accumulate calculations are performed alternately. While the previous set of weights is being calculated in the fully connected accumulator array, the next set of weights is prefetched via the weight loading buffer, achieving pipelined parallelism between weight loading and calculation. The accumulated membrane potential results enter the fully connected neural activation unit, which determines firing based on a fixed-point threshold register. When the membrane potential exceeds the threshold, the corresponding index is recorded and cleared. The final output events are processed by the impulse decision classification module. This module compares the cumulative firing counts of all output neurons using a maximum impulse detection circuit, selects the category corresponding to the maximum value as the inference result, and writes the result to the classification result register.

[0058] The output module is used to acquire the accumulated membrane potential values, and performs data reconstruction and output channel by channel. The scale recovery coefficient of the output data is calculated based on the scaling factors of the input, output, and weights of each layer. The scale recovery coefficient sf[i] = s_in[i]*s_w[i] / s_out[i], where i is the output channel number, s_in[i] is the scaling factor of the input of the i-th layer, s_w[i] is the scaling factor of the weight of the i-th channel, and s_out[i] is the scaling factor of the output of the i-th layer. The scaling factor is the excitation threshold of each layer. Divide by the pulse time window width T.

[0059] The on-chip cache is used to temporarily store the input, output, and intermediate data of the computation execution modules (transformation parameter loading module, input impulse event construction module, convolution module, pooling module, fully connected module, output module, etc.). The on-chip cache consists of an intermediate result cache array, a weight cache array, a channel pointer register, an event intensity table, and an event index table. The on-chip cache is directly connected to the computation execution modules via a high-bandwidth data channel. Data transmission is achieved through the on-chip cache to ARM, DDR, and other components.

[0060] In the hardware implementation, each layer of parameters is processed and mapped using an independent fixed-point number format. Parameters are written in batches to the FPGA's on-chip cache via the AXI bus during runtime. Inside the accelerator, a pipelined architecture is used to perform inference computation layer by layer. Specifically, a maximum impulse detection circuit compares the cumulative impulse values ​​of all output neurons, selects the category corresponding to the maximum value as the final inference result, writes the decision result to a classification result register, and returns it to the main processor via the AXI bus.

[0061] The operation scheduling module is responsible for model loading, state reset, inter-layer switching and data flow control. It achieves cross-layer shared storage through a unified pulse state register stack, avoiding duplicate allocation and relocation.

[0062] The performance monitoring module periodically collects the maximum pulse count, peak membrane potential, and average firing rate of each layer of neurons. The results are transmitted to the host computer via the host interface for algorithm verification and hardware performance evaluation. This module supports an event statistics interruption strategy, which can automatically trigger data throttling when pulse overload occurs, thereby ensuring system stability and real-time performance.

[0063] After the aforementioned hardware system starts up, the transformation parameter adaptive module performs collaborative optimization of model weights and transformation parameters. Then, the transformation parameter allocation module completes on-chip loading, and the impulse event construction module executes Poisson law encoding. Subsequently, convolutional accumulation, pooling compression, fully connected mapping, and result output are triggered sequentially. Each module operates in an event-triggered pipeline manner, enabling continuous data flow inference.

[0064] exist Figure 2 The device shown performs image classification tests on the MNIST dataset, converting the images in the test set into pulse sequences using Poisson rate encoding and inputting them into the SNN.

[0065] After simulating a certain number of time steps, classification decisions are made based on the pulse count of the output layer. With a pulse time window width of 127 and a weight bit width of 4 bits, the model accuracy is 98.6%, achieving low-loss conversion compared to the original ANN model (which has an accuracy of 99.15%). This demonstrates that the proposed method can improve real-time performance and energy efficiency while maintaining high accuracy.

[0066] This invention also discloses a storage medium storing a computer program that, when executed by a processor, implements the steps of the method. The computer-readable storage medium is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space containing the terminal's operating system. Furthermore, this storage space also contains one or more instructions suitable for loading and execution by a processor; these instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the technical solution of the present invention in any way. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can be modified and replaced in several simple ways, and these modifications and replacements are all within the scope of protection covered by the claims.

Claims

1. A spiking neural network conversion system based on differentiable adaptive optimization, characterized in that, include: Implement a spiking neural network conversion based on differentiable adaptive optimization on an ARM+FPGA hardware system, including a conversion parameter loading module, an input spiking event construction module, a convolution module, a pooling module, a fully connected module, an output module, an on-chip cache module, a runtime scheduling module, and a performance monitoring module. Each unit interacts with the ARM processor through the AXI bus. The transformation parameter loading module is used to load the thresholds obtained from each layer through learning. Pulse time window T and scaling factor; During the model deployment phase, the parameter loading module performs fixed-point representation on the trained network: the weights of each layer can be represented in 16-bit, 8-bit, or 4-bit fixed-point format, i.e., the bit width N of the numerical representation can be selected as 16-bit, 8-bit, or 4-bit, using a symmetric fixed-point format; the parameters are divided into convolutional kernel mapping region, fully connected mapping region, and threshold region. Register area; scaling factor is packaged into an extended word width structure in IW format through a wide-bit bus encapsulation unit, which can be transmitted to the FPGA internal cache in burst mode on the AXI or PCIe interface; After on-chip loading is complete, the parameters are distributed to the on-chip weight cache array, and the control unit establishes an index table from the convolutional layer to the fully connected layer; The input pulse event construction module is used to convert input image data into a time-series pulse event stream. This module includes an image linear rearrangement unit and a random firing generator. Specifically: First, the image linear rearrangement unit maps the input data into a three-dimensional feature tensor [C×H×W] and performs local buffering; then, the random firing generator uses a pseudo-random pulse generation circuit based on a linear feedback shift register to generate multiple random comparisons for each pixel; if the gray value multiplied by the scaling factor is greater than the random number threshold, the number of pulse firings is accumulated at the corresponding pixel index. The pulse index and pulse count are written to the event index table and event intensity table respectively, and the channel event pointer register is updated synchronously. The convolution module performs convolution calculations. Based on an event-driven strategy, it triggers computation only when a valid pulse input is detected. Specifically: the spatial index of the input event is mapped to the output feature map region via the convolution kernel localization unit; the membrane potential accumulation array within the module performs parallel summation based on the convolution kernel weights and pulse counts; the membrane potential corresponding to each output channel is compared in real-time in the threshold monitoring and reset unit, and when the membrane potential reaches a preset threshold... A new output pulse event is generated at that time, and the corresponding membrane potential is cleared to zero. The pooling module receives the pulse events output by the convolution module and performs spatial downsampling. The module has a region event aggregation unit, which aggregates and accumulates events in the same region according to the two-dimensional position index of the pulse. The pooling result is output to the input register array of the next convolution layer to achieve inter-layer feature compression and noise suppression. The fully connected module is used to process the sparse event stream output by the convolutional layer. This module includes a dual-channel weight loading buffer and a fully connected accumulator array. Specifically, in each clock cycle, weight loading and multiply-accumulate calculations are performed alternately. While the first set of weights is being calculated in the fully connected accumulator array, the second set of weights is prefetched through the weight loading buffer, achieving pipelined parallelism of weight loading and calculation. The accumulated membrane potential results enter the fully connected neural activation unit, which judges the firing based on a fixed-point threshold register. When the membrane potential exceeds the threshold, the corresponding index is recorded and cleared. The final output event is processed by impulse decision classification. The maximum impulse detection circuit compares the cumulative firing counts of all output neurons, selects the category corresponding to the maximum value as the inference result, and writes the result to the classification result register. The output module is used to obtain the numerical value of the accumulated membrane potential, and to reconstruct and output the data channel by channel; the scale recovery coefficient of the output data is calculated based on the scaling factor of the input, output and weight of each layer; The on-chip cache module is used to temporarily store the input, output, and intermediate data of the computational transformation parameter loading module, input impulse event construction module, convolution module, pooling module, fully connected module, and output module. The runtime scheduling module is responsible for model loading, state reset, inter-layer switching, and data flow control. The performance monitoring module is used to periodically collect the maximum pulse count, peak membrane potential and average firing rate of each layer of neurons. The results are transmitted to the host computer through the host interface for algorithm verification and hardware performance evaluation. When using a spiking neural network conversion system based on differentiable adaptive optimization for conversion, the steps are as follows: For the image dataset MNIST, construct and pre-train a standard Lenet network as the target ANN model to obtain the pre-trained ANN model; In the pre-trained ANN model, the activation values ​​of each layer are simulated using a globally differentiable numerical representation operator. Specifically, the simulation of the activation values ​​for each layer using a globally differentiable numerical representation operator in the pre-trained ANN model is as follows: For input Output a converted activation value. ; in, For trainable scaling factors; In SNN, it represents the firing threshold of a neuron; T represents the pulse time window width, which is the maximum upper limit of the pulse count allowed by the neuron; The approximate function used in the ANN-SNN conversion process is constructed as follows: Input: x Parameter: f Calculation steps: (1) The constraint parameter f is in the range of 0 to 1: f = max(0, min(1, f)) (2) Calculate the intermediate functions g1(x) ~ g3(x): g1(x) = sin( (x - 1) p ) g2(x) = arccos( f g1(x) ) g3(x) = 1 - (2 g2(x)) / π (3) Calculate the intermediate functions g4(x) ~ g6(x): g4(x) = sin( (x - 0.5) p ) g5(x) = g4(x) / (1-f) g6(x) = arctan( g5(x) ) (4) Calculate roundSNN(x) roundSNN(x) = x - (g3(x) g6(x)) / π in, The parameter used to control the smoothness of rounding operations is between 0 and 1; Based on the loss function, the original weight parameters in the target ANN model and the trainable activation threshold in the globally differentiable numerical representation operator are placed under the same optimization framework for joint fine-tuning to obtain the optimal weights and the optimal thresholds learned by each layer. Construct an equivalent SNN model based on the optimal weights and the thresholds learned at each layer.

2. The spiking neural network conversion system based on differentiable adaptive optimization according to claim 1, characterized in that, The loss function is the mean squared error (MSE) of the last layer output data of the ANN model before and after the transformation.

3. The spiking neural network conversion system based on differentiable adaptive optimization according to claim 1, characterized in that, The optimized framework uses backpropagation.

4. The spiking neural network conversion system based on differentiable adaptive optimization according to claim 1, characterized in that, The synaptic weights in the SNN model are the same as the weights of each layer in the ANN model.

5. The spiking neural network conversion system based on differentiable adaptive optimization according to claim 1, characterized in that, The SNN model is an integral-payout model, the first... The activation threshold of a layer neuron is the learned optimal threshold. .

6. The spiking neural network conversion system based on differentiable adaptive optimization according to claim 5, characterized in that, During SNN model inference, neurons accumulate membrane potential based on input, and when the membrane potential exceeds the optimal threshold... A pulse is delivered and the membrane potential is reset.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it performs the functions of the system according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the functions of the system according to any one of claims 1 to 6.

Images