A first-layer hybrid coding method and device of a pulse neural network based on membrane potential dynamic feedback
By adopting a hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential, and adjusting the encoding strategy dynamically in combination with the membrane potential state, the problem of poor adaptability of static bit width division in the prior art is solved, and efficient and accurate encoding scheduling is achieved, which meets the real-time requirements of edge intelligence and brain-like computing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2026-04-16
- Publication Date
- 2026-06-19
AI Technical Summary
In existing hybrid coding techniques for the first layer of spiking neural networks, static bit width partitioning has poor adaptability, and dynamic coding does not incorporate the internal state of neurons, making it difficult to achieve accurate and efficient coding scheduling in complex texture scenes or situations with sudden changes in local features.
A hybrid coding method based on dynamic feedback of membrane potential in the first layer of a spiking neural network is adopted. By introducing a pipeline mechanism of "coarse calculation-detection-refinement", the intermediate membrane potential is used as a feedback signal to dynamically determine whether to introduce pulse coding and adjust the coding strategy in combination with the membrane potential state.
It achieves the goal of minimizing invalid computation while ensuring high accuracy and not losing high-level features of subthreshold oscillations, and synergistically optimizing inference accuracy and hardware energy efficiency, making it suitable for edge intelligence and neuromorphic computing hardware acceleration scenarios.
Smart Images
Figure CN122242593A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of neuromorphic computing and integrated circuit design, specifically to a hybrid encoding method and apparatus for the first layer of a spiking neural network based on dynamic feedback of membrane potential. Background Technology
[0002] Spiking neural networks (NNs) have shown great potential in edge computing and low-power artificial intelligence applications due to their bio-inspired computational mechanisms and event-driven sparsity. However, in hardware implementation, the encoding method of the first-layer input directly affects the overall system energy efficiency and latency. Traditional encoding schemes are mainly divided into two categories: binary pulse coding, such as rate coding and time coding, which can fully utilize the sparsity advantage of NNs, but require a long time window to achieve sufficient accuracy, resulting in a significant increase in inference latency and making it difficult to meet real-time requirements. Direct encoding, using high-precision floating-point calculations, can retain all the information of the input data. Although it is highly accurate and fast, it loses the sparsity advantage of NNs and performs a large number of invalid high-precision multiplication operations on background regions or non-salient feature regions, resulting in a huge waste of hardware resources.
[0003] To balance accuracy and energy efficiency, existing technologies propose hybrid coding schemes. A typical approach is to use a static bit width partitioning strategy, that is, to directly encode the high bits and pulse encode the low bits. This cannot dynamically adjust the computational granularity according to the real-time characteristics of the input data, resulting in overcomputation on simple samples and insufficient accuracy on complex samples, making it difficult to achieve an optimal trade-off between accuracy and energy efficiency.
[0004] To address the limitation of static bit-level hybrid coding (BLHC) in adapting to input complexity, subsequent techniques have proposed dynamic adaptive bit-width partitioning strategies. For example, the high-low bit boundary is set as a learnable parameter, and through backpropagation optimization, the ratio of high-bit direct coding to low-bit pulse coding is dynamically allocated according to input complexity. This allows simple samples to automatically increase the proportion of low-bit pulse coding to improve sparsity, while complex samples automatically increase the proportion of high-bit direct coding to preserve accuracy. Alternatively, a lightweight bit-width controller can predict the optimal bit-width for each input sample in real time, achieving a dynamic trade-off between accuracy and energy efficiency with almost no increase in computational overhead. While these dynamic coding methods overcome the fixed constraints of static partitioning and improve energy efficiency, they still only adjust the bit-width based on the external features of the input sample, without considering the internal membrane potential state of the neuron for dynamic feedback. Therefore, they cannot adapt to the neuron's subthreshold oscillations, membrane potential accumulation trends, and internal state changes. Consequently, in complex texture scenes or situations with sudden changes in local features, it remains difficult to achieve truly accurate and efficient coding scheduling.
[0005] Therefore, there is an urgent need for a dynamic hybrid coding scheme that can adaptively adjust the coding method according to the internal state of neurons (especially the dynamic feedback of membrane potential). The scheme can dynamically adjust the first-layer coding strategy according to the real-time evolution of membrane potential, so as to ensure that the high-level features of subthreshold oscillations are not missed, while minimizing invalid computation and achieving the optimal balance between accuracy and energy efficiency. Summary of the Invention
[0006] To address the shortcomings of existing hybrid coding techniques for the first layer of spiking neural networks, such as poor adaptability of static bit width partitioning and dynamic coding relying solely on external input features without considering the internal state of neurons, this invention provides a hybrid coding method and apparatus for the first layer of spiking neural networks based on dynamic feedback of membrane potential. It introduces a pipeline mechanism of "coarse calculation-detection-refinement," using the intermediate membrane potential of neurons as a feedback signal to dynamically determine whether to introduce pulse coding to compensate for the neuron's membrane potential. This achieves synergistic optimization of inference accuracy and hardware energy efficiency while ensuring high precision and preventing the loss of high-level features in subthreshold oscillations, thus minimizing computational overhead.
[0007] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:
[0008] In a first aspect, the present invention discloses a hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential, the method comprising the following steps:
[0009] Obtain the raw pixel data of the input image, and split the pixel data into high-bit width pixel value (MSB) and low-bit width pixel value (LSB), wherein the high-bit width part is used as direct encoding input and the low-bit width part is used as source data for pulse coding input.
[0010] In the first time phase, the direct encoded input is multiplied and added with the corresponding weights to update the membrane potential of the first layer neurons, thus obtaining the intermediate state membrane potential. ;
[0011] The intermediate state membrane potential Each with a preset transmission threshold and sensitivity threshold Comparison, among which Less than ;
[0012] Control the encoding input mode for other time stages based on the comparison results:
[0013] If the intermediate state membrane potential Greater than or equal to the emission threshold When the neuron is determined to be in a saturated state, it blocks all weighted inputs in subsequent time steps and directly outputs a pulse signal.
[0014] If the intermediate state membrane potential Less than the emission threshold But greater than or equal to the sensitivity threshold At this point, the neuron is in a critical state and activates the impulse compensation mode;
[0015] If the intermediate state membrane potential Less than the sensitivity threshold The neuron is determined to be in a silent state. At this time, the neuron blocks all weighted inputs in subsequent time steps and enters a dormant state.
[0016] In pulse compensation mode, the binary pulse sequence generated by the conversion of the low-width portion is received, and accumulated with the weights to update the membrane potential; if the updated intermediate state membrane potential... Reaching the launch threshold If the signal is generated, a pulse signal will be output, and the neuron will no longer receive any weighted input.
[0017] Furthermore, the sensitivity threshold The value range is the emission threshold. 80% to 95%.
[0018] Furthermore, the sensitivity threshold It is determined based on the global statistical distribution of the membrane potential of the first layer of neurons after the first time phase ends; wherein, the distribution reflects the overall excitation level of the neurons by the current input image.
[0019] Furthermore, the specific process for determining the sensitivity threshold is as follows:
[0020] After the first time phase ends, the intermediate membrane potentials of all neurons in the first layer are obtained, and a membrane potential histogram is constructed.
[0021] Identify the peak regions in the histogram and extract the mean membrane potential corresponding to the peak regions as the statistical center value;
[0022] Calculate the statistical center value relative to the emission threshold. The offset ratio;
[0023] The sensitivity threshold is linearly adjusted within a preset range of 80% to 95% based on the offset ratio. This is then used as the unified criterion for determining all subsequent pulse compensation time steps.
[0024] Furthermore, the binary pulse sequence adopts a time-coded or rate-coded binary encoding form, and the accumulation operation only performs addition operations and does not perform multiplication operations.
[0025] In a second aspect, the present invention discloses a first-layer hybrid encoding device for a spiking neural network based on membrane potential dynamic feedback. The device includes a dual-modal input interface unit, a reconfigurable computing unit, a first-layer neuron module, and a state monitor.
[0026] The dual-modal input interface unit receives input image data and splits the pixel values of the input image, outputting high-order pixel values and low-order pixel values respectively. The high-order pixel values are used for direct encoding calculation, and the low-order pixel values are used for subsequent pulse coding compensation. Both the high-order and low-order pixel values are connected to a reconfigurable computing unit. The reconfigurable computing unit executes different operation modes in different time periods according to the control state of the current calculation stage. Specifically, in the first time period, a multiplication and accumulation operation is performed on the high-order pixel values and their corresponding weights. In subsequent time periods, an addition and accumulation operation is performed on the binary pulse signal generated from the low-order pixel values and the weights. The first-layer neuron module performs calculations on the results of the reconfigurable computational unit and the historical membrane potentials to update the current intermediate membrane potential of the neuron. The updated intermediate membrane potential is output to the state monitor in real time. The state monitor monitors and compares the intermediate membrane potentials and generates a membrane potential calculation control signal based on the relationship between the intermediate membrane potential and the sensitivity threshold and the emission threshold. The membrane potential calculation control signal is fed back to the first-layer neuron module to control the opening or blocking of the computation path in subsequent time steps, thereby realizing hybrid encoding calculation based on dynamic feedback of membrane potential and single-pulse neuron behavior constraints.
[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0028] First, the pulse neural network first-layer hybrid encoding method and device based on membrane potential dynamic feedback of the present invention accurately screens high-level features of subthreshold oscillations through sensitivity thresholds, and only enables low-level pulse encoding for fine compensation in critical states, ensuring inference accuracy equivalent to the fully direct encoding scheme, avoiding the problem of insufficient accuracy of static hybrid encoding on complex samples, while fully preserving the key amplitude information and subthreshold detail features in the input data.
[0029] Second, the hybrid encoding method and device for the first layer of the spiking neural network based on membrane potential dynamic feedback of the present invention truncates the computation of neurons in the silent state in advance and directly terminates the input of neurons in the saturated state, which greatly reduces the invalid high-precision multiplication and addition operations in the background region and non-significant feature region. While maintaining the inference accuracy equivalent to that of full direct encoding, it can significantly reduce the number of invalid multiplication operations and the number of output pulses, and is especially suitable for edge intelligence and brain-like computing hardware acceleration scenarios.
[0030] Third, the first-layer hybrid encoding method and device of the pulse neural network based on membrane potential dynamic feedback of the present invention adopts high-bit direct encoding to quickly establish the initial membrane potential. Most simple samples can directly complete pulse delivery or calculation truncation in the first stage without waiting for the long window of low-bit pulse encoding. Compared with the pure pulse encoding scheme, the inference delay is greatly reduced, which especially meets the real-time requirements of edge intelligence scenarios.
[0031] Fourth, the hybrid encoding method and device for the first layer of the pulse neural network based on membrane potential dynamic feedback of the present invention adopts a physical channel separation of high-bit direct encoding and low-bit pulse encoding in the encoding architecture. Combined with the reconfigurable computing unit driven by membrane potential state, the hardware implementation only requires a basic statistical module and state machine control, without the need for complex floating-point operations or backpropagation optimization. It can be directly adapted to edge hardware acceleration platforms such as FPGA, ASIC and neuromorphic chips, and has extremely strong engineering application value. Attached Figure Description
[0032] Figure 1 This is an overall flowchart of the hybrid encoding method provided in the embodiments of this patent;
[0033] Figure 2 A schematic diagram of the hardware device provided in this patent embodiment;
[0034] Figure 3 A schematic diagram of the membrane potential state space division and corresponding encoding control logic provided for the embodiments of this patent. Detailed Implementation
[0035] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0036] This invention discloses a hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential. The method includes the following steps:
[0037] Obtain the raw pixel data of the input image, and split the pixel data into high-bit width pixel value (MSB) and low-bit width pixel value (LSB), wherein the high-bit width part is used as direct encoding input and the low-bit width part is used as source data for pulse coding input.
[0038] In the first time phase, the direct encoded input is multiplied and added with the corresponding weights to update the membrane potential of the first layer neurons, thus obtaining the intermediate state membrane potential. ;
[0039] The intermediate state membrane potential Each with a preset transmission threshold and sensitivity threshold Comparison, among which Less than ;
[0040] Control the encoding input mode for other time stages based on the comparison results:
[0041] If the intermediate state membrane potential Greater than or equal to the emission threshold When the neuron is determined to be in a saturated state, it blocks all weighted inputs in subsequent time steps and directly outputs a pulse signal.
[0042] If the intermediate state membrane potential Less than the emission threshold But greater than or equal to the sensitivity threshold At this point, the neuron is in a critical state and activates the impulse compensation mode;
[0043] If the intermediate state membrane potential Less than the sensitivity threshold The neuron is determined to be in a silent state. At this time, the neuron blocks all weighted inputs in subsequent time steps and enters a dormant state.
[0044] In pulse compensation mode, the binary pulse sequence generated by the conversion of the low-width portion is received, and accumulated with the weights to update the membrane potential; if the updated intermediate state membrane potential... Reaching the launch threshold If the signal is generated, a pulse signal will be output, and the neuron will no longer receive any weighted input.
[0045] Figure 1 A detailed flowchart of the overall process of the present invention is shown. Specifically, Figure 1 The present invention provides a hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential, comprising the following steps: First, input image data is received, and the input pixel data is split into a significant bit portion (MSB) and a less significant bit portion (LSB) through a dual-modal input interface unit. The MSB is used for direct encoding calculation, and the LSB is used for subsequent pulse encoding calculation. Then, it is determined whether the current calculation is in the first time stage. When it is in the first time stage, the MSB is multiplied and accumulated with the corresponding weights, and the result is written into the membrane potential register of the first layer neuron of the network to quickly establish an intermediate membrane potential. When it is not in the first time stage, the LSB is encoded into a binary pulse signal and added with the corresponding weights. After the membrane potential is updated at each time step, the membrane potential V of the first layer neuron of the network is monitored in real time. m and compare it with the preset emission threshold V th Comparison, when the membrane potential V m ≥V thWhen the neuron reaches the firing condition, it immediately outputs a pulse signal and blocks all inputs to that neuron in subsequent time steps, preventing it from receiving any weighted inputs, thus ensuring the behavior of the single-pulse neuron; when the membrane potential V m <V th Then, further compare it with the sensitivity threshold V. sens Comparison, where V sens <V th If V m <V sens If V is in a low-activation state, then the neuron is stopped from further input calculation and put into a silent state; sens ≤V m <V th If the neuron continues to receive pulse inputs generated from the low effective bit portion in subsequent time steps, it can gradually increase the membrane potential through addition until the membrane potential reaches the emission threshold and outputs a pulse signal. The binary pulse sequence adopts binary encoding forms such as time encoding or rate encoding. The accumulation operation only performs addition operations and does not perform multiplication operations.
[0046] This invention constructs a two-stage encoding mechanism combining coarse-grained direct computation and fine-grained pulse compensation by splitting the input pixel data into bit widths, using the most significant bits as direct encoding input and the least significant bits as pulse encoding source. In the initial computation phase, the most significant bits are used to perform direct multiplication and addition operations with weights to quickly establish the intermediate membrane potential of the neuron. Subsequently, by monitoring the membrane potential state in real time and comparing it with a preset threshold range, it dynamically determines whether to introduce the pulse sequence corresponding to the least significant bits for residual compensation. When the membrane potential reaches or exceeds the emission threshold, the neuron directly outputs a pulse and terminates subsequent computation. When the membrane potential is far below the sensitivity threshold, it is determined to be a non-significant feature and computation is truncated early. Only when the membrane potential is in the critical range is a low-power pulse compensation process triggered to finely correct the membrane potential. Through this adaptive encoding strategy based on dynamic feedback of membrane potential, this invention significantly reduces the number of invalid multiplication operations and the number of output pulses while maintaining inference accuracy equivalent to fully direct encoding.
[0047] Sensitivity threshold It is determined based on the global statistical distribution of the membrane potential of the first layer of neurons after the first time phase ends; wherein, the distribution reflects the overall excitation level of the neurons by the current input image. Preferably, the specific process for determining the sensitivity threshold is as follows:
[0048] After the first time phase ends, the intermediate state membrane potentials of all neurons in the first layer are obtained, and a membrane potential histogram is constructed.
[0049] Identify the peak regions in the membrane potential histogram and extract the mean membrane potential corresponding to the peak regions as the statistical center value;
[0050] Calculate the statistical center value relative to the emission threshold. The offset ratio;
[0051] The sensitivity threshold is linearly adjusted within a preset range of 80% to 95% based on the offset ratio. This is then used as the unified criterion for all subsequent pulse compensation time steps, and is used to screen characteristic signals that are at the high end of the subthreshold oscillation.
[0052] Figure 2 This invention provides a hybrid encoding device for the first layer of a spiking neural network based on dynamic feedback of membrane potential, specifically including a dual-modal input interface unit, a reconfigurable computing unit, a first-layer neuron module, and a state monitor. The dual-modal input interface unit receives input image data and splits the pixel values of the input image, outputting high-order pixel values and low-order pixel values respectively. The high-order pixel values are used for direct encoding calculation, and the low-order pixel values are used for subsequent pulse encoding compensation. Both the high-order and low-order pixel values are connected to the reconfigurable computing unit. The reconfigurable computing unit executes different operation modes in different time periods according to the control state of the current calculation stage. In the first time period, it performs a multiplication-accumulation operation on the high-order pixel values and their corresponding weights. In subsequent time periods, it performs addition on the binary pulse signal generated from the low-order pixel values and the weights. The first-layer neuron module performs an accumulation operation; it calculates the results of the reconfigurable computational unit and the historical membrane potential to update the current membrane potential of the neuron. The membrane potential state of the first-layer neuron module is output to the state monitor in real time. The state monitor monitors and compares the membrane potential and generates a membrane potential operation control signal based on the relationship between the membrane potential and a preset threshold. The membrane potential operation control signal is fed back to the first-layer neuron module to control the opening or blocking of the computation path in subsequent time steps, thereby realizing hybrid encoding computation based on dynamic feedback of membrane potential and single-pulse neuron behavior constraints.
[0053] Figure 3 This diagram illustrates the partitioning of the membrane potential state space and the corresponding encoding control logic. (Example:) Figure 3 As shown, the vertical axis represents the neuron's membrane potential V. m The horizontal axis represents the state space of the neuron, where the emission threshold V is set. th and sensitivity threshold V sens And V sens <V th Based on the different regions of membrane potential, neuronal states are divided into quiescent, critical, and saturated states. When V m <V sensWhen V is in a silent state, all weighted inputs are blocked; when V sens ≤V m <V th At this time, the neuron is in a critical state, activating the pulse-compensated coding mode; when the membrane potential V m ≥V th Once the neuron enters a saturated state, it directly outputs a pulse signal and terminates subsequent input calculations.
[0054] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0055] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential, characterized in that, The method includes the following steps: Obtain the raw pixel data of the input image, and split the pixel data into high-bit width pixel value (MSB) and low-bit width pixel value (LSB), wherein the high-bit width part is used as direct encoding input and the low-bit width part is used as source data for pulse coding input. In the first time phase, the direct encoded input is multiplied and added with the corresponding weights to update the membrane potential of the first layer neurons, thus obtaining the intermediate state membrane potential. ; The intermediate state membrane potential Each with a preset transmission threshold and sensitivity threshold Comparison, among which Less than ; Control the encoding input mode for other time stages based on the comparison results: If the intermediate state membrane potential Greater than or equal to the emission threshold When the neuron is determined to be in a saturated state, it blocks all weighted inputs in subsequent time steps and directly outputs a pulse signal. If the intermediate state membrane potential Less than the emission threshold But greater than or equal to the sensitivity threshold At this point, the neuron is in a critical state and activates the impulse compensation mode; If the intermediate state membrane potential Less than the sensitivity threshold The neuron is determined to be in a silent state. At this time, the neuron blocks all weighted inputs in subsequent time steps and enters a dormant state. In pulse compensation mode, the binary pulse sequence generated by the conversion of the low-width portion is received, and accumulated with the weights to update the membrane potential; if the updated intermediate state membrane potential... Reaching the launch threshold If the signal is generated, a pulse signal will be output, and the neuron will no longer receive any weighted input.
2. The hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential according to claim 1, characterized in that, The sensitivity threshold The value range is the emission threshold. 80% to 95%.
3. The hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential according to claim 1 or 2, characterized in that, The sensitivity threshold is determined based on the global statistical distribution of neuronal membrane potentials after the first time phase ends; wherein, the distribution reflects the overall excitation level of the neuron by the current input image.
4. The hybrid encoding method for the first layer of a spiking neural network based on dynamic feedback of membrane potential according to claim 1 or 2, characterized in that, The specific process for determining the sensitivity threshold is as follows: After the first time phase ends, the intermediate membrane potentials of all neurons in the first layer are obtained, and a membrane potential histogram is constructed. Identify the peak regions in the histogram and extract the mean membrane potential corresponding to the peak regions as the statistical center value; Calculate the statistical center value relative to the emission threshold. The offset ratio; The sensitivity threshold is linearly adjusted within a preset range of 80% to 95% based on the offset ratio. This is then used as the unified criterion for determining all subsequent pulse compensation time steps.
5. The first-layer hybrid encoding method of a spiking neural network based on dynamic feedback of membrane potential according to claim 1, characterized in that, The binary pulse sequence adopts a time-coded or rate-coded binary encoding form, and the accumulation operation only performs addition operations and does not perform multiplication operations.
6. A hybrid encoding device for the first layer of a spiking neural network based on dynamic feedback of membrane potential, characterized in that, The device includes a dual-modal input interface unit, a reconfigurable computing unit, a first-layer neuron module, and a state monitor. The dual-modal input interface unit receives input image data and splits the pixel values of the input image, outputting high-order pixel values and low-order pixel values respectively. The high-order pixel values are used for direct encoding calculation, and the low-order pixel values are used for subsequent pulse coding compensation. Both the high-order and low-order pixel values are connected to a reconfigurable computing unit. The reconfigurable computing unit executes different operation modes in different time periods according to the control state of the current calculation stage. Specifically, in the first time period, a multiplication and accumulation operation is performed on the high-order pixel values and their corresponding weights. In subsequent time periods, an addition and accumulation operation is performed on the binary pulse signal generated from the low-order pixel values and the weights. The first-layer neuron module performs calculations on the results of the reconfigurable computational unit and the historical membrane potentials to update the current intermediate membrane potential of the neuron. The updated intermediate membrane potential is output to the state monitor in real time. The state monitor monitors and compares the intermediate membrane potentials and generates a membrane potential calculation control signal based on the relationship between the intermediate membrane potential and the sensitivity threshold and the emission threshold. The membrane potential calculation control signal is fed back to the first-layer neuron module to control the opening or blocking of the computation path in subsequent time steps, thereby realizing hybrid encoding calculation based on dynamic feedback of membrane potential and single-pulse neuron behavior constraints.