FHN neuron based on floating-point number approximation

Abstract

The invention relates to the field of neural network hardware implementation, in particular to an FHN neuron based on floating-point number approximation. According to the method, fixed-point data of square and cube calculation are converted into floating-point numbers through a floating-point approximation method, addition calculation is adopted for an index part, piecewise linear fitting approximation is adopted for multiplication calculation for a mantissa part, and dynamic characteristics consistent with an original model are effectively achieved; in digital implementation, the adder and the shifter are used for replacing a multiplier, so that the resource overhead is remarkably reduced, and low-cost and high-precision FHN neuron simulation is realized. Compared with the prior art, the method has the advantages that the performance cost ratio and error control are remarkably improved, low cost and high precision are taken into account when the FHN neurons are simulated and the FHN neural network topology is calculated, the biological characteristics of the realized FPA-FHN neurons are superior to those of LIF neurons, more complex neurodynamics behaviors can be shown under the condition that the deployment scale is reduced and limited, and the method is suitable for popularization and application. And a better choice is provided for large-scale neural network deployment.

Description

Technical Field

[0001] The present invention relates to the field of neural network hardware implementation, in particular to a FHN neuron based on floating point number approximation. Background Art

[0002] The human brain consists of about 10 billion neurons with powerful learning and generalization capabilities. It consumes relatively little power but can perform complex tasks. Exploring neuronal models and their interactions is an important research direction in neuroscience. Various simplified models are used for neuronal coupling and network composition, such as the Leaky, Integrated-Fire (LIF) model, the FitzHugh-Nagumo (FHN) model, and the Morris-Lecar model. Among them, the FHN model has high biological plausibility, and its two-dimensional nonlinear expression shows rich observable characteristics of biological neurons, such as phase spikes, tonic bursts, and subthreshold oscillations.

[0003] Spiking neural network (SNN) is a large-scale biologically interpretable neuron coupling framework. Some studies have used neuromorphic hardware to implement SNN, but due to the limitation of large-scale hardware networks, the LIF model is mostly used. Although the model is simple in calculation, it lacks neural dynamic characteristics. The FHN model is slightly more complex in structure, but it can better simulate the behavior of biological neurons. The high biological interpretability of its coupled network helps to generate complex dynamic behaviors, which is of great significance for understanding the brain and related diseases.

[0004] Approximate computing is an emerging computing paradigm that can balance power, performance, and area in circuits. When implementing FHN neural networks, a variety of approximate computing methods have been proposed, including approximate fixed-point and floating-point adders and multipliers, and approximate specific formulas using lookup tables (LUTs), polynomials, and coordinate rotation digital computation (CORDIC) methods. However, previous work on implementing the FHN model has problems such as lack of convincing comparison results with the FHN implemented by basic approximate units and failure to implement topological network applications based on FHN neurons. Summary of the invention

[0005] In view of the above problems or deficiencies, the present invention aims to provide a FHN neuron based on floating-point approximation to solve the problems of high cost, low precision or lack of practicality of the FHN model in the prior art. The fixed-point data of square and cube calculations are first converted into floating-point numbers, the exponential part is calculated by addition, and the mantissa part is calculated by piecewise linear fitting approximation. The floating-point approximation method effectively realizes dynamic characteristics consistent with the original model. In addition, the floating-point approximation method can replace the multiplier with an adder and a shifter in digital implementation, which takes into account both low cost and high precision when simulating FHN neurons and calculating the FHN neural network topology.

[0006] A FHN neuron based on floating point approximation, its structural block diagram is as follows Figure 3 As shown: It is a six-level register structure, and the FPA-FHN neuron is implemented through the Fix2Float module, FPA Square module, FPA Cube module, Sync Module module and Stimu Gen module.

[0007] The Fix2Float module implements fixed-point to floating-point conversion, converting the 1-4-11 fixed-point format to the 1-4-11 floating-point format, and is implemented through a simple decoder and shifter, reducing cost overhead.

[0008] The FPA Square module and FPA Cube module are based on the formula:

[0009] v 2 ≈FPA(v 2 )=2 2E-33 *(1792+3MAN)

[0010]

[0011] Replace constant multiplication with shift and addition, and limit oversized or undersized data to prevent overflow.

[0012] Finally, the floating-point approximate FPA calculations of the first four cycles are implemented through the Fix2Float module, the FPA Square module, and the FPA Cube module: FPA square and cube operations are performed on the membrane potential v(t), the fixed-point number v(t-1) input is converted to a floating-point number through the Fixed2Float module, and then approximated by the FPA cube and square modules to output a fixed-point number.

[0013] The Sync Module controls registers to ensure that the clock signals of the FPA calculation and the FHN calculation are synchronized.

[0014] The Stimu Gen module converts the external 1-bit spike signal into a fixed-point current I(t) and sends it to the fifth cycle for calculation, ensuring that the spike signal is correctly integrated into the FHN calculation; and completes the calculation in the sixth cycle.

[0015] Finally, the simulation of the entire FPA-FHN neuron in one time step is completed through six clock cycles.

[0016] Further, the fifth and sixth cycles are based on:

[0017]

[0018] Perform FHN calculations, avoid constant multiplication through shift and addition and subtraction operations, and use a tree structure to organize calculations to improve efficiency.

[0019] Furthermore, the Fix2Float module implements fixed-point to floating-point conversion specifically as follows:

[0020] Convert fixed-point data to IEEE 754 floating-point format to eliminate the need for square and cube operations in FHN neuron computations;

[0021] v=(-1) s *2 E-bias *(1+M)

[0022] Where S is the sign, E is the exponent, and M is the mantissa. M is a decimal number in the range of 0 to 1, which determines the precision of the floating-point number. E and bias are shifted relative to (M+1) to determine the representation range of the floating-point number. S, E, and M are calculated using the following three formulas:

[0023]

[0024] M=|v| / 2 E-bias -1

[0025] For square and cube calculations, the exponent part is added in floating-point calculations, and the mantissa part is multiplied using piecewise linear approximation;

[0026] v 2 =2 2E-2bias *(1+2M+M 2 )

[0027] v 3 =(-1) S *2 3E-3bias *(1+3M+3M 2 +M 3 )

[0028] Substitute the approximate formulas for M 2 and M 3 .

[0029] Furthermore, the M 2 Approximate the fit solely with a straight line:

[0030] M 2 ≈M-0.125,

[0031] M 3 Approximation is suitable using polylines:

[0032]

[0033] To strike a balance between approximation error and hardware implementation complexity, we get v 2 and v 3 The floating point approximation FPA formula

[0034] v 2 ≈FPA(v 2 )=2 2E-2bias *(0.875+3M)

[0035]

[0036] v 2 and v 3 The original function and FPA function of are smooth curves and broken lines respectively; the value range of v is -2 to 2; the floating point mantissa of (M+1) is always ∈[1,2), and the exponent part E is Change is needed.

[0037] Furthermore, the FHN neuron based on floating point number approximation is implemented by a field programmable gate array FPGA.

[0038] Furthermore, an eight-node circular diffusion neural network based on the FPA-FHN neuron model was constructed as the minimum unit; then, a mesh network cluster was constructed based on the unit, and the network cluster diffused in eight directions; the central unit in the structure drove the operation of the surrounding eight units through connections, and its motion equation was:

[0039]

[0040] Where i represents the i-th neuron N i ; The parameter connecting these 8 neurons is the current I i , if the membrane potential V i Not less than the threshold voltage V th , then the next neuron N i+1 Send out current; when the i-th neuron of the eight surrounding units sends out current I i When it is activated, it simultaneously transmits the initial excitation to the eight surrounding units, thus starting their operation.

[0041] Furthermore, an eight-node circular diffusion neural network based on the FPA-FHN neuron model was constructed as the minimum unit; subsequently, a ring network unit structure was formed based on this unit consisting of eight interconnected neurons; the eight neurons with excitation modules followed the directed diffusion rule described by their motion equations and were connected in a ring, with the first and last neurons connected together.

[0042] The input of the first FPA-FHN neuron model in the ring network is the external stimulus I inOr the output current I8(t) of the eighth module; a multiplexer is used at the input.

[0043] In summary, the present invention uses a floating-point approximation method to replace multipliers with adders and shifters in digital implementation, which significantly reduces resource overhead and realizes low-cost and high-precision FHN neuron simulation, which has significant improvements in performance-overhead ratio and error control compared to the prior art. The implemented FPA-FHN neurons are superior to LIF neurons in biological characteristics, and can exhibit more complex neural dynamic behaviors with limited reduction in deployment scale, providing a better choice for large-scale neural network deployment. The implementation results of the constructed FHN neural network hardware structure at different scales show its stability and feasibility. The linear relationship between network scale and resource overhead helps to reasonably plan and expand the scale of neural networks in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 v is in float16 format 2 Approximate graph;

[0045] Figure 2 v is in float16 format 3 Approximate graph;

[0046] Figure 3 It is the structure diagram of FPA-FHN, where the dotted lines represent registers;

[0047] Figure 4 This is a schematic diagram of a node circular diffusion FHN neural network in Example 8;

[0048] Figure 5 A diagram of a mesh diffusion network cluster and its internal structure in an embodiment;

[0049] Figure 6 : (a) Delay heat map of activation of eight diffusion units; (b) simulation result of network unit 1; (c) simulation result of network unit 9 in the embodiment. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0051] A FHN neuron based on floating point approximation is a six-level register structure. The FPA-FHN neuron is implemented through the Fix2Float module, FPASquare module, FPA Cube module, Sync Module module and Stimu Gen module. The structure of the FPA-FHN neuron is as follows Figure 3 shown.

[0052] This embodiment is based on the background of field programmable gate array FPGA. The FHN hardware structure based on the FPA algorithm adopts a six-level register structure to implement the FPA-FHN neuron hardware, so that the simulation of the neuron in one time step is completed through six clock cycles.

[0053] The Fix2Float module implements fixed-point to floating-point conversion, converting the 1-4-11 fixed-point format to the 1-4-11 floating-point format, which is implemented through a simple decoder and shifter. The FPA Square module and the FPA Cube module use shifts and additions to replace constant multiplication according to the formula, and limit the oversized or undersized data to prevent overflow.

[0054] Finally, the floating-point approximate FPA calculations of the first four cycles are implemented through the Fix2Float module, the FPA Square module, and the FPA Cube module: FPA square and cube operations are performed on the membrane potential v(t), the fixed-point number v(t-1) input is converted to a floating-point number through the Fixed2Float module, and then approximated by the FPA cube and square modules to output a fixed-point number.

[0055] The Sync Module controls the registers to ensure that the clock signals for FPA calculation and FHN calculation are synchronized. The StimuGen module converts the external 1-bit spike signal into a fixed-point current I(t) and sends it to the fifth cycle for calculation, ensuring that the spike signal is correctly integrated into the FHN calculation; and the calculation is completed in the sixth cycle.

[0056] Among them, the fifth and sixth cycles are based on:

[0057]

[0058] Perform FHN calculations, avoid constant multiplication through shift and addition and subtraction operations, and use a tree structure to organize the calculations.

[0059] The architecture of the present invention eliminates the need for multipliers in the hardware structure, making it different from the original algorithm, effectively reducing resource consumption and improving energy efficiency.

[0060] v 2 and v 3 The original function and FPA function are Figure 1 and Figure 2 The smooth curve and broken line shown. The value range of v is -2 to 2, covering most cases. The floating point mantissa of (M+1) is always ∈ [1,2), resulting in the exponent part E in The floating point number's mantissa changes with itself. Figure 1 and Figure 2 The approximation of is a broken line. Similarly, v 2 and v3 The absolute error AE between the original value and the approximate value is also as Figure 1 and Figure 2 As shown in Figure 2, the average value of the absolute error is around 0.05, indicating that the floating-point approximation method is effective.

[0061] This embodiment constructs an eight-node circular diffusion neural network based on the FPA-FHN neuron model as the minimum unit. Figure 4 (a). Then, a mesh network cluster is constructed based on this unit, as shown in Figure 4 As shown in (b), the network cluster spreads in eight directions. The dark-colored unit in the structure can drive the operation of the eight surrounding light-colored units through connections. Its equation is

[0062]

[0063] Where i represents the i-th neuron N i The parameter connecting these 8 neurons is the current I i , if the membrane potential V i Not less than the threshold voltage V th , then the next neuron N i+1 When the i-th neuron in the dark octagon emits a current I i , it simultaneously delivers the initial stimulus to the light-colored octagon, thereby starting its operation.

[0064] This embodiment is based on the hardware architecture of nine coupled circular networks in a mesh topology. Figure 5 shown. Figure 5 The detailed structure of a ring network unit consisting of eight interconnected neurons is also given. The eight neurons with excitation modules follow the directed diffusion rule described by their equations of motion and are connected in a ring, with the first and last neurons connected together. The input of the first FPA FHN module in the ring network is either an external stimulus I in , or the output current I8(t) of the eighth module. The reason for using a multiplexer at the input is that the network cannot generate diffusion independently and requires an external initialization stimulus to activate the system. Figure 5 As shown in the dashed box on the left, the simulation of the i-th neuron of the black unit is guided to the first neuron of the light-colored unit, thus realizing the simplest form of diffusion coupling between units. In this architecture, the membrane potential of each neuron needs to be recorded for analysis and verification.

[0065] Figure 5 The mesh network cluster shown is implemented on an FPGA, and the delay distribution and membrane potential error are analyzed. The results are shown in Figure 6As shown in the figure, the network unit 1 has the lowest delay, and the unit 8 has the highest delay. Although the network error (RMSE is 0.0861) is larger than that of a single neuron, it is smaller than the error of a single neuron in the prior art (accounting for 60% of it), which proves the stability and feasibility of the FHN neural network based on the FPA algorithm.

[0066] Figure 6 In the embodiment: (a) the delay heat map of the activation of eight diffusion units, where the darker the color, the higher the delay, and the lighter the color, the lower the delay; (b) the simulation result of network unit 1, showing lower delay; (c) the simulation result of network unit 9, showing higher delay.

[0067] Comparison of hardware implementation results:

[0068] 1) Comparison with other FHN implementations As shown in Table 1, the FPA-FHN neuron of the present invention adopts a combination of fixed-point and floating-point calculations and is implemented on Xilinx's Zynq development platform. Compared with the most advanced, the frequency is increased by 1.22 times, the performance overhead ratio is increased by 1.09 times, and the nRMSE is reduced to 3.5% of the prior art. Table 1:

[0069]

[0070] 2). Compared with approximate calculation, the FPA module is compared with the exact and approximate multipliers. Under the same clock constraints and without DSP utilization, the FPA-FHN method can reduce the overhead by up to 34% compared with the exact calculation and the nRMSE is in the same range. Compared with the approximate calculation methods such as the BFloat16 approximation algorithm, although the overhead increases by 2.8%, the nRMSE is reduced by 20%, which is suitable for large-scale FHN neural network implementation. Table 2 Comparison of hardware implementation resources and performance of different multipliers:

[0071]

[0072] 3). Compared with LIF neurons, FPA-FHN only increases LUTs by 10% in terms of resources, and FFs increase more due to multi-level registers, but the overall deployment scale is only reduced by 9%, while it can achieve more biological characteristics, such as phase spikes, bursts, and oscillations. Table 3 Comparison of FPA-FHN and LIF resources and biological characteristics:

[0073]

[0074]

[0075] The FPGA simulation results of the above embodiments show that the embodiment given by the present invention achieves a 1.09-fold performance overhead improvement and a 0.035-fold reduction in error rate compared to the most advanced SOTA. Compared with the Bfloat16 calculation implementation based on the approximate multiplier, the method of the present invention has an increased overhead of 2.8%, but a 20% reduction in error rate. Compared with LIF neurons, the model obtains additional biological characteristics such as bursts and oscillations while reducing the deployment scale by only 9%. In addition, circular networks and large network clusters were implemented on FPGA, demonstrating the linear relationship between network scale and area overhead. The coupled circular network topology of 72 neurons reduces the error rate by 60% compared to a single neuron of SOTA.

Claims

1. A FHN neuron based on floating point approximation, characterized in that: It is a six-level register structure, and the FPA-FHN neuron is implemented through the Fix2Float module, FPA Square module, FPA Cube module, Sync Module module and Stimu Gen module; The Fix2Float module implements fixed-point to floating-point conversion, converting the 1-4-11 fixed-point format to the 1-4-11 floating-point format, which is achieved through a simple decoder and shifter; The FPA Square module and FPA Cube module are based on the formula: v 2 ≈FPA(v 2 )=2 2E-33 *(1792+3MAN) Replace constant multiplication with shift and addition, and limit the oversized or undersized data to prevent overflow; Finally, the floating-point approximate FPA calculations of the first four cycles are implemented through the Fix2Float module, the FPA Square module, and the FPA Cube module: the FPA square and cube operations of the membrane potential v(t) are performed, the fixed-point number v(t-1) input is converted to a floating-point number through the Fixed2Float module, and then approximated by the FPA cube and square modules to output a fixed-point number; The Sync Module module controls the register to ensure the synchronization of the clock signals of the FPA calculation and the FHN calculation; The Stimu Gen module converts the external 1-bit spike signal into a fixed-point current I(t) and sends it to the fifth cycle for calculation, ensuring that the spike signal is correctly integrated into the FHN calculation; and completes the calculation in the sixth cycle; Finally, the simulation of the entire FPA-FHN neuron in one time step is completed through six clock cycles.

2. The FHN neuron based on floating point approximation as claimed in claim 1, characterized in that: The fifth and sixth cycles are based on: Perform FHN calculations, avoid constant multiplication through shift and addition and subtraction operations, and use a tree structure to organize calculations to improve efficiency.

3. The FHN neuron based on floating point approximation as claimed in claim 1, characterized in that: The Fix2Float module implements fixed-point to floating-point conversion specifically as follows: Convert fixed-point data to IEEE 754 floating-point format to eliminate the need for square and cube operations in FHN neuron computations; v=(-1) S *2 E-bias *(1+M) Where S is the sign, E is the exponent, and M is the mantissa; M is a decimal number in the range of 0 to 1, which determines the precision of the floating point number; E and bias are shifted relative to (M+1) to determine the representation range of the floating point number; S, E and M are calculated by the following three formulas: M=|v| / 2 E-bias -1 For square and cube calculations, the exponent part is added in floating-point calculations, and the mantissa part is multiplied using piecewise linear approximation; v 2 =2 2E-2bias *(1+2M+M 2 ) <h2 style=";text-align:left;direction:ltr">v<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> (-1)<h2 style=";text-align:left;direction:ltr"> S <h2 style=";text-align:left;direction:ltr"> *2<h2 style=";text-align:left;direction:ltr"> 3E-3bias <h2 style=";text-align:left;direction:ltr"> *(1+3M+3M<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +M<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> ) Substitute the approximate formula for M 2 and M 3 .

4. The FHN neuron based on floating point approximation as claimed in claim 3, characterized in that: The M 2 Approximate the fit solely with a straight line: M 2 ≈M-0.125, M 3 Approximation is suitable using polylines: To strike a balance between approximation error and hardware implementation complexity, we get v 2 and v 3 The floating point approximation FPA formula v 2 ≈FPA(v 2 )=2 2E-2bias *(0.875+3M) v 2 and v 3 The original function and FPA function of are smooth curves and broken lines respectively; the value range of v is -2 to 2; the floating point mantissa of (M+1) is always ∈[1,2), and the exponent part E is Change is needed.

5. The FHN neuron based on floating point approximation as claimed in claim 1, characterized in that: It is implemented through field programmable gate array FPGA.

6. The FHN neuron based on floating point approximation as claimed in claim 1, characterized in that: An eight-node circular diffusion neural network based on the FPA-FHN neuron model was constructed as the minimum unit. Subsequently, a mesh network cluster was constructed based on the unit, which diffused in eight directions. The central unit in the structure drove the operation of the eight surrounding units through connections, and its motion equation was: Where i represents the i-th neuron N i ; The parameter connecting these 8 neurons is the current I i , if the membrane potential V i Not less than the threshold voltage V th , then the next neuron N i+1 Send out current; when the i-th neuron of the eight surrounding units sends out current I i When it is activated, it simultaneously transmits the initial excitation to the eight surrounding units, thus starting their operation.

7. The FHN neuron based on floating point approximation as claimed in claim 1, characterized in that: An eight-node circular diffusion neural network based on the FPA-FHN neuron model was constructed as the minimum unit; then, a ring network unit structure was formed based on the unit consisting of eight interconnected neurons; the eight neurons with excitation modules followed the directional diffusion rule described by their motion equations and were connected in a ring, with the first and last neurons connected together; The input of the first FPA-FHN neuron model in the ring network is the external stimulus I in Or the output current I8(t) of the eighth module; a multiplexer is used at the input.

Images