Lightweight on-chip learning FPGA hardware architecture and its design method

Abstract

The present invention discloses a lightweight on-chip learning FPGA hardware architecture and a design method thereof. Based on a pulse neural network structure, a lightweight pulse neural network is constructed by combining compressed sensing rules and a Tempotron algorithm. The compressed sensing layer directly emits pulses, and a membrane potential and Trace collaborative reset mechanism are added to the traditional Tempotron algorithm. The event-driven method is used to improve the calculation efficiency and facilitate hardware implementation. The present invention improves the calculation speed and reduces the consumption of hardware resources.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and brain-like intelligent chips, and specifically relates to a lightweight on-chip learning FPGA hardware architecture and a design method thereof. Background Art

[0002] Artificial neural networks (ANNs) were the first to model the computing power of the human brain. Over the years, ANNs have achieved remarkable results. However, recent studies on the human brain have shown that information is transmitted in the form of action potentials in brain neurons. Based on this, Wolfgang Maass et al. proposed a spiking neural network that is more in line with the actual biological neuron model. Compared with artificial neural networks, the input pulses of spiking neural networks are sparse in time and space. Neurons only need to calculate when the input pulses arrive, and neurons only emit pulses when a certain threshold is reached. Therefore, compared with traditional artificial neural networks, spiking neural networks (SNNs) have lower energy consumption. In addition, the feedforward spiking neural networks in recent years do not require global backpropagation like artificial neural networks, which is conducive to on-chip learning. The von Neumann architecture, that is, the traditional computer system, has limitations for the implementation of SNNs. That is, the CPU can only execute programs in a serial manner, while SNNs involve a large amount of parallel computing, resulting in low SNN computing efficiency. Although GPUs (graphics processing units) can execute programs in parallel, the sparse characteristics of SNNs cannot be implemented on them. Therefore, people have proposed to implement SNN on FPGA (field programmable gate array) and ASIC (application-specific integrated circuit). For example, the TrueNorth processor with crossbar switch and support for arbitrary large-scale SNN topology and the SpiNNaker processor using ARM core to implement programmable SNN model. However, they are mainly aimed at research in the field of neuroscience, and the chip area consumption is large, which is not suitable for embedded edge or mobile computing scenarios.

[0003] Therefore, it is necessary to develop a new lightweight on-chip learning FPGA hardware architecture and its design method. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight on-chip learning FPGA hardware architecture and a design method thereof, which can improve the computing speed and reduce the consumption of hardware resources.

[0005] In a first aspect, a method for designing a lightweight on-chip learning FPGA hardware architecture according to the present invention comprises the following steps:

[0006] Step 1: Design the Compressive-Tempotron two-layer pulse neural network architecture and software fixed-point simulation, including:

[0007] Step 1-1. Determine how to encode the input image:

[0008] The input layer uses multi-pulse time coding to convert the pixels of the image into a pulse sequence, where the multi-pulse time coding uses a leaky cumulative firing neuron model:

[0009] V m (t) = V m (t-1)+β×x (1)

[0010] Where: t is the current time step, t-1 is the previous time step, V m is the neuron membrane potential, x is the pixel value of a single pixel, and β is the growth rate of the pixel growing linearly over time;

[0011] When the membrane potential accumulates to a pre-set threshold, the neuron fires a pulse, immediately returns to 0, and then increases linearly over time again;

[0012] Step 1-2. Determine the pulse feature extraction rules and improve the feature extraction layer:

[0013] The compressed sensing layer extracts pulse features with spatiotemporal information from the pulses sent from the input layer. In one image, two positive polarity patches and two negative polarity patches are randomly selected. Each patch is determined by three attributes: position, size, and polarity, where the position and size are randomly selected. Both the compressed sensing layer and the Tempotron layer are event-driven.

[0014] The compressed sensing layer adopts a biologically plausible leaky cumulative firing neuron model, whose membrane potential is described in an event-driven form as:

[0015]

[0016] Where: τ m is the first time decay constant, t p is the time when the presynaptic pulse arrives at the pth patch, t prev is the time before the current time when the presynaptic pulse arrives at any synapse, V is the membrane potential of the postsynaptic neuron, and the polarity value of Patch is represented by +1 / -1, that is, +1 indicates that the polarity of the patch is positive, and -1 indicates that the polarity of the patch is negative. p is the product of the area of ​​the pth patch and its polarity;

[0017] Step 1-3. Determine the learning layer algorithm and improve the Tempotron layer;

[0018] The traditional Tempotron learning rules are improved as follows:

[0019] The event-driven description of the postsynaptic neuron membrane potential is:

[0020]

[0021] Where: t i is the time it takes for the presynaptic pulse to arrive at the i-th synapse, ω i is the weight of the i-th synapse, V is the membrane potential of the postsynaptic neuron; when the neuron membrane potential exceeds the preset threshold, it is reset to α times the original value, 0≤α<1, and then the neuron membrane potential continues to accumulate-emit-reset;

[0022] Use single-core PSP for weight update, where single-core PSP is:

[0023]

[0024] Where: τ is the second time decay constant, t max is the time when the postsynaptic neuron membrane potential reaches its maximum value within the time window, t i is the time when the presynaptic pulse arrives at the i-th synapse;

[0025] When the weight is updated, t i Should be less than or equal to t max , the weight update is described as:

[0026]

[0027] Where: λ is the learning rate;

[0028] Steps 1-4, software fixed-point simulation: Use fixed-point software simulation to evaluate the accuracy of the Compressive-Tempotron neural network on the MNIST dataset.

[0029] Step 2: Design the hardware architecture: deploy the Compressive-Tempotron neural network in the FPGA.

[0030] Optionally, the accumulated term in formula (5) can be rewritten as an event-driven synaptic trace update method, that is, Replace it with formula (6), specifically:

[0031]

[0032] Where: τ is the second time decay constant, t max is the time when the neuron membrane potential reaches the maximum value within the time window, t prev is the time before the current time when the presynaptic pulse arrives at any synapse, Q i is the synaptic trace of the i-th synapse;

[0033] When a neuron receives a presynaptic pulse, its membrane potential is updated according to equation (3), and the synaptic trace (Q i (t)) multiplied by exp(-Δt i / τ), and then increase by 1; if the updated membrane potential is greater than the maximum membrane potential, the maximum membrane potential is updated with the current membrane potential of the neuron, and the current synaptic trace is backed up; when the postsynaptic neuron emits a pulse, the synaptic trace and the postsynaptic neuron membrane potential are reset to α times the original value; after processing all events of a graph, -λQ i (t max ) or λQ i (t max ) is added to the synaptic weight to complete equation (5).

[0034] In a second aspect, a lightweight on-chip learning FPGA hardware architecture described in the present invention includes a global controller, a compressed sensing layer and a Tempotron layer;

[0035] The global controller is connected to the compressed sensing layer and the Tempotron layer respectively, and the global controller is used to control the compressed sensing layer and the Tempotron layer;

[0036] The compressed sensing layer includes m1 first Tile units, wherein each first Tile unit processes k1 first neurons in a time multiplexing manner; wherein 1≤m1≤64, 1≤k1≤8; wherein the first Tile unit includes a first local controller, and a first membrane potential memory, a first emission management module, a first membrane potential update module, a Patch memory, a first index lookup table memory, and a first previous event memory respectively connected to the first local controller, the first membrane potential memory is also respectively connected to the first emission management module and the first membrane potential update module, the first emission management module is connected to the first membrane potential update module, and the first membrane potential update module is respectively connected to the Patch memory, the first index lookup table memory, and the first previous event memory;

[0037] The Tempotron layer includes m2 second Tile units, wherein each second Tile unit processes k2 second neurons in a time multiplexing manner; wherein 1≤m2≤64, 1≤k2≤4; wherein the second Tile unit includes a second local controller, and a learning module, a backup Trace memory, a maximum membrane potential management module, a second emission management module, a weight memory, a second membrane potential memory, a Trace memory, a Trace update module, a second membrane potential update module, a maximum membrane potential memory, a second previous event memory, and a second index lookup table respectively connected to the second local controller memory; the learning module is also connected to the weight memory, the second emission management module and the backup Trace memory respectively; the second membrane potential update module is also connected to the weight memory, the second membrane potential memory, the Trace update module and the second index lookup table memory respectively; the second membrane potential memory is also connected to the second emission management module; the Trace update module is also connected to the second index lookup table memory, the second previous event memory and the Trace memory respectively; the Trace memory is also connected to the second emission management module; the maximum membrane potential management module is also connected to the maximum membrane potential memory, the second emission management module and the backup Trace memory respectively.

[0038] Optionally, the second membrane potential updating module includes a first multiplier and a first adder, and the first multiplier and the first adder are connected;

[0039] When a pulse arrives, the membrane potential is updated, and t is first read from the second previous event memory. prev , subtract t from the current time t prev A time difference Δt is obtained, and an exponential decay coefficient is obtained by looking up an exponential lookup table stored in a second exponential lookup table memory through Δt. The exponential decay coefficient found and the membrane potential of the current neuron read from the second membrane potential memory are used as inputs of a first multiplier to obtain the output of the first multiplier, i.e., the attenuated membrane potential. The attenuated membrane potential and the weight of the synapse at which the pulse of the current neuron arrives read from the weight memory are used as inputs of a first adder to obtain the output of the first adder, i.e., the updated membrane potential V(t). The updated membrane potential is then written back to the second membrane potential memory.

[0040] Optionally, the learning module uses an 8-bit random number generation circuit;

[0041] The 8-bit random number generation circuit includes a first register to an eighth register, a first selector to an eighth selector, and a first XOR gate to a fourth XOR gate;

[0042] The first input end of the first selector is connected to the initialization seed, the second input end of the first selector is connected to the output end of the first register, the control end of the first selector is connected to whether to download the initialization seed, the output of the first selector is connected to the input end of the eighth register, the output of the eighth register and the output of the first register are connected to the second input end of the second selector through the first XOR gate, the first input end of the second selector is connected to the initialization seed, the control end of the second selector is connected to whether to download the initialization seed (the reset signal from the Tile), the output end of the second selector is connected to the input end of the seventh register, the output end of the seventh register is connected to the second input end of the third selector, the first input end of the third selector is connected to the initialization seed, the control end of the third selector is connected to whether to download the initialization seed, the output end of the third selector is connected to the input end of the sixth register, the output end of the sixth register and the input end of the first register are connected to the second input end of the fourth selector through the second XOR gate, the first input end of the fourth selector is connected to the initialization seed, the control end of the fourth selector is connected to whether to download the initialization seed, the output end of the fourth selector is connected to the input end of the fifth register, and the output end of the fifth register is connected to the The output end of the first register is connected to the second input end of the fifth selector through the third XOR gate, the first input end of the fifth selector is connected to the initialization seed, the control end of the fifth selector is connected to whether to download the initialization seed, the output end of the fifth selector is connected to the input end of the fourth register, the output end of the fourth register and the output end of the first register are connected to the second input end of the sixth selector through the fourth XOR gate, the first input end of the sixth selector is connected to the initialization seed, the control end of the sixth selector is connected to whether to download the initialization seed, the output end of the sixth selector is connected to the third register, the output end of the third register is connected to the second input end of the seventh selector, the first input end is connected to the initialization seed, the control end of the seventh selector is connected to whether to download the initialization seed, the output end of the seventh selector is connected to the input end of the second register, the output end of the second register is connected to the second input end of the eighth selector, the first input end of the eighth selector is connected to the initialization seed, the control end of the eighth selector is connected to whether to download the initialization seed, and the output end of the eighth selector is connected to the input end of the first register; the output end of each register will generate 1 bit of data, and the output ends of the 8 registers will generate an 8-bit random number R.

[0043] Subsequently, the weights of the Tempotron layer are updated using low 8-bit random weight update. A 16-bit Δω is calculated according to the weight update formula, and the part exceeding the 8-bit update accuracy is taken as the probability P. Then, an 8-bit random number R is generated by an 8-bit random number generation circuit. If P≥0 and P≥R, the part exceeding the 8-bit update accuracy is carried over as 1. If P≥0 and P<R, the part exceeding the 8-bit update accuracy is discarded. If P≤0 and |P|≥R, the part exceeding the 8-bit update accuracy is carried over as -1. If P≤0 and P<R, the part exceeding the 8-bit update accuracy is discarded;

[0044] Among them, the weight update formula is as follows:

[0045]

[0046] Among them: λ is the learning rate, t i is the time when the presynaptic pulse arrives at the i-th synapse, t max is the time when the postsynaptic neuron membrane potential reaches the maximum value, K(t max -t i ) represents the PSP kernel.

[0047] The present invention has the following advantages:

[0048] (1) Through the improvement of the compressive sensing layer and the Tempotron algorithm, the network recognition rate has been significantly improved, the computing efficiency has been increased, and it is easy to be implemented in hardware.

[0049] (2) The proposed efficient FPGA hardware framework for the lightweight SNN network with on-chip learning can maximize the parallel processing speed while consuming the least resources and having the lowest cost, and is suitable for edge and mobile terminal scenarios with low cost, low power consumption, and high real-time requirements.

[0050] (3) The proposed hardware system architecture can configure parameters such as the number of neurons in each layer of the hardware framework according to requirements. Description of the Drawings

[0051] Figure 1 is the flowchart of this embodiment;

[0052] Figure 2 is the schematic diagram of the Compressive-Tempotron model in this embodiment;

[0053] Figure 3 is the schematic diagram of the lightweight on-chip learning FPGA hardware architecture in this embodiment;

[0054] Figure 4 is the circuit diagram of the second membrane potential update module in this embodiment;

[0055] Figure 5 is a circuit diagram of an 8-bit random number generation circuit in this embodiment. DETAILED DESCRIPTION

[0056] The present invention will be described in detail below with reference to the accompanying drawings.

[0057] In this embodiment, a design method for a lightweight on-chip learning FPGA hardware architecture is based on a pulse neural network structure, combined with compressed sensing rules and Tempotron algorithm to construct a lightweight pulse neural network. The compressed sensing layer directly emits pulses, and adds membrane potential and Trace collaborative reset mechanism to the traditional Tempotron algorithm, and is carried out in an event-driven manner, which improves the computational efficiency and is easy to implement in hardware. For the two-layer SNN network we proposed, a parallel event-driven on-chip learning FPGA hardware architecture is designed.

[0058] like Figure 1 As shown, in this embodiment, a design method for a lightweight on-chip learning FPGA hardware architecture specifically includes the following steps:

[0059] Step 1: Design the Compressive-Tempotron two-layer pulse neural network architecture and software fixed-point simulation.

[0060] Step 1-1, determine the encoding method for the input image:

[0061] The input layer uses a multi-pulse temporal encoding scheme to convert the pixels of the image into a pulse sequence. The multi-pulse temporal encoding scheme uses a leaky accumulate-and-fire (LIF) neuron model:

[0062] V m (t) = V m (t-1)+β×x (3)

[0063] Where: t is the current time step, t-1 is the previous time step, V m is the neuron membrane potential, x is the pixel value of a single pixel, and β is the growth rate of the pixel linearly over time. When the membrane potential accumulates to a preset threshold, the neuron fires a pulse and immediately returns to 0, then grows linearly over time again.

[0064] Step 1-2. Determine the pulse feature extraction rules and improve the feature extraction layer:

[0065] The compressed sensing layer extracts pulse features with spatiotemporal information from the pulses sent from the input layer. As shown in Figure (2), in a picture, two positive polarity patches and two negative polarity patches are randomly selected. For each patch, it is determined by three attributes: position, size, and polarity. For a patch, its position and size are randomly selected. For the hardware-friendly network, the compressed sensing layer and the Tempotron layer are both event-driven. The compressed sensing layer adopts a biologically plausible leaky cumulative firing (LIF) neuron model, whose membrane potential is described in an event-driven form as:

[0066]

[0067] Where: τ m is the first time decay constant, t p is the time when the presynaptic pulse arrives at the pth patch, t prev is the time before the current time t when the presynaptic pulse arrives at any synapse connected to the neuron, V is the membrane potential of the postsynaptic neuron, and the polarity value of Patch is represented by +1 / -1, that is, +1 indicates that the polarity of the patch is positive, and -1 indicates that the polarity of the patch is negative. p is the product of the area of ​​the pth patch and its polarity. Compared with the previous network with a compressed sensing layer, the improvement of this embodiment is that when the absolute value of the neuron membrane potential exceeds a preset threshold, a pulse is immediately emitted and restored to the resting value, without the need to accumulate the neuron membrane potential within the time window and then encode it, which is conducive to hardware implementation. As shown in Figure (2), according to the polarity of the neuron membrane potential, the pulse is emitted from the non-shifted or shifted address and sent to the next layer in the form of event-driven (AER).

[0068] Step 1-3. Determine the learning layer algorithm and improve the Tempotron layer;

[0069] In the traditional Tempotron learning rule, applying the LIF neuron model, the membrane potential (V(t)) of the postsynaptic neuron is described as:

[0070]

[0071] Where: t i represents the time when the presynaptic pulse arrives at the i-th synapse, ω i is the weight of the ith synapse, V rest is the resting value. K(tt i ) is the normalized postsynaptic membrane potential (PSP) when the input pulse acts on the postsynaptic neuron, that is, the PSP core, which is defined as:

[0072]

[0073] Where: τ is the second time decay constant, τ s is the synaptic current time decay constant, V 0 The PSP kernel is normalized to 1, so that the amplitude of the postsynaptic membrane potential (PSP) depends only on the weight ω of the synapse with input pulses i When V(t) exceeds the preset threshold, the postsynaptic neuron fires a pulse, after which the neuron no longer receives any input pulses, and the membrane potential gradually resets to the resting value V rest In order to allow the output neuron to emit pulses or not according to whether its label corresponds to the input image label, the learning rule of Tempotron is as follows: If the label of the output neuron corresponds to the input image label but no pulse is emitted, increase the max (t max is the time when the postsynaptic neuron membrane potential reaches its maximum value); if the label of the output neuron does not correspond to the input image label but a pulse is emitted, then the weight of the synapse with an input pulse before t max The weight of the synapse that had an input spike before. The weight update is described as:

[0074]

[0075] Where: λ is the learning rate, t i is the time when the ith input pulse arrives at the synapse, t max is the time when the membrane potential of the postsynaptic neuron reaches its maximum value. This means that input pulses after the firing pulse moment will no longer have an effect on the weight update.

[0076] Improved Tempotron layer:

[0077] In traditional Tempotron, the postsynaptic neuron only emits one pulse, while in the STDP learning rule, the postsynaptic neuron can emit multiple pulses. There are significant differences in the weight update methods of the two algorithms, except that the weight update of Tempotron is updated only after all input pulses of an image are processed. In the Tempotron learning rule, the input pulse after the membrane potential reaches the maximum membrane potential will not affect the weight update. In the STDP learning rule, the input pulse always has an impact on the weight update, that is, when the postsynaptic neuron emits a pulse, the input pulse before its membrane potential is reset still acts on the next weight update.

[0078] However, the effect of the pulse before the membrane potential is reset on the next weight update should be the same as the effect of the membrane potential at the time of the pulse emission on the membrane potential after the reset, that is, the membrane potential and the trace reset are coordinated. Based on this, the traditional Tempotron learning rule is improved. In the improved algorithm, the event-driven postsynaptic neuron membrane potential is described as:

[0079]

[0080] Where: τ m is the time decay constant, t i is the time when the presynaptic pulse arrives at the i-th synapse, t prev is the time before the current time t to reach any synapse connected to the postsynaptic neuron, ω i is the weight of the i-th synapse, and V is the membrane potential of the postsynaptic neuron. When the neuron membrane potential exceeds the preset threshold, it is reset to α times the original value (0≤α<1), and then the neuron membrane potential continues to accumulate-emit-reset.

[0081] In this embodiment, the weight update method is similar to that of the traditional Tempotron, except that we use a single-core PSP to update the weights. The single-core PSP used in this embodiment is described as:

[0082]

[0083] Where: τ is the second time decay constant, t max is the time when the neuron membrane potential reaches the maximum value within the time window, t i is the time when the i-th input pulse arrives at the synapse. In formula (5), the input pulse when the membrane potential reaches the maximum value also affects the weight update, but because formula (5) is a dual-core PSP, when t i =t max When K(t max -t i )=0. Therefore t i =t max The input pulse at time has no effect on the weight update. In the single-core PSP, K(t max -t i )=1, so t i =t max The input pulse at time has an impact on the weight update. Therefore, when the weight is updated, t i Should be less than or equal to t max , the weight update is described as:

[0084]

[0085] Where: λ is the learning rate, t iis the time when the ith input pulse arrives at the synapse, t max is the time when the postsynaptic neuron membrane potential reaches its maximum value. The above formula will consume a lot of memory space to store and access the presynaptic pulse time t i Therefore, in order to save hardware resources and improve computing speed, the cumulative term in equation (8) is rewritten as an event-driven synaptic trace update method:

[0086]

[0087] Where: τ is the second time decay constant, t max is the time when the neuron membrane potential reaches the maximum value within the time window, t prev is the time before the current time t to reach any synapse connected to the postsynaptic neuron, Q i is the synaptic trace of the i-th synapse. When a neuron receives a presynaptic pulse, its membrane potential is updated according to equation (8), and the synaptic trace (Q i (t)) multiplied by exp(-Δt i / τ), and then increase by 1. If the updated membrane potential is greater than the maximum membrane potential, the maximum membrane potential is updated with the current membrane potential of the neuron, and the current synaptic trace is backed up. When the postsynaptic neuron fires a pulse, the synaptic trace and the postsynaptic neuron membrane potential are reset to α times the original value. After processing all events in a graph, -λQ i (t max ) or λQ i (t max ) is added to the synaptic weight to complete equation (10).

[0088] After the designed network is trained, the label of the test sample is inferred based on the label corresponding to the neuron group with the largest number of pulses. If more than two neuron groups have the same number of pulses, the label of the input sample is considered unknown.

[0089] Step 1-4, software fixed-point simulation: Through fixed-point software simulation, the accuracy of the Compressive-Tempotron neural network in the MNIST data set is evaluated to be 95.36%.

[0090] Step 2: Design the hardware architecture: deploy the Compressive-Tempotron neural network in the FPGA.

[0091] like Figure 3 As shown, in this embodiment, a lightweight on-chip learning FPGA hardware architecture includes a global controller, a compressed sensing layer and a Tempotron layer;

[0092] The global controller is connected to the compressed sensing layer and the Tempotron layer respectively, and the global controller is used to control the compressed sensing layer and the Tempotron layer;

[0093] The compressed sensing layer includes m1 first Tile units, wherein each first Tile unit processes k1 first neurons in a time multiplexing manner; wherein 1≤m1≤64, 1≤k1≤8; wherein the first Tile unit includes a first local controller, and a first membrane potential memory, a first emission management module, a first membrane potential update module, a Patch memory, a first index lookup table memory, and a first previous event memory respectively connected to the first local controller, the first membrane potential memory is also respectively connected to the first emission management module and the first membrane potential update module, the first emission management module is connected to the first membrane potential update module, and the first membrane potential update module is respectively connected to the Patch memory, the first index lookup table memory, and the first previous event memory;

[0094] The Tempotron layer includes m2 second Tile units, wherein each second Tile unit processes k2 second neurons in a time multiplexing manner; wherein 1≤m2≤64, 1≤k2≤4; wherein the second Tile unit includes a second local controller, and a learning module, a backup Trace memory, a maximum membrane potential management module, a second emission management module, a weight memory, a second membrane potential memory, a Trace memory, a Trace update module, a second membrane potential update module, a maximum membrane potential memory, a second previous event memory, and a second index lookup table memory respectively connected to the second local controller. memory; the learning module is also connected to the weight memory, the second emission management module and the backup Trace memory respectively; the second membrane potential update module is also connected to the weight memory, the second membrane potential memory, the Trace update module and the second index lookup table memory respectively; the second membrane potential memory is also connected to the second emission management module; the Trace update module is also connected to the second index lookup table memory, the second previous event memory and the Trace memory respectively; the Trace memory is also connected to the second emission management module; the maximum membrane potential management module is also connected to the maximum membrane potential memory, the second emission management module and the backup Trace memory respectively.

[0095] This lightweight on-chip learning FPGA hardware architecture can realize the configuration of different network scales and parameters, that is, the number of neurons, learning rate, threshold of the compressed sensing layer and the number of neurons, reset rate, learning rate, and threshold of the Tempotron layer can be flexibly configured according to different needs.

[0096] In this embodiment, in order to save hardware resources, for the Tempotron layer, the weights of the Tempotron layer are updated using low 8-bit random weight update. A 16-bit Δω is calculated according to the weight update formula (8), and the part exceeding the 8-bit update accuracy is used as the probability P. Then, an 8-bit random number R is generated by an 8-bit random number generation circuit. If P≥0 and P≥R, the part exceeding the 8-bit update accuracy is carried over as 1. If P≥0 and P<R, the part exceeding the 8-bit update accuracy is discarded. If P≤0 and |P|≥R, the part exceeding the 8-bit update accuracy is carried over as -1. If P≤0 and P<R, the part exceeding the 8-bit update accuracy is discarded.

[0097] As Figure 4 shown, in this embodiment, the second membrane potential update module includes a first multiplier 1 and a first adder 2, and the first multiplier 1 is connected to the first adder 2. When a pulse arrives, the update of the membrane potential is triggered. First, t prev is read from the second previous event memory, and the current time t is subtracted from t prev to obtain the time difference Δt. The exponential decay coefficient is obtained by looking up the exponential lookup table stored in the second exponential lookup table memory through Δt. The obtained exponential decay coefficient and the membrane potential of the current neuron read from the second membrane potential memory are used as the inputs of the first multiplier 1 to obtain the output of the first multiplier 1, that is, the decayed membrane potential. The decayed membrane potential and the weight of the synapse where the pulse of the current neuron arrives read from the weight memory are used as the inputs of the first adder 2 to obtain the output of the first adder 2, that is, the updated membrane potential V(t), and the updated membrane potential is written back to the second membrane potential memory.

[0098] In this embodiment, for the first membrane potential update module of each first Tile unit in the compressive sensing layer, the difference from the hardware circuit of the second membrane potential update module is that it is necessary to determine whether to perform membrane potential update according to whether the event is within the neuron patch.

[0099] As Figure 5 shown, in this embodiment, the learning module uses an 8-bit random number generation circuit;

[0100] The 8-bit random number generation circuit includes a first register to an eighth register, a first selector to an eighth selector, and a first exclusive OR gate to a fourth exclusive OR gate;

[0101] The first input end of the first selector 3 is connected to the initialization seed (the value 1 given directly by the initialization seed), the second input end of the first selector 3 is connected to the output end of the first register 15, the control end of the first selector 3 is connected to whether to download the initialization seed, the output of the first selector 3 is connected to the input end of the eighth register 22, the output of the eighth register 22 and the output of the first register 15 are connected to the second input end of the second selector 5 through the first XOR gate 4, the first input end of the second selector 5 is connected to the initialization seed, the control end of the second selector 5 is connected to whether to download the initialization seed (the reset signal from the Tile), and the output end of the second selector 5 is connected to the seventh register 2 1, the output end of the seventh register 21 is connected to the second input end of the third selector 6, the first input end of the third selector 6 is connected to the initialization seed, the control end of the third selector 6 is connected to whether to download the initialization seed, the output end of the third selector 6 is connected to the input end of the sixth register 20, the output end of the sixth register 20 is connected to the input end of the first register 15 through the second XOR gate 7 and the second input end of the fourth selector 8, the first input end of the fourth selector 8 is connected to the initialization seed, the control end of the fourth selector 8 is connected to whether to download the initialization seed, the output end of the fourth selector 8 is connected to the input end of the fifth register 19, and the output end of the fifth register 19 is connected to the The output end of the first register 15 is connected to the second input end of the fifth selector 10 through the third XOR gate 9, the first input end of the fifth selector 10 is connected to the initialization seed, the control end of the fifth selector 10 is connected to whether to download the initialization seed, the output end of the fifth selector 10 is connected to the input end of the fourth register 18, the output end of the fourth register 18 and the output end of the first register 15 are connected to the second input end of the sixth selector 12 through the fourth XOR gate 11, the first input end of the sixth selector 12 is connected to the initialization seed, the control end of the sixth selector 12 is connected to whether to download the initialization seed, the output end of the sixth selector 12 is connected to the third register 17, and the third register 18 is connected to the input end of the fourth register 18. The output end of the selector 17 is connected to the second input end of the seventh selector 13, the first input end is connected to the initialization seed, the control end of the seventh selector 13 is connected to whether to download the initialization seed, the output end of the seventh selector 13 is connected to the input end of the second register 16, the output end of the second register 16 is connected to the second input end of the eighth selector 14, the first input end of the eighth selector 14 is connected to the initialization seed, the control end of the eighth selector 14 is connected to whether to download the initialization seed, and the output end of the eighth selector 14 is connected to the input end of the first register 15; the output end of each register will generate 1 bit of data, and the output ends of the 8 registers will generate an 8-bit random number R.

[0102] Based on the improvement of previous compressed sensing rules and traditional Tempotron algorithms, an efficient FPGA hardware framework and circuit for lightweight SNN networks with on-chip learning are proposed. The hardware system adopts an event-driven pipeline architecture, which significantly improves the calculation speed, and adds low 8-bit random weight updates and synaptic trace update mechanisms, which greatly reduces hardware resource consumption. Based on the above characteristics, the hardware system is easy to integrate into edge or mobile terminal devices with limited cost and power consumption.

[0103] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A design method for a lightweight on-chip learning FPGA hardware architecture, characterized in that: The following steps are involved: Step 1: Design the Compressive-Tempotron two-layer pulse neural network architecture and software fixed-point simulation, including: Step 1-1. Determine how to encode the input image: The input layer uses multi-pulse time coding to convert the pixels of the image into a pulse sequence, where the multi-pulse time coding uses a leaky cumulative firing neuron model: V m (t)=V m (t-1)+β×x (1) Where: t is the current time step, t-1 is the previous time step, V m is the neuron membrane potential, x is the pixel value of a single pixel, and β is the growth rate of the pixel growing linearly over time; When the membrane potential accumulates to a pre-set threshold, the neuron fires a pulse, immediately returns to 0, and then increases linearly over time again; Step 1-2. Determine the pulse feature extraction rules and improve the feature extraction layer: The compressed sensing layer extracts pulse features with spatiotemporal information from the pulses sent from the input layer. In one image, two positive polarity patches and two negative polarity patches are randomly selected. Each patch is determined by three attributes: position, size, and polarity, where the position and size are randomly selected. Both the compressed sensing layer and the Tempotron layer are event-driven. The compressed sensing layer adopts a biologically plausible leaky cumulative firing neuron model, whose membrane potential is described in an event-driven form as: Where: τ m is the first time decay constant, t p is the time when the presynaptic pulse arrives at the pth patch, t prev is the time before the current time when the presynaptic pulse arrives at any synapse, V is the membrane potential of the postsynaptic neuron, and the polarity value of Patch is represented by +1 / -1, that is, +1 indicates that the polarity of the patch is positive, and -1 indicates that the polarity of the patch is negative. p is the product of the area of ​​the pth patch and its polarity; Step 1-3. Determine the learning layer algorithm and improve the Tempotron layer; The traditional Tempotron learning rules are improved as follows: The event-driven description of the postsynaptic neuron membrane potential is: Where: t i is the time it takes for the presynaptic pulse to arrive at the i-th synapse, ω i is the weight of the i-th synapse, V is the membrane potential of the postsynaptic neuron; when the neuron membrane potential exceeds the preset threshold, it is reset to α times the original value, 0≤α<1, and then the neuron membrane potential continues to accumulate-emit-reset; Use single-core PSP for weight update, where single-core PSP is: Where: τ is the second time decay constant, t max is the time when the postsynaptic neuron membrane potential reaches its maximum value within the time window, t i is the time when the presynaptic pulse arrives at the i-th synapse; When the weight is updated, t i Should be less than or equal to t max , the weight update is described as: Where: λ is the learning rate; Steps 1-4, software fixed-point simulation: Use fixed-point software simulation to evaluate the accuracy of the Compressive-Tempotron neural network on the MNIST dataset. Step 2: Design the hardware architecture: deploy the Compressive-Tempotron neural network in the FPGA; The accumulated term in equation (5) is rewritten as an event-driven synaptic trace update method, specifically: Where: τ is the second time decay constant, t max is the time when the neuron membrane potential reaches the maximum value within the time window, t prev is the time before the current time when the presynaptic pulse arrives at any synapse, Q i is the synaptic trace of the i-th synapse; When a neuron receives a presynaptic pulse, its membrane potential is updated according to equation (3), and the synaptic trace (Q i (t)) multiplied by exp(-Δt i / τ), and then increase by 1; if the updated membrane potential is greater than the maximum membrane potential, the maximum membrane potential is updated with the current membrane potential of the neuron, and the current synaptic trace is backed up; when the postsynaptic neuron emits a pulse, the synaptic trace and the postsynaptic neuron membrane potential are reset to α times the original value; after processing all events of a graph, -λQ i (t max ) or λQ i (t max ) is added to the synaptic weight to complete equation (5).

2. A lightweight on-chip learning FPGA hardware architecture, characterized by: Includes global controller, compressed sensing layer and Tempotron layer; The global controller is connected to the compressed sensing layer and the Tempotron layer respectively, and the global controller is used to control the compressed sensing layer and the Tempotron layer; The compressed sensing layer includes m1 first Tile units, wherein each first Tile unit processes k1 first neurons in a time multiplexing manner; wherein 1≤m1≤64, 1≤k1≤8; wherein the first Tile unit includes a first local controller, and a first membrane potential memory, a first emission management module, a first membrane potential update module, a Patch memory, a first index lookup table memory, and a first previous event memory respectively connected to the first local controller, the first membrane potential memory is also respectively connected to the first emission management module and the first membrane potential update module, the first emission management module is connected to the first membrane potential update module, and the first membrane potential update module is respectively connected to the Patch memory, the first index lookup table memory, and the first previous event memory; The Tempotron layer includes m2 second Tile units, where each second Tile unit processes k2 second neurons in a time-division multiplexing manner; where 1 ≤ m2 ≤ 64 and 1 ≤ k2 ≤ 4; the second Tile unit includes a second local controller, and a learning module, a backup Trace memory, a maximum membrane potential management module, a second firing management module, a weight memory, a second membrane potential memory, a Trace memory, a Trace update module, a second membrane potential update module, a maximum membrane potential memory, a second previous event memory, and a second exponential lookup table memory, which are respectively connected to the second local controller; the learning module is further connected to the weight memory, the second firing management module, and the backup Trace memory; the second membrane potential update module is further connected to the weight memory, the second membrane potential memory, the Trace update module, and the second exponential lookup table memory; the second membrane potential memory is further connected to the second firing management module; the Trace update module is further connected to the second exponential lookup table memory, the second previous event memory, and the Trace memory; the Trace memory is further connected to the second firing management module; the maximum membrane potential management module is further connected to the maximum membrane potential memory, the second firing management module, and the backup Trace memory.

3. The lightweight on-chip learning FPGA hardware architecture according to claim 2, characterized in that: The second membrane potential update module includes a first multiplier (1) and a first adder (2), and the first multiplier (1) is connected to the first adder (2); When a pulse arrives, the membrane potential is updated, and t is first read from the second previous event memory. prev , subtract t from the current time t prev A time difference Δt is obtained, and an exponential decay coefficient is obtained by looking up an exponential lookup table stored in a second exponential lookup table memory through Δt. The exponential decay coefficient found and the membrane potential of the current neuron read from the second membrane potential memory are used as inputs of a first multiplier (1), and the output of the first multiplier (1), i.e., the attenuated membrane potential, is obtained. The attenuated membrane potential and the weight of the synapse at which the pulse of the current neuron arrives read from the weight memory are used as inputs of a first adder (2), and the output of the first adder (2), i.e., the updated membrane potential V(t), is obtained, and the updated membrane potential is written back to the second membrane potential memory.

4. The lightweight on-chip learning FPGA hardware architecture according to claim 2 or 3, characterized in that: The learning module employs an 8-bit random number generation circuit; The 8-bit random number generation circuit includes a first register to an eighth register, a first selector to an eighth selector, and a first exclusive OR gate to a fourth exclusive OR gate; The first input end of the first selector (3) is connected to the initialization seed, the second input end of the first selector (3) is connected to the output end of the first register (15), the control end of the first selector (3) is connected to whether to download the initialization seed, the output of the first selector (3) is connected to the input end of the eighth register (22), the output of the eighth register (22) and the output of the first register (15) are connected to the second input end of the second selector (5) through the first XOR gate (4), the first input end of the second selector (5) is connected to the initialization seed, the control end of the second selector (5) is connected to whether to download the initialization seed, the output end of the second selector (5) is connected to the input end of the seventh register (21), and the seventh register (21) is connected to the input end of the seventh register (21). The output end of the register (21) is connected to the second input end of the third selector (6), the first input end of the third selector (6) is connected to the initialization seed, the control end of the third selector (6) is connected to whether to download the initialization seed, the output end of the third selector (6) is connected to the input end of the sixth register (20), the output end of the sixth register (20) and the input end of the first register (15) are connected to the second input end of the fourth selector (8) through the second XOR gate (7), the first input end of the fourth selector (8) is connected to the initialization seed, the control end of the fourth selector (8) is connected to whether to download the initialization seed, the output end of the fourth selector (8) is connected to the input end of the fifth register (19), and the fifth register (19) is connected to the first register (15). The output end of the register (19) and the output end of the first register (15) are connected to the second input end of the fifth selector (10) through the third XOR gate (9); the first input end of the fifth selector (10) is connected to the initialization seed; the control end of the fifth selector (10) is connected to whether to download the initialization seed; the output end of the fifth selector (10) is connected to the input end of the fourth register (18); the output end of the fourth register (18) and the output end of the first register (15) are connected to the second input end of the sixth selector (12) through the fourth XOR gate (11); the first input end of the sixth selector (12) is connected to the initialization seed; the control end of the sixth selector (12) is connected to whether to download the initialization seed; The output end of the sixth selector (12) is connected to the third register (17), the output end of the third register (17) is connected to the second input end of the seventh selector (13), the first input end is connected to the initialization seed, the control end of the seventh selector (13) is connected to whether to download the initialization seed, the output end of the seventh selector (13) is connected to the input end of the second register (16), the output end of the second register (16) is connected to the second input end of the eighth selector (14), the first input end of the eighth selector (14) is connected to the initialization seed, the control end of the eighth selector (14) is connected to whether to download the initialization seed, and the output end of the eighth selector (14) is connected to the input end of the first register (15);The output of each register will generate 1 bit of data, and the output of 8 registers will generate an 8-bit random number R. ; 5. The lightweight on-chip learning FPGA hardware architecture according to claim 4, characterized in that: The update of the weights in the Tempotron layer adopts low 8-bit random weight update. A 16-bit Δω is calculated according to the weight update formula, and the part exceeding the 8-bit update accuracy is taken as the probability P. Then, an 8-bit random number R is generated by the 8-bit random number generation circuit. If P ≥ 0 and P ≥ R, the part exceeding the 8-bit update accuracy is carried over as 1. If P ≥ 0 and P < R, the part exceeding the 8-bit update accuracy is discarded. If P ≤ 0 and |P| ≥ R, the part exceeding the 8-bit update accuracy is carried over as -1. If P ≤ 0 and P < R, the part exceeding the 8-bit update accuracy is discarded; Among them, the weight update formula is as follows: Where: λ is the learning rate, t i is the time when the presynaptic pulse arrives at the i-th synapse, t max is the time when the postsynaptic neuron membrane potential reaches its maximum value, K(t max -t i ) represents the PSP core.

Images