Chip architecture supporting risc-v instruction set and impulse neural network dedicated extension instruction set

Abstract

This invention proposes a chip architecture supporting the RISC-V instruction set and a dedicated extended instruction set for spiking neural networks (SNNs). The dedicated extended instructions for SNNs undergo a fetching, decoding, and transmission process before entering the SNN computation unit. The SNN initialization instructions enable training or inference and set the initialization voltage. Synaptic and input data are read from memory according to the pulse processing instructions in the dedicated extended instructions and processed in the pulse processing unit to obtain the update voltage required by the neurons. The neuron state is read from memory according to the neuron-related instructions in the dedicated extended instructions and enters the streamlined LIF neuron update unit to update the neurons according to the update voltage. The update result is stored in a dedicated SNN register. Synapses are updated according to the synaptic update instructions in the dedicated extended instructions. The results of the SNN computation unit are written back through a write-back stage.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture and deep learning and artificial intelligence, and particularly to a handwritten data recognition method and system based on dedicated extended instructions of spiking neural networks. Background Technology

[0002] In recent years, deep learning networks have been deployed using a "terminal-cloud server" architecture. However, this undoubtedly places high demands on communication technology and server scale. Distributed computing deployment on the terminal is one solution. In today's terminal applications, such as edge computing and the Internet of Things (IoT), there is a huge demand for accelerated computing in artificial intelligence. Neuromorphic computing based on spiking neural networks (SNNs) has become the best choice for low-power AI hardware implementation due to its high energy efficiency. In terms of hardware implementation, accelerated computing has become the main research direction for optimizing SNN computation. Current mainstream hardware acceleration solutions consist of a control core (CPU) and a dedicated neuromorphic computing accelerator for SNNs. The CPU controls the data input and output of the accelerator, while the accelerator is responsible for data computation. For example, the online-learning digital spiking neuromorphic processor (ODIN) supports two neuron models: the Leaky Integrate-and-Fire model (LIF model) and the izhikevich model, as well as low-power computation of spike-dependent synaptic plasticity learning rules. The implementation of ODIN requires a RISC-V Rocket core as its control core, with an SPI Flash memory buffer to handle communication between the control core and the accelerator. This system fails to fully leverage the accelerator's low-power advantages, increasing overall system power consumption.

[0003] The main technical problems with existing accelerated computing solutions for spiking neural networks are: 1. Accelerators require a control core as a system component to control and configure the accelerator, which prevents the low-power characteristics of SNN accelerators from being fully utilized; 2. Accelerators have their own unique instruction encoding. When they are not in the same architecture as the control core, storage buffers such as SPI Flash are needed between the control core and the accelerator to store the instructions to be executed, which makes it very difficult to execute instructions efficiently; 3. The neuron computing supported by existing neuromorphic accelerator chips is not suitable for integration into pipelined structures. Summary of the Invention

[0004] Firstly, regarding the first technical problem mentioned above, based on existing research on neuromorphic accelerated computing solutions, accelerators and control cores are widely used as a whole acceleration system. However, the total power consumption of such a system prevents the full realization of the low-power characteristics of SNNs. Therefore, this invention proposes integrating the accelerator's functionality into the processor core (control core) and using the same instruction set architecture as the control core for instruction set extension, thereby achieving pipelining processing.

[0005] In the second problem, the inventors discovered through research that the lack of a standard instruction set definition in existing neuromorphic accelerated computing led to this issue. Further research revealed that the RISC-V architecture is designed in a modular manner, meaning that this instruction set architecture has various instruction extension subsets that can be selectively implemented or customized as needed.

[0006] The third major issue is divided into two parts. First, regarding the implementation of neurons in the pipeline, the inventors discovered in their research on neuron model support that using the mathematical description of existing LIF models would significantly reduce the overall pipeline efficiency. This is because the mathematical description of existing LIF models includes multiplication and time-dependent operations. The inventors found that converting time-dependent calculations to sequential calculations could solve the problem of excessive multiplication. This transforms multiplication operations in neuron state updates into addition operations, reducing hardware overhead and power consumption. Therefore, the inventors proposed a streamlined LIF model. Second, regarding the implementation of learning rules, the inventors found that using higher-precision (multi-bit) synaptic weights for learning results in higher power consumption than using binary (one-bit) synaptic weights. Furthermore, multi-bit learning involves EXP (exponential expansion) operations, making it even less suitable for integration into the pipeline. Therefore, the inventors adopted Binary Stochastic Spike-timing-dependent plasticity (BS-STDP) as the learning rule, which further reduced power consumption and enabled integration into the pipeline.

[0007] Specifically, this invention proposes a chip architecture supporting the RISC-V instruction set and a dedicated extended instruction set for spiking neural networks. In a sequential single-issue RISC-V processor, the instruction fetch unit retrieves the instruction, which is then decoded by the decoder unit. After decoding, the instruction is sent to the instruction issue unit, which controls multiple general-purpose registers, the arithmetic unit, the multiplication unit, and the load-to-store unit to perform corresponding functions. Finally, the write-back unit returns the calculation result and a signal indicating instruction completion.

[0008] A dedicated SNN register is added to the issue stage of the sequential single-issue RISC-V processor, and an SNN computation unit is added to the execution stage of the sequential single-issue RISC-V processor. The SNN computation unit includes a streamlined LIF neuron update unit, a binary synapse update unit, and a pulse processing unit.

[0009] Add a dedicated extended instruction set for spiking neural networks to the decoder unit;

[0010] This dedicated SNN register is used to store synapses, neuron updates, leakage voltages, initial voltages, and firing results.

[0011] The dedicated extended instructions of the spiking neural network enter the SNN computation unit after being fetched, decoded, and emitted. The SNN initialization instructions enable training or inference and set the initialization voltage. According to the spiking processing related instructions in the dedicated extended instructions, synaptic and input data are read from memory and entered into the spiking processing unit for spiking processing to obtain the update voltage required by the neurons. According to the neuron-related instructions in the dedicated extended instructions, the neuron state is read from memory and entered into the streamlined LIF neuron update unit to update the neurons according to the update voltage. The update result is stored in the dedicated SNN register. Synapses are updated according to the synaptic update related instructions in the dedicated extended instructions. The results of the SNN computation unit are written back through the write-back stage.

[0012] The chip architecture described above supports the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks, wherein neurons are updated using the following formula:

[0013] S current =S previous +V valid -V leak

[0014] Among them, S current The current neuron membrane potential, S previous V represents the membrane potential of the current neuron in the previous cycle. valid The effective voltage input to the current neuron, V leak This represents the write-leak voltage of the current neuron; when S current If the threshold is exceeded, the neuron fires a pulse and resets the membrane potential.

[0015] The chip architecture that supports the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks includes the following process for updating synapses:

[0016] The input pulse that stimulates the neuron enters the long-range enhancement (LTP) module. If the current neuron fires, the input pulse is used to perform long-range enhancement with the synapses read from the synaptic storage, preserving or activating the synapses that contribute to the neuron's firing.

[0017] After passing through the LTP module, the currently processed synapses are determined by the number W of activated synapses. sum And expected weight accuracy W exp The long-range inhibition probability P can be obtained using the following formula. LTD :

[0018]

[0019] Δw=W sum -W exp P LTD =0 if Δw=0

[0020] The LTD module is based on P LTD To process synapses, the LTD module generates a 10-bit random number to process the activated synapse. When this random number is greater than P... LTD If the current synapse is updated, it will be set to 1; otherwise, it will be set to 0. After all synapses have been updated, the synapses will be saved to the synapse storage.

[0021] The chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks includes a pulse processing unit that reads synapses from storage into a synapse register, performs a bitwise AND operation with the pulse input, and finally counts the number of valid pulses to form the update voltage for the neuron update unit. The neuron update unit reads the neuron state into a neuron register, calculates the updated neuron state using the value in the drain voltage register and the number of valid pulses, and then compares it with a threshold voltage. If the value is greater than or equal to the threshold voltage, the neuron is released, its state is set to its initial value, and stored back into storage; otherwise, it is directly stored back into storage without release. The binary synapse update unit passes the value previously stored in the synapse register to the long-range enhancement unit, then counts the number of activated synapses. This number is then passed to the long-range inhibition unit to obtain the long-range inhibition probability for long-range inhibition, and finally, the update result is stored back into the synapse storage.

[0022] The chip architecture that supports the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks is described above, wherein the sequential single-issue RISC-V processor is the Wenquxing 22A processor.

[0023] As can be seen from the above solutions, the advantages of the present invention are:

[0024] Through research and experimentation, we applied this technology to handwritten digit recognition, achieving better accuracy compared to other similar accelerators. Under the same experimental conditions (deployed on the same FPGA platform, the processor implementing this technology, the Wenquxing 22A, consumes only one-fifth the power of the ODIN). After normalizing power efficiency, compared to other similar neuromorphic chips, the Wenquxing 22A achieves 2.4 TSOP / W (Tera Synaptic Operations per Second per Watt) at 300MHz; the area-to-power ratio reaches 339.9 SOP / LUT (Synaptic Operations per LUT). Furthermore, because this invention unifies the control and accelerator architecture, it no longer relies on SPI. Attached Figure Description

[0025] Figure 1 BS-STDP execution process flowchart;

[0026] Figure 2 Schematic diagram of the microarchitecture of the Wenquxing 22A;

[0027] Figure 3 This is a block diagram of the pulse processing unit.

[0028] Figure 4 A block diagram of a streamlined neuron processing unit structure;

[0029] Figure 5 This is a block diagram of the binary synaptic update unit structure;

[0030] Figure 6 This is a flowchart of the SNN online learning process. Detailed Implementation

[0031] The purpose of this invention is to solve the three problems of the prior art: 1. Accelerators require a control core as a system component to control and configure the accelerator, which results in the total power consumption of such a system preventing the full realization of the low-power characteristics of SNN; 2. Some accelerators have their own unique instruction encoding, which is not part of the same architecture as the control core, making instruction generation very difficult; 3. The neuronal computation supported by existing neuromorphic accelerator chips is not suitable for integration into the pipeline structure. Therefore, this invention proposes a dedicated binary SNN instruction set, RV-SNN, based on the RISC-V instruction set architecture. By supporting the streamlined LIF model, binary random impulse time-dependent plasticity (BS-STDP), and related basic operations, neuromorphic accelerated computation can be integrated into the RISC-V processor pipeline, achieving the goal of low power consumption.

[0032] To address the above problems, the present invention proposes the following key technical points:

[0033] Key Point 1: Streamlined LIF model and its hardware implementation; Technical effect: The LIF model is integrated into the RISC-V single-issue sequential core pipeline, and the behavior of the LIF neuron model is described by simple equations.

[0034] Key Point 2: A dedicated instruction set RV-SNN for binary spiking neural networks (SNNs) based on the RISC-V instruction set architecture and its encoding; Technical effect: SNN computation modules integrated into the processor core are controlled by a custom instruction set, enabling pipelined SNN computation; This instruction set extension is based on the RISC-V instruction format and is encoded according to the RISC-V instruction format. The decoding of the extended instructions in the processor is compatible with other RISC-V instructions. Therefore, the extended instructions here are not part of the RISC-V instruction set, but are encoded according to the RISC-V instruction set format and integrated into the RISC-V processor core, which is compatible with both RISC-V instructions and the extended instructions of this application.

[0035] Key point 3: The aforementioned neuron model, learning rules, and instruction set extensions are all deployed and implemented on a neuromorphic processor, such as the Wenquxing 22A chip, and can autonomously initiate online learning or inference.

[0036] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0037] Specifically, this invention proposes a chip architecture that supports both the general-purpose RISC-V instruction set and a dedicated extended instruction set for spiking neural networks, characterized by:

[0038] Retain components related to the sequentially single-issue RISC-V processor. For example... Figure 2 As shown in the pipeline, the instruction fetch unit (IFU, IAB) fetches the instruction. After fetching, the instruction enters the decoder unit (IDU) for decoding. After decoding, the instruction enters the issue unit (ISU), where it controls 32 general-purpose registers, the arithmetic logic unit (ALU, BRU), the multiplication unit (MDU), and the load-to-store unit (LSU) to perform their corresponding functions. Finally, the write-back unit (WBU) sends back the result and the instruction completion signal. This process demonstrates that this design retains support for general-purpose RISC-V instructions.

[0039] Support for the RV-SNN instruction set is added to the decoder unit. The RV-SNN instruction set encoding is shown in Table 1.

[0040]

[0041] Table 1 RV-SNN Encoding

[0042] A dedicated SNN register is added to the transmitter stage to store synapse / neuron updates, leakage voltage, initial voltage, and firing results.

[0043] An RV-SNN instruction execution unit is added at the execution level. This includes a spike processing unit (SPU), a neuron update unit (NU), and a synapse update unit (SU). These three parts together constitute the SNN computation unit.

[0044] like Figure 6 As shown, RV-SNN instructions enter the SNN computation unit after being fetched, decoded, and emitted. SNN initialization instructions enable training or inference and set the initial voltage; impulse processing instructions read synaptic and input data from memory, enter the SPU for impulse processing, and obtain the update voltage required by the neuron; neuron-related instructions read the neuron state from memory and enter the NU, then update the neuron using the update voltage, and the emission result is stored in the SNN dedicated register; synaptic update instructions use BS-STDP to update the synapses; the results of the SNN computation unit are written back through the write-back stage.

[0045] The above-mentioned neuron updates include the use of the following formula:

[0046] S current =S previous +V valid -V leak

[0047] Among them, S current The current neuron membrane potential, S previous V represents the membrane potential of the current neuron in the previous cycle. valid The effective voltage input to the current neuron, V leak This represents the write-leak voltage of the current neuron; when S current If the threshold is exceeded, the neuron fires a pulse and resets the membrane potential.

[0048] like Figure 1 As shown in Figure 5, the BS-STDP update algorithm described uses the following process:

[0049] The input pulse that stimulates the neuron enters the long-range enhancement (LTP) module. If the current neuron fires, the input pulse is used to perform long-range enhancement with the synapses read from the synaptic storage, preserving or activating the synapses that contribute to the neuron's firing.

[0050] After passing through the LTP module, the currently processed synapses are determined by the number W of activated synapses. sum And expected weight accuracy W exp The long-range inhibition probability P can be obtained using the following formula. LTD :

[0051]

[0052] Δw=W sum -W exp P LTD =0 if Δw=0

[0053] The LTD module is based on P LTD To process synapses, the LTD module generates a 10-bit random number to process the activated synapse. When this random number is greater than P... LTD If the current synapse is updated, it will be set to 1; otherwise, it will be set to 0. After all synapses have been updated, the synapses will be saved to the synapse storage.

[0054] like Figure 3 As shown, the pulse processing unit (SPU) reads the synapse from storage into the synapse register, performs a bitwise AND operation between the synapse register and the pulse input, and finally counts the number of valid pulses to form the update voltage entering the neuron update unit; as shown... Figure 4 As shown, the neuron update unit (NU) reads the neuron state into the neuron register, calculates the updated neuron state using the value in the drain voltage register and the number of effective pulses (update voltage), and then compares it with the threshold voltage. If it is greater than or equal to the threshold voltage, the neuron is released, its state is set to the initial value, and stored back; otherwise, it is directly stored back without being released. Figure 5 As shown, the binary synapse update unit passes the value previously stored in the synapse register to the long-range enhancement unit, then counts the number of activated synapses, and this number enters the long-range suppression unit to obtain the long-range suppression probability. A 64-bit random number generator generates a random number for long-range suppression, and finally the update result is stored back in the synapse storage.

[0055] This invention analyzes the problems existing in the current "control core-accelerator" neuromorphic accelerated computing solution and proposes a main idea: to integrate the accelerated computing function of spiking neural networks into the control core by utilizing an extended instruction set. This core can autonomously choose online learning or inference, thus integrating neuromorphic accelerated computing into the processor pipeline, achieving low power consumption and low area. The following are the key points of this invention.

[0056] 1. Streamlined LIF Model: Traditional LIF models are not easily integrated into pipelines, therefore, they need to be pruned. According to the traditional LIF model, it is driven by an arbitrary time-dependent input current I. To avoid event-dependent computations and to implement the LIF model in the Wenquxing 22A, the inventors pruned the LIF model to adapt it for streamlined computation. This invention omits the time parameter because time in a pipeline is decomposed into a series of clock cycles; instead, the model is driven by a sequential pulse input. The implementation equations are as follows:

[0057] S current =S previous +V valid -V leak (1)

[0058] Among them, S current The current neuron membrane potential, S previous V represents the membrane potential of the current neuron in the previous cycle. valid The effective voltage input to the current neuron, V leak This represents the current write / leak voltage of the neuron. Specifically, when S... current If the threshold is exceeded, the neuron fires a pulse and resets the membrane potential.

[0059] 2. Binary Stochastic STDP: This invention utilizes a hardware-friendly method: binary synaptic weights, to implement the synaptic model. The finite weights in this method may lead to a decrease in accuracy. To avoid this, we propose using an efficient STDP learning rule, namely Binary Stochastic STDP (BS-STDP). The specific process is as follows... Figure 1 As shown:

[0060] a. Input pulses that stimulate neurons enter the Long-Range Enhancement (LTP) module. If the current neuron is firing (membrane potential greater than a threshold), the input pulse performs long-range enhancement with synapses read from synaptic storage, preserving or activating synapses that contribute to neuronal firing. The processed synapses proceed to the next step.

[0061] b. After passing through the LTP module, the number of activated synapses in the currently processed synapses is counted, and the long-range inhibition probability is calculated based on the expected weight precision and the following equation:

[0062]

[0063] Δw=W sum -W exp P LTD =0 if Δw=0

[0064] In the above formula, P LTDW refers to the long-range inhibition probability. sum This refers to the count of activated synapses, W. exp This refers to the expected weight accuracy. Long-range inhibition probability and synapse progression proceed to the next step.

[0065] c. Long-range inhibition probability (hereinafter referred to as P) LTD After the probability is calculated, the Long Range Inhibition Module (LTD module) begins processing the synapse according to this probability. The LTD module generates a 10-bit random number for processing each activated synapse. When this random number is greater than P... LTD If the synapse is not updated, it will be preserved (set to 1); otherwise, it will be suppressed (set to 0). After all synapses have been updated, they will be saved to the synapse storage.

[0066] 3. RV-SNN instruction set encoding: as shown in Table 1

[0067]

[0068] Table 1 RV-SNN Encoding

[0069] These are divided into three categories:

[0070] a. Neuron-related:

[0071] a) Specify the drain voltage using the VLEAK instruction; the drain voltage is loaded into the vleak drain voltage register.

[0072] b) Add instruction NADD to neuron state, nr neuron register + valid impulse - vleak register -> nr neuron register;

[0073] c) The pulse comparison instruction SGE returns 1 if the value in the nr neuron register is greater than the threshold voltage rs;

[0074] d) The output left shift instruction SLS performs a left shift operation on the specified output register;

[0075] b. Synaptic-related:

[0076] a) Long-range augmentation (SUP) performs long-range LTP augmentation on the weights in the sr synaptic register;

[0077] b) Long-range suppression (LTD): Long-range suppression of weights in the SR protrusion register is applied using LTD.

[0078] c. Other categories:

[0079] a) Valid pulse and ANDS, used by the pulse processing unit to obtain valid pulse;

[0080] b) Valid Pulse Count (RPOP), used to count the number of valid 1s in the register;

[0081] c) Loading NLDs into neurons / synapses;

[0082] d) Neuron / synapse storage back to NST;

[0083] e) Specify the supervisor INF;

[0084] f) Initialize SINIT in SNN, initialize the initial voltage and whether to enable online learning (if online learning is disabled, it becomes inference);

[0085] A spiking neural network consists of neurons and synapses. Neurons fire impulses (spiking signals), while synapses transmit these impulses. The two are closely related; the plasticity of synapses allows for the selection of impulses transmitted to neurons. In other words, if a neuron is to be sensitive to specific data, the synapse is a crucial component in achieving this sensitivity. The recognition data after training is the same as the data input to the network during training. The features of the data to be recognized, after being pulse-coded, form a pulse sequence, which is then directly input into the network. The firing rate of the corresponding neuron will be higher than that of other neurons.

[0086] In a specific experiment, the inventors implemented the above-mentioned invention technology on the Wenquxing 22A based on the RISC-V instruction set architecture and applied it to the recognition of the MNIST handwritten dataset.

[0087] First, in terms of hardware, this invention adds an SNN computing unit (SNNU) to the execution stage of the Wenquxing 22A, including a streamlined LIF neuron update unit (NU), a binary synapse update unit (SU), and a spiking unit (SPU). The overall microarchitecture and the principle block diagram of each SNN unit component are shown below. Figure 2 As shown, the dark gray and gray-filled modules are extensions of the Wenquxing 22A.

[0088] For the entire SNN online learning process, according to Figure 6 The process proceeds as follows: First, the pulse sequence enters the pulse processing module. When the corresponding synapse is active, it will be transmitted to the corresponding neuron through the synapse (increasing the number of active pulses to the input neuron during this process). After pulse processing, it enters the neuron update unit to update the current neuron's state (potential). If the potential is greater than the threshold voltage, a pulse will be emitted into the output register, and the current neuron's state will be set to the initial value. If no pulse is emitted, it will be stored back in the neuron storage. Finally, the synaptic weights will be updated in the synaptic update unit according to the input pulse sequence and the neuron's emission status using the BS-STDP learning rule. The update result is then stored back in the synaptic storage.

[0089] The MNIST handwritten digit recognition process utilizes extended instructions during training to accelerate computation.

[0090] Step 1: Obtain images from the MNIST dataset and preprocess them (deskewing, image feature enhancement) to obtain grayscale data for inputting the network; initialize the SNN using the commands SINIT and VLEAK.

[0091] Step 2: Input the image grayscale data into the network input layer and encode it using a Poisson encoder. This process obtains the firing probability of each pixel based on the image grayscale value, then generates a random number and compares it with this firing probability. If the number is less than the firing probability, a pulse is fired. Depending on the training accuracy, an image can be fired multiple times, resulting in a series of pulse sequences that serve as the image information input into the network.

[0092] Step 3: The pulse sequence enters the pulse processing unit ( Figure 3 The system calculates the input voltage of the currently updated neuron based on the synaptic weights corresponding to the currently updated neuron; this process uses the instructions: ANDS, RPOP, NLD. It is important to note that in this invention, neurons are updated during both inference and training, while synapses are only updated during training.

[0093] Step 4: Input voltage enters the streamlined neuron processing unit ( Figure 4 The current neuron state is updated using equation (1). Then, based on the threshold and the updated state, it is determined whether to fire a pulse and whether to reset the voltage. If the updated state is greater than the threshold voltage, the neuron fires and the neuron state is reset to the initial voltage. If it is not greater than the threshold, the neuron will not fire and will maintain the current state. This process uses the following instructions: NLD, NADD, SGE, SLS, NST.

[0094] Step 5: Based on the image labels in the MNIST dataset, use the INF instruction to suppress the learning of neuronal synapses other than those labeled, thus ensuring the correctness of the learning. Specifically, this includes:

[0095] Neuronal spiking is recorded and enters the binary synaptic update unit (BSU). Figure 5 Synaptic updates using binary stochastic STDP ( Figure 1 At this point, long-range enhancement is performed based on the relationship between the pulse sequence input to the neuron and the synapse. If the current neuron is firing (membrane potential greater than the threshold), the input pulse and the synapse read from the synaptic storage are subjected to long-range enhancement, preserving or activating synapses that contribute to neuronal firing. Subsequently, long-range inhibition is performed based on the current synapse weight and the desired precision, calculating the probability of long-range inhibition. The number of activated synapses in the currently processed synapse is counted, and the probability of long-range inhibition is calculated based on the desired weight precision and the following equation:

[0096]

[0097] Δw=W sum -W exp P LTD =0 if Δw=0

[0098] In the above formula, P LTD W refers to the long-range inhibition probability. sum This refers to the count of activated synapses, W. exp This refers to the expected weight precision. The long-range inhibition probability and synapse proceed to the next step. After the long-range inhibition probability is calculated, the long-range inhibition module begins processing the synapse according to this probability. The long-range inhibition module generates a 10-bit random number for processing each activated synapse; when this random number is greater than P... LTD The synapse will be preserved (set to 1) if the synapse is not found; otherwise, it will be suppressed (set to 0). This process ultimately yields an updated synapse, which now possesses the ability to acquire and recognize features from handwritten images, and the corresponding neuron will be sensitive to specific handwritten images. In this step of synapse learning, neurons other than those recognizing labels are suppressed (each neuron learns and becomes sensitive to data corresponding to a label); this process uses the instructions: SUP, RPOP, LTD, INF, NST. The above learning process is as follows: Figure 6 As shown.

[0099] The process of using the trained network to recognize and infer other handwritten images is similar to the training process, the only difference being that the synapse update module is no longer activated to update the synapses. Training is complete after the set number of iterations. The final recognition result shows that specific neurons will be sensitive to specific handwritten images, and their firing rate will be higher within a certain timeframe.

[0100] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0101] Based on the aforementioned chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks, this invention also proposes a handwritten data recognition system based on the dedicated extended instruction set for spiking neural networks, comprising:

[0102] The initial module is used to acquire a handwritten image with a label, set the correspondence between neurons and labels in the spiking neural network, load the leakage voltage into the leakage voltage register of the spiking neural network using a specified leakage voltage instruction, and send the handwritten image into the spiking neural network.

[0103] The pulse generation module is used to encode the handwritten image using a Poisson encoder in the input layer of the spiking neural network. It obtains the firing probability of each pixel in the handwritten image based on the gray value of the pixel, generates a random number and compares it with the firing probability, and fires a pulse to obtain a pulse sequence.

[0104] The weighting module is used by the pulse processing unit of the spiking neural network to add the synaptic weights in the synaptic register and the pulses in the pulse sequence in turn to obtain an effective pulse sequence, which is used as the input voltage for updating the neuron.

[0105] The pulse delivery module is used to input the input voltage into the streamlined neuron processing unit of the spiking neural network, update the current neuron state according to the leakage voltage, and determine whether to deliver a pulse or reset the pulse voltage by comparing the threshold and the updated state.

[0106] The training module is used to perform long-range enhancement on the relationship between input and pulse based on the pulse firing records of neurons, obtain the current synaptic weights, calculate the long-range inhibition probability based on the current synaptic weights and the expected accuracy, and based on the long-range inhibition probability, inhibit the learning of neuronal synapses other than the label and finally obtain the updated synapse.

[0107] The initial module is used to input the pulse sequence of the handwritten image to be recognized into the trained spiking neural network, and select the label corresponding to the firing spiking neuron as the recognition result of the handwritten image to be recognized.

[0108] The handwritten data recognition system based on dedicated extended instructions of spiking neural networks includes a spiking module that updates the current neuron state using the following formula:

[0109] S current =S previous +V valid -V leak

[0110] Among them, S current The current neuron membrane potential, S previous V represents the membrane potential of the current neuron in the previous cycle. valid The effective voltage input to the current neuron, V leak This represents the write-leak voltage of the current neuron; when S current If the threshold is exceeded, the neuron fires a pulse and resets the membrane potential.

[0111] The handwritten data recognition system based on dedicated extended instructions of spiking neural networks, wherein the training module includes updating binary synapses using binary random operations on neuronal spikes:

[0112] The input pulse that stimulates the neuron enters the long-range enhancement (LTP) module. If the current neuron fires, the input pulse is used to perform long-range enhancement with the synapses read from the synaptic storage, preserving or activating the synapses that contribute to the neuron's firing.

[0113] After passing through the LTP module, the currently processed synapses are determined by the number W of activated synapses. sum And expected weight accuracy W exp The long-range inhibition probability P can be obtained using the following formula. LTD :

[0114]

[0115] Δw=W sum -W exp P LTD =0 if Δw=0

[0116] The LTD module is based on P LTD To process synapses, the LTD module generates a 10-bit random number to process the activated synapse. When this random number is greater than P... LTD If the current synapse is updated, it will be set to 1; otherwise, it will be set to 0. After all synapses have been updated, the synapses will be saved to the synapse storage.

[0117] The handwritten data recognition system based on dedicated extended instructions of a spiking neural network includes: performing tilt correction, sharpness enhancement, and grayscale processing on the handwritten image, and inputting the processed handwritten image into the spiking neural network.

Claims

1. A chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks, wherein the instruction fetch unit of the sequential single-issue RISC-V processor fetches instructions, the instructions are decoded by the decoder unit, and after decoding, the instructions are sent to the instruction issue unit. The instructions control multiple general-purpose registers, as well as the arithmetic unit, multiplication unit, and load-to-store unit to perform corresponding functions. Finally, the write-back unit transmits the calculation result and the instruction completion signal back, characterized in that... A dedicated SNN register is added to the issue stage of the sequential single-issue RISC-V processor, and an SNN computation unit is added to the execution stage of the sequential single-issue RISC-V processor. The SNN computation unit includes a streamlined LIF neuron update unit, a binary synapse update unit, and a pulse processing unit. Add a dedicated extended instruction set for spiking neural networks to the decoder unit; This dedicated SNN register is used to store synapses, neuron updates, leakage voltages, initial voltages, and firing results. The dedicated extended instructions of the spiking neural network enter the SNN computation unit after being fetched, decoded, and emitted. The SNN initialization instructions enable training or inference and set the initialization voltage. According to the ANDS and RPOP instructions in the dedicated extended instructions, synaptic and input data are read from memory and processed in the spiking processing unit to obtain the update voltage required by the neurons. According to the neuron-related instructions in the dedicated extended instructions, the neuron state is read from memory and enters the streamlined LIF neuron update unit to update the neurons according to the update voltage. The update result is stored in the dedicated SNN register. Synapses are updated according to the synapse update-related instructions in the dedicated extended instructions. The results of the SNN computation unit are written back through the write-back stage.

2. The chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks as described in claim 1, characterized in that, The neuron is updated using the following formula: S current =S previous +V valid -V leak Among them, S current The current neuron membrane potential, S previous V represents the membrane potential of the current neuron in the previous cycle. valid The effective voltage input to the current neuron, V leak This represents the write-leak voltage of the current neuron; when S current If the threshold is exceeded, the neuron fires a pulse and resets the membrane potential.

3. The chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks as described in claim 1, characterized in that, The process of updating synapses includes: The input pulse that stimulates the neuron enters the long-range enhancement (LTP) module. If the current neuron fires, the input pulse is used to perform long-range enhancement with the synapses read from the synaptic storage, preserving or activating the synapses that contribute to the neuron's firing. After passing through the LTP module, the currently processed synapses are determined by the number W of activated synapses. sum And expected weight accuracy W exp The long-range inhibition probability P can be obtained using the following formula. LTD : Δw=W sum -W exp ,P LTD =0 if Δw=0 The LTD module is based on P LTD To process synapses, the LTD module generates a 10-bit random number to process the activated synapse. When this random number is greater than P... LTD If the current synapse is updated, it will be set to 1; otherwise, it will be set to 0. After all synapses have been updated, the synapses will be saved to the synapse storage.

4. The chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks as described in claim 1, characterized in that, The pulse processing unit reads the synapse from storage into the synapse register, performs an AND operation with the pulse input, and finally counts the number of valid pulses to form the update voltage for entering the neuron update unit. The neuron update unit reads the neuron state into the neuron register, and uses the value in the leak voltage register and the number of valid pulses to calculate the updated state of the neuron. Then it compares it with the threshold voltage. If it is greater than or equal to the threshold voltage, it fires and sets the neuron state to the initial value and stores it back into storage. Otherwise, it will be directly stored back and not issued; the binary synapse update unit will pass the value previously stored in the synapse register to the long-range enhancement unit, then count the number of activated synapses, this number will enter the long-range suppression unit to obtain the long-range suppression probability for long-range suppression, and finally update the result and store it back in the synapse storage.

5. The chip architecture supporting the RISC-V instruction set and the dedicated extended instruction set for spiking neural networks as described in claim 1, characterized in that, The sequential single-issue RISC-V processor is the Wenquxing 22A processor.

Images