Neural morphology calculation system and method based on spiking neural network

Abstract

The invention relates to a neuromorphic computing system and computing method based on a spiking neural network, and the system comprises a neuromorphic processor which is used for constructing a hardware-based spiking neural network through employing a plurality of physical neurons, and carrying out the pulse coding and target recognition of a real-time input image; the central processing unit is realized on the basis of an RISC-V instruction architecture and is used for executing a control instruction required for constructing the pulse neural network; the neuromorphic processor is also used for carrying out online learning on the pulse neural network under the control of the control instruction; and the memory is used for storing program instructions required by the operation of the central processing unit and data transmitted between the central processing unit and the neuromorphic processor. The invention further provides a neuromorphic calculation method. According to the technical scheme of neural morphology calculation, high balance among precision, calculation speed and energy efficiency is achieved, and dynamic and complex task data can be processed accurately and efficiently with low power consumption.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial neural network, and in particular to a neuromorphic computing system and a computing method based on a spiking neural network. Background Art

[0002] By simulating the activity mechanism of brain neurons, combining biological principles and hardware design, neuromorphic processors based on spiking neural networks (SNNs) aim to achieve efficient computing and task execution, especially with online learning and event-driven features. Such processors are widely used in fields such as medical image processing, autonomous driving, intelligent monitoring and intelligent transportation, and are an important research direction in the field of neuromorphic computing. Unlike traditional artificial neural network circuits (such as convolutional neural networks, CNNs), they usually rely on continuous signal processing and fixed training models. These methods usually do not work well for processing dynamic changes and sparse data. In addition, CNN-based hardware design fails to fully reproduce the biological characteristics of the human brain and its dynamic learning ability. Its approximate computing simulation ability is poor, and its complex computing and huge computing capacity are poorly adaptable to environments with limited hardware resources and low energy efficiency, and usually rely on offline training for real-time applications and environments. The lack of online learning means that once the model is deployed in the hardware circuit, its computing power can only be based on the fixed original trained model for reasoning, and cannot adapt to further learning and accurate reasoning of new transactions.

[0003] At present, pulse neuromorphic processors face a series of key problems, such as excessive consumption of hardware resources, low efficiency of neuron calculation, limited online learning methods, lack of flexibility in data transmission between chips, etc. These problems restrict its deployment and popularization in actual scenarios. It also limits the deployment of neuromorphic processors in computer systems. Among these problems, the resource overhead of its internal computing core, that is, the pulse neuron, is greater, and the neuromorphic chip with low computing efficiency has become an important factor restricting its performance improvement. For example, the current-based model makes the behavior of neurons adjustable, but its complex membrane potential update process brings additional computing expenses. Its typical representatives include the digital leaky integrate-and-fire neuron model (abbreviated as LIF) and the Izh neuron model with up to dozens of neuron event states. The LIF neuron model has low computing efficiency because it is not optimized for the hardware system; and although the Izh neuron model can simulate up to dozens of neuron behaviors, the introduction of complex control logic brings a large amount of additional resource consumption. Considering the complexity of neuron computing, existing research has applied approximate computing techniques such as weight quantization and pruning to neuron model design to reduce the difficulty of hardware implementation and improve the approximate simulation performance of artificial neurons, but these methods still face the problem of affecting the recognition accuracy of neuron network models. Secondly, insufficient online learning capabilities or insufficient self-optimization are common problems of existing SNN processors. Considering the computational complexity and cost factors, many existing SNN processors, such as TrueNorth, Tsinghua Tianjic, SpiNNaker, etc., fail to provide online learning functions on-chip, but can only deploy weights offline, which limits their adaptability in changing actual scenarios. Another type of neural form chip with online learning capabilities, such as chips using the spike timing dependent plasticity (STDP) method, can design different learning rules through microcode, but when implemented in hardware, the learning rules of this method are more complex and will occupy more hardware resources.

[0004] The design of neuromorphic computing systems in the prior art usually lacks sufficient flexibility. For computing systems deployed with SNN processors, there are various technical problems, especially when designing time windows, the difference in processing time of each network layer will cause time mismatch problems. This mismatch will result in the need to wait for the calculation of the previous network layer to end after the calculation of a certain layer is completed, thus wasting computing resources. Therefore, how to design accurate artificial physical neurons that are closer to biological characteristics and combine multiple physical neurons to form a pulse neural network to achieve accurate target recognition is the main challenge facing current technology. How to dynamically adapt multi-scale image inputs in artificially designed physical neurons and support the calculation of pulse sequences and large-scale pulse neural networks in different time windows, achieve accuracy close to biological characteristics and save hardware computing resources as much as possible, is the core goal of current intelligent computer systems to achieve efficient and accurate image recognition. Summary of the invention

[0005] Existing neuromorphic processors and computing systems have technical problems such as large hardware resource overhead, low computing efficiency, limited online learning capabilities, and insufficient routing scheduling. In order to solve the problems existing in the above-mentioned prior art, the present invention is inspired by biological models and aims to apply neuromorphic computing technology to real-time intelligent computer systems. An approximate neural computing model is designed to efficiently simulate the complex computing functions of biological neural systems. Specifically, the present invention proposes a neuromorphic computing system that can dynamically update and efficiently process complex tasks and can learn online, so as to achieve a more efficient, more accurate, lighter, and more flexible neuromorphic computing effect, and achieve a high degree of balance between recognition accuracy, response speed, and hardware resource consumption, and finally realize the processing of intelligent image target recognition tasks. The technical solution provided by the present invention is applicable to various computer systems or embedded system technology fields that require artificial intelligence neural network calculations, such as autonomous driving, the Internet of Things, robots, drones, medical image processing, intelligent monitoring systems, and smart transportation, so as to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above object, the present invention provides the following technical solutions: A neuromorphic computing system based on a spiking neural network, comprising: A neuromorphic processor is used to construct a hardware-based spiking neural network using a number of physical neurons, and to perform pulse encoding and target recognition on a real-time input image through the spiking neural network; the physical neurons are constructed using an approximate biological neuron model; the approximate biological neuron model has pre-stored synaptic weights and activation thresholds; A central processing unit implemented based on the RISC-V instruction architecture, communicating with the neuromorphic processor, for executing control instructions required to construct the spiking neural network; The neuromorphic processor is further used to perform online learning on the spiking neural network under the control of the control instruction, and update the synaptic weight and activation threshold associated with each physical neuron of the spiking neural network; A memory for storing program instructions required for the operation of the central processing unit, wherein the program instructions include control instructions required for constructing the spiking neural network; and for storing data transmitted between the central processing unit and the neuromorphic processor.

[0007] Preferably, the central processing unit is a single-issue out-of-order RISC-V processor; the RISC-V processor includes a processor front end, an instruction buffer unit and a processor back end; The processor front end includes a program decoder, an instruction fetch unit and a dynamic branch predictor; The program decoder is used to convert the computer program that controls the operation of the pulse neural network into a RISC-V instruction stream, and transmit the RISC-V instruction stream to the memory for storage; The instruction fetch unit is used to access the memory, read a single RISC-V instruction corresponding to a program counter value of a current clock cycle and transmit it to the instruction buffer unit; the program counter value is used as a read address for accessing the memory; The dynamic branch predictor is used to predict the program counter value of the next clock cycle; The instruction buffer unit is used to buffer the RISC-V instructions read by the instruction fetch unit, and isolate the front-end computing thread from the back-end computing thread of the RISC-V processor; The processor backend is a five-stage pipeline architecture consisting of a decoding unit, an emission unit, a physical register stack, an execution unit and a retirement unit, which is used to read the RISC-V instructions of the instruction buffer unit and perform decoding, emission, reading and writing of the physical register stack, out-of-order calculation, and instruction retirement operations on them in sequence, so as to realize data interaction between the RISC-V processor and the neuromorphic processor.

[0008] Preferably, the approximate biological neuron model includes a charging model and a discharging model, wherein: The charging model is: Baseline decay charging model: ,or, Incremental decay charging model: , The discharge model is: Hard reset discharge model: ,or, Soft reset discharge model: , Among them, V(t) is the membrane potential of the physical neuron at the current time step, V(t-1) is the membrane potential of the physical neuron at the previous time step, and X(t) is the membrane potential increment formed by the pulse signal input at the current time step. is the preset membrane potential attenuation factor; is the reset reference potential of the physical neuron; V(t+) is the membrane potential of the physical neuron after reset at the current time step, V(t-) is the membrane potential of the physical neuron before reset at the current time step; Vth is the activation threshold of the current physical neuron.

[0009] Based on the same inventive concept, further, the present invention also provides an image target recognition method based on a pulse neural network, comprising: writing a computer program; inputting the computer program into any one of the neuromorphic computing systems based on a pulse neural network for execution, and using the pulse neural network to perform target recognition on a real-time input image.

[0010] The beneficial effects of the present invention are: The present invention proposes a structurally optimized neuromorphic computing system and image target recognition method based on a spiking neural network. Combined with the computing characteristics of the spiking neural network, hardware-friendly neuron structure design and high-precision approximate calculation are realized, and efficient online network learning can be realized, thereby obtaining a neuromorphic computing system with low computing resources and power consumption, high recognition accuracy, and high computing efficiency. Based on the neuromorphic computing system, an accurate, efficient, and flexible real-time image target recognition method is provided for various mobile scenarios and edge computing scenarios that have high requirements on timeliness and accuracy but limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 A schematic diagram of the internal structure of a neuromorphic computing system according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a central processing unit implemented based on the RISC-V instruction architecture according to an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a neuromorphic processor capable of online learning according to an embodiment of the present invention; Figure 4 A schematic diagram of a structure of a spiking neural computing core of a hidden layer of a neuromorphic processor according to an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a spiking neural decision core of an output layer of a neuromorphic processor according to an embodiment of the present invention; Figure 6 A preferred structural diagram of a spiking neural computing core of a hidden layer of a neuromorphic processor according to an embodiment of the present invention; Figure 7 An internal structure diagram of a physical neuron according to an embodiment of the present invention; Figure 8 The figure is a schematic diagram of the structure of routers at various levels according to an embodiment of the present invention. DETAILED DESCRIPTION The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] This embodiment provides a neuromorphic computing system based on a spiking neural network, such as Figure 1 As shown, the system includes: a neuromorphic processor 100, a central processing unit 200 implemented based on the RISC-V instruction architecture, and a memory 300. Among them: The neuromorphic processor 100 is used to construct a hardware-based spiking neural network using a plurality of physical neurons, and to perform pulse encoding and target recognition on a real-time input image through the spiking neural network; the physical neurons are constructed using an approximate biological neuron model; the approximate biological neuron model has pre-stored synaptic weights and activation thresholds. In particular, the hardware implementation of the spiking neural network is based on a CMOS digital process.

[0015] The central processing unit 200 implemented based on the RISC-V instruction architecture communicates with the neuromorphic processor 100 to execute the control instructions required to construct the pulse neural network. The neuromorphic processor 100 is also used to perform online learning of the pulse neural network under the control of the control instructions, and to update the synaptic weights and activation thresholds associated with each physical neuron of the pulse neural network. The memory 300 is used to store the program instructions required for the operation of the central processing unit 200, the program instructions including the control instructions required to construct the pulse neural network; and to store data transmitted between the central processing unit 200 and the neuromorphic processor 100.

[0016] For the neuromorphic processor 100, traditional neuron models mostly use two types: Integrate and Fire (IF) neurons and Leaky Integrate and Fire (LIF) neurons. However, the IF neuron model is an idealized integrator model, which is characterized by the fact that the membrane potential of the neuron remains constant without the influence of external input. For the LIF neuron model, in addition to the influence of input current, the effect of leakage conductance is also an important factor to be considered for changes in membrane potential. In addition, for the neuromorphic processor implemented in digital circuit hardware, the input signal of the pulse neural network it relies on is a discrete pulse signal, which makes it difficult to accurately implement the continuous input current signal in the biological model in the digital circuit, and it is impossible to perform differential operations through traditional continuous signal calculation methods.

[0017] In order to make the pulse neural network model adaptable to the implementation of digital circuit hardware, the embodiment of the present invention improves the LIF neuron model, and builds a hardware-friendly pulse neural network based on the improved LIF neuron model to implement the neuromorphic processor 100. The basic charging equation of the original LIF neuron (i.e., the neuron dynamic equation) is:

[0018] Where g is the leak conductance and E is the resting potential of the neuron. This equation reflects the relationship between the input current, membrane potential, and decay process. The differential equation of the LIF neuron charging model (neuron dynamics equation) can be approximated as a recursive equation for discrete-time signals.

[0019] In the embodiment of the present invention, the pulse sequence is serially input according to the overall clock pace of the neuromorphic computing circuit system. At the same time, multiple pulse signals (0 or 1) can be input into the physical neuron in parallel, combined into a multi-bit digital signal, and then transmitted to each pulse core of the hidden layer of the pulse neural network for parallel calculation. Within the accumulated discrete time step ∆t, according to the calculation model of the pulse neural network, the pulse sequence input current I(t) and the increment X(t) of the neuron membrane potential are in a linear relationship. In specific implementation, the membrane potential increment X(t) is expressed as the product of the input current I(t) and the time step ∆t in this calculation process, that is: X(t) = I(t)∙∆t, where its statistical discrete time step ∆t is also called the time window T. In specific implementation, the length of the discrete time window T is affected by the encoding method used when the image is pulse encoded; in the discrete time window, multiple time steps (Time Step) need to be executed to realize the accumulation of membrane potential. The membrane potential V(t) accumulated over time on the physical neuron is determined by a specifically constructed approximate biological neuron model.

[0020] This embodiment improves the LIF neuron model and provides the following optimized approximate biological neuron model, wherein the approximate biological neuron model includes a charging model and a discharging model, wherein: The charging model is: Baseline decay charging model: ,or, Incremental decay charging model: , The discharge model is: Hard reset discharge model: ,or, Soft reset discharge model: , Among them, V(t) is the membrane potential of the physical neuron at the current time step t, V(t-1) is the membrane potential of the physical neuron at the previous time step t-1, and X(t) is the membrane potential increment formed by the pulse signal input at the current time step t. is the preset membrane potential attenuation factor; is the reset reference potential of the physical neuron; V(t+) is the membrane potential of the physical neuron after reset at the current time step t, V(t-) is the membrane potential of the physical neuron before reset at the current time step t; Vth is the activation threshold of the current physical neuron. This embodiment adopts the pulse neural network constructed by the above approximate biological neuron model, which can support multiple types of synaptic connections and simulate neurons of different models, thereby more accurately reflecting the diversity in the biological nervous system. This makes the pulse neural network more flexible in information processing and can achieve higher-level neural computing.

[0021] In the implementation process of the present invention, the charging model is used as the dynamic equation of the physical neuron to accumulate the membrane potential of the physical neuron. The improved LIF neuron model provided in this embodiment takes into account the natural decay characteristics of the membrane potential. When there is no input, the membrane potential will gradually return to the resting potential. This model is closer to the actual behavior of biological neurons, can better simulate the signal processing process of the biological nervous system, and has a certain anti-noise ability, and can effectively deal with unforeseen interference in the environment. When the benchmark decay charging model is used to charge the physical neuron, the input membrane potential increment is Without attenuation, the increment of membrane potential will directly adjust the membrane potential of the neuron. Then according to the preset attenuation factor , to achieve the membrane potential of the previous time step relative to the reset reference potential The decay of When the input membrane potential increment is used The input membrane potential increment when performing the decaying LIF neuron charging model Also according to the preset attenuation factor Participation in attenuation makes the change of neuron membrane potential subject to the dual influence of the increment of input signal and the natural attenuation process. For images with more detailed changes and complex structures, the improved LIF neuron model provided by this embodiment can simulate the dynamic characteristics of biological neurons, such as the natural attenuation and refractory period of membrane potential, which makes it excellent in extracting time-related features in input images, thereby improving sensitivity to subtle differences and improving recognition accuracy.

[0022] In the optimized LIF neuron charging model above, the time constant in the decay factor ( ) can be adjusted according to the needs of the neuron model, biological rationality, etc. Preferably, if the neuron needs to respond to a rapidly changing input pulse signal, a smaller time constant (i.e., a larger attenuation factor) can be selected to accelerate the decay of the membrane potential; otherwise, if the neuron does not need to respond to the input pulse too quickly, in order to slow down the decay of the membrane potential, a larger time constant (i.e., a smaller attenuation factor) should be selected. This adjustment method is modeled after the actual response mechanism of biological neurons. For example, fast-paced neurons in the cerebral cortex usually have a smaller time constant, so that they can respond to the ever-changing input signals; while neurons related to learning and memory usually have a larger time constant in order to better maintain the response characteristics to the input. The choice of time constant is consistent with the behavioral characteristics of actual neurons in biological neuroscience. For resetting the baseline potential Preferably, it is set to a value lower than the resting membrane potential E to ensure that the neuron enters the refractory period after excitation. This setting can simulate the behavior of biological neurons, prevent neurons from being overactivated, and conform to the inhibitory mechanism that is prevalent in biological nervous systems.

[0023] Furthermore, in this embodiment, a central processing unit 200 based on a RISC-V instruction architecture is used to control the construction of a pulse neural network in the neuromorphic processor 100, so as to realize the transmission of the input image to the neuromorphic processor 100 for target recognition. RISC-V is an open source instruction set architecture (ISA) based on the principle of a reduced instruction set, which can realize the operation of various programs by controlling the computer to execute instructions. Due to its simplicity, open source (different from the traditional monopolistic instruction set architectures such as x86 and ARM), modularity, and strong instruction scalability, RISC-V can be easily extended to different application scenarios from microcontrollers to high-performance computer processors. However, the RISC-V instruction set itself is only an instruction set architecture, which provides the basic design principles of a specific RISC-V processor. Its actual hardware implementation needs to be designed according to the actual application scenarios and resource conditions, and instruction expansion is performed when necessary. Therefore, the circuit structure, resource consumption, computing performance, etc. of different RISC-V processors vary greatly.

[0024] Preferably, in this embodiment, the central processor 200 is a single-issue out-of-order RISC-V processor. Figure 2 As shown, the RISC-V processor includes a processor front end 210, an instruction buffer unit 220 and a processor back end 230.

[0025] The processor front end 210 includes a program decoder 211 , an instruction fetch unit 212 , and a dynamic branch predictor 213 .

[0026] The program decoder 211 is used to convert the computer program for controlling the operation of the pulse neural network into a RISC-V instruction stream, and transmit the RISC-V instruction stream to the memory 300 for storage; The instruction fetch unit 212 is used to access the memory 300, read a single RISC-V instruction corresponding to a program counter value (PC) of a current clock cycle and transmit it to the instruction buffer unit 220; the program counter value is used as a read address for accessing the memory 300; The dynamic branch predictor 213 is used to predict the program counter value (PC) of the next clock cycle. In most cases, the value of PC will automatically increase as the instruction is executed, thereby ensuring that the program is executed in sequence. However, the RISC-V instruction set contains unconditional jump instructions JAL (jump link) and J (direct jump) instructions, as well as conditional jump instructions such as BEQ (jump if equal), BNE (jump if not equal), and BLT (jump if less than). Therefore, the RISC-V processor designed in this embodiment can implement the execution of such branch instructions (also called jump instructions or branch jump instructions) through the dynamic branch predictor 213. In specific implementation, since it is impossible to know whether the instruction is a jump instruction before executing the instruction of the current cycle, the dynamic branch predictor 213 needs to perform branch prediction on the PC value corresponding to each instruction to be executed in the instruction fetching stage, and it can be known whether the PC value is predicted successfully in the decoding stage or the execution stage; if successful, the execution of the instruction of the next cycle can be quickly entered, if unsuccessful, it is necessary to flush the pipelines of the processor at all levels to restore the state of the RISC-V processor before the instruction jump. The RISC-V processor provided by the embodiment of the present invention has the functions of fast branch prediction and branch prediction failure recovery.

[0027] The instruction buffer unit 220 is used to buffer the RISC-V instructions read by the instruction fetch unit 212, and isolate the front-end computing thread of the RISC-V processor 200 from the back-end computing thread. In specific implementation, the instruction buffer unit 220 is composed of a sequential first-in-first-out (FIFO) buffer with a depth of d (preferably d=8 or d=16), and the RISC-V processor 200 is divided into a processor front end and a processor back end through the first-in-first-out queue. Among them, the processor front end is responsible for generating program instructions and reading instructions, while the processor back end is responsible for processing the fetched instructions. The FIFO setting with a depth d of 8 or 16 is a compromise solution, which can provide sufficient instruction capacity to cope with the front-end and back-end rate differences and the needs of out-of-order execution, while avoiding excessive increase in hardware resources and complexity. In traditional technical solutions, a total control logic is usually used to control the operation of the entire RISC-V processor. Due to the large number of front-end and back-end states, the control logic is huge and becomes the key path of the kernel. In this embodiment, the instruction buffer unit 220 is used. On the one hand, the front-end / back-end of the processor can be decoupled. The control logic of the decoupled front-end / back-end can be implemented separately, so that the control logic of the front-end / back-end is simplified and a faster working frequency is achieved; on the other hand, the problem of inconsistent calculation rates of the front-end / back-end can be solved. When accessing peripherals, the RISC-V processor may be blocked due to various factors (such as the clock cycle required to access the off-chip memory 300 when the front-end fetches instructions is too long, and the execution of multi-cycle instructions causes the back-end transmission queue to fill up, etc.). The instruction buffer unit 220 can match the problem of inconsistent rates of the front-end and back-end. When the back-end is blocked, the front-end can continue to run, and the throughput of the pipeline is improved. When the back-end of the processor is blocked, if the instruction buffer is not full at this time, the instruction fetching front-end can still fetch instructions normally. The decoupling of the front-end and back-end simplifies the design of the control logic and improves the pipeline throughput.

[0028] The processor back end 230 is a five-stage pipeline architecture consisting of a decoding unit 231, an emission unit 232, a physical register file 233, an execution unit 234 and a retirement unit 235, which is used to read the RISC-V instructions of the instruction buffer unit 220 and perform decoding, emission, reading and writing of physical register files, out-of-order calculations, and instruction retirement operations on them in sequence, so as to realize data interaction between the RISC-V processor 200 and the neuromorphic processor 100.

[0029] In this embodiment, the neuromorphic processor 100 is a neuromorphic processor that can be learned online based on a spiking neural network. The neuromorphic processor 100 includes: a pulse encoder 130, a plurality of spiking neural computing cores 110 for parallel computing, at least one spiking neural decision core 120, and a first-level router 140. Figure 3, which is a schematic diagram of a preferred structure of a neuromorphic processor provided in this embodiment. Specifically, Figure 3 A preferred implementation scheme of a neuromorphic processor 100 based on four pulse neural computing cores and one pulse neural decision core is schematically drawn. Among them, the four pulse neural computing cores 110 implement the pulse neural network hidden layer function in parallel, and the pulse neural decision core 120 is used to implement the pulse neural network output layer function. After the input image passes through the pulse encoder 130, a multi-frame binary image is generated based on a preset pulse encoding method. Each frame of the binary image is converted into a multi-bit pulse sequence and then distributed to the four pulse neural computing cores 110 for parallel computing; the parallel computing results of the four pulse neural computing cores 110 are aggregated to the pulse neural decision core 120 through the first-level router 140 for inference and recognition.

[0030] like Figure 4 and Figure 5 As shown, Figure 4 is a schematic diagram of a specific structure of a pulse neural computing core provided by an embodiment of the present invention; Figure 5 is a specific structural diagram of a pulse neural decision core provided by an embodiment of the present invention. Figure 4 The input terminals (virtual neurons) of the input layer in are only shown as schematic signals and are not included in the pulse neural computing core; similarly, Figure 5 The physical neurons of the hidden layer in are not included in the pulse neural decision core 120, and are only used to illustrate the signal source of the physical neurons of the output layer. Figure 6 1 is a schematic diagram of a preferred structure of a pulse neural computing core provided by an embodiment of the present invention. For the pulse neural decision core 120, the structure of the same components as the pulse neural computing core can be the same, so no further drawing is made for illustration. Specifically, the structure of the pulse neural decision core 120 is Figure 6 Based on the structure of the pulse neural computing core shown, it also includes an output classification decision maker, which is used to judge the recognition result of the output pulse neural network according to the calculation result of the physical neuron.

[0031] Regarding the commonalities between the pulse neural computing core and the pulse neural decision core, such as Figure 6 As shown, each of the pulse neural computing core and the pulse neural decision core is built with a synaptic computing module 111, a neuron computing module 112 and a routing interface 113. The neuron computing module 112 includes a transient data memory 1121, a neuron arbitrator 1122, and a plurality of physical neurons NE0~NE(N-1) cascaded using a pipeline architecture. The synaptic computing module 111 pre-stores the initial synaptic weights obtained by offline training of each physical neuron; the transient data memory 1121 pre-stores the initial activation threshold of each physical neuron.

[0032] The pulse encoder 130 is used to perform pulse encoding on the input image based on a convolutional neural network, generating a series of pulse signals every other time step within a preset time window to form a pulse sequence to be input into the pulse neural network.

[0033] The multiple parallel computing pulse neural computing cores 110 are used to form the hidden layer of the pulse neural network, connect the pulse sequence to each physical neuron in the hidden layer in batches, and control the activation state of each physical neuron according to the approximate biological neuron model.

[0034] The pulse neural decision core 120 is used to form the output layer of the pulse neural network, and distribute the new pulse signals generated by each physical neuron activated in the hidden layer to the physical neurons in the output layer for target prediction.

[0035] The first-level router 140 is used to construct a communication route between the pulse encoder 130 and the pulse neural computing core 110, and distribute the pulse sequence to the multiple pulse neural computing cores 110 in the hidden layer for parallel calculation; it is also used to construct a communication route between the multiple pulse neural computing cores 110 and the pulse neural decision core 120, and pass the calculation results of the hidden layer to the output layer.

[0036] Preferably, the pulse encoder 130 comprises a plurality of convolution filters, a pooler, and a spatiotemporal encoder. The multiple convolution filters are connected in sequence, and are used to perform multiple convolution operations on the input image through a trainable convolution kernel, so as to extract image features at different levels and fuse them into an image feature map; the pooler is used to perform data dimensionality reduction on the image feature map, so as to reduce the spatial information of the image feature map; the spatiotemporal encoder is used to divide the image feature map after dimensionality reduction into multiple local spatial regions, and encode each of the local spatial regions in sequence at a preset time interval, so as to quantize the intensity value of the pixel in the local spatial region into a pulse sequence emitted at a frequency.

[0037] Since the pulse neural network is driven by a pulse sequence, this feature is different from the traditional artificial neural network that directly receives image pixel values. Therefore, the image must be pulse-encoded before the two-dimensional image can be input into the pulse neural network for calculation. In this embodiment, the pulse encoder 130 will rate-encode the input image according to a preset time step (TimeStep) and time window (Time Window), and then a multi-frame binary image pulse sequence matching the length of the time window is output. In specific implementation, the pulse encoder 130 can batch transmit each frame of the binary image pulse sequence of the entire row or column to the corresponding physical neuron of the hidden layer; or, transmit each frame of the binary image pulse sequence to the corresponding physical neuron of the hidden layer at one time, and all pulse signals of the pulse sequence will be used to accumulate the membrane potential of the corresponding physical neuron.

[0038] In specific implementation, the time window is set to T, and each pixel performs encoding operations for multiple time steps on the input image with a probability proportional to the intensity value of its pixel. The pulse sequence generated by each pixel at each time step is obtained respectively, and finally merged into a complete pulse sequence corresponding to the input image.

[0039] The time window generator 114 is used to generate a time step t according to a preset RISC-V instruction. The combination of all time steps forms a time window T. In this time window T, the pulse sequence will be scheduled by the global controller 115 to be transmitted to the input terminal (also known as a virtual neuron) of the input layer at the beat of the time step. The pulse coding length of each pixel in the input image after rate coding can be adjusted by modifying the number of time steps in the time window, and the expression accuracy of each pixel point will change, which will eventually affect the training and reasoning accuracy of the entire pulse neural network. Specifically, the encoded whole frame image can be processed into a multi-frame binary image pulse sequence of a corresponding number of time steps in the above process. Subsequently, the global controller 115, under the control of the instruction from the RISC-V processor, distributes the pulse sequence to the hidden layer of each pulse neural computing core in the form of a whole row, a whole column or a whole frame for calculation. The physical neurons in the hidden layer calculate the real-time changes of the membrane potential of the physical neurons according to the preset neuron charging model, and decide whether to send pulses to the physical neurons in the next layer (output layer) by comparing with the activation threshold. In each time step within the same time window, if no activation event occurs in the neuron, its membrane potential will be temporarily stored after being leaked. These temporarily stored leaked membrane potential data will provide input for the next neuron event together with the membrane potential of the next step. According to the time change within the time window, the neuron membrane potential will be updated in real time with the number and frequency of input pulses. Specifically, when the input pulse causes the neuron membrane potential to reach the neuron activation threshold according to the membrane potential increment X(t) caused by the neuron charging model, the neuron will send a pulse (signal "1"), that is, an activation event occurs. Then the approximate biological neuron model will simulate the behavior of the biological neuron and reset the membrane potential state of the physical neuron, that is, the discharge behavior described in the discharge model will occur. After the discharge, the reset of the physical neuron will be initiated, and two modes can be used: in the hard reset mode, the reset of the neuron membrane potential is directly set to the preset reset reference potential; in the soft reset mode, the membrane potential does not need to be completely reset, but the membrane potential is adjusted by the attenuation factor and the activation threshold of the current membrane potential, and gradually restored to the reset potential.

[0040] The classic network structure of a pulse neural network includes an input layer, at least one hidden layer and an output layer. Among them, the input layer includes multiple input terminals, each terminal is used to connect an input pulse signal. Since the input terminal itself is not a real neuron, it can also be called a virtual neuron. The number of input terminals needs to match the scale of the input image in some way (parallel or serial, etc.) to receive the pulse sequence of the binary image after rate encoding the input image. The input layer receives the pulse sequence of the multi-scale binary image after pulse encoding, and transmits the pulse sequence in batches and in parallel to each physical neuron involved in the hidden layer for calculation according to its charge / discharge model. Preferably, the hidden layer includes four parallel processing pulse neural computing cores, and the output layer includes at least one pulse neural decision core. The pulse neural computing core and the pulse neural decision core contain multiple physical neurons, which are constructed by the aforementioned neuron model. The structure of the pulse neural computing core of each hidden layer is the same, and it can be specifically adopted as follows Figure 6 The structure shown in the figure is implemented; the structural design principles of the pulse neural computing core and the pulse neural decision core are roughly the same, but the structure of the pulse neural decision core is Figure 6 On the basis of the pulse neural computing core, it is necessary to add an output classification decision maker (such as Figure 5 In specific implementation, the number and function of physical neurons actually involved in neural network calculations will change due to the input signals and arrangement of the network layers.

[0041] by Figure 6 The pulse neural computing core shown in the figure is used to introduce the core working principles of each pulse neural computing core and the pulse neural decision core. The synaptic weight data storage in the synaptic computing module 111 pre-stores the synaptic weight W obtained through pre-training, and the synaptic weight W may be updated in the future online learning process. The neuron computing module pre-stores the neuron activation threshold Vth obtained through pre-training. Similarly, the neuron activation threshold Vth can be learned and updated. The neuron computing module is equipped with multiple physical neurons (NE), which transmit the received binary image pulse sequence in sequence according to the pipeline structure and calculate according to the approximate neuron model. In specific implementation, the number of physical neurons (NE) in the pulse neural computing core and the pulse neural decision core should be set according to the needs of the actual recognition task.

[0042] In particular, in the task of recognizing handwritten digits (0 to 9), the number of physical neurons NE in a spiking neural computing core can be set to 20 or other numbers, corresponding to some or all neurons in the hidden layer of the spiking neural network; and the number of physical neurons NE in the spiking neural decision core is consistent with the number of physical neurons required in the output layer, and also consistent with the number of target categories to be recognized. For example, for the handwritten digits 0 to 9 in the MNIST dataset, the number of physical neurons NE in the spiking neural decision core should be set to 10.

[0043] During the inference process of the neuromorphic processor 100 using the spiking neural network model, the spiking neural computing core and the spiking neural decision core perform statistics on the approximate calculation results of multiple time steps according to the pre-stored or updated synaptic weights and activation thresholds after online learning, thereby generating the recognition result of the multi-scale input image.

[0044] In the online learning process of the neuromorphic processor 100 using the spiking neural network model, multiple images can be input into the spiking neural computing core and the spiking neural decision core, and the synaptic weights and neuron activation thresholds can be updated using a specific online learning method. Different online learning mechanisms or architectures will determine the performance of the neuromorphic processor 100 on different recognition tasks.

[0045] The learning method widely used in the prior art to implement the training of spiking neural network models is the Spike Timing Dependent Plasticity (STDP) mechanism. However, the STDP learning mechanism still has two major defects: on the one hand, STDP relies on precise pulse time differences to adjust synaptic weights, which is difficult to achieve precise control when the circuit is implemented; on the other hand, it is necessary to perform exponential and logarithmic operations to obtain the various values ​​required to update the weights. These mathematical operations are difficult to implement accurately in digital circuits, and often require truncation and quantization of values. At the same time, their approximate implementation requires huge hardware computing resources and energy consumption. For example, a large amount of storage space is required to save the pulse information of each neuron at each time step, which is undoubtedly a considerable challenge for edge devices with limited resources and costs. In addition, the present invention has observed in a large number of experiments that for a trained spiking neural network model, when the spiking neural network model is trained for a new task, the knowledge of the previous old tasks cannot be guaranteed. That is, while the recognition accuracy of the new task is improved, the recognition accuracy of the old task may be reduced. The reason for this phenomenon is that in the case of learning these two similar tasks, a large number of identical neurons are activated. That is, when performing these two tasks, the synaptic connection behaviors between neurons in the network are very similar, and the network model has a technical problem of catastrophic forgetting. Furthermore, the present invention has found in a large number of studies that the connection strength (weight) of neuronal synapses is affected by the timing of the pulse emission of the pre-neuron (pre) and post-neuron (post) to which it is connected. Specifically, when the pre-neuron (pre) emits a pulse before the post-neuron (post), the synaptic weight is enhanced; and when the pre-neuron (pre) emits a pulse after the post-neuron (post), the synaptic weight is weakened. Therefore, in an embodiment of the present invention, in order to reduce the computational complexity of online training of a hardware-based pulse neural network model, rationally allocate the consumption of hardware resources and overcome the problem of catastrophic forgetting during model training, an embodiment of the present invention proposes a segmented online learning method, which updates the synaptic weights and activation thresholds of the current physical neurons by judging the strength of the correlation between the previous and next neurons.

[0046] Specifically, each of the spiking neural computing core and the spiking neural decision core further includes an online learning unit 116. The online learning unit 116 includes: a forgetting deactivator 1161, a freezer 1162, a random learner 1163 and a parameter updater 1164.

[0047] The forgetting deactivator 1161 is used to randomly deactivate the physical neurons involved in the hidden layer based on the random forgetting learning method.

[0048] The freezer 1162 is used to freeze the synaptic weights and activation thresholds of the physical neurons involved in the output layer.

[0049] The random learner 1163 is used to train the physical neurons that are not inactivated in the hidden layer, and is also used to train the physical neurons that are unfrozen in the output layer; The parameter updater 1164 is used to update the synaptic weights and activation thresholds of the physical neurons after being trained by the random learner 1163. Specifically: If the difference between the membrane potential of the physical neuron at the current time step and the activation threshold at the current time step (|Wcs|) is greater than a preset pseudo-random number (P), the following equations are used to update the synaptic weight and activation threshold of the current physical neuron respectively:

[0050]

[0051] Where, represents the synaptic weight at the next time step, represents the synaptic weight at the current time step, represents the neuron activation threshold at the next time step, represents the neuron activation threshold at the current time step, represents the time difference between the pre-neuron and the post-neuron firing pulses. a and b represent two different synaptic weight update amplitudes, and 0 < a < b; c represents the update amplitude of the neuron activation threshold, and c > 0.

[0052] Otherwise (i.e., |Wcs| ≤ P), the synaptic weight and neuron activation threshold of the physical neuron at the next time step remain unchanged. That is, when |Wcs| ≤ P, the physical neuron skips the online learning process, and the synaptic weight and activation threshold at the next time step remain unchanged, i.e., W(t) = W(t - 1), Vth(t) = Vth(t - 1). Specifically in implementation, the online learning unit 116 also has a historical data buffer 1165 for temporarily storing the synaptic weights and activation thresholds of the previous time step involved in the physical neurons that need to be online trained.

[0053] In the segmented online learning process provided by the present invention, in the first stage, the forgetting deactivator 1161 is used to randomly deactivate the physical neurons involved in the hidden layer, and the random learner 1163 and the parameter updater 1164 are used to adopt a random plasticity method based on historical weights to realize the online real-time update of the synaptic weights and neuron activation thresholds connected to each physical neuron in the hidden layer; in the second stage, the freezer 1162 is used to freeze the synaptic weights and activation thresholds of the physical neurons involved in the output layer, and some physical neurons or neuron parameters of the output layer are selected to thaw, and then the random learner 1163 and the parameter updater 1164 are used to adopt a random plasticity method based on historical weights to realize the training of the output layer, and the synaptic weights and neuron activation thresholds connected to the corresponding physical neurons are updated online in real time.

[0054] Specifically, in the first stage of online learning, the forget deactivator 1161 can randomly forget the connection between some neurons in the hidden layer and the output layer neurons by randomly setting the output pulses of some hidden layer neurons to "0". Based on the premise of hardware friendliness, the forget deactivator 1161 can be implemented by using the random deactivation pulse neuron method or the random deactivation synaptic connection method. Among them, the random deactivation pulse neuron method, after calculating the membrane potential of the current physical neuron, sets the membrane potential of the physical neuron to "0" (that is, the output pulse is set to "0") with a preset probability; the random deactivation synaptic connection method, between the two layers of physical neurons, randomly deactivates the synapses of the forgotten neurons, and only accumulates the activated synaptic inputs when calculating the membrane potential of the physical element. In the specific hardware circuit design, the random deactivation pulse neuron method can be implemented by using the method of randomly shutting down the enable end of the physical neuron; the random deactivation synaptic connection method can be implemented by randomly shutting down the read enable end of the synaptic weight, without too much additional circuit logic, which meets the characteristics of hardware friendliness. The embodiments of the present invention dynamically adjust the enabled state of neurons or synaptic weights, while maintaining the original task memory, and randomly build new synaptic connection paths for new tasks through synaptic connections between certain neurons, thereby giving the network higher adaptability, enabling it to efficiently learn new tasks under limited hardware resources while avoiding excessive interference with the learned content, thereby achieving the ability of continuous learning and improving the adaptability and versatility of the neural network to dynamic changes in the environment.

[0055] In the online learning process of the second stage, the physical neurons of the output layer are trained by the frozen learning method. Since the output layer no longer has synaptic connections to the next network layer, the above-mentioned forgetting learning is no longer applicable to the online learning of the physical neurons of the output layer. In order to overcome the catastrophic forgetting of the physical neurons of the output layer, the present invention adopts a parameter freezing learning method. For the pulse neural network that has been trained in the old task, all parameters of some physical neurons of the output layer are frozen when a new task arrives, or some parameters of all physical neurons of the output layer (such as synaptic weights) are frozen and only other trainable parameters (such as activation thresholds) are fine-tuned. This freezing learning method limits the interference of the new task on the synaptic weights of the original task, thereby reducing catastrophic forgetting. The synaptic weights and activation thresholds between the neurons with frozen parameters will not be updated, and only the synaptic connections of the neurons related to the task will be fine-tuned. This non-global update method can avoid the situation where catastrophic forgetting is caused by large-scale adjustment of weights in a well-trained neural network, and is very friendly to hardware implementation. When learning new tasks, the overhead of computing core storage is significantly reduced.

[0056] In the above two stages of model training, this embodiment adopts a random plasticity learning method based on historical information to train the corresponding physical neurons. The update of the synaptic weights and activation thresholds of the physical neurons is mainly realized by the random learner 1163 and the parameter updater 1164. The working principle of the random plasticity learning method based on historical information is: if the relative difference (|W) between the membrane potential and the activation threshold of the physical neuron at the current time step (t-1) cs |) is greater than the preset pseudo-random number (P), the synaptic weight and activation threshold of the current physical neuron at the next time step (t) are updated according to the aforementioned mathematical model; otherwise, the synaptic weight and activation threshold of the current physical neuron at the next time step (t) remain at the current value. This method can significantly reduce the computational complexity, improve the accuracy of the hardware-based neuromorphic processor during training and inference, and make the use of hardware resources more efficient. This random learning method that requires information from previous time steps is referred to as a random plasticity learning method based on historical information in an embodiment of the present invention.

[0057] The neuromorphic processor 100 with online learning capability can achieve more application scenario-specific target recognition, and implement model training and parameter adjustment through real-time input images, so that the neuromorphic processor 100 can still maintain a high recognition accuracy when facing different recognition tasks.

[0058] The neuron arbitrator 1122 in the neuron computing module 112 is used to transmit the relevant network parameters of the most active physical neurons to the online learning unit 116 and the synaptic computing module 111 for learning and updating. This decision-making process will determine the neuron target that will eventually participate in online learning, taking into account not only the factors of the current neuron such as membrane potential and synaptic weight, but also its contribution to the overall performance of the network. Once the neuron arbitrator 1122 determines the neurons that need to be updated, it will send an update signal to the online learning unit 116 and the synaptic computing module 111 in order to trigger the learning operation of specific weights and activation thresholds. Among them, the most active physical neurons are obtained through a competitive learning mechanism. The more active the neurons, the more likely they are to be activated, so that the synaptic weights and activation thresholds are updated, which meets the basic characteristics of biological learning.

[0059] The random learner 1163, based on the linear feedback shift register (LFSR), generates a pseudo-random number P as a reference value for whether to perform random learning. The LFSR includes a set of cascaded registers, the value in each register is shifted according to each clock beat, and a new input bit is generated based on the feedback signal output by some registers. Based on this principle, the random learner 1163 uses the LFSR to generate a cyclic pseudo-random sequence through the combined action of the shift register and feedback logic. The random learner 1163 compares the relative difference W of the neurons mentioned above. cs and pseudo-random number P, if W cs If it is greater than P, the next step is to control the parameter updater 1164 to start the online learning logic, otherwise the learning process of this time step will be skipped.

[0060] The parameter updater 1164 is used to update the synaptic weights and activation thresholds of the most active physical neurons in the historical time step based on historical information and using the random plasticity online learning (WSC-SDSP) strategy. The parameter updater 1164 is provided with a learning intensity regulator, an activation threshold learning module for updating the neuron model parameters using historical information, and a synaptic weight learning module.

[0061] Taking the working principle of a single spiking neural computing core in the hidden layer as an example, this embodiment introduces the online learning process of physical neurons. During the online learning process of a spiking neural network, the spike trains of each frame of binary images after spike coding are input into the neuromorphic processor 100 in a predetermined order for training, and at the same time, the number of training rounds is determined. Each time step triggers a training process. The global controller 115 is responsible for controlling the operation of the entire online learning process during training, coordinating the work of each sub-module, receiving the time step information from the time window generator 114 and the feedback from other modules, and ensuring the correct timing and function execution of the training process. When the online learning function is executed, the neuron computing module 112 will send the historical synaptic weights and historical input spikes that have participated in the accumulation within the same time window into the online learning unit 116 to participate in the learning update process, receive the updated neuron activation threshold, and control the output of the updated synaptic weights to the synaptic weight memory 1111 of the synaptic computing module 111 for storage. The transient data memory 1121 in the neuron computing module 112 is used to store parameters such as the membrane potential, activation threshold, and activation state of physical neurons.

[0062] Specifically, according to the modeling equation of the foregoing parameter updater 1164, the neuron online learning process provided by the embodiments of the present invention includes: in the first piecewise function, and are linearly related in each segment, and their change relationship is affected by two parameter values a and b, which are empirical values obtained through multiple sufficient experiments. Preferably, 0 < a < b, and the historical weight is selected as the synaptic weight value one time step away . Compared with in the conventional STDP online learning method, the time difference between the spikes emitted by the pre-neuron and the post-neuron adopted in the embodiments of the present invention only needs to be used to indicate the order of spike release of the pre-neuron and the post-neuron. Specifically,

[0063] greater than 0 indicates that the post-neuron releases spikes before the pre-neuron; on the contrary, it indicates that the pre-neuron releases spikes before the post-neuron. Therefore, in this embodiment, there is no need to store a large amount of historical spike release information, and only the spike release order of the pre-neuron and the post-neuron at the current time step needs to be obtained. At the same time, the embodiments of the present invention can reflect the influence of different historical weights on the synaptic weight adjustment amplitude by reasonably setting the values of a and b. In the embodiments of the present invention, particularly, a can be set to 1 and b can be set to 2 to represent different synaptic weight update amplitudes.

[0063] Furthermore, in this embodiment, an amplitude update table of the change amplitude c of the activation threshold can be obtained through experiments, and the activation threshold can be dynamically adjusted according to the spike emission situation of historical physical neurons at each time step. Specifically, if , which means that the synaptic weight is enhanced, the activation threshold is increased; if in the next time window If it always exists, the change amplitude c of the activation threshold will increase according to the preset value of the amplitude update table. , which means that the synaptic weight decreases, then the activation threshold is reduced; and if in the next time window If the activation threshold value is always present, the change amplitude c of the activation threshold value will decrease according to the preset value in the amplitude update table. In particular, a large number of experiments show that the change amplitude c can be set to 1, 2 and 3 in the amplitude update table in stages. Note that the change amplitude c will be affected by Initialized based on the changes in

[0064] In biological neural systems, the update of synaptic weights is a non-deterministic process, which shows significant randomness and probability (such as neuronal activity, voltage changes, neurotransmitter release, etc.), and is affected by various factors. By introducing a probabilistic mechanism to control the update of synaptic weights, the characteristics of biological learning can be simulated more realistically. In addition, this method helps to prevent the problem of overfitting of neural network models to a certain type of data due to frequent updates. Overfitting usually makes the network perform well on training data, but lacks good generalization ability in practical applications. The update of each neuron and the update of synapses are completed independently in the neural network. By setting the update probability, more diverse synaptic connections can be formed, thereby enhancing the ability to predict unknown data and facilitating a more comprehensive adaptation to the input space.

[0065] It should be noted that although the structures of each spiking neural computing core and spiking neural decision core are the same or similar, the circuit structure of each physical neuron stores synaptic weights and its own activation threshold. These model parameters will be dynamically updated during the training process according to the input pulses, resulting in calculation differences between each spiking neural computing core and spiking neural decision core. In other words, the membrane potential and activation state of each physical neuron have different positions in the spiking neural network.

[0066] In a preferred implementation, the embodiment of the present invention may use a digital integrated circuit (IC) chip to implement the pulse neural computing core 110 and the pulse neural decision core 120 to achieve high-speed and low-power neuron activation and reset.

[0067] Furthermore, if Figure 7 As shown, each physical neuron (NE for short) includes: a PE approximate calculation array, an approximate addition tree and a pulse emission decision unit.

[0068] Specifically, the PE approximate calculation array is composed of multiple PE (Processing Element) units using a parallel pipeline computing architecture, which is used to access the pulse sequence in batches, and issue each pulse signal of each batch in parallel to the corresponding PE unit for pipeline computing; each PE unit includes an accumulation control logic and at least one first approximate adder; each PE unit is used to accumulate the local pulse sequences of each batch under the control of the accumulation control logic, using the first approximate adder to obtain the local membrane potential increment within a preset time window; and according to the charging model, the local membrane potential of the physical neuron in the current PE unit is calculated using the local membrane potential increment; the approximate adder The method tree is composed of multiple second approximate adders hierarchically constructed with a binary tree structure, which is used to superimpose the local membrane potential output by each PE unit to obtain the complete membrane potential accumulated by the pulse sequence on the current physical neuron; the pulse emission decision unit is used to compare the complete membrane potential with the activation threshold of the current physical neuron to determine whether the current physical neuron is activated; if the current physical neuron is not activated, a new pulse signal is connected and the membrane potential of the current physical neuron is charged according to the charging model; if the current physical neuron is activated, a new pulse is emitted, and after the new pulse is emitted, the membrane potential of the current physical neuron is reset according to the discharge model.

[0069] In a neuron time step, the PE array calculates the local current of some input pulses, and the approximate addition tree accumulates the local current. The accumulated membrane potential obtained determines the behavior of the current physical neuron at this time step: when the accumulated membrane potential of the physical neuron exceeds the activation threshold, a pulse signal is issued and the membrane potential is reset; otherwise, the membrane potential decays according to the neuron model and is stored in the next time step. It can be seen that in this embodiment, the training process of the pulse neural network includes all modules and calculation steps in the reasoning process, that is, the training subsystem calls each hardware module of the reasoning subsystem to update the parameters such as the membrane potential, synaptic weight and activation threshold of the neuron, thereby optimizing the performance of the pulse neural network. The PE approximate calculation array accesses the local pulse sequence of the binary image in batches at each time step, and uses the local pulse sequence to screen the input synaptic weight sequence. If the pulse in the sequence is "1", the synaptic weight of the corresponding address is selected. After the corresponding synaptic weight is selected for each pulse, the synaptic weight is issued to the approximate addition tree for summation and temporary storage. By time-division multiplexing the PE approximate calculation array, the synaptic weights of the addresses corresponding to multiple local pulse sequences (rows or columns) of the binary image can be sent to the PE array in batches for selection according to the input image scale. Each PE uses the local pulse sequence input in each batch of time steps according to the preset neuron model to obtain the local current of each batch of time steps, which is used to calculate the local membrane potential increment of the physical neuron in the current time step. In each time step, the local pulse sequence will be transmitted to each cascade of physical neurons NE along the pipelines at each level for separate calculation. The approximate addition tree is used to pipeline the local current output by the PE approximate calculation array to calculate the complete membrane potential of the physical neuron corresponding to the connected binary image pulse sequence.

[0070] like Figure 7As shown, the pulse emission decision unit is a configurable hardware architecture, which has a built-in third approximate adder required for the neuron charging model and discharge model, as well as a logic circuit containing the membrane potential and pulse sequence output by the entire physical neuron. Among them, the third approximate adder includes the respective adders required for the calculation of the charging model and discharge model of the optimized approximate neuron mentioned above. Using the LIF neuron model optimized based on approximate calculation, the pulse emission decision unit calculates the active value of the current physical neuron, which is used to simulate the membrane potential change of the biological neuron and participates in data preparation in the online learning stage. Specifically, the pulse emission decision unit is used to compare the membrane potential obtained by summarizing the approximate addition tree with the neuron activation threshold to determine whether the current physical neuron needs to be activated; if the current physical neuron is not activated, its membrane potential is attenuated according to the preset neuron charging model; if the current physical neuron is activated, a new pulse will be issued to the next layer of neurons, and after the pulse is issued, the membrane potential of the current neuron will be reset according to the preset neuron discharge model. In addition, the pulse emission decision unit is also used to select the corresponding neuron model according to the preset conditions, and calculate the active value of the physical neuron, which will participate in the functional logic of online learning. The synaptic calculation module 111 stores the synaptic weights of the synapses connected to each physical neuron through the synaptic weight memory 1111, and transmits the synaptic weights corresponding to each physical neuron to the neuron calculation module through the synaptic arbiter 1112 to participate in reasoning or training. In each preset time window, the PE approximate calculation array and the approximate addition tree work together to calculate the complete membrane potential of the current physical neuron, and transmit the membrane potential value to the configurable pulse emission decision unit at each time step. The configurable pulse emission decision unit compares the preset neuron model (optimized LIF model) with the neuron activation threshold to determine whether the current physical neuron emits a pulse, and uses the synaptic arbiter 1112 to orderly store the neuron membrane potential after reset or leakage.

[0071] In this embodiment, multiple physical neurons (NEs) built into the neuron computing module form a pipeline parallel computing structure, and the pulse sequence of the connected binary image is transmitted to each NE along with the clock of the computing system. Time division multiplexing technology is applied inside each physical neuron, and multiple pulse signals corresponding to the input are distributed to each PE for parallel computing in each clock cycle, so as to realize the accumulation of the local membrane potential of the current physical neuron according to the approximate neuron model using the approximate addition tree. Figure 7As shown in the figure, the one-dimensional PE array is composed of multiple PEs. The array circuit correctly implements the membrane potential accumulation function through time division multiplexing, which significantly reduces the circuit area and power consumption. In order to further improve the computing efficiency, the pulse sequence of the input image is sent to the physical neuron in batches according to the circuit clock as the time step (T) for parallel superposition. Specifically, the pulse sequence of each row or column (i.e., M or N pixels) of the input image will be transmitted to the PE array of the physical neuron through each input. The PE array is composed of M PE units in parallel. By multiplexing N times the partial synaptic weights W0,0~W32,T, the M×N pulse signals of the input image can be fully transmitted to the physical neuron for membrane potential calculation. Through reasonable instruction control, input images of different sizes can be processed under limited hardware resources.

[0072] In the pulse neural computing core and the pulse neural decision core, the adder is the core component for realizing PE accumulation, approximate addition tree and configurable pulse emission decision unit in physical neurons. In order to obtain a hardware-friendly low-power, high-speed neuron circuit and better simulate the operation mechanism of biological neurons, in a preferred implementation method, the adders used in this embodiment are all speculative carry adders with error correction mechanism. Therefore, the above-mentioned first approximate adder and the second approximate adder are both speculative carry adders with error correction mechanism. Specifically, for two input signals A and B to be added with a bit width of n, signal A and signal B can be divided into m segments, and the bit width of each sub-signal is k bits, that is, n=m·k.

[0073] The speculative carry adder includes m k-bit sub-adders, which are used to accumulate the corresponding m-segment k-bit input sub-signals, wherein the m-segment k-bit input sub-signals are divided by the input signal with a total bit width of n=m·k; Each of the sub-adders includes a speculative carry generator; the speculative carry generator is used to infer the approximate carry signal of the corresponding sub-adder according to the following equation:

[0074] Where i represents the number of the sub-adder and 2≤i≤m, represents the approximate carry signal of the ith sub-adder; the approximate carry signal of the first sub-adder is =0; k represents the bit width of each sub-adder, represents the carry flag signal of the k-1th bit generated by the i-th sub-adder, represents the carry flag signal of the k-1th bit generated by the i-1th sub-adder, The carry propagation signal representing the j-th bit of the output of the i-th sub-adder; A multiplexer is also provided between each of the sub-adders, for selecting the carry identification signal of the previous sub-adder or the approximate carry signal of the previous sub-adder as the carry input signal of the current sub-adder according to the state value of the carry propagation signal of the current sub-adder.

[0075] The extension of the critical path of the adder will directly increase the computing time of the entire processor, thereby reducing computing efficiency. Therefore, an embodiment of the present invention proposes an approximate adder that uses carry speculation technology to shorten the carry propagation path to reduce the carry chain length.

[0076] When the sub-adder speculates that the carry signal is "0" but the actual carry signal is "1", this error will cause a 2i-level accumulation error, which will cause an irreversible and serious impact on the entire neural network reasoning or training process, causing the entire computing system to crash. In this embodiment, in order to prevent the above errors from occurring, an error correction mechanism is introduced: A "two-choose-one" multiplexer is inserted between the sub-adder blocks. Among them, the carry flag signal of the previous sub-adder and the approximate carry signal generated by the previous sub-adder As the input signal of the multiplexer, the carry propagation signal of the current sub-adder is used as the selection signal of the multiplexer. In this embodiment, no additional circuit needs to be designed to generate the carry propagation signal, because it has been generated in the carry generator of each sub-adder. In this embodiment, the function of the error correction circuit is: if the carry propagation signal is "1", the output selection of the multiplexer is , otherwise select , the output signal is used as the carry input signal of the current sub-adder. Through this improvement, although the area of ​​the adder is slightly increased, the cumulative error caused by the carry error can be significantly reduced, thereby enhancing the reliability of the circuit.

[0077] In addition, it cannot be ignored that in the signed number addition operation, the error of the sign bit will cause a great calculation deviation. Although the approximate adder allows a certain range of precision loss, the error of the sign bit is an unacceptable error type. To solve this error, this embodiment introduces a second error correction mechanism: by calculating the carry propagation signal and The result of logical "AND" is used as the symbol error correction signal . When any level is detected If the signal is "1", the sign bit error correction mechanism is triggered, and the carry propagation signal is used to determine whether to select the inferred approximate carry signal. As the final carry signal. This error correction mechanism can correct the sign bit to avoid irreversible large deviations or major calculation errors. The maximum relative error of the approximate adder in this embodiment is When k is greater than or equal to 3, due to the strong robustness of the spiking neural network to approximate calculations, this error has almost no effect on the pulse distribution of the neuron.

[0078] In summary, the speculative carry adder with error correction mechanism provided in this embodiment effectively reduces the delay of the critical path by optimizing the critical path delay, and significantly improves the maximum operating frequency of the processor. In addition, the dual error correction mechanism designed in this embodiment helps to ensure the accuracy of addition during the accumulation calculation of membrane potential in the spiking neural network, avoids the failure of the spiking neural network due to approximate calculation, and ensures the reliability of the neuromorphic processor 100. The use of approximate calculation technology in the entire neuromorphic processor 100 can effectively reduce the operating power consumption of the entire processor.

[0079] In addition, this embodiment further reduces unnecessary computational overhead according to the characteristics of the pulse sequence after the input image is pulse-coded. Furthermore, the neuromorphic processor 100 further includes: a pulse density discriminator.

[0080] The pulse density discriminator is used to discriminate the pulse sequence as a dense pulse sequence or a sparse pulse sequence according to the pulse density of the pulse sequence.

[0081] The PE approximate calculation array is also used to calculate the complete membrane potential of the pulse sequence on the current physical neuron using the approximate addition tree when the pulse sequence is a dense pulse sequence; and to calculate the complete membrane potential of the pulse sequence on the current physical neuron using a logic OR gate circuit when the pulse sequence is a sparse pulse sequence. Experimental data show that the resource consumption of the logic "OR" gate circuit is much less than that of the approximate adder, and the calculation efficiency of the logic gate is much higher than that of the approximate adder.

[0082] After a large number of experiments, the embodiments of the present invention found that since the pulse encoder is derived from multiple encodings of the same input image at different times (time steps), the corresponding positions of the obtained multi-frame binary input images often have many repeated pulse sequences. Specifically, when the pulse neural network is started for image recognition, each pulse neural computing core receives input pulse sequences of different time steps of the same input image. Since they come from the same image, the synaptic weights are exactly the same (without online learning), and the input pulse sequences are also partially repeated in different pulse neural computing cores. Therefore, in the reasoning process, part of the neuron membrane potential and all synaptic weights can be reused. Preferably, as Figure 7As shown, the complete pulse sequence of a frame input image can be decomposed into a non-multiplexed partial sequence and a reusable partial sequence. The synaptic weights are reused in all four pulse neural computing cores of the hidden layer, and in particular, the complete input pulse sequence is input into one of the pulse neural computing cores, while the non-reusable partial sequence is input into other pulse neural computing cores. After the first pulse neural computing core calculates the complete neuron membrane potential, the neuron membrane potential corresponding to the reusable partial sequence is separated from it, and added to the membrane potential of the non-reusable partial sequence in each computing core, so as to obtain the complete neuron membrane potential of multiple frames of binary input images. Then, the complete neuron membrane potential corresponding to the four cores is transmitted to the primary routing, and the neuron calculation results of the hidden layer are transmitted to the output layer for reasoning and decision making. The advantage of sparse activation of the physical neurons of the hidden layer in this way is that it can further reduce the redundant calculation overhead of the repeated part in the reasoning process, and achieve the design purpose of low power consumption. This embodiment utilizes the sparse characteristics of the pulse sequence, reduces unnecessary calculation overhead, improves learning efficiency and effect, and reduces energy consumption, which is of great help to the stability and accuracy of the model.

[0083] In addition, the communication system of the neuromorphic processor 100 may further include a secondary router 150, which is used to construct a large-scale spiking neural network by multiplexing multiple spiking neural computing cores and spiking neural decision cores to achieve target recognition of big data. The primary router 140 and the secondary router 150 are hierarchical routing architectures, which can realize data communication routing control of the reasoning process and training process of the large-scale neuromorphic computing system, as well as routing control of multi-level data transmission between each spiking neural computing core and spiking neural decision core.

[0084] The internal core structure and working principle of the primary router 140 and the secondary router 150 are basically the same, but the data they process and the directions of data transmission are different due to the application of data communications at different levels. Figure 8 The internal core components and working principles of the first-level router 140 and the second-level router 150 are shown. Specifically, the router includes at least four stages: input cache, path calculation and channel switching, virtual channel allocation and transmission. The first stage is the input cache, which is used to distribute data packets accessed from all directions to the caches of multiple channels inside it, and then extract specific information segments (filts) and distribute them to the corresponding channels or paths according to the encapsulation information carried by the data packets. Since the information segments (filts) may need to be distributed to multiple pulse neural computing cores or multiple overall approximate neuromorphic processors for calculation, the router needs to have a corresponding path calculation and channel switching mechanism (second stage). The third and fourth stages are mainly used to realize data interaction and external transmission of information segments.

[0085] The internal routing interface 113 of each pulse neural computing core and pulse neural decision core on the neuromorphic processor transmits pulse data packets with the first-level router 140, thereby forming an on-chip routing system with a star topology. The first-level router 140 communicates with the second-level router 150 to realize data interaction between the current neuromorphic processor and the external neuromorphic processor. In the routing mechanism of the first-level router 140, when the pulse data packet is received, it is decoded into the corresponding input signal and transmitted to the pulse computing core and the pulse decision core for processing; after the processing is completed, the output signal will be re-encoded into a pulse data packet, and data communication between different pulse cores will be carried out through the first-level router 140. The routing mechanism of the second-level router 150 is similar and will not be repeated.

[0086] In summary, the embodiments of the present invention successfully implement a neuromorphic computing system with significantly improved computing speed and recognition accuracy on the basis of ensuring efficient network transmission.

[0087] Further, based on the same inventive concept as the above-mentioned neuromorphic computing system based on a spiking neural network, an embodiment of the present invention also provides a neuromorphic computing method based on a spiking neural network. The method includes: Write a computer program; input the computer program into the neuromorphic computing system based on the spiking neural network for execution, and realize real-time target recognition of the input image. Since the main technical innovation of the image target recognition method is consistent with the neuromorphic computing system, the steps of the image target recognition method will not be described here.

[0088] Specifically, this embodiment further implements the neuromorphic computing system based on the spiking neural network based on the 40-nanometer CMOS semiconductor process. Among them, the accuracy of the neuromorphic computing system for handwritten digital recognition using the neuromorphic processor is verified on the MNIST handwritten digital data set (including 70,000 images of 10 types of handwritten digital images of the numbers 0-9, including 60,000 training images and 10,000 test images). This neuromorphic computing system builds 330 physical neurons on the neuromorphic processor 100 based on digital integrated circuits, and each physical neuron is connected to a total of 128,640 synapses, and the model parameters of the spiking neural network (mainly synaptic weights) are quantized to 4 bits for online learning. Finally, under the 40-nanometer CMOS digital process, the area of ​​the neuromorphic processor is only 0.95 mm2, the operating voltage is 0.8V, the maximum operating frequency is 500MHz, and the dynamic power consumption is between 75.6 and 93.52μW / MHz, demonstrating the performance advantages of the neuromorphic processor in terms of recognition accuracy and computing efficiency. At the same time, the classification accuracy of this embodiment on the MNIST data set reaches 95%, verifying the image target recognition ability of this neuromorphic computing system. Compared with existing recognition processors, the neuromorphic processor provided by this embodiment requires a larger number of physical neurons, and relies on the designed multi-layer routing to support the needs of neural computing tasks of various scales. Compared with other processors, this embodiment better balances the trade-offs of area, power consumption and accuracy of the neuromorphic computing system. Through careful design and optimization, the neuromorphic computing system in this embodiment shows significant advantages in terms of the number of neurons, computing accuracy, power consumption efficiency and area utilization. This makes it surpass existing similar processors in performance, achieves an excellent balance in power consumption control and performance integration, and provides strong support for the future application of neuromorphic computing.

[0089] This embodiment combines the bio-inspired computing model with the hardware architecture, and proposes a neuromorphic processor design that can flexibly adapt to and efficiently complete complex tasks. The processor is based on a pulse neural network, has low power consumption, high precision and online learning capabilities, and can efficiently simulate the dynamic computing process of the biological nervous system. The neuromorphic computing system provided in this embodiment not only improves the computing speed and energy efficiency, but also effectively reduces the consumption of hardware resources. It has a wide range of applications and is suitable for medical image analysis, autonomous driving, intelligent monitoring, smart cities and other fields, aiming to solve the shortcomings of existing technologies in terms of performance, power consumption and flexibility.

[0090] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A neuromorphic computing system based on a spiking neural network, characterized in that: The system includes: A neuromorphic processor is used to construct a hardware spiking neural network using a plurality of physical neurons, and to perform pulse encoding and target recognition on a real-time input image through the spiking neural network; the physical neurons are constructed using an approximate biological neuron model; the approximate biological neuron model has pre-stored synaptic weights and activation thresholds; A central processing unit implemented based on the RISC-V instruction architecture, communicating with the neuromorphic processor, for executing control instructions required to construct the spiking neural network; The neuromorphic processor is further used to perform online learning on the spiking neural network under the control of the control instruction, and update the synaptic weight and activation threshold associated with each physical neuron of the spiking neural network; A memory for storing program instructions required for the operation of the central processing unit, wherein the program instructions include control instructions required for constructing the spiking neural network; and for storing data transmitted between the central processing unit and the neuromorphic processor.

2. The neuromorphic computing system based on a spiking neural network according to claim 1, characterized in that: The central processing unit is a single-issue out-of-order RISC-V processor; the RISC-V processor includes a processor front end, an instruction buffer unit and a processor back end; The processor front end includes a program decoder, an instruction fetch unit and a dynamic branch predictor; The program decoder is used to convert the computer program that controls the operation of the pulse neural network into a RISC-V instruction stream, and transmit the RISC-V instruction stream to the memory for storage; The instruction fetch unit is used to access the memory, read a single RISC-V instruction corresponding to a program counter value of a current clock cycle and transmit it to the instruction buffer unit; the program counter value is used as a read address for accessing the memory; The dynamic branch predictor is used to predict the program counter value of the next clock cycle; The instruction buffer unit is used to buffer the RISC-V instructions read by the instruction fetch unit, and isolate the front-end computing thread from the back-end computing thread of the RISC-V processor; The processor backend is a five-stage pipeline architecture consisting of a decoding unit, an emission unit, a physical register stack, an execution unit and a retirement unit, which is used to read the RISC-V instructions of the instruction buffer unit and perform decoding, emission, reading and writing of the physical register stack, out-of-order calculation, and instruction retirement operations on them in sequence, so as to realize data interaction between the RISC-V processor and the neuromorphic processor.

3. The neuromorphic computing system based on a spiking neural network according to claim 1, characterized in that: The approximate biological neuron model includes a charging model and a discharging model, wherein: The charging model is: Baseline decay charging model: ,or, Incremental decay charging model: , The discharge model is: Hard reset discharge model: ,or, Soft reset discharge model: , Among them, V(t) is the membrane potential of the physical neuron at the current time step, V(t-1) is the membrane potential of the physical neuron at the previous time step, and X(t) is the membrane potential increment formed by the pulse signal input at the current time step. is the preset membrane potential attenuation factor; is the reset reference potential of the physical neuron; V(t+) is the membrane potential of the physical neuron after reset at the current time step, V(t-) is the membrane potential of the physical neuron before reset at the current time step; Vth is the activation threshold of the current physical neuron.

4. The neuromorphic computing system based on a spiking neural network according to claim 3, characterized in that: The neuromorphic processor includes: a pulse encoder, a plurality of pulse neural computing cores for parallel computing, at least one pulse neural decision core, and a first-level router; Each of the pulse neural computing core and the pulse neural decision core has a built-in synaptic computing module, a neuron computing module and a routing interface; the neuron computing module includes a transient data storage, a neuron arbitrator, and a plurality of physical neurons cascaded using a pipeline architecture; the synaptic computing module pre-stores the initial synaptic weights obtained by offline training of each physical neuron; the transient data storage pre-stores the initial activation threshold of each physical neuron; The pulse encoder is used to pulse encode the input image based on the convolutional neural network, generate a series of pulse signals every other time step within a preset time window, and form a pulse sequence to be input into the pulse neural network; The plurality of parallel computing pulse neural computing cores are used to form a hidden layer of the pulse neural network, connect the pulse sequence to each physical neuron in the hidden layer in batches, and control the activation state of each physical neuron according to the approximate biological neuron model; The pulse neural decision core is used to form the output layer of the pulse neural network, and distribute the new pulse signals generated by each physical neuron activated in the hidden layer to the physical neurons in the output layer for target prediction; The first-level router is used to construct a communication route between the pulse encoder and the pulse neural computing core, and distribute the pulse sequence to the multiple pulse neural computing cores in the hidden layer for parallel calculation; it is also used to construct a communication route between the multiple pulse neural computing cores and the pulse neural decision core, and transmit the calculation results of the hidden layer to the output layer.

5. The neuromorphic computing system based on a spiking neural network according to claim 4, characterized in that: Each of the spiking neural computing core and the spiking neural decision core also includes an online learning unit; the online learning unit includes: a forgetting deactivator, a freezer, a random learner and a parameter updater; The forgetting deactivator is used to randomly deactivate the physical neurons involved in the hidden layer based on a random forgetting learning method; The freezer is used to freeze the synaptic weights and activation thresholds of the physical neurons involved in the output layer; The random learner is used to train the physical neurons that are not inactivated in the hidden layer, and to train the physical neurons that are unfrozen in the output layer; A parameter updater is used to update the synaptic weights and activation thresholds of the physical neurons trained by the stochastic learner, specifically: If the difference between the membrane potential of the physical neuron at the current time step and the activation threshold at the current time step is greater than the preset pseudo-random number, the following equations are used to update the synaptic weight and activation threshold of the current physical neuron respectively: Among them, represents the synaptic weight at the next time step, represents the synaptic weight at the current time step, represents the neuron activation threshold at the next time step, represents the neuron activation threshold at the current time step, represents the time difference between the spikes of the pre-neuron and the post-neuron. a and b represent two different synaptic weight update amplitudes, and 0 < a < b; c represents the update amplitude of the neuron activation threshold, and c > 0; Otherwise, the synaptic weights and neuron activation thresholds of the physical neuron at the next time step remain unchanged.

6. The neuromorphic computing system based on a spiking neural network according to claim 4, characterized in that: Each of the physical neurons comprises: a PE approximate calculation array, an approximate addition tree and a pulse emission decision unit; The PE approximate calculation array is composed of multiple PE units using a parallel pipeline calculation architecture, which is used to access the pulse sequence in batches and issue each batch of accessed pulse signals in parallel to the corresponding PE units for pipeline calculation; Each PE unit includes accumulation control logic and at least one first approximate adder; Each of the PE units is used to accumulate the local pulse sequences connected in each batch by using the first approximate adder under the control of the accumulation control logic to obtain the local membrane potential increment within the preset time window; and according to the charging model, calculate the local membrane potential of the physical neuron in the current PE unit by using the local membrane potential increment; The approximate addition tree is composed of a plurality of second approximate adders hierarchically formed by a binary tree structure, and is used to superimpose the local membrane potential output by each PE unit to obtain the complete membrane potential accumulated by the pulse sequence on the current physical neuron; The pulse emission decision unit is used to compare the complete membrane potential with the activation threshold of the current physical neuron to determine whether the current physical neuron is activated; if the current physical neuron is not activated, a new pulse signal is connected and the membrane potential of the current physical neuron is charged according to the charging model; if the current physical neuron is activated, a new pulse is emitted, and after the new pulse is emitted, the membrane potential of the current physical neuron is reset according to the discharge model.

7. The neuromorphic computing system based on a spiking neural network according to claim 6, characterized in that: The neuromorphic processor further includes: a pulse density discriminator; The pulse density discriminator is used to discriminate the pulse sequence as a dense pulse sequence or a sparse pulse sequence according to the pulse density of the pulse sequence; The PE approximate calculation array is also used to calculate the complete membrane potential of the pulse sequence on the current physical neuron using the approximate addition tree when the pulse sequence is a dense pulse sequence; and to calculate the complete membrane potential of the pulse sequence on the current physical neuron using a logic OR gate circuit when the pulse sequence is a sparse pulse sequence.

8. The neuromorphic computing system based on a spiking neural network according to claim 4, characterized in that: The pulse encoder includes a plurality of convolution filters, a pooler, and a spatiotemporal encoder; The multiple convolution filters are connected in sequence, and are used to perform multiple convolution operations on the input image through a trainable convolution kernel, extract image features at different levels and fuse them into an image feature map; The pooler is used to perform data dimensionality reduction on the image feature map to reduce the spatial information of the image feature map; The spatiotemporal encoder is used to divide the image feature map after dimensionality reduction into multiple local spatial regions, and encode each of the local spatial regions in sequence according to a preset time interval, and quantize the intensity value of the pixel in the local spatial region into a pulse sequence emitted at a frequency.

9. The neuromorphic computing system based on a spiking neural network according to claim 6, characterized in that: The first approximate adder and the second approximate adder are both speculative carry adders with error correction mechanisms; The speculative carry adder includes m k-bit sub-adders, which are used to accumulate the corresponding m-segment k-bit input sub-signals, wherein the m-segment k-bit input sub-signals are divided by the input signal with a total bit width of n=m·k; Each of the sub-adders includes a speculative carry generator; the speculative carry generator is used to infer the approximate carry signal of the corresponding sub-adder according to the following equation: Where i represents the number of the sub-adder and 2≤i≤m, represents the approximate carry signal of the ith sub-adder; the approximate carry signal of the first sub-adder is =0; k represents the bit width of each sub-adder, represents the carry flag signal of the k-1th bit generated by the i-th sub-adder, represents the carry flag signal of the k-1th bit generated by the i-1th sub-adder, The carry propagation signal representing the j-th bit of the output of the i-th sub-adder; A multiplexer is also provided between each of the sub-adders, for selecting the carry identification signal of the previous sub-adder or the approximate carry signal of the previous sub-adder as the carry input signal of the current sub-adder according to the state value of the carry propagation signal of the current sub-adder.

10. A neuromorphic computing method based on a spiking neural network, characterized in that: include: Write computer programs; The computer program is input into a neuromorphic computing system based on a spiking neural network as described in any one of claims 1 to 9 for execution, so as to realize real-time target recognition of an input image.

Images