A neuromorphic computing device

Abstract

The present invention relates to the field of neuronal computing, and in particular to a neuromorphic computing device. The device includes a neuronal state memory, a neuronal interconnection memory, a controller, a physical neuron, an external neuronal excitation input unit, and an external neuronal activation output unit. The biological state of all neurons in the neural network system is stored in the neuronal state memory, and the interconnection state of all neurons in the neural network system is stored in the neuronal interconnection memory. The device works periodically at a time interval of Δt, and completes the update of all neuronal states in the neuronal state memory within the Δt time: the controller reads the neuronal interconnection state, sequentially configures the physical neurons as corresponding virtual neurons, and the physical neurons actively obtain data from the state memory according to the configuration information, completes the corresponding calculation, and finally updates the state of the corresponding neuron. This device can realize flexible, universal and arbitrary scale neural network calculations.

Description

Technical Field

[0001] The present invention relates to the field of neuromorphic computing, and in particular to a neuromorphic computing device (or neuronal computer). Background Art

[0002] In 1936, British mathematician Alan Turing proposed an abstract computing model to replace human mathematical operations. This computing model is called the Turing Machine. The Turing Machine is very simple and consists of three parts: (1) an infinitely long paper tape for storing data content, (2) a read / write head for reading and updating the tape content, and (3) a controller that determines the specific operation mode of the read / write head according to specific program rules.

[0003] After nearly 100 years of development, Turing machines have become the basic computing model of most modern computer systems, from micro embedded computers to large supercomputers. All modern computer systems belong to Turing machines. The core function of the Turing machine computing model is to realize mathematical operations. The rules of mathematical operations are stored in the form of programs. Therefore, Turing machines are a computing system that "updates data according to program rules."

[0004] A neural network is a mathematical abstract model that simulates the human brain's nervous system. The neural network model can be decomposed into a program that can be run by a Turing machine to implement the operation of the neural network model in a Turing computer. However, the Turing computer's calculation method of "updating data according to program rules" will introduce a large number of additional operations such as program control, resulting in low calculation efficiency.

[0005] In addition to traditional Turing computer solutions, some neuromorphic dedicated computing solutions have also emerged in recent years, such as IBM TrueNorth, Intel Loihi, and Tsinghua Tianjic.

[0006] Existing neuromorphic computing solutions physically construct a computing system composed of neurons and their interconnections. For example, IBM TrueNorth is a neuromorphic computing system composed of 1 million neurons connected to each other; Tsinghua University Tianjic is a neuromorphic computing system composed of 40,000 neurons connected to each other.

[0007] A major drawback of existing computing solutions is that they have a limited neuron scale. Neural networks larger than the physical neuron scale cannot run in this neuromorphic computing system and can only run through expansion (for example, merging multiple computing systems to achieve larger-scale computing).

[0008] Since the human brain nervous system is very large, the human brain is composed of about 100 billion neurons, each of which contains 1,000 to 10,000 synapses. If a human brain-scale neuromorphic computing system is implemented according to existing technical solutions, the relevant system will become very large and almost impossible to implement. Summary of the invention

[0009] In order to solve the above problems, the present invention proposes a neuromorphic computing device to realize neural network computing with unlimited scale.

[0010] To achieve the above-mentioned object, the present invention provides a neuromorphic computing device, including a neuron state memory, a neuron interconnect memory, a controller, a physical neuron, an external neuron excitation input unit, and an external neuron activation output unit.

[0011] The neuron state memory and the neuron interconnection memory together realize the digital storage of the neural network, each neuron in the neural network corresponds to a virtual neuron, wherein the neuron state memory is used to store the neuron state information of each virtual neuron, and the neuron interconnection memory is used to store the neuron interconnection information of each virtual neuron;

[0012] The physical neurons are used to implement calculations of various neuron models;

[0013] The controller is used to: periodically read the neuron interconnection information of the virtual neurons from the neuron interconnection memory, sequentially configure the neuron models in the physical neurons as the corresponding virtual neurons according to the sequence of the virtual neurons, and start the physical neuron calculation until all virtual neurons are traversed;

[0014] The external neuron excitation input unit is used to connect to the external neural network system, receive and store excitation information from external neurons, and is ultimately used by the physical neurons;

[0015] The external neuron activation output unit is used to connect to an external neural network system, receive and store activation information from internal neurons, and is ultimately used by the external neural network system.

[0016] Furthermore, the physical neuron includes a virtual neuron state cache module, a virtual neuron state read-write module, a virtual neuron control module, a synaptic calculation module, a dendrite calculation module and a cell body calculation module; wherein the virtual neuron control module is used to receive configuration information from the controller, and configure the virtual neuron state read-write module to a corresponding interconnection mode, and configure the synaptic calculation module, the dendrite calculation module and the cell body calculation module to a corresponding functional mode according to the configuration information;

[0017] After the configuration is completed, the virtual neuron state read-write module, the virtual neuron state cache module, the synapse calculation module, the dendrite calculation module and the cell body calculation module form a data calculation path, the virtual neuron state read-write module reads the input neuron state information from the neuron state memory, reads the excitation input from the external neuron excitation input unit, and stores the input neuron state and the excitation input in the virtual neuron state cache;

[0018] The synaptic calculation module obtains the data required for calculation from the virtual neuron state cache, completes the synaptic calculation according to the functional mode, and passes the calculation result to the dendrite calculation module; after the dendrite calculation module obtains the data, it completes the dendrite calculation according to the functional mode, and passes the calculation result to the cell body calculation module; after the cell body calculation module obtains the data, it completes the cell body calculation according to the functional mode, and passes the calculation result to the virtual neuron state read-write module; finally, the virtual neuron state read-write module obtains the calculation result from the cell body calculation, and updates the calculation result to the neuron state memory or the external neuron activation output unit.

[0019] Furthermore, the state of a neuron refers to the biological state of the neuron model, which includes the following information: (1) axon pulse information, which represents whether the neuron is activated at a certain moment to output a pulse of 0, 1, or -1; (2) membrane potential, which represents the membrane potential of the neuron at a certain moment; (3) synaptic weight, which represents the weight of the interconnection between neurons.

[0020] Furthermore, the order of the virtual neurons is based on whether the input neurons have completed updating, that is, the neurons whose input neurons have completed updating are calculated first.

[0021] Furthermore, the neuromorphic computing device dynamically updates the state data of all neurons in the neural network, and its implementation method is as follows: under normal working conditions, the state data of all neurons in the neural network are periodically updated at a time interval of Δt: external input neurons send excitation information at a time interval of Δt; within the time interval of Δt, the states of all neurons in the neural network are updated; finally, activation information is output to external output neurons at a time interval of Δt.

[0022] Furthermore, the excitation information includes axon pulse information and membrane potential information; the activation information includes axon pulse information and membrane potential information.

[0023] Furthermore, the neuron state memory and the neuron interconnection memory are two functional blocks of the same memory, or are two independent memories.

[0024] Furthermore, the neuron state memory, the neuron interconnection memory, the controller, the external neuron excitation input unit and the external neuron activation output unit are connected to the physical neuron via a system bus.

[0025] Furthermore, the neuromorphic computing device includes one or more physical neurons.

[0026] Furthermore, the neuromorphic computing device includes one or more controllers, or a multi-core controller.

[0027] The present invention achieves the following technical effects:

[0028] The neuromorphic computing device (neural computer for short) of the present invention is a computing system based on the digitized storage of neural networks, which completes external neuron excitation input, neuron state update and external neuron activation output with a Δt time interval as a period. It has the characteristics of separation of neuron storage and neuron computing, separation of neuron state and neuron interconnection, and autonomous calculation of physical neurons, and can realize flexible, general and arbitrary scale neural network calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a neural network model of the prior art;

[0030] Figure 2 It is a schematic diagram of the digital storage of the neural network of the present invention;

[0031] Figure 3 is a schematic diagram of dynamic update of neural network data of the present invention;

[0032] Figure 4 is a functional block diagram of the neuron computer of the present invention;

[0033] Figure 5 It is a block diagram of the internal structure of the physical neuron of the present invention. DETAILED DESCRIPTION

[0034] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0035] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0036] The present invention proposes a neuromorphic computing device (abbreviated as neuron computer). The core function of the neuron computer is to realize the calculation of the neural network, and the neural network is stored in the form of data. Therefore, the neuron computer is a computing system that "updates the neuron state according to the neural network structure".

[0037] The neural network refers to a mathematical abstract model of the human brain nervous system. The neural network model is composed of interconnected neurons. A typical neural network is as follows: Figure 1 shown.

[0038] In a neural computer, the neural network is stored in a digital form. The digitalization of a neural network refers to expressing the state of neurons in a neural network and the interconnection between neurons in a specific data format. The state of a neuron refers to the biological state of a neuron model, which generally includes several types of information: (1) axon pulse information, which represents whether a neuron is activated at a certain moment to output a pulse of 0, 1, or -1; (2) membrane potential, which represents the membrane potential of a neuron at a certain moment; (3) synaptic weight, which represents the weight of the interconnection between neurons. The interconnection between neurons refers to the input-output connection relationship between different neurons.

[0039] After the neural network is expressed in data form, the data is usually stored in a memory, the state of the neuron is stored in the neuron state memory, and the interconnection between neurons is stored in the neuron interconnection memory, such as Figure 2 shown.

[0040] In practical applications, the neuron state memory and the neuron interconnection memory may be two storage blocks of the same memory or may be different memories.

[0041] In a neural computer, neural networks are stored in a digitized form. From a static point of view, the data not only expresses the structure and interconnection of the complete neural network, but also represents the real-time state of the neural network at a certain time t, such as the axon pulse information and membrane potential of all neurons at a certain time t.

[0042] The core function of the neural computer is to dynamically update the status data of all neurons in the neural network, such as Figure 3 As shown, under normal working conditions, the state data of all neurons in the neural network are periodically updated at Δt time intervals: external input neurons send input excitation information at Δt time intervals; within the Δt time interval, the states of all neurons in the neural network are updated; finally, activation information is output to external output neurons at Δt time intervals.

[0043] The neural computer can realize the neural network computing tasks based on data storage, that is, complete the excitation input of external neurons, the update of neuron status and the activation output of external neurons within the Δt time interval.

[0044] We give the following definition of a neural computer: A neural computer is a computing system that is based on the data storage of neural networks and completes external neuron stimulus input, neuron state update, and external neuron activation output at a time interval of Δt.

[0045] Example 1

[0046] In this embodiment, the neuron computer is composed of six parts, including a neuron state memory 10, a neuron interconnect memory 20, a controller 30, a physical neuron 40, an external neuron excitation input unit 50, and an external neuron activation output unit 60. Figure 4 shown.

[0047] The neuron state memory 10 and the neuron interconnection memory 20 jointly realize the digital storage of the neural network, wherein the neuron state memory 10 stores the biological state of the neuron (neuron state information), and the neuron interconnection memory 20 stores the input-output connection relationship between neurons (i.e., neuron interconnection information). The neurons described in the neuron state memory 10 and the neuron interconnection memory 20 are called virtual neurons. Each neuron in the neural network corresponds to a virtual neuron, wherein the neuron state memory 10 is used to store the neuron state information of each virtual neuron, and the neuron interconnection memory 20 is used to store the neuron interconnection information of each virtual neuron. In theory, the scale of the neural network that can be processed by the neuron computer is only limited by the capacity of the above memory.

[0048] The physical neuron 40 is the core computing component of the neuron computer. The physical neuron 40 is an independent neuron computing module that can realize the calculation of multiple neuron models. When configured in a certain computing mode (corresponding to a certain neuron model), the physical neuron 40 can independently complete the reading of the neuron state from the neuron state memory 10, the reading of the excitation input from the external neuron excitation input unit 50, the completion of the calculation corresponding to the neuron model, and finally write the result to the neuron state memory 10 to update the neuron state or write it to the external neuron activation output unit 60.

[0049] The controller 30 is the core component of the neuron computer. The controller 30 reads the neuron interconnection information of the virtual neurons from the neuron interconnection memory 20 at a time interval of Δt. According to the sequence of virtual neurons, the neuron models in the physical neurons 40 are sequentially configured as corresponding virtual neurons, and the physical neuron calculation is started until all virtual neurons are traversed, that is, the calculation of the neural network model is divided into multiple neuron calculations in steps, and the corresponding neuron models are configured in the physical neurons 40 in steps to perform the corresponding neuron calculations. The sequence of the virtual neurons is based on whether the input neurons have completed the update, that is, the neurons whose input neurons have completed the update are calculated first.

[0050] The external neuron excitation input unit 50 is used to connect to an external neural network system. Its main function is to receive and store excitation information from external neurons, such as axon pulse information or membrane potential information, which is eventually used by the physical neuron 40.

[0051] The external neuron activation output unit 60 is used to connect to an external neural network system. Its main function is to receive and store activation information from internal neurons, such as axon pulse information or membrane potential information, and is ultimately used by the external neural network system.

[0052] A neuron computer is a computing system in which neuron storage and neuron computation are separated. One major advantage of the separation of storage and computation is that it makes the neuron computer versatile and flexible. On the one hand, the scale of the neural network that a neuron computer can handle (the number of neurons and neuron interconnections) is completely determined by the capacity of the memory. If the capacity of the memory is large enough, the neuron computer can realize neural network computations of any scale. On the other hand, the scale of the computational resources of a neuron computer only determines the computational delay of the neural network system. The computational delay can be reduced by using a neuron computer with larger-scale computational resources.

[0053] A neuron computer is a computing system in which the neuron state is separated from the neuron interconnection. One major advantage of separating the neuron state from the interconnection is that physical neurons can realize the calculation of any virtual neurons, thus making the neuron computer universal for any neural network calculation. Among them, the neuron interconnection determines which neuron state data the physical neurons read and update in the form of control, and the neuron state is read and written by the physical neurons in the form of data. Therefore, by continuously updating the relevant control logic of the neuron interconnection in the physical neurons, the physical neurons can change the target address of the neuron reading and neuron updating, thereby realizing the calculation of any virtual neuron.

[0054] The neuron computer saves the biological state of all neurons in the neural network system through the neuron state memory, and saves the interconnection state of all neurons in the neural network system through the neuron interconnection memory. The neuron computer works periodically at a time interval of Δt. Within the Δt time, the neuron computer completes the update of all neuron states in the neuron state memory: the controller reads the neuron interconnection state, configures the physical neurons as corresponding virtual neurons in turn, and the physical neurons actively obtain data from the state memory according to the configuration information, completes the corresponding calculation, and finally updates the state of the corresponding neuron.

[0055] The physical neuron 40 is similar to the GPU in a general-purpose computer, and the neuron computer includes one or more (including 2) physical neurons 40. The controller 30 is equivalent to the general-purpose processor CPU in a general-purpose computer, and the neuron computer includes one or more (including 2) controllers 30, or a multi-core controller. Similar to a general-purpose computer, the neuron state memory 10, the neuron interconnect memory 20, the controller 30, the physical neuron 40, the external neuron excitation input unit 50 and the external neuron activation output unit 60 are connected through a system bus (including a data bus, an address bus and a control bus, such as a PCI-E bus), so that multiple physical neurons 40 can share the neuron state memory 10, the neuron interconnect memory 20, the external neuron excitation input unit 50 and the external neuron activation output unit 60, and are controlled by a single or multiple controllers 30 to realize parallel computing or distributed computing of neurons.

[0056] Both the external neuron excitation input unit and the external neuron activation output unit can be composed of multiple functional modules to provide a rich input and output interface, supporting the input and output of analog signals and digital signals. The external neuron excitation input unit can be an image sensor or a dynamic sensor receiving device, which is used to receive external sensor signals and convert them into neuron excitation signals. The external neuron activation output unit can be an image sending device, which is used to convert the neuron activation output signal into an image signal and send it to an external system.

[0057] In this embodiment, the internal structure of the physical neuron 40 is as follows: Figure 5 shown.

[0058] The physical neuron 40 is composed of a virtual neuron state cache module 401 , a virtual neuron state read-write module 402 , a virtual neuron control module 403 , a synapse calculation module 404 , a dendrite calculation module 405 and a cell body calculation module 406 .

[0059] The virtual neuron control module 403 receives configuration information from the controller 30, configures the virtual neuron state read / write module 402 to a corresponding interconnection mode, and configures the synapse calculation module 404, the dendrite calculation module 405 and the cell body calculation module 406 to a corresponding functional mode.

[0060] After configuration is completed, the virtual neuron state read-write module 402, the virtual neuron state cache module 401, the synaptic calculation module 404, the dendrite calculation module 405 and the cell body calculation module 406 form a data calculation path. The virtual neuron state read-write module 402 reads the input neuron state from the neuron state memory 10, reads the excitation input from the external neuron excitation input unit 50, and stores the input neuron state and the excitation input in the virtual neuron state cache module 401.

[0061] The synaptic calculation module 404 obtains the data required for calculation from the virtual neuron state cache module 401, completes the synaptic calculation according to the functional mode, and passes the calculation result to the dendrite calculation module 405. After obtaining the data, the dendrite calculation module 405 completes the dendrite calculation according to the functional mode, and passes the calculation result to the cell body calculation module 406. After obtaining the data, the cell body calculation module 406 completes the cell body calculation according to the functional mode, and passes the calculation result to the virtual neuron state read-write module 402. Finally, the virtual neuron state read-write module 402 obtains the calculation result from the cell body calculation, and updates the result to the neuron state memory 10 or the external neuron activation output unit 60.

[0062] The functions of the synaptic calculation module 404, the dendrite calculation module 405 and the cell body calculation module 406 are determined by the neuron model. For example, under a certain neuron model, the synaptic calculation module 404 implements the multiplication calculation of data and parameters, the dendrite calculation module 405 implements the accumulation of synaptic calculation results, and the cell body calculation module 406 implements the upper and lower limit threshold activation of the dendrite calculation results.

[0063] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A neuromorphic computing device, characterized in that It includes a neuron state memory, a neuron interconnect memory, a controller, a physical neuron, an external neuron excitation input unit, and an external neuron activation output unit, The neuron state memory and the neuron interconnection memory together realize the digital storage of the neural network, each neuron in the neural network corresponds to a virtual neuron, wherein the neuron state memory is used to store the neuron state information of each virtual neuron, and the neuron interconnection memory is used to store the neuron interconnection information of each virtual neuron; the physical neurons are used to realize the calculation of multiple neuron models; The controller is used to: periodically read the neuron interconnection information of the virtual neurons from the neuron interconnection memory, sequentially configure the neuron models in the physical neurons as the corresponding virtual neurons according to the sequence of the virtual neurons, and start the physical neuron calculation until all virtual neurons are traversed; The external neuron excitation input unit is an image sensor, which is used to connect to the external neural network system, receive and store excitation information from external neurons, and is ultimately used by physical neurons; The external neuron activation output unit is an image sending device, which is used to convert the neuron activation output signal into an image signal and send it to the external system; The physical neuron includes a virtual neuron state cache module, a virtual neuron state read-write module, a virtual neuron control module, a synaptic calculation module, a dendrite calculation module and a cell body calculation module; wherein the virtual neuron control module is used to receive configuration information from the controller, and configure the virtual neuron state read-write module to a corresponding interconnection mode, and configure the synaptic calculation module, the dendrite calculation module and the cell body calculation module to a corresponding functional mode according to the configuration information; After the configuration is completed, the virtual neuron state read-write module, the virtual neuron state cache module, the synapse calculation module, the dendrite calculation module and the cell body calculation module form a data calculation path, the virtual neuron state read-write module reads the input neuron state information from the neuron state memory, reads the excitation input from the external neuron excitation input unit, and stores the input neuron state and the excitation input in the virtual neuron state cache; The synaptic calculation module obtains the data required for calculation from the virtual neuron state cache, completes the synaptic calculation according to the functional mode, and passes the calculation result to the dendrite calculation module; after the dendrite calculation module obtains the data, it completes the dendrite calculation according to the functional mode, and passes the calculation result to the cell body calculation module; after the cell body calculation module obtains the data, it completes the cell body calculation according to the functional mode, and passes the calculation result to the virtual neuron state read-write module; finally, the virtual neuron state read-write module obtains the calculation result from the cell body calculation, and updates the calculation result to the neuron state memory or the external neuron activation output unit; The neuron state information refers to the biological state of the neuron model, including the following information: axon pulse information, representing whether the neuron is activated to output a pulse of 0, 1 or -1 at a certain moment; membrane potential, representing the membrane potential of the neuron at a certain moment; synaptic weight, representing the weight of the interconnection between neurons.

2. The neuromorphic computing device of claim 1, wherein: The sequence of the virtual neurons is based on whether the input neurons have completed updating.

3. The neuromorphic computing device of claim 1, wherein: The neuromorphic computing device dynamically updates the state data of all neurons in the neural network, and the implementation method is: under normal working conditions, the state data of all neurons in the neural network are updated in the form of △ t time interval periodic update: external input neurons are updated with △ Send incentive information at intervals of t; △ Within the time interval t, the states of all neurons in the neural network are updated; finally, △ Output activation information to external output neurons at time intervals t.

4. The neuromorphic computing device of claim 3, wherein: The excitation information includes axon pulse information and membrane potential information; the activation information includes axon pulse information and membrane potential information.

5. The neuromorphic computing device of claim 1, wherein: The neuron state memory and the neuron interconnection memory are two functional blocks of the same memory, or are two independent memories.

6. The neuromorphic computing device of claim 1, wherein: The neuron state memory, the neuron interconnection memory, the controller, the external neuron excitation input unit, the external neuron activation output unit and the physical neuron are connected via a system bus.

7. The neuromorphic computing device of claim 6, wherein: The neuromorphic computing device includes one or more physical neurons.

8. The neuromorphic computing device of claim 6, wherein: The neuromorphic computing device includes one or more controllers, or a multi-core controller.

Images