Arithmetic device, neural network system, neuron model device, arithmetic method, and program
By defining input and output time intervals for spiking neural networks, the inefficiencies in data processing are addressed, allowing synchronized neuron model operations for efficient data processing and increased throughput.
Patent Information
- Application Number
- JP2023539531
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-08-06
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-08-06
AI Technical Summary
Spiking neural networks face inefficiencies in data processing due to unpredictable firing timings of neuron models, leading to delays in processing multiple inputs sequentially.
Implementing a spiking neural network with defined input and output time intervals, where neuron models fire and transmit spikes within specified output time intervals, and receive spikes within input time intervals, using index value calculations to manage membrane potential changes.
This approach allows for efficient data processing by synchronizing neuron models, enabling pipeline processing and reducing wait times between data inputs, thereby enhancing throughput.
Smart Images

Figure 0007589818000012 
Figure 0007589818000013 
Figure 0007589818000014
Abstract
Description
[Technical field]
[0001] The present invention relates to a computing device, a neural network system, a neuron model device, a computing method, and a program. [Background technology]
[0002] One type of neural network is a spiking neural network (SNN). For example, Patent Document 1 describes a neuromorphic computing system that implements a spiking neural network on a neuromorphic computing device. In a spiking neural network, a neuron model has an internal state called a membrane potential, and outputs signals called spikes based on the time evolution of the membrane potential. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-136919 Summary of the Invention [Problem to be solved by the invention]
[0004] It is preferable that a spiking neural network be efficient at processing data.
[0005] An example of an object of the present invention is to provide a calculation device, a neural network system, a neuron model device, a calculation method, and a program that can solve the above-mentioned problems. [Means for solving the problem]
[0006] According to a first aspect of the present invention, a computing device is provided with a neuron model that is divided into an input time interval in which spikes are received and an output time interval in which spikes are fired and transmitted, the neuron model firing within the output time interval and limiting firing within the input time interval.
[0007] According to a second aspect of the present invention, a neural network system includes a neural network body and a learning means, and is divided into an input time interval in which a signal is received and an output time interval in which a signal is permitted to be transmitted in association with forced firing, the neural network body includes a neuron model including an index value calculation means for changing an index value of a signal output based on the input status of the signal in the input time interval, and a signal output means for outputting a signal within an output time interval after the end of the input time interval based on the index value, and the learning means learns a weighting coefficient for the signal.
[0008] According to a third aspect of the present invention, a neuron model device is provided with an input time interval in which spikes are received and an output time interval in which spikes are permitted to be transmitted, in association with forced firing, and the device is equipped with an index value calculation unit that changes an index value of a signal output based on the input status of a signal in the input time interval, and a signal output unit that transmits spikes in the output time interval by firing based on the index value of the signal output.
[0009] According to a fourth aspect of the present invention, a calculation method includes identifying an input time interval for receiving a spike and an output time interval for allowing a spike to be transmitted, the input time interval being divided in association with forced firing, changing an index value of a signal output based on an input condition of a signal in the input time interval, and transmitting a spike to the output time interval by firing based on the index value of the signal output.
[0010] According to a fourth aspect of the present invention, a program is provided for causing an apparatus executing the program to identify an input time interval for receiving a spike and an output time interval for allowing a spike to be transmitted, the input time intervals being divided in association with forced firing, change an index value of a signal output based on the input status of a signal in the input time interval, and transmit a spike to the output time interval by firing based on the index value of the signal output. Effect of the Invention
[0011] According to the present invention, a spiking neural network can efficiently process data. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a neural network device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a spiking neural network included in the neural network device according to the embodiment. [Diagram 3] FIG. 13 is a diagram showing an example of a change in membrane potential over time in a spiking neuron model in which the output timing of a spike signal is not restricted according to an embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of the output timing of a spike signal of a spiking neuron model in a spiking neural network in a case where the output timing of the spike signal of the spiking neuron model according to an embodiment is not restricted. [Figure 5A] 1 is a diagram showing an example of timing of passing of spike signals between neuron models in a neural network device according to an embodiment. FIG. [Figure 5B] 1 is a diagram showing an example of timing of passing of spike signals between neuron models in a neural network device according to an embodiment. FIG. [Figure 6] FIG. 11 is a diagram illustrating an example of setting a time interval according to the embodiment. [Figure 7] FIG. 4 is a diagram showing a first example of a clip function according to the embodiment. [Figure 8] FIG. 11 is a diagram illustrating a second example of a clip function according to the embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of a system configuration during learning in an embodiment. [Figure 10] 2 is a diagram showing an example of input and output of signals in the neural network system 1 according to the embodiment. FIG. [Figure 11] 1A to 1C are diagrams illustrating an example of signal input and output in a neural network device during operation in an embodiment. [Figure 12] FIG. 11 is a diagram showing an example of a threshold reaching time during learning in an embodiment. [Figure 13] FIG. 11 is a diagram showing an example of ignition times during learning in the embodiment. [Figure 14] FIG. 13 is a diagram showing an example of a threshold reaching time during a test in the embodiment. [Figure 15] FIG. 13 is a diagram showing an example of firing times during testing in the embodiment. [Figure 16] FIG. 1 is a diagram illustrating an example of the configuration of a neural network device according to an embodiment. [Figure 17] FIG. 1 is a diagram illustrating an example of the configuration of a neuron model device according to an embodiment. [Figure 18] FIG. 1 is a diagram illustrating an example of the configuration of a neural network system according to an embodiment. [Figure 19] 1 is a flowchart illustrating an example of a processing procedure in a calculation method according to an embodiment. [Figure 20] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. [Figure 21] FIG. 11 is a diagram illustrating an example of the configuration of a neural network device according to a second embodiment. [Figure 22] FIG. 11 is a diagram for explaining a result of the classification process according to the second embodiment. [Diagram 23] FIG. 13 is a diagram for explaining output results of a first convolution layer and a first pooling layer according to the second embodiment. [Figure 24] FIG. 13 is a diagram for explaining output results of a second convolution layer and a second pooling layer according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. (First embodiment) Fig. 1 is a diagram showing an example of the configuration of a neural network device according to an embodiment. In the configuration shown in Fig. 1, a neural network device 10 (arithmetic unit) includes a neuron model 100. The neuron model 100 includes an index value calculation unit 110, a comparison unit 120, and a signal output unit .
[0014] The neural network device 10 processes data using a spiking neural network. The neural network device 10 is an example of a computing device. The neural network device here is a device in which a neural network is implemented. The spiking neural network may be implemented in the neural network device 10 using dedicated hardware. For example, the spiking neural network may be implemented in the neural network device 10 using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Alternatively, the spiking neural network may be implemented in the neural network device 10 in the form of software using a computer or the like.
[0015] An apparatus including an ASIC, an apparatus including an FPGA, and a computer are all examples of programmable apparatuses. In the case of ASICs and FPGAs, describing the hardware using a hardware description language and implementing the described hardware on the ASIC or FPGA is an example of programming. When the neural network device 10 is configured using a computer, the function of the spiking neural network may be described by programming, and the obtained program may be executed by the computer.
[0016] The spiking neural network referred to here is a neural network in which a neuron model outputs a signal at a timing based on a state quantity called membrane potential, which changes over time depending on the input state of the signal to the neuron model itself. The membrane potential is also called the index value of the signal output, or simply the index value. The term "time change" used here means a change according to time.
[0017] A neuron model in a spiking neural network is also called a spiking neuron model. A signal output from a spiking neuron model is also called a spike signal or a spike. In a spiking neural network, a binary signal can be used as a spike signal, and information can be transmitted between spiking neuron models by the transmission timing of the spike signal or the number of spike signals. In the case of the neuron model 100, the index value calculation section 110 calculates the membrane potential based on the input state of the spike signal to the neuron model 100. The signal output section 130 outputs the spike signal at a timing according to the time change of the membrane potential.
[0018] As the spike signal in the neural network device 10, a pulse signal or a step signal may be used, but is not limited to these. In the following, a case will be described in which a time method of transmitting information at the transmission timing of a spike signal is used as an information transmission method between the neuron models 100 in the spiking neural network of the neural network device 10. However, the information transmission method between the neuron models 100 in the spiking neural network of the neural network device 10 is not limited to a specific method.
[0019] The processing performed by the neural network device 10 may be various processing that can be performed using a spiking neural network. For example, the neural network device 10 may perform image recognition, biometric authentication, or numerical prediction, but is not limited to these.
[0020] The neural network device 10 may be configured as one device, or may be configured as a combination of multiple devices. For example, each neuron model 100 may be configured as a device, and the devices constituting each neuron model 100 may be connected by a signal transmission path to form a spiking neural network.
[0021] 2 is a diagram showing an example of the configuration of a spiking neural network included in the neural network device 10. The spiking neural network included in the neural network device 10 is also referred to as neural network 11. Moreover, the neural network 11 is also referred to as the neural network main body.
[0022] In the example of Fig. 2, the neural network 11 is configured as a four-layer spiking neural network of the forward propagation type. Specifically, the neural network 11 includes an input layer 21, two intermediate layers 22-1 and 22-2, and an output layer 23. The two intermediate layers 22-1 and 22-2 are also collectively referred to as an intermediate layer 22. The intermediate layer is also referred to as a hidden layer. The input layer 21, the intermediate layer 22, and the output layer 23 are also collectively referred to as a layer 20.
[0023] The input layer 21 includes an input node 31. The hidden layer 22 includes a hidden node 32. The output layer 23 includes an output node 33. The input node 31, the hidden node 32, and the output node 33 are collectively referred to as nodes 30.
[0024] The input node 31 converts, for example, input data to the neural network 11 into a spike signal. Alternatively, when the input data to the neural network 11 is represented by a spike signal, a neuron model 100 may be used as the input node 31.
[0025] The neuron model 100 may be used as both the intermediate node 32 and the output node 33. In addition, the operation of the neuron model 100 may be different between the intermediate node 32 and the output node 33, for example, the constraints on the output timing of a spike signal, which will be described later, may be relaxed in the output node 33 compared to the intermediate node.
[0026] The four layers 20 of the neural network 11 are arranged in the following order from the upstream side in terms of signal transmission: input layer 21, intermediate layer 22-1, intermediate layer 22-2, and output layer 23. Between two adjacent layers 20, nodes 30 are connected by a transmission path 40. The transmission path 40 transmits spike signals from the node 30 in the upstream layer 20 to the node 30 in the downstream layer 20.
[0027] However, when the neural network 11 is configured as a forward propagation spiking neural network, the number of layers is not limited to four, but may be two or more. Also, the number of neuron models 100 in each layer is not limited to a specific number, but may be one or more. Each layer may have the same number of neuron models 100, or the number of neuron models 100 may differ depending on the layer.
[0028] In addition, the neural network 11 may be configured as a fully connected type, but is not limited to this. In the example of Fig. 2, all the neuron models 100 in the layer 20 on the front side of the adjacent layers and all the neuron models 100 in the layer 20 on the rear side of the adjacent layers may be connected by the transmission paths 40, but there may be some neuron models 100 in the adjacent layers that are not connected by the transmission paths 40.
[0029] In the following description, it is assumed that the delay time in the transmission of the spike signal can be ignored, and that the spike signal output time of the neuron model 100 on the spike signal output side is the same as the spike signal input time to the neuron model 100 on the spike signal input side. If the delay time in the transmission of the spike signal cannot be ignored, the time obtained by adding the delay time to the spike signal output time may be used as the spike signal input time.
[0030] A spiking neuron model outputs a spike signal when the time-varying membrane potential reaches a threshold. In a general spiking neural network, where the timing of outputting a spike signal is not restricted, when there are multiple data to be processed, the spiking neural network must wait for the next input data to be input to the spiking neural network before receiving the input data and outputting the calculation result.
[0031] Fig. 3 shows an example of the change in membrane potential over time in a spiking neuron model in which the output timing of spike signals is not restricted. The horizontal axis of the graph in Fig. 3 represents time, and the vertical axis represents membrane potential. Figure 3 shows an example of the membrane potential of the spiking neuron model of the i-th node in the l-th layer. The membrane potential of the spiking neuron model of the i-th node in the l-th layer at time t is expressed as v i (l) In the explanation of Fig. 3, the spiking neuron model of the i-th node in the l-th layer is also called the target model. Time t indicates the elapsed time from the start time of the time interval assigned to the processing of the first layer.
[0032] In the example in Figure 3, the target model receives spike signals as input from three spiking neuron models. time t2 *(l-1) indicates the input time of a spike signal from the second spiking neuron model in the l-1th layer. Time t1 *(l-1) indicates the input time of a spike signal from the first spiking neuron model in the l-1th layer. Time t3 *(l-1) indicates the input time of a spike signal from the third spiking neuron model in the l-1th layer. In addition, the target model is i *(l) The spiking neuron model outputs a spike signal. The output of a spike signal from the spiking neuron model is called firing. The time when the spiking neuron model fires is called the firing time.
[0033] In the example of Figure 3, the initial value of the membrane potential is set to 0. The initial value of the membrane potential corresponds to the resting membrane potential. Before the target model fires, after the spike signal is input, the membrane potential of the target model v i (l) (t) continues to change at a rate (speed of change) according to the weighting set for each spike signal transmission path. In addition, the rate of change of the membrane potential for each spike signal input is linearly added. In the example of Figure 3, the membrane potential v i (l) The differential equation for (t) is expressed as equation (1).
[0034]
number
[0035] In formula (1), w ij (l) indicates the weight set in the transmission path of the spike signal from the j-th spiking neuron model in the l-1th layer to the target model. ij (l) is the object of learning. Weight w ij (l)can have both positive and negative values.
[0036] θ is a step function, as shown in equation (2). Therefore, the membrane potential v i (l) The rate of change of (t) depends on the input status of the spike signal and the weight w ij (l) It changes depending on the value of and can take on both positive and negative values in the process.
[0037] For example, at time t i *(l) , the membrane potential v of the target model i (l) (t) is the threshold V th The target model fires. The firing causes the target model's membrane potential v i (l) After (t) becomes 0, the membrane potential does not change even if the target model receives a spike signal as input.
[0038] FIG. 4 is a diagram showing an example of the output timing of a spike signal of a spiking neuron model in a spiking neural network when the output timing of the spike signal of the spiking neuron model is not restricted.
[0039] The horizontal axis of Fig. 4 indicates time as the elapsed time from the start of data input to the spiking neural network. The vertical axis indicates the node numbers in the input layer, the first layer, the second layer, and the output layer. The node numbers in each layer are used as identification numbers to identify the spiking neuron models in the layer.
[0040] In the example of Figure 4, the spiking neuron models in the input layer fire within about 5 milliseconds from the start of data input to the spiking neural network. In contrast, the firing timings of the spiking neuron models in the first layer are scattered in the range of about 10 milliseconds to about 120 milliseconds, which is the elapsed time from the start of data input to the spiking neural network. The firing timings of the spiking neuron models in the second layer are scattered in the range of about 20 milliseconds to about 70 milliseconds, which is the elapsed time from the start of data input to the spiking neural network.
[0041] In general, it is difficult to know in real time when a spiking neural network is executing processing, the timing of firing of the spiking neural network, and which of the firings of the spiking neuron models in the first and second layers affects the firing of the spiking neuron model in the output layer.
[0042] For this reason, although it depends on the spiking neural network method, it is necessary to wait at least the time T11 until the first spiking neuron model fires in the output layer before inputting the next input data to the spiking neural network. To strictly set up so that no interference occurs within the spiking neural network between the input data to be processed in sequence by the spiking neural network, it is necessary to wait until the last spiking neuron model fires in the output layer before inputting the next input data to the spiking neural network. On the other hand, when there are multiple input data to be processed in sequence, if the next input data can be input into the spiking neural network before an answer for the first input data is obtained, the time from inputting the first input data into the spiking neural network to obtaining an answer for the last input data can be made relatively short, and in this respect the throughput of the spiking neural network can be increased.
[0043] Therefore, in the neural network device 10, a data processing time for the neuron model 100 is set so that the neuron model 100 completes data processing within the data processing time. As a result, the next data can be input to the neuron model 100 after the data processing time has elapsed.
[0044] 5A and 5B are diagrams showing an example of the timing of the transfer of spike signals between neuron models 100 in the neural network device 10. FIG. 5A shows an example of the time change in membrane potential of each neuron model 100 that transfers spike signals for each of the first to third layers of the neural network 11, and the firing timing based on the membrane potential. The horizontal axis of the graph in FIG. 5A indicates time, which is the elapsed time since the first input data was input to the first layer. The vertical axis indicates the membrane potential of each neuron model 100 from the first to third layers.
[0045] In the example of Fig. 5A, the data processing time set in the neuron model 100 is composed of a combination of an input time interval and an output time interval. The input time interval is a time interval during which the neuron model 100 accepts the input of a spike signal. The output time interval is a time interval during which the neuron model 100 outputs a spike signal. The neural network device 10 synchronizes the neuron models 100 in each layer so that the same data processing time is set for all the neuron models 100 included in the same layer, and sets the same input time interval and the same output time interval. T is the time width of each time interval.
[0046] Furthermore, the neural network device 10 also synchronizes between layers so that the output time interval of a neuron model 100 in a certain layer overlaps with the input time interval of a neuron model 100 in the next layer. In particular, the time length of the input time interval and the time length of the output time interval are set to the same length, and the input time interval and output time interval are set so that the output time interval in a neuron model 100 of a certain layer completely overlaps with the input time interval in the neuron model 100 of the next layer.
[0047] In this case, the "neuron model 100 in a certain layer" corresponds to an example of the first neuron model. The "neuron model 100 in the next layer" corresponds to an example of the second neuron model. The output time interval and the input time interval are set so that the output time interval of the first neuron model overlap with the input time interval of the second neuron model that receives the spike signal input from the first neuron model.
[0048] Since the time when the spike signal is input to the neuron model 100 is limited to the input time interval, the index value calculation unit 110 calculates the membrane potential so as to change the membrane potential over time based on the input state of the spike signal in the input time interval. The index value calculation unit 110 corresponds to an example of an index value calculation means. In the following, as in the explanation of the spiking neural network with reference to equations (1) and (2), the neuron model 100 of the i-th node of the l-th layer is referred to as the target model, and the membrane potential of the target model is denoted by v i (l) This is written as (t). The index value calculation unit 110 of this embodiment uses the following formula (1A) instead of the above formula (1). By using formula (1A), the membrane potential in the input time interval and the output time interval can be calculated as v i (l) The rate of change of (t) can be changed.
number
[0049] w ij (l) w denotes the weight set in the transmission path of the spike signal from the j-th spiking neuron model in the l-1th layer to the target model. i (l)is the inherent weight of the i-th neuron in the l-th layer. The membrane potential v i (l) The rate of change of (t) is the weighting factor w ij (l) This results in a function using the membrane potential v i (l) The rate of change of (t) is the weighting factor w ij (l) In addition, the membrane potential v in the output time section after time t is lT is calculated by a function using i (l) The rate of change of (t) is the weighting factor w i (l) For example, the function is a function using the weighting factor w i (l) It may be a linear function with the slope of w. i (l) can take positive values. This causes the membrane potential v i (l) The rate of change of (t) is the weighting factor w i (l) The weight w ij (l) And w i (l) is the target of learning. Also, the weight set in the transmission path of the spike signal from the j-th spiking neuron model in the l-1th layer to the target model is w ij (l) It is written as follows.
[0050] The description of the neuron model 100 of the target model applies to all the neuron models 100 included in the neural network device 10, except that the output timing of the spike signal of the neuron model 100 of the output layer, which will be described later, may be relaxed. That is, the l-th layer may be any layer that includes the neuron model 100 as a node. Any neuron model 100 in the l-th layer may be the i-th neuron model 100.
[0051] In this way, by using equation (1A), the membrane potential v i (l)(t) is the slope w i (l) Alternatively, a similar function can be implemented by fixing the membrane potential during the output time interval and varying the threshold V th This can also be achieved by changing the firing threshold instead of the threshold. Specifically, in each neuron model, the firing threshold is set to a unique value determined for each neuron model, and the firing threshold is changed in the output time interval with a slope of -w i (l) In the following explanation, the firing threshold is changed to the threshold V th The membrane potential is fixed at v i (l) Although the case where (t) changes is shown as an example, a similar argument can be applied to the case where the membrane potential is fixed and the firing threshold changes during the output time interval.
[0052] Below, the membrane potential v i (l) (t) will be explained separately for the input time interval and the output time interval. The index value calculation unit 110 calculates the membrane potential v i (l) (t) is the threshold V th The differential equation is calculated using the input time interval formula (1A) until the membrane potential v i (l) (t) is calculated. The membrane potential v i (l) (t) is expressed as equation (3A). The membrane potential v i (l) (t) is expressed as equation (3B).
number
[0053] The index value calculation unit 110 calculates the membrane potential v i (l) (t) may be calculated. In the input time interval, the index value calculation unit 110 calculates the membrane potential v at a rate of change according to the input state of the spike signal to the target model as shown in Equation (1A) and Equation (3A). i (l) Change (t). On the other hand, in the output time interval, the target model does not receive a spike signal as shown in equation (1A). Therefore, in the input time interval, the membrane potential v i (l) (t) is the threshold V th If the membrane potential v at the end of the input time interval is not reached, the index value calculation unit 110 calculates the membrane potential v at the end of the input time interval in the output time interval. i (l) Based on (lT) and the elapsed time, the membrane potential v increases monotonically from this potential. i (l) (t) is changed. The equation shown in (3B) is an example. The membrane potential v i (l) The rate of change of (t) may be maintained at a predetermined value.
[0054] First, the membrane potential v i (l) We will explain (t) in more detail. Within the input time interval, the membrane potential v i (l) (t) is the threshold V th The time when i (l,vth) and substituting it for time t in equation (3A), we obtain equation (4). This equation (4) is applied to the input time interval.
[0055]
number
[0056] Gamma i (l) t j *(l-1) <t i (l,vth) represents the set of indexes j such that In the neural network device 10, the firing time is limited within the output time interval. i *(l) is the membrane potential v i (l) (t) is the threshold V th Since the time may be later than the time when the membrane potential v i (l) (t) is the threshold V th The time when i (l,vth) The time at which the membrane potential reaches the threshold is also called the threshold reaching time. From equation (4), the threshold arrival time t i (l,vth) is expressed as in equation (5A). Next, the membrane potential v i (l) We will explain (t) in more detail. The index value calculation unit 110 calculates the membrane potential v i (l) (t) is the threshold V th The membrane potential v, whose differential equation is expressed as equation (1A), is expressed as i (l) Calculate (t). Membrane potential v i (l) (t) is expressed as in the above-mentioned formula (3B). By rearranging the linear expression of this formula (3B), the threshold arrival time t i (l,vth) is expressed as in equation (5B).
[0057]
number
[0058] The comparison unit 120 detects the membrane potential v i (l) (t) and threshold V th Compared with the membrane potential v i (l) (t) is the threshold V th It is determined whether or not The signal output unit 130 outputs the membrane potential v i (l) Based on (t), a spike signal is output within the output time interval. Specifically, within the input time interval, the membrane potential v i (l) (t) is the threshold V th When the membrane potential v reaches the threshold value, the signal output unit 130 outputs a spike signal at the start of the output time interval. i (l) (t) is the threshold V th When the membrane potential v i (l) (t) is the threshold V th When the membrane potential v reaches the threshold value within the output time interval, the signal output unit 130 outputs a spike signal. i (l) (t) is the threshold V th If it does not reach, the signal output section 130 outputs a spike signal at the end of the output time interval. In the example of Figure 5A, the output layer has no counterpart to which the spike signal can be transferred, so it is possible to output spikes even in the time interval corresponding to the input time interval. The time interval corresponding to the input time interval in this case is also called the input / output time interval.
[0059] Referring to FIG. 5B, the membrane potential v of the neuron model 100 is i (l) The change in (t) will be explained by dividing it into several cases. FIG. 5B shows an example of spike signals passed from the lth layer to the l+1th layer of the neural network 11. The horizontal axis of the graph in FIG. 5B indicates time, which is the elapsed time since the first input data was input to the lth layer. The vertical axis, from the top of FIG. 5B, indicates the membrane potential of the neuron model 100 in the lth layer, its output spike, and the membrane potential v i (l) 5B, interval A indicates the input time interval, and interval B indicates the output time interval. As shown in CASE0 in FIG. 5B, the membrane potential v i (l) (t) is the threshold V thWhen the membrane potential v reaches the start of the output time interval, the index value calculation unit 110 calculates the membrane potential v i (l) (t) is the threshold V th In this way, at the start of the output time period, the comparison unit 120 calculates the membrane potential v i (l) (t) is the threshold V th Based on the result of the determination by the comparing section 120, the signal output section 130 outputs a spike signal at the start of the output time interval.
[0060] In this case, as shown in FIG. 5B as CASE0 instead of the above, at the start of the output time period, the comparison unit 120 detects the membrane potential v i (l) (t) is the threshold V th After it is determined that the membrane potential v i (l) (t) may be calculated as 0. This allows the index value calculation unit 110 to process the next data in the next input time interval based on the membrane potential v i (l) It is also possible to start from a state in which (t) is reset to 0.
[0061] In addition, the membrane potential v i (l) (t) is the threshold V th If it has not reached v, the membrane potential v until the spike signal is output is i (l) The control of (t) can be divided into several cases. The index value calculation unit 110 controls so as to output a spike signal within the output time interval. For example, as shown in CASE1 in FIG. 5B, if the membrane potential v i (l) (t) is the threshold V th When the membrane potential v i (l) (t) is the threshold V thThe signal output unit 130 outputs a spike signal based on the determination result of the comparison unit 120. As a result, the signal output unit 130 determines that the membrane potential v i (l) (t) is the threshold V th When it reaches this value, a spike signal is output.
[0062] In this case, the comparison unit 120 also detects the membrane potential v i (l) (t) is the threshold V th After determining that the membrane potential v i (l) (t) may be calculated as 0. This allows the index value calculation unit 110 to process the next data in the next input time interval based on the membrane potential v i (l) It is also possible to start from a state in which (t) is reset to 0.
[0063] As shown in CASE2 in FIG. 5B, by the end of the output time period, the membrane potential v i (l) (t) is the threshold V th If the index value calculation section 110 or the comparison section 120 does not reach the threshold value, the index value calculation section 110 or the comparison section 120 may cause the signal output section 130 to output a spike signal at the end of the output time period. The output of the spike signal in this case is also called forced firing. For example, by the end of the output time interval, the membrane potential v i (l) (t) never reaches the threshold V th If the membrane potential does not reach the threshold, the comparison section 120 may output a dummy determination result indicating that the membrane potential has reached the threshold. Then, the signal output section 130 may output a spike signal at the end of the output time interval based on the determination result of the comparison section 120.
[0064] By the end of the output time interval, the membrane potential v i (l) (t) is the threshold V th If it does not reach v, the index value calculation unit 110 calculates the membrane potential v at the start of the next input time interval. i(l) (t) may be set to 0. This allows the index value calculation unit 110 to process the next data in the next input time interval based on the membrane potential v i (l) It is also possible to start from a state in which (t) is reset to 0.
[0065] Fig. 6 is a diagram showing an example of setting a time interval. The horizontal axis of Fig. 6 indicates time, which is the elapsed time from the start of data input to the neural network 11. Input and output of data between layers is performed by input and output of spike signals. In the example of FIG. 6, the time shown on the horizontal axis is divided into time intervals of time T. FIG. 6 also shows the types of time intervals for each of the input layer, first layer, second layer, and output layer. Specifically, the input time interval is indicated by the notation "input," and the output time interval is indicated by the notation "output." Furthermore, the input / output time interval is indicated by the notation "input" and "output." 6 shows an example in which the neural network 11 processes three pieces of data: first data, second data, and third data. For each layer, a time interval during which each piece of data is processed in that layer is shown.
[0066] In the example of FIG. 6, each of the neuron models 100 in the first, second and third layers operates to complete data processing within one input time interval, one output time interval following the input time interval and the data processing time. Specifically, each neuron model 100 completes the membrane potential v i (l) (t) is the threshold V th If it does not reach, a spike signal is output and the process ends. This allows the neuron model 100 to process the next data in the next data processing time. In the entire neural network 11, like a pipeline process, the processing of the next data can be started without waiting for the processing of the input data to be completed.
[0067] 6, the neural network device 10 synchronizes between layers and between the neuron models 100 in each layer so that the output time interval of the layer that outputs the spike signal coincides with the input time interval of the layer that receives the input of the spike signal. That is, the neural network device 10 synchronizes between layers and between the neuron models 100 in each layer so that the output time interval of the layer that outputs the spike signal completely overlaps with the input time interval of the layer that receives the input of the spike signal.
[0068] The neural network device 10 may include a synchronization processor that notifies each neuron model 100 of the timing of switching the time interval. Alternatively, each neuron model 100 may detect the timing of switching the time interval based on a clock signal common to all the neuron models 100.
[0069] In the example of FIG. 6, the process in which the i-th neuron model 100 in the l-th layer outputs a spike signal in the output time interval in the processing of the first data can be expressed as in equation (6).
[0070]
number
[0071] The expression clip is expressed as in expression (7).
[0072]
number
[0073] The function clip shown in equation (7) is called a clip function. Fig. 7 is a diagram showing a first example of a clip function when the first layer neuron model 100 processes the first data in the example of Fig. 6. The horizontal axis of the graph in Figure 7 is the membrane potential v i (1) (t) is the threshold Vth 7 corresponds to the horizontal axis in FIG. 6. The vertical axis indicates the time when the neuron model 100 outputs a spike signal, as the elapsed time from the start of data input to the neural network 11. Both the horizontal axis and the vertical axis in FIG. 7 correspond to the horizontal axis in FIG. 6.
[0074] As shown in FIG. 7, the threshold arrival time t i (l,vth) is earlier than time T, the neuron model 100 outputs a spike signal at time T. Time T is the start time of the output time interval. Threshold arrival time t i (l,vth) If is within the interval from time T to 2T, the neuron model 100 determines whether the threshold is reached at time t i (l,vth) The time 2T is the end time of the output time section. Note that the membrane potential v i (1) (t) is the threshold V th It is assumed that the delay time from when the input current Vp reaches Vt1 until when the neuron model 100 outputs a spike signal can be ignored. Threshold arrival time t i (l,vth) is later than time 2T, i.e., by time 2T, the membrane potential v i (1) (t) is the threshold V th If it does not reach 2T, the neuron model 100 outputs a spike signal at time 2T.
[0075] In the example of FIG. 6, the process in which the i-th neuron model 100 in the l-th layer outputs a spike signal in the application time interval in processing the d-th data can be expressed as in equation (8).
[0076]
number
[0077] Fig. 8 is a diagram showing a second example of the clip function. Fig. 8 generalizes the example of Fig. 7. Specifically, Fig. 8 shows an example of the clip function when the l-th layer neuron model 100 processes the d-th data in the example of Fig. 6. The horizontal axis of the graph in Figure 8 is the membrane potential v i (l) (t) is the threshold V th 8 corresponds to the horizontal axis in FIG. 6. The vertical axis indicates the time when the neuron model 100 outputs a spike signal, as the elapsed time from the start of data input to the neural network 11. Both the horizontal axis and the vertical axis in FIG. 8 correspond to the horizontal axis in FIG. 6.
[0078] As shown in FIG. 8, the threshold arrival time t i (l,vth) is earlier than time (2d+l-2)T, the neuron model 100 outputs a spike signal at time T. Time (2d+l-2)T is the start time of the output time interval. Threshold arrival time t i (l,vth) is within the interval from time (2d+l-2)T to (2d+l-1)T, the neuron model 100 determines whether the threshold is reached at time t i (l,vth) The time (2d+l-1)T is the end time of the output time interval. Note that the membrane potential v i (l) (t) is the threshold V th It is assumed that the delay time from when the input signal reaches the output terminal Vcc until when the neuron model 100 outputs a spike signal can be ignored. Threshold arrival time t i (l,vth) is later than time (2d+l-1)T, that is, by time (2d+l-1)T, the membrane potential v i (1) (t) is the threshold V th If not, the neuron model 100 outputs a spike signal at time (2d+l−1)T.
[0079] The clip function in the neuron model 100 can be considered to be equivalent to an activation function in an artificial neural network (ANN). Therefore, in the neural network 11, the concept of an activation function can be considered to be introduced into a spiking neural network. In this respect, the neural network device 10 is expected to exhibit high recognition performance. Moreover, the clip function in the neuron model 100 may be the subject of learning.
[0080] Fig. 9 is a diagram showing an example of a system configuration during learning. In the configuration shown in Fig. 9, the neural network system 1 includes a neural network device 10 and a learning device 50. The neural network device 10 and the learning device 50 may be integrated into one device. Alternatively, the neural network device 10 and the learning device 50 may be configured as separate devices. As described above, the neural network 11 formed by the neural network device 10 is also called the neural network main body.
[0081] FIG. 10 is a diagram showing an example of signal input and output in the neural network system 1. As shown in FIG. 10, input data and a teacher label indicating a correct answer to the input data are input to the neural network system 1. The neural network device 10 may receive the input of the input data, and the learning device 50 may receive the input of the teacher label. The combination of the input data and the teacher label corresponds to an example of training data in supervised learning. Furthermore, the neural network device 10 obtains a clock signal. The neural network device 10 may be provided with a clock circuit. Alternatively, the neural network device 10 may receive an input of a clock signal from outside the neural network device 10.
[0082] The neural network device 10 receives input data and outputs an estimated value based on the input data. When calculating the estimated value, the neural network device 10 uses a clock signal to synchronize the time intervals between layers and between the neuron models 100 in the same layer.
[0083] The learning device 50 performs learning of the neural network device 10. Learning here refers to adjusting the parameter values of the learning model by machine learning. The learning device 50 performs learning of the weighting coefficients for the spike signals input to the neuron model 100. The weighting W ij (l) corresponds to an example of a weighting coefficient whose value is adjusted by the learning device 50 through learning.
[0084] The learning device 50 may use an evaluation function indicating an evaluation of the error between the estimated value output by the neural network device 10 and the correct value indicated by the teacher label to learn the weighting coefficient so as to reduce the magnitude of the error between the estimated value and the correct value. The learning device 50 corresponds to an example of the learning means. The learning device 50 is configured using, for example, a computer.
[0085] The learning method performed by the learning device 50 can be, for example, an existing learning method such as backpropagation, which is arranged in accordance with the input time interval and output time interval that are set. For example, when the learning device 50 performs learning using a method that is an arrangement of the backpropagation algorithm, the weight W ij (l) The change ΔW shown in equation (9A) ij (l) The weight W ij (l) Similarly, the weight Wi (l) The change amount ΔWi shown in equation (9B) (l) The weights Wi (l) may be updated.
[0086]
number
[0087] Here, η is a constant indicating a learning rate. The learning rates in formula (9A) and formula (9B) may be the same or different. C is expressed as in equation (10).
[0088]
number
[0089] C corresponds to an example of an evaluation function indicating an evaluation of the error between the estimated value output by the neural network device 10 and the correct value indicated by the teacher label. C is set as a loss function that outputs a smaller value as the error becomes smaller. M represents the index value indicating the output layer (final layer). (M) represents the number of neuron models 100 included in the output layer.
[0090] κ i Here, the neural network device 10 calculates the number of classes as N (M) We assume that we perform classification into classes, and the teacher labels are represented by one-hot vectors. If the value of index i indicates the correct class, then κ i = 1, otherwise kappa i =0.
[0091] t (ref) represents the Reference Spike. "γ / 2(t i (M) -t (ref) ) 2" is a term provided to avoid learning difficulty. This term is also called the temporal penalty term. γ is a constant for adjusting the degree of influence of the temporal penalty term, and γ>0. γ is also called the temporal penalty coefficient. S i is a softmax function and is expressed as equation (11).
[0092]
number
[0093] σ soft is the softmax function S i is a constant set as a scale factor to adjust the value of σ soft >0. Here, the i-th spike firing time t i (M) 0≦t i (M) For example, the spike firing time of the output layer may indicate the probability that the classification object indicated by the input data is classified into the class. i For i = 1, t i (M) The closer the value of is to 1, the greater the "-Σ i=1 N(M) (κ i ln(S i (t (M) )))” term becomes smaller, and the learning device 50 calculates a smaller loss (the value of the evaluation function C). However, the processing performed by the neural network device 10 is not limited to class classification.
[0094] Weight W ij (l) and W i (l)In addition to or instead of the above, the clip function in the neuron model 100 may be the object of learning. For example, "a" and "b" in formula (7) may be treated as learning parameters, and the values of "a" and "b" may be adjusted by learning. Furthermore, for example, the learning device 50 may determine the time widths of the input time interval and the output time interval by learning, and set the values of "a" and "b" according to the obtained time widths. In FIG. 6, time T corresponds to an example of the time widths of the input time interval and the output time interval.
[0095] FIG. 11 is a diagram showing an example of signal input and output in the neural network device 10 during operation. 10, during learning, the neural network device 10 receives input data and acquires a clock signal. The neural network device 10 may be provided with a clock circuit. Alternatively, the neural network device 10 may receive a clock signal input from outside the neural network device 10. The neural network device 10 receives input data and outputs an estimated value based on the input data. When calculating the estimated value, the neural network device 10 uses a clock signal to synchronize the time intervals between layers and between the neuron models 100 in the same layer.
[0096] Next, the results of an experiment using the neural network device 10 will be described. We trained and tested the recognition of handwritten digit images using MNIST, a dataset of handwritten digit images. The neural network 11 was configured as a fully connected forward propagation type having four layers: an input layer, a first layer, a second layer, and an output layer. The number of neuron models 100 in the input layer was 784, the number of neuron models 100 in the first layer and the second layer was 200 each, and the number of neuron models 100 in the output layer was 10.
[0097] The time width of both the input time interval and the output time interval was set to 0.05 milliseconds. As a method for displaying the results of estimation by the neural network 11, a time method is adopted in which the results of estimation are displayed based on which neuron model 100 among the neuron models 100 in the output layer outputs a spike signal first. As a result of the experiment, a recognition rate of 97.4% was achieved during testing.
[0098] Fig. 12 is a diagram showing an example of the threshold reaching time during learning. Fig. 12 shows an example of the threshold reaching time in the case where the index value calculation section 110 does not limit the time for calculating the membrane potential. 12, the horizontal axis indicates time as the elapsed time from the start of data input to the neural network 11. The vertical axis indicates the node numbers in each of the input layer, the first layer, the second layer, and the output layer.
[0099] In the example of Fig. 12, among the neuron models 100 in the first layer, there is a neuron model 100 whose membrane potential reaches the threshold value after 0.010 milliseconds, which is the end time of the output time interval of the first layer. For example, there is a neuron model 100 whose membrane potential reaches the threshold value around 0.025 milliseconds. If we were to wait for the membrane potential of this neuron model 100 to reach a threshold, a spike signal for processing the next input data cannot be input to the first layer neuron model 100 until around 0.025 milliseconds.
[0100] As for the second layer, there are some neuron models 100 whose membrane potential reaches the threshold value after 0.015 milliseconds, which is the end of the output time interval of the first layer. For example, there is a neuron model 100 whose membrane potential reaches the threshold value at around 0.019 milliseconds. If we were to wait for the membrane potential of this neuron model 100 to reach a threshold, a spike signal for processing the next input data cannot be input to the second layer neuron model 100 until approximately 0.019 milliseconds.
[0101] Alternatively, if we were to continue calculating the membrane potential in each of the first layer neuron models 100 and each of the second layer neuron models 100 until the neural network 11 outputs an estimated result, a spike signal for processing the next input data cannot be input to any of the first layer, second layer, and output layer neuron models 100 until around 0.013 milliseconds, when the membrane potential of the output layer neuron model 100 first reaches the threshold.
[0102] FIG. 13 is a diagram showing an example of firing times during learning. 13, the horizontal axis indicates time as the elapsed time from the start of data input to the neural network 11. The vertical axis indicates the node numbers in each of the input layer, the first layer, the second layer, and the output layer. In the example of Fig. 13, all the neuron models 100 in the first layer output spike signals during the output time interval of the first layer, from 0.005 seconds to 0.010 seconds. As a result, after 0.010 seconds, spike signals for processing the next input data can be input to the neuron models 100 in the first layer.
[0103] In addition, since the firing time of the neuron model 100 in the first layer is limited to 0.005 seconds or later, the start time of the input time interval in the second layer can also be set to 0.005 seconds. If there is data before the data to be processed in the example of Fig. 13, the neuron model 100 in the second layer can process the previous data up to 0.005 seconds.
[0104] In addition, all neuron models 100 in the second layer output spike signals during the output time interval of the second layer, from 0.010 seconds to 0.015 seconds. This allows the neuron models 100 in the second layer to input spike signals for processing the next input data after 0.015 seconds.
[0105] In addition, since the firing time of the neuron model 100 of the second layer is limited to 0.010 seconds or later, the start time of the input / output time interval of the output layer can also be set to 0.010 seconds. If there is data before the data to be processed in the example of Fig. 13, the neuron model 100 of the output layer can process the previous data up to 0.010 seconds.
[0106] In the experiment, the timing at which the neuron model 100 in the output layer outputs a spike signal is not limited. Therefore, among the neuron models 100 in the output layer, there is a neuron model 100 that outputs a spike after 0.020 milliseconds, which is the end of the output time interval of the output layer. By making the neuron model 100 in the output layer also fire within the output time interval, a spike signal for processing the next input data can be input to the neuron model 100 in the output layer after 0.020 milliseconds, which is the end of the output time interval.
[0107] Fig. 14 is a diagram showing an example of the threshold reaching time during a test, in which the index value calculation section 110 does not limit the time for calculating the membrane potential. 14, the horizontal axis indicates time as the elapsed time from the start of data input to the neural network 11. The vertical axis indicates the node numbers in each of the input layer, the first layer, the second layer, and the output layer.
[0108] In the example of Fig. 14, among the neuron models 100 in the first layer, there is a neuron model 100 whose membrane potential reaches the threshold value after 0.010 milliseconds, which is the end time of the output time interval of the first layer. For example, there is a neuron model 100 whose membrane potential reaches the threshold value around 0.025 milliseconds. If we were to wait for the membrane potential of this neuron model 100 to reach a threshold, a spike signal for processing the next input data cannot be input to the first layer neuron model 100 until around 0.025 milliseconds.
[0109] As for the second layer, there are some neuron models 100 whose membrane potential reaches the threshold value after 0.015 milliseconds, which is the end of the output time interval of the first layer. For example, there is a neuron model 100 whose membrane potential reaches the threshold value at around 0.022 milliseconds. If we were to wait for the membrane potential of this neuron model 100 to reach a threshold, a spike signal for processing the next input data cannot be input to the second layer neuron model 100 until approximately 0.022 milliseconds.
[0110] Alternatively, if we were to continue calculating the membrane potential in each of the first layer neuron models 100 and each of the second layer neuron models 100 until the neural network 11 outputs an estimated result, a spike signal for processing the next input data cannot be input to any of the first layer, second layer, and output layer neuron models 100 until around 0.013 milliseconds, when the membrane potential of the output layer neuron model 100 first reaches the threshold.
[0111] FIG. 15 is a diagram showing an example of firing times during a test. 15, the horizontal axis indicates time as the elapsed time from the start of data input to the neural network 11. The vertical axis indicates the node numbers in each of the input layer, the first layer, the second layer, and the output layer. In the example of Fig. 15, all the neuron models 100 in the first layer output spike signals during the output time interval of the first layer, from 0.005 seconds to 0.010 seconds. As a result, after 0.010 seconds, spike signals for processing the next input data can be input to the neuron models 100 in the first layer.
[0112] In addition, since the firing time of the neuron model 100 in the first layer is limited to 0.005 seconds or later, the start time of the input time interval in the second layer can also be set to 0.005 seconds. If there is data before the data to be processed in the example of Fig. 15, the neuron model 100 in the second layer can process the previous data up to 0.005 seconds.
[0113] In addition, all neuron models 100 in the second layer output spike signals during the output time interval of the second layer, from 0.010 seconds to 0.015 seconds. This allows the neuron models 100 in the second layer to input spike signals for processing the next input data after 0.015 seconds.
[0114] In addition, since the firing time of the neuron model 100 of the second layer is limited to 0.010 seconds or later, the start time of the input / output time interval of the output layer can also be set to 0.010 seconds. If there is data before the data to be processed in the example of Fig. 15, the neuron model 100 of the output layer can process the previous data up to 0.010 seconds.
[0115] By making the neuron model 100 in the output layer also fire within the output time interval, a spike signal for processing the next input data can be input to the neuron model 100 in the output layer after 0.020 milliseconds, which is the end of the output time interval.
[0116] According to the embodiment, the neural network device 10 is divided into an input time interval in which spikes are received and an output time interval in which spikes are fired and transmitted. The neural network device 10 includes a neuron model 100 that fires in the output time interval and is restricted from firing in the input time interval. This allows the input time interval and the output time interval to be used in their respective modes, enabling the spiking neural network to efficiently process data. For example, by making the neuron model 100 fire within a certain time interval to transmit spikes, the trailing edge of the output time interval becomes clear, and it becomes easy to distinguish between the input time interval in which spikes are received and the output time interval in which spikes are permitted to be transmitted. This makes it easy to adjust the neuron model 100 that outputs relatively significant information so that it uses the information without leaking it. By configuring the neuron model 100 that outputs relatively significant information to fire within the output time interval, the spiking neural network can efficiently process data. Also, the neuron model 100 that outputs relatively insignificant information may be configured to omit firing. For example, the neuron model 100 may be configured to be forced to fire when the membrane potential of the neuron model 100 satisfies a predetermined condition at the trailing edge of the output time interval, or not to fire in the same case. This prevents the neuron model 100 from firing after the output time interval has elapsed. This allows the spiking neural network to process data more efficiently.
[0117] The firing time of the neuron model 100 in the output time interval may be defined as a function of the membrane potential of the neuron model 100 at the last time of the input time interval. The neuron model 100 may be formed so as to limit firing when the membrane potential of the neuron model 100 at the last time of the input time interval is outside a predetermined range. The neuron model 100 may be formed so that the membrane potential of the neuron model 100 in the output time interval increases with a slope specific to the neuron model 100. The neuron model 100 generates a spike with a predetermined pulse waveform when the membrane potential of the neuron model 100 satisfies a predetermined condition in the output time interval. For example, the neuron model 100 may start transmitting a spike with a rectangular wave shape when the membrane potential of the neuron model 100 satisfies a predetermined condition in the output time interval, and may stop transmitting the spike at the time when the output time interval ends.
[0118] The type of the neural network 11 is not limited to a specific type. For example, the neural network 11 may be configured as a convolutional neural network (CNN) using a spiking neural network.
[0119] Second embodiment An application example to a CNN type neural network device 10A as the second embodiment will be described with reference to FIGS.
[0120] 21 is a diagram showing an example of the configuration of a neural network device 10A. The neural network device 10A includes a neural network 11A. The neural network 11A is configured as a CNN. For example, the neural network 11A includes, in order from the input side, an input layer 21, an intermediate layer 22, an output layer 23, and a feature extraction layer 24.
[0121] The feature extraction layer 24 is disposed between the input layer 21 and the intermediate layer 22. The feature extraction layer 24 is configured to extract desired features from input image data by calculation processing using the output of the input layer 21. The feature extraction layer 24 supplies the calculation results to the subsequent intermediate layer 22.
[0122] For example, the feature extraction layer 24 is configured to stack any number of pairs of convolution layers and pooling layers suitable for processing, combining a convolution layer (Conv) and a pooling layer (Pooling). The feature extraction layer 24 shown in FIG. 21 includes a first convolution layer (Conv) 241, a first pooling layer (Pooling) 251, a second convolution layer (Conv) 242, and a second pooling layer (Pooling) 252. The feature extraction layer 24 is an example equipped with two pairs of convolution layers and pooling layers. Each layer is configured by a neuron model 100 that is activated according to local features in the image space.
[0123] The intermediate layer 22 includes, for example, a first fully connected layer 221 (Dense) and a second fully connected layer 222 (Dense). Note that the output layer 23 may be omitted.
[0124] An example of the results of an experiment using the neural network device 10A configured in this way is shown below. As input data for this experiment, a portion of the MNIST data is used. The data shown here is a model of a handwritten figure eight.
[0125] Fig. 22 is a diagram for explaining the result of the discrimination process. Fig. 23 is a diagram for explaining the output results of the first convolution layer 241 and the first pooling layer 251. Fig. 24 is a diagram for explaining the output results of the second convolution layer 242 and the second pooling layer 252. The value of each pixel in the images shown in Figs. 23 and 24 is obtained by converting the time when the spike of each layer was confirmed.
[0126] In order from the top of FIG. 22, the input layer 21 (input), the first convolution layer 241 (Conv), the first pooling layer 251 (Pooling), the second convolution layer 242 (Conv), the second pooling layer 252 (Pooling), the first fully connected layer 221 (Dense), and the second fully connected layer 222 (Dense) each have a membrane potential v i (l) (t) is the threshold V th The timing at which the
[0127] For example, the size of the two-dimensional image to be classified is 28 pixels x 28 pixels. The first convolution layer 241 forms a convolution filter of size 5x5 and has 6 channels. The first pooling layer 251 performs maximum pooling on the processing result of the first convolution layer 241 in units of size 2x2. The second convolution layer 242 forms a convolution filter of size 5x5 and has 16 channels. The second pooling layer 252 performs maximum pooling on the processing result of the second convolution layer 242 in units of size 2x2. The number of neuron models 100 in the first fully connected layer 221 is 200, and the number of neuron models 100 in the second fully connected layer 222 is 10.
[0128] 22 shows the occurrence of firing of the neuron model 100 in each layer, with a circle marking the firing time. It was confirmed that the timing of firing of each neuron model 100 in each layer was appropriately distributed within the output time interval, and it was confirmed that the neuron model 100 corresponding to the index value 8 fired the earliest as a result of the calculation of the second fully connected layer 222.
[0129] The results are shown in Figure 23 and Figure 24 (a) to (f). The results also show that activation was relatively strong in the diagonally upward direction along the trajectory of the number 8. As a result of this experiment, a recognition rate of 99.1% was obtained during testing. In this way, the second embodiment also provides the same effects as the first embodiment.
[0130] (Modifications common to all embodiments) As described above, when the neural network 11 is configured as a forward propagation spiking neural network, the number of layers of the neural network 11 may be two or more, and is not limited to a specific number of layers. Also, the number of neuron models 100 included in each layer is not limited to a specific number, and each layer may have one or more neuron models 100. Each layer may have the same number of neuron models 100, or each layer may have a different number of neuron models 100. Also, the neural network 11 may be a fully connected type, or may not be a fully connected type.
[0131] In addition, the membrane potential after firing of the neuron model 100 is not limited to the one that does not change from the above-mentioned potential 0. For example, after a predetermined time has elapsed from firing, the membrane potential may be changed according to the input of a spike signal. The number of firings of each neuron model 100 is also not limited to once for each input data.
[0132] The configuration of the neuron model 100 as a spiking neuron model is not limited to a specific configuration. For example, the rate of change of the neuron model 100 from when it receives the input of a spike signal until when it receives the input of the next spike signal does not have to be constant. The learning method of the neural network 11 is not limited to supervised learning, but the learning device 50 may also learn the neural network 11 by unsupervised learning.
[0133] As described above, the index value calculation unit 110 changes the membrane potential over time based on the input state of the signal in the input time interval. The signal output unit 130 outputs a signal within an output time interval after the end of the input time interval based on the membrane potential. In this way, by setting the input time interval during which the neuron model 100 accepts the input of a spike signal and the output time interval during which the neuron model 100 outputs a spike signal, the time during which the index value calculation unit 110 should calculate the membrane potential can be limited to the time from the start of the input time interval to the end of the output time interval. During other times, the neuron model 100 can process other data. In this respect, the neural network device 10 allows the spiking neural network to efficiently process data.
[0134] Moreover, the index value calculation unit 110 changes the membrane potential at a rate of change according to the input state of the signal during the input time interval. If the membrane potential does not reach the threshold within the input time interval, the index value calculation unit 110 changes the membrane potential during the output time interval at the rate of change at the end of the input time interval. If the membrane potential reaches the threshold within the input time interval, the signal output unit 130 outputs a spike signal at the start of the output time interval. If the membrane potential reaches the threshold within the output time interval, the signal output unit 130 outputs a spike signal when the membrane potential reaches the threshold. If the membrane potential does not reach the threshold within the output time interval, the signal output unit 130 outputs a spike signal at the end of the output time interval.
[0135] This makes it possible to limit the time during which the neuron model 100 outputs a spike signal to the output time interval. After the output time interval ends, the neuron model 100 can process other data. In this respect, the neural network device 10 allows the spiking neural network to efficiently process data.
[0136] In addition, the output time interval and the input time interval of the first neuron model 100 are set so as to overlap with the input time interval of the second neuron model that receives the spike signal input from the first neuron model 100. This allows data to be efficiently transmitted from the first neuron model 100 to the second neuron model 100 via spike signals, and the first neuron model 100 and the second neuron model 100 can perform processing in a manner similar to pipeline processing. In this respect, the neural network device 10 allows the spiking neural network to efficiently process data.
[0137] The index value calculation unit 110 changes the membrane potential over time based on the input state of the signal in the input time interval. The signal output unit 130 outputs a spike signal within an output time interval after the end of the input time interval based on the membrane potential. The learning device 50 learns the weighting coefficient for the spike signal. This makes it possible to adjust the weighting coefficients through learning, thereby improving the estimation accuracy of the neural network device 10.
[0138] Fig. 16 is a diagram showing an example of the configuration of a neural network device according to an embodiment. In the configuration shown in Fig. 11, a neural network device 610 includes a neuron model 611. The neuron model 611 includes an index value calculation unit 612 and a signal output unit 613. In this configuration, the neuron model 611 is configured to be able to transmit spikes by firing within a certain time interval. In the neural network device 610, an input time interval in which spikes are received and an output time interval in which spikes are permitted to be transmitted are separated in association with firing the neuron model 611. For example, the index value calculation unit 612 changes the index value of the signal output based on the input state of the signal in the input time interval. The signal output unit 613 outputs a signal within an output time interval after the end of the input time interval by firing based on the index value. The index value calculation unit 612 corresponds to an example of an index value calculation means, and the signal output unit 613 corresponds to an example of a signal output means.
[0139] In this way, by setting the input time interval during which the neuron model 611 accepts the input of a signal and the output time interval during which the neuron model 611 outputs a spike signal and firing within the output time interval, the time during which the index value calculation unit 612 should calculate the index value can be limited to the time from the start of the input time interval to the end of the output time interval. During other times, the neuron model 611 can process other data. In this respect, the neural network device 610 allows a spiking neural network to process data efficiently. In addition, if the layer is one in which the signal input and the signal output do not interfere with each other, the input time interval and the output time interval may be defined to overlap. For example, the above-mentioned output layer 23 is an example of a layer in which the signal input and the signal output do not interfere with each other. The neuron model 611 applied to such an output layer 23 may be set with an input / output time interval in which signal reception and transmission are permitted in association with firing. The input / output time interval in which signal reception and transmission are permitted may be set instead of the input time interval in which signal output is restricted.
[0140] 17 is a diagram showing an example of the configuration of a neuron model device according to an embodiment. In the configuration shown in FIG. 17, a neuron model device 620 includes an index value calculation unit 621 and a signal output unit 622. In this configuration, the neuron model device 620 is divided into an input time interval in which a signal is received and an output time interval in which a signal is permitted to be transmitted, in association with firing. The index value calculation unit 621 changes the index value of the signal output based on the input status of the signal in the input time interval. The signal output unit 622 outputs a signal within an output time interval after the end of the input time interval by firing based on the index value. The index value calculation unit 621 corresponds to an example of an index value calculation means, and the signal output unit 622 corresponds to an example of a signal output means.
[0141] In this way, by setting the input time interval during which the neuron model device 620 accepts the input of a signal and the output time interval during which the neuron model device 620 outputs a spike signal, the time during which the index value calculation unit 621 should calculate the index value can be limited to the time from the start of the input time interval to the end of the output time interval. During other times, the neuron model device 620 can process other data. In this respect, the neuron model device 620 allows spiking neural networks to process data efficiently.
[0142] FIG. 18 is a diagram illustrating an example of the configuration of a neural network system according to the embodiment. 18, neural network system 630 includes neural network body 631 and learning unit 635. Neural network body 631 includes neuron model 632. Neuron model 632 includes index value calculation unit 633 and signal output unit 634.
[0143] In this configuration, the neural network system 630 is divided into an input time interval in which a signal is received and an output time interval in which a signal is permitted to be transmitted, in association with firing the neuron model 632. The index value calculation unit 633 changes the index value of the signal output based on the input state of the signal in the input time interval. The signal output unit 634 outputs a signal within an output time interval after the end of the input time interval based on the index value. The learning unit 635 learns the weighting coefficient for the signal input to the neuron model 632.
[0144] The index value calculation unit 633 corresponds to an example of an index value calculation means, the signal output unit 634 corresponds to an example of a signal output means, and the learning unit 635 corresponds to an example of a learning means. Learn numbers. In this way, in the neural network system 630, the weighting coefficients can be adjusted by learning, and the estimation accuracy by the neural network main body 631 can be improved.
[0145] Fig. 19 is a flowchart showing an example of a processing procedure of a calculation method according to the embodiment. The calculation method shown in Fig. 19 includes identifying a division of a time interval (step S610), calculating an index value (step S611), and outputting a signal (step S612).
[0146] In identifying the division of the time interval (step S610), an input time interval in which a spike is received and an output time interval in which a spike is permitted to be transmitted are identified. For example, identifying the division of the time interval may include setting a flag according to the identification result. In calculating an index value (step S611), if the identification result indicates an input time interval, a signal input is permitted and an index value of the signal output is changed based on the input status of the signal in the input time interval. In outputting a signal (step S612), for example, a transition from an input time interval to an output time interval is detected based on the identification result. In outputting a signal (step S612), a signal is output within an output time interval after the end of the input time interval by firing based on the index value according to the identification result (flag value).
[0147] 19, an input time interval for receiving a signal input and an output time interval for outputting a signal are set, and firing occurs within the output time interval, thereby limiting the time for calculating the index value to the time from the start of the input time interval to the end of the output time interval. During other times, processing of other data can be performed. In this respect, the computational method shown in FIG. 19 allows the spiking neural network to process data efficiently.
[0148] FIG. 20 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 20, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, an interface 740, and a non-volatile recording medium 750.
[0149] Any one or more of the above-mentioned neural network device 10, learning device 50, neural network device 610, neuron model device 620, and neural network system 630, or a part thereof, may be implemented in the computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures a storage area corresponding to each of the above-mentioned storage units in the main storage device 720 according to the program. Communication between each device and other devices is executed by the interface 740 having a communication function and performing communication according to the control of the CPU 710.
[0150] When neural network device 10 is implemented in computer 700, the operations of neural network device 10 and each of its components are stored in the form of a program in auxiliary storage device 730. CPU 710 reads the program from auxiliary storage device 730, loads it into main storage device 720, and executes the above-mentioned processing in accordance with the program.
[0151] Furthermore, the CPU 710 allocates a memory area for processing of the neural network device 10 in the main memory device 720 in accordance with the program. Communication between the neural network device 10 and other devices is performed by an interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the neural network device 10 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.
[0152] When learning device 50 is implemented in computer 700, the operation of learning device 50 is stored in the form of a program in auxiliary storage device 730. CPU 710 reads the program from auxiliary storage device 730, loads it into main storage device 720, and executes the above-mentioned processing according to the program.
[0153] Furthermore, CPU 710 allocates a memory area in main memory device 720 for processing of learning device 50 in accordance with the program. Communication between learning device 50 and other devices is performed by interface 740, which has a communication function and operates under the control of CPU 710. Interaction between learning device 50 and a user is performed by interface 740, which has a display device and an input device, displaying various images under the control of CPU 710, and accepting user operations.
[0154] When the neural network device 610 is implemented in the computer 700, the operations of the neural network device 610 and each of its components are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program.
[0155] Furthermore, the CPU 710 allocates a storage area for processing of the neural network device 610 in the main memory device 720 in accordance with the program. Communication between the neural network device 610 and other devices is performed by an interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the neural network device 610 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.
[0156] When the neuron model device 620 is implemented in the computer 700, the operation of the neuron model device 620 and each part thereof is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program.
[0157] In addition, the CPU 710 secures a memory area for processing the neuron model device 620 in the main memory device 720 according to the program. Communication between the neuron model device 620 and other devices is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the neuron model device 620 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.
[0158] When the neural network system 630 is implemented in the computer 700, the operations of the neural network system 630 and each of its components are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing in accordance with the program.
[0159] Furthermore, the CPU 710, in accordance with the program, secures a storage area in the main storage device 720 for processing of the neural network system 630. Communication between the neural network system 630 and other devices is performed by an interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the neural network system 630 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.
[0160] In addition, the programs for executing all or part of the processes performed by the neural network device 10, the learning device 50, the neural network device 610, the neuron model device 620, and the neural network system 630 may be recorded in a computer-readable recording medium, and the programs recorded in the recording medium may be read into a computer system and executed to perform the processes of each part. Note that the "computer system" referred to here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The above-mentioned program may be for realizing part of the above-mentioned functions, or may be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.
[0161] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]
[0162] 1, 630 Neural Network System 10, 10A, 610 Neural network device 11, 11A Neural Networks 21 Input Layer 22 Middle Class 23 Output layer 24 Feature Extraction Layer 50 Learning Device 100, 611, 632 Neuron Model 110, 612, 621, 633 Index value calculation section 120 Comparison section 130, 613, 622, 634 Signal output section 620 Neuron Model Device 631 Neural Network Main Unit 635 Learning Department
Claims
1. A neuron model in which an input time interval for receiving spikes and an output time interval for firing and transmitting spikes are separated, firing occurs within the output time interval, and firing within the input time interval is restricted. A computing device comprising:
2. The firing time at which the neuron model fires within the output time interval is a function of the membrane potential of the neuron model at the last time of the input time interval. The computing device of claim 1 .
3. The neuron model limits firing when a membrane potential of the neuron model at the last time of the input time interval is outside a predetermined range. The computing device according to claim 2 .
4. The neuron model is The membrane potential of the neuron model within the output time interval is formed to increase with a slope specific to each of the neuron models. The computing device of claim 1 .
5. The neuron model is The neuron model is configured to be forced to fire when the membrane potential of the neuron model satisfies a predetermined condition at the trailing edge of the output time interval, or not to fire in the same case. The computing device according to claim 4.
6. The neuron model is when the membrane potential of the neuron model satisfies a predetermined condition within the output time interval, transmission of the spike having a rectangular wave shape is started, and transmission of the spike is interrupted at the time when the output time interval ends. The computing device according to claim 4.
7. The neural network includes a main body and a learning means. In relation to firing, an input time interval during which a signal is received and an output time interval during which the signal is permitted to be transmitted are separated, an input / output time during which a signal is permitted to be received and transmitted is set in association with firing of a part of layers among a plurality of layers constituting the neural network body; The neural network body includes: an index value calculation means for changing an index value of a signal output based on an input state of a signal in an input time section; a signal output means for outputting a signal within an output time interval following the end of the input time interval based on the index value; A neuron model with Equipped with The learning means learns a weighting factor for the signal. Neural network system.
8. In association with the forced firing, an input time interval in which a spike is received and an output time interval in which a spike is permitted to be transmitted are separated, an index value calculation unit that changes an index value of a signal output based on an input state of the signal in the input time section; a signal output unit that transmits a spike within the output time interval by firing based on an index value of the signal output; A neuron model device comprising:
9. Identifying an input time interval during which a spike is received and an output time interval during which a spike is permitted to be transmitted, the input time interval being associated with a forced firing; Varying an index value of a signal output based on an input state of the signal in the input time section; transmitting a spike within the output time interval by firing based on an index value of the signal output; A computational method executed by a device in which a neural network is implemented.
10. A device that executes a program Identifying an input time interval during which a spike is received and an output time interval during which a spike is permitted to be transmitted, the input time interval being associated with a forced firing; Varying an index value of a signal output based on an input state of the signal in the input time section; transmitting a spike within the output time interval by firing based on an index value of the signal output; A program for executing the above.
Citation Information
Patent Citations
Network traversal using neuromorphic instantiations of spike-timing dependent plasticity
JP2018136919A
Neural network circuit learning method
WO2013111200A1
