Computing device, neural network system, neuron model device, computing method, and program

Abstract

To allow a spiking neural network to efficiently perform data processing.SOLUTION: An operation device has a spiking neural model including: index value calculation means for changing the index value of a signal output on the basis of the input status of a signal for a time interval according to each time interval; detection means for detecting a generation timing of a predetermined event about the index value; and signal output means for outputting a signal at a timing in a first time interval of the time intervals and also at a timing according to the generation timing of the predetermined event in a second time interval before the first time interval.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an arithmetic device, a neural network system, a neuron model device, an arithmetic method, and a program.

Background Art

[0002] One type of neural network is a spiking neural network (SNN). For example, Patent Document 1 describes a neuromorphic computing system that implements a spiking neural network on a neuromorphic computing device. In a spiking neural network, a neuron model has an internal state called a membrane potential and outputs a signal called a spike based on the temporal evolution of the membrane potential.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is preferable that a spiking neural network can perform data processing efficiently.

[0005] An example of an object of the present invention is to provide an arithmetic device, a neural network system, a neuron model device, an arithmetic method, and a program that can solve the above-described problems.

Means for Solving the Problems

[0006] According to a first aspect of the present invention, an arithmetic device includes an index value calculation means for changing an index value of signal output based on an input status of a signal for each time interval, a detection means for detecting a timing of occurrence of a predetermined event related to the index value, and a signal output means for outputting a signal at a timing within a first time interval of the time intervals and at a timing corresponding to the timing of occurrence of the predetermined event in a second time interval which is a time interval earlier than the first time interval, and includes a spiking neuron model.

[0007] According to a second aspect of the present invention, a neural network system includes a spiking neural network main body and a learning means. The spiking neural network main body includes an index value calculation means for changing an index value of signal output based on an input status of a signal for each time interval, a detection means for detecting a timing of occurrence of a predetermined event related to the index value, and a signal output means for outputting a signal at a timing within a first time interval of the time intervals and at a timing corresponding to the timing of occurrence of the predetermined event in a second time interval which is a time interval earlier than the first time interval, and includes a spiking neuron model. The learning means performs learning of a weight coefficient for the signal.

[0008] According to a third aspect of the present invention, a neuron model device includes an index value calculation means for changing an index value of signal output based on an input status of a signal for each time interval, a detection means for detecting a timing of occurrence of a predetermined event related to the index value, and a signal output means for outputting a signal at a timing within a first time interval of the time intervals and at a timing corresponding to the timing of occurrence of the predetermined event in a second time interval which is a time interval earlier than the first time interval.

[0009] According to a fourth aspect of the present invention, the calculation method includes, for each time interval, changing an index value of signal output based on an input status of a signal in that time interval, detecting a timing of occurrence of a predetermined event regarding the index value, and outputting a signal at a timing corresponding to a timing within a first time interval among the time intervals and a timing of occurrence of the predetermined event in a second time interval that is a time interval prior to the first time interval.

[0010] According to a fifth aspect of the present invention, the program is a program for causing a programmable device to, for each time interval, change an index value of signal output based on an input status of a signal in that time interval, detect a timing of occurrence of a predetermined event regarding the index value, and output a signal at a timing corresponding to a timing within a first time interval among the time intervals and a timing of occurrence of the predetermined event in a second time interval that is a time interval prior to the first time interval.

Advantages of the Invention

[0011] According to the present invention, a spiking neural network can efficiently perform data processing.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described. However, the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution means of the invention. FIG. 1 is a diagram showing an example of the configuration of a neural network device according to an embodiment. In the configuration shown in FIG. 1, the neural network device 10 includes a neuron model 100. The neuron model 100 includes an index value calculation unit 110, a detection unit 120, a delay unit 130, and a signal output unit 140.

[0014] The neural network device 10 performs data processing using a spiking neural network. Data processing is also referred to as an operation. The neural network device 10 corresponds to an example of an arithmetic device. The neural network device referred to here is a device in which a neural network is implemented. A spiking neural network may be implemented in the neural network device 10 using dedicated hardware. For example, the spiking neural network may be implemented in the neural network device 10 using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Alternatively, the spiking neural network may be implemented in the neural network device 10 software-wise using a computer or the like.

[0015] Any of the device equipped with an ASIC, the device equipped with an FPGA, and the computer falls within the example of a programmable device. In the case of an ASIC and an FPGA, describing hardware using a hardware description language and implementing the described hardware on the ASIC or FPGA corresponds to an example of programming. When the neural network device 10 is configured using a computer, the function of the spiking neural network may be described by programming and the obtained program may be executed by the computer.

[0016] The spiking neural network referred to here is a neural network in which a neuron model outputs a signal at a timing based on a state quantity called a membrane potential, which changes over time according to the input situation of signals to the neuron model itself. The membrane potential is also referred to as an index value for signal output, or simply an index value. The change over time referred to here means changing according to time.

[0017] The neuron model in the spiking neural network is also referred to as the spiking neuron model. The signal output by the spiking neuron model is also referred to as a spike signal or a spike. In a spiking neural network, a binary signal can be used as the spike signal, and information transmission between spiking neuron models can be performed according to the transmission timing of the spike signal or the number of spike signals, etc. In the case of the neuron model 100, the index value calculation unit 110 calculates the membrane potential based on the input situation of the spike signal to the neuron model 100. The signal output unit 140 outputs a spike signal at a timing corresponding to the temporal change of the membrane potential.

[0018] As the spike signal in the neural network device 10, a pulse signal or a step signal may be used, but it is not limited thereto. Hereinafter, as an information transmission method between neuron models 100 in the spiking neural network by the neural network device 10, a time method of transmitting information at the transmission timing of the spike signal is used.

[0019] The processing performed by the neural network device 10 can be various processes executable using a spiking neural network. For example, the neural network device 10 may perform image recognition, biometric authentication, or numerical prediction, but is not limited thereto.

[0020] The neural network device 10 may be configured as one device, or may be configured by combining a plurality of devices. For example, each neuron model 100 may be configured as a device, and a spiking neural network may be configured by connecting the devices constituting these individual neuron models 100 through a signal transmission path.

[0021] FIG. 2 is a diagram showing an example of the configuration of the spiking neural network included in the neural network device 10. The spiking neural network included in the neural network device 10 is also denoted as the neural network 11. The neural network 11 is also referred to as the neural network main body.

[0022] In the example of FIG. 2, the neural network 11 is configured as a feedforward four-layer spiking neural network. Specifically, the neural network 11 includes an input layer 21, two intermediate layers 22-1 and 22-2, and an output layer 23. The two intermediate layers 22-1 and 22-2 are collectively denoted as the intermediate layer 22. The intermediate layer is also referred to as the hidden layer. The input layer 21, the intermediate layer 22, and the output layer 23 are collectively denoted as the layer 20.

[0023] The input layer 21 includes input nodes 31. The intermediate layer 22 includes intermediate nodes 32. The output layer 23 includes output nodes 33. The input nodes 31, the intermediate nodes 32, and the output nodes 33 are collectively denoted as nodes 30.

[0024] The input nodes 31, for example, convert the input data to the neural network 11 into spike signals. Alternatively, when the input data to the neural network 11 is represented by spike signals, a neuron model 100 may be used as the input nodes 31.

[0025] As the intermediate nodes 32 and the output nodes 33, the neuron model 100 may be used for both. Also, in the output nodes 33, the operation of the neuron model 100 may be different between the intermediate nodes 32 and the output nodes 33, such as the constraint conditions for the spike signal output timing described later being relaxed compared to the case of the intermediate nodes.

[0026] The four layers 20 of the neural network 11 are arranged in the order of the input layer 21, the intermediate layer 22-1, the intermediate layer 22-2, and the output layer 23 from the upstream side in signal transmission. Between two adjacent layers 20, nodes 30 are connected by transmission paths 40. The transmission path 40 transmits a spike signal from the node 30 of the upstream layer 20 to the node 30 of the downstream layer 20.

[0027] However, when the neural network 11 is configured as a forward propagation type spiking neural network, the number of layers is not limited to four and may be two or more. Also, the number of neuron models 100 provided in each layer is not limited to a specific number, and each layer may include one or more neuron models 100. Each layer may have the same number of neuron models 100, or may have different numbers of neuron models 100 depending on the layer. Also, the destination of the spike signal from each layer is not limited to the next layer. The output of a certain layer may be transmitted to a layer further downstream across an arbitrary number of layers.

[0028] Also, the neural network 11 may be configured as a fully connected type, but is not limited to this. In the example of FIG. 2, all neuron models 100 of the upstream layer 20 and all neuron models 100 of the downstream layer 20 in adjacent layers may be connected by the transmission path 40, but there may be neuron models 100 in adjacent layers that are not connected by the transmission path 40.

[0029] Hereinafter, it is assumed that the delay time in the transmission of the spike signal can be ignored, and the spike signal output time of the neuron model 100 on the spike signal output side and the spike signal input time to the neuron model 100 on the spike signal input side are the same. When the delay time in the transmission of the spike signal cannot be ignored, the time obtained by adding the delay time to the spike signal output time may be used as the spike signal input time.

[0030] In a general spiking neuron model, the output timing of spike signals is not restricted, and spike signals are output at the timing when the membrane potential that changes over time reaches the threshold. In a spiking neural network based on a spiking neuron model in which the output timing of spike signals is not restricted, when there are multiple data to be processed, it is necessary to wait for the input of the next input data to the spiking neural network until the spiking neural network receives the input of the input data and outputs the calculation result.

[0031] Figure 3 is a diagram showing an example of the temporal change of the membrane potential in a spiking neuron model in which the output timing of spike signals is not restricted. The horizontal axis of the graph in Figure 3 indicates time. The vertical axis indicates the membrane potential. Figure 3 shows an example of the membrane potential of the spiking neuron model of the i-th node in the l-th layer. The membrane potential of the spiking neuron model of the i-th node in the l-th layer at time t is denoted as v i (l) (t). In the description of Figure 3, the spiking neuron model of the i-th node in the l-th layer is also referred to as the target model.

[0032] In the example of Figure 3, the target model receives the input of spike signals from three spiking neuron models. Time t2 *(l-1) indicates the input time of the spike signal from the second spiking neuron model in the l-1-th layer. Time t1 *(l-1) indicates the input time of the spike signal from the first spiking neuron model in the l-1-th layer. Time t3 *(l-1) indicates the input time of the spike signal from the third spiking neuron model in the l-1-th layer. Also, the target model outputs a spike signal at time t i *(l) Outputting a spike signal by a spiking neuron model is called firing. The time when a spiking neuron model fires is called the firing time.

[0033] In the example of FIG. 3, the initial value of the membrane potential is set to 0. The initial value of the membrane potential corresponds to the resting membrane potential. Before the target model fires, after the input of the spike signal, the membrane potential v i (l) (t) of the target model continues to change at a rate of change (rate of change) according to the weights set for each transmission path of the spike signal. Also, the rates of change of the membrane potential for each input of the spike signal are linearly added. The differential equation of the membrane potential v i (l) (t) in the example of FIG. 3 is expressed as in Equation (1).

[0034]

Number

[0035] w ij (l) represents the weight set for the transmission path of the spike signal from the j-th spiking neuron model in the (l-1)-th layer to the target model. The weight w ij (l) is the object of learning. θ is a step function and is shown as in Equation (2).

[0036]

Number

[0037] At time t i *(l) the membrane potential v i (l) (t) of the target model reaches the firing threshold V th and the target model fires. Due to the firing, the membrane potential v i (l) (t) of the target model becomes 0, and thereafter the membrane potential does not change even if the target model receives an input of a spike signal.

[0038] FIG. 4 is a diagram showing an example of the output timing of spike signals of a spiking neuron model in a spiking neural network when the output timing of the spike signals of the spiking neuron model is not restricted. The horizontal axis in FIG. 4 indicates time as the elapsed time since the start of data input to the spiking neural network. The vertical axis indicates the node numbers in each of the input layer, the first layer, the second layer, and the output layer. The node number for each layer is used as an identification number for identifying the spiking neuron model within the layer.

[0039] In the example of FIG. 4, the spiking neuron model in the input layer fires within about 5 milliseconds from the start of data input to the spiking neural network. In contrast, the firing timings of the spiking neuron models in the first layer are scattered in the range from about 10 milliseconds to about 120 milliseconds as the elapsed time from the start of data input to the spiking neural network. The firing timings of the spiking neuron models in the second layer are scattered in the range from about 20 milliseconds to about 70 milliseconds as the elapsed time from the start of data input to the spiking neural network.

[0040] Generally, it is difficult to know in real time during the execution of the processing of the spiking neural network the firing timing of the spiking neural network and which of the firings of the spiking neuron models in the first layer and the second layer affect the firing of the spiking neuron model in the output layer.

[0041] Therefore, depending on the method of the spiking neural network, it is necessary to wait without inputting the next input data to the spiking neural network for at least the time T11 until the spiking neuron model in the output layer fires first. On the other hand, when there are multiple input data, if the next input data can be input to the spiking neural network before the operation result for one input data is obtained, the time from when the first input data is input to the spiking neural network until the operation result for the last input data is obtained can be made relatively short. In this regard, the throughput of the spiking neural network can be increased.

[0042] Therefore, in the neural network device 10, the data processing time of the neuron model 100 is set so that the neuron model 100 completes a predetermined process within the data processing time. Specifically, the index value calculation unit 110 calculates a membrane potential based on input data input in the form of a spike signal for each predetermined time interval. The time width of this time interval corresponds to an example of the data processing time. Then, the detection unit 120 detects the occurrence timing of a predetermined event regarding the membrane potential calculated by the index value calculation unit 110 for each time interval. Each time the detection unit 120 detects the occurrence timing of an event, the delay unit 130 determines the output timing of the spike signal within the next time interval of the time interval including that timing. The signal output unit 140 outputs a spike signal at the timing determined by the delay unit 130 for each time interval. The spike signal output by the signal output unit 140 corresponds to the output data from the neuron model 100. Thereby, the neuron model 100 receives data input for each time interval. Then, the neuron model 100 sequentially processes the input data in the time of two time intervals and outputs data for each time interval.

[0043] FIG. 5 is a diagram showing an example of the timing of data processing for each layer in the neural network device 10. In FIG. 5, taking the case of the four-layer configuration shown in FIG. 2 as an example, the timing at which each node 30 of the input layer 21, the first layer of the intermediate layer (intermediate layer 22-1), the second layer of the intermediate layer (intermediate layer 22-2), and the output layer 23 processes data is shown. In both the first layer and the second layer, the neuron model 100 is used as the node 30.

[0044] The horizontal axis of FIG. 5 represents time as the elapsed time since the start of data input to the neural network device 10. In the example of FIG. 5, the time indicated by the horizontal axis is divided into time intervals for each time width T. The neural network device 10 synchronizes among the neuron models 100 to set the same time interval so that the time intervals of the same time width are set for all the neuron models 100 at the same timing.

[0045] Alternatively, when the delay time in the transmission of spike signals cannot be ignored, the neural network device 10 may set the same time interval by synchronizing among all the neuron models 100 within each layer, and in terms of correlation, set the time interval shifted by the delay time in the transmission of spike signals.

[0046] The node 30 of the input layer 21 outputs data in the form of spike signals to the node 30 of the first layer for each time interval. In the example of FIG. 5, the node 30 of the input layer 21 outputs three pieces of data, namely the first data, the second data, and the third data, to the node 30 of the first layer. However, the number of data output by the node 30 of the input layer 21 is not limited to a specific number.

[0047] Each time the node 30 of the first layer receives data input from the node 30 of the input layer 21, it outputs intermediate data for processing that data in the form of spike signals to the node 30 of the second layer in the next time interval after the time interval when the data input was received. For example, the node 30 of the first layer receives the input of the first data in the time interval from time 0 to T, and outputs intermediate data for processing the first data to the node 30 of the second layer in the time interval from time T to 2T.

[0048] Each time the node 30 of the second layer receives data input from the node 30 of the first layer, it outputs, in the form of a spike signal, intermediate data for processing that data to the node 30 of the output layer in the next time interval after the time interval when the data input was received. For example, the node 30 of the second layer receives input of intermediate data for processing the first data in the time interval from time T to 2T, and outputs, to the node 30 of the output layer, intermediate data obtained by further processing the data from the node 30 of the first layer in the time interval from time 2T to 3T.

[0049] Each time the node 30 of the output layer receives data input from the node 30 of the second layer, it outputs, in the form of a spike signal, output data obtained by further processing that data. Since the output data from the node 30 of the output layer is not further input to a node, in the example of FIG. 5, the node 30 of the output layer outputs data in the time interval when the data input was received.

[0050] In this way, in the neural network device 10, since the time interval for data processing is determined, the processing of the next data can be started without waiting for the completion of the processing of one data. Also, in the neural network device 10, as in the examples of the first layer and the second layer, the neuron model 100 outputs a spike signal in the next time interval after the time interval when it receives the input of the spike signal. Thereby, the neuron model 100 can adjust the timing of the spike signal output within the time interval. By adjusting the timing of the spike signal output, in the neural network device 10, for example, it is possible to avoid the timing of the input of the spike signal being biased towards the latter part within the time interval (near the end of that time interval) in the downstream layer in signal transmission.

[0051] In order to perform data processing at the timing illustrated in FIG. 5, the neuron model 100 outputs a spike signal in the next time interval according to the input status of the spike signal for each time interval, as described above. Hereinafter, similar to the description of the spiking neural network with reference to Formula (1) and Formula (2), the neuron model 100 of the i-th node in the l-th layer is referred to as the target model, and the membrane potential of the target model is denoted as v i (l) (t). Also, the weight set in the transmission path of the spike signal from the j-th spiking neuron model in the l-1-th layer to the target model is denoted as w ij (l) .

[0052] The index value calculation unit 110 changes the membrane potential based on the input situation of the spike signal to the neuron model 100 in each time interval. The index value calculation unit 110 corresponds to an example of the index value calculation means. The index value calculation unit 110 calculates the membrane potential v i (l) (t) until either the membrane potential v th reaches the threshold value V i (l) (t) or the time interval ends, whichever event occurs earlier, according to the differential equation represented as in Formula (1). The membrane potential v i (l) (t) is represented as in Formula (3).

[0053]

Equation

[0054] The index value calculation unit 110 may calculate the membrane potential based on Formula (3). On the other hand, at the timing when the membrane potential v i (l) (t) reaches the threshold value V th or at the timing when the time interval ends, the index value calculation unit 110 resets the value of the membrane potential v i (l) (t) to 0. The index value calculation unit 110 maintains the value of the membrane potential v i (l) (t) at 0 until the start of the next time interval. Thereby, the index value calculation unit 110 calculates the membrane potential v i(l) The calculation of (t) starts from the state where the value of the membrane potential v i (l) (t) is 0.

[0055] The membrane potential v i (l) (t) reaches the threshold value V th at time t i (l,vth) When substituting into the time t in Equation (3) and denoted as such, Equation (4) is obtained.

[0056]

Equation

[0057] Γ i (l) represents the set of indices j for which t j *(l-1) < t i (l,vth) holds. In the neural network device 10, since the firing time is restricted within the output time interval, the firing time t i *(l) may be at a time later than the time when the membrane potential v i (l) (t) reaches the threshold value V th . Thus, since there may be a difference between the time when the membrane potential v i (l) (t) reaches the threshold value V th and the firing time t i *(l) , the time when the membrane potential v i (l) (t) reaches the threshold value V th is denoted as t i (l,vth) . The time when the membrane potential reaches the threshold value is also referred to as the threshold arrival time. From Equation (4), the threshold arrival time t i (l,vth) is expressed as in Equation (5).

[0058]

Equation

[0059] The detection unit 120 detects the occurrence timing of a predetermined event related to the membrane potential. The detection unit 120 corresponds to an example of detection means. The predetermined event related to the membrane potential here may be that the membrane potential has reached a predetermined threshold value, or that the membrane potential has not reached the predetermined threshold value by the end of the data processing time. The occurrence timing of the predetermined event related to the membrane potential here may be the timing when the membrane potential reaches the predetermined threshold value, or the end of the data processing time.

[0060] The occurrence timing of the predetermined event related to the membrane potential detected by the detection unit 120 is also referred to as the event occurrence timing. Within a time interval, the membrane potential v i (l) (t) reaches the threshold value V th If so, the detection unit 120 detects the timing when the membrane potential v i (l) (t) reaches the threshold value V th as the event occurrence timing. On the other hand, if the membrane potential v i (l) (t) does not reach the threshold value V th within the time interval, the detection unit 120 detects the end of that time interval as the event occurrence timing.

[0061] The delay unit 130 determines the timing at which the neuron model 100 outputs a spike signal. In particular, the delay unit 130 determines the timing at which the neuron model 100 outputs a spike signal for each time interval based on the event occurrence timing in the previous time interval.

[0062] Specifically, the delay unit 130 detects the timing delayed by a fraction of a predetermined condition from the event occurrence timing. The timing delayed by a fraction of a predetermined condition from the event occurrence timing detected by the delay unit 130 is also referred to as the delayed timing.

[0063] When the post-delay timing is included in the same time interval as the event occurrence timing, the delay unit 130 determines the start of the next time interval of that time interval as the timing at which the neuron model 100 outputs a spike signal. When the time interval including the post-delay timing is the next time interval of the time interval including the event occurrence timing, the delay unit 130 determines the post-delay timing as the timing at which the neuron model 100 outputs a spike signal. If the post-delay timing is not detected by the end of the next time interval including the event occurrence timing, the delay unit 130 determines the end of the next time interval including the event occurrence timing as the timing at which the neuron model 100 outputs a spike signal.

[0064] The signal output unit 140 outputs a spike signal at the timing determined by the delay unit 130. Thereby, the signal output unit 140 outputs a spike signal at a timing within the first time interval of the time intervals and at a timing corresponding to the occurrence timing of a predetermined event in the second time interval, which is a time interval prior to the first time interval. Here, the first time interval is the time interval in which the neuron model 100 outputs a spike signal. The second time interval is the time interval in which the neuron model 100 receives the input of the spike signal that is the basis for determining the spike signal output timing in the first time interval. The signal output unit 140 corresponds to an example of signal output means.

[0065] FIG. 6 is a diagram showing a first example of a more specific configuration of the neural network device. In the configuration shown in FIG. 6, the neural network device 10a includes a neuron model 100a. The neuron model 100a includes a first sub-model 210-1, a switching unit 220a, a second sub-model 210-2, a third sub-model 210-3, and an integration unit 230. The first sub-model 210-1 includes a first index value calculation unit 211-1, a first detection unit 212-1, and a first signal output unit 213-1. The second sub-model 210-2 includes a second index value calculation unit 211-2, a second detection unit 212-2, and a second signal output unit 213-2. The third sub-model 210-3 includes a third index value calculation unit 211-3, a third detection unit 212-3, and a third signal output unit 213-3.

[0066] The first sub-model 210-1, the second sub-model 210-2, and the third sub-model 210-3 are collectively referred to as the sub-model 210. The second sub-model 210-2 and the third sub-model 210-3 are collectively referred to as the delay sub-model 210f. The first index value calculation unit 211-1, the second index value calculation unit 211-2, and the third index value calculation unit 211-3 are collectively referred to as the index value calculation unit 211. The second index value calculation unit 211-2 and the third index value calculation unit 211-3 are collectively referred to as the delay index value calculation unit 211f.

[0067] The first detection unit 212-1, the second detection unit 212-2, and the third detection unit 212-3 are collectively referred to as the detection unit 212. The second detection unit 212-2 and the third detection unit 212-3 are collectively referred to as the delay detection unit 212f. The first signal output unit 213-1, the second signal output unit 213-2, and the third signal output unit 213-3 are collectively referred to as the signal output unit 213. The second signal output unit 213-2 and the third signal output unit 213-3 are collectively referred to as the delay signal output unit 213f.

[0068] The sub-model 210 is configured as a spiking neuron model. Specifically, the index value calculation unit 211 changes the membrane potential over time based on the input status of the spike signal to the sub-model 210. The first sub-model 210-1 corresponds to an example of the first spiking neuron model. The two delay sub-models 210f correspond to examples of the two second spiking neuron models.

[0069] The detection unit 212 detects the occurrence timing of a predetermined event related to the membrane potential. Specifically, the first detection unit 212-1 compares the membrane potential with a predetermined threshold value. When the membrane potential reaches the threshold value within the time interval, the first detection unit 212-1 detects the timing at which the membrane potential reaches the threshold value. When the time interval ends without the membrane potential reaching the threshold value, the first detection unit 212-1 detects the end timing of that time interval.

[0070] In the delay sub-model 210f, the time interval is divided into a time interval during which the delay sub-model 210f receives the input of the spike signal and a time interval during which the delay sub-model 210f outputs the spike signal. The time interval during which the delay sub-model 210f receives the input of the spike signal is also referred to as the input time interval. The time interval during which the delay sub-model 210f outputs the spike signal is also referred to as the output time interval. The input time interval is the time interval during which the delay sub-model 210f receives the input of the spike signal from the first sub-model 210-1. The output time interval is the time interval following the input time interval.

[0071] The delay detection unit 212f compares the membrane potential with a predetermined threshold value. When the membrane potential reaches the threshold value within the output time interval, the detection unit 212 detects the timing at which the membrane potential reaches the threshold value. When the output time interval ends without the membrane potential reaching the threshold value, the detection unit 212 detects the end timing of that time interval. Also, when the membrane potential reaches the threshold value within the input time interval, the delay detection unit 212f detects the start timing of the next time interval. In this case, the "next time interval" corresponds to the output time interval.

[0072] The signal output unit 213 outputs a spike signal at the timing of the occurrence of a predetermined event detected by the detection unit 120. For example, the first signal output unit 213-1 outputs a spike signal at the timing when the membrane potential reaches the threshold value detected by the first detection unit 212-1, or at the end timing of the time interval when the time interval ends without the membrane potential reaching the threshold value.

[0073] Also, the delay signal output unit 213f outputs a spike signal at the timing when the membrane potential reaches the threshold value within the output time interval detected by the delay detection unit 212f, at the end timing of the time interval when the output time interval ends without the membrane potential reaching the threshold value, or at the start timing of the next time interval when the membrane potential reaches the threshold value within the input time interval.

[0074] The switching unit 220a switches the transmission destination of the spike signal from the first sub-model 210-1 to either of the two delay sub-models 210f. The integration unit 230 integrates the spike signals output from each of the two delay sub-models 210f.

[0075] Integrating the spike signals here may be to take the logical OR of the spike signals output from each of the two delay sub-models 210f. Specifically, for each time interval, either one of the two delay sub-models 210f outputs a spike signal. The integration unit 230 outputs the spike signal output by either one of the delay sub-models 210f for each time interval as the output of the neuron model 100a.

[0076] The neural network device 10a corresponds to an example of the neural network device 10. The first index value calculation unit 211-1 corresponds to an example of the index value calculation unit 110. The first index value calculation unit 211-1 also corresponds to an example of the index value calculation means. The first detection unit 212-1 corresponds to an example of the detection unit 120. The first detection unit 212-1 also corresponds to an example of the detection means.

[0077] The combination of the delay index value calculation unit 211f and the delay detection unit 212f corresponds to an example of the delay unit 130. The combination of the delay index value calculation unit 211f and the delay detection unit 212f also corresponds to an example of the delay means. The delay signal output unit 213f corresponds to an example of the signal output unit 140. The delay signal output unit 213f also corresponds to an example of the signal output means.

[0078] FIG. 7 is a diagram showing an example of the configuration of the switching unit 220a and the integration unit 230. In the example of FIG. 7, the switching unit 220a includes a switch 221. The switching unit 220a switches the switch 221 for each time interval based on the input clock signal. Thereby, the switching unit 220a switches the delay sub-model 210f, which is the output destination of the spike signal from the first sub-model 210-1, for each time interval. The switching unit 220a corresponds to an example of the switching means.

[0079] The integration unit 230 includes a switch 231 and an inverter 232. The integration unit 230 inverts the input clock signal with the inverter 232 and switches the switch 231 for each time interval based on the inverted clock signal. Thereby, the integration unit 230 switches the delay sub-model 210f that outputs the spike signal to the delay sub-model 210f that does not receive the input of the spike signal from the first sub-model 210-1 for each time interval. However, the configurations of the switching unit 220a and the integration unit 230 are not limited to the configurations illustrated in FIG. 7.

[0080] FIG. 8 is a diagram showing an example of the timing of passing spike signals between neuron models 100a in the neural network device 10a. FIG. 8 shows the passing of spike signals between the first sub-model 210-1 and the two delay sub-models 210f in one neuron model 100a, and the passing of spike signals between the two delay sub-models 210f and the first sub-model 210-1 of the neuron model 100a that receives the input of the spike signals from those delay sub-models 210f.

[0081] The horizontal axis of the graph in FIG. 8 indicates the time elapsed since the first input data was input to the neuron model 100a on the side that outputs the spike signal in FIG. 8. The vertical axis indicates the membrane potential of each of the first sub-model 210-1, the second sub-model 210-2, the third sub-model 210-3, and the first sub-model 210-1 that receives the input of the spike signal from the delay sub-model 210f. Similar to the description of the neural network device 10 in the example of FIG. 5, the time shown on the horizontal axis of FIG. 8 is divided into time intervals for each time width T.

[0082] At the start of the time interval, the membrane potential of the first sub-model 210-1 is reset to 0. The first index value calculation unit 211-1 may set the membrane potential to 0 when the first sub-model 210-1 fires and maintain the setting of the membrane potential to 0 until the start of the next time interval. From the start of the time interval until the first sub-model 210-1 fires, the first index value calculation unit 211-1 changes the membrane potential over time according to the acquisition status of the spike signals received within the time interval. The first sub-model 210-1 fires at the timing when the membrane potential reaches the threshold value. If the membrane potential does not reach the threshold value within the time interval, the first sub-model 210-1 fires at the end of the time interval. Firing in a state where the membrane potential has not reached the threshold value is also referred to as forced firing.

[0083] The spike signal from the first sub-model 210-1 is input into one of the two delay sub-models 210f. For example, the spike signal in the processing of odd-numbered data may be input into the second sub-model 210-2, and the spike signal in the processing of even-numbered data may be input into the third sub-model 210-3. As described above, the switching unit 220a distributes the spike signal from the first sub-model 210-1 to one of the two delay sub-models 210f.

[0084] The delay sub-model 210f outputs a spike signal in the time interval following the time interval when it receives the spike signal from the first sub-model 210-1. As described above, the time interval when the delay sub-model 210f receives the spike signal is also referred to as the input time interval. The time interval when the delay sub-model 210f outputs the spike signal is also referred to as the output time interval. The input time interval and the output time interval are reversed between the second sub-model 210-2 and the third sub-model 210-3. That is, when the time interval in the second sub-model 210-2 is the input time interval, the time interval in the third sub-model 210-3 is the output time interval. When the time interval in the second sub-model 210-2 is the output time interval, the time interval in the third sub-model 210-3 is the input time interval.

[0085] At the start of the input time interval, the membrane potential of the delay sub-model 210f is reset to 0. The index value calculation unit 211 of the delay sub-model 210f may set the membrane potential to 0 when the delay sub-model 210f fires and maintain the setting of keeping the membrane potential at 0 until the start of the next time interval. The firing of the delay sub-model 210f occurs in the output time interval, and the "next time interval" here is the input time interval.

[0086] The delay sub-model 210f receives the input of the spike signal only from one first sub-model 210-1 included in the same neuron model 100a as the neuron model 100a in which the delay sub-model 210f itself is included. When receiving the spike signal input from the first sub-model 210-1, the delay index value calculation unit 211f increases the membrane potential at a rate of change (rate of change) corresponding to the weight set in the transmission path of the spike signal.

[0087] When the membrane potential reaches the threshold value within the input time interval, the delay signal output unit 213f outputs a spike signal at the start of the output time interval. For example, from when the membrane potential reaches the threshold value until the start of the output time interval, the delay index value calculation unit 211f maintains the membrane potential at the same value as the threshold value. Then, at the start of the output time interval, the delay detection unit 212f detects that the membrane potential has reached the threshold value, and the delay signal output unit 213f outputs a spike signal based on the detection result of the delay detection unit 212f. After firing, the delay index value calculation unit 211f sets the membrane potential to 0.

[0088] When the membrane potential reaches the threshold value within the output time interval, the delay detection unit 212f detects that the membrane potential has reached the threshold value, and the delay signal output unit 213f outputs a spike signal based on the detection result of the delay detection unit 212f. After firing, the delay index value calculation unit 211f sets the membrane potential to 0.

[0089] When the membrane potential does not reach the threshold value until the end of the output time interval, the delay signal output unit 213f outputs a spike signal by forced firing. For example, if the membrane potential has never reached the threshold value until the end of the output time interval, the delay detection unit 212f may output a dummy determination result indicating that the membrane potential has reached the threshold value to the delay signal output unit 213f. Then, the delay signal output unit 213f may output a spike signal at the end of the output time interval based on the dummy determination result of the delay detection unit 212f. After firing, the delay index value calculation unit 211f sets the membrane potential to 0.

[0090] FIG. 9 is a diagram showing an example of the configuration of a sublayer in the neural network device 10a. The spiking neural network included in the neural network device 10a is also denoted as the neural network 11a. The neural network 11a corresponds to an example of the neural network 11.

[0091] In the example of FIG. 9, the layer 20 including the neuron model 100a as the node 30 is configured to include sub-layers. The sub-layer including the first sub-model 210-1 is referred to as the calculation layer, and the symbol 25-1 is added. The sub-layer including the second sub-model 210-2 is referred to as the first delay layer, and the symbol 25-2 is added. The sub-layer including the third sub-model 210-3 is referred to as the second delay layer, and the symbol 25-3 is added. When the sub-layer is included in the i-th layer (i is a positive integer), the calculation layer is also referred to as the i-th calculation layer, the first delay layer is also referred to as the (i-1)-th delay layer, and the second delay layer is also referred to as the (i-2)-th delay layer.

[0092] When the first sub-model 210-1 outputs a spike signal, the switching unit 220a transmits the spike signal from the first sub-model 210-1 to any one of the two delay sub-models 210f. The spike signal output by the delay sub-model 210f that has received the input of the spike signal from the first sub-model 210-1 in the next time interval is input to the node 30 of the next layer 20 via the integration unit 230.

[0093] FIG. 10 is a diagram showing an example of the timing of data processing in the neural network device 10a. The horizontal axis of FIG. 10 indicates the time as the elapsed time from the start of data input to the neural network 11a. In the example of FIG. 10, the time indicated on the horizontal axis is divided into time intervals for each time T.

[0094] Further, FIG. 10 shows an example in which the neural network 11a is composed of four layers, namely an input layer, a first layer, a second layer, and an output layer, and processes three pieces of data, namely first data, second data, and third data. For each layer or each sub-layer, the time interval for processing each piece of data in that layer or sub-layer is shown.

[0095] In the example of FIG. 10, node 30 of the input layer outputs one piece of data to node 30 of the first arithmetic layer in one time interval. In the arithmetic layer, the first sub-model 210-1 processes the data within the time interval in which the data is input, and outputs the data as a spike signal to the delay sub-model 210f of the delay layer. The data processing in the first sub-model 210-1 may be processing for determining the firing timing.

[0096] In the delay layer, the delay sub-model 210f receives the input of data in the input time interval. The delay sub-model 210f outputs the data to the nodes of the next layer by outputting a spike signal at a timing corresponding to the timing at which the data was input in the input time interval among the output time intervals that are the next time intervals after the input time interval. It can be said that the delay sub-model 210f delays the spike signal received at the input time interval until the output time interval and then outputs it. Node 30 of the output layer processes the data within the time interval in which the data is input, and outputs the processed data as a spike signal.

[0097] In this way, since the processing time for one piece of data is determined for each layer, the neural network device 10a can start the next data processing in a pipeline processing manner without waiting for the completion of the processing of one piece of data. Further, by providing two delay sub-models 210f for one first sub-model 210-1, even if each of the delay sub-models 210f processes one piece of data over two time intervals, the first sub-model 210-1 can output the data to the delay sub-models 210f for each time interval. Also, by providing two delay sub-models 210f for one first sub-model 210-1, even if each of the delay sub-models 210f performs the processing of one piece of data over two time intervals, the first sub-model 210-1 can output the data to the delay sub-models 210f for each time interval.

[0098] The membrane potential v before firing of the first sub-model 210-1 of the i-th neuron model 100 in the l-th layer i_1 (l) (t) can be expressed as in Equation (6).

[0099]

Equation

[0100] Let the threshold arrival time in this first sub-model 210-1 be t i_1 (l,vth) Then, Equation (7) can be obtained.

[0101]

Equation

[0102] Γ i_1 (l) is the set of indices j such that t j *(l-1) < t i_1 (l,vth) holds. The threshold arrival time t i_1 (l,vth) is expressed as in Equation (8).

[0103]

Equation

[0104] The firing time t when this first sub-model 210-1 processes the d-th data i_1 *(l) can be expressed as in Equation (9).

[0105]

Equation

[0106] The function clip is shown as in Equation (10).

[0107] [Mathematics]

[0108] The function clip shown in formula (10) is also referred to as the clip function. FIG. 11 is a diagram showing a first example of the clip function. FIG. 11 shows an example of the clip function when the first sub-model 210-1 of the i-th neuron model 100 in the l-th layer processes the d-th data. The horizontal axis of the graph in FIG. 11 indicates the time when the membrane potential reaches the threshold value, represented by the elapsed time from the start of data input to the neural network 11a. The vertical axis indicates the firing time of the first sub-model 210-1, represented by the elapsed time from the start of data input to the neural network 11a.

[0109] The time from time (l + d - 2)T to (l + d - 1)T corresponds to the time interval during which the first sub-model 210-1 of the i-th neuron model 100 in the l-th layer processes the d-th data. If the membrane potential reaches the threshold value within this time interval, when the first detection unit 212-1 detects that the membrane potential has reached the threshold value, the first signal output unit 213-1 outputs a spike signal. On the other hand, if the membrane potential does not reach the threshold value within this time interval, it can be regarded that the membrane potential reaches the threshold value after time (l + d - 1)T. In this case, the first signal output unit 213-1 outputs a spike signal at time (l + d - 1)T, which is the end of this time interval.

[0110] The membrane potential v before firing of the delay sub-model 210f of the i-th neuron model 100 in the l-th layer i_f (l) (t) can be expressed as in formula (11).

[0111] [Mathematics]

[0112] w findicates the weight set for the transmission path of the spike signal from the first sub-model 210_1 to the delay sub-model 210f. w f may be the object of learning. Let the threshold arrival time in this delay sub-model 210f be t i_f (l,vth) Then, Equation (12) can be obtained.

[0113]

Equation

[0114] The threshold arrival time t i_f (l,vth) is expressed as in Equation (13).

[0115]

Equation

[0116] When this delay sub-model 210f processes the d-th data, the firing time t i_f *(l) can be expressed as in Equation (14).

[0117]

Equation

[0118] Figure 12 is a diagram showing a second example of the clip function. Figure 12 shows an example of the clip function when the delay sub-model 210f of the i-th neuron model 100 in the first layer processes the d-th data. The horizontal axis of the graph in Figure 12 indicates the time when the membrane potential reaches the threshold, represented as the elapsed time from the start of data input to the neural network 11a. The vertical axis indicates the firing time of the delay sub-model 210f, represented as the elapsed time from the start of data input to the neural network 11a.

[0119] The time from (l + d - 2)T to (l + d - 1)T corresponds to the input time interval when the delay sub-model 210f of the i-th neuron model 100 in the l-th layer processes the d-th data. The time from (l + d - 1)T to (l + d)T corresponds to the output time interval when the delay sub-model 210f of the i-th neuron model 100 in the l-th layer processes the d-th data. When the membrane potential reaches the threshold within the input time interval, the delay signal output unit 213f outputs a spike signal at the start of the output time interval, which is the next time interval after this input time interval.

[0120] When the membrane potential reaches the threshold within the output time interval, when the delay detection unit 212f detects that the membrane potential has reached the threshold, the delay signal output unit 213f outputs a spike signal. If the membrane potential does not reach the threshold by the end of the output time interval, it can be regarded as if the membrane potential reaches the threshold after time (l + d)T. In this case, the delay signal output unit 213f outputs a spike signal at time (l + d)T, which is the end of the output time interval.

[0121] FIG. 13 is a diagram showing a first example of a function indicating the timing when the i-th neuron model 100a in the l-th layer processes the d-th data. The horizontal axis of the graph in FIG. 13 indicates the time when the membrane potential of the first sub-model 210-1 reaches the threshold, represented as the elapsed time from the start of data input to the neural network 11a. The vertical axis indicates the firing time of the delay sub-model 210f, represented as the elapsed time from the start of data input to the neural network 11a. The time when the membrane potential of the first sub-model 210-1 reaches the threshold can be treated as the same time as the firing time of the first sub-model 210-1. Also, the time when the membrane potential of the first sub-model 210-1 reaches the threshold can be treated as the same time as the time when the delay sub-model 210f receives the input of the spike signal.

[0122] FIG. 13 shows an example in which the time from when the delay sub-model 210f receives the input of the spike signal until it outputs the spike signal is T. The time from when the delay sub-model 210f receives the input of the spike signal until it outputs the spike signal is also referred to as the delay time by the delay sub-model 210f.

[0123] When the delay time by the delay sub-model 210f is T, the delay sub-model 210f outputs the spike signal at a timing within the output time interval that is the same as the timing when it received the input of the spike signal within the input time interval. Here, the same timing within the two time intervals may mean that the elapsed time from the start of each time interval is the same.

[0124] In FIG. 13, the elapsed time from time (l + d - 2)T to the threshold arrival time in the first sub-model 210-1 is equal to the elapsed time from time (l + d - 1)T to the firing time of the delay sub-model 210f. Here, time (1 + d - 2)T is the start of the input time interval. Time (1 + d - 1)T is the start of the output time interval. Also, it is assumed that the delay time from when the membrane potential of the first sub-model 210-1 reaches the threshold until the delay sub-model 210f receives the input of the spike signal can be ignored. Therefore, FIG. 13 shows the case where the delay sub-model 210f outputs the spike signal at a timing within the output time interval that is the same as the timing when it received the input of the spike signal within the input time interval.

[0125] When the delay time by the delay sub-model 210f is equal to the time width of the time interval, the timing at which the membrane potential of the first sub-model 210-1 reaches the threshold within the time interval is directly reflected in the timing at which the delay sub-model 210f fires within the output time interval. In this regard, it can be said that no information is lost.

[0126] FIG. 14 is a diagram showing a second example of a function indicating the timing when the i-th neuron model 100a in the first layer processes the d-th data. The horizontal axis of the graph in FIG. 14 indicates the time when the membrane potential of the first sub-model 210-1 reaches the threshold value, represented as the elapsed time from the start of data input to the neural network 11a. The vertical axis indicates the firing time of the delay sub-model 210f, represented as the elapsed time from the start of data input to the neural network 11a.

[0127] FIG. 14 shows an example where the delay time by the delay sub-model 210f is shorter than T. When the delay time by the delay sub-model 210f is shorter than T, the timing at which the delay sub-model 210f fires within the output time interval becomes earlier than the timing at which the first sub-model 210-1 fires within the time interval. In the example of FIG. 14, when the membrane potential of the first sub-model 210-1 reaches the threshold value between the times (l + d - 2)T and (l + d - 2 + c1)T, the delay sub-model 210f fires at the time (l + d - 1)T, which is the start of the output time interval.

[0128] Also, when the membrane potential of the first sub-model 210-1 reaches the threshold value between the times (l + d - 2 + c1)T and (l + d - 1)T, the delay sub-model 210f fires after a lapse of time T - c1. Looking at the timing within the time interval, the timing at which the delay sub-model 210f fires within the output time interval is earlier than the timing at which the first sub-model 210-1 fires within the time interval by the time c1. By advancing the timing in this way, the output timing of the spike signal within the time interval can be adjusted. For example, it is possible to avoid the concentration of the input of the spike signal near the end of the time interval in the layer on the downstream side of the data flow.

[0129] FIG. 15 is a diagram showing a third example of a function indicating the timing when the i-th neuron model 100a in the first layer processes the d-th data. The horizontal axis of the graph in FIG. 15 indicates the time when the membrane potential of the first sub-model 210-1 reaches the threshold value, represented as the elapsed time from the start of data input to the neural network 11a. The vertical axis indicates the firing time of the delay sub-model 210f, represented as the elapsed time from the start of data input to the neural network 11a.

[0130] FIG. 15 shows an example where the delay time by the delay sub-model 210f is longer than T. When the delay time by the delay sub-model 210f is longer than T, the timing at which the delay sub-model 210f fires within the output time interval becomes later than the timing at which the first sub-model 210-1 fires within the time interval.

[0131] In the example of FIG. 15, when the membrane potential of the first sub-model 210-1 reaches the threshold value between the times (l + d - 2)T and (l + d - 1 - c2)T, the delay sub-model 210f fires after a lapse of time T + c2. Looking at the timing within the time interval, the timing at which the delay sub-model 210f fires within the output time interval is delayed by a time c2 compared to the timing at which the first sub-model 210-1 fires within the time interval. Also, when the membrane potential of the first sub-model 210-1 reaches the threshold value between the times (l + d - 1 - c2)T and (l + d - 1)T, the delay sub-model 210f fires at the time (1 + d)T, which is the end of the output time interval.

[0132] FIG. 16 is a diagram showing a second example of a more specific configuration of the neural network device. In the configuration shown in FIG. 16, the neural network device 10b includes a neuron model 100b. The neuron model 100b includes an index value calculation unit 110, a detection unit 120, a delay unit 130b, a signal output unit 140, and a switching unit 220b.

[0133] Parts having the same functions corresponding to the parts in FIG. 1 among the parts in FIG. 16 are assigned the same reference numerals (110, 120, 140), and detailed descriptions thereof are omitted here. The neural network device 10b corresponds to an example of the neural network device 10, and the neuron model 100b corresponds to an example of the neuron model 100. In FIG. 16, as the configuration of the neuron model 100b, a delay unit 130b is shown instead of the delay unit 130 in the configuration of the neuron model 100 in FIG. 1, and a switching unit 220b is further shown.

[0134] The switching unit 220b switches the transmission destination of the signal indicating that the membrane potential has reached the threshold value from the detection unit 120 to either of the two delay units 130b. The switching unit 220b is different from the switching unit 220a of the neural network device 10a in that the signal indicating that the membrane potential has reached the threshold value is not limited to a spike signal. When a spike signal is used as the signal indicating that the membrane potential has reached the threshold value, the switching unit 220b is the same as the switching unit 220a. The switching unit 220a corresponds to an example of the switching unit 220b. The switching unit 220b corresponds to an example of switching means.

[0135] The delay unit 130b outputs a signal instructing firing to the signal output unit 140 at a timing corresponding to the timing of receiving the input of the signal from the detection unit 120 in the next time interval after the time interval in which the input of the signal indicating that the membrane potential has reached the threshold value from the detection unit 120 is received. Thereby, the signal output unit 140 outputs a spike signal at a timing within the first time interval of the time intervals as described above and at a timing corresponding to the occurrence timing of a predetermined event in the second time interval which is a time interval earlier than the first time interval. The delay unit 130b corresponds to an example of the delay unit 130. The delay unit 130b also corresponds to an example of delay means.

[0136] A signal obtained by taking the logical OR of the signals output from the two delay units 130b is input to the signal output unit 140. Therefore, regardless of which of the two delay units 130b outputs a signal instructing firing, the signal output unit 140 outputs a spike signal. When the delay unit 130b is configured using a spiking neuron model, the delay unit 130b is the same as the sub-model 210. The sub-model 210 corresponds to an example of the delay unit 130b. On the other hand, the configuration of the delay unit 130b is not limited to the configuration using a spiking neuron model. For example, the delay unit 130b may be configured with a timer, and the time from when the signal is input until the signal is output may be measured using the timer.

[0137] The signal output unit 140 outputs a spike signal in response to a signal instructing firing from the delay unit 130b. When the signal from the delay unit 130b is a spike signal, the signal output unit 140 may output the spike signal from the delay unit 130b as it is. When the time obtained by adding a predetermined delay time to the time when the detection unit 120 detects the occurrence of an event is included in the same time interval as the time interval when the detection unit 120 detects the occurrence of the event, the process of causing the signal output unit 140 to output a spike signal at the start of the next time interval may be performed by the delay unit 130b, or may be performed by the signal output unit 140.

[0138] When the time obtained by adding a predetermined delay time to the time when the detection unit 120 detects the occurrence of an event is a time later than the next time interval of the time interval when the detection unit 120 detects the occurrence of the event, the process of causing the signal output unit 140 to output a spike signal at the end of the next time interval of the time interval when the detection unit 120 detects the occurrence of the event may be performed by the delay unit 130b, or may be performed by the signal output unit 140.

[0139] In the configuration shown in FIG. 1, the delay unit 130 can receive signal inputs at a plurality of times and output signals at times delayed from each of those times. For example, the combination of the switching unit 220b in FIG. 16 and two delay units 130b corresponds to an example of the delay unit 130. Alternatively, the delay unit 130 may include a path capable of receiving a plurality of signals. For example, the delay unit 130 may include a shift register, and when receiving a signal input, the least significant bit of the shift register may be set to 1, and the shift register may be shifted up every predetermined clock cycle. Then, when the most significant bit of the shift register becomes 1, the delay unit 130 may output a signal instructing ignition to the signal output unit 140.

[0140] In the configuration shown in FIG. 1, when the time obtained by adding a predetermined delay time to the time when the detection unit 120 detects the occurrence of an event is within the same time interval as the time interval when the detection unit 120 detects the occurrence of the event, the signal output unit 140 may output a spike signal at the start of the next time interval. This process may be performed by the delay unit 130 or by the signal output unit 140.

[0141] In the configuration shown in FIG. 1, when the time obtained by adding a predetermined delay time to the time when the detection unit 120 detects the occurrence of an event is later than the next time interval after the time interval when the detection unit 120 detects the occurrence of the event, the signal output unit 140 may output a spike signal at the end of the next time interval after the time interval when the detection unit 120 detects the occurrence of the event. This process may be performed by the delay unit 130c or by the signal output unit 140.

[0142] FIG. 17 is a diagram showing an example of a system configuration during learning. In the configuration shown in FIG. 17, the neural network system 1 includes a neural network device 10 and a learning device 50. The neural network device 10 and the learning device 50 may be integrally configured as one device. Alternatively, the neural network device 10 and the learning device 50 may be configured as separate devices. As described above, the neural network 11 configured by the neural network device 10 is also referred to as the neural network main body.

[0143] FIG. 18 is a diagram showing an example of signal input and output in the neural network system 1. In the example of FIG. 18, input data and teacher labels indicating correct answers for the input data are input to the neural network system 1. The neural network device 10 may receive the input of the input data, and the learning device 50 may receive the input of the teacher labels. The combination of the input data and the teacher labels corresponds to an example of training data in supervised learning. Further, the neural network device 10 acquires a clock signal. The neural network device 10 may include a clock circuit. Alternatively, the neural network device 10 may receive the input of a clock signal from outside the neural network device 10.

[0144] The neural network device 10 receives the input of the input data and outputs an estimated value based on the input data. When calculating the estimated value, the neural network device 10 synchronizes the time intervals between layers and the time intervals between neuron models 100 within the same layer using the clock signal.

[0145] The learning device 50 performs learning of the neural network device 10. The learning here is to adjust the parameter values of the learning model by a machine learning algorithm. The learning device 50 performs learning of the weight coefficients for the spike signals input to the neuron model 100. The weight W in Equation (4) ij (l) corresponds to an example of the weight coefficient whose value is adjusted by the learning device 50 through learning.

[0146] The learning device 50 may perform learning of the weight coefficients so that the magnitude of the error between the estimated value and the correct value becomes small, using an evaluation function that shows an evaluation of the error between the estimated value output by the neural network device 10 and the correct value indicated by the teacher label. The learning device 50 corresponds to an example of a learning means. The learning device 50 is configured using, for example, a computer.

[0147] As a learning method performed by the learning device 50, for example, an existing learning method such as the backpropagation method can be used, and a method arranged according to the setting of the input time interval and the output time interval can be used. For example, when the learning device 50 performs learning using a method in which the backpropagation method is arranged, the weight W ij (l) is changed by the change amount ΔW shown in Equation (15) ij (l) so that the weight W ij (l) may be updated.

[0148]

Equation

[0149] η is a constant indicating the learning rate. C is expressed as in Equation (16).

[0150]

Equation

[0151] C corresponds to an example of an evaluation function that shows an evaluation of the error between the estimated value output by the neural network device 10 and the correct value indicated by the teacher label. C is set as a loss function that outputs a smaller value as the error is smaller. M represents an index value indicating the output layer (final layer). N (M) represents the number of neuron models 100 included in the output layer.

[0152] κ i represents the teacher label. Here, it is assumed that the neural network device 10 performs class classification with N (M) classes, and the teacher label is indicated by a one-hot vector. When the value of index i indicates the correct class, κ i = 1, and in other cases, κ iLet it be 0.

[0153] t (ref) represents a reference spike. "γ / 2(t i *(M) -t (ref) ) 2 " is a term provided to avoid learning difficulties. This term is also called the Temporal Penalty Term. Due to the Temporal Penalty Term, the firing timing of the output layer will be distributed around the reference spike, and as a result, the output neuron will fire stably. γ is a constant for adjusting the degree of influence of the Temporal Penalty Term, and γ > 0. γ is also called the Temporal Penalty Coefficient. S i is the softmax function and is expressed as in Equation (17).

[0154]

Equation

[0155] σ soft is a constant provided as a scale factor for adjusting how much the value of the softmax function S i changes when the firing timing of the output layer changes. σ soft > 0. With this scale factor having a value of about the same order as the output time interval (from one-tenth to ten times), stable learning becomes possible. In Equation (17), let the firing time t i *(M) of the i-th spike in the output layer be such that 0 ≦ t i *(M) ≦ 1. For example, the firing time of the spikes in the output layer may be such that, for each class, it indicates the probability that the classification target indicated by the input data is classified into that class. For κ i = 1 for i, the closer the value of t i *(M) is to 1, the more "-Σ i=1 N(M) (κi ln(S i (t *(M) )))」 becomes smaller, and the learning device 50 calculates a smaller loss (the value of the evaluation function C). However, the processing performed by the neural network device 10 is not limited to class classification.

[0156] weight W ij (l) In addition to, or instead of, the weight W shown in Equation (11), the weight w f shown in Equation (11) may be the target of the weight whose value is adjusted by learning. For example, the value of the weight w f may be settable for each neuron model 100. In this case, the weight w f in the i-th neuron model 100 of the l-th layer is denoted as w i_f (l) When the learning device 50 performs learning using a method in which the error backpropagation method is arranged, the weight W i_f (l) is changed by the change amount ΔW i_f (l) shown in Equation (18), and the weight W i_f (l) may be updated.

[0157]

Equation

[0158] FIG. 19 is a diagram showing an example of signal input and output in the neural network device 10 during operation. Similar to the operation shown in FIG. 19, during learning as well, the neural network device 10 receives the input of input data and also acquires a clock signal. The neural network device 10 may be provided with a clock circuit. Alternatively, the neural network device 10 may receive the input of a clock signal from outside the neural network device 10. The neural network device 10 receives the input of input data and outputs an estimated value based on the input data. When calculating the estimated value, the neural network device 10 synchronizes the time intervals between layers and the time intervals between neuron models 100 within the same layer using a clock signal.

[0159] Note that the type of the neural network 11 is not limited to a specific type. For example, the neural network 11 may be configured as a convolutional neural network (CNN) using a spiking neural network.

[0160] As described above, when the neural network 11 is configured as a forward-propagation type spiking neural network, the number of layers of the neural network 11 may be two or more and is not limited to a specific number of layers. Also, the number of neuron models 100 included in each layer is not limited to a specific number, and each layer may include one or more neuron models 100. Each layer may include the same number of neuron models 100, or may include different numbers of neuron models 100 depending on the layer. Also, the neural network 11 may be fully connected or may not be fully connected.

[0161] Also, the membrane potential after the firing of the neuron model 100 is not limited to not changing from the above-described potential 0. For example, after a predetermined time has elapsed since firing, the membrane potential may change in response to the input of a spike signal. The number of firings of each neuron model 100 is also not limited to once for each input data.

[0162] The configuration of the neuron model 100 as a spiking neuron model is also not limited to a specific configuration. For example, the change speed of the neuron model 100 may not be constant from the reception of the input of a spike signal until the reception of the input of the next spike signal. The learning method of the neural network 11 is not limited to supervised learning. The learning device 50 may perform the learning of the neural network 11 by unsupervised learning.

[0163] As described above, the index value calculation unit 110 changes the membrane potential, which is the index value of the spike signal output, over time for each time interval based on the input situation of the spike signal in that time interval. The detection unit 120 detects the occurrence timing of a predetermined event related to the membrane potential. Specifically, the detection unit 120 detects the timing when the membrane potential reaches the threshold value and the timing when the time interval ends without the membrane potential reaching the threshold value. The signal output unit 140 outputs a spike signal at a timing corresponding to the occurrence timing of a predetermined event in the input time interval, which is a time interval before the output time interval and within the time interval, and at a timing within the output time interval.

[0164] In this way, the neuron model 100 receives the input of the spike signal for each time interval and outputs the spike signal in the next time interval after the time interval in which the spike signal is received. Thus, the neural network 11 can receive the input of data for each time interval and process each input data. In this regard, according to the neural network device 10, the neural network 11, which is a spiking neural network, can efficiently perform data processing. Also, by the neuron model 100 outputting the spike signal in the next time interval after the time interval in which the spike signal is received, the output timing of the spike signal within the time interval can be adjusted. In the neural network device 10, for example, it is possible to avoid the input of the spike signal concentrating near the end of the time interval in the layer on the downstream side of the data flow in the neural network 11.

[0165] Further, the delay unit 130 determines the timing for causing the signal output unit 140 to output a spike signal based on the timing after a predetermined time from the timing detected by the detection unit 120. The switching unit 220 switches the delay unit 130 to which the detection result of the detection unit 120 is input among the two delay units 130 for each time interval. In the neural network device 10b, with a relatively simple configuration using two delay units 130 and the switching unit 220, it is possible to cope with the overlap between the time interval during which the signal output unit 140 outputs a spike signal and the time interval during which the index value calculation unit 110 receives the input of the spike signal regarding the next data.

[0166] Also, when the first detection unit 212-1 detects the occurrence timing of a predetermined event regarding the membrane potential, the first sub-model 210-1 outputs a spike signal to the delay sub-model 210f. The delay detection unit 212f determines the timing for causing the delay signal output unit 213f to output a spike signal based on the timing within the output time interval and the timing after a predetermined time from the timing when the spike signal is input from the first sub-model 210-1 in the input time interval.

[0167] In the neural network device 10a, the spiking neuron model can be used as the first sub-model 210-1 and the delay sub-model 210f to configure the neuron model 100a. It is expected that the burden on the designer to design the neural network device 10a is relatively light in that there is no need to separately design the means for delaying the signal output from the spiking neuron model. Also, by using a large number of spiking neuron models, the production cost per spiking neuron model is reduced, and it is expected that the overall production cost of the neural network device 10a is relatively low.

[0168] FIG. 20 is a diagram showing a configuration example of an arithmetic device according to an embodiment. In the configuration shown in FIG. 20, the arithmetic device 610 includes a neuron model 611. The neuron model 611 includes an index value calculation unit 612, a detection unit 613, and a signal output unit 614. With such a configuration, the index value calculation unit 612 changes the index value of the signal output based on the input status of the signal for each time interval. The detection unit 613 detects the occurrence timing of a predetermined event regarding the index value. The signal output unit 614 outputs a signal at a timing corresponding to the timing within the first time interval among the time intervals and the occurrence timing of the predetermined event in the second time interval which is a time interval earlier than the first time interval. The index value calculation unit 612 corresponds to an example of the index value calculation means. The detection unit 613 corresponds to an example of the detection means. The signal output unit 614 corresponds to an example of the signal output means.

[0169] In this way, the neuron model 611 receives the input of spike signals for each time interval and outputs spike signals in the time interval next to the time interval in which the spike signals are received. Thus, the arithmetic unit 610 functioning as a spiking neural network using the neuron model 611 can receive the input of data for each time interval and process each input data. In this regard, according to the arithmetic unit 610, the spiking neural network can perform data processing efficiently.

[0170] Also, since the neuron model 611 outputs spike signals in the time interval next to the time interval in which the spike signals are received, the output timing of the spike signals within the time interval can be adjusted. In the arithmetic unit 610, for example, it is possible to avoid the concentration of the input of spike signals near the end of the time interval in the layer on the downstream side of the data flow in the spiking neural network.

[0171] FIG. 21 is a diagram showing a configuration example of the neural network system according to the embodiment. In the configuration shown in FIG. 21, the neural network system 620 includes a neural network main body 621 and a learning unit 626. The neural network main body 621 includes a neuron model 622. The neuron model 622 includes an index value calculation unit 623, a detection unit 624, and a signal output unit 625.

[0172] With such a configuration, the index value calculation unit 623 changes the index value of the signal output based on the input status of the signal in each time interval. The detection unit 624 detects the occurrence timing of a predetermined event related to the index value. The signal output unit 625 outputs a signal at a timing corresponding to the timing within the first time interval of the time intervals and the occurrence timing of the predetermined event in the second time interval which is a time interval prior to the first time interval. The learning unit 626 performs learning of the weight coefficient for the signal.

[0173] The index value calculation unit 623 corresponds to an example of the index value calculation means. The signal output unit 625 corresponds to an example of the signal output means. The learning unit 626 corresponds to an example of the learning means. In the neural network system 620, thereby, the weight coefficient can be adjusted by learning, and the estimation accuracy by the neural network main body 621 can be improved.

[0174] FIG. 22 is a diagram showing a configuration example of the neuron model device according to the embodiment. In the configuration shown in FIG. 22, the neuron model device 630 includes an index value calculation unit 631, a detection unit 632, and a signal output unit 633. With such a configuration, the index value calculation unit 631 changes the index value of the spike signal output based on the input status of the spike signal in each time interval. The detection unit 632 detects the occurrence timing of a predetermined event related to the index value. The signal output unit 633 outputs a spike signal at a timing corresponding to the timing within the first time interval of the time intervals and the occurrence timing of the predetermined event in the second time interval which is a time interval prior to the first time interval.

[0175] In this way, by setting the input time interval during which the neuron model device 630 receives signal input and the output time interval during which the neuron model device 630 outputs spike signals, the time during which the index value calculation unit 631 should calculate the index value can be limited to the time from the start of the input time interval to the end of the output time interval. At other times, the neuron model device 630 can perform processing on other data. According to the neuron model device 630, in this regard, the spiking neural network can efficiently perform data processing.

[0176] Also, by the neuron model device 630 outputting a spike signal in the time interval next to the time interval in which the spike signal input is received, the output timing of the spike signal within the time interval can be adjusted. According to the neuron model device 630, for example, it is possible to avoid the input of spike signals concentrating near the end of the time interval in the layer on the downstream side of the data flow in the spiking neural network.

[0177] FIG. 23 is a flowchart showing an example of a processing procedure in the arithmetic method according to the embodiment. The arithmetic method shown in FIG. 23 includes changing an index value (step S611), detecting the occurrence timing of an event (step S612), and outputting a signal (step S613). In changing the index value (step S611), for each time interval, the index value of signal output is changed based on the signal input situation in that time interval. In detecting the occurrence timing of an event (step S612), the occurrence timing of a predetermined event regarding the index value is detected. In outputting a signal (step S613), the signal is output at a timing within the first time interval of the time interval and at a timing corresponding to the occurrence timing of the predetermined event in the second time interval which is a time interval earlier than the first time interval.

[0178] In the operation method shown in FIG. 23, by accepting the input of spike signals for each time interval and outputting spike signals in the time interval next to the time interval in which the input of spike signals is received, the spiking neural network can accept the input of data for each time interval and process each input data. In this regard, according to the operation method shown in FIG. 23, the spiking neural network can efficiently perform data processing.

[0179] Also, in the operation method shown in FIG. 23, by outputting spike signals in the time interval next to the time interval in which the input of spike signals is received, the output timing of spike signals within the time interval can be adjusted. According to the operation method shown in FIG. 23, for example, it is possible to avoid the concentration of the input of spike signals near the end of the time interval in the layer on the downstream side of the data flow in the spiking neural network.

[0180] FIG. 24 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 24, the computer 700 includes a CPU 710, a main storage device 720, an auxiliary storage device 730, an interface 740, and a non-volatile recording medium 750.

[0181] Any one or more or a part of the above neural network device 10, learning device 50, arithmetic device 610, neural network system 620, and neuron model device 630 may be implemented in the computer 700. In that case, the operations of the above-described respective processing units are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above processing according to the program. Further, the CPU 710 secures a storage area corresponding to each of the above-described storage units in the main storage device 720 according to the program. Communication between each device and other devices is executed by the interface 740 having a communication function and performing communication under the control of the CPU 710.

[0182] When the neural network device 10 is implemented in the computer 700, the operations of the neural network device 10 and its respective parts are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above processing according to the program.

[0183] Also, the CPU 710 secures a storage area for the processing of the neural network device 10 in the main storage device 720 according to the program. Communication between the neural network device 10 and other devices is executed by the interface 740 having a communication function and operating according to the control of the CPU 710. Interaction between the neural network device 10 and the user is executed by the interface 740 including a display device and an input device, performing the display of various images according to the control of the CPU 710, and accepting user operations.

[0184] When the learning device 50 is implemented in the computer 700, the operation of the learning device 50 is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above processing according to the program.

[0185] Also, the CPU 710 secures a storage area for the processing of the learning device 50 in the main storage device 720 according to the program. Communication between the learning device 50 and other devices is executed by the interface 740 having a communication function and operating according to the control of the CPU 710. Interaction between the learning device 50 and the user is executed by the interface 740 including a display device and an input device, performing the display of various images according to the control of the CPU 710, and accepting user operations.

[0186] When the arithmetic unit 610 is implemented in the computer 700, the operations of the arithmetic unit 610 and its respective parts are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above processing according to the program.

[0187] Also, the CPU 710 secures a storage area for the processing of the arithmetic unit 610 in the main storage device 720 according to the program. Communication between the arithmetic unit 610 and other devices is executed by the interface 740 having a communication function and operating according to the control of the CPU 710. Interaction between the arithmetic unit 610 and the user is executed by the interface 740 including a display device and an input device, displaying various images according to the control of the CPU 710, and accepting user operations.

[0188] When the neural network system 620 is implemented in the computer 700, the operations of the neural network system 620 and its respective parts are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it in the main storage device 720, and executes the above processing according to the program.

[0189] Also, the CPU 710 secures a storage area for the processing of the neural network system 620 in the main storage device 720 according to the program. Communication between the neural network system 620 and other devices is executed by the interface 740 having a communication function and operating according to the control of the CPU 710. Interaction between the neural network system 620 and the user is executed by the interface 740 including a display device and an input device, displaying various images according to the control of the CPU 710, and accepting user operations.

[0190] Note that a program for executing all or part of the processes performed by the neural network device 10, the learning device 50, the arithmetic device 610, the neural network system 620, and the neuron model device 630 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform the processes of each part. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices. Also, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or a storage device such as a hard disk incorporated in a computer system. Further, the above program may be for realizing a part of the above-described functions, and may also be for realizing the above-described functions in combination with a program already recorded in the computer system.

[0191] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Explanation of Reference Numerals

[0192] 1, 620 Neural network system 10 Neural network device 11 Neural network 50 Learning device 100, 611, 622 Neuron model 110, 211, 612, 623, 631 Index value calculation unit 120, 212, 613, 624, 632 Detection unit 130, 130b Delay unit 140, 213, 614, 625, 633 Signal output unit 210 Sub-model 210f Delay sub-model 210-1 First Sub-Model 210-2 Second Sub-Model 210-3 Third Sub-Model 211f Delay Index Value Calculation Unit 211-1 First Index Value Calculation Unit 211-2 Second Index Value Calculation Unit 211-3 Third Index Value Calculation Unit 212f Delay Detection Unit 212-1 First Detection Unit 212-2 Second Detection Unit 212-3 Third Detection Unit 213f Delay Signal Output Unit 213-1 First Signal Output Unit 213-2 Second Signal Output Unit 213-3 Third Signal Output Unit 220a, 220b Switching Unit 221, 231 Switch 230 Integration Unit 232 Inverter 610 Arithmetic Unit 621 Neural Network Main Body 626 Learning Unit 630 Neuron Model Device

Claims

1. An index value calculation means for changing an index value of signal output based on an input situation of a signal in each time interval; A detection means for detecting a generation timing of a predetermined event regarding the index value; A signal output means for outputting a signal at a timing corresponding to a timing within a first time interval among the time intervals and a generation timing of the predetermined event in a second time interval which is a time interval earlier than the first time interval; A spiking neuron model comprising the above; An arithmetic unit comprising the above.

2. A delay means for determining a timing for causing the signal output means to output a signal based on a timing after a predetermined time from the timing detected by the detection means; A switching means for switching the delay means for determining the timing among two delay means for each time interval; Further comprising the above; The arithmetic unit according to Claim 1.

3. Further comprising a first spiking neuron model comprising the index value calculation means and the detection means, and two second spiking neuron models comprising the delay means and the signal output means; When the detection means detects the generation timing of the predetermined event, the first spiking neuron model outputs the signal to the second spiking neuron model; The delay means determines the timing for causing the signal output means to output the signal based on a timing after a predetermined time from a timing within the first time interval and a timing when the signal is input from the first spiking neuron model in the second time interval; The arithmetic unit according to Claim 2.

4. Comprising a spiking neural network main body and a learning means; The spiking neural network main body is For each time interval, an index value calculation means for changing an index value of signal output based on an input status of a signal in that time interval; a detection means for detecting a timing of occurrence of a predetermined event regarding the index value; a signal output means for outputting a signal at a timing corresponding to a timing within a first time interval among the time intervals and a timing of occurrence of the predetermined event in a second time interval which is a time interval prior to the first time interval; A spiking neuron model comprising: The learning means performs learning of a weight coefficient for the signal. A neural network system.

5. For each time interval, an index value calculation means for changing an index value of signal output based on an input status of a signal in that time interval; a detection means for detecting a timing of occurrence of a predetermined event regarding the index value; a signal output means for outputting a signal at a timing corresponding to a timing within a first time interval among the time intervals and a timing of occurrence of the predetermined event in a second time interval which is a time interval prior to the first time interval; A neuron model device comprising:

6. For each time interval, changing an index value of signal output based on an input status of a signal in that time interval; detecting a timing of occurrence of a predetermined event regarding the index value; outputting a signal at a timing corresponding to a timing within a first time interval among the time intervals and a timing of occurrence of the predetermined event in a second time interval which is a time interval prior to the first time interval; An arithmetic method including:

7. In a programmable device, for each time interval, changing an index value of signal output based on an input status of a signal in that time interval; Detecting the occurrence timing of a predetermined event regarding the index value; Outputting a signal at a timing corresponding to the timing within the first time interval among the time intervals and the occurrence timing of the predetermined event in the second time interval which is a time interval prior to the first time interval; A program for causing the above to be executed.

Images