Information processing device, and method for controlling information processing device

Abstract

To provide: an information processing device which includes a chip circuit which can reduce electric power consumption in data giving and receiving between chips; and a method for controlling the information processing device.SOLUTION: An information processing device includes a chip circuit. The chip circuit includes: a signal input unit which receives input of a signal; an arithmetic circuit unit which executes predetermined arithmetic using the signal received by the signal input unit; and a sparsification circuit unit which converts information outputted by the arithmetic circuit unit into data in which non-zero bits are sparse.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device having a plurality of chip circuits and a method for controlling the information processing device. [Background technology]

[0002] In recent years, the development of chips for artificial intelligence has progressed. In addition, in the field of chip development, Dennard's law has broken down, and there is a limit to how much the clock frequency can increase, so multi-core chips are being promoted. Specifically, artificial intelligence chips using many processing elements (PEs) have become common in recent years, and it is also becoming common to combine multiple chips for processing.

[0003] On the other hand, reducing the power consumption of such chips is also an issue. To address this issue, for example, sparse compression technology has been developed that stops the operation of a PE to reduce power consumption when the input data to that PE is zero (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Kei Momose, Tetsuya Asai, Deep Learning Chips and AI Calculations, Artificial Intelligence, Japanese Society for Artificial Intelligence, January 2018, Vol. 33, No. 1, p. 23-30 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above conventional technology does not take into account the number of non-zero bits in the data exchanged between PEs or chips, and the reality is that no consideration is given to reducing power consumption when exchanging data between PEs or chips.

[0006] The present invention has been made in consideration of the above-mentioned situation, and one of its objects is to provide an information processing device equipped with a chip circuit that can reduce power consumption, for example, when transmitting data between chips, and a control method for the information processing device. [Means for solving the problem]

[0007] One aspect of the present invention for solving the problems of the conventional examples described above is an information processing device equipped with a chip circuit, the chip circuit including a signal input section that accepts an input signal, an arithmetic circuit section that performs a predetermined operation using the signal accepted by the signal input section, and a sparse circuit section that sparses the information output by the arithmetic circuit section into data in which non-zero bits are sparse. Effect of the Invention

[0008] According to the present invention, it is possible to reduce power consumption in data transmission between PEs on a chip or the like. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating an example of a configuration of an information processing circuit according to an embodiment of the present invention. [Diagram 2] 2 is a schematic configuration diagram illustrating an example of a calculation unit of an information processing circuit according to an embodiment of the present invention. [Diagram 3] 1 is a schematic configuration diagram illustrating an example of a coarsening circuit unit of an information processing circuit according to an embodiment of the present invention. [Figure 4] 1 is a schematic configuration diagram illustrating an example of implementation of an information processing circuit according to an embodiment of the present invention. [Diagram 5] 4 is a flowchart illustrating an example of the operation of the information processing circuit according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An embodiment of the present invention will be described with reference to the drawings. An information processing device 1 according to the embodiment of the present invention is generally configured to include a plurality of information processing circuits (chip circuits) 10 that are communicatively connected to each other, as shown in Fig. 1. In this embodiment, each chip circuit mimics a biological neuron.

[0011] Here, each chip circuit 10 corresponds to a processing element (PE) and has a signal input unit 11, an encoder unit 12, a transmitter unit 13, a receiver unit 14, and a decoder unit 15, which are connected to each other via a communication path L. The encoder unit 12 of the chip circuit 10 includes a calculation unit 121 and a coarsening circuit unit 122. The information processing device 1 may further include a processor unit 50 that controls each chip circuit 10 and processes its output data, and includes a programmable CPU, a memory device, an input / output circuit, etc.

[0012] The signal input unit 11 of the chip circuit 10 receives an analog signal x (x is a set of D-dimensional real values) input from the outside or from a decoder unit 15 described later, and outputs it to the encoder unit 12. Note that the chip circuit 10 that only receives the analog signal x input from the outside does not necessarily have to include the receiver unit 14 and the decoder unit 15 described later.

[0013] The arithmetic unit 121 of the encoder unit 12 can be individually configured in accordance with the problem that the information processing device 1 is to solve. One example of this arithmetic unit 121 is configured as a hierarchical neural network. This arithmetic unit 121 performs a calculation to convert an input analog signal into a binary representation (i.e., quantizes it). When an input analog signal x has D=1 (i.e., when the analog signal x is a one-dimensional real value), this arithmetic unit 121 converts the analog signal x into

number

[0014] In an example of this embodiment, if this arithmetic unit 121 satisfies x > 2 -1 , then set σ0 = 1 (the decision in the first stage). Otherwise, set σ0 = 0. The arithmetic unit 121

Number

Number

[0015] Note that here, bi is defined as, for example

Number

[0016] In an example of this embodiment, this arithmetic unit 121 is realized by a neural network of a binary decision tree. That is, this arithmetic unit 121 is realized by a hierarchical neural network including a pair of i-th stage nodes 21 (i = 1, 2,...), as exemplified in FIG. 2, for example. In the example of FIG. 2, the nodes 21 up to the fourth stage are illustrated, but this is just an example, and there may be more stages.

[0017] In this example, the state h ± i(t) of the i-th stage node 21 at a certain time t + 1 is determined using the outputs (spikes) ω + j and ω - j of the j-th stage node 21 (j < i) at the previous time (time t), and bj,

number

number

number

[0018] In this example, the calculation unit 121 updates the state of each node 21 using equation (1) based on the input analog signal x and the value of the spike at that time, and further calculates the spike ω ± For example, in equation (2), P(ω ± If i(t)=1)>T, then ω ± Let i(t) = 1, otherwise ω ± Let i(t) = 0. Here, T is a threshold value, which is determined in advance as, for example, T = 1 / 2.

[0019] The setting of bi is not limited to the above example, and it is sufficient to set bi>0. For example,

number

[0020] The sparsification circuit unit 122 receives data that is the result of the calculation output by the calculation unit 121, and generates and outputs n-bit data based on the data. Regardless of the value of the analog signal x input to the encoder unit 12, the sparsification circuit unit 122 converts at most m bits of the n bits to non-zero bits that are active bits (bits expressed by a state in which a current flows) (this is called sparsification).

[0021] As a specific example, the sparsifying circuit unit 122 can be realized by applying a winner-takes-all network (hereinafter referred to as a WTA network) as illustrated in Fig. 3. The sparsifying circuit unit 122 of this embodiment using the WTA network includes an input layer having n nodes 31 (n is a natural number of 2 or more), and a competition layer 32. This competitive layer 32 has two nodes 321j (j=1, 2).

[0022] The i-th node 31i of the input layer receives bi ω + The sparsifying circuit unit 122 receives an input of i(t) (hereinafter simply denoted as bi·ωi(t)) of the node 31i at time t. The sparsifying circuit unit 122 then calculates the internal state ui(t+1) of the node 31i at time t+1 by using the output (spike) yi(t) of the node 31i at time t, this bi·ωi(t), and the output z of the competitive layer 32 (this z is a vector quantity in which the output of each node 321 included in the competitive layer 32 is arranged in a predetermined order), as follows:

number

[0023] Furthermore, the sparsifying circuit unit 122 calculates the internal state vj(t+1) of the j-th node 321j of the competitive layer 32 at time t+1 by using the output y(t+1) of the node 31 at time t+1 (where y is a vector quantity with yi(t+1) as an element):

number

[0024] Here, ωE is the weight from node 31 to node 321 of the competitive layer 32, and is set to, for example, {1 / m}i (a vector with the same dimension (number of elements) as yi(t+1) and each element being "1 / m"). Also, δj is the bias (a predetermined constant) of node 321 (an inhibitory neuron) of the competitive layer 32, and is set to, for example,

number

[0025] The sparsifying circuit unit 122 updates the internal states using equations (3) and (4) and calculates the probability that the output (spike) of the corresponding node 31, 321 will be “1” using the internal states ui(t) and vj(t) at time t as follows:

number

[0026] If P(yi(t)=1)>Q, the thinning circuit unit 122 sets yi(t)=1; otherwise, yi(t)=0. Here, Q is a threshold value, which is predefined, for example, as Q = 1 / 2. Similarly, if P(zi(t)=1)>Q′, the thinning circuit unit 122 sets zi(t)=1; otherwise, zi(t)=0. This Q′ is also a threshold value, which is predefined, for example, as Q′ = 1 / 2. The threshold values T, Q, and Q′ may be determined independently or may be made to coincide with each other.

[0027] Each time an analog signal x is input to the encoder unit 12, the thinning circuit unit 122 updates (increments) the time t, determines u, v, y, and z at that time, and outputs y(t) (a vector with yi(t) as components). As already explained, this y(t) is determined by the function of the WTA network such that m (m < n) of its components become "1" and the others become "0".

[0028] The transmission unit 13 is a transmitter circuit that transmits an n-bit signal including transistors provided corresponding to each of yi(t) (i = 1, 2,..., n), which are components of y(t). Each transistor turns on when the corresponding yi(t) is "1" and supplies current to the corresponding line of the communication path L. Also, the transistors included in this transmission unit 13 are controlled to turn off when the corresponding yi(t) is "0" and not supply current to the corresponding line of the communication path L (therefore, the communication path L has at least n lines).

[0029] In this embodiment, since the number of yi(t) that become "1" is limited to at most m (m in the above example, m < n), the number of transistors that turn on (active transistors) is reduced (it can be in a dark silicon state), and the power density is limited regardless of the power efficiency of each transistor. Also, because of this, according to the example of this embodiment, chip design in a relatively small manufacturing process can be considered.

[0030] The receiver 14 receives a signal from the transmitter 13 of its own or another chip circuit 10 via the communication path L, and outputs an n-bit binary signal. The receiver 14 generates an n-bit binary signal in which the bits to which current is supplied to the corresponding line are set to "1" and the bits to which no current is supplied are set to "0" among the n bits, and outputs the signal to the decoder 15.

[0031] The decoder unit 15 obtains an analog signal (real value) by calculating the inner product of an n-dimensional vector y (vector with yi as an element) with each bit of y(t) received by the receiver unit 14 as an element, and a vector b={bi} (vector with bi as an element), and outputs the value of the analog signal to the signal input unit 11 or to the outside. That is, the value of the analog signal output by the decoder unit 15 is

number

[0032] When the receiving unit 14 receives y(t) from multiple chip circuits 10, the decoder unit 15 may combine these (sparse data) (for example, by weighting and summing corresponding components) and use the combined result as vector y (coupling).

[0033] [Operation] The information processing device 1 according to the embodiment of the present invention basically has the above-mentioned configuration and operates as follows: In the following, an example of the case where the information processing device 1 solves a combination problem will be described.

[0034] As a concrete example of a combinatorial problem, here is the Ising Hamiltonian:

number

number

number

number

[0035] It is widely known that these equations can be used to find the ground state of the Ising Hamiltonian, so we will not explain them here.

[0036] Here, ui is the initial state of the node 21 (neuron) of the calculation unit 121, and ei represents the amplitude heterogeneity error correction term (KP Kalinin and NGBerloff, New J. Phys. 20, 113023 (2018)). This ei is used as a post-synaptic coupling strength for updating the analog signal xi. In this example of the present embodiment, ei is normalized so that the average of ei is 1 at any time.

[0037] The analog signal xi represents the firing rate of the Hopfield neural network and corresponds to the value to be calculated. Furthermore, the time step in each equation is taken to be the same as in the Euler method, TIFF2025019438000021.tif14170 takes a value between "-1" and "1" depending on the probability represented by the corresponding xi. The value representing the sign of TIFF2025019438000022.tif14170, i.e. The value ("-1" if TIFF2025019438000023.tif14170 is less than 0, and "1" otherwise) corresponds to the desired state s.

[0038] The information processing device 1 of this example has chip circuits 10l (l=1, 2...; in FIG. 4, shown as 10-1, 10-2... to distinguish the symbols) connected to each other by communication paths L, as shown in FIG. 4, and each chip circuit 10l accepts and combines (couples) signals (sparse data) output from its own and other chip circuits 10 and accepts them as input data. Specifically, under the above settings, each chip circuit 10l operates as follows.

[0039] Each chip circuit 10 executes the following process illustrated in FIG. 5 at each time t=0, 1, ... τ. Note that bi is set in advance to b=1.3 (a positive real value of 2 or less), etc.

number

[0040] The k-th chip circuit 10k receives the signals (sparse data) output by itself and the other chip circuits 10, weights them by ωij as follows, decodes them as the operation of the decoder unit 15, and then couples them (S11; in the following process description, the equal sign in the formula is used to mean substituting the calculation result on the right side for the left side):

number

[0041] In this example of the present embodiment, the output of each chip circuit 10 may be determined randomly at first.

[0042] Next, the chip circuit 10k uses the sparsified data yk,j output by itself to

number

number

number

[0043] Next, the chip circuit 10k repeats the following process for time t'=1, 2, ...T. That is, the chip circuit 10k

number

number

[0044] Also, the chip circuit 10k [Number] is obtained, and using this, [Number] as the probability that zk,j is "1", for example if TIFF2025019438000035.tif18170, then zk,j = 1; otherwise, zk,j = 0 (S16: Setting of the conflict layer). The chip circuit 10k repeats the processes from S14 to S16 T times.

[0045] The chip circuit 10k outputs the yk,i obtained after T times of repeated processing to another chip circuit 10 via the transmission unit 13. At this time, the yk,i determined by the processing here is n-bit binary, and among them, only m (m < n) bits determined by the settings of ωR, ωI, ωE are "1" (non-zero bits), and the other bits are set to be "0" (zero bits).

[0046] The transmission unit 13 includes transistors provided corresponding to each of the yk,i (i = 1, 2,..., n) to be transmitted. Each transistor supplies current to the corresponding line of the communication path L when the corresponding yi(t) is "1". Also, the transistors included in this transmission unit 13 are controlled to be off when the corresponding yk,i is "0" and not supply current to the corresponding line of the communication path L.

[0047] The chip circuit 10k, in the same manner as in step S12 obtains TIFF2025019438000036.tif14170 and designates its sign as si, and stores the vector (the vector with elements si) as sk in a storage device such as a memory (not shown) (S17).

[0048] Each chip circuit 10 repeats the above processing at each time t = 0, 1, … τ. Then, for example, the information processing apparatus 1 uses the value sk obtained by each chip circuit 10k through the above processing by its processor unit 50, and calculates the Hamiltonian, that is, the energy of the Ising model: calculates TIFF2025019438000037.tif38170 and outputs it together with σk (k = 1, 2, …), which is the output of each chip circuit. Here, let σk = sk, and let σkσl denote the inner product of σk and σl.

[0049] According to this embodiment, in the transmission and reception of data between a plurality of chip circuits 10 performed in the process of this calculation, on the communication path L (n-bit communication path L) between the chip circuits 10, m or more bits (m < n) are controlled so as not to become "1" simultaneously. Also, the value of m is controllable. Thereby, the number of transistors that become active all at once among the transistors that control the output signals of each chip circuit 10 can be limited, and thus the power consumption in the data transfer between chips can be reduced.

[0050] [Another example of the calculation unit] In the examples so far, the calculation unit 121 has been assumed to convert the input analog signal into a binary representation (i.e., quantize it), but the present embodiment is not limited to this. For example, the calculation unit 121 may perform a linear transformation. In this example, the calculation unit 121 uses a vector quantity (analog information) x consisting of D-dimensional (i.e., D) real values inputted to perform the conversion of Q[x]=βx + α Here, β is an n×D matrix, and α is an n-dimensional vector. That is, Q[x] becomes an n-dimensional vector quantity.

[0051] In this example, the thinning circuit unit 122 receives the input of the data of the n-dimensional vector quantity, which is the result of the calculation output by the calculation unit 121, and based on the data, generates and outputs data W[Q[x]] which is n-bit data and in which m bits (m < n) are active bits.

[0052] [Machine Learning] Furthermore, the neural network of this calculation unit 121 may determine parameters (eg, the above-mentioned α, β, and b described below) by machine learning.

[0053] In this example, the input data x of the calculation unit 121 and the output of the calculation unit 121 for the input data x are sparsified by the sparsification circuit unit 122 to obtain data y (n-bit binary value). The data y is treated as an n-dimensional vector with each bit as an element, and multiplied from the left by b (the elements are real numbers and are initially initialized randomly), which is a matrix of dimensions D×n, to obtain a D-dimensional real value o. The root of the squared error of the difference between o and x is:

number

[0054] As for the error, the squared error is

number

[0055] [When the inter-chip communication path is an optical communication path] In the explanation so far, the communication path L between the chip circuits 10 has not been particularly limited and may be an electrical communication path, but the communication path L in this embodiment may include at least n optical communication paths (realized by light guide paths such as optical fibers).

[0056] In this case, the transmitter 13 of the chip circuit 10 is a transmitter circuit capable of simultaneously transmitting signals for n bits, including n sets of transistors provided corresponding to each of yi(t) (i = 1, 2, …, n) and VCSELs (Vertical Cavity Surface Emitting Lasers) driven by these transistors. Each transistor turns on (becomes active, i.e., in an energized state supplying current) when the corresponding yi(t) is “1” to cause the corresponding VCSEL to emit light. Also, the transistors included in this transmitter 13 turn off when the corresponding yi(t) is “0” to turn off the corresponding VCSEL.

[0057] Each VCSEL is coupled to a corresponding light guiding path such as an optical fiber in the communication path L, and the light emitted by the VCSEL is sent through the corresponding light guiding path such as an optical fiber.

[0058] In this example of the present embodiment, since the number of yi(t) that become “1” is limited to at most m (m in the above example, m < n), the number of transistors that turn on (active transistors) is reduced, and by limiting the number of VCSELs that emit light, the power density is limited regardless of the power efficiency of each transistor.

[0059] Also, the receiver 14 in this example receives an optical signal via n or more light guiding paths that are the communication path L with the transmitter 13 of another chip circuit 10. The receiver 14 in this example includes a photodetector such as a photodiode for each light guiding path. Then, when an optical signal arrives via the corresponding light guiding path, it outputs a bit representing “1”, and outputs a bit representing “0” while no optical signal is arriving. Thereby, based on the optical signal received via the communication path L, this receiver 14 outputs an n-bit binary signal represented by the optical signal to the decoder unit 15.

[0060] Furthermore, in this example of the present embodiment, a half mirror or the like may be provided on the light guide path to split the optical signal sent through the light guide path. Also, a modulator corresponding to each chip circuit 10 may be provided, and the signal from the light guide path connected to the corresponding transmitting unit 13 may be modulated and combined by this modulator, and transmitted through one light guide path (called a backbone path). In this example, a demodulator may be further provided for each chip circuit 10 on the backbone path, and the optical signal sent from the chip circuit 10 other than the corresponding chip circuit 10 may be demodulated, and the demodulated optical signal may be sent to the corresponding receiving unit 14 via the light guide path. [Explanation of symbols]

[0061] 1 information processing device, 10 chip circuit, 11 signal input section, 12 encoder section, 13 transmitter section, 14 receiver section, 15 decoder section, 21, 31, 321 node, 32 competitive layer, 50 processor unit, 121 calculation section, 122 sparsification circuit section.< / x>

Claims

1. A signal input unit for receiving an input of a signal; an arithmetic circuit section that performs a predetermined operation using the signal received by the signal input section; and a sparsifying circuit section that sparsifies the information output by the arithmetic circuit section into data in which non-zero bits are sparse.

2. 2. The information processing device according to claim 1, The arithmetic circuit unit is an information processing device having a quantization circuit that quantizes the signal received by the signal input unit.

3. 2. The information processing device according to claim 1, The information processing device, wherein the sparsifying circuit unit is composed of a Winner-TakeAll (WTA) network, and encodes input data so that the number of non-zero bits in the n-bit output data does not exceed a predetermined number m (m<n).

4. The information processing device according to claim 1 . a plurality of the chip circuits are provided, each chip circuit being communicatively connected to each other via a communication path having an n-bit width; a signal input unit included in at least a portion of the plurality of chip circuits receives an input of a signal output by another chip circuit via the communication path; The coarsening circuit unit coarsens the information output by the arithmetic circuit unit into n-bit wide data with m non-zero bits (m<n), and transmits the coarsened data to other chip circuits via the communication path.

5. 5. The information processing device according to claim 4, The information processing device includes at least n communication paths each having an n-bit width.

6. A method for controlling an information processing device having a plurality of chip circuits, comprising the steps of: Each of the chip circuits is Accepts signal input, A predetermined calculation is performed using the signal received by the signal input unit, A control method for an information processing device that controls the result of the operation to be sparsely populated into data having sparse non-zero bits.

Images