Calculation device, learning control device, calculation method, learning control method and program

Abstract

The arithmetic device includes a matrix product calculation means for calculating the product of a connection representation matrix having a plurality of elements each taking a value of 1, 0, or -1 and a vector indicating an intermediate node value in a machine learning model; a shift operation means for performing a shift operation on a bit string in binary representation of each of the plurality of elements of the vector obtained by the product; a summation means for summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; an activation function application means for applying an activation function to each of the plurality of elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values ​​to calculate a vector indicating an update value of the intermediate node value as time steps progress; and an output value calculation means for weighting the update value of the intermediate node value to calculate a plurality of output values.

Description

[Technical field]

[0001] The present invention relates to a calculation device, a learning control device, a calculation method, a learning control method, and program Regarding. [Background technology]

[0002] One of the information processing techniques using neural networks is reservoir computing (RC) (see, for example, Patent Document 1). In reservoir computing, a recurrent neural network (RNN) is used as the neural network structure, and the learning target is limited to the weight coefficients of the connections in the output layer. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2018 / 0309266 Summary of the Invention [Problem to be solved by the invention]

[0004] It would be preferable to be able to simplify calculations in machine learning models having recursive network structures such as reservoir computing.

[0005] One example of the object of the present invention is to provide a calculation device, a learning control device, a calculation method, a learning control method, and program The purpose of this project is to provide [Means for solving the problem]

[0006] According to a first aspect of the present invention, a calculation device includes: a matrix product calculation means for calculating a product of a connection representation matrix having a plurality of elements each taking a value of 1, 0, or -1 and a vector indicating an intermediate node value in a machine learning model; a shift operation means for performing a shift operation on a bit string in binary representation of each of the plurality of elements of the vector obtained by the product; a summation means for summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; an activation function application means for applying an activation function to each of the plurality of elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values ​​to calculate a vector indicating an update value of the intermediate node value as a time step progresses; and an output value calculation means for weighting the update value of the intermediate node value to calculate a plurality of output values.

[0007] According to a second aspect of the present invention, a calculation method is implemented in a calculation device by: calculating a product of a connection representation matrix, which is a matrix having a plurality of elements each taking a value of 1, 0, or -1, and a vector indicating an intermediate node value in a machine learning model; performing a shift operation on a bit string in binary representation of each of the plurality of elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; applying an activation function to each of the plurality of elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values ​​to calculate a vector indicating an update value of the intermediate node value as time steps progress; and weighting the update value of the intermediate node value to calculate a plurality of output values.

[0008] According to a third aspect of the present invention, a learning control method includes: A learning control devicea vector indicating an update value of the intermediate node value as a time step progresses by applying an activation function to each of the elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the weighted input values, and a learning control method adapted to a calculation device that calculates a plurality of output values ​​by weighting the update values ​​of the intermediate node values, the learning control method comprising: calculating a product of a connection representation matrix having a plurality of elements taking the values ​​of 1, 0, or -1 and a vector indicating an intermediate node value in a machine learning model; performing a shift operation on a bit string in binary representation of the element for each of the elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; applying an activation function to each of the elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values, the learning control method being adapted to a calculation device that calculates a plurality of output values ​​by weighting the update values ​​of the intermediate node ... including: setting a value of each element of the plurality of elements of the connection representation matrix of the calculation device so that the value of the element becomes 0 with a predetermined probability; and causing the calculation device, for which the values ​​of the elements of the connection representation matrix have been set, to perform learning and update the weighting coefficient for calculating the output values.

[0009] According to a fourth aspect of the present invention, program a device operable according to a program that implements the following steps: calculating a product of a connection representation matrix having a plurality of elements each taking a value of 1, 0, or -1 and a vector indicating an intermediate node value in a machine learning model; performing a shift operation on a bit string in binary representation of each of the plurality of elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; applying an activation function to each of the plurality of elements of the vector obtained by summing the vector obtained by the shift operation and the vector including the plurality of weighted input values ​​to calculate a vector indicating an update value of the intermediate node value as a time step progresses; and weighting the update value of the intermediate node value to calculate a plurality of output values. do.

[0010] According to a fifth aspect of the present invention, programa computer implements the following steps in a calculation device: calculates a product of a connection representation matrix having a plurality of elements taking the values ​​of 1, 0, or -1 and a vector indicating an intermediate node value in machine learning; for each of the plurality of elements of the vector obtained by the product, performs a shift operation on a bit string in binary representation of the element; sums the vector obtained by the shift operation and a weighted input value vector; applies an activation function to each of the plurality of elements of a vector obtained by summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted by a weighting coefficient in the machine learning to calculate a vector indicating an update value of the intermediate node value as a time step progresses; and weights the update value of the intermediate node value to calculate a plurality of output values. The computer implements the following steps in a calculation device: calculates a product of a connection representation matrix having a plurality of elements taking the values ​​of 1, 0, or -1 and a vector indicating an intermediate node value in machine learning; do. Effect of the Invention

[0011] According to the present invention, calculations in a calculation model having a recursive network structure can be performed relatively easily. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a device configuration of a computing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a configuration of a calculation device according to an embodiment. [Diagram 3] FIG. 1 is a diagram illustrating an example of a configuration of a learning control device according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating a first example of a relationship between sparsity and a spectral radius of a machine learning model of a computing device according to an embodiment. [Diagram 5] FIG. 11 is a diagram illustrating a second example of the relationship between the sparsity and the spectral radius of the machine learning model of the arithmetic device according to the embodiment. [Figure 6]4 is a flowchart showing an example of a processing procedure of the learning control device according to the embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of a relationship between sparsity and estimation accuracy of a computing device according to an embodiment. [Figure 8] FIG. 13 is a diagram illustrating another example of the configuration of a calculation device according to the embodiment. [Figure 9] FIG. 4 is a diagram illustrating another example of the configuration of the learning control device according to the embodiment. [Figure 10] 1 is a flowchart showing an example of a processing procedure in a calculation method according to the embodiment. [Figure 11] 4 is a flowchart showing an example of a processing procedure in a learning control method according to the embodiment. [Figure 12] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. 1 is a diagram showing an example of the device configuration of a computing system according to an embodiment. In the configuration shown in FIG.

[0014] The arithmetic device 10 performs calculations using a machine learning model having a recursive network structure. However, the recursive network structure does not have to be explicitly stated. For example, the signal lines in the hardware constituting the arithmetic device 10 may be wired in a shape including a loop, but the method of configuring the arithmetic device 10 is not limited to a specific method. In the following, an example will be described in which a machine learning model having a recursive network structure is represented by a calculation formula using a matrix. The machine learning model having a recursive network structure used by the calculation device 10 is also simply referred to as a model. Machine learning is also simply referred to as learning.

[0015] The learning control device 20 controls the arithmetic device 10 during learning to make it learn. The arithmetic device 10 has parameters that are to be updated by learning and parameters that are not to be updated by learning. The learning control device 20 sets the values ​​of the parameters that are not to be updated by learning, and then makes the arithmetic device 10 perform learning. The learning control device 20 may change the values ​​of the parameters that are not to be updated by learning and make the arithmetic device 10 perform learning with each value. Then, the learning control device 20 may determine the values ​​of the parameters that are not to be updated by learning based on the learning results.

[0016] Here, the value of the intermediate node in the reservoir computing system can be expressed as Equation (1).

[0017]

number

[0018] Both t and t-1 are time step indexes and indicate time. Time t-1 is the time step one step before (past) time t. In reservoir computing, the input node obtains an input value at each time step, and the value of the output node is updated at each time step. The value of the output node corresponds to the output value from the reservoir computing system.

[0019] x(t) is a vector that indicates the value of the intermediate node at time t. The intermediate nodes of the reservoir computing system and the elements of the vector x(t) are in one-to-one correspondence. The vector x(t) is represented as a vertical vector. The intermediate tier in reservoir computing is also called a reservoir tier, and the intermediate nodes in reservoir computing are also called reservoir nodes.

[0020] W res is a matrix that indicates the weighting coefficient for the value of the intermediate node. res x(t-1)" indicates the weighting of the intermediate node values ​​at time t-1. res is a square matrix whose number of rows and columns is equal to the number of elements of the vector x(t). res The calculation result of "x(t-1)" is represented as a column vector with the same number of elements as the vector x(t).

[0021] u(t) is a vector that indicates the value of the input node at time t. Therefore, the vector u(t) indicates the input value to the reservoir computing system at time t. The vector u(t) is represented as a vertical vector. W in is a matrix that indicates the weighting coefficient for the input node value. in "u(t)" indicates the weighting of the input node value at time t. The matrix W in The number of rows is equal to the number of elements in vector x(t), and the number of columns is equal to the number of elements in vector u(t). in The calculation result of "u(t)" is expressed as a column vector with the same number of elements as the vector x(t).

[0022] f indicates the activation function. res x(t-1)+W in The calculation result of "f(W res x(t-1)+W in u(t)" represents the application of an activation function to each element of this column vector.

[0023] The product of the matrix and vector in equation (1) "W res x(t-1) and W in For each of "u(t)", the weight coefficient indicated by the matrix element is multiplied by the node value indicated by the vector element. It is possible that the number of intermediate nodes is greater than the number of input nodes, and in particular, "W resIt is likely that there are many multiplications in "x(t-1)".

[0024] Multiplication involves more complex processing than addition or subtraction, such as repeated additions being performed for each multiplication. For this reason, the power consumption of a device performing multiplication is greater than that of adding or subtracting. In addition, the size of the hardware required to implement a multiplier in hardware is relatively large, which may hinder efforts to miniaturize the device and increase the production cost of the device.

[0025] In response to this, the arithmetic device 10 is able to calculate the value of the intermediate node with fewer multiplications than the calculation of equation (1). By simplifying the calculation, it is expected that the processing speed of the arithmetic device 10 will be faster. In addition, by reducing the number of multiplications, the power consumption of the arithmetic device 10 will be reduced, and it is expected that the arithmetic device 10 can be installed in places where sufficient power supply is not expected, such as near the installation location of a sensor. In addition, by making the arithmetic device 10 more compact, it is expected that the number of places where the arithmetic device 10 can be installed will increase. In addition, by reducing the production cost of the arithmetic device 10, it will become relatively easy to arrange a large number of arithmetic devices 10 in a distributed arrangement such as edge computing.

[0026] The computing device 10 calculates the values ​​of the intermediate nodes based on equation (2).

[0027]

number

[0028] Z represents an integer. T is a matrix in which all elements have a value of 1, 0, or -1. The number of rows and the number of columns of the matrix T are both equal to the number of elements of the vector x(t). In other words, the number of rows and the number of columns of the matrix T are both equal to the number of intermediate nodes of the machine learning model of the arithmetic device 10.

[0029] Matrix T can be interpreted as representing the connection relationships between intermediate nodes. An element value of "1" in matrix T can be interpreted as representing that intermediate nodes are connected to each other and information is transmitted. An element value of "0" in matrix T can be interpreted as representing that intermediate nodes are not connected to each other and information is not transmitted. An element value of "-1" in matrix T can be interpreted as representing that intermediate nodes are connected to each other and information is transmitted, and that the transmitted information is multiplied by a weighting coefficient of -1. The matrix T is also called a connection representation matrix.

[0030] Comparing equation (2) with equation (1), "W res The term "x(t-1)" in equation (2) is "2 Z Tx(t-1)". In all other respects, equation (2) is the same as equation (1). The calculation result of "Tx(t-1)" is expressed as a vertical vector with the same number of elements as the vector x(t). The i-th element of the vector is <> i and the element in the i-th row and j-th column of the matrix is ​​<> i,j Then, the i-th element of the vector resulting from the calculation of “Tx(t-1)” is expressed as in equation (3).

[0031]

number

[0032] Elements of matrix T <t> i,j Since the value of x(t-1) can be 1, 0, or -1, in the calculation of formula (3), the element of vector x(t-1)<x(t-1)> j For each bit, it is sufficient to perform either "addition," "subtraction," or "no calculation." Therefore, the calculation of equation (3) can be performed using only an adder, without the need for a multiplier.

[0033] Also, as above, Z is an integer, so "2 Z 2 in the calculation of "Tx(t-1)" Z The multiplication of the elements of the column vector of the calculation result of "Tx(t-1)" with "Tx(t-1)" can be performed by shifting the bit string. Therefore, "2 Z The calculation of "Tx(t-1)" can be done using adders and shift registers, and does not require multipliers.

[0034] The calculation of equation (3) will be explained with an example. Consider an example in which the values ​​of vector x(t-1), matrix T, and integer Z are given by equation (4).

[0035]

number

[0036] In this case, the calculation of "Tx(t-1)" can be performed by addition and subtraction as shown in equation (5).

[0037]

number

[0038] Also, 2 Z The multiplication of the calculation result of "Tx(t-1)" with "Tx(t-1)" can be performed by a shift operation as shown in equation (6).

[0039]

number

[0040] In the example shown in formula (6), for each of the elements "3", "-7" and "-4" of the vertical vector of the calculation result of "Tx(t-1)", the bit string expressed in binary is shifted one bit to the left (towards the larger digit) to obtain 2 1 is multiplied. In addition, in equation (6), negative numbers are expressed as two's complements.

[0041] Regarding the activation function f in formula (1), the activation function used in reservoir computing is not limited to a specific function, but a complex function such as a tangent hyperbolic function may be used. When a complex function is used as the activation function, it is possible that the power consumption of the device will increase, the scale of the hardware implementing the activation function will become relatively large, which will hinder the miniaturization of the device, and the production cost of the device will increase.

[0042] The activation function (function f in equation (2)) used in the arithmetic unit 10 is not limited to a specific function, either, but may be, for example, a polynomial function poly shown in equation (7).

[0043]

number

[0044] Both a and b are real constants. The polynomial function poly corresponds to an example of a third-order polynomial function made up of a third-order term and a first-order term. In an experiment using the polynomial function poly as the activation function of the arithmetic device 10, good results were obtained. When a polynomial function poly is used as the activation function, equation (2) can be expressed as equation (8).

[0045]

number

[0046] By using a relatively simple function such as the polynomial function poly as an activation function, the arithmetic device 10 is expected to have a relatively fast processing speed. In addition, since the processing load of the arithmetic device 10 is relatively light, the power consumption of the arithmetic device 10 is relatively small, and it is expected that the arithmetic device 10 can be installed even in places where sufficient power supply is not expected, such as near the installation location of a sensor. In addition, since the arithmetic device 10 is miniaturized, it is expected that the number of places where the arithmetic device 10 can be installed will increase. In addition, since the configuration of the arithmetic device 10 is relatively simple, the production cost is relatively low, and it is expected that it will be relatively easy to arrange a large number of arithmetic devices 10 in a distributed arrangement such as edge computing.

[0047] Moreover, the arithmetic device 10 calculates its own output value at time t based on the formula (9).

[0048]

number

[0049] y(t) represents the value of the output node of the machine learning model of the arithmetic device 10 at time t. The arithmetic device 10 outputs the value to be set in the output node. W out is a matrix that indicates the weighting coefficients for the values ​​transmitted from the intermediate nodes to the output nodes. out The number of rows is equal to the number of elements in vector y(t), and the number of columns is equal to the number of elements in vector x(t). out "x(t)" indicates that for each output node, the value of the intermediate node at time t is weighted and added.

[0050] The machine learning model of reservoir computing is shown in equations (1) and (9). In reservoir computing, the matrix W out The value of is updated through learning. The machine learning model of the computing device 10 can be used as a kind of reservoir computing. The machine learning model of the computing device 10 shown in formulas (2) and (9) can be obtained by subtracting "W" in formula (1) from the machine learning model of reservoir computing shown in formulas (1) and (9). res x(t-1)" to "2 Z Tx(t-1)".

[0051] Fig. 2 is a diagram showing an example of the configuration of the calculation device 10. In the configuration shown in Fig. 2, the calculation device 10 includes an input value weighting unit 11, a matrix product calculation unit 12, a shift calculation unit 13, a summation unit 14, an activation function application unit 15, an output value calculation unit 16, and an intermediate node value storage unit 17.

[0052] The input value weighting unit 11 acquires an input value to the arithmetic device 10 for each time step, and weights the input value. The processing performed by the input value weighting unit 11 is in This corresponds to an example of the calculation of "u(t)". The input value weighting unit 11 outputs the weighting result to the summation unit 14. The input value weighting unit 11 may be configured using a plurality of multipliers and a plurality of adders. The numerical values ​​handled by the arithmetic unit 10 may be expressed as fixed-length bit strings. For example, the numerical values ​​handled by the arithmetic unit 10 may be expressed as fixed-point 16-digit binary numbers.

[0053] The matrix product calculation unit 12 calculates the product of the connection representation matrix and a vector indicating an intermediate node value. As described above, the connection representation matrix is ​​a matrix in which all elements have a value of 1, 0, or -1. The process performed by the matrix product calculation unit 12 corresponds to an example of the calculation of "Tx(t-1)" in formula (2). The matrix multiplication calculation unit 12 outputs the calculation result of the product to the shift operation unit 13. The matrix multiplication calculation unit 12 may be configured using adders the number of which is the same as the number of intermediate nodes. The matrix multiplication calculation unit 12 corresponds to an example of a matrix multiplication calculation means.

[0054] The shift operation unit 13 performs a shift operation on the bit string of the binary representation of each element of the vector obtained by the multiplication performed by the matrix multiplication calculation unit 12. Z This corresponds to an example of a process of multiplying "Tx(t-1)" by "Tx(t-1)". The shift operation unit 13 outputs the result of the shift operation to the summing unit 14. The shift operation unit 13 may be configured using shift registers the number of which is the same as the number of intermediate nodes. The shift calculation unit 13 corresponds to an example of a shift calculation means.

[0055] The summation unit 14 sums up the vector obtained by the shift operation performed by the shift operation unit 13 and the input value vector weighted by the input value weighting unit 11. The process performed by the summation unit 14 is Z Tx(t-1) and W in This corresponds to an example of calculating the sum of "u(t)". The summing unit 14 outputs the sum result to the activation function application unit 15. The summing unit 14 may be configured using one adder. The totalizing unit 14 corresponds to an example of a totalizing means.

[0056] The activation function application unit 15 applies a function defined as an activation function to each element of the vector obtained by the summation performed by the summation unit 14 to calculate a vector indicating the update value of the intermediate node value due to the progress of the time step. Z Tx(t-1)+W in This corresponds to an example of a process in which the calculation result of "u(t)" is input into the function f to calculate "x(t)".

[0057] As described above, the activation function application unit 15 may use a third-order polynomial function consisting of a third-order term and a first-order term as the activation function. Specifically, the activation function application unit 15 may use a polynomial poly shown in equation (7) as the activation function. The activation function application unit 15 outputs the calculated vector to the output value calculation unit 16. When the activation function application unit 15 uses the polynomial poly as the activation function, the activation function application unit 15 may be configured using one multiplier that calculates a third-order term, one multiplier that calculates a first-order term, and one adder that sums up the third-order term and the first-order term. The activation function application unit 15 corresponds to an example of an activation function application means.

[0058] The output value calculation unit 16 performs weighting on the update value of the intermediate node value to calculate the output value. out This corresponds to an example of the calculation of "x(t)". The calculation result by the output value calculation section 16 is used as the output value of the arithmetic device 10. The output value calculation section 16 may be configured using a plurality of multipliers and a plurality of adders. The output value calculation section 16 corresponds to an example of the output value calculation means.

[0059] The intermediate node value storage unit 17 stores the values ​​of the intermediate nodes calculated by the activation function application unit 15 for the time step of time t. The values ​​stored in the intermediate node value storage unit 17 are output to the matrix product calculation unit 12. The output of the value from the intermediate node value storage unit 17 to the matrix product calculation unit 12 corresponds to an example of using the value of "x(t)" resulting from the calculation of equation (2) as the value of "x(t-1)" for the calculation of equation (2) in the next time step. The intermediate node value storage unit 17 may be configured using D-FFs (D Flip-Flops) in the same number as the product of the number of intermediate nodes and the number of bits per intermediate node.

[0060] The arithmetic device 10 may be configured using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), or a combination of these. In particular, at least the matrix multiplication calculation unit 12 or the activation function application unit 15 may be configured using an FPGA or an ASIC, or a combination of these.

[0061] As a result, the matrix multiplication calculation unit 12 or the activation function application unit 15 is implemented in hardware, and the effects of a relatively fast processing speed of the calculation device 10, a relatively low power consumption of the calculation device 10, a relatively small size of the calculation device 10, a relatively low production cost of the calculation device 10, or a combination of these are expected.

[0062] Fig. 3 is a diagram showing an example of the configuration of the learning control device 20. In the configuration shown in Fig. 3, the learning control device 20 includes a provisional setting unit 21, a learning control unit 22, an evaluation unit 23, and a calculation device setting unit 24. The provisional setting unit 21 provisionally sets values ​​of hyperparameters in the machine learning model of the arithmetic device 10. The hyperparameters referred to here are parameters of the machine learning model other than the parameters to be learned. The parameters to be learned here are parameters whose values ​​are updated by learning.

[0063] When the arithmetic device 10 is used for reservoir computing, the weighting coefficient for the weighting of the value transmitted from the intermediate node to the output node (the matrix W in Equation (9)) out ) are the parameters to be learned. The other parameters are treated as hyperparameters. Therefore, the connection representation matrix and the powers of 2 (matrix T and integer Z in equation (2)) are treated as hyperparameters. In addition, the weight coefficients for weighting the values ​​of the input nodes (matrix W in equation (2) in ) is also treated as a hyperparameter. In addition, when a polynomial function poly is used as the activation function f, the coefficients a and b in equation (8) are also treated as hyperparameters. The provisional setting unit 21 corresponds to an example of a provisional setting means.

[0064] When the provisional setting unit 21 provisionally sets the values ​​of elements of the matrix T (connection representation matrix), the provisional setting unit 21 may set the values ​​of the elements to be 0 with a determined probability. The probability that the value of an element of the matrix T becomes 0 is called sparsity and is represented by "s". For example, the provisional setting unit 21 determines the value of each element of the matrix T such that the value of the element of the matrix T becomes "0" with a probability of sparsity s, becomes "1" with a probability of (1-s) / 2, and becomes "-1" with a probability of (1-s) / 2. The value of the sparsity degree s may be set by the user, or may be set or provisionally set by the provisional setting unit 21.

[0065] Here, the spectral radius can be used as an index value of the accuracy of processing in reservoir computing. The spectral radius is calculated as the maximum eigenvalue of a matrix. In the case of the calculation device 10, Z It is possible to calculate the maximum eigenvalue of the matrix indicated by "T" as the spectral radius, and the sparsity and the value of Z affect the spectral radius.

[0066] Fig. 4 is a diagram showing a first example of the relationship between the sparsity and the spectral radius of the machine learning model of the arithmetic device 10. Fig. 4 shows an example in which the number of intermediate nodes is 100. The horizontal axis of the graph in Fig. 4 indicates the sparsity, and the vertical axis indicates the spectral radius.

[0067] Line L11 shows the relationship between the sparsity and the spectral radius when the value of Z in formula (2) is 0. Line L12 shows the relationship between the sparsity and the spectral radius when the value of Z is 1. Line L13 shows the relationship between the sparsity and the spectral radius when the value of Z is 2. Line L14 shows the relationship between the sparsity and the spectral radius when the value of Z is 3.

[0068] For all of the lines L11 to L14, the spectral radius decreases monotonically as the sparsity increases. This shows that the spectral radius can be adjusted by adjusting the sparsity. Furthermore, comparing lines L11, L12, L13, and L14, the spectral radius becomes smaller as the value of Z becomes larger. From this, when the adjustment of the spectral radius is insufficient only by adjusting the sparsity, the spectral radius can be further adjusted by adjusting the value of Z.

[0069] Fig. 5 is a diagram showing a second example of the relationship between the sparsity and the spectral radius of the machine learning model of the arithmetic device 10. Fig. 5 shows an example in which the number of intermediate nodes is 200. The horizontal axis of the graph in Fig. 5 indicates the sparsity, and the vertical axis indicates the spectral radius.

[0070] Line L21 shows the relationship between the sparsity and the spectral radius when the value of Z in formula (2) is 0. Line L22 shows the relationship between the sparsity and the spectral radius when the value of Z is 1. Line L23 shows the relationship between the sparsity and the spectral radius when the value of Z is 2. Line L24 shows the relationship between the sparsity and the spectral radius when the value of Z is 3.

[0071] In all of the lines L21 to L24, the spectral radius decreases monotonically as the sparsity increases. This shows that the spectral radius can be adjusted by adjusting the sparsity. Furthermore, comparing lines L21, L22, L23, and L24, the spectral radius becomes smaller as the value of Z becomes larger. From this, when the adjustment of the spectral radius is insufficient only by adjusting the sparsity, the spectral radius can be further adjusted by adjusting the value of Z.

[0072] In this way, it is expected that the spectral radius can be adjusted by adjusting the sparsity and the value of Z, regardless of the number of intermediate nodes. The provisional setting unit 21 sets the element values ​​of the connection representation matrix (matrix T) based on the sparsity, so that the user or the arithmetic device 10 can relatively easily adjust the spectral radius. A parameter in the process performed by the learning control device 20, such as the sparsity degree s, is also referred to as a parameter of the learning control device 20.

[0073] The learning control unit 22 causes the arithmetic device 10, in which values ​​of hyperparameters such as values ​​of elements of a connection representation matrix are set, to perform learning, and calculates weighting coefficients (matrix W out The value of The learning control unit 22 corresponds to an example of a learning control means. The evaluation unit 23 evaluates the learning result of the arithmetic device 10 .

[0074] The arithmetic device setting section 24 sets the arithmetic device 10 based on the learning result of the arithmetic device 10 . The provisional setting unit 21 may set the values ​​of the hyperparameters in a plurality of ways, and the learning control unit 22 may cause the arithmetic device 10 to learn for each setting of the value of the hyperparameter. Then, the evaluation unit 23 may evaluate the learning result for each setting of the value of the hyperparameter, and the arithmetic device setting unit 24 may set the value of the hyperparameter when the learning result is most highly evaluated, to the arithmetic device 10. This is expected to enable the value of the hyperparameter to be set so as to increase the accuracy of the processing of the arithmetic device 10.

[0075] FIG. 6 is a flowchart showing an example of a processing procedure of the learning control device 20. 6, the provisional setting unit 21 provisionally sets parameter values ​​of the learning control device 20 (step S11). In particular, the learning control device 20 provisionally sets the value of the sparseness degree s. Next, the provisional setting unit 21 provisionally sets parameter values ​​of the arithmetic device 10 (step S12). Specifically, the provisional setting unit 21 provisionally sets hyperparameter values ​​of the arithmetic device 10, and initially sets parameter values ​​of the learning target. The provisional setting unit 21 sets element values ​​of the connection representation matrix T based on the value of the sparsity degree s.

[0076] Next, the learning control unit 22 controls the arithmetic device 10 to perform learning (step S13). The learning control unit 22 learns the matrix W out Update the value of the element. Next, the evaluation unit 23 evaluates the learning result (step S14). For example, when the arithmetic device 10 performs class classification, the evaluation unit 23 evaluates the accuracy of class classification by the arithmetic device 10 after learning.

[0077] Next, the learning control unit 22 judges whether or not a learning end condition is satisfied (step S15). The condition used by the learning control unit 22 for the judgment is not limited to a specific one. For example, the learning control unit 22 may judge whether or not the number of times the learning has been performed is equal to or greater than a predetermined number. Alternatively, the learning control unit 22 may judge whether or not the evaluation value of the learning result calculated by the evaluation unit 23 is equal to or greater than a predetermined value.

[0078] When the learning control unit 22 determines that the learning termination condition is not satisfied (step S15: NO), the provisional setting unit 21 determines whether or not a reset condition for the parameter value of the learning control device 20 is satisfied (step S21). The condition used by the provisional setting unit 21 for the determination may be, but is not limited to, whether learning has been performed a predetermined number of times or more with the set parameter value.

[0079] If the provisional setting unit 21 determines that the resetting condition is not satisfied (step S21: NO), the process returns to step S12. In this case, the provisional setting unit 21 does not change the parameter values ​​of the learning control device 20, but changes the parameter values ​​of the arithmetic device 10 by provisionally resetting the parameter values ​​of the arithmetic device 10 in step S12.

[0080] On the other hand, if the provisional setting unit 21 determines that the resetting condition is satisfied (step S21: YES), the process returns to step S11. In this case, the provisional setting unit 21 changes the parameter values ​​of the learning control device 20 by provisionally resetting the parameter values ​​of the learning control device 20 in step S11.

[0081] On the other hand, when the learning control unit 22 determines that the learning end condition is satisfied (step S15: YES), the arithmetic unit setting unit 24 determines the parameter values ​​of the arithmetic unit 10 (step S31). For example, the arithmetic unit setting unit 24 selects the learning that has been most highly evaluated by the evaluation unit 23. Then, the arithmetic unit setting unit 24 determines the parameter values ​​of the arithmetic unit 10 at the end of the selected learning as the parameter values ​​to be set in the arithmetic unit 10. Then, the arithmetic device setting unit 24 sets the determined parameter values ​​in the arithmetic device 10. After step S32, the learning control device 20 ends the process of FIG.

[0082] Fig. 7 is a diagram showing an example of the relationship between the sparsity and the estimation accuracy of the calculation device 10. Fig. 7 shows the results of an experiment using NARMA10 as a benchmark task. In the experiment, the calculation device 10 receives time series data up to time t-1 and estimates the value of the data at time t.

[0083] The calculation device 10 was trained using 2000 steps of NARMA10 (NARMA stands for nonlinear autoregressive moving average) time series data, and the estimation accuracy of the calculation device 10 was tested using 3000 steps in addition to the 2000 steps used for training. The first 200 steps of the NARMA10 time series data were deleted to avoid the influence of the initial state of the intermediate nodes. In addition, the normalized mean-squared error (NMSE) was calculated as the error rate.

[0084] In Fig. 7, the spectral radius and error rate are plotted for each set value of the integer Z and the sparsity s. The results of a similar experiment using a general reservoir computing system are also shown. In the example of Figure 7, the spectral radius is adjusted by adjusting the value of the integer Z and the value of the sparsity s. Also, the error rate changes according to the spectral radius. In particular, the error rate is smaller when the spectral radius is 0.76 and 0.74, and in both cases, the error rate is smaller than when a general reservoir computing system is used.

[0085] As described above, the matrix product calculation unit 12 calculates the product of the connection representation matrix, which is a matrix in which all elements have a value of 1, 0, or -1, and the vector indicating the intermediate node value. The shift operation unit 13 performs a shift operation on the bit string in binary representation of each element of the vector obtained by the product of the connection representation matrix and the vector indicating the intermediate node value. The summation unit 14 sums the vector obtained by the shift operation and the weighted input value vector. The activation function application unit 15 applies a function defined as an activation function to each element of the vector obtained by the sum of the vector obtained by the shift operation and the weighted input value vector to calculate a vector indicating the update value of the intermediate node value due to the progress of the time step. The output value calculation unit 16 performs weighting on the update value of the intermediate node value to calculate an output value.

[0086] According to the arithmetic device 10, it is possible to perform calculations in a calculation model having a recursive network structure relatively easily. In particular, according to the arithmetic device 10, it is possible to perform calculations in a calculation model having a recursive network structure shown in formula (2). Z The calculation of "Tx(t-1)" can be performed using adders and shift registers, and does not require the use of multipliers.

[0087] Moreover, the activation function application unit 15 uses a third-order polynomial function consisting of a third-order term and a first-order term as the activation function. According to the activation function application unit 15, it is possible to calculate the activation function by calculating a relatively simple formula, that is, a cubic polynomial function using a cubic term and a linear term.

[0088] At least the matrix multiplication calculation unit 12 or the activation function application unit 15 is configured using an FPGA or an ASIC. As a result, the matrix multiplication calculation unit 12 or the activation function application unit 15 is implemented in hardware, and the effects of a relatively fast processing speed of the calculation device 10, a relatively low power consumption of the calculation device 10, a relatively small size of the calculation device 10, a relatively low production cost of the calculation device 10, or a combination of these are expected.

[0089] Further, the provisional setting unit 21 of the learning control device 20 sets the value of each element of the connection representation matrix of the arithmetic device 10 so that the element value becomes 0 with a determined probability. The learning control unit 22 causes the arithmetic device 10, for which the element values ​​of the connection representation matrix have been set, to perform learning and update the weight coefficients for calculating output values.

[0090] According to the learning control unit 22, it is possible to adjust the spectral radius by a simple method of setting a value of sparsity, which is the probability that the value of an element of the connection representation matrix becomes 0, and a value of integer Z, which is an exponent of a power of 2. By adjusting the spectral radius, it is expected that the accuracy of the processing of the arithmetic device 10 will be improved.

[0091] 8 is a diagram showing another example of the configuration of the arithmetic device according to the embodiment. In the configuration shown in FIG. 8, the arithmetic device 610 includes a matrix multiplication calculation unit 611, a shift calculation unit 612, a summation unit 613, an activation function application unit 614, and an output value calculation unit 615.

[0092] In this configuration, the matrix product calculation unit 611 calculates the product of a connection representation matrix, which is a matrix in which all elements have a value of 1, 0, or -1, and a vector indicating an intermediate node value. The shift operation unit 612 performs a shift operation on a bit string in binary representation of each element of a vector obtained by the product of the connection representation matrix and the vector indicating an intermediate node value. The summation unit 613 sums the vector obtained by the shift operation and a weighted input value vector. The activation function application unit 614 applies a function determined as an activation function to each element of a vector obtained by summing the vector obtained by the shift operation and the weighted input value vector, thereby calculating a vector indicating an update value of the intermediate node value due to the progress of the time step. The output value calculation unit 615 performs weighting on the update value of the intermediate node value to calculate an output value. The matrix multiplication calculation unit 611 corresponds to an example of a matrix multiplication calculation means. The shift operation unit 612 corresponds to an example of a shift operation means. The summation unit 613 corresponds to an example of a summation means. The activation function application unit 614 corresponds to an example of an activation function application means. The output value calculation unit 615 corresponds to an example of an output value calculation means.

[0093] The arithmetic device 610 can perform calculations in a computation model having a recursive network structure relatively easily. In particular, the arithmetic device 610 can perform calculations in a computation model having a recursive network structure by using a matrix multiplication calculation unit 611, a shift calculation unit 612, a summing unit 613, and an activation function application unit 614. The matrix multiplication calculation unit 611 can be configured using an adder, and does not need to use a multiplier. The shift calculation unit 612 can be configured using a shift register, and does not need to use a multiplier.

[0094] 9 is a diagram showing another example of the configuration of the learning control device according to the embodiment. In the configuration shown in FIG. In this configuration, the provisional setting unit 621 sets the value of each element of the connection representation matrix of the arithmetic device so that the value of the element becomes 0 with a determined probability. The connection representation matrix is ​​a matrix in which all elements have a value of 1, 0, or -1. The arithmetic device includes a matrix product calculation unit, a shift calculation unit, a summing unit, an activation function application unit, and an output value calculation unit. The matrix product calculation unit calculates the product of the connection representation matrix and a vector indicating an intermediate node value. The shift calculation unit performs a shift operation on a bit string in binary representation of the element for each element of the vector obtained by the product of the connection representation matrix and the vector indicating the intermediate node value. The summing unit sums the vector obtained by the shift operation and a weighted input value vector. The activation function application unit applies a function determined as an activation function to each element of the vector obtained by the sum of the vector obtained by the shift operation and the weighted input value vector to calculate a vector indicating an update value of the intermediate node value due to the progress of the time step. The output value calculation unit performs weighting on the update value of the intermediate node value to calculate an output value. The learning control unit 622 causes the arithmetic device in which the element values ​​of the connection representation matrix have been set to perform learning, and updates the weighting coefficients for calculating the output values. The provisional setting unit 621 corresponds to an example of a provisional setting means, and the learning control unit 622 corresponds to an example of a learning control means.

[0095] According to the learning control device 620, it is possible to adjust the spectral radius in the machine learning model of the arithmetic device by setting the probability that the value of an element of the connection representation matrix will be 0. It is expected that the accuracy of the processing of the arithmetic device will be improved by adjusting the spectral radius.

[0096] FIG. 10 is a flowchart showing an example of a processing procedure in the calculation method according to the embodiment. The calculation method shown in FIG. 10 includes calculating a product (step S611), performing a shift operation (step S612), performing a sum (step S613), applying an activation function (step S614), and calculating an output value (step S615). In calculating a product (step S611), a product of a connection representation matrix, which is a matrix in which all elements have a value of 1, 0, or −1, and a vector indicating an intermediate node value is calculated. In performing a shift operation (step S612), a shift operation is performed on a bit string in binary representation of each element of a vector obtained by multiplying the connection representation matrix and the vector indicating an intermediate node value. In performing a sum (step S613), a vector obtained by the shift operation is summed with a weighted input value vector. In applying the activation function (step S614), a function defined as the activation function is applied to each element of the vector obtained by summing the vector obtained by the shift operation and the weighted input value vector to calculate a vector indicating the update value of the intermediate node value due to the progress of the time step. In calculating the output value (step S615), the output value is calculated by weighting the update value of the intermediate node value.

[0097] According to the calculation method shown in Fig. 10, calculations in a calculation model having a recursive network structure can be performed relatively easily. In particular, according to the calculation method shown in Fig. 10, calculations in a calculation model having a recursive network structure can be performed by the processes in steps S611 to S614. The process in step S611 can be performed by addition, and multiplication is not required. The process in step S612 can be performed by shift operation, and multiplication is not required.

[0098] Fig. 11 is a flowchart showing an example of a process procedure in a learning control method according to the embodiment. The learning control method shown in Fig. 11 includes setting a matrix (step S621) and performing learning (step S622). In setting the matrix (step S621), the value of each element of the connection representation matrix of the arithmetic device is set so that the value of the element becomes 0 with a determined probability. The connection representation matrix is ​​a matrix in which all elements have a value of 1, 0, or -1. The arithmetic device includes a matrix product calculation unit, a shift calculation unit, a summing unit, an activation function application unit, and an output value calculation unit. The matrix product calculation unit calculates the product of the connection representation matrix and a vector indicating an intermediate node value. The shift calculation unit performs a shift operation on a bit string in binary representation of the element for each element of the vector obtained by the product of the connection representation matrix and the vector indicating the intermediate node value. The summing unit sums the vector obtained by the shift operation and a weighted input value vector. The activation function application unit applies a function determined as an activation function to each element of the vector obtained by the sum of the vector obtained by the shift operation and the weighted input value vector to calculate a vector indicating an update value of the intermediate node value due to the progress of the time step. The output value calculation unit performs weighting on the update value of the intermediate node value to calculate an output value. In the step of performing learning (step S622), the arithmetic device in which the element values ​​of the connection representation matrix have been set is made to perform learning, and the weight coefficients for calculating the output values ​​are updated.

[0099] According to the learning control method shown in Fig. 11, the spectral radius in the machine learning model of the arithmetic device can be adjusted by setting the probability that the element value of the connection representation matrix will be 0. It is expected that the accuracy of the processing of the arithmetic device will be improved by adjusting the spectral radius.

[0100] FIG. 12 is a schematic block diagram illustrating a computer configuration according to at least one embodiment. In the configuration shown in FIG. 12, a computer 700 includes a CPU (Central Processing Unit) 710, a main memory device 720, an auxiliary memory device 730, and an interface 740.

[0101] Any one or more of the arithmetic device 10, the learning control device 20, the arithmetic device 610, and the learning control device 620, or a part thereof, may be implemented in the computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures a storage area corresponding to each of the above-mentioned storage units in the main storage device 720 according to the program. Communication between each device and other devices is executed by the interface 740 having a communication function and performing communication according to the control of the CPU 710.

[0102] When the calculation device 10 is implemented in a computer 700, the operations of the input value weighting unit 11, the matrix multiplication calculation unit 12, the shift calculation unit 13, the summation unit 14, the activation function application unit 15, and the output value calculation unit 16 are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program.

[0103] Furthermore, the CPU 710 allocates a storage area for processing of the arithmetic device 10, such as the intermediate node value storage unit 17, in the main memory device 720 according to the program. Communication between the arithmetic device 10 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the arithmetic device 10 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.

[0104] When the learning control device 20 is implemented in a computer 700, the operations of the provisional setting unit 21, the learning control unit 22, the evaluation unit 23, and the calculation device setting unit 24 are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program.

[0105] Furthermore, the CPU 710 allocates a storage area for processing of the learning control device 20 in the main storage device 720 according to the program. Communication between the learning control device 20 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the learning control device 20 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.

[0106] When the arithmetic unit 610 is implemented in the computer 700, the operations of the matrix multiplication calculation unit 611, the shift calculation unit 612, the summation unit 613, the activation function application unit 614, and the output value calculation unit 615 are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program.

[0107] Furthermore, the CPU 710 allocates a storage area for processing by the arithmetic device 610 in the main memory device 720 in accordance with the program. Communication between the arithmetic device 610 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the arithmetic device 610 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.

[0108] When the learning control device 620 is implemented in the computer 700, the operations of the provisional setting unit 621 and the learning control unit 622 are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program.

[0109] Furthermore, the CPU 710 allocates a storage area for processing of the learning control device 620 in the main storage device 720 according to the program. Communication between the learning control device 620 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the learning control device 620 and a user is performed by the interface 740 having a display device and an input device, displaying various images under the control of the CPU 710, and accepting user operations.

[0110] Note that the programs for executing all or part of the processing performed by the arithmetic device 10, the learning control device 20, the arithmetic device 610, and the learning control device 620 may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be read into a computer system and executed to perform the processing of each part. Note that the term "computer system" here includes software such as an operating system (OS) and hardware such as peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The above-mentioned program may be for realizing part of the above-mentioned functions, or may be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.

[0111] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and also includes designs within the scope of the invention defined in the attached claims. [Industrial Applicability]

[0112] The present invention may be applied to a calculation device, a learning control device, a calculation method, a learning control method, and a recording medium. [Explanation of symbols]

[0113] 1. Computing System 10, 610 Arithmetic unit 11 Input value weighting section 12, 611 Matrix product calculation section 13,612 Shift operation unit 14,613 Total 15, 614 Activation function application part 16,615 Output value calculation section 17 Intermediate node value storage unit 20, 620 Learning control device 21, 621 Provisional Setting Section 22, 622 Learning control unit 23 Evaluation Department 24 Calculation device setting section< / t>

Claims

1. a matrix product calculation means for calculating a product of a connection representation matrix having a plurality of elements each having a value of 1, 0, or −1 and a vector indicating an intermediate node value in the machine learning model; a shift operation means for performing a shift operation on a bit string in binary representation of each of the elements of the vector obtained by the product; A summing means for summing a vector obtained by the shift operation and a vector including a plurality of input values ​​weighted by a weighting factor in the machine learning model; an activation function application means for applying an activation function to each of a plurality of elements of a vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values, to calculate a vector indicating an update value of the intermediate node value due to the progress of a time step; an output value calculation means for calculating a plurality of output values ​​by weighting the update values ​​of the intermediate node values; A computing device comprising:

2. The activation function application means uses a third-order polynomial function having a third-order term and a first-order term as the activation function. The computing device of claim 1 .

3. The matrix multiplication calculation means or the activation function application means is configured using an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The computing device according to claim 1 or 2.

4. a provisional setting unit for setting the value of each of the elements of the connection representation matrix of the arithmetic device according to claim 1 to 3 to be 0 with a predetermined probability; a learning control means for causing the arithmetic device, in which the element values ​​of the connection representation matrix have been set, to perform learning and update the weighting coefficients for calculating the plurality of output values; A learning control device comprising:

5. The computing device includes: Calculating the product of a connectivity representation matrix, which is a matrix having a plurality of elements each having a value of 1, 0, or −1, and a vector indicating intermediate node values ​​in the machine learning model; performing a shift operation on a bit string in a binary representation of each of the elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted by weighting factors in the machine learning model; calculating a vector indicating an update value of the intermediate node value due to the progress of a time step by applying an activation function to each of a plurality of elements of a vector obtained by summing the vector obtained by the shift operation and the vector having the plurality of weighted input values; calculating a plurality of output values ​​by weighting the update values ​​of the intermediate node values; An arithmetic method that implements.

6. A learning control device, a vector indicating an update value of the intermediate node value as a time step progresses by applying an activation function to each of the elements of the vector obtained by summing the vector obtained by the shift operation and the vector having the weighted input values, and a learning control method adapted to a calculation device that calculates a plurality of output values ​​by weighting the update value of the intermediate node value, the learning control method comprising: calculating a product of a connection representation matrix having a plurality of elements taking the values ​​of 1, 0, or -1 and a vector indicating an intermediate node value in a machine learning model; performing a shift operation on a bit string in binary representation of the element for each of the elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted using a weighting coefficient in the machine learning model; calculating a vector indicating an update value of the intermediate node value as a time step progresses by applying a weight to the update value of the intermediate node value; causing the arithmetic device, in which the element values ​​of the connection representation matrix have been set, to perform learning and update weight coefficients for calculating the output values; A learning control method that implements

7. A programmable device, Calculating a product of a connectivity representation matrix having a plurality of elements each having a value of 1, 0, or −1 and a vector indicating intermediate node values ​​in the machine learning model; performing a shift operation on a bit string in a binary representation of each of the elements of the vector obtained by the product; summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted by weighting factors in the machine learning model; calculating a vector indicating an update value of the intermediate node value due to the progress of a time step by applying an activation function to each of a plurality of elements of a vector obtained by summing the vector obtained by the shift operation and a vector including a plurality of weighted input values; calculating a plurality of output values ​​by weighting the update values ​​of the intermediate node values; A program that implements the above.

8. On the computer, a calculation device that calculates a product of a connection representation matrix having a plurality of elements each taking a value of 1, 0, or -1 and a vector indicating an intermediate node value in machine learning, performs a shift operation on a bit string in binary representation of each of the plurality of elements of the vector obtained by the product, sums the vector obtained by the shift operation and a weighted input value vector, applies an activation function to each of the plurality of elements of the vector obtained by summing the vector obtained by the shift operation and a vector including a plurality of input values ​​weighted by a weighting coefficient in the machine learning to calculate a vector indicating an update value of the intermediate node value as a time step progresses, and calculates a plurality of output values ​​by weighting the update value of the intermediate node value, and sets a value of each element of the plurality of elements of the connection representation matrix of the calculation device so that the value of the element becomes 0 with a predetermined probability; causing the arithmetic device, in which the element values ​​of the connection representation matrix have been set, to perform learning and update weight coefficients for calculating the output values; A program that implements the above.

Images