Learning model device, computing device production system, computing method, computing device production method, and program

Abstract

To configure a learning model provided with a node using binary data to be able to take relatively various output values without requiring increase in the number of layers of the node.SOLUTION: A learning model device includes a node which receives input of a binary vector and determines a binarized output value on the basis of an element other than elements of a coordinate value based on the binary vector, of elements of the coordinate value of a point of points included in a hypersurface in a real-number space having the number of dimensions one more than that of the binary vector, the point having the coordinate value including the elements of the coordinate value which is obtained when the binary vector is treated as a coordinate value in a subspace of the real-number space.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning model device, a computing device production system, a computing method, a computing device production method, and a program. [Background technology]

[0002] A binary neural network has been proposed that uses a node that receives input of multiple binary data, compares the sum of the multiple binary data with a threshold, and calculates an output value of the binary data (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-61496 Summary of the Invention [Problem to be solved by the invention]

[0004] A single-layer binary neural network is limited in the output values it can take for a given input value, as it cannot express exclusive OR. Increasing the number of layers in a binary neural network allows for a greater variety of output values, but the network structure becomes more complex. In addition, increasing the number of layers in a neural network can result in a decrease in learning accuracy and speed. It is desirable for a learning model with nodes that use binary data to be able to take on a relatively wide variety of output values without having to increase the number of layers of nodes.

[0005] One example of the objective of the present invention is to provide a learning model device, a computing device production system, a computing method, a computing device production method, and a program that enable a learning model having nodes that use binary data to take on a relatively wide variety of output values without the need to increase the number of layers of nodes. [Means for solving the problem]

[0006] According to a first aspect of the present invention, a learning model device receives an input of a binary vector and includes a node that determines a binarized output value based on elements of the coordinate values of a point included in a hypersurface in a real number space that has a number of dimensions one more than the number of dimensions of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, other than the elements of the coordinate values of the binary vector.

[0007] According to a second aspect of the present invention, an arithmetic device production system includes a learning model system, a learning control unit, and a setting unit, wherein the learning model system receives an input of a binary vector and includes a node that determines an output value based on elements of coordinate values of a point included in a hypersurface in a real number space having a dimensionality one more than the dimensionality of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, other than the elements of the coordinate values of the binary vector, and the learning control unit controls the learning of the learning model system, and the setting unit generates a lookup table showing the relationship between input and output at the node of the learning model system after learning, and sets the generated lookup table as a template for the arithmetic device.

[0008] According to a third aspect of the present invention, a computing method includes a computer receiving an input of a binary vector, and determining a binarized output value based on elements of coordinate values of a point included in a hypersurface in a real number space having a number of dimensions one more than the number of dimensions of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as coordinate values in a subspace of the real number space, other than elements of coordinate values of the binary vector.

[0009] According to a fourth aspect of the present invention, a method for producing an arithmetic device includes: training a learning model system having a node that receives an input of a binary vector and determines an output value based on elements of the coordinate values of a point that is included in a hypersurface in a real number space having a number of dimensions one more than the number of dimensions of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, other than elements of the coordinate values of the binary vector; generating a lookup table showing the relationship between input and output at the node of the learning model system after training; and setting the generated lookup table as a template for the arithmetic device.

[0010] According to a fifth aspect of the present invention, a program causes a computer to receive an input of a binary vector and determine a binarized output value based on elements of coordinate values of a point included in a hypersurface in a real number space having a number of dimensions one more than the number of dimensions of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as coordinate values in a subspace of the real number space, other than elements of coordinate values of the binary vector.

[0011] According to a sixth aspect of the present invention, a program includes the steps of: training a learning model system including a node that receives an input of a binary vector and determines an output value based on elements of coordinate values of a point included in a hypersurface in a real number space having a number of dimensions one more than the number of dimensions of the binary vector, the coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, other than elements of the coordinate values of the binary vector; and generating a lookup table indicating the relationship between input and output at the node of the learning model system after training; and setting the generated lookup table as a template for the arithmetic unit. [Effects of the Invention]

[0012] According to the above-mentioned learning model device, computing device production system, computing method, computing device production method and program, a learning model having nodes that use binary data can be made to be able to take on a relatively wide variety of output values without the need to increase the number of layers of nodes. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a configuration of a computing device production system according to an embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of input and output of data in a table type node according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of the configuration of a hypersurface node according to the embodiment. [Figure 4] FIG. 10 is a diagram showing a first example of a binary operation used to check the operation of a hypersurface processing unit using a B-Spline surface according to the embodiment. [Figure 5] FIG. 10 is a diagram showing a first example of a B-Spline surface obtained in an operation check. [Figure 6] FIG. 10 is a diagram showing a second example of binary operations used to check the operation of a hypersurface processing unit using a B-Spline surface according to the embodiment. [Figure 7] FIG. 10 is a diagram showing a second example of a B-Spline surface obtained in an operation check. [Figure 8] FIG. 10 is a diagram showing a third example of binary operations used to check the operation of a hypersurface processing unit using a B-Spline surface according to the embodiment. [Figure 9] FIG. 10 is a diagram showing a third example of a B-Spline surface obtained in an operation check. [Figure 10] FIG. 10 is a diagram showing a fourth example of a binary operation used to check the operation of the hypersurface processing unit 111 using a B-Spline surface according to the embodiment. [Figure 11] FIG. 10 is a diagram showing a fourth example of a B-Spline surface obtained in an operation check. [Figure 12] 10 is a flowchart showing an example of a processing procedure in which the computing device production system 1 according to the embodiment produces a computing device. [Figure 13]10A and 10B are diagrams illustrating an example of data input and output when a table-type node according to an embodiment outputs a binary vector. [Figure 14] 10A and 10B are diagrams illustrating an example of data input and output when a hypersurface node according to an embodiment outputs a binary vector. [Figure 15] FIG. 10 is a diagram showing an example of the configuration of a hypersurface type node in a learning model device in the case where one learning model device according to an embodiment includes one hypersurface type node. [Figure 16] FIG. 10 is a diagram showing a first example of the configuration of a hypersurface type node in a learning model device in the case where one learning model device according to an embodiment includes multiple hypersurface type nodes. [Figure 17] FIG. 10 is a diagram showing a second example of the configuration of a hypersurface type node in a learning model device in the case where one learning model device according to an embodiment includes multiple hypersurface type nodes. [Figure 18] FIG. 1 is a diagram illustrating a configuration of a convolutional neural network used in an experiment according to an embodiment. [Figure 19] FIG. 10 is a diagram showing the recognition rate obtained as an experimental result. [Figure 20] FIG. 1 is a diagram illustrating an example of a configuration of an FPGA. [Figure 21] FIG. 10 is a diagram illustrating an example of the configuration of a computing device when a lookup table is provided in common for a plurality of table-type nodes according to an embodiment. [Figure 22] FIG. 1 is a schematic block diagram illustrating an example configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. Fig. 1 is a diagram showing an example of the configuration of a computing device production system according to an embodiment. In the configuration shown in Fig. 1, the computing device production system 1 includes a learning model device 100, a learning control unit 300, and a setting unit 400. The learning model device 100 includes a hypersurface type node 110. 1 also shows a computing device 200. The computing device 200 includes a table-type node 210. The computing device 200 may be configured as a part of the computing device production system 1, or may be configured as a device external to the computing device production system 1.

[0015] The computing device production system 1 performs learning of a learning model having nodes that use binary data, and acquires a lookup table that indicates the relationship between input values and output values for each node of the learning model. The computing device production system 1 produces the computing device 200 by setting the acquired lookup table in the table-type node 210 of the template of the computing device 200. The template of the computing device 200 here does not have a lookup table set, but is otherwise similar to the computing device 200. In the arithmetic unit production system 1, the learning model device 100 has the function of a learning model, and the hypersurface type node 110 corresponds to a node that uses binary data.

[0016] The learning model here has parameters whose parameter values are adjustable, and outputs output data values corresponding to the input data values and the parameter values. Adjusting the parameter values of the learning model is referred to as learning the learning model. Learning the learning model of the learning model device 100 is also referred to as learning of the learning model device 100. The operation of binary data is also referred to as binary data operation. The operation here may refer to a lookup table to determine an output value.

[0017] The arithmetic device 200 performs binary data arithmetic using table-type nodes 210. The number of table-type nodes 210 included in the arithmetic device 200 is not limited to a specific number and can be one or more. In particular, the arithmetic device 200 may be provided with the same number of table-type nodes 210 as the number of hypersurface-type nodes 110 included in the learning model device 100, and the table-type nodes 210 may be connected in the same network structure as the network structure to which the hypersurface-type nodes 110 are connected in the learning model device 100. This allows the lookup table that the arithmetic device production system 1 acquires for each hypersurface-type node 110 during learning by the learning model device 100 to be set directly in the table-type node 210.

[0018] The arithmetic unit 200 can be used for various operations that handle binary data, such as bit operations, logical operations, or binary image processing, but the uses of the arithmetic unit 200 are not limited to these. The table type node 210 refers to a lookup table and outputs an output value that is associated with an input binary vector in the lookup table.

[0019] Fig. 2 is a diagram showing an example of data input and output in a table type node 210. Fig. 2 shows an example of a table type node 210 with two inputs and one output. However, the number of input data in the table type node 210 is not limited to a specific number, and can be any number greater than or equal to one. Furthermore, as will be described later, the number of output data in the table type node 210 may be multiple.

[0020] In the example of Fig. 2, the table-type node 210 receives two binary data items x0 and x1 as input, and outputs a binary data value that is associated with the input data value in a lookup table. The two binary data items x0 and x1 correspond to an example of a binary vector. Also, in the example of Fig. 2, the output data value that is associated with the input data value in the lookup table is the output data value shown in the same row as the input data value.

[0021] In the table-type node 210, any output data value can be set for each combination of input data values in a lookup table. In this respect, the table-type node 210 has high expressive power. For example, while a single node in a binary neural network cannot perform an exclusive OR operation, a single table-type node 210 can perform an exclusive OR operation. Furthermore, in the table-type node 210, the output data value for the input data value is determined by referring to a lookup table, so that even in the case of input / output that corresponds to a complex calculation, data can be output in a relatively short time and with relatively low power consumption.

[0022] On the other hand, the lookup table used by the table-type node 210 represents the correspondence between discrete input values and discrete output values, and this lookup table cannot be differentiated as a function. For this reason, a learning method that uses the differentiation of a function, such as the backpropagation method, cannot be applied to the arithmetic device 200. Therefore, the arithmetic unit production system 1 performs learning on the learning model device 100 and reflects the learning results in the arithmetic unit 200.

[0023] The learning model device 100 performs binary data operations using a hypersurface node 110. The learning model device 100 may include one hypersurface node 110, or multiple hypersurface nodes 110. When the learning model device 100 includes multiple hypersurface nodes 110, data may be passed between the hypersurface nodes 110. In this case, the data input / output relationship in the learning model device 100 can be represented in the form of a directed graph, as in the case of a neural network. The learning model device 100 corresponds to an example of a learning model system.

[0024] 3 is a diagram showing an example of the configuration of the hypersurface node 110. In the configuration shown in FIG. 3 shows an example of a hypersurface node 110 with two inputs and one output. However, the number of input data and the number of output data in the hypersurface node 110 are not limited to a specific number. For example, the hypersurface node 110 of the learning model device 100 and the table node 210 of the arithmetic device 200 may be in one-to-one correspondence, and the corresponding hypersurface node 110 and table node 210 may receive the same number of input data and output the same number of output data.

[0025] The hypersurface node 110 performs binary data operations similar to those performed by the learning model device 100, or a part of them. Specifically, the hypersurface node 110 receives an input vector as a binary vector. That is, the hypersurface node 110 receives an input of one or more binary data values. Then, the hypersurface node 110 performs binary data operations on the input vector in the hypersurface processing unit 111 and threshold calculation unit 112, and determines and outputs an output value.

[0026] The hypersurface processing unit 111 detects coordinate values of points included in a hypersurface in a real number space with one more dimension than the dimension of the input vector, the points having coordinate values including each element of the coordinate values when the input vector is treated as coordinate values in a subspace of the real number space.The hypersurface processing unit 111 then outputs the values of elements other than the elements of the coordinate values of the input vector among the detected coordinate value elements.

[0027] In the example of FIG. 3, two input data values x0 and x1 correspond to examples of input vectors. In this case, the number of dimensions of the input vector is two. The x0x1z coordinate space shown in FIG. 3 corresponds to an example of a three-dimensional real number space, which is one dimension more than the number of dimensions of the input vector. Here, z is the output value of the hypersurface processing unit 111 and serves as an input data value to the threshold calculation unit 112.

[0028] In this way, a coordinate space can be used that is a real number space with one more dimension than the number of dimensions of the input vector, which is a combination of an input coordinate axis, which is the coordinate axis for each element when each element value of the input vector is treated as a real value, and an output coordinate axis, which is the coordinate axis for the output value of the hypersurface processing unit 111. A real space with one more dimension than the number of dimensions of the input vector is also called an input / output real space.

[0029] In the example of FIG. 3, the surface shown in the x0x1z coordinate space corresponds to an example of a hypersurface in a three-dimensional real space (input / output real space) that is one dimension greater than the number of dimensions of the input vector. Additionally, the x0x1 coordinate plane, which is composed of the x0 coordinate and the x1 coordinate, is an example of a subspace of the x0x1z coordinate space. By treating the input data values x0 and x1, which are binary data values, as real data values, the input vector can be treated as a coordinate value on the x0x1 coordinate plane. For example, when x0=1 and x1=0, the input vector (1,0) can be treated as a coordinate value (1,0) on the x0x1 coordinate plane.

[0030] In this way, once the coordinate value (1,0) on the x0x1 coordinate plane is determined, among the points included in the surface shown in the x0x1z coordinate space, the point having coordinate values including the elements x0=1 and x1=0 of this coordinate value is uniquely identified. On the surface shown in Figure 3, when x0=1 and x1=0, the z coordinate value is z=0, and the coordinate value (1,0,0) is identified.

[0031] To this end, a hypersurface in the input / output real number space is used such that, among the points included in the hypersurface, points having coordinate values including each element of the coordinate values when the input vector is treated as coordinate values in a subspace of the input / output real number space are uniquely determined. An example of such a hypersurface is a B-Spline hypersurface whose control points are points with coordinate values obtained by combining the coordinate values of an input vector when each possible value of the input vector is treated as a coordinate value in a subspace of real number space with the output coordinate values set for each possible value of the input vector.

[0032] In the example of Figure 3, the input data values x0 and x1 can both be 0 or 1. Therefore, the possible values of the input vector are (x0, x1) = (0, 0), (0, 1), (1, 0), and (1, 1). In the example of Figure 3, the points with coordinate values (x0, x1, z) = (0, 0, 0), (0, 1, 0), (1, 0, 0), and (1, 1, 1), which are combinations of the possible values of the input vector and the output value, are set as control points and are indicated by white circles (○). In the example of Figure 3, a B-Spline surface based on the values of these control points is used as a surface in the x0x1z coordinate space.

[0033] The output coordinate value of each control point can be treated as a learning parameter value. By changing the output coordinate value as a learning parameter value, the output value of the hypersurface processing unit 111 corresponding to the value that the input vector can take is changed. Furthermore, a B-Spline hypersurface can be expressed as a differentiable function that takes an input vector as an argument and outputs an output coordinate value. This makes it possible to apply a learning method that uses function differentiation, such as backpropagation, to learning the learning parameter values included in the coordinate values of the control points. However, the hypersurface in the input / output real number space is not limited to a B-Spline hypersurface, but can be any of various hypersurfaces in which the relationship between the input vector and the output data value in the hypersurface processing unit 111 changes depending on the learning parameter value and which are expressed by differentiable functions.

[0034] The hypersurface processing unit 111 outputs the values of elements other than the elements determined by the input vector among the elements of the coordinate values identified as the coordinate values of the points included in the hypersurface. In the above example, the hypersurface processing unit 111 outputs the z coordinate value "0" among the identified coordinate values (x0, x1, z) = (1, 0, 0). The output values of the hypersurface processing unit 111 are real values, and can take values other than those corresponding to the binary values in the binary data.

[0035] During forward propagation in the learning model device 100, the threshold calculation unit 112 binarizes the output value of the hypersurface processing unit 111 using a step function. For example, the threshold calculation unit 112 may compare the output value of the hypersurface processing unit 111 with a threshold and output one of the two values in the binary data depending on the comparison result. In this case, the threshold may be a fixed value or may be variable as a learning parameter value. An example of forward propagation in the learning model device 100 is when the learning model device 100 executes binary data operations.

[0036] On the other hand, the threshold calculation unit 112 approximates the step function with a differentiable function during backpropagation in the learning model device 100. Examples of differentiable functions that approximate the step function include a sigmoid function and a hyperbolic tangent function, but the functions used by the threshold calculation unit 112 are not limited to these.

[0037] An example of backpropagation in the learning model device 100 is calculation of a correction amount for a learning parameter in the backpropagation method. When the threshold calculation unit 112 approximates a step function with a differentiable function, a learning method using the differentiation of a function, such as the backpropagation method, can be applied to learning in the learning model device 100.

[0038] For at least a part of the learning period, the threshold calculation unit 112 may not binarize the data, and the hypersurface node 110 may output real-valued data. For example, in the early stage of learning from the start of learning until a predetermined condition is met, the threshold calculation unit 112 may not binarize the data, and after the early stage when the predetermined condition is met ends, the threshold calculation unit 112 may binarize the data. This is expected to enable learning to proceed relatively quickly, and also to reduce the likelihood that the learning results will fall into a local optimum.

[0039] In the example of FIG. 3, the hypersurface in the input / output real number space is expressed as in equation (1).

[0040]

number

[0041] Here, x0 and x1 are real variables that take on real values when the input values to the hypersurface node 110, which are binary data values, are treated as real values. z is a real variable that takes on the output value of the hypersurface processing unit 111. The output value of the hypersurface processing unit 111 is the z coordinate value of the x0x1z coordinate values of a point that has x0x1z coordinate values that include the input values to the hypersurface node 110 when these coordinate values are treated as x0 and x1 coordinate values, among the points included on the surface in the x0x1z coordinate space.

[0042] w0, w1, w2, and w3 are real variables used as learning parameters. As described above, the values of these learning parameters may be used as z coordinate values at the control points of the B-Spline surface. f is a differentiable function. The calculation performed by the entire hypersurface node 110, which is a combination of the hypersurface node 110 and the threshold calculation unit 112, is expressed as in equation (2).

[0043]

number

[0044] Here, y is a real variable that takes on a real value when the output value of the hypersurface type node 110 based on binary data values is treated as a real value. In the case of back propagation in the learning model device 100, f R is a differentiable function. The correct value of the output value of the hypersurface node 110 is y * and the output value y of the hypersurface node 110 and its correct value y * The error E between is defined as in equation (3).

[0045]

number

[0046] If the learning coefficient is α, the correction amount Δw0 of the learning parameter w0 can be calculated as Δw0 = -α(∂E / ∂w0). By rewriting the notation of E using equations (3) and (2), the correction amount Δw0 of the learning parameter w0 can be calculated using equation (4).

[0047]

number

[0048] The same applies to the learning parameters w1, w2, and w3. In this way, the hypersurface node 110 allows for the use of a learning method that uses the differentiation of a function.

[0049] The learning control unit 300 controls the learning of the learning model device 100. For example, the learning control unit 300 acquires training data from another device such as a database, and causes the learning model device 100 to perform learning using the acquired training data. Through the learning, the learning parameter values of the learning model device 100 are adjusted.

[0050] The setting unit 400 generates a lookup table indicating the relationship between input values and output values in the hypersurface type node 110 of the learning model device 100 after learning, and sets the generated lookup table in the table type node 210 of the template of the calculation device 200. For example, for the learning model device 100 in the example of Fig. 3, the setting unit 400 observes the output value y for each of all possible values (x0, x1) = (0, 0), (0, 1), (1, 0), and (1, 1) of the input vector, and generates the lookup table in the example of Fig. 2. Then, the setting unit 400 sets the generated lookup table in the table-type node 210 of the template of the arithmetic device 200 as in the example of Fig. 2.

[0051] The learning control unit 300, the setting unit 400, and the learning model device 100 may be configured as separate devices. In this case, each of these devices may be configured using a computer such as a personal computer. Alternatively, one or more of the learning control unit 300, the setting unit 400, and the learning model device 100 may be configured using hardware dedicated to that device, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0052] Alternatively, any two or more of the learning control unit 300, the setting unit 400, and the learning model device 100 may be integrally configured. For example, the learning control unit 300, the setting unit 400, and the learning model device 100 may be incorporated into the same device. In this case, too, the device may be configured using a computer, or may be configured using hardware dedicated to the device.

[0053] The arithmetic device 200 may also be configured using a computer, or may be configured using hardware dedicated to the arithmetic device 200. As will be described later, the arithmetic device 200 is considered to have a configuration particularly suitable for implementation in an FPGA, and the arithmetic device 200 may be configured using an FPGA.

[0054] Furthermore, when additional learning is performed during operation of the device, the learning model device 100 may be used for operation. In this case, the arithmetic unit production system 1 does not need to include the setting unit 400 and the arithmetic unit 200.

[0055] When the operation of the hypersurface processing unit 111 was checked using a B-Spline surface for learning binary operations, good results were obtained. FIG. 4 is a diagram showing a first example of binary operations used to check the operation of the hypersurface processing unit 111 using a B-Spline surface. Operation was confirmed using input and output data of the logical operation "AND" (logical product) as shown in Figure 4, and the surface shown in Figure 5 was obtained.

[0056] FIG. 5 is a diagram showing a first example of a B-Spline surface obtained in the operation check. For the surface shown in Figure 5, the value of z is approximately 0 when x0 = 0 and x1 = 0. Also, the value of z is approximately 0 when x0 = 0 and x1 = 1. Also, the value of z is approximately 0 when x0 = 1 and x1 = 0. Also, the value of z is approximately 1 when x0 = 1 and x1 = 1. In this way, a curved surface showing inputs and outputs similar to the "AND" inputs and outputs of the logic circuit shown in Figure 4 was obtained.

[0057] FIG. 6 is a diagram showing a second example of binary operations used to check the operation of the hypersurface processing unit 111 using a B-Spline surface. Operation was confirmed using input and output data of the logical operation "OR" (logical sum) as shown in Figure 6, and the surface shown in Figure 7 was obtained.

[0058] FIG. 7 is a diagram showing a second example of a B-Spline surface obtained in the operation check. For the surface shown in Figure 7, the value of z is approximately 0 when x0 = 0 and x1 = 0. Also, the value of z is approximately 1 when x0 = 0 and x1 = 1. Also, the value of z is approximately 1 when x0 = 1 and x1 = 0. Also, the value of z is approximately 1 when x0 = 1 and x1 = 1. In this way, a curved surface showing inputs and outputs similar to the "OR" inputs and outputs of the logic circuit shown in Figure 6 was obtained.

[0059] FIG. 8 is a diagram showing a third example of binary operations used to check the operation of the hypersurface processing unit 111 using a B-Spline surface. Operation was confirmed using input and output data of the logical operation "EXOR" (exclusive OR) as shown in Figure 8, and the surface shown in Figure 9 was obtained.

[0060] FIG. 9 is a diagram showing a third example of a B-Spline surface obtained in the operation check. For the surface shown in Figure 9, the z value is approximately 0 when x0 = 0 and x1 = 0. Also, the z value is approximately 1 when x0 = 0 and x1 = 1. Also, the z value is approximately 1 when x0 = 1 and x1 = 0. Also, the z value is approximately 0 when x0 = 1 and x1 = 1. In this way, a curved surface showing inputs and outputs similar to the inputs and outputs of the "EXOR" logic circuit shown in Figure 8 was obtained.

[0061] FIG. 10 is a diagram showing a fourth example of binary operations used to check the operation of the hypersurface processing unit 111 using a B-Spline surface. Operation was confirmed using input and output data of the logical operation "NOT x0" (negation of input signal x0) as shown in FIG. 10, and a curved surface as shown in FIG. 11 was obtained.

[0062] FIG. 11 is a diagram showing a fourth example of a B-Spline surface obtained in the operation check. For the surface shown in Figure 11, the value of z is approximately 1 when x0 = 0 and x1 = 0. Also, the value of z is approximately 1 when x0 = 0 and x1 = 1. Also, the value of z is approximately 0 when x0 = 1 and x1 = 0. Also, the value of z is approximately 0 when x0 = 1 and x1 = 1. In this way, a curved surface showing inputs and outputs similar to the inputs and outputs of "NOT x0" in the logic circuit shown in FIG. 10 was obtained.

[0063] FIG. 12 is a flowchart showing an example of a processing procedure by which the computing device production system 1 produces the computing device 200. 12, the learning model device 100 performs learning under the control of the learning control unit 300 and adjusts the learning parameter values (step S11). The learning model device 100 performs learning of the learning model device 100 including the hypersurface type node 110 under the control of the learning control unit 300. In particular, when a plurality of hypersurface type nodes 110 constitute a network, learning of the entire network is performed, as in the case of neural network learning. As described above, the learning parameters may be elements of the coordinates of the control points of the B-Spline hypersurface.

[0064] Next, the setting unit 400 generates a lookup table indicating the relationship between input values and output values in the hypersurface type node 110 after learning is completed, and sets the generated lookup table in the table type node 210 of the template of the calculation device 200 (step S12). When the learning model device 100 includes a plurality of hypersurface type nodes 110, the calculation device 200 is configured so that the hypersurface type nodes 110 and the table type nodes 210 are in one-to-one correspondence with each other. Specifically, the number of table type nodes 210 included in the calculation device 200 is set to be the same as the number of hypersurface type nodes 110 included in the learning model device 100, and the table type nodes 210 are configured to form a network having the same structure as the network formed by the hypersurface type nodes 110. The setting unit 400 generates a lookup table for each hypersurface type node 110 indicating the relationship between input values and output values in that hypersurface type node 110, and sets the generated lookup table in the table type node 210 that is in one-to-one correspondence with that hypersurface type node 110. After step S12, the computing device production system 1 ends the processing of FIG.

[0065] The hypersurface node 110 and the table node 210 may each output a binary vector. That is, the hypersurface node 110 and the table node 210 may each output a plurality of binary data.

[0066] FIG. 13 is a diagram showing an example of data input and output when the table type node 210 outputs a binary vector. In the example of FIG. 13, the table-type node 210b stores an N-dimensional (N is a positive integer) binary vector (x0, x1, . . . , x N-1 ) and outputs an M-dimensional (M is a positive integer) binary vector (y0, y1, . . . , y M-1) The table type node 210b is an example of the table type node 210.

[0067] In this case, the table-type node 210b receives the binary vector (x0, x1, . . . , x N-1 ) can take, the binary vector (y0, y1, , y M-1 ) is a binary vector (x0, x1, . . . , x N-1 ) can take on the value 2 N The lookup table is N Shows the data for a line.

[0068] FIG. 14 is a diagram showing an example of data input / output when the hypersurface type node 110 outputs a binary vector. In the example of FIG. 14, the hypersurface node 110b uses the same binary vector (x0, x1, . . . , x) as the table node 210b. N-1 ), and the same binary vector (y0, y1, . . . , y M-1 ) is output. In this case, the hypersurface is expressed as, for example, equation (5).

[0069]

number

[0070] L is a positive integer that represents the number of learning parameters when a B-Spline hypersurface is used as the hypersurface. A control point of the B-Spline hypersurface is provided for each possible value of the input vector, and one learning parameter is provided for each control point. Therefore, the value of L is expressed as in equation (6).

[0071]

number

[0072] In this case, a differentiable function f can be obtained, and a learning method using the differentiation of the function can be applied. For example, if the function f in Equation (5) is N-1 , and one output variable y i (where i is an integer in the range of 0≦i≦M−1) and may be represented by M hypersurfaces in an N+1-dimensional coordinate space having coordinate axes for each variable.

[0073] Alternatively, the learning model device 100 may be provided with one-output hypersurface nodes 110, the number of which is equal to the number of binary data output by the table type node 210. The learning model device 100 may be provided with M N-input, one-output hypersurface nodes 110, corresponding to the N-input, M-output table type node 210b exemplified in Fig. 13. After completion of learning, the setting unit 400 may compile the relationships between input values and output values in these M hypersurface type nodes 110 into an N-input, M-output lookup table as exemplified in Fig. 13, and set the obtained lookup table in the table type node 210b.

[0074] There are several possible variations in the configuration of the hypersurface node 110 in the learning model device 100. Fig. 15 is a diagram showing an example of the configuration of the hypersurface node 110 in the learning model device 100 when one learning model device 100 has one hypersurface node 110. In the configuration shown in Fig. 15, the learning model device 100c has one hypersurface node 110c. The learning model device 100c is an example of the learning model device 100. The hypersurface node 110c is an example of the hypersurface node 110.

[0075] The learning model device 100c receives four binary data x0, x1, x2, and x3 as input, and outputs three binary data y0, y1, and y2. In response to this, the hypersurface node 110c receives four binary data x0, x1, x2, and x3 as input, and outputs three binary data y0, y1, and y2.

[0076] In this way, the learning model device 100 may be configured to include one hypersurface node 110. The hypersurface node 110 may then receive input data for the learning model device 100 and output output data for the learning model device 100. The arithmetic device 200 corresponding to this learning model device 100 can also have the same configuration as this learning model device 100. Specifically, the arithmetic device 200 may be configured to include one table-type node 210. The table-type node 210 may then receive input data to the arithmetic device 200 and output output data from the arithmetic device 200.

[0077] Alternatively, as described above, the learning model device 100 may be provided with the same number of hypersurface nodes 110 as the number of output data. After the learning is completed, the setting unit 400 may collect the relationships between input values and output values in the multiple hypersurface nodes 110 into one lookup table, and set the obtained lookup table in one table type node 210.

[0078] For example, in the example of FIG. 15, the learning model device 100 may be provided with three hypersurface nodes 110 instead of one hypersurface node 110c. In this case, four binary data x0, x1, x2, and x3 are input to each of the three hypersurface nodes 110. As for output data, each hypersurface node 110 outputs a different binary data. Specifically, the first hypersurface node 110 outputs binary data y0, the second hypersurface node 110 outputs binary data y1, and the third hypersurface node 110 outputs binary data y3. After the learning is completed, the setting unit 400 may compile the relationships between the input values and output values of the three hypersurface type nodes 110 into one lookup table, and set the obtained lookup table in one table type node 210.

[0079] Fig. 16 is a diagram showing a first example of the configuration of the hypersurface type nodes 110 in the learning model device 100 when one learning model device 100 is equipped with a plurality of hypersurface type nodes 110. In the configuration shown in Fig. 16, the learning model device 100d is equipped with a hypersurface type node 110d-1, a hypersurface type node 110d-2, a hypersurface type node 110d-3, and a hypersurface type node 110d-4. The learning model device 100d is an example of the learning model device 100. The hypersurface type node 110d-1, the hypersurface type node 110d-2, the hypersurface type node 110d-3, and the hypersurface type node 110d-4 are examples of the hypersurface type node 110, respectively.

[0080] 16 shows an example in which the structure of the network formed by the hypersurface type nodes 110 is fixed. For example, a person such as a designer of the learning model device 100 determines the structure of the network in advance and implements it in the learning model device 100. In this way, one learning model device 100 may be equipped with a plurality of hypersurface type nodes 110, and these plurality of hypersurface type nodes 110 may constitute a network. In this case, the structure of the network may be various, as in the case of a neural network.

[0081] The arithmetic device 200 corresponding to this learning model device 100 can also be configured in the same manner as this learning model device 100. Specifically, the arithmetic device 200 is configured to have the same number of table type nodes 210 as the number of hypersurface type nodes 110 included in the learning model device 100. The table type nodes 210 are then configured to form a network with the same structure as the network formed by the hypersurface type nodes 110.

[0082] Fig. 17 is a diagram showing a second example of the configuration of the hypersurface type nodes 110 in the learning model device 100 when one learning model device 100 has a plurality of hypersurface type nodes 110. In the configuration shown in Fig. 17, the learning model device 100e has hypersurface type nodes 110e-1 to 110e-12. The learning model device 100e corresponds to an example of the learning model device 100. Each of the hypersurface type node 110e-1 to the hypersurface type node 110e-12 corresponds to an example of the hypersurface type node 110.

[0083] 17 shows an example in which the structure of the network formed by the hypersurface type nodes 110 is variable during learning. In this case, the network structure may be determined by machine learning. For example, a genetic programming technique may be used as a learning method for the network structure.

[0084] Furthermore, consider a case where the learning control unit 300 searches for a network structure using Cartesian genetic programming (CGP), a type of genetic programming. In this case, the learning control unit 300 provisionally sets a certain network structure as the structure of the network made up of the hypersurface type nodes 110. The provisionally set network structure can be said to be a candidate network structure.

[0085] Then, the learning control unit 300 calculates an evaluation value of the provisionally set network structure. Specifically, the learning control unit 300 learns the network of the provisionally set hypersurface type nodes 110, calculates an evaluation score (e.g., recognition rate) of the learning result, and sets it as the evaluation value of the provisionally set network structure.

[0086] The learning control unit 300 repeatedly changes the provisionally set network structure and calculates the evaluation value of the network structure until a predetermined condition is met as an end condition for learning the network structure. When changing the network structure, the learning control unit 300 can determine the degree to which to change the network structure based on the evaluation value. For example, if the evaluation value indicates a good evaluation equal to or greater than a predetermined evaluation threshold, the learning control unit 300 may change the network structure to a relatively small degree, such as by changing only one edge in the network structure. On the other hand, if the evaluation value indicates a low evaluation equal to or less than the predetermined evaluation threshold, the learning control unit 300 may change the network structure to a relatively large degree, such as by changing ten edges in the network structure. However, the method for learning the structure of the network formed by the hypersurface type nodes 110 is not limited to a specific method.

[0087] We conducted character recognition experiments by applying a learning method using B-Spline hypersurfaces to a convolutional neural network, and obtained good results. Fig. 18 shows the configuration of the convolutional neural network used in the experiment. As shown in Fig. 18, the experiment used a convolutional neural network that includes a first convolutional layer, a pooling layer, and a second convolutional layer and selects classes using a softmax function.

[0088] As training data, we used a dataset of images of handwritten digits shown in MNIST (Modified National Institute of Standards and Technology), which were reduced to 8 pixels x 8 pixels and binarized. In the first convolutional layer, a convolution operation is performed on each 3 pixel x 3 pixel partial image of the 8 pixel x 8 pixel input image data, and 6 pixel x 6 pixel image data is generated with no padding (zero padding). The first convolutional layer receives one image data input and outputs ten image data. These ten image data are treated as features for each of the ten classes from "0" to "9." In this way, the first convolutional layer receives 9-dimensional data input generated by convolution using 3 pixel x 3 pixel image patches, and outputs 10-dimensional data (10-channel data).

[0089] In the experiment, we compared the results of using a conventional convolutional neural network, using a B-Spline hypersurface and converting the node output data into real data without binarizing it, and using a B-Spline hypersurface and binarizing the node output data. In the conventional convolutional neural network, ReLU (Rectified Linear Unit) was used as the activation function, and the output data of the nodes was real data.

[0090] On the other hand, when a B-Spline hypersurface is used, it can be said that the B-Spline hypersurface (B-Spline function) includes the meaning of an activation function. In a general neural network, linear processing and processing using an activation function are performed for each node, but with the hypersurface node 110, processing equivalent to a combination of these two processing can be performed by processing using a B-Spline hypersurface.

[0091] When binarizing the output data, the values obtained using a B-Spline hypersurface are binarized in the same manner as in the case of the threshold calculation unit 112 in the hypersurface type node 110. In addition, in the convolutional neural network where the B-Spline hypersurface is used and the output data of the nodes is binarized, the binarized data is output in the form of real number data. It is believed that the same recognition rate can be obtained even if this data is output as 1-bit data.

[0092] In the pooling layer, image data of 6 pixels x 6 pixels is reduced to image data of 3 pixels x 3 pixels using max pooling. The pooling layer receives 10 pieces of image data as input and outputs 10 pieces of image data.

[0093] In the second convolutional layer, depthwise convolution is performed on 3 pixel x 3 pixel image data in 3 pixel x 3 pixel units, and 1 pixel x 1 pixel image data is output without padding. In the second convolutional layer, the activation function used in the conventional convolutional neural network and the number of dimensions in the node using the B-Spline hypersurface were the same as in the first convolutional layer.

[0094] The second convolutional layer receives 10 pieces of image data as input and outputs 10 pieces of image data. Therefore, the second convolutional layer outputs a scalar score for each of the 10 classes from "0" to "9." Class estimation is performed by using the softmax function to select the class with the highest score from the 10 classes from "0" to "9".

[0095] FIG. 19 is a diagram showing the recognition rate obtained as a result of the experiment. In the experimental results shown in Figure 19, a higher recognition rate was obtained than when using a conventional convolutional neural network in both cases: when a B-Spline hypersurface was used and the node output data was converted to real data without being binarized, and when a B-Spline hypersurface was used and the node output data was binarized.

[0096] The setting unit 400 may implement the arithmetic device 200 in an FPGA. Fig. 20 is a diagram showing an example of the configuration of an FPGA. In the configuration shown in Fig. 20, the FPGA includes a configurable logic block (CLB) and a switching block. The configurable logic block includes a basic logic element (BLE). The BLE includes a lookup table (LUT), a flip flop (FF), and a multiplexer (MUX).

[0097] In a basic logic element, a lookup table receives input data (I / P's) and outputs data of a value corresponding to the value of the input data to a flip-flop and a multiplexer. The flip-flop stores the data value from the look-up table when a clock signal (CLK) is input. When a reset signal (RST) is input, the flip-flop resets the stored data. The flip-flop outputs the stored data to the multiplexer.

[0098] The multiplexer receives data from the lookup table and data from the flip-flop as inputs and outputs one output data (O / P). For example, the multiplexer receives a control signal as input and outputs either data from the lookup table or data from the flip-flop. The switching block switches the connection (On / Off) of the data lines between the configurable logic blocks.

[0099] When the arithmetic device 200 is implemented in an FPGA, the setting unit 400 may set the lookup table generated based on the relationship between the input value and the output value in the hypersurface node 110 as the lookup table of the basic logic element. In this case, by making the multiplexer output data from the lookup table, it is possible to make the basic logic element perform the same operation as the operation performed by the hypersurface node 110.

[0100] As in the example of Figure 17, when the learning model device 100 determines the structure of the network formed by the hypersurface type nodes 110 through learning, the setting unit 400 may adjust the settings of the switching block so that a network with a structure similar to the network formed by the hypersurface type nodes 110 is implemented in the FPGA.

[0101] A lookup table may be shared by multiple table type nodes 210 . Fig. 21 is a diagram showing an example of the configuration of a calculation device 200 when a lookup table is provided to be shared by multiple table type nodes 210. In the configuration shown in Fig. 21, the calculation device 200f includes table type nodes 210f-1 to 210f-4, table storage units 220f-1 and 220f-2.

[0102] The table type nodes 210f-1 to 210f-4 are also collectively referred to as table type nodes 210f. The table storage unit 220f-1 and the table storage unit 220f-2 are also collectively referred to as table storage unit 220f. The arithmetic device 200f corresponds to an example of the arithmetic device 200. The table type node 210f corresponds to a modified example of the table type node 210. The combination of the table type node 210f and the table storage unit 220f corresponds to an example of the table type node 210. The table storage unit 220f stores a lookup table.

[0103] Each of the table type nodes 210f does not have its own lookup table, but refers to a lookup table stored in the table storage unit 220. In other respects, the table type node 210f is similar to the table type node 210. The lookup table stored in the table storage unit 220f-1 is a lookup table that is commonly referenced by the table type node 210f-1 and the table type node 210f-2. The lookup table stored in the table storage unit 220f-2 is a lookup table that is commonly referenced by the table type node 210f-3 and the table type node 210f-4.

[0104] When the learning of the learning model device 100 is completed and the setting unit 400 generates a lookup table for each hypersurface node 110, the "similar lookup tables" may be aggregated into a single lookup table, and the aggregated single lookup table may be stored in the table storage unit 220f. Then, the setting unit 400 may set the reference destination of the lookup table of each hypersurface node 110 so that the hypersurface nodes 110 that were supposed to refer to the lookup tables before the aggregation will commonly refer to the aggregated lookup table.

[0105] As a criterion for determining whether a lookup table is similar, for example, a condition may be used in which the output values of the "common rows" in the lookup table are the same for a predetermined threshold percentage or more (for example, 90% or more). If the correspondence between the input data to the table-type node 210 and the input data shown in the lookup table can be set for each table-type node 210, there are multiple ways to set the "common rows," which increases the possibility of sharing the lookup table.

[0106] 21, the input data to the table type node 210f-1 is (x1, x2), and the input data to the table type node 210f-2 is (x0, x1). Also, it is assumed that the input data in the lookup table stored in the table storage unit 220f-1 is (I0, I1). It is also assumed that the output values in the lookup table are set so that the table type node 210f-1 refers to the lookup table by associating x1 with I0 and x2 with I1.

[0107] The table type node 210f-2 may associate x0 with I0 and x1 with I1, or may associate x1 with I0 and x0 with I1. If the determination condition is met for at least one of these two associations, the lookup table referenced by the table type node 210f-1 and the lookup table referenced by the table type node 210f-2 can be shared.

[0108] If the number of input data to the table-type node 210f is smaller than the number of input data shown in the lookup table, the number of cases in which the "common row" setting method is used increases, further increasing the possibility that the lookup table can be shared. For example, if the output data of the table type node 210f-1 is expressed as v1, the input data of the table type node 210f-3 is two pieces of data (x0, v1), and the input data of the table type node 210f-4 is three pieces of data (y0, x2, x3). It should be noted that here, the "number of input data" does not refer to the number of input vectors, but to the number of individual binary data (hence, the number of elements of the input vector).

[0109] Also, it is assumed that the input data in the lookup table stored in the table storage unit 220f-2 are three, (I2, I3, I4). It is also assumed that the output values in the lookup table are set so that the table-type node 210f-4 associates y0 with I2, x2 with I3, and x3 with I4 and refers to the lookup table.

[0110] There are six ways that the table type node 210f-3 associates the input data (x0, v1) with the input data (I2, I3, I4) in the lookup table: (x0, v1) = (I2, I3), (I2, I4), (I3, I2), (I3, I4), (I4, I2), (I4, I3). If the determination condition is met with at least one of these six associations, the lookup table referenced by the table type node 210f-3 and the lookup table referenced by the table type node 210f-4 can be shared.

[0111] Furthermore, when a lookup table referenced by multiple table-type nodes 210f is shared, the shared lookup table may have multiple columns of output values, as in the example of Figure 13, and the output values for each table-type node 210f may be entered. For example, when three single-output table-type nodes 210f are shared, three columns of output values may be provided in the lookup table, and the output values of each table-type node 210f may be written therein. In this case, it is possible to share the description of input data used as a search key when the table type node 210f refers to the lookup table, and in this respect, it is possible to reduce the memory capacity required to store the lookup table.

[0112] As described above, the hypersurface type node 110 receives an input of a binary vector, and determines a binarized output value based on elements of the coordinate values of a point that has coordinate values including each element of the coordinate values of the input binary vector when the input binary vector is treated as a coordinate value in a subspace of the input / output real space, among points included in a hypersurface in the input / output real space, which is a real space with one more dimension than the number of dimensions of the input binary vector, other than the elements of the coordinate values of the input binary vector.

[0113] The hypersurface node 110 can express the relationship between input and output values using a hypersurface, thereby achieving expressive power equivalent to that of a truth table. In this respect, the learning model device 100 can enable a learning model equipped with a hypersurface node 110 that uses binary data to take on a relatively wide variety of output values without the need to increase the number of layers of the hypersurface node 110.

[0114] Furthermore, the hypersurface provided in the hypersurface type node 110 is a B-Spline hypersurface whose control points are points of coordinate values in the input / output real number space, which are determined by combining the coordinate values when each of the values that the binary vector input to the hypersurface type node 110 can take is treated as a coordinate value in a subspace of the input / output real number space with the learning parameter value.

[0115] According to the hypersurface node 110, a differentiable function can be obtained by using a B-Spline hypersurface in this way, and a learning method that uses the differentiation of a function, such as the backpropagation method, can be applied. Furthermore, by using the input data values to the hypersurface node 110 as control points of the B-Spline hypersurface, the output coordinate values of the control points indicate the output data values for the input data values. In this respect, the hypersurface node 110 makes it relatively easy to grasp the relationship between the input data values and the output data values.

[0116] Furthermore, the hypersurface processing unit 111 acquires the values of elements of the coordinate values of points included in the hypersurface that include each element of the coordinate values when the binary vector input to the hypersurface type node 110 is treated as a coordinate value in a subspace of the input / output real number space, other than the elements of the coordinate values of the binary vector input to the hypersurface type node 110. During forward propagation in the learning model device 100, the threshold calculation unit 112 binarizes the value acquired by the hypersurface processing unit 111 using a step function, and during backpropagation in the hypersurface processing unit 111, it approximates the step function with a differentiable function.

[0117] In this way, the threshold calculation unit 112 switches the threshold function so that the hypersurface node 110 outputs binary data during forward propagation, and a learning method that uses the differentiation of a function, such as backpropagation, can be applied.

[0118] Furthermore, the learning model device 100 receives an input of a binary vector, and determines a binarized output value based on elements of the coordinate values of a point that has coordinate values including each element of the coordinate values of the input binary vector when the input binary vector is treated as a coordinate value in a subspace of the input / output real space, among points included in a hypersurface in the input / output real space, which is a real space with one more dimension than the number of dimensions of the input binary vector, other than the elements of the coordinate values of the input binary vector. The learning control unit 300 controls the learning of the learning model device 100 . The setting unit 400 generates a lookup table indicating the relationship between input values and output values at the nodes of the learning model device 100 after learning, and sets the generated lookup table in the template of the calculation device 200.

[0119] According to the arithmetic unit production system 1, it is possible to obtain expressive power equivalent to that of a truth table by using a hypersurface to represent the relationship between input values and output values in the hypersurface processing unit 111. In this respect, according to the arithmetic unit production system 1, it is possible to make a learning model including hypersurface type nodes 110 that use binary data capable of taking a relatively wide variety of output values without the need to increase the number of layers of the hypersurface type nodes 110. Furthermore, according to the computing device production system 1, the lookup table obtained by learning is set as a template for the computing device 200 and the computing device 200 is produced, so that the computing device 200 can perform binary operations by referring to the lookup table. According to the computing device production system 1, the table-type node 210 determines an output data value for an input data value by referring to the lookup table, and therefore can output data in a relatively short time and with relatively low power consumption even in the case of input / output corresponding to a complex operation.

[0120] The arithmetic device 200 is configured using a field programmable gate array. According to the computing device production system 1, an existing FPGA can be used as a template for the computing device 200, and there is no need to separately generate a template for the computing device 200. In this way, according to the computing device production system 1, the burden of producing the computing device 200 is relatively small.

[0121] 22 is a schematic block diagram illustrating an example of a computer configuration according to at least one embodiment. In the configuration shown in FIG. 22, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, and an interface 740.

[0122] Any one or more of the learning model device 100, the calculation device 200, the calculation device 200f, the learning control unit 300, and the setting unit 400 may be implemented in the computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program. The CPU 710 also allocates storage areas in the main storage device 720 corresponding to each of the above-mentioned storage units in accordance with the program.

[0123] When the learning model device 100 is implemented in a computer 700, the operation of the hypersurface node 110 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program.

[0124] Furthermore, the CPU 710 allocates a memory area in the main memory device 720 for the learning model device 100 to perform processing in accordance with the program. Communication between the learning model device 100 and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the learning model device 100 and the user is carried out by the interface 740 having an input device and an output device, presenting information to the user via the output device under the control of the CPU 710, and accepting user operations via the input device.

[0125] When the arithmetic device 200 is implemented in a computer 700, the operation of the table type node 210 is stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program.

[0126] Furthermore, the CPU 710 allocates a storage area in the main memory device 720 for the arithmetic device 200 to perform processing in accordance with the program. Communication between the arithmetic device 200 and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the arithmetic device 200 and a user is performed by the interface 740 having an input device and an output device, which presents information to the user via the output device under the control of the CPU 710, and receives user operations via the input device.

[0127] When the arithmetic device 200f is implemented in the computer 700, the operation of the table type node 210f is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program.

[0128] Furthermore, the CPU 710, in accordance with the program, allocates a storage area corresponding to the table storage unit 220f in the main storage device 720. Communication between the arithmetic device 200f and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the arithmetic device 200f and the user is performed by the interface 740 having an input device and an output device, presenting information to the user via the output device under the control of the CPU 710, and accepting user operations via the input device.

[0129] When the learning control unit 300 is implemented in a computer 700, the operation of the learning control unit 300 is stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0130] Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for the learning control unit 300 to perform processing in accordance with the program. Communication between the learning control unit 300 and other devices is performed by an interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the learning control unit 300 and the user is carried out by the interface 740 having an input device and an output device, presenting information to the user via the output device under the control of the CPU 710, and accepting user operations via the input device.

[0131] When the setting unit 400 is implemented in the computer 700, the operation of the setting unit 400 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0132] Furthermore, the CPU 710, in accordance with the program, allocates a storage area in the main storage device 720 for the setting unit 400 to perform processing. Communication between the setting unit 400 and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the setting unit 400 and the user is performed by the interface 740 having an input device and an output device, presenting information to the user via the output device under the control of the CPU 710, and accepting user operations via the input device.

[0133] In addition, a program for realizing all or part of the functions of the learning model device 100, the calculation device 200, the calculation device 200f, the learning control unit 300, and the setting unit 400 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform processing of each unit. Note that the term "computer system" here includes hardware such as an OS (Operating System) and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0134] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0135] 1. Computing Device Production System 100 Learning model device 110 Hypersurface Nodes 111 Hypersurface Processing Unit 112 Threshold calculation unit 200, 200f calculation unit 210, 210f Table type node 220f Table memory section 300 Learning control unit 400 Settings 220 Table storage section

Claims

1. A node that receives a binary vector as an input, and determines a binarized output value based on elements of the coordinate values of a point that has coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector. A learning model device comprising:

2. The hypersurface is a B-Spline hypersurface whose control points are points of coordinate values in the real number space, which are determined by combining the coordinate values when each of the values that the binary vector can take is treated as a coordinate value in a subspace of the real number space with a learning parameter value. The learning model device according to claim 1 .

3. The node a hypersurface processing unit that acquires values of elements other than the coordinate values of the binary vector among elements of coordinate values of points included in the hypersurface, the points having coordinate values including elements of the coordinate values when the binary vector is treated as coordinate values in a subspace of the real number space; and a threshold calculation unit that, in forward propagation in the learning model device, binarizes the value acquired by the hypersurface processing unit using a step function, and, in back propagation in the learning model device, approximates the step function using a differentiable function; The learning model device according to claim 1 or 2, comprising:

4. A learning model system, a learning control unit, and a setting unit are provided, The learning model system includes: A node that receives a binary vector as an input, and determines a binarized output value based on elements of the coordinate values of a point that has coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector. Equipped with the learning control unit controls learning of the learning model system; the setting unit generates a lookup table indicating a relationship between an input value and an output value at a node of the learning model system after learning, and sets the generated lookup table in a template of the arithmetic device. Computing device production system.

5. The arithmetic unit is configured using a Field Programmable Gate Array.

5. The computing device production system of claim 4.

6. The computer A binary vector is received as an input, and a binarized output value is determined based on elements of the coordinate values of a point that has coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector. A calculation method including:

7. learning a learning model system including a node that receives an input of a binary vector and determines an output value based on elements of coordinate values of a point that has coordinate values including each element of the coordinate values when the binary vector is treated as a coordinate value in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector; generating a lookup table indicating the relationship between inputs and outputs at the nodes of the learning model system after learning; The generated lookup table is set as a template for the computing device. A method for producing a computing device, comprising:

8. On the computer, A method of receiving an input of a binary vector and determining a binarized output value based on elements of the coordinate values of a point that has coordinate values including elements of the coordinate values when the binary vector is treated as coordinate values in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector A program to execute.

9. On the computer, learning a learning model system including a node that receives an input of a binary vector and determines an output value based on elements of coordinate values of a point that has coordinate values including elements of the coordinate values when the binary vector is treated as coordinate values in a subspace of the real number space, among points included in a hypersurface in a real number space that has one dimension greater than the dimension of the binary vector, other than the elements of the coordinate values of the binary vector; generating a lookup table indicating the relationship between inputs and outputs at the nodes of the learning model system after learning; Setting the generated lookup table as a template for the arithmetic device; A program to execute.

Images