Machine learning programs, machine learning methods, and information processing devices.

Abstract

This method speeds up the search for solutions to systems of linear equations represented by asymmetric coefficient matrices. [Solution] The information processing device 10 obtains a coefficient matrix 13 showing the coefficients included in the system of linear equations. The information processing device 10 includes nodes corresponding to the variables included in the system of linear equations and edges corresponding to the linear expressions included in the system of linear equations, and each edge generates a hypergraph 14 connecting nodes corresponding to the positions of non-zero coefficients included in the coefficient matrix. The information processing device 10 uses a machine learning model 15 including parameter values ​​to convert multiple first features associated with multiple nodes into multiple second features. Based on the connection relationships between nodes, the information processing device 10 calculates multiple third features associated with multiple nodes from the multiple second features. The information processing device 10 updates the parameter values ​​based on the error determined from the multiple third features.

Description

Technical Field

[0001] The present invention relates to a machine learning program, a machine learning method, and an information processing apparatus.

Background Art

[0002] A computer may execute a linear solver that solves a system of linear equations (Linear System). For example, in a scientific and technological simulation using the finite element method (FEM: Finite Element Method), a computer may solve a large-scale system of linear equations with a large number of variables. A typical linear solver using an iterative method such as the conjugate gradient (CG: Conjugate Gradient) method has a large amount of computation. Therefore, there is a technique for predicting the solution of a system of linear equations using a machine learning model. For example, there is a technique for representing a large-scale sparse symmetric coefficient matrix as an undirected graph and predicting the solution using a graph neural network (GNN: Graph Neural Network).

[0003] In addition, there is a technique for compressing sparse matrix data to reduce memory access. There is also a technique for rearranging the coefficient matrix and decomposing it into a triangular matrix, and obtaining the solution by forward elimination and backward substitution. There is also a technique for generating a hypergraph having a plurality of nodes corresponding to a plurality of entities, adding a directed edge indicating a partial order between the entities to the hypergraph, representing the hypergraph as a matrix, and generating a prediction function so that the value of the objective function is minimized.

[0004] In addition, there is a technique for synchronizing a plurality of threads that process a plurality of non-zero elements included in a sparse matrix in parallel by a synchronization method according to the number of non-zero elements. There is also a technique for predicting the medical effect of a treatment plan using a hypergraph.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] [Non-Patent Document 1] Luca Grementieri and Paolo Galeone, "Towards Neural Sparse Linear Solvers", arXiv:2203.06944, March 2022 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] The graph neural network described in Non-Patent Document 1 imposes the constraint that the coefficient matrix is ​​a symmetric matrix. Therefore, it is difficult to predict the solution to a system of linear equations represented by an asymmetric coefficient matrix using the graph neural network described in Non-Patent Document 1. As a result, the search for a solution to a system of linear equations represented by an asymmetric coefficient matrix can take a long time.

[0008] For example, Non-Patent Document 1 restricts the coefficient matrix to a symmetric matrix for the following reason: The order of multiple linear equations included in a system of linear equations does not affect the solution to the system of linear equations. Therefore, it is preferable for a linear solver to have row permutation invariance, meaning that the order of multiple rows included in the coefficient matrix does not affect the output.

[0009] If the coefficient matrix is ​​a symmetric matrix, then one row and one column with identical coefficient arrangements are linked. When one node of a graph neural network is generated from one row of the coefficient matrix, that node also corresponds to one column of the coefficient matrix. Therefore, even if the rows of the coefficient matrix are rearranged to maintain symmetry, the same undirected graph as before the transformation is generated from the transformed coefficient matrix. As a result, the graph neural network in Non-Patent Document 1 exhibits row order invariance for symmetric coefficient matrices.

[0010] On the other hand, if no symmetry constraint is imposed on the coefficient matrix, one row and one column of the coefficient matrix are not automatically linked. Therefore, if the rows of an asymmetric coefficient matrix are swapped, an undirected graph different from the original may be generated from the transformed coefficient matrix. As a result, the graph neural network described in Non-Patent Literature 1 does not have row order invariance for asymmetric coefficient matrices. Therefore, it is not advisable to apply the technique described in Non-Patent Literature 1 to a system of linear equations represented by an asymmetric coefficient matrix. Thus, in one aspect, the present invention aims to speed up the search for solutions to a system of linear equations represented by an asymmetric coefficient matrix. [Means for solving the problem]

[0011] In one aspect, a machine learning program is provided that causes a computer to perform the following processing: obtain a coefficient matrix showing the coefficients included in a system of linear equations; based on the coefficient matrix, generate a hypergraph that includes multiple nodes corresponding to multiple variables included in the system of linear equations and multiple edges corresponding to multiple linear expressions included in the system of linear equations; each of the multiple edges generates a hypergraph that connects nodes corresponding to the positions of non-zero coefficients included in the coefficient matrix; use a machine learning model that includes parameter values ​​to convert multiple first features associated with multiple nodes into multiple second features; calculate multiple third features associated with multiple nodes from the multiple second features based on the connection relationships between the multiple nodes shown by the multiple edges; and update the parameter values ​​based on the error determined from the multiple third features. [Effects of the Invention]

[0012] On one side, the solution search for a system of linear equations represented by an asymmetric coefficient matrix is accelerated.

Brief Description of the Drawings

[0013] [Figure 1] It is a diagram for explaining the information processing apparatus of the first embodiment. [Figure 2] It is a diagram showing a hardware example of the information processing apparatus of the second embodiment. [Figure 3] It is a diagram showing an example of the data structure of a system of linear equations. [Figure 4] It is a diagram showing an example of an undirected graph for predicting the solution of a system of linear equations. [Figure 5] It is a diagram showing an example of an undirected graph that satisfies row order invariance. [Figure 6] It is a diagram showing an example of a directed graph that does not satisfy row order invariance. [Figure 7] It is a diagram showing an example of a hypergraph for predicting the solution of a system of linear equations. [Figure 8] It is a diagram showing an example of row order invariance in a hypergraph. [Figure 9] It is a diagram showing an example of the data structure of a hypergraph. [Figure 10] It is a diagram showing an example of the structure of a hypergraph neural network. [Figure 11] It is a diagram showing an example of the structure of a node feature conversion unit. [Figure 12] It is a diagram showing an example of the calculation of an edge residual vector. [Figure 13] It is a diagram showing an example of the calculation of a node residual vector. [Figure 14] It is a diagram showing an example of the use of a hypergraph neural network. [Figure 15] It is a diagram showing an example of the input / output of a hypergraph neural network. [Figure 16] It is a diagram showing an example of a finite element method simulation. [Figure 17] This is a block diagram showing an example of the functions of an information processing device according to the second embodiment. [Figure 18] This is a flowchart showing an example of the model training procedure. [Figure 19] This flowchart shows an example of the solution prediction procedure. [Modes for carrying out the invention]

[0014] This embodiment will now be described with reference to the drawings. (a) First embodiment Figure 1 is a diagram illustrating an information processing device of a first embodiment. The information processing device 10 of the first embodiment trains a machine learning model to predict the solution to a system of linear equations. The information processing device 10 may use the trained machine learning model to predict the solution to the system of linear equations. The information processing device 10 may be a client device or a server device. The information processing device 10 may also be called a computer or a machine learning device.

[0015] The information processing device 10 includes a storage unit 11 and a processing unit 12. The storage unit 11 may be a volatile memory such as RAM (Random Access Memory). Alternatively, the storage unit 11 may be a non-volatile storage such as an HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0016] The processing unit 12 is a processor, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). However, the processing unit 12 may also include electronic circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The processor executes a program stored in memory, such as RAM. The processor is sometimes called a processor circuit. A collection of processors is sometimes called a multiprocessor or simply a "processor." Different processing steps among multiple processing steps may be executed by different processors.

[0017] The memory unit 11 stores the coefficient matrix 13. The coefficient matrix 13 represents the coefficients included in the system of linear equations. Typically, the columns of the coefficient matrix 13 correspond to multiple variables, and the rows of the coefficient matrix 13 correspond to multiple linear equations. However, the roles of rows and columns may be reversed.

[0018] Systems of linear equations are sometimes used in scientific and technological simulations using the finite element method (FEM). Examples of scientific and technological simulations include structural analysis to calculate the displacement and stress of objects, and computational fluid dynamics (CFD) to calculate the pressure of fluids. Variables representing physical quantities are assigned to nodes or sub-regions on a computer-aided design (CAD) model. The coefficient matrix used in the finite element method is typically a square matrix with rows and columns corresponding to the variables.

[0019] The coefficient matrix 13 may be a symmetric or asymmetric matrix. In the finite element method, the propagation of physical quantities between two locations on the CAD model can be asymmetric because it is directional. In such cases, an asymmetric coefficient matrix may be used.

[0020] Iterative linear solvers are sometimes used to solve systems of linear equations. Examples of iterative methods include the conjugate gradient method, GMRES (Generalized Minimal Residual) method, and Jacobi method. However, for large systems of linear equations, iterative linear solvers are computationally intensive and can take a long time to find a solution. Therefore, the information processing device 10 trains a machine learning model 15 to quickly predict the solution to the system of linear equations. The machine learning model 15 may also be called a surrogate model. The predicted values ​​calculated by the machine learning model 15 may be used as the final approximate solution. Alternatively, the predicted values ​​may be used as initial values ​​for other solution-finding algorithms such as iterative methods.

[0021] The processing unit 12 generates a hypergraph 14 based on the coefficient matrix 13. The hypergraph 14 includes multiple nodes corresponding to multiple variables in the system of linear equations, and multiple edges corresponding to multiple linear expressions in the system of linear equations. Each of the multiple edges connects to a node among the multiple nodes that corresponds to the position of a non-zero coefficient in the coefficient matrix 13. Unlike a general graph, one edge may connect three or more nodes.

[0022] Typically, one column of the coefficient matrix 13 corresponds to one node, and one row of the coefficient matrix 13 corresponds to one edge. An edge connects one or more nodes in the row corresponding to that edge that correspond to columns containing non-zero elements. In the example in Figure 1, the hypergraph 14 includes four nodes (nodes x1, x2, x3, x4) corresponding to columns of the coefficient matrix 13 and four edges corresponding to rows of the coefficient matrix 13.

[0023] The first row of coefficient matrix 13 contains three non-zero elements. The edges corresponding to the first row connect nodes x1, x2, and x4. The second row of coefficient matrix 13 contains two non-zero elements. The edges corresponding to the second row connect nodes x1 and x3. The third row of coefficient matrix 13 contains two non-zero elements. The edges corresponding to the third row connect nodes x2 and x4. The fourth row of coefficient matrix 13 contains four non-zero elements. The edges corresponding to the fourth row connect nodes x1, x2, x3, and x4.

[0024] The processing unit 12 associates multiple first features with multiple nodes included in the hypergraph 14. These multiple first features are, for example, initial values ​​for multiple variables. The initial values ​​may be predetermined fixed values, specified by the user, or randomly selected. The processing unit 12 uses the machine learning model 15 to convert the multiple first features into multiple second features. These multiple second features correspond to, for example, predicted solutions to a system of linear equations, and are expected to be closer to the true values ​​than the initial values.

[0025] The machine learning model 15 includes parameter values ​​that are trained through machine learning. The machine learning model 15 may be a neural network. Preferably, the machine learning model 15 performs nonlinear transformations on multiple features. For example, the machine learning model 15 includes a fully connected layer that transforms multiple features using parameter values ​​and a nonlinear activation layer that transforms the output of the fully connected layer using a nonlinear function. The fully connected layer is implemented, for example, as a parameter matrix multiplied by a vector of features. The nonlinear activation layer is, for example, a sigmoid function or a hyperbolic tangent function (tanh function). The machine learning model 15 may include multiple sets of fully connected layers and nonlinear activation layers alternatingly.

[0026] The processing unit 12 calculates multiple third features that correspond to multiple nodes from multiple second features, based on the connection relationships between multiple nodes indicated by multiple edges of the hypergraph 14. For example, the processing unit 12 calculates multiple fourth features that correspond to multiple edges from multiple second features that are node features, and calculates multiple third features that are node features from multiple fourth features that are edge features.

[0027] The processing unit 12 may calculate a fourth feature of an edge from the second features of one or more nodes connected to that edge. The fourth feature may be a weighted sum of one or more second features. The weights may be non-zero elements included in the coefficient matrix 13. In the example in Figure 1, the fourth feature of the edge corresponding to the first row of the coefficient matrix 13 may be the sum of twice the second feature of node x1, -3 times the second feature of node x2, and -5 times the second feature of node x4. Alternatively, the fourth feature may represent the residual obtained by subtracting the constant term on the right-hand side of the system of linear equations from the above weighted sum.

[0028] Furthermore, the processing unit 12 may calculate the third feature of a node from the fourth features of one or more edges that contain that node. The third feature may be the sum of one or more fourth features. In the example in Figure 1, the third feature of node x1 may be the sum of the fourth feature of the edge corresponding to the first row of the coefficient matrix 13, the fourth feature of the edge corresponding to the second row, and the fourth feature of the edge corresponding to the fourth row. The fourth feature is, for example, the residual under the second feature, which is the predicted value of the solution, evaluated on a variable-by-variable basis. Note that the connection relationship between nodes and edges is many-to-many and may be represented as a bipartite graph.

[0029] The processing unit 12 updates the parameter values ​​included in the machine learning model 15 based on the error determined from a plurality of third features. Error metrics include, for example, the mean squared error (MSE), cosine distance, and L2 norm. If the machine learning model 15 is a neural network, the processing unit 12 may update the parameter values ​​to reduce the error using backpropagation. The machine learning model 15 may also be called a hypergraph neural network (HGNN).

[0030] The processing unit 12 may repeat the conversion from the first feature to the second feature, the calculation of the third feature, and the updating of parameter values ​​until a certain stopping condition is met. In this case, the processing unit 12 may use the converted second feature in one iteration as the first feature in the next iteration. As the iterations progress, the error determined from the third feature is reduced, and the predicted value of the solution shown by the second feature is improved. The stopping condition may be that the number of iterations reaches a threshold, that the error falls below a threshold, or the logical AND of both.

[0031] During the training phase, the processing unit 12 outputs the trained machine learning model 15. The processing unit 12 may store the parameter values ​​of the machine learning model 15 in non-volatile storage, display them on a display device, or transmit them to another information processing device.

[0032] In the prediction phase, for example, the processing unit 12 reads out the trained machine learning model 15, generates a hypergraph corresponding to the system of linear equations to be computed, and inputs initial values ​​of multiple variables into the machine learning model 15. The processing unit 12 may also refine the parameter values ​​using backpropagation, similar to the training phase, to fit the system of linear equations to be computed. The processing unit 12 may output the transformed second features as predicted values ​​of the solution to the system of linear equations. The processing unit 12 may store the predicted values ​​in non-volatile storage, display them on a display device, or transmit them to another information processing device.

[0033] As described above, the information processing device 10 of the first embodiment obtains a coefficient matrix 13 showing the coefficients included in the system of linear equations. Based on the coefficient matrix 13, the information processing device 10 generates a hypergraph 14 that includes multiple nodes corresponding to multiple variables included in the system of linear equations and multiple edges corresponding to multiple linear expressions. Each of the multiple edges connects to a node among the multiple nodes that corresponds to the position of a non-zero coefficient included in the coefficient matrix 13.

[0034] The information processing device 10 uses a machine learning model 15 that includes parameter values ​​to convert multiple first features, which are associated with multiple nodes, into multiple second features. Based on the connection relationships between multiple nodes indicated by multiple edges, the information processing device 10 calculates multiple third features, which are associated with multiple nodes, from the multiple second features. The information processing device 10 updates the parameter values ​​based on the error determined from the multiple third features.

[0035] As a result, the information processing device 10 can speed up the search for solutions to systems of linear equations compared to using an iterative linear solver. Furthermore, even if the coefficient matrix 13 is asymmetric, the same hypergraph 14 can be generated from other coefficient matrices with the rows of coefficient matrix 13 swapped. Therefore, the machine learning model 15 that searches for solutions using the hypergraph 14 has row order invariance with respect to the asymmetric coefficient matrix 13. Thus, the stability of the solution search is improved, and the accuracy of solution prediction is improved. In addition, if the size of the system of linear equations is constant, the computational complexity and prediction accuracy of the machine learning model 15 are generally constant. Therefore, the information processing device 10 does not need to perform preprocessing to optimize the order of the multiple linear expressions included in the system of linear equations.

[0036] (b) Second embodiment Figure 2 shows an example of the hardware of the information processing device according to the second embodiment. The information processing device 100 of the second embodiment trains a machine learning model to predict the solution to a system of linear equations. The information processing device 100 also uses the trained machine learning model to predict the solution to the system of linear equations. However, training and prediction may be performed by different information processing devices. The information processing device 100 corresponds to the information processing device 10 of the first embodiment.

[0037] The information processing device 100 includes a CPU 101, RAM 102, HDD 103, GPU 104, input interface 105, media reader 106, and communication interface 107. The CPU 101 corresponds to the processing unit 12 of the first embodiment. The RAM 102 or HDD 103 corresponds to the storage unit 11 of the first embodiment.

[0038] The CPU 101 is a processor that executes program instructions. The CPU 101 loads the program and data from the HDD 103 into the RAM 102 and executes the program. The information processing device 100 may have multiple processors.

[0039] RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by CPU 101 and data used for calculations by CPU 101. The information processing device 100 may have a type of volatile memory other than RAM.

[0040] The HDD 103 is a non-volatile storage device that stores software programs such as operating systems, middleware, and application software, as well as data. The information processing device 100 may have other types of non-volatile storage, such as an SSD or flash memory.

[0041] The GPU 104 works in conjunction with the CPU 101 to perform image processing and outputs the image to the display device 111 connected to the information processing device 100. The display device 111 is, for example, a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, or a projector.

[0042] Furthermore, the GPU 104 may be used as a GPGPU (General Purpose Computing on Graphics Processing Unit). The GPU 104 can execute programs in response to instructions from the CPU 101. The information processing device 100 may have volatile semiconductor memory other than RAM 102 as GPU memory.

[0043] The input interface 105 receives input signals from an input device 112 connected to the information processing device 100. The input device 112 is, for example, a mouse, a touch panel, or a keyboard. Multiple input devices may be connected to the information processing device 100.

[0044] The media reader 106 is a reading device that reads programs and data recorded on the recording medium 113. The recording medium 113 is, for example, a magnetic disk, an optical disk, or semiconductor memory. Magnetic disks include flexible disks (FD) and HDDs. Optical disks include CDs (Compact Discs) and DVDs (Digital Versatile Discs). The media reader 106 copies the programs and data read from the recording medium 113 to other recording media such as RAM 102 or HDD 103. The read programs may be executed by the CPU 101.

[0045] The recording medium 113 may be a portable recording medium. The recording medium 113 may be used for distributing programs and data. The recording medium 113 and the HDD 103 may also be referred to as computer-readable recording media.

[0046] The communication interface 107 communicates with other information processing devices via the network 114. The communication interface 107 may be a wired communication interface connected to a wired communication device such as a switch or router, or a wireless communication interface connected to a wireless communication device such as a base station or access point.

[0047] Next, we will explain how to predict the solution to a system of linear equations. Systems of linear equations are sometimes used in scientific and technological simulations using the finite element method. Examples of scientific and technological simulations include structural analysis to calculate the displacement and stress of an object, and computational fluid dynamics to calculate the pressure of a fluid. Linear solvers using iterative methods are sometimes used to solve systems of linear equations. Examples of iterative methods include the conjugate gradient method, the GMRES method, and the Jacobi method.

[0048] However, for large systems of linear equations, iterative linear solvers are computationally intensive and can take a long time to find a solution. Therefore, the information processing device 100 uses a machine learning model as a surrogate model to quickly predict the solution to the system of linear equations. The predicted value calculated by the information processing device 100 may be used as the final approximate solution, or as an initial value for other solution-finding algorithms such as iterative methods.

[0049] Figure 3 shows an example of a data structure for a system of linear equations. The system of linear equations 201 contains multiple variables and multiple linear equations. The left-hand side of each linear equation contains multiple linear terms, which are the products of the variables and coefficients. The right-hand side of each linear equation contains one constant term. The system of linear equations 201 is decomposed into a coefficient matrix 202, a solution vector 203, and a right-hand side vector 204.

[0050] The coefficient matrix 202 is a matrix obtained by extracting and arranging the coefficients contained in the left-hand side of the system of linear equations 201. Each row of the coefficient matrix 202 corresponds to one linear expression, and each column corresponds to one variable. The arrangement of multiple coefficients in the coefficient matrix 202 depends on the order of the linear expressions and the order of the variables specified in the system of linear equations 201. The system of linear equations 201 with the order of the linear expressions changed is essentially the same system of linear equations. However, the coefficient matrix 202 with the rows changed is formally treated as a different coefficient matrix.

[0051] The coefficient matrix 202 can be either a symmetric or asymmetric matrix. A symmetric matrix is ​​one in which the elements at row i and column j are identical to the elements at row j and column i (where i and j are non-negative integers). In the finite element method, the coefficient matrix 202 can be symmetric if the propagation of a physical quantity between two adjacent locations is symmetric. Conversely, the coefficient matrix 202 can be asymmetric if the propagation of a physical quantity between two adjacent locations is asymmetric.

[0052] The solution vector 203 is a column vector obtained by extracting and arranging the variables contained in the left-hand side of the system of linear equations 201. Each element of solution vector 203 corresponds to one variable. At the start of the solution search, solution vector 203 is a variable vector consisting of unknowns. Through the solution search, the numerical values ​​of each element of solution vector 203 are calculated. The right-hand side vector 204 is a column vector obtained by extracting and arranging the constants contained in the right-hand side of the system of linear equations 201. Each element of right-hand side vector 204 corresponds to one linear expression.

[0053] If we denote the coefficient matrix 202 as A, the solution vector 203 as x, and the right-hand side vector 204 as b, then the system of linear equations 201 can be expressed as the matrix equation Ax=b. The solution to the system of linear equations can sometimes be predicted using a graph neural network as a surrogate model, as in the technique described in Non-Patent Document 1 mentioned above.

[0054] Figure 4 shows an example of an undirected graph for predicting the solution to a system of linear equations. The coefficient matrix 205 is a 4x4 coefficient matrix. Here, we consider the case where the coefficient matrix 205 is a symmetric matrix. The right-hand side vector 206 is a right-hand side vector of length 4.

[0055] As an example, a machine learning model generates an undirected graph 207 from a coefficient matrix 205. The undirected graph 207 contains nodes corresponding to the rows of the coefficient matrix 205. Specifically, the undirected graph 207 contains nodes 208, 208-1, 208-2, and 208-3. Node 208 corresponds to the first row of the coefficient matrix 205. Node 208-1 corresponds to the second row of the coefficient matrix 205. Node 208-2 corresponds to the third row of the coefficient matrix 205. Node 208-3 corresponds to the fourth row of the coefficient matrix 205.

[0056] The undirected graph 207 contains weighted undirected edges corresponding to the non-zero elements of the coefficient matrix 205. The undirected graph 207 contains an edge with weight 1 that returns from node 208 to node 208 itself. The undirected graph 207 also contains an edge with weight 9 that connects node 208 to node 208-2. The undirected graph 207 also contains an edge with weight 77 that connects node 208 to node 208-3. The undirected graph 207 also contains an edge with weight 4 that returns from node 208-1 to node 208-1 itself.

[0057] Furthermore, undirected graph 207 includes an edge with weight 15 connecting node 208-1 and node 208-3. Also, undirected graph 207 includes an edge with weight 17 returning from node 208-2 to node 208-2 itself. Furthermore, undirected graph 207 includes an edge with weight -21 returning from node 208-3 to node 208-3 itself.

[0058] The machine learning model assigns the values ​​of the right-hand vector 206 to the nodes as initial values ​​for the features. Node 208 is assigned the first value of the right-hand vector 206, which is 0.2. Node 208-1 is assigned the second value of the right-hand vector 206, which is 0.4. Node 208-2 is assigned the third value of the right-hand vector 206, which is 0.1. Node 208-3 is assigned the fourth value of the right-hand vector 206, which is 0.9.

[0059] The machine learning model updates the features of nodes by sending and receiving features between nodes according to the weighted edges contained in the undirected graph 207. The machine learning model iteratively updates the features. For example, the machine learning model transforms the features of each node at time t using trained parameter values. The machine learning model calculates the weighted sum of the transformed features of neighboring nodes connected to a given node as the features of that node at time t+1. The weights used here are the edge weights.

[0060] The machine learning model outputs the final features of the nodes in the undirected graph 207 as predicted values ​​for the solutions to the system of linear equations. For example, the features of node 208 correspond to the value of the first variable. The features of node 208-1 correspond to the value of the second variable. The features of node 208-2 correspond to the value of the third variable. The features of node 208-3 correspond to the value of the fourth variable.

[0061] Here, it is preferable that the machine learning model predicting the solution to a system of linear equations has row order invariance. Row order invariance is the property that the order of the linear expressions included in the system of linear equations does not affect the solution. Machine learning models with row order invariance have improved stability and prediction accuracy compared to those without row order invariance.

[0062] Furthermore, in iterative linear solvers, the order of the linear equations can affect the computation time of the linear solver. For example, the closer the coefficient matrix is ​​to a diagonal matrix, the shorter the computation time of the linear solver may be. On the other hand, in graph neural networks with row order invariance, the order of the linear equations is expected not to affect the computation time. Therefore, it is expected that preprocessing to optimize the order of the linear equations can be omitted. In this respect, when the coefficient matrix is ​​a symmetric matrix, graph neural networks using undirected graphs have row order invariance.

[0063] Figure 5 shows an example of an undirected graph that satisfies row order invariance. The coefficient matrix 211 is a symmetric coefficient matrix of size 3 × 3. An undirected graph 213 is generated from the coefficient matrix 211. The undirected graph 213 contains nodes 214, 214-1, and 214-2. Node 214 corresponds to the first row of the coefficient matrix 211. Node 214-1 corresponds to the second row of the coefficient matrix 211. Node 214-2 corresponds to the third row of the coefficient matrix 211.

[0064] The undirected graph 213 includes an edge with weight -3 that returns from node 214 to node 214 itself. It also includes an edge with weight -5 connecting node 214 and node 214-2. Furthermore, it includes an edge with weight 7 that returns from node 214-1 to node 214-1 itself. Finally, it includes an edge with weight 4 connecting node 214-1 and node 214-2. And finally, it includes an edge with weight 1 that returns from node 214-2 to node 214-2 itself.

[0065] Since the coefficient matrix 211 is a symmetric matrix, one row and one column with identical arrangements of coefficients are linked. A node in the undirected graph 213 corresponds to both one row and one column of the coefficient matrix 211. If we swap two rows of the coefficient matrix 211 while maintaining symmetry, the new coefficient matrix representing essentially the same system of linear equations will also have the two columns linked to those rows swapped.

[0066] For example, the coefficient matrix 212 is generated by swapping the first and second rows of the coefficient matrix 211, as well as swapping the first and second columns of the coefficient matrix 211. The coefficient matrix 212 represents essentially the same system of linear equations as the coefficient matrix 211. A machine learning model using an undirected graph generates an undirected graph 213 from the coefficient matrix 212 that is identical to the coefficient matrix 211. However, node 214 corresponds to the second row, and node 214-1 corresponds to the first row.

[0067] Therefore, the machine learning model calculates the same solution from coefficient matrices 211 and 212. However, when attempting to apply a machine learning model that utilizes an undirected graph to an asymmetric coefficient matrix, the machine learning model loses its row-order invariance.

[0068] Figure 6 shows an example of a directed graph that does not satisfy row order invariance. The coefficient matrix 215 is an asymmetric coefficient matrix of size 3x3. A machine learning model can generate a directed graph 217 from the asymmetric coefficient matrix 215. The directed graph 217 contains nodes 218, 218-1, and 218-2. Node 218 corresponds to the first row of the coefficient matrix 215. Node 218-1 corresponds to the second row of the coefficient matrix 215. Node 218-2 corresponds to the third row of the coefficient matrix 215.

[0069] The directed graph 217 contains weighted directed edges corresponding to the non-zero coefficients of the coefficient matrix 215. The directed graph 217 contains an edge with weight -3 that returns from node 218 to node 218 itself. The directed graph 217 also contains an edge with weight 2 that transitions from node 218 to node 218-1, and an edge with weight -5 that transitions from node 218 to node 218-2.

[0070] Furthermore, directed graph 217 includes an edge with weight -3 that returns from node 218-1 to node 218-1 itself. Also, directed graph 217 includes an edge with weight 4 that transitions from node 218-1 to node 218-2. Also, directed graph 217 includes an edge with weight 1 that transitions from node 218-2 to node 218, and an edge with weight 1 that transitions from node 218-2 to node 218-1.

[0071] Since the coefficient matrix 215 is an asymmetric matrix, one row and one column are not linked. A new coefficient matrix obtained by swapping two rows of coefficient matrix 215 but not swapping columns represents essentially the same system of linear equations with the same solution as coefficient matrix 215. For example, coefficient matrix 216 is generated by swapping the first and second rows of coefficient matrix 215. However, coefficient matrix 216 generates a directed graph different from the directed graph 217. Therefore, machine learning models may calculate different solutions from coefficient matrices 215 and 216.

[0072] Therefore, the information processing device 100 of the second embodiment trains a machine learning model that is row-order invariant even with respect to an asymmetric coefficient matrix. The information processing device 100 then uses the trained machine learning model to predict the solution to a system of linear equations. The machine learning model of the second embodiment is a hypergraph neural network that utilizes a hypergraph.

[0073] Figure 7 shows an example of a hypergraph for predicting the solution to a system of linear equations. A hypergraph is an extended graph that extends the general graph, where one edge connects two nodes, and one edge can connect three or more nodes. A hypergraph is defined by (V,E), where V is the set of nodes and E is the set of edges. A hyperedge, which is an element of the edge set, has one or more nodes as its target. The number of nodes is sometimes called the order of the hypergraph. The number of edges is sometimes called the size of the hypergraph.

[0074] The system of linear equations 131 contains four variables and four linear equations. The coefficient matrix corresponding to the system of linear equations 131 is an asymmetric matrix of size 4 × 4. The information processing device 100 generates a hypergraph 140 from the system of linear equations 131. The information processing device 100 generates one node from one variable of the system of linear equations 131 and one edge from one linear equation of the system of linear equations 131. The edge connects the nodes among the multiple nodes that correspond to the variables in the given linear equation that have non-zero coefficients.

[0075] Hypergraph 140 includes nodes 141, 142, 143, 144 and edges 145, 146, 147, 148. Node 141 corresponds to variable x1, node 142 to variable x2, node 143 to variable x3, and node 144 to variable x4. Edge 145 corresponds to the first linear expression and connects nodes 141, 142, 144, which correspond to variables x1, x2, x4 with non-zero coefficients.

[0076] Edge 146 corresponds to the second linear equation and connects nodes 141 and 143, which correspond to variables x1 and x3 with non-zero coefficients. Edge 147 corresponds to the third linear equation and connects nodes 142 and 144, which correspond to variables x2 and x4, which have non-zero coefficients. Edge 148 corresponds to the fourth linear equation and connects nodes 141, 142, 143, and 144, which correspond to variables x1, x2, x3, and x4, which have non-zero coefficients.

[0077] Figure 8 shows an example of row order invariance in a hypergraph. Hypergraph 140 exhibits row order invariance. For example, the information processing device 100 can generate a system of linear equations 132 by swapping the second and fourth linear equations of the system of linear equations 131. The system of linear equations 132 is substantially identical to the system of linear equations 131 and has the same solutions. The information processing device 100 also generates the same hypergraph 140 from the system of linear equations 132 as it does from the system of linear equations 131.

[0078] Figure 9 shows an example of a hypergraph data structure. The structure of the hypergraph 140 is represented by an incidence matrix 133. The incidence matrix 133 contains multiple rows corresponding to multiple nodes and multiple columns corresponding to multiple edges. In relation to the system of linear equations 131, the incidence matrix 133 contains four rows corresponding to the variables x1, x2, x3, and x4, and four columns corresponding to the four linear equations.

[0079] The elements of the connection matrix 133 are flags with values ​​of 0 or 1. The i-th row and j-th column of the connection matrix 133 indicate whether the edge corresponding to the j-th column contains the node corresponding to the i-th row. That is, the i-th row and j-th column of the connection matrix 133 indicate whether the linear expression corresponding to the j-th column contains a non-zero linear term with the variable corresponding to the i-th row. A flag of 1 indicates that the edge contains the node. A flag of 0 indicates that the edge does not contain the node.

[0080] Edge 145 contains nodes 141, 142, and 144, so the first column of the connection matrix 133 is (1,1,0,1). Edge 146 contains nodes 141 and 143, so the second column is (1,0,1,0). Edge 147 contains nodes 142 and 144, so the third column is (0,1,0,1). Edge 148 contains nodes 141, 142, 143, and 144, so the fourth column is (1,1,1,1).

[0081] The connection matrix 133 may be a sparse matrix with few non-zero elements. In that case, the information processing device 100 can reduce the amount of data by converting the connection matrix 133 into an index tensor 134. The index tensor 134 is equivalent to the connection matrix 133 in that it represents the hypergraph 140. The index tensor 134 is an enumeration of pairs of row and column numbers that are non-zero elements. Since the connection matrix 133 contains 11 non-zero elements, the index tensor 134 contains 11 pairs. Multiple pairs are sorted, for example, in ascending order of column number, and then sorted in ascending order of row number among pairs with the same column number.

[0082] Next, we will describe hypergraph neural networks that utilize hypergraphs. Hypergraph neural networks are also described in the following non-patent document. However, the structure of the hypergraph neural network used in the second embodiment differs from that of the following non-patent document: Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao, "Hypergraph Neural Networks", Proc. of the 33rd AAAI (Association for the Advancement of Artificial Intelligence) Conference on Artificial Intelligence (AAAI-19), pages 3558-3565, January 2019.

[0083] Figure 10 shows an example of the structure of a hypergraph neural network. The hypergraph neural network 150 includes a node feature transformation unit 151, an edge residual calculation unit 152, and a node residual calculation unit 153.

[0084] The information processing device 100 first assigns initial values ​​to node features for each node included in the hypergraph. These initial values ​​are the first predicted values ​​of the variable. The initial values ​​may be fixed values, provided by the user, or randomly selected. As a result, the information processing device 100 generates a node feature vector 154 (node ​​feature vector X) which enumerates the initial values ​​of multiple nodes. The length of the node feature vector 154 is the same as the number of nodes in the hypergraph and the same as the number of variables.

[0085] The information processing device 100 inputs the node feature vector 154 to the node feature transformation unit 151. The node feature transformation unit 151 is a multilayer neural network containing multiple layers. The node feature transformation unit 151 transforms the node feature vector 154 into a node feature vector 155 (node ​​feature vector X') using a parameter matrix Θ and a nonlinear activation function σ. The parameter matrix Θ is a set of parameter values ​​trained through machine learning. As will be described later, the node feature transformation unit 151 alternately arranges fully connected layers using the parameter matrix Θ and nonlinear activation layers using the nonlinear activation function σ.

[0086] The length of node feature vector 155 is the same as that of node feature vector 154. The node feature transformation unit 151 calculates a different node feature for each node from the initial value of the node feature. The transformed node feature corresponds to an improved predicted value of the variable's value and is expected to be an approximation closer to the true value than the initial value shown by node feature vector 154.

[0087] The information processing device 100 inputs the node feature vector 155 to the edge residual calculation unit 152. The edge residual calculation unit 152 calculates the edge residual vector 156 (edge ​​residual vector Y) from the node feature vector 155 according to the hypergraph. The edge residual calculation unit 152 uses the coefficient matrix A and the right-hand side vector b. The length of the edge residual vector 156 is the same as the number of edges in the hypergraph and the same as the number of linear equations.

[0088] The edge residual calculation unit 152 aggregates edge features from the node features of the nodes included in each edge. However, as will be described later, the node features are weighted by the coefficients included in the coefficient matrix A. The edge residual calculation unit 152 also subtracts a constant included in the right-hand side vector b from the weighted sum of the node features for each edge. The edge features correspond to the residual between the right-hand and left-hand sides of a linear equation calculated under the predicted solution value shown by the node feature vector 155. It is preferable for the absolute value of the edge features to be small.

[0089] The information processing device 100 inputs the edge residual vector 156 to the node residual calculation unit 153. The node residual calculation unit 153 calculates the node residual vector 157 (node ​​residual vector R) from the edge residual vector 156 according to the hypergraph. The node residual calculation unit 153 uses the connection matrix H. The length of the node residual vector 157 is the same as the number of nodes in the hypergraph and the same as the number of variables.

[0090] The node residual calculation unit 153 calculates new node features for each node from the edge features of the edge containing that node. Here, the edge features are not weighted. The new node features correspond to the residuals aggregated on a variable-by-variable basis. It is preferable that the absolute value of the new node features be small. In the ideal situation where the node feature vector 155 matches the true value of the solution, the new node features will be zero.

[0091] Figure 11 shows an example of the structure of the node feature transformation unit. The node feature transformation unit 151 includes multiple fully connected layers, including fully connected layers 161, 163, and 165, and multiple nonlinear activation layers, including nonlinear activation layers 162, 164, and 166. The node feature transformation unit 151 connects the fully connected layers and nonlinear activation layers alternately in series. For example, the node feature transformation unit 151 includes several to several dozen layers. The number of layers may be specified by the user.

[0092] The fully connected layers 161, 163, and 165 each transform the input node feature vector using the parameter matrix Θ of their respective layers. The vector length does not change before and after the fully connected layers 161, 163, and 165. The nonlinear activation layers 162, 164, and 166 each perform a nonlinear transformation of the input node feature vector using a nonlinear activation function σ. The vector length does not change before and after the nonlinear activation layers 162, 164, and 166. The nonlinear activation function σ may be common to all nonlinear activation layers 162, 164, and 166, or it may be different for each layer. The nonlinear activation function σ may be, for example, a sigmoid function or a hyperbolic tangent function.

[0093] The fully connected layer 161 accepts a node feature vector 167 (node ​​feature vector X). The node feature vector 167 corresponds to the initial predicted values ​​of the solution to a system of linear equations. The node feature vector 167 contains the initial values ​​of variables x1, x2, x3, and x4. The fully connected layer 161 transforms the node feature vector 167 by multiplying it by the parameter matrix Θ1 of the fully connected layer 161. The nonlinear activation layer 162 applies a nonlinear activation function σ to the node feature vector output by the fully connected layer 161.

[0094] The fully connected layer 163 multiplies the node feature vector output by the nonlinear activation layer 162 by its own parameter matrix Θ2. The nonlinear activation layer 164 applies a nonlinear activation function σ to the node feature vector output by the fully connected layer 163.

[0095] The fully connected layer 165 has a parameter matrix Θ n This is multiplied by the node feature vector output by the preceding nonlinear activation layer. The nonlinear activation layer 166 applies the nonlinear activation function σ to the node feature vector output by the fully connected layer 165 and outputs node feature vector 168 (node ​​feature vector X'). Node feature vector 168 corresponds to an improved prediction value compared to node feature vector 167. Node feature vector 168 includes improved values ​​for variables x1, x2, x3, and x4.

[0096] Figure 12 shows an example of edge residual vector calculation. The edge residual calculation unit 152 calculates the modified transpose connection matrix 171 (modified transpose connection matrix H T Prepare '). When calculating edge features from node features in the hypergraph 140, the edge residual calculation unit 152 may use the transpose of the connection matrix 133. However, as mentioned above, in the second embodiment, node features are weighted using the coefficients of the system of linear equations.

[0097] Therefore, the edge residual calculation unit 152 obtains the transposed connection matrix H by transposing the connection matrix 133. T Replace the "1" elements in with the corresponding coefficients in the system of linear equations. Transpose connection matrix H T Each row corresponds to a single linear expression, and the transpose connection matrix H T Each column corresponds to a single variable. Transposed connection matrix H T If the i-th row and j-th column of the equation is "1", the edge residual calculation unit 152 replaces that element with the coefficient of the j-th linear term of the i-th linear equation.

[0098] As a result, the edge residual calculation unit 152 generates a modified transpose connection matrix 171, which is a weighted transpose connection matrix. This modified transpose connection matrix 171 ultimately coincides with the coefficient matrix A. The modified transpose connection matrix 171 is the coefficient matrix corresponding to the system of linear equations 131. The edge residual calculation unit 152 calculates the edge residual vector 173 by multiplying the node feature vector 168 by the modified transpose connection matrix 171 and subtracting the right-hand side vector 172. This matrix operation is H T This can be expressed as 'X'-b=Y or AX'-b=Y.

[0099] Let the predicted solutions to the system of linear equations 131 be denoted as (x1', x2', x3', x4'). Then, the first value of the edge residual vector 173 is 2x1'-3x2'-5x4'+14, representing the residual of the first linear equation. The second value of the edge residual vector 173 is 13x1'-3x3'-10, representing the residual of the second linear equation. The third value of the edge residual vector 173 is x2'+x4', representing the residual of the third linear equation. The fourth value of the edge residual vector 173 is 6x1'+4x2'+9x3'+2x4'-22, representing the residual of the fourth linear equation.

[0100] Figure 13 shows an example of calculating a node residual vector. The node residual calculation unit 153 calculates the node residual vector 174 by multiplying the edge residual vector 173 by the connection matrix 133. The node residual calculation unit 153 does not need to weight the non-zero elements of the connection matrix 133 with coefficients. The node residual vector 174 represents the residuals aggregated on a variable-by-variable basis.

[0101] The first value of node residual vector 174 is 21x1'+x2'+6x3'-3x4'-18, representing the residual for variable x1. The second value of node residual vector 174 is 8x1'+2x2'+9x3'-2x4'-8, representing the residual for variable x2. The third value of node residual vector 174 is 19x1'+4x2'+6x3'+2x4'-32, representing the residual for variable x3. The fourth value of node residual vector 174 is 8x1'+2x2'+9x3'-2x4'-8, representing the residual for variable x4.

[0102] In this way, the hypergraph neural network 130 calculates node feature vectors 168 that correspond to the predicted values ​​of the solutions to the system of linear equations, and node residual vectors 174 that show the evaluation of the predicted values. Next, the training phase and prediction phase using the hypergraph neural network 130 will be described.

[0103] Figure 14 shows an example of the use of a hypergraph neural network. In order to train the hypergraph neural network 130, the information processing device 100 prepares training data. The training data includes one or more coefficient matrices and one or more right-hand side vectors corresponding to one or more systems of linear equations.

[0104] On the other hand, the training data does not need to include solution vectors corresponding to labels that indicate the correct answer. Therefore, machine learning with the hypergraph neural network 130 is equivalent to unsupervised learning. This is because it is obvious that the node feature vector 168 such that the node residual vector 174 is a zero vector is the correct answer. The user only needs to provide a sample of a system of linear equations as training data, and does not need to calculate the solution to the system of linear equations in advance. For example, the user may generate multiple samples of a system of linear equations from a CAD model used in the finite element method, while changing the boundary conditions.

[0105] The information processing device 100 generates a hypergraph from the coefficient matrix of the system of linear equations included in the training data. The information processing device 100 also initializes the parameter matrix included in the hypergraph neural network 130. The information processing device 100 generates a node feature vector 167, which corresponds to the initial values ​​of the solution vector, and inputs it to the hypergraph neural network 130. The information processing device 100 obtains the node feature vector 168 and the node residual vector 174 from the hypergraph neural network 130.

[0106] The information processing device 100 calculates the error by inputting the node residual vector 174 into the loss function 181. The error is an index value that indicates the magnitude of the node residual vector 174. For example, the loss function 181 is the mean squared error between the node residual vector 174 and the zero vector. However, the loss function 181 may be the cosine similarity or the L2 norm.

[0107] The information processing device 100 updates the parameter matrix included in the hypergraph neural network 130 using backpropagation to reduce the error. In the second embodiment, the transformation from node feature vector 167 to node feature vector 168 and node residual vector 174 is implemented as a neural network. Therefore, the information processing device 100 can apply backpropagation to the parameter matrix.

[0108] When the parameter matrix is ​​updated, the information processing device 100 uses the transformed node feature vector 168 as input to the hypergraph neural network 130, replacing the original node feature vector 167. The information processing device 100 then re-executes the above process with the updated parameter matrix and the replaced node feature vector.

[0109] The information processing device 100 repeats the iteration until the termination condition is met. The termination condition may be that the number of iterations reaches a threshold, that the error falls below a threshold, or a combination of both. If the training data contains two or more systems of linear equations, the information processing device 100 performs the above process on those two or more systems of linear equations. When the termination condition is met, the information processing device 100 saves the trained hypergraph neural network 130.

[0110] In the above explanation, the parameter matrix is ​​updated once and the input node feature vector is replaced once per iteration. In contrast, the information processing device 100 may repeat the process of transforming the node feature vector and feeding it back to the input multiple times while updating the parameter matrix once. Alternatively, after updating the parameter matrix, the information processing device 100 may re-evaluate the error without changing the input node feature vector before changing the input node feature vector.

[0111] In the prediction phase, the information processing device 100 reads out the trained hypergraph neural network 130. The information processing device 100 generates a hypergraph from the coefficient matrix of the system of linear equations to be predicted. The information processing device 100 generates node feature vectors 167, which correspond to the initial values ​​of the solution vectors, and inputs them into the hypergraph neural network 130. The information processing device 100 obtains node feature vectors 168 and node residual vectors 174 from the hypergraph neural network 130.

[0112] In the prediction phase, the information processing device 100 fine-tunes the parameter matrix of the hypergraph neural network 130 to fit the system of linear equations to be predicted. Then, the information processing device 100 performs the same iterations as in the training phase. The information processing device 100 calculates the error by inputting the node residual vector 174 into the loss function 181. The information processing device 100 updates the parameter matrix included in the hypergraph neural network 130 using backpropagation to reduce the error.

[0113] When the parameter matrix is ​​updated, the information processing device 100 uses the transformed node feature vector 168 as input to the hypergraph neural network 130, replacing the original node feature vector 167. The information processing device 100 then re-executes the above process with the updated parameter matrix and the replaced node feature vector.

[0114] The information processing device 100 repeats the iteration until the termination condition is met. When the termination condition is met, the information processing device 100 outputs the node feature vector 168, which was last calculated by the hypergraph neural network 130, as the predicted value of the solution. The information processing device 100 may use this predicted value as an approximate solution to the system of linear equations, or it may use it as the initial value of an iterative linear solver.

[0115] The information processing device 100 may perform training and prediction phases using a system of linear equations generated from the CAD model when repeating finite element method simulations while changing boundary conditions on the same CAD model. In this case, the vector length of the node feature vector 167 is constant. The information processing device 100 may also predict the solution to a system of linear equations of a different size than that of the training phase during the prediction phase. In this case, the information processing device 100 may perform a process to match the vector length of the node feature vector 167 with the size of the parameter matrix, such as interpolating missing dimensions with zero.

[0116] Figure 15 shows an example of input and output for a hypergraph neural network. The information processing device 100 generates a connection matrix 185 representing the hypergraph from the coefficient matrix 183. The information processing device 100 also extracts and stores weights 186 to be applied to the "1" elements of the connection matrix 185 from the coefficient matrix 183. The information processing device 100 also obtains a right-hand side vector 184 that pairs with the coefficient matrix 183.

[0117] The information processing device 100 generates an initial solution vector 187. The hypergraph neural network 130 receives the right-hand side vector 184, the connection matrix 185, the weights 186, and the initial solution vector 187, and outputs the solution vector 188 and the residual vector 189. The information processing device 100 updates the parameter values ​​of the hypergraph neural network 130 using the residual vector 189 and feeds the solution vector 188 back into the input of the hypergraph neural network 130. The information processing device 100 outputs the solution vector 188 after repeating the above iterations.

[0118] Figure 16 shows an example of a finite element method simulation. CAD model 219 is a three-dimensional CAD model used for structural analysis using the finite element method. In this finite element method simulation, a force is applied from outside the object in a specific direction as a boundary condition. Therefore, there may be methodological differences in the stress and displacement amounts at multiple nodes, and the coefficient matrix may be asymmetric.

[0119] As an example, the CAD model 219 has 2142 nodes. Since the physical quantities defined for each node are represented by three-dimensional vectors, the size of the coefficient matrix is ​​6426 × 6426. The hypergraph neural network that predicts the solution to this large system of linear equations includes 20 layers. According to the hypergraph neural network of the second embodiment, the final predicted value is calculated with an accuracy of cosine distance (error) of 0.054. Next, the functions and processing procedures of the information processing device 100 will be described.

[0120] Figure 17 is a block diagram showing an example of the functions of an information processing device according to a second embodiment. The information processing device 100 includes an equation storage unit 121, a model storage unit 122, a training unit 123, and a prediction unit 124. The equation storage unit 121 and the model storage unit 122 are implemented using, for example, RAM 102 or HDD 103. The training unit 123 and the prediction unit 124 are implemented using, for example, a CPU 101, a GPU 104, and a program.

[0121] The equation storage unit 121 stores one or more samples of systems of linear equations used as training data. The equation storage unit 121 also stores the system of linear equations to be predicted. The system of linear equations is stored in the form of a coefficient matrix and a right-hand side vector. The model storage unit 122 stores the trained hypergraph neural network. The trained hypergraph neural network includes the trained parameter matrix.

[0122] The training unit 123 trains the hypergraph neural network using training data. As described above, the training unit 123 generates a hypergraph from a sample of a system of linear equations. The training unit 123 generates initial values ​​for the node feature vectors and calculates the transformed node feature vectors and node residual vectors according to the hypergraph. The training unit 123 evaluates the error from the node residual vectors and updates the parameter values ​​of the hypergraph neural network using backpropagation to minimize the error. The training unit 123 saves the parameter values ​​after repeating the above iterations.

[0123] The prediction unit 124 applies the system of linear equations to be predicted to the trained hypergraph neural network and predicts the solution to the system of linear equations. As described above, the prediction unit 124 generates a hypergraph from the system of linear equations to be predicted. The prediction unit 124 generates initial values ​​for the node feature vectors and calculates the transformed node feature vectors and node residual vectors according to the hypergraph. The prediction unit 124 evaluates the error from the node residual vectors and adjusts the parameter values ​​using backpropagation to reduce the error. The prediction unit 124 outputs the node feature vectors after repeating the above iterations.

[0124] The training unit 123 may display the trained hypergraph neural network on the display device 111 or transmit it to another information processing device. The prediction unit 124 may store the predicted solution values ​​in non-volatile storage, display them on the display device 111, or transmit them to another information processing device.

[0125] Figure 18 is a flowchart illustrating an example of the model training procedure. In step S10, the training unit 123 initializes the parameter values ​​of the machine learning model. In step S11, the training unit 123 generates a hypergraph from the coefficient matrix of the system of linear equations. The nodes of the hypergraph correspond to the variables of the system of linear equations. The edges of the hypergraph correspond to the linear equations of the system of linear equations. The hypergraph is represented by a connection matrix.

[0126] In step S12, the training unit 123 selects initial values ​​for the node features for the nodes included in the hypergraph. These initial values ​​correspond to the initial values ​​of the variables. In step S13, the training unit 123 transforms the node features using the parameter values ​​and the nonlinear activation function. The transformed node features correspond to the improved predicted values ​​of the variables.

[0127] In step S14, the training unit 123 calculates edge residuals from node features according to the hypergraph. The edge residual of a given edge is calculated from the node features of one or more nodes included in that edge. In calculating the edge residuals, the coefficients included in the coefficient matrix are used as weights for the node features. In addition, constants included in the right-hand side vector are subtracted from the weighted sum of the node features.

[0128] In step S15, the training unit 123 calculates node residuals from edge residuals according to the hypergraph. The node residual of a given node is calculated from the edge residuals of one or more edges that include that node. In step S16, the training unit 123 calculates the error from the node residuals of multiple nodes using a loss function. In step S17, the training unit 123 updates the parameter values ​​of the machine learning model using backpropagation.

[0129] In step S18, the training unit 123 replaces the node features to be input to the machine learning model in the next iteration with the transformed node features from step S13. In step S19, the training unit 123 determines whether the stopping condition is met. The stopping condition may be, for example, that the number of iterations from steps S13 to S18 has reached a threshold, or that the error in step S16 has decreased to below a threshold. If the stopping condition is met, the process proceeds to step S20; otherwise, the process returns to step S13.

[0130] In step S20, the training unit 123 saves the trained machine learning model. If the training data contains two or more systems of linear equations, the training unit 123 may sequentially perform the processes in steps S11 to S19 for different systems of linear equations.

[0131] Figure 19 is a flowchart showing an example of the solution prediction procedure. In step S30, the prediction unit 124 reads out a machine learning model with trained parameter values. In step S31, the prediction unit 124 generates a hypergraph from the coefficient matrix of the system of linear equations. In step S32, the prediction unit 124 selects initial values ​​for node features for the nodes included in the hypergraph.

[0132] In step S33, the prediction unit 124 transforms the node features using the parameter values ​​and the nonlinear activation function. In step S34, the prediction unit 124 calculates edge residuals from the node features according to the hypergraph. In step S35, the prediction unit 124 calculates node residuals from the edge residuals according to the hypergraph.

[0133] In step S36, the prediction unit 124 calculates the error from the node residuals of multiple nodes using a loss function. In step S37, the prediction unit 124 updates the parameter values ​​of the machine learning model using backpropagation. In step S38, the prediction unit 124 replaces the node features to be input to the machine learning model in the next iteration with the transformed node features from step S33.

[0134] In step S39, the prediction unit 124 determines whether the stopping condition is met. The stopping condition may be, for example, that the number of iterations in steps S33 to S38 has reached a threshold, or that the error in step S36 has decreased to below a threshold. If the stopping condition is met, the process proceeds to step S40; otherwise, the process returns to step S33. In step S40, the prediction unit 124 outputs a solution vector, which is a sequence of the latest node features, as the predicted value of the solution to the system of linear equations.

[0135] As described above, the information processing device 100 of the second embodiment predicts the solution to a system of linear equations using a machine learning model as a surrogate model. This allows the information processing device 100 to speed up solution searching compared to using an iterative linear solver. Furthermore, the information processing device 100 can rapidly perform finite element method simulations using large-scale systems of linear equations, such as structural analysis and computational fluid dynamics.

[0136] Furthermore, the information processing device 100 represents the coefficient matrix as a hypergraph and predicts the solution to the system of linear equations using a hypergraph neural network that performs convolutional processing of node features on the hypergraph. As a result, even if the coefficient matrix is ​​asymmetric, the same hypergraph can be generated from other coefficient matrices by swapping the rows of the coefficient matrix, and the machine learning model has row order invariance. Therefore, the stability of solution search is improved and the prediction accuracy is improved.

[0137] Furthermore, unlike iterative linear solvers, the order of the rows in the coefficient matrix does not affect computation time in hypergraph neural networks. Therefore, the information processing device 100 does not need to perform preprocessing to optimize the row order. Also, the training data for training the machine learning model does not need to include labels indicating the correct solution to the system of linear equations. This reduces the burden of creating training data. In addition, the information processing device 100 adjusts the parameter values ​​of the machine learning model even in the prediction phase. Therefore, the information processing device 100 can flexibly adapt the machine learning model to various systems of linear equations.

[0138] Furthermore, the machine learning model uses a multilayer neural network with fully connected layers and nonlinear activation layers arranged alternately to transform the variable values ​​so that the residuals approach zero. As a result, the information processing device 100 can quickly and stably bring the variable values ​​closer to the true values, regardless of the initial values, by repeatedly transforming the variable values ​​and evaluating the residuals. [Explanation of Symbols]

[0139] 10 Information Processing Devices 11 Storage section 12 Processing Units 13 Coefficient Matrix 14 Hypergraph 15 Machine Learning Models

Claims

1. Obtain the coefficient matrix showing the coefficients included in the system of linear equations, Based on the coefficient matrix, the system includes multiple nodes corresponding to multiple variables in the system of linear equations and multiple edges corresponding to multiple linear expressions in the system of linear equations, and each of the multiple edges generates a hypergraph that connects nodes among the multiple nodes corresponding to the positions of non-zero coefficients in the coefficient matrix. Using a machine learning model that includes parameter values, multiple first features associated with the multiple nodes are converted into multiple second features. Based on the connection relationships between the multiple nodes indicated by the multiple edges, a plurality of third feature quantities corresponding to the plurality of nodes are calculated from the plurality of second feature quantities. The parameter value is updated based on the error determined from the plurality of third features. A machine learning program that instructs a computer to perform a task.

2. The plurality of first features indicate the initial values ​​of the plurality of variables, the plurality of second features indicate the predicted values ​​of the solutions to the system of linear equations, and the plurality of third features indicate the residuals of the system of linear equations under the predicted values. The machine learning program according to claim 1.

3. The machine learning model includes a fully connected layer having the parameter values ​​and a nonlinear activation layer that transforms the output of the fully connected layer using a nonlinear function. The machine learning program according to claim 1.

4. The calculation process includes calculating a plurality of fourth feature quantities associated with the plurality of edges from the non-zero coefficient and the plurality of second feature quantities, and calculating the plurality of third feature quantities from the plurality of fourth feature quantities. The machine learning program according to claim 1.

5. The aforementioned plurality of fourth features are calculated by subtracting the constant term included in the system of linear equations from a weighted sum obtained by weighting the plurality of second features with non-zero coefficients. The machine learning program according to claim 4.

6. If the update process does not satisfy the termination condition, the computer is further instructed to perform a process to further transform the plurality of second features using the machine learning model that includes the updated parameter values. The machine learning program according to claim 1.

7. If the updating process satisfies the termination condition, the computer is further instructed to output the plurality of second feature quantities as predicted values ​​of the solutions to the system of linear equations. The machine learning program according to claim 6.

8. Obtain the coefficient matrix showing the coefficients included in the system of linear equations, Based on the coefficient matrix, the system includes multiple nodes corresponding to multiple variables in the system of linear equations and multiple edges corresponding to multiple linear expressions in the system of linear equations, and each of the multiple edges generates a hypergraph that connects nodes among the multiple nodes corresponding to the positions of non-zero coefficients in the coefficient matrix. Using a machine learning model that includes parameter values, multiple first features associated with the multiple nodes are converted into multiple second features. Based on the connection relationships between the multiple nodes indicated by the multiple edges, a plurality of third feature quantities corresponding to the plurality of nodes are calculated from the plurality of second feature quantities. The parameter value is updated based on the error determined from the plurality of third features. A machine learning method in which a computer performs a process.

9. A memory unit that stores a coefficient matrix showing the coefficients included in a system of linear equations, Based on the coefficient matrix, a processing unit generates a hypergraph that includes multiple nodes corresponding to multiple variables in the system of linear equations and multiple edges corresponding to multiple linear expressions in the system of linear equations, each of the multiple edges connecting to a node corresponding to the position of a non-zero coefficient in the coefficient matrix, and uses a machine learning model including parameter values ​​to convert multiple first features associated with the multiple nodes into multiple second features, calculates multiple third features associated with the multiple nodes from the multiple second features based on the connection relationships between the multiple nodes indicated by the multiple edges, and updates the parameter values ​​based on the error determined from the multiple third features. An information processing device having

Images