Method for generating training data, method for generating a trained model, training data generation program, trained model generation program, and training data generation device.

Abstract

The present invention provides a method for generating training data, a method for generating a trained model, a training data generation program, a trained model generation program, and a training data generation apparatus that can optimize the graph structure used as training data. [Solution] The training data generation method includes: a graph update step of updating at least a portion of first graph information, which includes nodes divided into constituent units and edges indicating the relationships between each node, to generate second graph information; a trained model update step of updating a trained model by retraining using the second graph information; a prediction value acquisition step of obtaining predicted values ​​of the ground truth data for the second graph information; a graph evaluation value calculation step of calculating graph evaluation values ​​for the second graph information based on the predicted values ​​and ground truth data; and a determination step of determining whether or not to repeat the re-evaluation step based on pre-set conditions.

Description

[Technical Field]

[0001] The present invention relates to a method for generating training data, a method for generating a trained model, a training data generation program, a trained model generation program, and a training data generation apparatus. [Background technology]

[0002] In recent years, machine learning methods that handle graph information have attracted attention not only in the field of materials, but also in diverse fields such as search engines, social networking services (SNS), and transportation. This graph information is composed of nodes and edges, and by assigning attribute information to nodes and representing connection information between nodes via edges, it becomes possible to teach artificial intelligence (AI) complex structural information (see, for example, Patent Document 1). A common learning method using graph information is to aggregate information from surrounding nodes connected by edges. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2024-54431 [Overview of the project] [Problems that the invention aims to solve]

[0004] Incidentally, the graph structure in graph information is crucial for building highly accurate AI. However, conventional technologies rely on artificially set conditions for generating graph structures, and there was a need for technologies that could further optimize graph structures.

[0005] The present invention has been made in view of the above, and aims to provide a method for generating training data, a method for generating a trained model, a training data generation program, a trained model generation program, and a training data generation apparatus that can optimize the graph structure used as training data. [Means for solving the problem]

[0006] To solve the above-mentioned problems and achieve the objective, the present invention provides a method for generating training data for a computer to generate a trained model that estimates correct data, comprising: a graph update step to update the first graph information by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information using a predetermined algorithm, thereby generating a second graph information for a trained model generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and correct data of a structure based on each graph information; and the second graph information The method includes: a trained model update step that updates the trained model by retraining using the report and the ground truth data of the second graph information; a prediction value acquisition step that inputs the second graph information into the updated trained model and obtains a predicted value of the ground truth data of the second graph information; a graph evaluation value calculation step that calculates a graph evaluation value of the second graph information based on the predicted value obtained in the prediction value acquisition step and the ground truth data of the second graph information; and a determination step that determines whether or not to repeat a re-evaluation step consisting of the graph update step, the trained model update step, the prediction value acquisition step and the graph evaluation value calculation step, based on a set of conditions.

[0007] Furthermore, the learning data generation method according to the present invention further includes, in the above invention, an extraction step of extracting the plurality of graph information and the plurality of correct answer data corresponding to each of the plurality of graph information by reading them from a storage unit; a division step of dividing each of the plurality of graph information into nodes in terms of constituent elements; and an initial graph generation step of generating edges connecting the nodes and generating an initial graph.

[0008] Furthermore, in the learning data generation method according to the present invention, the graph update step includes an in-subgraph update step which updates each subgraph by adding and / or deleting edges and / or nodes in at least two subgraphs obtained by dividing the graph shown by the first graph information, and an inter-subgraph update step which updates the first graph information by adding and / or deleting edges and / or nodes between the subgraphs.

[0009] Furthermore, in the learning data generation method according to the present invention, the determination step determines whether or not to repeat the re-evaluation step based on the first graph evaluation value calculated in the graph evaluation value calculation step and the second graph evaluation value calculated in the graph evaluation value calculation step by the re-evaluation step.

[0010] Furthermore, in the learning data generation method according to the present invention, the judgment step involves comparing the amount of change in a plurality of index values ​​obtained by repeating the re-evaluation step, which are index values ​​calculated using the first graph evaluation value and the second graph evaluation value, with a preset threshold, and stopping the re-evaluation step according to the comparison result.

[0011] Furthermore, in the learning data generation method according to the present invention, the determination step determines whether or not to repeat the re-evaluation step based on the degree of change of the second graph information.

[0012] Furthermore, in the learning data generation method according to the present invention, the determination step determines whether or not to repeat the re-evaluation step based on the number of times the re-evaluation step is repeated.

[0013] Furthermore, in the learning data generation method according to the present invention, the graph update step generates a plurality of updated graph information to be used as candidates for learning data, the re-evaluation step is performed for each updated graph information, and the determination step determines whether or not to repeat the re-evaluation step for each updated graph information.

[0014] Furthermore, the trained model generation method according to the present invention is a trained model generation method in which a computer generates a trained model that estimates correct data, and the trained model is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and correct data of a structure based on each graph information, and updates the first graph information by adding and / or deleting the edges and / or nodes to at least a portion of the plurality of graph information to generate second graph information, and trained model update step updates the trained model by retraining using the second graph information and the correct data of the second graph information, and the second graph information The method includes: a prediction value acquisition step of inputting the information into the updated trained model to obtain predicted values ​​of the ground truth data of the second graph information; a graph evaluation value calculation step of calculating a graph evaluation value of the second graph information based on the predicted values ​​obtained in the prediction value acquisition step and the ground truth data of the second graph information; a determination step of determining whether or not to repeat a re-evaluation step consisting of the graph update step, the trained model update step, the prediction value acquisition step and the graph evaluation value calculation step, based on pre-set conditions; and a trained model setting step of setting the trained model generated in the trained model update step immediately preceding the decision to stop the re-evaluation step in the determination step as the trained model for estimating the ground truth data.

[0015] Furthermore, the training data generation program according to the present invention is a training data generation program that causes a computer to generate training data for generating a trained model that estimates correct data, and includes a graph update step which updates the first graph information by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information of the plurality of graph information, using a predetermined algorithm, to generate a trained model that is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and correct data of a structure based on each graph information, thereby generating a second graph information, and the second graph information The computer is instructed to perform the following steps: a trained model update step, which updates the trained model by retraining using the correct data; a prediction value acquisition step, which inputs the second graph information into the updated trained model and obtains predicted values ​​of the correct data for the second graph information; a graph evaluation value calculation step, which calculates a graph evaluation value for the second graph information based on the predicted values ​​obtained in the prediction value acquisition step and the correct data for the second graph information; and a determination step, which determines whether or not to repeat a re-evaluation step consisting of the graph update step, the trained model update step, the prediction value acquisition step and the graph evaluation value calculation step, based on pre-set conditions.

[0016] Furthermore, the trained model generation program according to the present invention is a trained model generation program that causes a computer to generate a trained model that estimates correct data, and includes a graph update step to update the first graph information by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information using a predetermined algorithm, thereby generating a second graph information for the trained model generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and correct data of structures based on each graph information, and the first graph information update step to update the trained model by retraining using the second graph information and the correct data of the second graph information, and the second graph information The computer is made to execute the following steps: a prediction value acquisition step, which inputs the updated trained model to obtain predicted values ​​of the ground truth data of the second graph information; a graph evaluation value calculation step, which calculates a graph evaluation value of the second graph information based on the predicted values ​​obtained in the prediction value acquisition step and the ground truth data of the second graph information; a determination step, which determines whether or not to repeat a re-evaluation step consisting of the graph update step, the trained model update step, the prediction value acquisition step and the graph evaluation value calculation step, based on pre-set conditions; and a trained model setting step, which sets the trained model generated in the trained model update step immediately preceding the decision to stop the re-evaluation step in the determination step as the trained model for estimating the ground truth data.

[0017] Further, a learning data generation device according to the present invention is a learning data generation device that generates learning data for generating a learned model that estimates correct data, and includes a plurality of graph information including nodes divided in units of components and edges indicating the relationships of each node, and correct data of a structure based on each graph information. For at least a part of the first graph information of the plurality of graph information, the graph update unit updates the first graph information by adding and / or deleting the edges and / or the nodes using a predetermined algorithm to generate second graph information; a learned model update unit that updates the learned model by re-learning using the second graph information and the correct data of the second graph information; a predicted value acquisition unit that inputs the second graph information into the updated learned model to obtain a predicted value of the correct data of the second graph information; a graph evaluation value calculation unit that calculates a graph evaluation value of the second graph information based on the predicted value acquired by the predicted value acquisition unit and the correct data of the second graph information; and a determination unit that determines whether to repeat a re-evaluation process including a graph update process, a learned model update process, a predicted value acquisition process, and a graph evaluation value calculation process as a set based on a preset condition.

Effect of the Invention

[0018] According to the present invention, the graph structure used as learning data can be optimized.

Brief Description of the Drawings

[0019] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an estimation system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing a configuration of a learning device included in an estimation system according to an embodiment of the present invention. [Figure 3] FIG. 3 is a block diagram showing a configuration of an estimation device included in an estimation system according to an embodiment of the present invention. [Figure 4]FIG. 4 is a flowchart showing an outline of learning data generation processing performed by a learning device according to an embodiment of the present invention. [Figure 5] FIG. 5 is a diagram for explaining initial graph structure generation processing performed by a learning device according to an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram (part 1) for explaining an example of graph update processing performed by a learning device according to an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram (part 2) for explaining an example of graph update processing performed by a learning device according to an embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart showing an outline of learning data generation processing performed by a learning device according to Modification 1 of the present invention. [Figure 9] FIG. 9 is a flowchart showing an outline of learning data generation processing performed by a learning device according to Modification 2 of the present invention. [Figure 10] FIG. 10 is a diagram for explaining graph update processing performed by a learning device according to Modification 3 of the present invention. [Figure 11] FIG. 11 is a diagram (part 1) for explaining learning data generation processing performed by a learning device according to Modification 4 of the present invention. [Figure 12] FIG. 12 is a diagram (part 2) for explaining learning data generation processing performed by a learning device according to Modification 4 of the present invention. MODE FOR CARRYING OUT THE INVENTION

[0020] Hereinafter, embodiments of a learning data generation method, a learned model generation method, a learning data generation program, a learned model generation program, and a learning data generation device according to the present invention will be described in detail based on the drawings. Note that the present invention is not limited by this embodiment. In addition, the individual embodiments of the present invention are not independent, and can be appropriately implemented in combination.

[0021] (Embodiment) [System Configuration] Figure 1 is a diagram showing the schematic configuration of an estimation system according to an embodiment of the present invention. The estimation system 1 shown in these figures comprises a learning device 2 that generates training data and uses the generated training data to train and generate a trained model, an estimation device 3 that uses the trained model generated by the learning device 2 to estimate a graph structure exhibiting predetermined characteristics, a display device 4 that displays information including the selection results of the estimation device 3, and an input device 5.

[0022] The graph structure estimated by the estimation device 3 consists of, for example, multiple nodes representing the raw materials, composition, and properties that constitute the material, as well as the characteristics of the material, and edges that connect the nodes. Here, each node is assigned one of the attributes such as material, composition, and properties, and is arranged in space according to the relationships between these attributes. Edges connect the nodes according to the relationships between them. Information that can be represented as a graph structure includes molecular structures, crystal structures, experimental data, plant operation data, citation relationships in papers, infrastructure-related data such as transportation networks and maps, e-commerce, social media, knowledge graphs, networks, databases, circuit structures, text, and images. Furthermore, characteristics include those for each node, those for each edge, and those corresponding to the entire graph structure. Specifically, these characteristics include numerical values ​​such as physical properties, categories, text such as descriptions and responses to queries, images, and the presence or absence of edges between nodes.

[0023] The learning device 2 is electrically connected to the estimation device 3. The learning device 2 generates and outputs a trained model by learning using training data. Figure 2 is a block diagram showing the configuration of the learning device in the estimation system according to an embodiment of the present invention. The learning device 2 has a training data generation unit 21, a learning unit 22, a control unit 23, and a storage unit 24. In this embodiment, the training data generation device is composed of at least the training data generation unit 21 and the storage unit 24.

[0024] The training data generation unit 21 generates training data using data stored in the storage unit 24 and data acquired from external sources. The training data is a dataset consisting of the graph structure described above and the characteristics shown by that graph structure. This dataset corresponds to graph information.

[0025] The learning data generation unit 21 includes a predicted value acquisition unit 211, a graph evaluation value calculation unit 212, a graph update unit 213, a model update unit 214, and a decision unit 215.

[0026] The prediction value acquisition unit 211 acquires predicted values ​​of the characteristics shown by the graph structure using a trained model.

[0027] The graph evaluation value calculation unit 212 calculates a graph evaluation value using the predicted value and the characteristics of the graph structure (ground truth data) obtained in advance. The graph evaluation value is, for example, the Mean Squared Error (MSE), the Mean Absolute Error, and the coefficient of determination (R). 2 ), precision, accuracy, recall, F1 score, AUC (Area Under the Curve), BLEU score, ROUGE score, perplexity, mAP (mean Average Precision), and IoU (Intersection over Union) can be used.

[0028] The graph update unit 213 updates the graph structure represented by nodes and edges by adding, deleting, rearranging nodes, and adding and deleting edges. The graph update unit 213 updates the graph structure using methods such as random search, grid search, greedy algorithm, branch-and-bound method, gradient descent, particle swarm optimization, Monte Carlo method, Bayesian optimization, genetic algorithm, evolutionary computation, reinforcement learning, and adversarial learning. Here, from the viewpoint of efficiently searching for the optimal solution, it is preferable to use gradient descent, particle swarm optimization, Monte Carlo method, Bayesian optimization, genetic algorithm, evolutionary computation, reinforcement learning, and adversarial learning for updating the graph structure.

[0029] The model update unit 214 updates the trained model by learning using graph information, including the updated graph structure. The model update unit 214 supports graph neural networks (graph convolutional neural networks (GCN), graph attention networks (GAT), gated graph neural networks (GGNN), graph isomorphism networks, GraphSAGE, message passing neural networks (MPNN), DeepWalk, Node2Vec, graph transformers, graph diffusion models, etc.), neural networks (feedforward neural networks (FFNN), convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory (LSTM), gated recurrent units (GRU), autoencoders (AE), variational autoencoders (VAE), generative adversarial networks (GAN), diffusion models, attention mechanisms, transformers, diffusion models, etc.), support vector machines, decision trees, random forests, gradient boosting trees and other boosting models, Bayesian estimation models, linear multiple regression, Ridge, LASSO, Elastic The trained model is updated using known methods such as Net, Partial Least Squares (PLS), KernelRidge regression, and Gaussian process regression. Here, for updating the model, it is preferable to use GCN, GAT, GGNN, Graph Isomorphism Network, GraphSAGE, MPNN, DeepWalk, Node2Vec, Graph Transformer, or Graph Diffusion Model, as these effectively capture the dependencies between nodes in the graph structure.

[0030] The decision unit 215 determines whether or not to repeat the update process of the graph and the trained model based on pre-set conditions.

[0031] The learning unit 22 generates a trained model by performing training using the training data generated by the training data generation unit 21. For example, it generates a trained model by learning graph information about the initial model structure read from the memory unit 24 or the like. Here, the trained model is a multilayer neural network consisting of an input layer, an intermediate layer, and an output layer, with each layer having one or more nodes. The trained model is generated by performing training. Information such as network parameters in the trained model is stored in the memory unit 24. Network parameters include information about the weights and biases between layers of the neural network.

[0032] The learning unit 22 can employ known learning methods. For example, when generating a trained model using regularization, multiple candidate values ​​for the regression model's hyperparameters are provided, and learning is performed for each of the given candidate hyperparameter values. Subsequently, the prediction error is calculated for the model obtained by learning with each candidate value using cross-validation or holdout validation with the training data, and the regression model that gives the smallest prediction error is selected. The selected regression model is output as the trained model. Here, hyperparameters refer to, for example, the number of layers in the neural network or the regularization coefficients.

[0033] The learning unit 22 generates a trained model by learning the graph structure, consisting of nodes and edges, contained in the graph information, as explanatory variables, and the characteristics exhibited by the graph structure as the target variable.

[0034] The control unit 23 comprehensively controls the operation of the learning device 2.

[0035] The memory unit 24 stores data including various programs for operating the learning device 2, and various parameters necessary for the operation of the learning device 2. The various programs include a learning data generation program that generates training data for generating a trained model, and a trained model generation program that uses the training data to train and generate a trained model. The memory unit 24 also has a dataset storage unit 241 that stores multiple datasets that constitute the training data.

[0036] The memory unit 24 is composed of a ROM (Read Only Memory) on which various programs are pre-installed, and RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., which store calculation parameters and data for each process.

[0037] Various programs can be recorded on computer-readable recording media such as HDDs, flash memory, CD-ROMs, DVD-ROMs, and Blu-ray® discs and widely distributed. The communication network referred to here is configured using existing public telephone networks, LANs (Local Area Networks), WANs (Wide Area Networks), etc., and can be wired or wireless.

[0038] The learning device 2 having the above functional configuration is a computer composed of one or more hardware components such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array).

[0039] The estimation device 3 is electrically connected to the learning device 2 and the display device 4. The estimation device 3 uses the characteristics to be estimated and the trained model acquired from the learning device 2 to output a graph structure representing the characteristics to be estimated as the estimation result. Figure 3 is a block diagram showing the configuration of the estimation device in the estimation system according to an embodiment of the present invention. The estimation device 3 has a calculation unit 31, a control unit 32, and a storage unit 33.

[0040] The calculation unit 31 calculates a graph structure that represents the characteristics to be estimated using the trained model.

[0041] The control unit 32 comprehensively controls the operation of the estimation device 3. The control unit 32 includes a display control unit 321 that displays the estimation results from the calculation unit 31 on the display device 4. In addition to the estimation results, the display control unit 321 may also display the characteristics of the object to be estimated on the display device 4.

[0042] The memory unit 33 stores data including various programs for operating the estimation device 3, and various parameters necessary for the operation of the estimation device 3. The various programs also include physical property estimation programs executed using trained models. The memory unit 33 is composed of a ROM with various programs pre-installed, and RAM, HDD, SSD, etc., for storing calculation parameters and data for each process.

[0043] Various programs can be recorded on computer-readable recording media such as HDDs, flash memory, CD-ROMs, DVD-ROMs, and Blu-ray® discs and widely distributed. Furthermore, the estimation device 3 can acquire various programs via a communication network. This communication network can be, for example, an existing public network, LAN, or WAN, and can be wired or wireless.

[0044] The estimation device 3, having the above functional configuration, is a computer composed of one or more hardware components such as a CPU, GPU, ASIC, and FPGA.

[0045] The display device 4 is a display made of liquid crystal or organic EL (Electro-Luminescence), and is electrically connected to the estimation device 3. The display device 4 acquires and displays display data output from the estimation device 3 under the control of the display control unit 321. The display device 4 may also have an audio output function such as a speaker.

[0046] Input device 5 accepts various types of information, including settings related to the process of estimating the graph structure, and outputs the received information to learning device 2 and estimation device 3. Input device 5 is configured using a user interface such as a keyboard, mouse, microphone, and touch panel.

[0047] In estimation system 1, first, the training data generation unit 21 generates multiple datasets (for example, initial graph information) as training data. Then, the learning unit 22 trains using the multiple datasets to generate a trained model. Furthermore, when the graph information is updated by the graph update unit 213, the trained model is updated by the model update unit 214. The calculation unit 31 inputs the graph structure into this trained model 100, calculates the estimation characteristics, and extracts a graph structure that exhibits the desired characteristics.

[0048] [Training data generation process] Figure 4 is a flowchart illustrating the overview of the learning data generation process performed by a learning device according to one embodiment of the present invention. First, the learning device 2 generates an initial graph structure as graph information (step S101). The learning data generation unit 21 generates an initial graph structure, for example, by reading a dataset stored in the storage unit 24 and generating an initial graph structure that includes the graph structure, or by setting a graph structure input via the input device 5 as the initial graph structure.

[0049] Figure 5 is a diagram illustrating the initial graph structure generation process performed by a learning device according to one embodiment of the present invention. The initial graph structure is generated using pre-set data such as composition. For example, if tabular data with material number, raw material (ethanol), composition (numerical value, unit), and properties (Yield: correct data) is pre-set, as shown in Figure 5(a), it is first divided into nodes according to each component (see Figure 5(b)). At this time, it is divided into subgraphs according to the relationships between the components. In the example shown in Figure 5(b), it is divided into a first subgraph SG1 composed of nodes N1 to N3 that represent material and composition, and a second subgraph SG2 composed of nodes N4 to N6 that represent things other than material and composition (e.g., properties). Then, for example, an initial graph structure is generated in which the nodes are connected by edges E1 to E5 according to user instructions (see Figure 5(c)).

[0050] Once the initial graph structure is generated, the control unit 23 uses this initial graph structure to generate a trained model (step S102). For example, the control unit 23 instructs the learning unit 22 to generate a trained model by learning using the initial graph structure.

[0051] Then, the graph update unit 213 performs a graph structure update process on the initial graph structure (step S103). The graph update unit 213 updates the graph structure using, for example, instructions input via the input device 5 and the algorithm described above. In this case, if there are multiple initial graph structures, the graph update unit 213 performs the update process on at least some of the graph structures, such as the graph structure specified by the user or the initial graph structure set in advance.

[0052] Figures 6 and 7 illustrate an example of graph update processing performed by a learning device according to one embodiment of the present invention. The graph update unit 213 adds or deletes nodes or edges using the algorithm described above. For example, in the example shown in Figure 6, within the first subgraph SG1, edge E1 is deleted, and edge E11 is added, and within the second sub-graph SG2, node N 11 , as well as the edge E connecting nodes N4 and N 11 are added. 12 Node N5 and N 11 are connected by edge E 13 has been added. Also, in the example shown in FIG. 7, between the first sub-graph SG1 and the second sub-graph SG2, node N 12 is added, and the edge E connecting nodes N1 and N 12 is added. 14 Nodes N4 and N 12 are connected by E 15 has been added. In this embodiment, as shown in FIGS. 6 and 7, nodes and edges are changed either within or between sub-graphs. In addition, the weights of nodes and edges may be adjusted. For example, penalties may be imposed on the weights of specific nodes by regularization. Also, domain knowledge for each field may be reflected when updating the graph structure. For example, weights can be set based on domain knowledge for the update probability of graphs and edges. Furthermore, when updating the graph structure using a learned model, multiple learned models may be used, different models may be applied for each sub-graph, and learning may be performed collectively during learning. Note that text data is preferably vectorized using a language model from the perspective of suppressing notation variations. At this time, after the text data is vectorized, a group of similar texts is determined based on the vector, and the group of texts is treated as the same vector. Also, for the updated graph structure, correct data for the estimation target is calculated. This updated graph structure corresponds to the first graph information.

[0053] After the graph is updated, the model update unit 214 updates the trained model using the updated graph structure (step S104). The model update unit 214 updates the trained model, which has been retrained using the updated graph structure, using the algorithm described above.

[0054] When the trained model is updated, this trained model is used to calculate the predicted values ​​for the updated graph structure (step S105).

[0055] Then, the graph evaluation value calculation unit 212 calculates the graph evaluation value of the updated graph structure relative to the correct data of the predicted value calculated in step S105 (step S106). The graph evaluation value calculation unit 212 calculates, for example, the MSE.

[0056] After calculating the graph evaluation value, the determination unit 215 calculates the rate of change of the graph evaluation value and determines whether the rate of change is below a threshold (step S107). The determination unit 215 reads the graph evaluation value calculated in the initial graph update process from, for example, the storage unit 24, and calculates the rate of change using the latest graph evaluation value and the graph evaluation value immediately preceding it. The rate of change may be a ratio, MSE, or difference. The threshold is set according to the type of rate of change to be calculated.

[0057] If the determination unit 215 determines that the rate of change is below a threshold (step S107: Yes), the control unit 23 proceeds to step S109. Conversely, if the determination unit 215 determines that the rate of change is greater than a threshold (step S107: No), the control unit 23 proceeds to step S108. In the initial update process, there is no fluctuation rate because there is only one graph evaluation value. In this case, the control unit 23 proceeds to step S108.

[0058] In step S108, the graph update unit 213 updates the graph structure in the same manner as in step S103. After updating the graph, the control unit 23 proceeds to step S104 and repeats the processing from step S104 onward for the graph structure updated in step S108. At this time, the graph structure before the update by the graph update unit 213 becomes the first graph information.

[0059] Furthermore, in step S109, the control unit 23 sets a graph structure in which the rate of change is below a threshold as training data and outputs the training data. The output destination in this case is, for example, the storage unit 24, an external server, or the display device 4.

[0060] In this way, training data is generated. Here, the trained model generated by this training data generation process may be newly generated by the training unit 22 training using this training data, or it may be the latest trained model updated in step S104.

[0061] Furthermore, in the estimation device 3, the calculation unit 31 uses the training data and trained model generated as described above to calculate a graph structure that represents the characteristics to be estimated. For example, the calculation unit 31 inputs several candidate graph structures into the trained model and obtains the predicted characteristics for each graph structure. From these, it extracts the graph structure that has the desired characteristics and sets it as the optimal predicted graph structure.

[0062] The graph structure calculated by the calculation unit 31 is then output to the display device 4 by the display control unit 321, or output to and stored in the storage unit 33. At this time, the display device 4 displays the estimation result.

[0063] In the embodiments described above, the graph structure is updated using a predetermined algorithm to improve prediction accuracy compared to the ground truth data, and estimation processing is performed using a trained model that employs a graph structure with guaranteed prediction accuracy as training data. This allows for the optimization of the graph structure used as training data.

[0064] Furthermore, graph structures are considered excellent for multimodal modeling because they can process data from various modalities, such as numerical data, text, images, audio, and chemical structures, allowing for the incorporation of diverse information to build highly accurate machine learning models. According to this embodiment, a trained model is generated using training data optimized for such a graph structure, making it possible to predict characteristics with high accuracy.

[0065] In this embodiment, the correct answer data may be calculated using analytical values ​​in addition to measured values.

[0066] (Variation 1) Next, a modification 1 of this embodiment will be described. The estimation system according to this modification 1 is the same as the estimation system 1 according to the embodiment, so its description will be omitted. The following describes the differences from the embodiment. In this modification 1, the manner in which the training data generation process is stopped differs from the embodiment described above.

[0067] In this modified example 1, the learning data generation unit 21 generates learning data using data stored in the storage unit 24 and data acquired from an external source. Figure 8 is a flowchart showing an overview of the learning data generation process performed by the learning device according to this modified example 1. In this modified example 1, the learning device 2 generates an initial graph and calculates graph evaluation values ​​(steps S201 to S206) in the same manner as steps S101 to S106 shown in Figure 4.

[0068] After calculating the graph evaluation value, the judgment unit 215 determines whether the graph evaluation value is below a threshold (step S207). The threshold is set according to the desired prediction accuracy.

[0069] If the determination unit 215 determines that the graph evaluation value is below the threshold (step S207: Yes), the control unit 23 proceeds to step S211. Conversely, if the determination unit 215 determines that the graph evaluation value is greater than the threshold (step S207: No), the control unit 23 proceeds to step S208.

[0070] In step S208, the determination unit 215 calculates the degree of change in the graph structure. The determination unit 215 uses a known method to calculate the degree of change between the updated graph structure and the graph structure immediately before it.

[0071] The determination unit 215 then determines whether the degree of change in the graph structure is below a predetermined threshold (step S209). If the determination unit 215 determines that the degree of change in the graph structure is below the threshold (step S209: Yes), the control unit 23 proceeds to step S211. Conversely, if the determination unit 215 determines that the degree of change in the graph structure is greater than the threshold (step S209: No), the control unit 23 proceeds to step S210.

[0072] In step S210, the graph update unit 213 updates the graph structure in the same manner as in step S102. After updating the graph, the control unit 23 proceeds to step S204 and repeats the processing from step S204 onward for the graph structure updated in step S210.

[0073] Furthermore, in step S211, the control unit 23 sets a graph structure whose degree of change is below a threshold as training data, and outputs the training data. The output destination at this time is, for example, the storage unit 24, an external server, or the display device 4.

[0074] In the first modified example, training data is generated as described above. Here, the trained model generated by this training data generation process may be newly generated by the training unit 22 training using this training data, or it may be the latest trained model updated in step S204.

[0075] Furthermore, in the estimation device 3, the calculation unit 31 uses the training data and trained model generated as described above to calculate a graph structure that represents the characteristics to be estimated.

[0076] The graph structure calculated by the calculation unit 31 is then output to the display device 4 by the display control unit 321, or output to and stored in the storage unit 33. At this time, the display device 4 displays the estimation result.

[0077] In the modified example 1 described above, similar to the embodiment, the graph structure is updated using a predetermined algorithm to improve prediction accuracy compared to the ground truth data, and estimation processing is performed using a trained model that employs a graph structure with guaranteed prediction accuracy as training data. This allows for the optimization of the graph structure used as training data.

[0078] In Modification Example 1, an example was described in which the termination of the process is determined using the degree of change in the graph structure, but the termination of the process may also be determined using the degree of change in the graph evaluation value.

[0079] (Modification 2) Next, a modified example 2 of this embodiment will be described. The estimation system in this modified example 2 is the same as the estimation system 1 in the embodiment, so its description will be omitted. The following describes the differences from the embodiment. In this modified example 2, the manner in which the training data generation process is stopped differs from the embodiment described above.

[0080] In this modified example 2, the learning data generation unit 21 generates learning data using data stored in the storage unit 24 and data acquired from an external source. Figure 9 is a flowchart showing an overview of the learning data generation process performed by the learning device according to this modified example 2. In this modified example 2, the learning device 2 generates an initial graph in the same manner as steps S201 to S207 shown in Figure 8, and determines whether the graph evaluation value is below a threshold (steps S301 to S307). The control unit 23 proceeds to step S310 if the determination unit 215 determines that the graph evaluation value is below a threshold (step S307: Yes). Conversely, the control unit 23 proceeds to step S308 if the determination unit 215 determines that the graph evaluation value is greater than a threshold (step S307: No).

[0081] In step S308, the determination unit 215 determines whether the number of iterations of updating the graph structure is less than or equal to a preset threshold. If the determination unit 215 determines that the degree of change in the graph structure is less than or equal to the threshold (step S308: Yes), the control unit 23 proceeds to step S309. Conversely, if the determination unit 215 determines that the degree of change in the graph structure is greater than the threshold (step S308: No), the control unit 23 proceeds to step S310.

[0082] In step S309, the graph update unit 213 updates the graph structure in the same manner as in step S102. After updating the graph, the control unit 23 proceeds to step S304 and repeats the processing from step S304 onward for the graph structure updated in step S309.

[0083] Furthermore, in step S310, the control unit 23 sets graph structures whose degree of change is below a threshold as training data, and outputs the training data. The output destination in this case is, for example, the storage unit 24, an external server, or the display device 4.

[0084] In the modified example 2, training data is generated as described above. Here, the trained model generated by this training data generation process may be newly generated by the training unit 22 training using this training data, or it may be the latest trained model updated in step S304.

[0085] Furthermore, in the estimation device 3, the calculation unit 31 uses the training data and trained model generated as described above to calculate a graph structure that represents the characteristics to be estimated.

[0086] The graph structure calculated by the calculation unit 31 is then output to the display device 4 by the display control unit 321, or output to and stored in the storage unit 33. At this time, the display device 4 displays the estimation result.

[0087] In the modified example 2 described above, similar to the embodiment, the graph structure is updated using a predetermined algorithm to improve prediction accuracy compared to the ground truth data, and estimation processing is performed using a trained model that employs a graph structure with guaranteed prediction accuracy as training data. This allows for the optimization of the graph structure used as training data.

[0088] (Variation 3) Next, a third modification of this embodiment will be described. The estimation system in this third modification is the same as the estimation system 1 in the embodiment, so its description will be omitted. The following describes the differences from the embodiment. In this third modification, the update method of the training data generation process is different from that of the embodiment described above.

[0089] In this modified example 3, the graph structure update process in step S102 or S108 shown in Figure 4 modifies nodes and edges within and between subgraphs. Figure 10 is a diagram illustrating the graph update process performed by a learning device according to Modification 3 of the present invention. The graph update unit 213, for example, adds node N1, edges E3 and E4 within the second subgraph SG2, then adds node N2 between the first subgraph SG1 and the second subgraph SG2 (intra-subgraph update step), and adds edges E5 and E6 that connect node N2 to the nodes of the first subgraph SG1 and the nodes of the second subgraph SG2 (inter-subgraph update step). Note that the update process within a subgraph and the update process between subgraphs can be executed in any order, or they can be executed simultaneously. The other processes described above are the same as those shown in Figure 4.

[0090] In the third modified example, training data is generated as described above. Here, the trained model generated by this training data generation process may be newly generated by the training unit 22 training using this training data, or it may be the latest trained model updated in step S303.

[0091] Furthermore, in the estimation device 3, the calculation unit 31 uses the training data and trained model generated as described above to calculate a graph structure that represents the characteristics to be estimated.

[0092] The graph structure calculated by the calculation unit 31 is then output to the display device 4 by the display control unit 321, or output to and stored in the storage unit 33. At this time, the display device 4 displays the estimation result.

[0093] Here, we will explain an example of the graph update process related to this modified example 3. First, we prepared an initial graph structure consisting of five subgraphs, each with 10 nodes. The X attribute of each node was a 5-dimensional vector randomly sampled from a normal distribution. The Y attribute (Yield: ground truth data) of each node was a weighted average of the X attribute vectors of neighboring nodes, weighted by randomly sampling from a standard normal distribution. We then created a dataset for a node regression problem with these attributes.

[0094] For this dataset, 50% of the edges were randomly removed, and the same number of new edges were generated to create a randomized graph structure (corresponding to step S301). A machine learning model using a graph convolutional neural network was trained on this randomized graph structure (corresponding to step S302), and the coefficient of determination was calculated to be 0.42.

[0095] Next, for the randomized graph structure, 10% of the edges were removed within each subgraph and between subgraphs, and new edges were added to generate 50 graph structures (corresponding to step S303). For each graph structure, a machine learning model using a graph convolutional neural network was trained to generate a trained model (corresponding to step S304). Subsequently, the coefficient of determination was calculated for each trained model (corresponding to steps S305-S306). Based on the top 5 graph structures with the highest coefficient of determination among the trained models, 50 updated graph structures were generated by removing 10% of the edges and adding new edges. For each updated graph structure, a machine learning model was trained, the trained model was updated, and the coefficient of determination was calculated. After repeating this process of updating the graph structures and trained models 30 times (corresponding to step S038), the final obtained coefficient of determination for the trained model was 0.67. Thus, it can be said that the graph structure can be optimized by performing the process described in this modified example 3.

[0096] In the modified example 3 described above, similar to the embodiment, the graph structure is updated using a predetermined algorithm to improve prediction accuracy compared to the ground truth data, and estimation processing is performed using a trained model that employs a graph structure with guaranteed prediction accuracy as training data. This allows for the optimization of the graph structure used as training data.

[0097] (Modification 4) Next, a modification 4 of this embodiment will be described. The estimation system according to this modification 4 is the same as the estimation system 1 according to the embodiment, so its description will be omitted. The following describes the differences from the embodiment. In this modification 4, the update method of the training data generation process is different from that of the embodiment described above.

[0098] In this modified example 4, the graph structure update process in step S102 or S107 shown in Figure 4 involves modifying nodes and edges within and between subgraphs. Figures 11 and 12 are diagrams illustrating the learning data generation process performed by the learning device according to Modification 4 of the present invention. The graph update unit 213 generates different graph structures, for example, as shown in Figures 11 and 12.

[0099] When the graph is updated, the model update unit 214 updates each trained model using the updated graph structure (step S103). Therefore, in this modified example 4, the model update unit 214 generates separately trained models for each updated graph structure using the algorithm described above.

[0100] When a trained model is updated, the predicted values ​​for the updated graph structure are calculated using each trained model (step S104).

[0101] Then, the graph evaluation value calculation unit 212 calculates a graph evaluation value for each of the predicted values ​​calculated in step S104 (step S105).

[0102] After calculating the graph evaluation value, the determination unit 215 calculates the rate of change of the graph evaluation value and determines whether the rate of change is below a threshold (step S106). At this time, the determination unit 215 selects the graph structure with the smaller graph evaluation value, calculates the rate of change of the graph evaluation value of that graph structure, and compares it with the threshold. Note that the graph structure selected at this time is not limited to one; for example, the graph structure shown in the figure below, which is set in advance, can be selected. If multiple selections are set, the processing after the rate of change judgment process will be executed for each graph structure, and an optimized trained model may be output for each, or the optimal trained model may be selected based on pre-set conditions.

[0103] After comparing the rates of change, the training data generation unit 21 executes the processing in step S107 and / or step S108.

[0104] In the modified example 4, training data is generated as described above. Here, the trained model generated by this training data generation process may be newly generated by the learning unit 22 learning using this training data, or it may be the latest trained model updated in step S303.

[0105] Furthermore, in the estimation device 3, the calculation unit 31 uses the training data and trained model generated as described above to calculate a graph structure that represents the characteristics to be estimated.

[0106] The graph structure calculated by the calculation unit 31 is then output to the display device 4 by the display control unit 321, or output to and stored in the storage unit 33. At this time, the display device 4 displays the estimation result.

[0107] In the modified example 4 described above, similar to the embodiment, the graph structure is updated using a predetermined algorithm to improve prediction accuracy compared to the ground truth data, and estimation processing is performed using a trained model that employs a graph structure with guaranteed prediction accuracy as training data. This allows for the optimization of the graph structure used as training data.

[0108] (Other embodiments) While embodiments for carrying out the present invention have been described so far, the present invention should not be limited to the embodiments described above. For example, the estimation device may also include the functions of a learning unit. In this case, the estimation device generates training data and sequentially updates the trained model.

[0109] Furthermore, in the embodiments and modifications described above, examples were given in which the update process is stopped when the rate of change of the graph evaluation value, the degree of change in the graph structure, or the number of iterations is below a threshold. However, for example, the coefficient of determination (R 2 When using ) as the determination, the process is terminated when it exceeds the threshold.

[0110] Furthermore, while the pre-trained model described above was an example where a graph structure was input and characteristics were output, it is also possible to generate a pre-trained model with characteristics as explanatory variables and the graph structure as the target variable, input the desired characteristics into the pre-trained model, and obtain the estimated graph structure.

[0111] Furthermore, machine learning is not limited to deep learning as described above; for example, support vector machines, decision trees, random forests, or gradient boosting trees may also be used. [Explanation of symbols]

[0112] 1 Estimation System 2 Learning device 3 Estimation device 4 Display device 5 Input devices 21. Training Data Generation Unit 22 Learning Department 23, 32 Control Unit 24, 33 Storage section 31 Calculation Section 211 Prediction Value Acquisition Unit 212 Graph Evaluation Value Calculation Unit 213 Graph Update Section 214 Model Update Section 215 Judgment Department 321 Display Control Unit

Claims

1. A method for generating training data for a computer to generate a trained model that estimates correct data, A graph update step to generate second graph information by updating a trained model, which is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and ground truth data of structures based on each graph information, by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information among the plurality of graph information using a predetermined algorithm, A trained model update step, which updates the trained model by retraining using the second graph information and the ground truth data of the second graph information, A prediction value acquisition step involves inputting the second graph information into the updated trained model to obtain the predicted values ​​of the ground truth data for the second graph information, A graph evaluation value calculation step calculates a graph evaluation value of the second graph information based on the predicted value obtained in the predicted value acquisition step and the ground truth data of the second graph information, A determination step that determines whether or not to repeat a re-evaluation step, which consists of the graph update step, the trained model update step, the predicted value acquisition step, and the graph evaluation value calculation step, based on pre-set conditions, A method for generating training data that includes [the specified element].

2. Extraction step of extracting by reading the aforementioned plurality of graph information and the plurality of correct answer data corresponding to each of the plurality of graph information from the storage unit, For each of the aforementioned multiple graph information, a division step is performed to divide it into nodes based on its constituent elements, An initial graph generation step generates edges connecting nodes and generates an initial graph, The method for generating training data according to claim 1, further comprising:

3. The aforementioned graph update step is: A subgraph update step updates each subgraph by adding and / or deleting edges and / or nodes within at least two subgraphs obtained by dividing the graph shown by the first graph information, A subgraph update step that updates the first graph information by adding and / or deleting edges and / or nodes between the subgraphs, A method for generating training data according to claim 1, including the following:

4. The determination step determines whether or not to repeat the re-evaluation step based on the first graph evaluation value calculated in the graph evaluation value calculation step and the second graph evaluation value calculated in the graph evaluation value calculation step by the re-evaluation step. The method for generating training data according to claim 1.

5. The aforementioned determination step involves determining an index value calculated using the first graph evaluation value and the second graph evaluation value, and stopping the re-evaluation step when the amount of change in the multiple index values ​​obtained by repeating the re-evaluation step falls below a predetermined threshold. The method for generating training data according to claim 4.

6. The aforementioned determination step determines whether or not to repeat the re-evaluation step based on the degree of change of the second graph information. The method for generating training data according to claim 1.

7. The determination step determines whether or not to repeat the re-evaluation step based on the number of times the re-evaluation step has been repeated. The method for generating training data according to claim 1.

8. The aforementioned graph update step generates multiple updated graph information to be used as candidate training data, For each updated graph information, perform the re-evaluation step described above. The aforementioned determination step determines whether or not to repeat the re-evaluation step for each updated graph information. The method for generating training data according to claim 1.

9. A method for generating a trained model in which a computer generates a trained model that estimates correct data, A graph update step to generate second graph information by updating a trained model, which is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and ground truth data of structures based on each graph information, by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information among the plurality of graph information using a predetermined algorithm, A trained model update step, which updates the trained model by retraining using the second graph information and the ground truth data of the second graph information, A prediction value acquisition step involves inputting the second graph information into the updated trained model to obtain the predicted values ​​of the ground truth data for the second graph information, A graph evaluation value calculation step calculates a graph evaluation value of the second graph information based on the predicted value obtained in the predicted value acquisition step and the ground truth data of the second graph information, A determination step that determines whether or not to repeat a re-evaluation step, which consists of the graph update step, the trained model update step, the predicted value acquisition step, and the graph evaluation value calculation step, based on pre-set conditions, A trained model setting step in which the trained model generated in the trained model update step immediately preceding the decision step in which it was determined to stop the re-evaluation step is set as the trained model for estimating the correct data, A method for generating a pre-trained model that includes this.

10. A training data generation program that causes a computer to generate training data for creating a trained model that estimates correct data, A graph update step to generate second graph information by updating a trained model, which is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and ground truth data of structures based on each graph information, by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information among the plurality of graph information using a predetermined algorithm, A trained model update step, which updates the trained model by retraining using the second graph information and the ground truth data of the second graph information, A prediction value acquisition step involves inputting the second graph information into the updated trained model to obtain the predicted values ​​of the ground truth data for the second graph information, A graph evaluation value calculation step calculates a graph evaluation value of the second graph information based on the predicted value obtained in the predicted value acquisition step and the ground truth data of the second graph information, A determination step that determines whether or not to repeat a re-evaluation step, which consists of the graph update step, the trained model update step, the predicted value acquisition step, and the graph evaluation value calculation step, based on pre-set conditions, A training data generation program that causes the aforementioned computer to execute.

11. A pre-trained model generation program that causes a computer to generate a pre-trained model for estimating correct data, A graph update step to generate second graph information by updating a trained model, which is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and ground truth data of structures based on each graph information, by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information among the plurality of graph information using a predetermined algorithm, A trained model update step, which updates the trained model by retraining using the second graph information and the ground truth data of the second graph information, A prediction value acquisition step involves inputting the second graph information into the updated trained model to obtain the predicted values ​​of the ground truth data for the second graph information, A graph evaluation value calculation step calculates a graph evaluation value of the second graph information based on the predicted value obtained in the predicted value acquisition step and the ground truth data of the second graph information, A determination step that determines whether or not to repeat a re-evaluation step, which consists of the graph update step, the trained model update step, the predicted value acquisition step, and the graph evaluation value calculation step, based on pre-set conditions, A trained model setting step in which the trained model generated in the trained model update step immediately preceding the decision step in which it was determined to stop the re-evaluation step is set as the trained model for estimating the correct data, A trained model generation program that causes the aforementioned computer to execute.

12. A training data generation device that generates training data for generating a trained model that estimates correct data, A graph update unit generates second graph information by updating a trained model, which is generated by learning a plurality of graph information including nodes divided into constituent units and edges indicating the relationships between each node, and ground truth data of structures based on each graph information, by adding and / or deleting the edges and / or nodes to at least a portion of the first graph information among the plurality of graph information using a predetermined algorithm. A trained model update unit updates the trained model by retraining using the second graph information and the correct data of the second graph information. A prediction value acquisition unit inputs the second graph information into the updated trained model to obtain the predicted value of the correct data for the second graph information, A graph evaluation value calculation unit calculates a graph evaluation value of the second graph information based on the predicted values ​​acquired by the predicted value acquisition unit and the correct data of the second graph information. A determination unit that determines whether or not to repeat a re-evaluation process consisting of a graph update process, a trained model update process, a predicted value acquisition process, and a graph evaluation value calculation process based on pre-set conditions, A training data generation device equipped with [the following features].

Images