Machine learning programs, machine learning methods, and information processing devices.

Abstract

The challenge is to reduce the computational cost when performing distributed learning with parallel data generation. [Solution] An information processing device that performs machine learning on a machine learning model that extracts molecular structure features using different molecules as multiple training data obtains the number of atoms contained in each of the multiple training data. If there is a first training data with a number of atoms greater than or equal to a threshold among the obtained number of atoms in each of the multiple training data, the first training data is replaced with another number of atoms that is less than the threshold, based on the other training data among the multiple training data.

Description

[Technical Field]

[0001] The present invention relates to a machine learning program, a machine learning method, and an information processing device. [Background technology]

[0002] In recent years, materials informatics (hereinafter sometimes referred to as MI) has been advancing globally as a data-driven research and development approach aimed at shortening research and development periods and reducing costs in materials development. MI relies heavily on the accumulation of high-quality materials data, and efficient data accumulation is expected through the use of simulation technologies such as computational chemistry and neural network potentials (hereinafter sometimes referred to as NNP), in addition to experiments.

[0003] The potential (interatomic potential) is a computational model that directly estimates the energy surface from atomic structure. In recent years, NNP has become one of the main methods for calculating potentials, using neural networks. Due to the enormous computational cost of training NNP using large-scale models and large-scale data aimed at improving the inference accuracy and generalization of NNP, distributed learning of NNP is being employed.

[0004] For distributed learning of NNPs, two methods are known: model parallelism, which divides a single model and deploys multiple processes within it, and data parallelism, which duplicates the same model and multiplies the batch size by a factor of two. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-181372 [Patent Document 2] Japanese Patent Publication No. 2024-2637 [Patent Document 3] Japanese Patent Publication No. 2023-117513 [Patent Document 4] U.S. Patent Application Publication No. 2024 / 0028889 [Patent Document 5] U.S. Patent Application Publication No. 2013 / 0290223 [Overview of the project] [Problems that the invention aims to solve]

[0006] However, when performing distributed learning of NNPs in data parallelism, computational costs can increase. For example, load balancing in distributed learning of NNPs becomes a partitioning problem (NP-complete problem) in which a given set of n training data is divided into sets of a number of processes. The combination of training data in which the sum of the number of atoms in each training data is uniform is then found as an optimization problem using methods such as greedy algorithms.

[0007] However, if some of the training data contains training data with an extremely large number of atoms, there is no combination of training data where the sum of the number of atoms in each process is uniform. As a result, some processes will have a large sum of atoms in their training data, leading to uneven load balancing. This can actually increase computational costs.

[0008] One aspect of this project is to provide machine learning programs, machine learning methods, and information processing devices that can reduce the computational cost when performing distributed learning in parallel with data. [Means for solving the problem]

[0009] In the first proposal, a machine learning program that causes a computer to perform machine learning on a machine learning model that extracts molecular structure features using multiple different molecules as training data, is characterized in that it obtains the number of atoms contained in the molecules shown in each of the multiple training data, and if there is a first training data with a number of atoms that is equal to or greater than a threshold among the obtained number of atoms of each of the multiple training data, it performs a process to replace the first training data with a number of atoms that is less than the threshold, based on the other training data among the multiple training data.

Advantages of the Invention

[0010] According to one embodiment, the computational cost in the case of distributed learning in data parallelism can be reduced.

Brief Description of the Drawings

[0011] [Figure 1] FIG. 1 is a diagram for explaining an information processing apparatus according to Example 1. [Figure 2] FIG. 2 is a diagram for explaining an example of non-uniform load distribution by the greedy method. [Figure 3] FIG. 3 is a diagram for explaining load distribution according to Example 1. [Figure 4] FIG. 4 is a functional block diagram showing the functional configuration of the information processing apparatus according to Example 1. [Figure 5] FIG. 5 is a diagram for explaining the setting of replacement conditions. [Figure 6] FIG. 6 is a diagram for explaining replacement processing. [Figure 7] FIG. 7 is a diagram for explaining a specific example 1 of replacement. [Figure 8] FIG. 8 is a diagram for explaining a specific example 2 of replacement. [Figure 9] FIG. 9 is a flowchart showing the overall flow of processing. [Figure 10] FIG. 10 is a flowchart showing the detailed flow of processing. [Figure 11] FIG. 11 is a diagram showing an example of a dataset at the time of evaluation. [Figure 12] FIG. 12 is a diagram showing Evaluation Result 1. [Figure 13] FIG. 13 is a diagram showing Evaluation Result 2. [Figure 14] FIG. 14 is a diagram for explaining an example of hardware configuration.

Modes for Carrying Out the Invention

[0012] The following describes in detail, with reference to the drawings, embodiments of the machine learning program, machine learning method, and information processing device disclosed in this application. However, the present invention is not limited by these embodiments. Each embodiment can be combined as appropriate within a non-consistent range. [Examples]

[0013] (Description of information processing device) Figure 1 is a diagram illustrating the information processing device 10 according to Example 1. The information processing device 10 shown in Figure 1 is an example of a computer that performs machine learning, such as a machine learning model that estimates molecular structural features using different molecules as multiple training data, or a NNP that performs material discovery.

[0014] In this embodiment, we describe an example of distributed learning of NNP using two processes. For example, NNP is an example of a computational model or machine learning model that calculates potential (interatomic potential), which is a computational model that estimates the energy surface with the aim of reproducing the motion of atoms. Each training data is a molecule composed of multiple atoms, and its molecular structure, number of atoms, and molecular use are known. The molecular use refers to, for example, whether the molecule or the material structure of the molecule is solid or gaseous, and the intended use of the molecule (e.g., used in pharmaceuticals).

[0015] In this state, the information processing device 10 solves a partitioning problem (NP-complete problem) that divides the given n training data into sets of processes. For example, as shown in Figure 1, the information processing device 10 finds combinations of training data such that the sum of the number of atoms in each process is uniform, as an optimization problem using a greedy method or the like. Specifically, the information processing device 10 arranges the n training data in descending order of the number of atoms and assigns them to the process with the smallest sum of the number of atoms in the training data within the set at that time, in descending order of the number of atoms.

[0016] For example, the information processing device 10 rearranges the training data in the following order: number of atoms "25", number of atoms "19", number of atoms "12", number of atoms "9", number of atoms "8", number of atoms "4", number of atoms "3", and number of atoms "1". Next, the information processing device 10 assigns the training data with "25" atoms to process 1, and assigns the training data with "19" atoms to process 2, which has fewer atoms already assigned.

[0017] Furthermore, the information processing device 10 assigns the training data with 12 atoms to process 2, which has fewer assigned atoms, and the training data with 9 atoms to process 1, which also has fewer assigned atoms. Using a similar method, the information processing device 10 assigns the training data with 8 atoms to process 2, the training data with 4 atoms to process 1, the training data with 3 atoms to process 1, and the training data with 1 atom to process 2.

[0018] The information processing device 10 then assigns the training data sets with atomic numbers of "25", "9", "4", and "3" assigned to process 1 as training dataset A, and the training data sets with atomic numbers of "19", "12", "8", and "1" assigned to process 2 as training dataset B.

[0019] Next, the information processing device 10 executes process 1 using training dataset A to perform machine learning on the machine learning model, and similarly executes process 2 using training dataset B to perform machine learning on the machine learning model. After that, the information processing device 10 generates the desired machine learning model by sharing and integrating the results of each machine learning process.

[0020] However, if some of the training data includes training data with an extremely large number of atoms, the load balancing may become uneven. Figure 2 illustrates an example of uneven load balancing using the greedy method. As shown in Figure 2, consider the case where the training data is rearranged in the following order: 90 atoms, 19 atoms, 12 atoms, 9 atoms, 8 atoms, 4 atoms, 3 atoms, and 1 atom. In this example, when the training data is assigned to each process, the training data with 90 atoms is assigned to process 1, and then all the other training data is assigned to process 2. As a result, the load balancing of the training data is not performed correctly, and the computational cost of distributed learning of NNP in data parallel cannot be reduced.

[0021] Therefore, in learning aimed at extracting atomic structure features (e.g., pre-training), when focusing on a single atom in the training data, we notice that highly similar atomic structures are frequently included in each training data set, and training data with a large number of atoms appear infrequently. In other words, the impact on the learning results of excluding training data with a large number of atoms is considered to be limited. For this reason, if some of the training data contains training data with an extremely large number of atoms, the following training data replacement method is used to replace the relevant training data with other training data to improve load balancing.

[0022] Specifically, the information processing device 10 according to Example 1 obtains the number of atoms contained in each of the multiple training data when performing data parallelism on machine learning for a machine learning model that extracts molecular structure features using different molecules as multiple training data. If the information processing device 10 finds that among the number of atoms in each of the multiple training data obtained, there is a first training data with a number of atoms greater than or equal to a threshold, the first training data is replaced with a number of atoms less than the threshold based on the other training data among the multiple training data.

[0023] Figure 3 illustrates the load balancing in Example 1. As shown in Figure 3, the information processing device 10 rearranges the training data in the following order: number of atoms "90", number of atoms "19", number of atoms "12", number of atoms "9", number of atoms "8", number of atoms "4", number of atoms "3", and number of atoms "1". Next, the information processing device 10 assigns the training data with the highest number of atoms, "90", to process 1. At this time, since the number of atoms "90" exceeds a threshold (for example, 50), the information processing device 10 replaces the training data.

[0024] Specifically, the information processing device 10 identifies the training data with the next highest number of atoms, "19," and replaces the training data with "90" atoms with the training data with "19" atoms. Then, the information processing device 10 assigns the replaced training data with "19" atoms to process 1.

[0025] Next, the information processing device 10 assigns the training data with the number of atoms selected as the substitution source, "19," to process 2, which has a smaller number of already assigned atoms, and assigns the training data with the number of atoms, "12," to process 1, which also has a smaller number of already assigned atoms. If the number of already assigned atoms is the same, the process may be selected arbitrarily.

[0026] Using a similar method, the information processing device 10 assigns training data with 9 atoms to process 2, training data with 8 atoms to process 2, training data with 4 atoms to process 1, training data with 3 atoms to process 1, and training data with 1 atom to process 2.

[0027] As described above, the information processing device 10 can perform appropriate load balancing without significantly impacting the learning results, thereby reducing the computational cost when learning NNP in a data-parallel distributed manner.

[0028] (Functional Configuration) Figure 4 is a functional block diagram showing the functional configuration of the information processing device 10 according to Embodiment 1. As shown in Figure 4, the information processing device 10 has a communication unit 11, a storage unit 12, and a control unit 20.

[0029] The communication unit 11 is a processing unit that controls communication with other devices, and is implemented, for example, by a communication interface. For example, the communication unit 11 receives input of training data and various instructions from administrators, etc., and outputs the results of machine learning to an administrator terminal, etc.

[0030] The memory unit 12 is a processing unit that stores various data and various programs executed by the control unit 20, and is implemented by, for example, memory or a hard disk. This memory unit 12 stores the learning data DB 13 and the NNP model 14.

[0031] The training data DB13 is a database that stores the training data used to train the NNP model 14. Specifically, the training data DB13 stores training data that associates molecules, which are composed of multiple atoms, with the features and NNP values ​​of those molecules. The molecules (molecular structures) in the training data are the explanatory variables, and the features and NNP values ​​are the target variables. For example, the training data shown in Figure 3 is an example.

[0032] NNP model 14 is an example of a machine learning model that outputs an NNP or features in response to a molecular (molecular structure) input, such as a neural network. In the example, it is the model being trained.

[0033] The control unit 20 is a processing unit that oversees the entire information processing device 10 and is implemented by, for example, a processor. This control unit 20 includes a setting unit 21, a distributed processing unit 22, and a machine learning unit 23. The setting unit 21, the distributed processing unit 22, and the machine learning unit 23 are examples of electronic circuits and processes executed by a processor, for example.

[0034] The settings unit 21 is a processing unit that performs various settings, such as replacement conditions for learning data, in response to input from an administrator or other user. Figure 5 is a diagram illustrating the setting of replacement conditions.

[0035] As shown in Figure 5, the configuration unit 21 sets the divisions of the training data. For example, the configuration unit 21 assigns a unique number (data_no) to each of the n training data and divides the n training data into sets of m (m≧1) or more training data (datasets). In the example in Figure 5, the configuration unit 21 divides "data_no:1~10" into "dataset A, B, C". Specifically, the configuration unit 21 divides "data_no=1,4,5,8" into "dataset A", "data_no=2,6,9" into "dataset B", and "data_no=3,7,10" into "dataset C". Note that the data divisions are based on the intended use of the molecules, etc.

[0036] Furthermore, the configuration unit 21 sets the selection of training data (selection_set). Specifically, the configuration unit 21 selects one or more sets of training data to be used as replacement candidates. As shown in Figure 5, the configuration unit 21 sets "selection_set,A,C" as an example of the data structure of selection_set. In other words, the configuration unit 21 sets that the training data of dataset A and the training data of dataset C are interchangeable. Note that selection_set may include data categories with the same molecular use, data categories with similar molecular uses, or data categories that can be used for the same molecular use.

[0037] Furthermore, the setting unit 21 sets the maximum number of replacement data (max_num_data). Specifically, the setting unit 21 sets the maximum number of training data to be replaced. As shown in Figure 5, the setting unit 21 sets "max_num_data,2" as an example of the data structure for max_num_data. In other words, the setting unit 21 sets the number of replacements during parallel data execution to a maximum of 2.

[0038] Furthermore, the setting unit 21 sets the minimum number of atoms (min_num_atoms). Specifically, the setting unit 21 sets the minimum number of atoms in the training data to be used as replacement candidates. As shown in Figure 5, the setting unit 21 sets "A, 50" and "C, 70" as examples of the data structure "dataset, min_num_atoms" for min_num_atoms. In other words, the setting unit 21 sets that training data from dataset A with 50 or more atoms is to be replaced, and training data from dataset C with 70 or more atoms is to be replaced.

[0039] Furthermore, the setting unit 21 sets the atomic ratio, which is a value representing the ratio of the number of atoms in one training data to the adjacent training data. The atomic ratio is calculated using the formula: "Atomic ratio = (Number of atoms in the i-th training data) / (Number of atoms in the i+1-th training data)".

[0040] The setting unit 21 then sets the minimum value of the atom number ratio (min_num_atoms_ratio). Specifically, the setting unit 21 sets the minimum value of the atom number ratio of the training data to be used as replacement candidates. As shown in Figure 5, the setting unit 21 sets "A, 1.5" and "C, 2.0" as examples of the data structure "dataset, min_num_atoms_ratio" for min_num_atoms_ratio. In other words, the setting unit 21 sets the training data of dataset A to be used as a replacement candidate if the atom number ratio of the training data to the training data with the next highest atom number is 1.5 or greater.

[0041] Furthermore, the configuration unit 21 sets the selection of replacement training data (replacement_set). Specifically, the configuration unit 21 selects one or more sets of training data to be used as candidates for the replacement training data. As shown in Figure 5, the configuration unit 21 sets "A,A" or "C,A / C" as an example of the data structure "dataset,replacement_set" for replacement_set. In other words, the configuration unit 21 sets the training data of dataset A to be used as a candidate for the replacement training data set, and the training data of dataset C to be used as a candidate for the replacement training data set, and selects either dataset A or C.

[0042] Returning to Figure 4, the distributed processing unit 22 is a processing unit that performs distributed processing of training data using data parallelism. For example, the distributed processing unit 22 divides n training data into sets of m or more training data, and arranges the n training data in descending order of the number of atoms. Then, the distributed processing unit 22 replaces training data that satisfies the following conditions defined for each set of training data with other training data.

[0043] conditions "The set of training data is defined as the set of training data to be used as replacement candidates." The condition is "the order of the training data (i-th: i≧1) ≤ the maximum number of training data to be replaced". The condition is "Number of atoms in the training data ≥ Minimum number of atoms in the training data to be used as replacement candidates". The condition is "Number of atoms in the training data > Number of atoms in the training data after substitution". "The training data after substitution is the set of candidate training data after substitution." Note that the training data after substitution is the (i+1)th training data point.

[0044] Here, we will explain a specific example of the substitution process. First, we will explain the overview of the substitution process. Figure 6 is a diagram illustrating the substitution process. The distributed processing unit 22 sorts the training data in descending order of the number of atoms. Specifically, as shown in Figure 6, the distributed processing unit 22 generates "1,A,A1", "2,B,B1", "3,C,C1", "4,A,A2", etc., as "data_no,dataset,number of atoms". Note that the data shown in Figure 6 is assumed to satisfy the above conditions.

[0045] In this case, the distributed processing unit 22 determines whether substitution is possible for the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Next, the distributed processing unit 22 determines whether substitution is possible for the second training data 2, based on its relationship with the next most numerous training data 3, and then determines whether substitution is possible for training data 3, based on its relationship with the next most numerous training data 4. In this way, the distributed processing unit 22 executes the substitution process sequentially from the first training data, as long as the above conditions are met.

[0046] Figure 7 illustrates a specific example of substitution (concrete example 1). In Figure 7, we explain an example of determining whether substitution is possible using the selection of training data (selection_set) as an example. It is assumed that other conditions are met.

[0047] As shown in Figure 7, the distributed processing unit 22 generates a data structure similar to that in Figure 6 by rearranging the training data in descending order of the number of atoms. In this state, the distributed processing unit 22 determines whether to replace the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Here, the distributed processing unit 22 determines that the condition "selection_set = "A,C"" is not satisfied because training data 1 belongs to dataset A and training data 2 belongs to dataset B, and therefore does not perform the replacement of training data 1.

[0048] Next, the distributed processing unit 22 determines whether to replace training data 2 based on its relationship with training data 3, which has the next highest number of atoms. In this case as well, the distributed processing unit 22 determines that the condition "selection_set = "A,C"" is not satisfied because training data 2 belongs to dataset B and training data 3 belongs to dataset C, and therefore does not perform the replacement of training data 2.

[0049] Next, the distributed processing unit 22 determines whether training data 3 can be replaced based on its relationship with training data 4, which has the next highest number of atoms. In this case, the distributed processing unit 22 determines that training data 3 belongs to dataset C and training data 4 belongs to dataset A, thus satisfying the condition "selection_set = "A,C"", and performs the replacement of training data 3. Specifically, the distributed processing unit 22 replaces training data 3 (dataset C, number of atoms C1) with training data 4 (dataset A, number of atoms A2).

[0050] Figure 8 illustrates a specific example of substitution, part 2. Figure 8 uses the selection of training data (selection_set) and the maximum number of replacement data points (max_num_data) as examples to illustrate how substitution is determined. It is assumed that other conditions are met.

[0051] As shown in Figure 8, the distributed processing unit 22 sorts the training data in descending order of the number of atoms. Specifically, the distributed processing unit 22 generates "data_no,dataset,number of atoms" such as "1,A,A1", "2,A,A2", "3,A,A3", etc.

[0052] In this state, the distributed processing unit 22 determines whether substitution is possible for the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Here, the distributed processing unit 22 determines that the condition "selection_set = "A,C"" is satisfied because training data 1 belongs to dataset A and training data 2 belongs to dataset A. Furthermore, the distributed processing unit 22 determines that the number of substitutions is less than the maximum number of data to be substituted (max_num_data) because no substitution has been performed yet.

[0053] As a result, the distributed processing unit 22 performs the replacement of training data 1. Specifically, the distributed processing unit 22 replaces training data 1 (dataset A, number of atoms A1) with training data 2 (dataset A, number of atoms A2).

[0054] Next, the distributed processing unit 22 determines whether a substitution is possible for training data 2 based on its relationship with training data 3, which has the next highest number of atoms. Here, the distributed processing unit 22 determines that the condition "selection_set = "A,C"" is satisfied because training data 2 belongs to dataset A and training data 3 belongs to dataset A. Furthermore, the distributed processing unit 22 determines that the number of substitutions (1) is less than the maximum number of substituted data (max_num_data) because only one substitution has been performed.

[0055] As a result, the distributed processing unit 22 performs the replacement of training data 2. Specifically, the distributed processing unit 22 replaces training data 2 (dataset A, number of atoms A2) with training data 3 (dataset A, number of atoms A3).

[0056] Next, the distributed processing unit 22 determines whether to substitute training data 3 based on its relationship with training data 4, which has the next highest number of atoms. Here, the distributed processing unit 22 determines that training data 3 belongs to dataset A and training data 4 belongs to dataset A, thus satisfying the condition "selection_set = "A,C"". However, since substitution has been performed twice, the distributed processing unit 22 determines that the number of substitutions (2) is greater than or equal to the maximum number of substituted data (max_num_data), and therefore does not perform the substitution on training data 3.

[0057] In this way, the distributed processing unit 22 uses the above conditions to determine whether each training data can be replaced.

[0058] Returning to Figure 4, the machine learning unit 23 is the processing unit that performs machine learning on the NNP model 14. Specifically, the machine learning unit 23 performs distributed learning of the NNP model 14 using a dataset of training data that has been parallelized by the distributed processing unit 22.

[0059] For example, the machine learning unit 23 performs machine learning on a first replica model of the NNP model 14 using dataset A, and then performs machine learning on a second replica model of the NNP model 14 using dataset B. The machine learning unit 23 then integrates the learning results of the first replica model and the second replica model to generate the NNP model 14. To give one example, the machine learning unit 23 integrates the learning results by applying the average value of the updated parameters corresponding to each replica model to the NNP model 14. Various learning methods can be employed in machine learning, such as supervised learning, unsupervised learning, and semi-supervised learning.

[0060] (Overall processing flow) Figure 9 is a flowchart showing the overall flow of the process. As shown in Figure 9, when the information processing device 10 is instructed to start processing by an administrator or the like, it sets the replacement conditions for the learning data according to the input of the administrator or the like (S101).

[0061] Next, the information processing device 10 performs the replacement of the training data according to the training data replacement conditions (S102), and then performs machine learning of the NNP model 14 using data parallelism with the training data dataset including the replaced training data (S103).

[0062] (Detailed processing flow) Figure 10 is a flowchart showing the detailed flow of the process. Here, it is assumed that the substitution conditions shown in Figure 5 are set.

[0063] As shown in Figure 10, the information processing device 10 sorts the training data in descending order of the number of atoms (S201). Next, the information processing device 10 sets the counter variable i to "max_num_data" (S202).

[0064] Then, if the counter variable i is greater than 0 (S203: Yes), the information processing device 10 checks the i-th training data D i is "D i It is determined whether the condition "∈selection_set" is satisfied (S204). Here, the information processing device 10 determines "D i If the condition "∈selection_set" is met (S204:Yes), then "D i Determine whether the number of atoms ≥ min_num_atoms (S205).

[0065] Then, the information processing device 10 is "D i If the number of atoms ≥ min_num_atoms is satisfied (S205: Yes), then "D i Determine whether the atomic ratio of ≥ min_num_atoms_ratio is satisfied (S206).

[0066] Here, the information processing device 10 is "D i If the atomic ratio of ≥ min_num_atoms_ratio is satisfied (S206: Yes), then "D i+1 Determine whether the condition "∈replacement_set" is satisfied (S207).

[0067] Then, when the information processing apparatus 10 satisfies "D i+1 ∈replacement_set" (S207: Yes), it replaces the learning data D i =D i+1 with the learning data D i as "D i+1 " (S208), decrements the counter variable i (S209), and repeats the processes after S203.

[0068] Note that when the information processing apparatus 10 does not satisfy "D i ∈selection_set" (S204: No), does not satisfy "the number of atoms of D i ≥min_num_atoms" (S205: No), does not satisfy "the atomic number ratio of D i ≥min_num_atoms_ratio" (S206: No), or does not satisfy "D i+1 ∈replacement_set" (S207: No), it repeats the processes after S203. When the counter variable becomes 0 or less in S203 (S203: No), the process ends.

[0069] (Effect) As described above, the information processing apparatus 10 can improve the inference accuracy and generalization of the NNP. In the learning of the NNP using a large-scale model and large-scale data, the information processing apparatus 10 can achieve a reduction in the calculation cost and the memory capacity used as an improvement in the performance in the distributed learning of the NNP.

[0070] Next, we will explain the results of evaluating each method by comparing the method according to Example 1 with the conventional method (Figure 2). Figure 11 shows an example of the dataset used during evaluation. As shown in Figure 11, "1,A", "2,A", "3,A", ..., "n,A" were used as the training dataset "data_no,dataset" during evaluation. In addition, the following settings were set for the substitution conditions: "selection_set,A", "max_num_data,2", "min_num_atoms,50", "min_num_atoms_ratio,1.5", and "replacement_set,A". As training data, a mixed set of publicly available material datasets (HME21: generated periodic structure, QM7-X: small molecule, OC22: part of the catalyst) was used.

[0071] Under these conditions, distributed learning using data parallelism was performed using the same number of training data for both the conventional method without substitution (Figure 2) and the method according to Example 1 which includes substitution. Figure 12 shows the results of evaluation 1. As shown in Figure 12, the training time for the conventional method was "841.4(s)", and the training time for Example 1 was "547.7(s)", demonstrating that the method according to Example 1 was 1.54 times faster than the conventional method.

[0072] Figure 13 shows the results of evaluation 2. In Figure 13, the vertical axis shows the mean absolute error (MAE), and the horizontal axis shows the number of learning steps. As shown in Figure 13, the learning curves of the method according to Example 1 and the conventional method show similar trends. In other words, it can be seen that there is no deterioration in learning accuracy even when the training data is replaced.

[0073] As a result, the method according to Example 1 can significantly reduce training time while maintaining the same training accuracy as the conventional method. [Examples]

[0074] Now, although embodiments of the present invention have been described, the present invention may be implemented in various other forms besides those described above.

[0075] (Numerical values, etc.) The machine learning model, training data, load balancing method during data distribution, and molecular applications used in the above examples are merely examples and can be changed as needed. Furthermore, the processing flow described in each flowchart can be modified as appropriate within a consistent range.

[0076] (Data replacement example) In the above embodiment, we described an example in which the first training data, whose number of atoms is above a threshold, is replaced with training data that satisfies the condition within the training dataset, but the invention is not limited to this. For example, the first training data can also be replaced with training data that is not present in the training dataset. In this case, the first training data is replaced with training data that has the same or similar use as the molecule in the first training data. If the number of atoms in the replaced training data is no longer the largest number of atoms in the subsequent training data, the data is sorted again, and then the replacement judgment and data parallelism are performed.

[0077] Furthermore, the first training data, whose number of atoms is above a threshold, can be replaced with a nutrient whose number of atoms is closest to the average number of atoms in the training data set as the replacement target, and whose purpose is the same as that of the nutrient in the first training data. In this case as well, the data is sorted again after the replacement, and then the replacement judgment and data parallelism are performed.

[0078] (system) Unless otherwise specified, the processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed at will.

[0079] Furthermore, the specific forms of distribution and integration of the components of each device are not limited to those shown in the diagram. For example, the distributed processing unit 22 and the machine learning unit 23 may be integrated. In other words, all or part of the components may be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Moreover, all or any part of the processing functions of each device may be implemented by a CPU and a program executed for analysis on that CPU, or as hardware using wired logic.

[0080] Furthermore, each processing function performed by each device may be implemented, in whole or in part, by a CPU and a program executed for analysis by that CPU, or by hardware using wired logic.

[0081] (Hardware) Figure 14 illustrates an example of a hardware configuration. As shown in Figure 14, the information processing device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, memory 10c, and a processor 10d. Furthermore, the components shown in Figure 14 are interconnected by a bus or the like.

[0082] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and databases that operate the functions shown in Figure 4.

[0083] The processor 10d operates a process that performs the functions described in Figure 4 by reading a program that performs the same processing as each processing unit shown in Figure 4 from the HDD 300b or the like and loading it into memory 10c. For example, this process performs the same functions as each processing unit of the information processing device 10. Specifically, the processor 10d reads a program that has the same functions as the setting unit 21, distributed processing unit 22, machine learning unit 23, etc. from the HDD 10b or the like. Then, the processor 10d executes a process that performs the same processing as the setting unit 21, distributed processing unit 22, machine learning unit 23, etc.

[0084] Thus, the information processing device 10 operates as an information processing device that executes a machine learning method by reading and executing a program. Furthermore, the information processing device 10 can also achieve the same functionality as the embodiment described above by reading the program from a recording medium using a media reader and executing the read program. Note that the program referred to in this other embodiment is not limited to being executed by the information processing device 10. For example, the above embodiment may also be applied to cases where another computer or server executes the program, or where these computers or servers collaborate to execute the program.

[0085] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and executed by being read from the recording medium by a computer. [Explanation of symbols]

[0086] 10 Information Processing Devices 11 Communications Department 12 Storage section 13. Training Data Database 14 NNP Model 20 Control Unit 21 Setting section 22 Distributed Processing Unit 23 Machine Learning Department

Claims

1. In a machine learning program that uses multiple different molecules as training data to perform machine learning on a machine learning model that extracts molecular structural features, The number of atoms contained in each of the aforementioned multiple training data is obtained, If, among the number of atoms in each of the acquired plurality of training data, there is a first training data set having a number of atoms equal to or greater than the threshold, the first training data set is replaced with other training data sets among the plurality of training data sets that have a number of atoms less than the threshold. A machine learning program characterized by executing a process.

2. The substitution process described above is: The first training data is replaced with specific training data, which is identified based on the material structure of the molecule shown by the first training data among the plurality of training data, and in which the number of atoms is less than the threshold. The machine learning program according to feature 1.

3. The substitution process described above is: The number of atoms in the first training data is replaced with the specific training data from among the plurality of training data whose application of the molecular material structure shown in the first training data is the same. The machine learning program according to feature 2.

4. The substitution process described above is: Each of the aforementioned training data is sorted in descending order of the number of atoms, In order of the training data with the largest number of atoms, it is determined whether the conversion conditions are met, which indicate that the number of atoms is equal to or greater than the threshold and the number of converted training data is less than the threshold. If the above conversion conditions are met, the training data to be replaced is replaced with training data containing fewer atoms than the training data to be replaced. A machine learning program according to claim 1 or 2.

5. In a machine learning method where a computer performs machine learning on a machine learning model that extracts molecular structural features using multiple different molecules as training data, The number of atoms contained in each of the aforementioned multiple training data is obtained, If, among the number of atoms in each of the acquired plurality of training data, there is a first training data set having a number of atoms equal to or greater than the threshold, the first training data set is replaced with other training data sets among the plurality of training data sets that have a number of atoms less than the threshold. A machine learning method characterized by performing a process.

6. In an information processing device that performs machine learning on a machine learning model that extracts molecular structural features using multiple different molecules as training data, The number of atoms contained in each of the aforementioned multiple training data is obtained, If, among the number of atoms in each of the acquired plurality of training data, there is a first training data set having a number of atoms equal to or greater than the threshold, the first training data set is replaced with other training data sets among the plurality of training data sets that have a number of atoms less than the threshold. An information processing device characterized by having a control unit.

Images