Machine learning program, machine learning method, and information processing device

Abstract

This information processing device executes machine learning of a machine learning model that extracts a feature amount of a molecular structure using different molecules as a plurality of pieces of learning data. The information processing device acquires the number of atoms included in a molecule indicated by each of the plurality of pieces of learning data, and when there is first learning data having the number of atoms equal to or greater than a threshold value among the acquired numbers of atoms of the respective plurality of pieces of learning data, replaces the first learning data with another number of atoms less than the threshold value on the basis of learning data other than the first learning data among the plurality of pieces of learning data.

Description

Machine learning programs, machine learning methods, and information processing devices.

[0001] The present invention relates to a machine learning program, a machine learning method, and an information processing device.

[0002] In recent years, materials informatics (hereinafter sometimes referred to as MI) has been advancing globally as a data-driven research and development approach aimed at shortening research and development periods and reducing costs in materials development. MI relies heavily on the accumulation of high-quality materials data, and efficient data accumulation is expected through the use of simulation technologies such as computational chemistry and neural network potentials (hereinafter sometimes referred to as NNP), in addition to experiments.

[0003] Potential (interatomic potential) is a computational model that directly estimates the energy surface from atomic structure. In recent years, NNP has become one of the main methods for calculating potential, using neural networks. Due to the enormous computational cost of training NNPs using large-scale models and large-scale data aimed at improving the inference accuracy and generalization of NNPs, distributed learning of NNPs is being employed.

[0004] For distributed learning of NNPs, two methods are known: model parallelism, which divides a single model and deploys multiple processes within it, and data parallelism, which duplicates the same model and multiplies the batch size by a factor of two.

[0005] Japanese Patent Publication No. 2023-181372, Japanese Patent Publication No. 2024-2637, Japanese Patent Publication No. 2023-117513, U.S. Patent Application Publication No. 2024 / 0028889, U.S. Patent Application Publication No. 2013 / 0290223

[0006] However, when performing distributed learning of NNPs in parallel with data, the computational cost can increase. For example, load balancing in distributed learning of NNPs becomes a partitioning problem (NP-complete problem) in which a given n training data is divided into sets of a number of processes. The combination of training data in which the sum of the number of atoms in each training data is uniform is then found as an optimization problem using methods such as the greedy method.

[0007] However, if some of the training data contains training data with an extremely large number of atoms, there is no combination of training data where the sum of the number of atoms in each process is uniform. As a result, some processes will have a large sum of atoms in their training data, leading to uneven load balancing. This can actually increase computational costs.

[0008] One aspect of this project is to provide machine learning programs, machine learning methods, and information processing devices that can reduce the computational cost when performing distributed learning in parallel with data.

[0009] In the first proposal, a machine learning program that causes a computer to perform machine learning on a machine learning model that extracts molecular structure features using multiple different molecules as training data is characterized in that the number of atoms contained in the molecules shown in each of the multiple training data is obtained, and if there is a first training data among the obtained number of atoms of the multiple training data that has a number of atoms equal to or greater than a threshold, the first training data is replaced with a number of atoms that is less than the threshold, based on the other training data among the multiple training data.

[0010] According to one embodiment, the computational cost when performing distributed learning in parallel with data can be reduced.

[0011] Figure 1 is a diagram illustrating the information processing device according to Example 1. Figure 2 is a diagram illustrating an example of uneven load balancing using the greedy method. Figure 3 is a diagram illustrating load balancing according to Example 1. Figure 4 is a functional block diagram showing the functional configuration of the information processing device according to Example 1. Figure 5 is a diagram illustrating the setting of substitution conditions. Figure 6 is a diagram illustrating the substitution process. Figure 7 is a diagram illustrating specific example 1 of substitution. Figure 8 is a diagram illustrating specific example 2 of substitution. Figure 9 is a flowchart illustrating the overall flow of the process. Figure 10 is a flowchart illustrating the detailed flow of the process. Figure 11 is a diagram illustrating an example of a dataset during evaluation. Figure 12 is a diagram illustrating evaluation result 1. Figure 13 is a diagram illustrating evaluation result 2. Figure 14 is a diagram illustrating an example of a hardware configuration.

[0012] The following describes in detail, with reference to the drawings, embodiments of the machine learning program, machine learning method, and information processing device disclosed in this application. However, the present invention is not limited by these embodiments. Each embodiment can be combined as appropriate within a non-consistent range.

[0013] (Description of Information Processing Device) Figure 1 is a diagram illustrating the information processing device 10 according to Example 1. The information processing device 10 shown in Figure 1 is an example of a computer that performs machine learning, such as machine learning models that estimate molecular structural features and NNPs that perform material discovery, using different molecules as multiple training data.

[0014] In this embodiment, we describe an example of distributed learning of NNP using two processes. For example, NNP is an example of a computational model or machine learning model that calculates potential (interatomic potential), which is a computational model that estimates the energy surface with the aim of reproducing the motion of atoms. Each training data is a molecule composed of multiple atoms, and its molecular structure, number of atoms, and molecular use are known. The molecular use refers to, for example, whether the molecule or the material structure of the molecule is solid or gaseous, and the intended use of the molecule (e.g., used in pharmaceuticals).

[0015] In this state, the information processing device 10 solves a partitioning problem (NP-complete problem) that divides the given n training data into sets of processes. For example, as shown in Figure 1, the information processing device 10 finds a combination of training data such that the sum of the number of atoms in the training data within each process is uniform, as an optimization problem using a greedy method or the like. Specifically, the information processing device 10 arranges the n training data in descending order of the number of atoms and assigns them to the process with the smallest sum of the number of atoms in the training data within the set at that time, in descending order of the number of atoms.

[0016] For example, the information processing device 10 rearranges the training data in the following order: number of atoms "25", number of atoms "19", number of atoms "12", number of atoms "9", number of atoms "8", number of atoms "4", number of atoms "3", and number of atoms "1". Next, the information processing device 10 assigns the training data with "25" atoms to process 1, and assigns the training data with "19" atoms to process 2, which has fewer atoms already assigned.

[0017] Furthermore, the information processing device 10 assigns the training data with 12 atoms to process 2, which has a small number of atoms already assigned, and the training data with 9 atoms to process 1, which also has a small number of atoms already assigned. Using a similar method, the information processing device 10 assigns the training data with 8 atoms to process 2, the training data with 4 atoms to process 1, the training data with 3 atoms to process 1, and the training data with 1 atom to process 2.

[0018] The information processing device 10 then uses the training data set with atomic numbers of "25", "9", "4", and "3" assigned to process 1 as training dataset A, and the training data set with atomic numbers of "19", "12", "8", and "1" assigned to process 2 as training dataset B.

[0019] Next, the information processing device 10 executes process 1 using training dataset A to perform machine learning on the machine learning model, and similarly executes process 2 using training dataset B to perform machine learning on the machine learning model. After that, the information processing device 10 generates the desired machine learning model by sharing and integrating the results of each machine learning process.

[0020] However, if some of the training data includes training data with an extremely large number of atoms, the load balancing may become uneven. Figure 2 illustrates an example of uneven load balancing using the greedy method. As shown in Figure 2, consider the case where the training data is rearranged in the following order: 90 atoms, 19 atoms, 12 atoms, 9 atoms, 8 atoms, 4 atoms, 3 atoms, and 1 atom. In this example, when the training data is assigned to each process, the training data with 90 atoms is assigned to process 1, and then all the other training data is assigned to process 2. As a result, the load balancing of the training data is not performed correctly, and the computational cost of distributed learning of NNP in data parallelism cannot be reduced.

[0021] Therefore, in learning aimed at extracting atomic structure features (e.g., pre-training), when focusing on a single atom in the training data, we notice that highly similar atomic structures are included in many training data, and training data with a large number of atoms appear infrequently. In other words, the impact on the learning results of excluding training data with a large number of atoms is considered to be limited. For this reason, if some of the training data includes training data with an extremely large number of atoms, the following training data replacement method is used to replace the relevant training data with other training data to improve load balancing.

[0022] Specifically, the information processing device 10 according to Example 1 obtains the number of atoms contained in each of the multiple training data when performing data parallelism on machine learning for a machine learning model that extracts molecular structure features using different molecules as multiple training data. If the information processing device 10 finds that among the number of atoms in each of the multiple training data obtained, there is a first training data with a number of atoms greater than or equal to a threshold, the first training data is replaced with a number of atoms less than the threshold based on the other training data among the multiple training data.

[0023] Figure 3 illustrates the load balancing in Example 1. As shown in Figure 3, the information processing device 10 rearranges the training data in the following order: number of atoms "90", number of atoms "19", number of atoms "12", number of atoms "9", number of atoms "8", number of atoms "4", number of atoms "3", and number of atoms "1". Next, the information processing device 10 assigns the training data with the largest number of atoms, "90", to process 1. At this time, since the number of atoms "90" exceeds a threshold (for example, 50), the information processing device 10 replaces the training data.

[0024] Specifically, the information processing device 10 identifies the training data with the next highest number of atoms, "19," and replaces the training data with "90" atoms with the training data with "19" atoms. Then, the information processing device 10 assigns the replaced training data with "19" atoms to process 1.

[0025] Next, the information processing device 10 assigns the training data with the number of atoms selected as the substitution source, "19," to process 2, which has a small number of already assigned atoms, and assigns the training data with the number of atoms, "12," to process 1, which also has a small number of already assigned atoms. If the number of already assigned atoms is the same, the process may be selected arbitrarily.

[0026] Using a similar method, the information processing device 10 assigns training data with 9 atoms to process 2, training data with 8 atoms to process 2, training data with 4 atoms to process 1, training data with 3 atoms to process 1, and training data with 1 atom to process 2.

[0027] As described above, the information processing device 10 can perform appropriate load balancing without significantly impacting the learning results, thus reducing the computational cost when learning NNP in a data-parallel distributed manner.

[0028] (Functional Configuration) Figure 4 is a functional block diagram showing the functional configuration of the information processing device 10 according to Embodiment 1. As shown in Figure 4, the information processing device 10 has a communication unit 11, a storage unit 12, and a control unit 20.

[0029] The communication unit 11 is a processing unit that controls communication with other devices, and is implemented, for example, by a communication interface. For example, the communication unit 11 receives input of learning data and various instructions from administrators, and outputs the results of machine learning to an administrator terminal, etc.

[0030] The memory unit 12 is a processing unit that stores various data and various programs executed by the control unit 20, and is implemented by, for example, memory or a hard disk. This memory unit 12 stores the learning data DB 13 and the NNP model 14.

[0031] The training data DB13 is a database that stores training data used to train the NNP model 14. Specifically, the training data DB13 stores training data that associates molecules, which are composed of multiple atoms, with the features and NNPs of those molecules. The molecules (molecular structures) in the training data are the explanatory variables, and the features and NNPs are the target variables. For example, the training data shown in Figure 3 is an example.

[0032] The NNP model 14 is an example of a machine learning model that outputs NNPs and features in response to a molecular (molecular structure) input, such as a neural network. In this embodiment, it is the model being trained.

[0033] The control unit 20 is a processing unit that oversees the entire information processing device 10 and is implemented by, for example, a processor. This control unit 20 includes a setting unit 21, a distributed processing unit 22, and a machine learning unit 23. The setting unit 21, the distributed processing unit 22, and the machine learning unit 23 are examples of electronic circuits or processes executed by a processor.

[0034] The setting unit 21 is a processing unit that performs various settings, such as replacement conditions for learning data, in response to input from an administrator or other user. Figure 5 is a diagram illustrating the setting of replacement conditions.

[0035] As shown in FIG. 5, the setting unit 21 sets the classification of the learning data. For example, the setting unit 21 assigns a unique number (data_no) to each of the n learning data, and classifies the n learning data into a set (dataset) of m (m ≧ 1) or more learning data. In the example of FIG. 5, the setting unit 21 classifies "data_no: 1 to 10" into "dataset A, B, C". Specifically, the setting unit 21 classifies "data_no = 1, 4, 5, 8" into "dataset A", "data_no = 2, 6, 9" into "dataset B", and "data_no = 3, 7, 10" into "dataset C". Note that the data classification is classified according to the use of the molecule or the like.

[0036] In addition, the setting unit 21 sets the selection (selection_set) of the learning data. Specifically, the setting unit 21 selects one or more sets of learning data to be replacement candidates. As shown in FIG. 5, the setting unit 21 sets "selection_set, A, C" as an example of the data structure of selection_set. That is, the setting unit 21 sets that the learning data of dataset A and the learning data of dataset C are replaceable. Note that in selection_set, data classifications with the same use of the molecule, data classifications with similar uses of the molecule, data classifications that can be used as the use of the molecule, etc. are set.

[0037] In addition, the setting unit 21 sets the maximum value (max_num_data) of the number of replacement data. Specifically, the setting unit 21 sets the maximum value of the number of learning data to be replaced. As shown in FIG. 5, the setting unit 21 sets "max_num_data, 2" as an example of the data structure of max_num_data. That is, the setting unit 21 sets the number of replacement times during data parallel execution to up to 2 times.

[0038] Also, the setting unit 21 sets the minimum value of the number of atoms (min_num_atoms). Specifically, the setting unit 21 sets the minimum value of the number of atoms of the learning data to be a substitution candidate. As shown in FIG. 5, the setting unit 21 sets "A, 50" or "C, 70" as an example of the data structure of min_num_atoms, "dataset, min_num_atoms". That is, the setting unit 21 sets that the learning data having the number of atoms of "50" or more among the learning data of the dataset A is a substitution target, and the learning data having the number of atoms of "70" or more among the learning data of the dataset C is a substitution target.

[0039] Also, the setting unit 21 sets the atomic number ratio, which is a value representing the ratio of the number of atoms of one learning data and the number of atoms of the adjacent learning data. The atomic number ratio is calculated by "atomic number ratio = (number of atoms of the i-th learning data) / (number of atoms of the (i + 1)-th learning data)".

[0040] Then, the setting unit 21 sets the minimum value of the atomic number ratio (min_num_atoms_ratio). Specifically, the setting unit 21 sets the minimum value of the atomic number ratio of the learning data to be a substitution candidate. As shown in FIG. 5, the setting unit 21 sets "A, 1.5" or "C, 2.0" as an example of the data structure of min_num_atoms_ratio, "dataset, min_num_atoms_ratio". That is, for the learning data of the dataset A, the setting unit 21 sets that when the atomic number ratio between the number of atoms of the learning data and the number of atoms of the next learning data with a larger number of atoms is 1.5 or more, the learning data is set as a substitution candidate.

[0041] Furthermore, the setting unit 21 sets the selection of replacement training data (replacement_set). Specifically, the setting unit 21 selects one or more sets of training data to be used as candidates for the replacement training data. As shown in Figure 5, the setting unit 21 sets "A, A" or "C, A / C" as an example of the data structure "dataset, replacement_set" for replacement_set. In other words, the setting unit 21 sets the training data of dataset A to be used as a candidate for the replacement training data, and sets the training data of dataset C to be used as a candidate for the replacement training data, and selects either dataset A or C.

[0042] Returning to Figure 4, the distributed processing unit 22 is a processing unit that performs distributed processing of training data using data parallelism. For example, the distributed processing unit 22 divides n training data into sets of m or more training data, and arranges the n training data in descending order of the number of atoms. Then, the distributed processing unit 22 replaces training data that satisfies the following conditions defined for each set of training data with other training data.

[0043] The conditions are: "The set of training data ∈ The set of training data to be used as replacement candidates." "The order of the training data (i-th: i≧1) ≤ the maximum number of training data to be replaced." "The number of atoms in the training data ≥ the minimum number of atoms in the training data to be used as replacement candidates." "The number of atoms in the training data > the number of atoms in the training data after replacement." "The training data after replacement ∈ The set of candidate training data to be replaced." Note that the training data after replacement is the (i+1)th training data.

[0044] Here, we will explain a specific example of the substitution process. First, we will explain the overview of the substitution process. Figure 6 is a diagram illustrating the substitution process. The distributed processing unit 22 sorts the training data in descending order of the number of atoms. Specifically, as shown in Figure 6, the distributed processing unit 22 generates "1, A, A1", "2, B, B1", "3, C, C1", "4, A, A2", etc., as "data_no, dataset, number of atoms". Note that the data shown in Figure 6 is assumed to satisfy the above conditions.

[0045] In this case, the distributed processing unit 22 determines whether substitution is possible for the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Next, the distributed processing unit 22 determines whether substitution is possible for the second training data 2, based on its relationship with the next most numerous training data 3, and then determines whether substitution is possible for training data 3, based on its relationship with the next most numerous training data 4. In this way, the distributed processing unit 22 executes the substitution process sequentially from the first training data, as long as the above conditions are met.

[0046] Figure 7 illustrates a specific example of substitution (concrete example 1). In Figure 7, we explain an example of determining whether substitution is possible using the selection of training data (selection_set) as an example. Note that other conditions are assumed to be met.

[0047] As shown in Figure 7, the distributed processing unit 22 generates a data structure similar to that in Figure 6 by rearranging the training data in descending order of the number of atoms. In this state, the distributed processing unit 22 determines whether to replace the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Here, the distributed processing unit 22 determines that the condition "selection_set = "A, C"" is not satisfied because training data 1 belongs to dataset A and training data 2 belongs to dataset B, and therefore does not perform the replacement of training data 1.

[0048] Next, the distributed processing unit 22 determines whether to replace training data 2 based on its relationship with training data 3, which has the next highest number of atoms. In this case as well, the distributed processing unit 22 determines that the condition "selection_set = "A, C"" is not satisfied because training data 2 belongs to dataset B and training data 3 belongs to dataset C, and therefore does not perform the replacement of training data 2.

[0049] Next, the distributed processing unit 22 determines whether training data 3 can be replaced based on its relationship with training data 4, which has the next highest number of atoms. In this case, the distributed processing unit 22 determines that training data 3 belongs to dataset C and training data 4 belongs to dataset A, thus satisfying the condition "selection_set = "A, C"", and performs the replacement of training data 3. Specifically, the distributed processing unit 22 replaces training data 3 (dataset C, number of atoms C1) with training data 4 (dataset A, number of atoms A2).

[0050] Figure 8 illustrates a specific example of substitution, part 2. Figure 8 explains an example of determining whether substitution is possible, using the selection of training data (selection_set) and the maximum number of substitution data (max_num_data) as examples. It is assumed that other conditions are met.

[0051] As shown in Figure 8, the distributed processing unit 22 sorts the training data in descending order of the number of atoms. Specifically, the distributed processing unit 22 generates "1, A, A1", "2, A, A2", "3, A, A3", etc., as "data_no, dataset, number of atoms".

[0052] In this state, the distributed processing unit 22 determines whether substitution is possible for the first training data 1, which has the most atoms, based on its relationship with the next most numerous training data 2. Here, the distributed processing unit 22 determines that the condition "selection_set = "A, C"" is satisfied because training data 1 belongs to dataset A and training data 2 belongs to dataset A. Furthermore, the distributed processing unit 22 determines that the number of substitutions is less than the maximum number of data to be substituted (max_num_data) because no substitution has been performed yet.

[0053] As a result, the distributed processing unit 22 performs the replacement of training data 1. Specifically, the distributed processing unit 22 replaces training data 1 (dataset A, number of atoms A1) with training data 2 (dataset A, number of atoms A2).

[0054] Next, the distributed processing unit 22 determines whether substitution is possible for training data 2 based on its relationship with training data 3, which has the next highest number of atoms. Here, the distributed processing unit 22 determines that the condition "selection_set = "A, C"" is satisfied because training data 2 belongs to dataset A and training data 3 belongs to dataset A. Furthermore, the distributed processing unit 22 determines that the number of substitutions (1) is less than the maximum number of substituted data (max_num_data) because substitution has only been performed once.

[0055] As a result, the distributed processing unit 22 performs the replacement of training data 2. Specifically, the distributed processing unit 22 replaces training data 2 (dataset A, number of atoms A2) with training data 3 (dataset A, number of atoms A3).

[0056] Next, the distributed processing unit 22 determines whether to substitute training data 3 based on its relationship with training data 4, which has the next highest number of atoms. Here, the distributed processing unit 22 determines that training data 3 belongs to dataset A and training data 4 belongs to dataset A, thus satisfying the condition "selection_set = "A, C"". However, since substitution has been performed twice, the distributed processing unit 22 determines that the number of substitutions (2) is greater than or equal to the maximum number of substituted data (max_num_data), and therefore does not perform the substitution on training data 3.

[0057] In this way, the distributed processing unit 22 uses the above conditions to determine whether each piece of training data can be replaced.

[0058] Returning to Figure 4, the machine learning unit 23 is the processing unit that executes machine learning on the NNP model 14. Specifically, the machine learning unit 23 uses a dataset of training data parallelized by the distributed processing unit 22 to perform distributed learning of the NNP model 14.

[0059] For example, the machine learning unit 23 performs machine learning on a first replica model of the NNP model 14 using dataset A, and then performs machine learning on a second replica model of the NNP model 14 using dataset B. The machine learning unit 23 then integrates the learning results of the first replica model and the second replica model to generate the NNP model 14. To give one example, the machine learning unit 23 integrates the learning results by applying the average value of the updated parameters corresponding to each replica model to the NNP model 14. Various learning methods can be employed in machine learning, such as supervised learning, unsupervised learning, and semi-supervised learning.

[0060] (Overall processing flow) Figure 9 is a flowchart showing the overall processing flow. As shown in Figure 9, when the information processing device 10 is instructed to start processing by an administrator or the like, it sets the replacement conditions for the learning data according to the input of the administrator or the like (S101).

[0061] Next, the information processing device 10 performs the replacement of the training data according to the training data replacement conditions (S102), and then performs machine learning of the NNP model 14 using data parallelism with the training data dataset including the replaced training data (S103).

[0062] (Detailed processing flow) Figure 10 is a flowchart showing the detailed processing flow. Here, it is assumed that the substitution conditions shown in Figure 5 are set.

[0063] As shown in Figure 10, the information processing device 10 sorts the learning data in descending order of the number of atoms (S201). Next, the information processing device 10 sets the counter variable i to "max_num_data" (S202).

[0064] Then, if the counter variable i is greater than 0 (S203: Yes), the information processing device 10 checks the i-th training data D ｉ is "D ｉ It is determined whether the condition "∈selection_set" is satisfied (S204). Here, the information processing device 10 determines whether "D ｉ If the condition "∈selection_set" is met (S204: Yes), then "D ｉDetermine whether it satisfies "the number of atoms of D ≥ min_num_atoms" (S205).

[0065] And when the information processing apparatus 10 satisfies "the number of atoms of D ≥ min_num_atoms" (S205: Yes), it determines whether it satisfies "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206). ｉ And when the information processing apparatus 10 satisfies "the number of atoms of D ≥ min_num_atoms" (S205: Yes), it determines whether it satisfies "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206). ｉ Determine whether it satisfies "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206).

[0066] Here, when the information processing apparatus 10 satisfies "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206: Yes), it determines whether D ∈ replacement_set (S207). ｉ Here, when the information processing apparatus 10 satisfies "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206: Yes), it determines whether D ∈ replacement_set (S207). ｉ＋１ Determine whether D ∈ replacement_set (S207).

[0067] And when the information processing apparatus 10 satisfies D ∈ replacement_set (S207: Yes), set the learning data D as D (S208), decrement the counter variable i (S209), and repeat the processing after S203. ｉ＋１ And when the information processing apparatus 10 satisfies D ∈ replacement_set (S207: Yes), set the learning data D as D (S208), decrement the counter variable i (S209), and repeat the processing after S203. ｉ = D ｉ＋１ and replace the learning data D with the learning data D (S208), decrement the counter variable i (S209), and repeat the processing after S203. ｉ and replace the learning data D with the learning data D (S208), decrement the counter variable i (S209), and repeat the processing after S203. ｉ＋１ and replace the learning data D with the learning data D (S208), decrement the counter variable i (S209), and repeat the processing after S203.

[0068] In addition, when the information processing apparatus 10 does not satisfy D ∈ selection_set (S204: No), does not satisfy "the number of atoms of D ≥ min_num_atoms" (S205: No), does not satisfy "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206: No), or does not satisfy D ∈ replacement_set (S207: No), it repeats the processing after S203. When the counter variable i becomes 0 or less in S203 (S203: No), the processing ends. ｉ In addition, when the information processing apparatus 10 does not satisfy D ∈ selection_set (S204: No), does not satisfy "the number of atoms of D ≥ min_num_atoms" (S205: No), does not satisfy "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206: No), or does not satisfy D ∈ replacement_set (S207: No), it repeats the processing after S203. When the counter variable i becomes 0 or less in S203 (S203: No), the processing ends. ｉ does not satisfy "the number of atoms of D ≥ min_num_atoms" (S205: No), ｉ does not satisfy "the atomic number ratio of D ≥ min_num_atoms_ratio" (S206: No), ｉ＋１ does not satisfy D ∈ replacement_set (S207: No), it repeats the processing after S203. When the counter variable i becomes 0 or less in S203 (S203: No), the processing ends.

[0069] (Effects) As described above, the information processing device 10 can improve the inference accuracy and generalizability of NNP. In NNP training using large-scale models and large-scale data, the information processing device 10 can improve performance in distributed learning of NNP by reducing computational costs and memory usage.

[0070] Next, we will explain the results of evaluating each method by comparing the method according to Example 1 with the conventional method (Figure 2). Figure 11 shows an example of a dataset used during evaluation. As shown in Figure 11, "1, A", "2, A", "3, A", ..., "n, A" were used as the training dataset "data_no, dataset" during evaluation. In addition, the following settings were set for the substitution conditions: "selection_set, A", "max_num_data, 2", "min_num_atoms, 50", "min_num_atoms_ratio, 1.5", and "replacement_set, A". As training data, a mixed set of publicly available material datasets (HME21: generated periodic structure, QM7-X: small molecule, OC22: part of the catalyst) was used.

[0071] Under these conditions, distributed learning using data parallelism was performed using the same number of training data for both the conventional method without substitution (Figure 2) and the method according to Example 1 which includes substitution. Figure 12 shows the results of evaluation 1. As shown in Figure 12, the training time for the conventional method was "841.4 (s)", and the training time for Example 1 was "547.7 (s)", demonstrating that the method according to Example 1 achieved a speedup of 1.54 times compared to the conventional method.

[0072] Figure 13 shows the results of evaluation 2. In Figure 13, the vertical axis shows the mean absolute error (MAE), and the horizontal axis shows the number of learning steps. As shown in Figure 13, the learning curves of the method according to Example 1 and the conventional method show similar trends. In other words, it can be seen that there is no deterioration in learning accuracy even when the training data is replaced.

[0073] As a result, the method according to Example 1 can significantly reduce the training time while maintaining the same training accuracy as the conventional method.

[0074] Now, although embodiments of the present invention have been described, the present invention may be implemented in various other forms besides those described above.

[0075] (Numerical values, etc.) The machine learning model, training data, load balancing method during data distribution, and molecular applications used in the above examples are merely examples and can be changed at will. Furthermore, the processing flow described in each flowchart can be modified as appropriate within a consistent range.

[0076] (Data Replacement Example) In the above embodiment, an example was described in which the first training data, whose number of atoms is equal to or greater than a threshold, is replaced with training data that satisfies the condition within the training dataset. However, the example is not limited to this. For example, the first training data can also be replaced with training data that is not present in the training dataset. In this case, the first training data is replaced with training data that has the same or similar use as the molecule in the first training data. If the number of atoms in the replaced training data is no longer the number of atoms in the training data thereafter, the data is sorted again, and then replacement judgment and data parallelism are performed.

[0077] Furthermore, the first training data, whose number of atoms is above a threshold, can be replaced with a nutrient whose number of atoms is closest to the average number of atoms in the training data set as the replacement target, and whose purpose is the same as that of the nutrient in the first training data. In this case as well, the data is sorted again after the replacement, and then the replacement judgment and data parallelism are performed.

[0078] (System) Unless otherwise specified, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings may be changed at will.

[0079] Furthermore, the specific forms of distribution and integration of the components of each device are not limited to those shown in the diagram. For example, the distributed processing unit 22 and the machine learning unit 23 may be integrated. In other words, all or part of the components may be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Moreover, all or any part of the processing functions of each device may be implemented by a CPU and a program executed for analysis on that CPU, or as hardware using wired logic.

[0080] Furthermore, each processing function performed by each device may be implemented, in whole or in part, by a CPU and a program executed for analysis by that CPU, or by wired logic hardware.

[0081] (Hardware) Figure 14 is a diagram illustrating an example of hardware configuration. As shown in Figure 14, the information processing device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, memory 10c, and a processor 10d. Furthermore, each of the parts shown in Figure 14 is interconnected by a bus or the like.

[0082] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and databases that operate the functions shown in Figure 4.

[0083] The processor 10d operates a process that performs the functions described in Figure 4 by reading a program that performs the same processing as each processing unit shown in Figure 4 from the HDD 10b or the like and loading it into memory 10c. For example, this process performs the same functions as each processing unit of the information processing device 10. Specifically, the processor 10d reads a program that has the same functions as the setting unit 21, distributed processing unit 22, machine learning unit 23, etc. from the HDD 10b or the like. Then, the processor 10d executes a process that performs the same processing as the setting unit 21, distributed processing unit 22, machine learning unit 23, etc.

[0084] Thus, the information processing device 10 operates as an information processing device that executes a machine learning method by reading and executing a program. Furthermore, the information processing device 10 can also achieve the same functionality as in the above-described embodiment by reading the program from a recording medium using a media reader and executing the read program. Note that the program referred to in this other embodiment is not limited to being executed by the information processing device 10. For example, the above embodiment may also be applied similarly when another computer or server executes the program, or when they collaborate to execute the program.

[0085] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), or DVD (Digital Versatile Disc), and executed by being read from the recording medium by a computer.

[0086] 10 Information Processing Unit 11 Communication Unit 12 Storage Unit 13 Learning Data DB 14 NNP Model 20 Control Unit 21 Setting Unit 22 Distributed Processing Unit 23 Machine Learning Unit

Claims

1. A machine learning program that causes a computer to perform machine learning on a machine learning model that extracts molecular structure features using multiple different molecules as training data, characterized in that the program obtains the number of atoms contained in each of the multiple training data, and if there is a first training data among the obtained number of atoms of each of the multiple training data that has a number of atoms equal to or greater than a threshold, the program performs a process to replace the first training data with other numbers of atoms that fall below the threshold, based on the training data other than the first training data among the multiple training data.

2. The machine learning program according to claim 1, characterized in that the substitution process replaces the first training data with specific training data, which is identified based on the material structure of the molecule shown by the first training data among the plurality of training data, and in which the number of atoms is less than the threshold.

3. The machine learning program according to claim 2, characterized in that the substitution process replaces the number of atoms in the first training data with a specific training data from among the plurality of training data whose application is the same as that of the molecular material structure represented by the first training data.

4. The machine learning program according to claim 1 or 2, characterized in that the replacement process involves sorting each of the multiple training data in descending order of the number of atoms, determining whether the training data in descending order of the number of atoms satisfies the conversion condition that the number of atoms is equal to or greater than the threshold and the number of converted training data is less than the threshold, and if the conversion condition is satisfied, replacing the training data to be replaced with training data having fewer atoms than the training data to be replaced.

5. A machine learning method in which a computer performs machine learning on a machine learning model that extracts molecular structure features using multiple different molecules as training data, characterized in that the number of atoms contained in the molecules represented by each of the multiple training data is obtained, and if there is a first training data among the obtained number of atoms of each of the multiple training data that has a number of atoms equal to or greater than a threshold, the first training data is replaced with a number of atoms less than the threshold based on the other training data among the multiple training data.

6. An information processing device that performs machine learning on a machine learning model that extracts molecular structure features using multiple different molecules as training data, wherein the information processing device has a control unit that acquires the number of atoms contained in the molecules indicated by each of the multiple training data, and if there is a first training data among the acquired number of atoms of each of the multiple training data that has a number of atoms equal to or greater than a threshold, the first training data is replaced with another number of atoms that is less than the threshold, based on the training data other than the first training data among the multiple training data.

Images