State estimation device, state estimation method, and state estimation program

Abstract

To cause interactions between unit components in both short distance and long distance to be appropriately reflected.SOLUTION: A state estimation device takes as inputs the type and position of each point of a point cloud part which is a finite part of a point cloud structure in which one or more types of point infinitely expand in a space and estimates the state of each point of the point cloud part. The state estimation device comprises: an acquisition unit for acquiring the type and position of an inputted point as input information; and a state estimation unit for estimating a feature vector corresponding to the input information by a state estimation model which is a learning model having been trained by machine learning about a relationship between the type and position of points in the point cloud part and a feature vector representing the state of each point of the point cloud part.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a state estimation device, a state estimation method, and a state estimation program. [Background technology]

[0002] The substances that make up various materials such as permanent magnets, superconductors, and batteries are represented by a pattern of atomic arrangement in three-dimensional space called a crystal structure. The physical properties of such materials, such as the formation energy and band gap, can be calculated by simulation based on the crystal structure. However, since simulations based on the crystal structure require a large amount of calculation time, methods for quickly predicting physical properties using machine learning models are being researched.

[0003] For example, Non-Patent Document 1 describes a method in which a machine learning model using a graph neural network (GNN) is used to repeatedly calculate interactions between atoms in close proximity in a crystal structure in real space, calculate the atomic state as a feature vector, and predict physical properties, etc. based on the calculated feature vector. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Tian Xie and Jeffrey C. Grossman, "Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties", Phys. Rev. Lett. 120, 145301 Published 6 April 2018. [Non-Patent Document 2] Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 15066-15080. Curran Associates, Inc., 2022. [Non-Patent Document 3] Arthur Kosmala, Johannes Gasteiger, Nicholas Gao, Stephan Gunnemann (u with umlaut), “Ewald-based Long-Range Message Passing for Molecular Graphs”, arXiv:2303.04791, 2023. [Non-Patent Document 4] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? In Advances in Neural Information Processing Systems, volume 34, pp. 28877-28888. Curran Associates, Inc., 2021. Summary of the Invention [Problem to be solved by the invention]

[0005] Interactions between atoms include interactions that extend over relatively long distances in the unit lattice arrangement that constitutes a crystal, such as Coulomb force. It is difficult to appropriately incorporate such long-distance interactions with the methods described in Non-Patent Documents 1 and 2. Non-Patent Document 3 describes a method for incorporating long-range interactions by incorporating a Fourier transform into a GNN, but since it is composed of a GNN based on a finite graph structure, an infinitely expanding structure is not directly formulated. In addition, Non-Patent Document 4 incorporates interactions between atoms in a molecular structure, but since a crystal structure is a structure that extends infinitely in space, this method cannot be applied to a crystal structure.

[0006] The present invention has been made in consideration of the above, and aims to provide a state estimation device, a state estimation method, and a state estimation program that are capable of appropriately reflecting interactions at both short and long distances between each point that makes up an infinitely expanding point cloud structure. [Means for solving the problem]

[0007] A state estimation device according to the present invention is a state estimation device that estimates a state of each of the points in a point cloud part, which is a finite part of a point cloud structure in which one or more types of points extend infinitely in space, by inputting the type and position of each of the points in the point cloud part, and includes an acquisition unit that acquires the input type and position of the point as input information, and a state estimation unit that estimates the feature vector corresponding to the input information using a state estimation model that is a learning model that has been machine-learned to learn a correspondence between the type and position of the point in the point cloud part and a feature vector that represents the state of each of the points in the point cloud part, and the state estimation model has been machine-learned to estimate the feature vector based on a calculation result obtained by calculating an interaction between each of the points in the point cloud part and other points in the point cloud structure in real space, in an inverse space obtained by performing a Fourier transform on the real space, or in both the real space and the inverse space, using a function defined to include an infinite series sum of an interaction function that attenuates the state of each of the points in the point cloud part depending on the distance from the other points in the point cloud structure.

[0008] A state estimation method according to the present invention is a state estimation method for estimating a state of each of a point cloud part, which is a finite part of a point cloud structure in which one or more types of points extend infinitely in space, by inputting the type and position of each of the points in the point cloud part, the state estimation method including: an acquisition step of acquiring the type and position of the input point as input information; and a state estimation step of estimating the feature vector corresponding to the input information by a state estimation model that is a learning model that has been machine-learned to learn a correspondence between the type and position of the point in the point cloud part and a feature vector representing the state of each of the points in the point cloud part, the state estimation model being machine-learned to estimate the feature vector based on a calculation result obtained by calculating an interaction between each of the points in the point cloud part and other points in the point cloud structure in a real space, an inverse space obtained by performing a Fourier transform on the real space, or both the real space and the inverse space, by a function defined to include an infinite series sum of an interaction function that attenuates the state of each of the points in the point cloud part according to the distance from the other points in the point cloud structure.

[0009] A state estimation program according to the present invention is a state estimation program that estimates a state of each of a point cloud part, which is a finite part of a point cloud structure in which one or more types of points extend infinitely in space, by inputting the type and position of each of the points in the point cloud part. The state estimation program causes a computer to execute an acquisition process of acquiring the type and position of the input point as input information, and a state estimation process of estimating the feature vector corresponding to the input information using a state estimation model that is a learning model that has been machine-learned to learn a correspondence between the type and position of the point in the point cloud part and a feature vector that represents the state of each of the points in the point cloud part, and the state estimation model has been machine-learned to estimate the feature vector based on a calculation result obtained by calculating an interaction between each of the points in the point cloud part and other points in the point cloud structure in real space, in an inverse space obtained by Fourier transforming the real space, or in both the real space and the inverse space, using a function defined to include an infinite series sum of an interaction function that attenuates the state of each of the points in the point cloud part depending on the distance from the other points in the point cloud structure. Effect of the Invention

[0010] According to the present invention, it is possible to provide a state estimation device, a state estimation method, and a state estimation program that are capable of appropriately reflecting interactions at both short and long distances between each point that constitutes a point cloud structure. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a state estimating device according to the present embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing an example of a state estimation device according to the present embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a state estimation model. [Figure 4] FIG. 4 is a diagram illustrating an example of processing in the self-attention layer. [Diagram 5]FIG. 5 is a diagram illustrating an example of processing in the self-attention layer. [Figure 6] FIG. 6 is a diagram showing an example of the arrangement of a plurality of self-attention blocks provided in a state estimation model. [Figure 7] FIG. 7 is a diagram showing an example of the arrangement of a plurality of self-attention blocks provided in a state estimation model. [Figure 8] FIG. 8 is a diagram showing an example of the arrangement of a plurality of self-attention blocks provided in a state estimation model. [Figure 9] FIG. 9 is a diagram showing an example of the arrangement of a plurality of self-attention blocks provided in a state estimation model. [Figure 10] FIG. 10 is a graph showing an example of the magnitude of interaction between atoms calculated by multiple self-attention blocks. [Figure 11] FIG. 11 is a flowchart showing an example of a state estimation method according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of a state estimation device, a state estimation method, and a state estimation program according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments. In addition, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same.

[0013] FIG. 1 is a diagram showing a schematic diagram of an example of a state estimation device 100 according to the present embodiment. FIG. 2 is a functional block diagram showing an example of the state estimation device 100 according to the present embodiment. The state estimation device 100 shown in FIG. 1 and FIG. 2 estimates the state of each point of a point cloud portion, which is a finite part of a point cloud structure in which one or more types of points extend infinitely in space. In the present embodiment, as an example of such a point cloud structure, a substance having a three-dimensional crystal structure will be described. In this case, each point is an atom constituting the substance. Also, the point cloud portion is a unit lattice of the crystal structure. As shown in FIG. 2, the state estimation device 100 includes a processing unit 10, a storage unit 20, and a communication unit 30.

[0014] The processing unit 10 performs various information processing. The processing unit 10 includes a processor such as a CPU (Central Processing Unit) and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processing unit 10 includes an acquisition unit 11, a state estimation unit 12, and a physical property estimation unit 13.

[0015] The acquisition unit 11 acquires the type and position of the input point as input information. In this embodiment, for a substance to be estimated, configuration information indicating the arrangement in real space of atoms included in a unit lattice and a lattice vector in the real space of the unit lattice are acquired as input information. The configuration information and the lattice vector can be input using, for example, an input device (not shown). The configuration information includes information on the type of atom and the position of the atom. The position information of the atom can be, for example, coordinates in a Cartesian coordinate system in which the starting point of the lattice vector is the origin.

[0016] The state estimation unit 12 estimates a feature vector based on the input information (arrangement information, lattice vector) acquired by the acquisition unit 11. The feature vector is a vector that indicates, as a feature, a state of an atom included in a unit lattice of a substance based on interactions between the atom and other atoms of the substance in real space and interactions in reciprocal space. The reciprocal space is a space obtained by performing a Fourier transform on the real space. The state estimation unit 12 estimates the feature vector using a state estimation model 40 described below. The state estimation unit 12 outputs the estimated feature vector.

[0017] The state estimation model 40 is machine-learned to estimate a feature vector based on the calculation results obtained by calculating the interactions between each atom of the unit cell and other atoms in the crystal structure in real space, in reciprocal space obtained by Fourier transforming the real space, or in both real space and reciprocal space, using a function defined to include an infinite series sum of interaction functions in which the state of each atom of the unit cell attenuates depending on the distance from the other atoms in the crystal structure.

[0018] In the following, in this embodiment, the principle of estimating the state of each atom of a finite structure (target structure) in an infinitely expanding structure will be described. Let S={1, 2, 3, ...} be the division number of the atom in the infinite structure, and X be the type of the atom in the structure S. 0 ={x1 0 ,x2 0 ,x3 0 ,…}, the position of an atom in the structure S is P={p1,p2,p3,…}, and the type of atom X 0 Let X = {x1, x2, x3, ...} be the input information based on the position P of the atom, and let T ⊂ S be a part of S consisting of N finite atoms. Let f be the interaction function that attenuates for atom i in structure S, which is the target atom of the target structure T, according to the distance between the atom i and another atom j in structure S. ij Let y be the feature vector that indicates the state of atom i. i Then, the feature vector y i is the total interaction coefficient Z i and the total interaction vector a i A function g that receives and returns a vector i Using

number

number

number

number

number

[0019] When the infinite structure S has a periodic crystal structure, the type of atoms X 0 and the atomic position P is a finite basic repeating element S cell and D translation direction vectors L={l1,…,lD}, a D-dimensional integer vector n, and an atom i(n) at a position moved by a D-dimensional translation Ln,

number

number

number

number

[0020] Total interaction coefficient Z in real space i and the total interaction vector a i is the coordinate variable r = (x1, ..., x d ), using

number

number

number

number

[0021] Total interaction coefficient Z in reciprocal space i and the total interaction vector a i is the spatial angular frequency vector ω m Using the Fourier series expansion by

number

number

number

number

number

[0022] Here, the principle of state estimation according to this embodiment will be described when applied to a Transformer encoder. When the principle of state estimation according to this embodiment is applied to a Transformer encoder, in the real space, a feature vector y i teeth,

number

number

number

number

number

number

number

number

number

[0023] In addition, when the principle of state estimation according to this embodiment is applied to the Transformer encoder, in the inverse space, the feature vector y i teeth,

number

number

number

number

number

number

number

[0024] When the principle of state estimation according to this embodiment is applied to a Transformer encoder, the state estimation model 40 is Q , W K , W V There are H different query vectors q i , key vector k i and the value vector v i The interactions are calculated in parallel, and the results are combined to produce a feature vector y i =(y i 1 , y i 2 , …, y i H ) In this case, the state estimation unit 12 inputs the feature vector estimated by the state estimation model to another neural network NN, and calculates the following as the output value X'. X´ = NN(y1, y2, …, y N ) In this case, the state estimation unit 12 uses X', which is the output value of the neural network NN, as input information, and calculates a function g i Then, the feature vector y i As shown above, we can calculate H query vectors q i , key vector k i and the value vector v i Since the interactions can be calculated using these in parallel, the amount of calculation can be reduced and the feature vector can be estimated with high accuracy.

[0025] A specific example to which the above principle is applied will be described below. The state estimation model 40 according to this embodiment estimates a feature vector using a calculation configuration of QKV in an encoder of a deep learning model called Transformer, which is used in language translation, for example. In the state estimation model 40, the lattice vector, which is input information, is treated as a token in Transformer, and the arrangement information (position) of each atom is treated as information indicating the position of the token. The state estimation model 40 estimates a feature vector taking into account interactions occurring between atoms using a calculation configuration of QKV in Transformer. The state estimation model 40 estimates a feature vector based on one or both of a function indicating interactions in real space and a function indicating interactions in reciprocal space. This makes it possible to efficiently estimate a feature vector taking into account both strong interactions occurring between atoms in close proximity and interactions that cannot be ignored even if the atoms are far apart. The state estimation model 40 is generated by connecting a feature vector output when certain input information is input to a neural network that estimates physical properties using the feature vector, and by performing learning using various well-known learning methods such as backpropagation so as to reduce the error between the predicted physical property values ​​and the correct values.

[0026] FIG. 3 is a diagram showing a schematic example of a state estimation model 40 according to the present embodiment. As shown in FIG. 3, the state estimation model 40 has a plurality of self-attention blocks 41 that estimate feature vectors in stages. As the plurality of self-attention blocks 41, for example, a self-attention block, a multi-head attention block, or the like applied to an encoder of a deep learning model called Transformer used in language translation or the like can be applied. The self-attention block 41 includes a self-attention layer 41a and a transformation layer 41b. The self-attention block 41 may further include a multi-layer perceptron (MLP). In the present embodiment, by repurposing the configuration of the self-attention block 41 arranged in the encoder of Transformer, it is possible to effectively incorporate both short-range and long-range interactions for atoms.

[0027] Next, an example of the processing in the self-attention layer 41a will be described. FIG. 4 is a diagram showing a schematic diagram of an example of the processing in the self-attention layer 41a. As shown in FIG. 4, in the self-attention layer 41a, an input vector x (x1, x2, ..., x i ,x n ), query q(q1,q2,…,q i ,q n ), key k(k1, k2, …, k j ,k n ), values ​​v(v1, v2, …, v j ,v n )

[0028] The self-attention layer 41a calculates the input vector x i Query q corresponding to i For key k, the relevance q i T k j Calculate the probability distribution p i The self-attention layer 41a calculates the calculated probability distribution pi Based on the value v j The expected value between i Output y i Let us assume that.

[0029] By providing multiple self-attention layers 41a, the feature vector can be estimated in stages.

[0030] A substance having a three-dimensional crystal structure has a structure in which unit lattices are repeated multiple times in three dimensions. Therefore, the self-attention layer 41a obtains a repeating pattern in which unit lattices are repeated multiple times, and estimates a feature vector for each atom included in a reference unit lattice in the repeating pattern based on the interaction between the atom and other atoms included in the repeating pattern. When obtaining a repeating pattern, the self-attention layer 41a can calculate the position information of the atoms included in the repeating pattern based on the input position information.

[0031] 5 is a diagram showing an example of processing in the self-attention layer 41a. As shown in FIG. 5, for example, for atoms a, b, and c, an input vector x(x a ,x b ,x c ), the self-attention layer 41a will a ,q b ,q c ), key k(k i a ,k i b ,k i c ), value v(v i a ,v i b ,v i c The self-attention layer 41a generates position information P i K , P i V is calculated as preprocessing.i K is the key k(k i a ,k i b ,k i c ) is a vector to be added to the position information P i V is the value v(v i a ,v i b ,v i c ) is a vector to be added to the calculated position information P i K , P i V to the key k and the value v, respectively, and then performs an estimation process of the feature vector y. This makes it possible to appropriately reflect the interactions between atoms in a repeating pattern in which unit lattices are repeated multiple times. Note that a three-dimensional crystal structure is composed of unit lattices repeated multiple times, and the state of atoms in each unit lattice is equivalent for each unit lattice that is repeated. Therefore, the self-attention layer 41a calculates the feature vector y by calculating the states of atoms a, b, and c in the unit lattice (y a ,y b ,y c ) can be estimated. This allows the self-attention layer 41a to appropriately reflect the influence of atoms (atoms located close and far away) that compose countless other unit lattices around the unit lattice of interest, by simply using the states of atoms a, b, and c in the unit lattice as basic variables and utilizing the position information of the atoms and the property of the periodic arrangement of the unit lattice.

[0032] 6 to 9 are diagrams showing examples of arrangements of multiple self-attention blocks provided in a state estimation model.

[0033] 6 and 7, the state estimation models 40A and 40B have a real space attention block 42 and a reciprocal space attention block 43 as multiple self-attention blocks 41. The real space attention block 42 is a self-attention block that estimates feature vectors for atoms included in a unit lattice to be estimated based on interactions between the atoms and other atoms in real space. When calculating interactions by finding repetitive patterns in the real space attention block 42, processing is performed in order from atoms with small interatomic distances to atoms with large interatomic distances.

[0034] The reciprocal space attention block 43 is a self-attention block that estimates feature vectors for atoms included in a unit lattice to be estimated based on interactions between the atoms and other atoms in the reciprocal space. In the reciprocal space attention block 43, a reciprocal lattice vector is calculated by Fourier transforming the lattice vector, and the feature vector is estimated using the reciprocal lattice vector. In the reciprocal space attention block 43, a variable corresponding to an atomic distance becomes a frequency. That is, in the reciprocal space attention block 43, processing is performed in order from atoms with small interatomic frequencies to atoms with large interatomic frequencies. Therefore, in the reciprocal space attention block 43, processing is performed in order from atoms with large interatomic distances in real space to atoms with small interatomic distances.

[0035] By providing the real space attention block 42 and the reciprocal space attention block 43 separately, it is possible to effectively reflect in the feature vector both the influence of the interaction between atoms with short interatomic distances and the influence of the interaction between atoms with long interatomic distances. Note that the learning parameters of the real space attention block 42 and the reciprocal space attention block 43 can be configured to be independently learned when the state estimation model 40 is learned.

[0036] The state estimation models 40A and 40B are configured to alternate between processing by the real space attention block 42 and processing by the inverse space attention block 43. The state estimation model 40A shown in Fig. 6 is configured to first perform processing by the real space attention block 42. The state estimation model 40B shown in Fig. 7 is configured to first perform processing by the inverse space attention block 43.

[0037] 8 is configured such that all of the multiple self-attention blocks 41 are real space attention blocks 42, and no inverse space attention block 42 is provided. The state estimation model 40C is configured to continuously perform processing by the real space attention blocks 42. Note that the learning parameters of the real space attention blocks 42 can be configured to be independently learned when the state estimation model 40 is learned.

[0038] The state estimation model 40D shown in FIG. 9 has a multi-head attention block 44 as the self-attention block 41. The multi-head attention block 44 estimates feature vectors in parallel for atoms included in the atomic unit lattice to be estimated based on the interactions between the atoms and other atoms in real space and reciprocal space. A plurality of multi-head attention blocks 44 are provided. The plurality of multi-head attention blocks 44 estimate the feature vectors in stages. The learning parameters of the multi-head attention blocks 44 can be configured to be independently learned when the state estimation model 40 is trained.

[0039] The above-mentioned various self-attention blocks 41 (real space attention block 42, reversal space attention block 43, multi-head attention block 44) may be configured in various ways, for example, according to the type of atoms constituting the substance to be estimated or the type of interaction occurring between atoms. In addition, when estimating a feature vector using a state estimation model and when learning the state estimation model, any setting can be made according to the allowable calculation resources, the required accuracy, and the like.

[0040] FIG. 10 is a graph showing an example of the magnitude of interaction between atoms calculated by a plurality of self-attention blocks 41. In FIG. 10, a certain atom is taken as an example, and the magnitude of interaction with other atoms is shown according to the interatomic distance. In each of the upper and lower parts of FIG. 10, the horizontal axis indicates the interatomic distance, and the vertical axis indicates the magnitude of interaction. In the upper part of FIG. 10, a curve L1 shown by a solid line indicates the magnitude of interaction calculated for the real space. In the lower part of FIG. 10, a curve L2 shown by a dashed line indicates the magnitude of interaction calculated for the reciprocal space. As shown in FIG. 10, by calculating the interaction in the reciprocal space, it is possible to appropriately reflect long-distance interactions that cannot be sufficiently ignored by only calculating the interaction in the real space.

[0041] In addition, the real space attention block 42 and the reciprocal space attention block 43 can set the distance range from the atom of the reference unit lattice to the atom for which the interaction between the atom and the atom is to be calculated individually. In other words, the real space attention block 42 and the reciprocal space attention block 43 can set the calculation range for the distance to which the interaction with other atoms is calculated with respect to the atom included in the reference unit lattice. FIG. 10 shows a case where the range of the interatomic distance when calculating with the real space attention block 42 is R1 (for example, a range from a short distance to a fixed distance D1 with respect to the reference atom), and the range of the interatomic distance when calculating with the reciprocal space attention block 43 is R2 (for example, a range from infinity to a fixed distance D2). For example, the calculation ranges of the real space attention block 42 and the reciprocal space attention block 43 can be set to be different when the estimation target is a semiconductor or metal and when the estimation target is an insulator. For example, when the estimation target is a semiconductor or metal, the calculation range of the real space attention block 42 can be narrowed and the calculation range of the reciprocal space attention block 43 can be widened. Also, for example, when the estimation target is an insulator, the calculation range of the real space attention block 42 can be widened and the calculation range of the inverse space attention block 43 can be narrowed.

[0042] The physical property estimation unit 13 can estimate the physical property of the substance based on the estimated feature vector. The physical property estimation unit 13 can estimate the physical property using the physical property estimation model 50. The physical property estimation model 50 is a learning model in which the feature vector output from the state estimation unit 12 is used as an explanatory variable, the physical property of the substance is used as an objective variable, and the correspondence between the explanatory variable and the objective variable is machine-learned. The physical property estimation unit 13 does not need to use the physical property estimation model 50 when estimating the physical property. The physical property of the substance may be, for example, a physical property value such as a band gap or formation energy, a category distribution indicating a material category or the like (for example, "metal, nonmetal", "superconducting, metal, nonmetal", etc.), or a feature vector of a crystal structure. In addition, it may be a vector sequence indicating the physical property of each atom of the substance. The feature vector output from the state estimation unit 12 may be used as an input of another machine learning model or the like. The state estimation unit 12 may estimate another state of the atom.

[0043] The storage unit 20 stores information such as various programs and data. The storage unit 20 includes storage such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 20 stores, for example, the state estimation model 40 and the physical property estimation model 50 described above.

[0044] The storage unit 20 stores a state estimation program for estimating the state of each point of a point cloud part, which is a finite part of a point cloud structure in which one or more types of points extend infinitely in space, by inputting the type and position of each point of the point cloud part, the state estimation program causing a computer to execute an acquisition process for acquiring the type and position of the input point as input information, and a state estimation process for estimating a feature vector corresponding to the input information by a state estimation model 40, which is a learning model that has machine-learned the correspondence between the type and position of the point in the point cloud part and the feature vector representing the state of each point of the point cloud part. In this state estimation program, the state estimation model 40 is machine-learned to estimate a feature vector based on a calculation result obtained by calculating the interaction between each point of the point cloud part and other points in the point cloud structure in the real space, in the inverse space obtained by Fourier transforming the real space, or in both the real space and the inverse space, by a function determined to include an infinite series sum of an interaction function that attenuates the state of each point of the point cloud part according to the distance from the other points in the point cloud structure.

[0045] The communication unit 30 performs wired or wireless communication with an external device and includes an interface such as a network interface card.

[0046] When generating the state estimation model 40 and the physical property estimation model 50, for example, a composite model may be configured in which a feature vector output from the state estimation model 40 is input to the physical property estimation model 50, and machine learning may be performed on this composite model. That is, a composite model is generated in which the substance arrangement information and lattice vector input to the state estimation model 40 are used as explanatory variables, and the physical properties of the substance output from the physical property estimation model 50 are used as objective variables, and the correspondence between the explanatory variables and the objective variables is machine-learned. In this case, in the composite model, the state estimation model 40 in which the substance arrangement information and lattice vector are used as explanatory variables and the feature vector of the substance is used as the objective variable, and the physical property estimation model 50 in which the feature vector of the substance is used as the explanatory variable and the physical properties of the substance are used as the objective variable are individually generated.

[0047] Next, a state estimation method according to this embodiment will be described. Fig. 11 is a flowchart showing an example of the state estimation method according to this embodiment. As shown in Fig. 11, the state estimation method according to this embodiment includes an acquisition step S10, a state estimation step S20, and a state estimation step S30.

[0048] In the acquisition step S10, for the substance to be estimated, configuration information indicating the arrangement in real space of atoms included in a unit lattice and a lattice vector in real space of the unit lattice are acquired as input information. For example, when an operator inputs the configuration information and the lattice vector to the state estimation device 100 using an input device (not shown) or the like, the acquisition unit 11 of the processing unit 10 acquires the input configuration information and the lattice vector as input information.

[0049] In the state estimation step S20, a feature vector for the estimation target is estimated based on the above-mentioned state estimation model 40 and the input information acquired in the acquisition step S10. The state estimation unit 12 inputs the arrangement information and the lattice vector, which are the input information, to the state estimation model 40, and acquires the feature vector, which is the estimation result by the state estimation model 40. The state estimation unit 12 outputs the acquired feature vector.

[0050] In the physical property estimation step S30, the physical properties of the substance are estimated based on the feature vector estimated in the state estimation step S20. The physical property estimation unit 13 inputs the feature vector output from the state estimation unit 12 to the physical property estimation model 50, and obtains the physical properties that are the estimation results by the physical property estimation model 50.

[0051] Table 1 below shows an example of the mean absolute error when estimating each physical property value of the crystal structure when the "Materials Project" dataset is machine-learned. Examples of physical properties include formation energy, band gap, bulk modulus, and shear modulus. Table 2 also shows an example of the mean absolute error when estimating each physical property value of the crystal structure when the "JARVIS-DFT 3D 2021" dataset is machine-learned. Examples of physical properties include formation energy, total energy, band gap (OPT) obtained using "the OptB88vdW functional", band gap (MBJ) obtained using "Tran-Blaha modified Becke-Johnson potential", and Energy Above Hull (E hull). In Tables 1 and 2, the three numbers under each physical property name indicate the size of the dataset subset for training, validation, and testing, from left to right. [Table 1] [Table 2]

[0052] In addition, in Tables 1 and 2, Comparative Examples 1 and 7 are the mean absolute errors of the estimation results by the method described in "Tian Xie and Jeffrey C. Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett., 120:145301, Apr 2018." Comparative Examples 2 and 8 are the mean absolute errors of the estimation results by the method described in "KT Schutt (u with umlaut), HE Sauceda, P.-J. Kindermans, A. Tkatchenko, and K.-R. Muller (u with umlaut). SchNet - A deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24):241722, 03 2018." Comparative Examples 3 and 9 are the mean absolute errors of the estimation results using the method described in "Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials, 31(9):3564-3572, May 2019."Comparative Examples 4 and 10 are the mean absolute errors of the estimation results by the method described in "Steph-Yves Louis, Yong Zhao, Alireza Nasiri, XiranWang, Yuqi Song, Fei Liu, and Jianjun Hu. Graph convolutional neural networks with global attention for improved materials property prediction. Phys. Chem. Chem. Phys., 22:18141-18148, 2020." Comparative Examples 5 and 11 are the mean absolute errors of the estimation results by the method described in "Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions. npj Computational Materials, 7(1):185, Nov 2021." Comparative Examples 6 and 12 are the mean absolute errors of the estimation results using the method described in "Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 15066-15080. Curran Associates, Inc., 2022."

[0053] Furthermore, in Tables 1 and 2, Examples 1 and 3 are mean absolute errors of the estimation results obtained by calculating the interactions between atoms in real space based on the above-mentioned principle of state estimation according to this embodiment. Examples 2 and 4 are mean absolute errors of the estimation results when the stochastic weighted average is not used in Examples 1 and 3. Example 5 is the mean absolute error of the estimation results obtained by calculating the interactions between atoms in real space and reciprocal space based on the above-mentioned principle of state estimation according to this embodiment.

[0054] As shown in Table 1, the mean absolute errors of the estimation results according to Examples 1 and 2 are lower than the mean absolute errors of the estimation results according to Comparative Examples 1 to 6 for all physical property values.

[0055] As shown in Table 2, the mean absolute errors of the estimation results according to Examples 3 to 5 are lower than the mean absolute errors of the estimation results according to Comparative Examples 1 to 6 for all physical property values. In addition, when Example 3 is compared with Example 5, the mean absolute errors for the band gap (OPT) and E hull are smaller in Example 5 than in Example 3. In this way, by calculating the interactions in both real space and reciprocal space, it is possible to obtain more accurate estimation results.

[0056] As described above, the state estimation device 100 according to the present embodiment is a state estimation device that estimates the state of each atom in a unit lattice, which is a finite part of a crystal structure in which one or more types of atoms extend infinitely in space, by inputting the type and position of each atom in the unit lattice. The state estimation device 100 includes an acquisition unit 11 that acquires the input type and position of the atom as input information, and a state estimation unit 12 that estimates a feature vector corresponding to the input information using a state estimation model 40 that is a learning model that has been machine-learned to learn a correspondence between the type and position of the atom in the unit lattice and a feature vector that represents the state of each atom in the unit lattice. The state estimation model 40 has been machine-learned to estimate a feature vector based on a calculation result obtained by calculating an interaction between each atom in the unit lattice and other atoms in the crystal structure in real space, in a reciprocal space obtained by Fourier transforming the real space, or in both real space and reciprocal space, using a function defined to include an infinite series sum of an interaction function that attenuates the state of each atom in the unit lattice depending on the distance from the other atoms in the crystal structure.

[0057] Moreover, the state estimation method according to the present embodiment is a state estimation method for estimating a state of each atom in a unit lattice, which is a finite part of a crystal structure in which one or more types of atoms extend infinitely in space, by inputting the type and position of each atom in the unit lattice. The state estimation method includes an acquisition step S10 for acquiring the type and position of the input atom as input information, and a state estimation step S20 for estimating a feature vector corresponding to the input information by a state estimation model 40, which is a learning model that has been machine-learned to learn a correspondence between the type and position of the atom in the unit lattice and a feature vector representing the state of each atom in the unit lattice. The state estimation model 40 has been machine-learned to estimate a feature vector based on a calculation result obtained by calculating an interaction between each atom in the unit lattice and other atoms in the crystal structure in real space, in a reciprocal space obtained by Fourier transforming the real space, or in both real space and reciprocal space, using a function defined to include an infinite series sum of an interaction function that attenuates the state of each atom in the unit lattice depending on the distance from the other atoms in the crystal structure.

[0058] Furthermore, the state estimation program according to this embodiment is a state estimation program that estimates the state of each atom in a unit lattice by inputting the type and position of each atom in the unit lattice, which is a finite part of a crystal structure in which one or more types of atoms extend infinitely in space. The state estimation program causes a computer to execute an acquisition process that acquires the type and position of the input atom as input information, and a state estimation process that estimates a feature vector corresponding to the input information using a state estimation model 40, which is a learning model that has been machine-learned to learn the correspondence between the type and position of the atom in the unit lattice and a feature vector representing the state of each atom in the unit lattice. The state estimation model 40 has been machine-learned to estimate a feature vector based on a calculation result obtained by calculating interactions between each atom in the unit lattice and other atoms in the crystal structure in real space, in a reciprocal space obtained by Fourier transforming the real space, or in both real space and reciprocal space, using a function defined to include an infinite series sum of an interaction function that attenuates the state of each atom in the unit lattice depending on the distance from the other atoms in the crystal structure.

[0059] According to this configuration, a state estimation model 40 is used that has been machine-learned to estimate a feature vector based on the calculation results obtained by calculating interactions between each atom of a unit lattice and other atoms in the crystal structure in real space, reciprocal space, or both real space and reciprocal space using a function that is determined to include an infinite series sum of interaction functions that attenuate the state of each atom of the unit lattice according to the distance between the atom and other atoms in the crystal structure, so that it is possible to estimate a feature vector that reflects the calculation results of interactions between atoms in an infinitely expanding crystal structure. Therefore, the effects of both interactions between atoms with short interatomic distances and interactions between atoms with long interatomic distances can be appropriately reflected in the feature vector.

[0060] In the state estimation device 100 according to this embodiment, the state estimation model 40 has a plurality of self-attention blocks 41 that estimate a feature vector in a step-by-step manner.

[0061] According to this configuration, since the feature vector is estimated stepwise by the multiple self-attention blocks 41, it is possible to estimate the feature vector with high accuracy and high speed. In this embodiment, by repurposing the configuration of the self-attention block 41 arranged in the encoder of a deep learning model called Transformer used in language translation, etc., it is possible to effectively incorporate both short-distance and long-distance interactions.

[0062] In the state estimation device 100 according to this embodiment, the self-attention block 41 determines a repeating pattern in which unit lattices are repeated multiple times, and estimates a feature vector for each atom included in a reference unit lattice in the repeating pattern based on interactions between the atoms and other atoms included in the repeating pattern.

[0063] According to this configuration, it is possible to perform processing corresponding to the structure of a substance in which unit lattices containing atoms are repeatedly arranged in multiple layers, and therefore it is possible to estimate a feature vector with high accuracy.

[0064] In the state estimation device 100 according to this embodiment, the self-attention block 41 calculates position information of atoms included in the repetitive pattern based on the configuration information.

[0065] According to this configuration, position information of atoms included in the repetitive pattern can be appropriately calculated.

[0066] In the state estimation device 100 of this embodiment, the multiple self-attention blocks 41 include a real space attention block 42, which is a self-attention block that estimates feature vectors for atoms included in the unit lattice to be estimated based on interactions between other atoms in real space, and a reciprocal space attention block 43, which is a self-attention block that estimates feature vectors for atoms included in the unit lattice to be estimated based on interactions between other atoms in reciprocal space.

[0067] According to this configuration, the real space attention block 42 and the reverse space attention block 43 process the influence of the interaction between the real space and the reverse space separately, so that the processing can be performed efficiently.

[0068] In the state estimation device 100 according to this embodiment, the state estimation model 40 is configured to alternately perform estimation by a real space attention block 42 and estimation by a reciprocal space attention block 43.

[0069] According to this configuration, by alternately performing estimation by the real space attention block 42 and estimation by the reverse space attention block 43, interactions in real space and reverse space can be appropriately obtained.

[0070] In the state estimation device 100 according to this embodiment, the state estimation model 40 is configured to first perform estimation by the real space attention block 42.

[0071] According to this configuration, by first performing estimation by the real space attention block 42, it is possible to appropriately reflect the influence of interactions in the real space, which are more dominant in the value of the feature vector.

[0072] In the state estimation device 100 according to this embodiment, the state estimation model 40 is capable of individually setting the distance range from an atom in a reference unit cell to an atom for which an interaction between the atom and the reference atom is to be calculated, in the real space attention block 42 and the reciprocal space attention block 43.

[0073] According to this configuration, the real space attention block 42 and the reciprocal space attention block 43 individually set the range of distances from the atom of the reference unit cell to the atom for which the interaction between the atom is calculated, so that the effects of both interactions between atoms with short interatomic distances and interactions between atoms with long interatomic distances can be appropriately reflected in the feature vector.

[0074] In the state estimation device 100 according to this embodiment, the multiple self-attention blocks 41 have a multi-head attention block 44 that estimates feature vectors in parallel for atoms included in the unit cell to be estimated based on their respective interactions with other atoms in real space and reciprocal space.

[0075] According to this configuration, the calculations of the interactions in the real space and the reciprocal space can be performed in parallel, so that the feature vector can be estimated with high accuracy while suppressing the amount of calculation.

[0076] The state estimation device 100 according to this embodiment further includes a physical property estimation unit 13 that estimates the physical properties of a substance based on the feature vector estimated by the state estimation unit 12.

[0077] According to this configuration, the physical properties of a substance are estimated based on a feature vector that appropriately reflects the provision of both interactions between atoms with short interatomic distances and interactions between atoms with long interatomic distances, thereby making it possible to estimate the physical properties with high accuracy.

[0078] The technical scope of the present invention is not limited to the above-mentioned embodiment, and appropriate modifications can be made without departing from the spirit of the present invention. For example, in the above-mentioned embodiment, the substance of the state estimation device 100 is described as a substance having a three-dimensional crystal structure. The substance of the state estimation device 100 is not limited to a substance having a three-dimensional crystal structure, and may be a one-dimensional object, a two-dimensional object, or an object having four or more dimensions, as long as a unit pattern including unit constituents is repeatedly arranged. In addition, the position of an atom input as input information may indicate a relative position. [Explanation of symbols]

[0079] L1, L2... curve, 10... processing unit, 11... acquisition unit, 12... state estimation unit, 13... state estimation unit, 20... memory unit, 30... communication unit, 40, 40A, 40B, 40C, 40D... state estimation model, 41... self-attention block, 42... real space attention block, 43... inverse space attention block, 44... multi-head attention block, 50... state estimation model, 100... state estimation device

Claims

1. A state estimation device that estimates a state of each of one or more types of points in a point cloud portion, the point cloud portion being a finite part of a point cloud structure in which the one or more types of points extend infinitely in space, by inputting the type and position of each of the points in the point cloud portion, the device comprising: an acquisition unit that acquires the type and position of the point to be input as input information; a state estimation unit that estimates the feature vector corresponding to the input information using a state estimation model that is a learning model that has been machine-learned to learn correspondences between types and positions of the points in the point cloud portion and feature vectors that represent states of the points in the point cloud portion; Equipped with The state estimation model is The feature vector is estimated based on a calculation result obtained by calculating an interaction between each of the points in the point cloud portion and other points in the point cloud structure in a real space, an inverse space obtained by Fourier transforming the real space, or both the real space and the inverse space, using a function defined to include an infinite series sum of an interaction function in which the state of each of the points in the point cloud portion is attenuated according to the distance from the other points in the point cloud structure. State estimator.

2. The type of the point is X ０ = {x １ ０ , x ２ ０ , x ３ ０ ,…}year, The position of the point is P = {p １ , p ２ , p ３ ,…}year, The type of point X ０ and the input information based on the position P of the point is expressed as X={x １ , x ２ , x ３ ,…}year, The interaction function attenuated according to the distance between the point i, which is a target point of the point cloud part in the point cloud structure, and another point j of the point cloud structure is defined as f ｉｊ year, The feature vector indicating the state of the point i is expressed as y ｉ Then, The feature vector y ｉ is the total interaction coefficient Z ｉ and the total interaction vector a ｉ A function g that receives and returns a vector ｉ Using [0010] It is shown that The total interaction coefficient Z ｉ teeth, [0025] It is expressed as The total interaction vector a ｉ teeth, [0030] It is expressed as An interaction vector v from point j to point i ｉｊ teeth, [0045] It is expressed as The interaction function f ｉｊ teeth, [0050] is The state estimation device according to claim 1 .

3. When the point group structure has periodicity, the point type X ０ and the position P of the point is a set of N finite basic repeating elements S ｃｅｌｌ and D translation direction vectors L = {l １ , …, l Ｄ }, a D-dimensional integer vector n, and a point i(n) moved by a D-dimensional translation Ln, [006] [0070] is shown in The point group part is divided into the basic repeating elements S ｃｅｌｌ Then, the feature vector y of the point i which is the target atom ｉ In the above, Z ｉ and a ｉ teeth, [0080] [0090] Represented by The state estimation device according to claim 2 .

4. The total interaction coefficient Z ｉ and the total interaction vector a ｉ teeth, The coordinate variable r = (x １ , …, x ｄ ), using [0010] ##EQU00011## It is expressed as f' is an interaction function that takes into account the periodicity ｉｊ , f' ｉｊ teeth, ##EQU00012## ##EQU00013## It is expressed as The total interaction coefficient Z in the reciprocal space ｉ and the total interaction vector a ｉ is the spatial angular frequency ω ｍ Using the Fourier series expansion by ##EQU14## ##EQU00015## It is expressed as The spatial angular frequency ω ｍ teeth, ##EQU00016## It is expressed as The interaction function f' taking into account the periodicity ｉｊ , f' ｉｊ F' is the Fourier coefficient of ｉｊ , F' ｉｊ teeth, ##EQU00017## [0018] Represented by The state estimation device according to claim 3 .

5. In the real space, the feature vector y ｉ teeth, [0019] It is expressed as The interaction function f ｉｊ (r) is the query vector q ｉ , key vector k ｉ and the value vector v ｉ , and the radius σ that determines the attenuation range ｉ Using [0020] It is expressed as σ ｉ is a constant or a radius r that determines the scale of the attenuation range ０ Using the function ρ that returns a positive value and the learning parameters w (scalar) and w (bold, vector), ##EQU00021## or, [0022] Define it as: The distance variable r is [0023] It is expressed as The query vector q ｉ , key vector k ｉ and the value vector v ｉ is the learning parameter W of the state estimation model Ｑ , W Ｋ , W Ｖ Using ##EQU00024## It is expressed as The interaction vector v ｉｊ is a constant W Ｇ and a K-dimensional one-hot vector b indicating the distance between the two points, [0025] It is expressed as The one-hot vector b is a vector that can express a maximum distance r ｍａｘ Using [0026] It is expressed as constant μ ｋ teeth, [0027] Represented by The state estimation device according to claim 3 .

6. In the inverse space, the feature vector y ｉ teeth, [0028] It is expressed as The interaction function f' taking into account the periodicity ｉｊ F' is the Fourier coefficient of (r) ｉｊ (ω ｍ ) is the query vector q ｉ , key vector k ｉ and the value vector v ｉ and the radius σ that determines the attenuation range ｉ bar ("σ ｉ " with "-" on top, [0029] [0030] It is expressed as σ ｉ The bars are either a constant or a radius r that determines the scale of the attenuation range. ０ Bar ("r ０ "), a function ρ that returns a positive value, and learning parameters w (a "-" over "w", a scalar) and w (a "-" over a bold "w", a vector), [0031] or, [0032] Define it as: The interaction function f'' taking into account the periodicity ｉｊ F' is the Fourier coefficient of (r) ｉｊ (ω ｍ )teeth, [Equation 33] It is expressed as The query vector q ｉ , the key vector k ｉ and the value vector v ｉ is the learning parameter W of the state estimation model Ｑ , W Ｋ , W Ｖ Using [0034] Represented by The state estimation device according to claim 5 .

7. The state estimation model is the learning parameter W Ｑ , W Ｋ , W Ｖ There are H different query vectors q ｉ , key vector k ｉ and the value vector v ｉ The interactions are calculated using parallel parallel processing, and the calculation results are combined to obtain the feature vector y ｉ () ｉ １ y ｉ ２ 、…、y ｉ Ｈ ) Calculate The state estimation device according to claim 6.

8. The state estimation unit inputs the feature vector estimated by the state estimation model to another neural network NN and outputs the result as an output value X'. X´=NN(y １ y ２ 、…、y Ｎ ) Calculate The state estimation device according to claim 7.

9. The state estimation unit uses X′, which is an output value of the neural network NN, as the input information, and calculates the function g ｉ The feature vector y ｉ Calculate The state estimation device according to claim 8.

10. A state estimation method for estimating a state of each of one or more types of points in a point cloud portion, which is a finite part of a point cloud structure in which the points extend infinitely in space, by inputting the type and position of each of the points in the point cloud portion, the method comprising: an acquisition step of acquiring the type and position of the point to be input as input information; a state estimation step of estimating the feature vector corresponding to the input information by a state estimation model which is a learning model that machine-learns a correspondence relationship between the type and position of the point in the point cloud part and a feature vector representing a state of each of the points in the point cloud part; Including, The state estimation model is The feature vector is estimated based on a calculation result obtained by calculating an interaction between each of the points in the point cloud portion and other points in the point cloud structure in a real space, an inverse space obtained by Fourier transforming the real space, or both the real space and the inverse space, using a function defined to include an infinite series sum of an interaction function in which the state of each of the points in the point cloud portion is attenuated according to the distance from the other points in the point cloud structure. State estimation methods.

11. A state estimation program for estimating a state of each of one or more types of points in a point cloud portion, which is a finite part of a point cloud structure in which the points extend infinitely in space, by inputting the type and position of each of the points in the point cloud portion, the program comprising: an acquisition process for acquiring the type and position of the point to be input as input information; a state estimation process of estimating the feature vector corresponding to the input information using a state estimation model that is a learning model that has been machine-learned to learn correspondences between the types and positions of the points in the point cloud portion and feature vectors that represent the states of each of the points in the point cloud portion; on the computer, The state estimation model is The feature vector is estimated based on a calculation result obtained by calculating an interaction between each of the points in the point cloud portion and other points in the point cloud structure in a real space, an inverse space obtained by Fourier transforming the real space, or both the real space and the inverse space, using a function defined to include an infinite series sum of an interaction function in which the state of each of the points in the point cloud portion is attenuated according to the distance from the other points in the point cloud structure. State estimation program.

Images