Quantum chemistry method of quantum Monte Carlo algorithm based on neural network

By introducing Fourier transform and high-frequency information modeling layers into the quantum Monte Carlo algorithm, the neural network's ability to capture electron states is improved, and the problem of the quantum Monte Carlo method's dependence on wave function assumption accuracy is solved, and the implementation of Higher computational accuracy and adaptability, especially in heavy atomic systems.

CN118917427AActive Publication Date: 2024-11-08PEKING UNIV

Patent Information

Application Number
CN202411117406.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-08
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

The accuracy of the quantum Monte Carlo method depends to a great extent on the accuracy of the wave function assumption, resulting in large errors in electronic structure calculations, especially when dealing with large-scale molecular systems, it is difficult to take into account both the calculation cost and the accuracy.

Method used

Using the quantum Monte Carlo algorithm based on neural network, the high-frequency information modeling layer is designed by pre-training the Transformer model and introducing the Fourier transform to enhance the modeling ability of the model to capture high-frequency changes in electronic states and to improve the modeling of wave functions. Accuracy and application performance.

Benefits of technology

It significantly improves the model's modeling accuracy and adaptability of electron behavior, especially capturing multi-scale features in heavy atomic systems, reduces computational costs, and improves the understanding and prediction ability of electron position uncertainty.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917427A_ABST
    Figure CN118917427A_ABST
Patent Text Reader

Abstract

The invention discloses a quantum chemistry method of a quantum Monte Carlo algorithm based on a neural network, and the method comprises the steps: carrying out the pre-training of a Transform model, and enabling an orbit outputted by the pre-trained Transform model to be matched with an orbit obtained through the solving of a Schrodinger equation in an atomic nucleus potential field through a Hartreree-Fock method; respectively designing a first high-frequency information modeling layer and a second high-frequency information modeling layer by utilizing Fourier transform; the first high-frequency information modeling layer serves as an input layer of a pre-trained Transform model, the second high-frequency information modeling layer serves as a middle layer of the pre-trained Transform model, an explicit modeling high-frequency information Transform model is obtained, and the explicit modeling high-frequency information Transform model takes an electronic position as input and takes a wave function as output; the method comprises the following steps of: training a Transform model of explicit modeling high-frequency information to obtain a trained Transform model of the explicit modeling high-frequency information; and carrying out prediction on a new electronic position by utilizing the trained Tranformer model of the explicit modeling high-frequency information to obtain a predicted wave function, and realizing modeling of an electronic behavior.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of quantum chemistry, and in particular is a quantum chemistry method based on a quantum Monte Carlo algorithm of a neural network. Background Art

[0002] The behavior of electrons is essentially strictly governed by the laws of quantum mechanics, and the core task of calculating electronic structure is to solve the Schrödinger equation in the nuclear potential field. Determining the electronic structure is to model the behavior of electrons. Specifically, the equation can be expressed as:

[0003]

[0004] Among them, Ψ(r,t) represents the wave function of the electron, is the Hamiltonian operator, r and t represent position and time respectively, i represents the unit imaginary number, represents the reduced Planck constant.

[0005] Electronic structure not only determines most of the properties of materials, but also affects the characteristics of chemical reactions. Therefore, in chemistry, condensed matter physics and related fields, the study of electronic structure plays an extremely important role. The study of electronic structure not only provides key information for understanding the basic properties of matter, but also provides a theoretical basis for the design and development of new materials and the analysis of chemical reaction mechanisms.

[0006] Given that the Schrödinger equation is a complex high-dimensional partial differential equation, the calculation of electronic structure can be roughly divided into two categories of methods, including empirical methods and first-principles methods. The empirical method simplifies the difficulty of solving the Schrödinger equation by introducing an alternative model. Although the empirical method cannot directly solve the Schrödinger equation, it provides relatively reasonable electronic structure calculation results and has a relatively simple calculation process. Therefore, they have been widely used in fields such as molecular dynamics. Machine learning-based methods also tend to adopt this strategy and have made some significant progress in electronic structure prediction. However, there is a large deviation between the energy obtained by this type of method and the actual energy, which is not suitable for occasions requiring high-precision calculations. Another type of method is the first-principles method, which obtains accurate electronic structure information by directly solving the Schrödinger equation. These methods are significantly better than empirical methods in terms of calculation accuracy, but they also bring higher calculation costs accordingly. For example, the post-Hartree-Fock method can achieve chemical accuracy, but its calculation overhead increases sharply with the increase of the size of the molecular system, and is usually limited to processing small and medium-sized molecular systems. Although density functional theory is fast in calculation speed, it is often not accurate enough in predicting chemical reaction processes, and its effect is highly dependent on the selected exchange-correlation functional. Different systems may require different functionals, which reduces the reliability of its predictions.

[0007] In the field of first-principles calculations, quantum Monte Carlo methods play an important role. The Monte Carlo technique is used to sample wave functions to calculate the expected value of physical quantities. Although its computational cost increases with the number of electrons (usually between the cubic and the fourth power), it is widely used in the study of medium-sized systems and solid-state systems. The accuracy of the quantum Monte Carlo method depends greatly on the accuracy of the wave function assumption. Summary of the invention

[0008] Purpose of the invention: In order to solve the problem that the accuracy of the quantum Monte Carlo method depends to a great extent on the accuracy of the wave function assumption, the present invention proposes a quantum chemistry method based on a quantum Monte Carlo algorithm of a neural network, which uses Fourier transform to enhance the neural network model's ability to capture high-frequency changes in electronic states, and is used to improve the model's application performance in the field of physics, which helps to gain a deeper understanding of the basic properties of matter.

[0009] Technical solution: A quantum chemical method based on a quantum Monte Carlo algorithm of a neural network, comprising the following steps:

[0010] Step 1: Pre-train the Transformer model so that the orbits output by the pre-trained Transformer model match the orbits obtained by solving the Schrödinger equation in the nuclear potential field using the Hartree-Fock method;

[0011] Step 2: Design the first high-frequency information modeling layer and the second high-frequency information modeling layer respectively by Fourier transform;

[0012] Step 3: Using the first high-frequency information modeling layer as the input layer of the pre-trained Transformer model, or using the second high-frequency information modeling layer as the intermediate layer of the pre-trained Transformer model, to obtain a Transformer model that explicitly models high-frequency information, wherein the Transformer model that explicitly models high-frequency information takes the electron position as input and the wave function as output;

[0013] Step 4: training the Transformer model for explicitly modeling high-frequency information to obtain a trained Transformer model for explicitly modeling high-frequency information;

[0014] Step 5: Use the trained Transformer model that explicitly models high-frequency information to make predictions at the new electron position to obtain the predicted wave function and achieve modeling of electron behavior.

[0015] Furthermore, the first high-frequency information modeling layer is expressed as:

[0016] hj (x) = ∑ k∈K c j,k exp(k T x)

[0017] Among them, h j (x0 represents the j-th dimension electronic feature, K represents a pre-specified frequency set, c j,k is a learnable parameter of the network, representing the j-dimensional electronic feature corresponding to frequency k, and x represents the electronic position.

[0018] Furthermore, the first high-frequency information modeling layer is expressed as:

[0019]

[0020] In the formula, h 2j (x) represents the electronic characteristics of the 2jth dimension, c 2j,k represents the 2j-dimensional electronic characteristics corresponding to frequency k, h 2j+1 (x) represents the electronic characteristics of the 2j+1th dimension, c 2j+1,k represents the 2j+1th dimensional electronic characteristic corresponding to frequency k, x represents the electron position, and K represents a pre-specified frequency set.

[0021] Furthermore, the second high-frequency information modeling layer is expressed as:

[0022] Calculate the structural features of atomic or molecular systems k :

[0023] s k =∫h(x0exp(ik T xDx

[0024] In the formula, h(x) represents the structural characteristics of the atomic or molecular system at the electronic position x;

[0025] Assuming that the atomic or molecular system has non-zero structural characteristics only at the positions where electrons are present, the electronic characteristics h are used. j Denote h(x):

[0026] h(x)=∑ j h j δ(xx j )

[0027] In the formula, δ(·) represents the Dirac function, x j represents the position of the jth electron.

[0028] Structural characteristics of atomic or molecular systems k Apply a learnable frequency filter, and assume that the feature after processing by the frequency filter is M, and its expression is:

[0029] M k =s k Φ θ (k)

[0030] In the formula, Φ θ (·) represents the frequency filter, θ is the model parameter, M k Represents the output feature corresponding to frequency k;

[0031] The features processed by the frequency filter are converted back to the coordinate space by inverse Fourier transform:

[0032]

[0033] in, represents the output characteristic of the jth electron, x j represents the position of the jth electron.

[0034] Furthermore, the training of the Transformer model for explicitly modeling high-frequency information to obtain a trained Transformer model for explicitly modeling high-frequency information includes:

[0035] The probability density p(x) of a known wave function is sampled using the MH algorithm:

[0036] p(x)=|Ψ(x)| 2 / ∫dΩ|Ψ(x)| 2

[0037] In the formula, Ψ(x) represents the wave function, and Ω represents the integral space;

[0038] The probability density of the wave function is then used to calculate the local energy E L Expected value;

[0039] E VMC =E X~p [E L ]

[0040] In the formula, E X~p represents the expectation of probability density p, represents the local energy, represents the Hamiltonian of the system;

[0041] The electron positions obtained in the above process are taken out to obtain training data;

[0042] The KFAC optimizer is used to train the Transformer model for explicitly modeling high-frequency information using the obtained electron positions to obtain a trained Transformer model for explicitly modeling high-frequency information.

[0043] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0044] (1) The present invention introduces Fourier transform, which provides an effective way for the network to capture and encode the fluctuation and oscillation patterns of functions. This is particularly important in quantum physics because the rapidly changing region of the wave function is usually closely related to the uncertainty of the electron position and the interaction strength between electrons. The periodic characteristics of trigonometric functions enable the network to obtain information about periodic and repetitive patterns, which is very helpful for understanding and predicting physical systems with periodic properties.

[0045] (2) The present invention greatly enhances the adaptability of the model to scale changes by introducing Fourier transform; in actual physical systems, the wave function of electrons may exhibit different behaviors at different scales. For example, when close to the nucleus, the wave function may exhibit rapid oscillations, while in areas far away from the nucleus it may behave more smoothly. By selecting an appropriate set of frequencies, the network can simultaneously capture these multi-scale features, thereby more accurately modeling the behavior of electrons. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a training curve diagram of a quantum chemistry method based on a neural network quantum Monte Carlo algorithm proposed in the present invention; wherein Input FT represents the introduction of Fourier transform in the input layer, and Model FT represents the introduction of Fourier transform in the intermediate layer. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is now further described in conjunction with the accompanying drawings and embodiments.

[0048] This embodiment discloses a quantum chemistry method based on a quantum Monte Carlo algorithm of a neural network. For the input layer and the middle layer of the self-attention network model, two high-frequency information modeling layers are designed respectively by using Fourier transform. The high-frequency information modeling layer is used as the input layer in any existing Transformer model, or the high-frequency information modeling layer is used as the middle layer in any existing Transformer model, that is, a Transformer model for explicitly modeling high-frequency information is obtained, whose input is electronic state information and output is a wave function, thereby realizing efficient calculation of the state of microscopic particles.

[0049] The method of this embodiment mainly includes the following steps:

[0050] Step 1: Pre-train the Transformer model to match the trajectory calculated using the Hartree-Fock method; this includes using the Adam optimizer as the pre-trained optimizer and training for 5000 steps.

[0051] Step 2: Use Fourier transform to design high-frequency information modeling of the input layer and the middle layer respectively; the specific operations include:

[0052] According to the electron position x, determine a set of frequencies k and calculate sin(k T x) and cos(k T x); these two functions capture the sine and cosine components of the distance information at different frequencies. By combining these trigonometric function values ​​with the original input feature vector, the network’s sensitivity and ability to capture high-frequency information can be significantly improved.

[0053] Therefore, the high-frequency information modeling of the input layer can be expressed as:

[0054] h j (x) = ∑ k∈K c j,k exp(k T x) (1)

[0055] Among them, h j (x) represents the j-th dimension electronic feature, K represents a pre-specified frequency set, c j,k is a learnable parameter of the network, representing the j-th dimension electronic feature corresponding to frequency k.

[0056] In actual code implementation, the introduction of complex numbers will lead to a decrease in computational efficiency. Therefore, sin and cos functions are used to use two real number features to represent a complex number feature, namely:

[0057] h 2j (x) = ∑ k∈K c 2j,k sin(k T x) (2)

[0058] h 2j+1 (x) = ∑ k∈K c 2j+1,k cos(k T x) (3)

[0059] In the formula, h 2j (x) represents the electronic characteristics of the 2jth dimension, c 2j,k represents the 2j-dimensional electronic characteristics corresponding to frequency k, h 2j+1 (x) represents the electronic characteristics of the 2j+1th dimension, c 2j+1,k represents the 2j+1th dimensional electronic characteristic corresponding to frequency k, x represents the electron position, and K represents a pre-specified frequency set.

[0060] The application of Fourier transform is extended to the middle layer of the network, that is, the high-frequency information modeling of the middle layer is designed using Fourier transform. The specific approach is:

[0061] First, calculate the structural characteristics of the molecular or atomic system s k , whose expression is:

[0062] s k =∫h(x)exp(ik T x)dx

[0063] In the formula, h(x) represents the structural characteristics of the molecule or atomic system at the electronic position x.

[0064] Assuming that the structural characteristics of a molecule or atomic system are non-zero only when there are electrons present, the electronic characteristics h j To represent h(x):

[0065] h(x)=∑ j h j δ(xx j )

[0066] In the formula, δ(·) represents the Dirac function, x j represents the coordinates of the jth electron;

[0067] Structural features of molecular or atomic systems k Apply a learnable filter to emphasize or weaken the influence of certain frequencies. Let the feature after the filter processing be M, and its expression is:

[0068] M k =s k Φ θ (k)

[0069] s k =∫∑ j h j δ(xx j )exp(ik T x)dx

[0070] In the formula, Φ θ (·) represents the frequency filter, θ is the model parameter, M k Represents the output feature corresponding to frequency k;

[0071] The feature M processed by the filter is converted back to the general coordinate space by inverse Fourier transform:

[0072]

[0073] in, represents the output characteristic of the jth electron, x j represents the position of the jth electron.

[0074] Step 3: Modeling the high-frequency information of the input layer as the input layer of the Transformer model pre-trained in step 1, or modeling the high-frequency information of the intermediate layer as the intermediate layer of the Transformer model pre-trained in step 1, to obtain a Transformer model that explicitly models high-frequency information, whose input is electronic state information and output is a wave function;

[0075] The Transformer model that explicitly models high-frequency information is trained. The specific operations include:

[0076] Electronic position initialization, including:

[0077] To generate electron positions from the wave function, the MH algorithm with a symmetric Gaussian proposal distribution is used to sample the probability density p(x) of the wave function:

[0078] p(x)=|Ψ(x)| 2 / ∫dΩ|Ψ(x)| 2

[0079] Where Ψ(x0) represents the wave function and Ω represents the integration space.

[0080] The probability density of the wave function is then used to calculate the local energy E L Expected value;

[0081] E VMC =E X~p [E L ]

[0082] In the formula, E X~p represents the expectation of probability density p, represents the local energy, represents the Hamiltonian of the system;

[0083] The electron positions obtained in the above process are taken out to obtain training data;

[0084] The obtained electron position is used to train the Transformer model for explicitly modeling high-frequency information to obtain a trained Transformer model for explicitly modeling high-frequency information; specifically, the method includes:

[0085] The Kronecker-factored Approximate Curvature (KFAC) optimizer was used, with a batch size of 4096 for carbon or nitrogen molecules and 8192 for iron atoms, and all models were trained for 100,000 optimization iterations. This step was not trained using the Adam optimizer commonly used in traditional neural networks. This decision was based on the superiority of KFAC shown in previous studies. Specifically, researchers generally believe that the KFAC optimizer shows higher stability and faster convergence than the Adam optimizer. KFAC was used based on its ability to effectively handle large-scale parameter updates, which is particularly critical for deep learning models. KFAC achieves efficient parameter updates by approximating the Hessian matrix of the loss function and decomposing it into smaller Kronecker products. This approach not only improves the efficiency of optimization, but also helps avoid the vanishing or exploding gradient problems encountered during training. During the optimization of these models, as usual, we pre-trained all networks to match the orbits calculated using the Hartree-Fock method. Here, the Adam optimizer is used as the pre-trained optimizer and trained for 5000 steps.

[0086] Step 4: Use the trained Transformer model that explicitly models high-frequency information to make predictions at the new electron position and obtain the predicted wave function, thereby realizing the calculation of the state of microscopic particles.

[0087] In order to improve the efficiency of sampling and reduce the correlation between samples, the number of decorrelated MCMC steps between optimization iterations is set to 30 in this embodiment. In addition, for larger systems (such as iron atoms), the positions of all electrons are not updated simultaneously in one Metropolis step. Instead, the electrons are divided into multiple blocks and updated block by block in each step. For example, in iron atoms, the usual practice is to divide the electrons into two blocks, and each step only updates the electron positions in one block, while keeping the electron positions in the other block unchanged. It is worth noting that although all models have undergone 100,000 optimization iterations of training, this does not mean that all systems require so many iterations to reach convergence. In fact, many smaller systems (such as carbon atoms) have reached convergence in iterations far less than this number. However, in order to maintain consistency and reduce operational confusion, the same number of iterations is set for all systems. This approach ensures the consistency of experimental conditions and facilitates comparison of performance differences between different systems. In addition, due to the large variance in the iron atom system, the learning batch size of carbon atoms or nitrogen molecules is set to 4096, while the learning batch size on iron atoms is increased to 8192. The following table lists the important hyperparameter settings:

[0088]

[0089] The main experimental results of this example are shown in the table below. Among them, the two methods of this example were tried on three systems. The energy obtained by the final reasoning is the average value calculated by 5000 steps of reasoning.

[0090] system Wave function Transformer Psiformer Input FT Model FT C -37.84481 -37.84483(-0.02mHa) -37.84489(-0.08mHa) N -109.53864 -109.53867(-0.03mHa) -109.53870(-0.06mHa) Fe -1263.6445 -1263.6456(-1.1mHa) -1263.6471(-2.6mHa)

[0091] Figure 1 is the training curve of the present invention and the baseline model on the iron atom. Among them, Input FT means the introduction of Fourier transform in the input, and Model FT means the introduction of Fourier transform in the model. The first 50,000 steps of training have been truncated. As shown in Table and Figure 1 As shown, the method of this embodiment has achieved a slight improvement on carbon atoms and nitrogen molecules, and a significant improvement on iron atoms. This is in line with the initial expectation, that is, in the heavy atom system, the wave function in the area near the nucleus oscillates faster and shows high-frequency characteristics, so a large improvement can be achieved in the heavy atom system.

Claims

1. A quantum chemical method based on a quantum Monte Carlo algorithm of a neural network, characterized in that: The following steps are involved: Step 1: Pre-train the Transformer model so that the orbits output by the pre-trained Transformer model match the orbits obtained by solving the Schrödinger equation in the nuclear potential field using the Hartree-Fock method; Step 2: Design the first high-frequency information modeling layer and the second high-frequency information modeling layer respectively by Fourier transform; Step 3: Using the first high-frequency information modeling layer as the input layer of the pre-trained Transformer model, or using the second high-frequency information modeling layer as the intermediate layer of the pre-trained Transformer model, to obtain a Transformer model that explicitly models high-frequency information, wherein the Transformer model that explicitly models high-frequency information takes the electron position as input and the wave function as output; Step 4: training the Transformer model for explicitly modeling high-frequency information to obtain a trained Transformer model for explicitly modeling high-frequency information; Step 5: Use the trained Transformer model that explicitly models high-frequency information to make predictions at the new electron position to obtain the predicted wave function and achieve modeling of electron behavior.

2. The quantum chemical method according to claim 1, characterized in that: The first high-frequency information modeling layer is expressed as: h j (x)=∑ k∈K c j,k exp(k T x) Among them, h j (x) represents the j-th dimension electronic feature, K represents a pre-specified frequency set, c j,k is a learnable parameter of the network, representing the j-dimensional electronic feature corresponding to frequency k, and x represents the electronic position.

3. The quantum chemical method according to claim 1, characterized in that: The first high-frequency information modeling layer is expressed as: In the formula, h 2j (x) represents the electronic characteristics of the 2jth dimension, c 2j,k represents the 2j-dimensional electronic characteristics corresponding to frequency k, h 2j+1 (x) represents the electronic characteristics of the 2j+1th dimension, c 2j+1,k represents the 2j+1th dimensional electronic characteristic corresponding to frequency k, x represents the electron position, and K represents a pre-specified frequency set.

4. The quantum chemical method according to claim 1, characterized in that: The second high-frequency information modeling layer is expressed as: Calculate the structural features of atomic or molecular systems k : s k =∫h(x)exp(i) T x)dx In the formula, h(x) represents the structural characteristics of the atomic or molecular system at the electronic position x; Assuming that the atomic or molecular system has non-zero structural characteristics only at the positions where electrons are present, the electronic characteristics h are used. j Denote h(x): h(x)=∑ j h j δ(x-x j ) In the formula, δ(·) represents the Dirac function, x j represents the position of the jth electron. Structural characteristics of atomic or molecular systems k Apply a learnable frequency filter, and assume that the feature after processing by the frequency filter is M, and its expression is: M k =s k F θ (k) In the formula, Φ θ (·) represents the frequency filter, γ is the model parameter, M k Represents the output feature corresponding to frequency k; The features processed by the frequency filter are converted back to the coordinate space by inverse Fourier transform: in, represents the output characteristic of the jth electron, x j represents the position of the jth electron.

5. The quantum chemical method according to claim 1, characterized in that: The training of the Transformer model for explicitly modeling high-frequency information to obtain a trained Transformer model for explicitly modeling high-frequency information specifically includes: The probability density p(x) of a known wave function is sampled using the MH algorithm: p(x)=|Ψ(x)| 2 / ∫dΩ|Ψ(x)| 2 In the formula, Ψ(x) represents the wave function, and Ω represents the integral space; The probability density of the wave function is then used to calculate the local energy E L Expected value; AND VMC =And X~p [AND L ] In the formula, E X~p represents the expectation of probability density p, represents the local energy, represents the Hamiltonian of the system; The electron positions obtained in the above process are taken out to obtain training data; The KFAC optimizer is used to train the Transformer model for explicitly modeling high-frequency information using the obtained electron positions to obtain a trained Transformer model for explicitly modeling high-frequency information.

Citation Information

Patent Citations

  • Multi-target molecule generation method and system based on diffusion model

    CN117727374A

  • Quantum chemistry model training method, device and equipment based on machine learning

    CN118212996A

  • Quantum computing system and method

    EP4053756A1

  • Quantum circuit generation device, quantum circuit generation method, and quantum circuit generation program

    WO2022039183A1

  • Expected value estimation method and apparatus in quantum system, device, and system

    WO2023060737A1

Cited By

  • Lattice Fermi processing method and electronic equipment

    CN121352056A