Crystal electronic structure prediction method based on large language model and deep learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-15
- Publication Date
- 2026-08-11
AI Technical Summary
[0010]为了解决现有晶体电子结构预测方法计算复杂度高、无法保留高维谱图完整信息、预测精度低的技术问题，本发明提供了一种基于大语言模型和深度学习的晶体电子结构预测方法
[0070]1、降低高维电子结构预测难度。
Smart Images
Figure CN122552001A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to methods for predicting the electronic structure of crystals, specifically to a method for predicting the electronic structure of crystals based on a large language model and deep learning. Background Technology
[0002] Electronic structure (including band structure and density of states) is a core physical quantity in materials science, determining the electrical, optical, and thermal properties of materials. Accurately obtaining the electronic structure of materials is fundamental to the discovery and design of new materials. Existing methods for predicting electronic structure can be mainly classified into the following categories:
[0003] 1. First-principles calculation method (DFT)
[0004] First-principles methods such as density functional theory can accurately calculate electronic structures, but they have the following problems: high computational complexity, requiring hours to days of computation for a single material; difficulty in high-throughput screening, making it impossible to quickly evaluate the electronic structures of thousands of candidate materials; and high resource requirements, necessitating large-scale high-performance computing clusters.
[0005] 2. Direct High-Dimensional Spectral Regression Method
[0006] In recent years, researchers have attempted to use graph neural networks or convolutional networks to directly map crystal structures into band tensors and DOS vectors. However, this method has fundamental drawbacks: the output dimension is high, and the supervision signal is sparse; under small sample conditions, the model is very likely to converge to the "average spectrum", which is a featureless noise level line, losing key physical features such as band gap and peak structure; and it is difficult to capture the band dispersion shape (slope) and the multi-scale distribution of DOS at the same time.
[0007] 3. Methods for predicting low-dimensional electronic properties
[0008] Some works employ graph neural networks or compositional models to predict low-dimensional scalar indices such as band gap, valence band top, and conduction band bottom. The drawback of these methods is that they lose rich physical information such as band dispersion, effective mass, and fine structure of states, making them unsuitable for spectral-level analysis and visualization.
[0009] In summary, there is a lack of existing technologies for electronic structure prediction that can both preserve the complete information of high-dimensional electronic structure spectra and overcome the defects of direct regression. Summary of the Invention
[0010] To address the technical problems of high computational complexity, inability to retain complete information from high-dimensional spectra, and low prediction accuracy in existing crystal electronic structure prediction methods, this invention provides a crystal electronic structure prediction method based on large language models and deep learning.
[0011] To achieve the above objectives, the present invention adopts the following technical solution:
[0012] A method for predicting the electronic structure of crystals based on large language models and deep learning is characterized by the following steps:
[0013] Step 1: Construct a standard sample set of electronic structures;
[0014] Step 2: Construct the target statistics file for the electronic structure model based on the aforementioned standard sample set of electronic structures;
[0015] Step 3: Construct an autoencoder. Train the autoencoder based on the target statistics file and verify its performance using the electronic structure standard sample set to obtain a low-dimensional electronic latent variable space, the trained encoder, and the parameter-fixed electronic structure decoder.
[0016] Step 4: Construct a structure prediction model. Based on this model, the electronic structure is reconstructed using a standard electronic structure sample set, a target statistics file, a trained encoder, a parameter-fixed electronic structure decoder, and a low-dimensional electronic latent variable space, thus completing the prediction of the crystal electronic structure.
[0017] Furthermore, step 1 specifically includes:
[0018] Step 1.1: Collect the original crystal structure data and the original electronic structure data;
[0019] The original crystal structure data includes CIF structure files; the original electronic structure data includes band structure data and density of states (DOS) data.
[0020] Step 1.2: Standardize and organize the band structure data;
[0021] The band structure data is resampled to a fixed number of k points along a preset high-symmetry path, and bands within a preset range from the Fermi level are selected to construct a band tensor of a fixed shape; at the same time, the high-symmetry path signature band_path is extracted and retained.
[0022] Step 1.3: Perform gridding and unification processing on the density of states (DOS) data;
[0023] The density of states (DOS) data is resampled to a preset fixed energy grid to generate a fixed-length DOS vector, thus unifying the DOS data of different materials to the same sampling coordinate system.
[0024] Step 1.4: Perform text serialization processing on the CIF structure file;
[0025] The CIF structure file is converted into a discrete token sequence using a crystal tokenizer, generating the input token number sequence input_ids and the valid token mask attention_mask, while retaining the crystal structure data of atom type, cell parameters, space group, and fractional coordinates;
[0026] Step 1.5: Generate standardized sample files;
[0027] The input_ids, attention_mask, band tensor, high symmetric path signature band_path, and fixed-length DOS vector of each material sample are encapsulated into a .npz format sample file;
[0028] Step 1.6: Fixed data set partitioning;
[0029] All .npz format sample files are divided according to material ID to obtain a standard sample set of electronic structures, including training sample set, validation sample set and test sample set, thus completing the construction of the standard sample set of electronic structures.
[0030] Furthermore, step 2 specifically includes:
[0031] Step 2.1: Determine the target structural dimensions;
[0032] Read the shape of the band tensor and the dimension of the fixed-length DOS vector in the training sample set, define the boundary between the band segment and the DOS segment, and determine the dimension of the target structure.
[0033] Step 2.2: Perform a log1p transformation on the fixed-length DOS vector;
[0034] Add one to the fixed-length DOS vector and take the natural logarithm to obtain the log1p transformed DOS vector.
[0035] Step 2.3: Data assembly and statistics;
[0036] According to the boundaries of the band segment and the DOS segment and the dimension of the target structure, the band tensor and the DOS vector after log1p transformation are concatenated to obtain the electronic structure target vector of the training sample set. The mean and standard deviation are calculated, and the target statistics file target_stats.npz containing the mean, standard deviation, shape of the band tensor, dimension of the fixed-length DOS vector, and DOS transformation method is generated, thus completing the construction of the target statistics file of the electronic structure model.
[0037] Furthermore, step 3 specifically includes:
[0038] Step 3.1, Standardization Processing;
[0039] Based on the target statistics file, the electronic structure target vector in the training sample set is subjected to Z-Score normalization transformation. The band segment uses the original energy value to participate in the Z-Score normalization transformation, and the DOS segment uses the log1p transformed value to participate in the Z-Score normalization transformation, thus obtaining the normalized electronic structure target vector.
[0040] Step 3.2: Construct an autoencoder;
[0041] The autoencoder includes an encoder and a decoder; the encoder is used to compress a standardized electronic structure target vector into a fixed-dimensional low-dimensional latent electronic variable vector; the decoder is used to reconstruct a complete band tensor and a fixed-length DOS vector based on the low-dimensional latent electronic variable vector.
[0042] Step 3.3: Construct the stage loss function for the autoencoder;
[0043] The stage loss function includes a band reconstruction loss function, a band slope constraint loss function, and a log1p domain DOS reconstruction loss function. The band reconstruction loss function is used to constrain the pointwise energy consistency between the predicted band and the true band. The band slope constraint loss function is used to constrain the band dispersion variation trend on the k-path. The log1p domain DOS reconstruction loss function is used to constrain the DOS prediction accuracy in the logarithmic transformation space.
[0044] Step 3.4: Train the autoencoder;
[0045] The autoencoder is trained using the standardized electronic structure target vector obtained in step 3.1 to minimize the stage loss function and obtain a low-dimensional electronic latent variable space. Then, the model performance of the autoencoder is verified using the validation sample set to obtain the trained encoder and the parameter-fixed electronic structure decoder.
[0046] Furthermore, step 4 specifically includes:
[0047] Step 4.1: Construct a structural prediction model;
[0048] The structural prediction model includes an embedded layer, a crystal structure encoder, a fusion module, and a multi-layer fully connected layer connected sequentially from the input end to the output end.
[0049] Step 4.2: Construct a highly symmetric path condition vector;
[0050] The embedding layer reads the input_ids, attention_mask and high symmetric path signature band_path from the training sample set, and discretizes the high symmetric path signature band_path into a path category index to generate a high symmetric path condition vector.
[0051] Step 4.3: Obtain global characterization of the crystal structure;
[0052] The Transformer network of the crystal structure encoder processes the discrete token sequence corresponding to the CIF structure file layer by layer, and pools the hidden states of the effective tokens in combination with attention_mask to obtain the global representation of the crystal structure.
[0053] Step 4.4: Feature fusion processing;
[0054] The fusion module concatenates the global representation of the crystal structure with the high symmetry path condition vector to generate a joint representation that includes the crystal structure and the high symmetry path condition vector.
[0055] Step 4.5: Prediction of electronic latent variables;
[0056] The multilayer fully connected layer maps the joint representation to a predicted latent vector that is consistent with the low-dimensional electronic latent variable space dimension obtained in step 3.4, and inputs the standardized electronic structure target vector into the trained encoder to output the true latent vector.
[0057] Step 4.6: Electronic structure reconstruction constraints;
[0058] The predicted latent vector is input into the electronic structure decoder with solidified parameters obtained in step 3.4 to reconstruct the band tensor and fixed-length DOS vector after Z-Score normalization transformation, and the reconstruction loss is calculated using the stage loss function calculation rule.
[0059] Step 4.7: Obtain the optimal structure prediction model;
[0060] The mean square error between the predicted latent vector and the true latent vector is calculated to obtain the latent regression loss; the latent regression loss and the reconstruction loss are weighted and combined to obtain the total loss function; under the condition that the electronic structure decoder parameters are fixed, the parameters of the structure prediction model are updated on the training sample set based on the total loss function, and the performance is evaluated and verified through the validation sample set to finally obtain the optimal structure prediction model.
[0061] Step 4.8: Inverse transformation processing;
[0062] Read the input_ids, attention_mask, and high-symmetric path signature band_path from the test sample set, and perform discrete encoding according to the same path category index established in step 4.2 to obtain the high-symmetric path condition vector. Input the input_ids, attention_mask, and high-symmetric path condition vector into the optimal structure prediction model obtained in step 4.7 for inference to obtain the optimal prediction latent vector. Reconstruct the normalized band tensor and fixed-length DOS vector through the parameter-fixed electronic structure decoder. Perform Z-Score normalization inverse transform on the reconstructed normalized band tensor and fixed-length DOS vector based on the target statistics file. Then, further apply non-negative physical constraints to the inversely transformed fixed-length DOS vector to reconstruct the crystal electronic structure and complete the crystal electronic structure prediction.
[0063] Furthermore, in step 3.2, the fixed dimension of the latent vector is set to 256 dimensions.
[0064] Furthermore, in step 4.1, the crystal structure encoder adopts CrystalLLM; the fusion module consists of a linear layer, a normalization layer, and a nonlinear activation layer.
[0065] Furthermore, in step 1.6, the training sample set, validation sample set, and test sample set are independently divided based on the material ID, ensuring that the same material sample belongs to only a single dataset and avoiding data leakage.
[0066] Further, in step 4.8, the non-negative physical constraint includes expm1 transformation and numerical truncation; the expm1 transformation is used to restore the fixed-length DOS vector from the log1p domain to the original physical quantity space to mathematically guarantee non-negativity; the numerical truncation is used to assign 0 to the negative elements in the fixed-length DOS vector after the expm1 transformation.
[0067] Furthermore, in step 1.2, the screening of energy bands within a preset range from the Fermi level specifically involves selecting 3 to 10 nearest neighbor energy bands above and below the Fermi level to match the electronic structure characteristics of different crystals.
[0068] In step 3.3, the band slope constraint loss is calculated by the gradient difference of band energy at adjacent k points to constrain the consistency of the dispersion trend between the predicted band and the real band.
[0069] The beneficial effects of this invention are:
[0070] 1. Reduce the difficulty of predicting high-dimensional electronic structures.
[0071] This invention constructs an autoencoder to compress a high-dimensional target vector formed by concatenating a band tensor and a fixed-length DOS vector into a low-dimensional latent vector of electronic variables with a fixed dimension. This transforms the high-dimensional electronic structure spectrum regression problem into a low-dimensional latent variable prediction problem, significantly reducing the difficulty of direct regression in subsequent structure prediction models and making high-dimensional electronic structure prediction more computationally feasible and efficient.
[0072] 2. Achieve end-to-end prediction from crystal structure to electronic structure.
[0073] This invention employs CrystalLLM (Crystal Large Language Model) as a crystal structure encoder to extract the global representation of the crystal structure from the CIF structure file; then it fuses this representation with the high-symmetry path condition vector to construct a joint representation; the optimal structure prediction model is obtained by training using the methods in steps 4.2 to 4.7; step 4.8 utilizes the fixed electronic structure decoder to reconstruct the complete band tensor and fixed-length DOS vector from low-dimensional latent variables, thereby realizing end-to-end prediction from the crystal structure file to the high-dimensional electronic structure spectrum without relying on point-by-point calculations of first-principles methods such as DFT (Density Functional Theory).
[0074] 3. Explicitly model highly symmetric path conditions to reduce path signature ambiguity.
[0075] In step 4.2 of this invention, the high-symmetric path signature band_path is discretely encoded into a path category index and mapped to a high-symmetric path condition vector via an embedding layer. This enables the structure prediction model to effectively distinguish different high-symmetric paths, overcomes the prediction bias that may be caused by path signature ambiguity, and improves the prediction accuracy of the model under multi-path conditions.
[0076] 4. Supports multi-path electronic structure prediction.
[0077] The electronic structure standard sample set constructed in step 1 of this invention covers a variety of high-symmetry path signatures. The optimal structure prediction model obtained by training in steps 4.2 to 4.7 does not need to be trained separately for each high-symmetry path. It can output the corresponding band structure and density of states under multi-path conditions and has good generalization ability.
[0078] 5. Data independence and experimental reproducibility.
[0079] In step 1.6, this invention employs a fixed data segmentation strategy based on material ID to strictly divide the electronic structure standard sample set into non-overlapping training sample set, validation sample set, and test sample set, thereby avoiding the risk of data leakage from the source. The target statistics file generated in step 2 is calculated only based on the training sample set, ensuring the independence of data preprocessing and the comparability between different experiments.
[0080] 6. Compatibility with crystal structure encoders.
[0081] This invention uses CrystalLLM as the crystal structure encoder without changing its pre-training process. By adding a fusion module, it enhances the electronic structure prediction capability of the crystal large language model, enabling the two major tasks of crystal structure generation and electronic structure prediction to work collaboratively under a unified CIF structure text representation system. At the same time, it can output a prediction comparison diagram of band tensor and fixed-length DOS vector, which facilitates intuitive review and analysis of key physical features such as band edge position, band dispersion trend and DOS peak shape. Attached Figure Description
[0082] Figure 1 This is a flowchart of an embodiment of the crystal electronic structure prediction method based on large language models and deep learning of the present invention;
[0083] Figure 2 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-978862(SrAlO3) material test sample set; Figure 3 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-1186154 (NaNiO3) material test sample set; Figure 4 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-973958 (KRuO3) material test sample set; Figure 5 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-4783 (BaPrO3) material test sample set; Figure 6 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-1183447 (BePdO3) material test sample set; Figure 7 This is a visualization of the crystal electronic structure prediction method in this embodiment of the invention on the mp-769284 (Dy2SeO2) material test sample set;
[0084] Figures 2 to 7In the middle, the left figure is a comparison of the band structure of the corresponding materials, with the horizontal axis representing the high symmetry path and the vertical axis representing the energy (eV) of the relative Fermi level; the right figure is a comparison of the density of states (DOS) of the corresponding materials, with the horizontal axis representing the DOS and the vertical axis representing the energy (eV) of the relative Fermi level.
[0085] The solid black lines represent the actual values, while the dashed blue lines represent the predicted values output by this invention. Detailed Implementation
[0086] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0087] This invention provides a method for predicting the electronic structure of crystals based on a large language model and deep learning, such as... Figure 1 As shown, the method for predicting the electronic structure of a crystal includes the following steps:
[0088] Step 1: Construct a standard sample set for electronic structures; the standard sample set for electronic structures includes a training sample set, a validation sample set, and a test sample set; specifically, it includes the following steps:
[0089] Step 1.1: Collect the original crystal structure data and the original electronic structure data.
[0090] The original crystal structure data includes the CIF structure file; the original electronic structure data includes band structure data and density of states (DOS) data.
[0091] Step 1.2: Standardize and organize the band structure data.
[0092] The band structure data is resampled to a fixed number of k points along a preset high-symmetry path. Bands within a preset range from the Fermi level are selected to construct a band tensor of a fixed shape. At the same time, the high-symmetry path signature band_path is extracted and retained. Specifically, 3 to 10 nearest neighbor bands above and below the Fermi level are selected to adapt to the electronic structure characteristics of different crystals.
[0093] Step 1.3: Perform gridding and unification processing on the density of states (DOS) data.
[0094] The density of states (DOS) data is resampled to a preset fixed energy grid to generate a fixed-length DOS vector, thus unifying the DOS data of different materials into the same sampling coordinate system.
[0095] Step 1.4: Perform text serialization processing on the CIF structure file.
[0096] The CIF structure file is converted into a discrete token sequence by a crystal tokenizer, generating the input token number sequence input_ids and the valid token mask attention_mask, while retaining the crystal structure data such as atom type, cell parameters, space group, and fractional coordinates.
[0097] Step 1.5: Generate standardized sample files.
[0098] The input_ids, attention_mask, band tensor, high symmetric path signature band_path, and fixed-length DOS vector of each material sample are encapsulated into a .npz format sample file.
[0099] Step 1.6: Fixed split of the dataset.
[0100] All .npz format sample files are partitioned according to their material IDs to obtain a standard electronic structure sample set, which includes a training sample set, a validation sample set, and a test sample set, thus completing the construction of the standard electronic structure sample set. The training sample set, validation sample set, and test sample set are independently partitioned based on their material IDs, ensuring that the same material sample belongs to only a single dataset and avoiding data leakage.
[0101] In this embodiment, the original crystal structure data and original electronic structure data of the oxide material are obtained from the Materials Project database. The band structure data is resampled into a fixed number of 128 k-points along a preset high-symmetry path, and the eight nearest neighbor bands above and below the Fermi level are selected, with the energy range aligned to [-10eV, 10eV] relative to the Fermi level. A band tensor of shape 128×16 is constructed, and the high-symmetry path signature band_path is extracted and retained, such as "XG|GY|LG|GZ|NG|GM|RG". The density of states (DOS) data is resampled to a preset fixed energy grid, including 1024 energy points with an energy range of [-10eV, 10eV], and a fixed-length DOS vector is generated. Subsequently, the CIF structure file is converted into a discrete token sequence using a crystal tokenizer, generating the input token number sequence input_ids and the effective token mask attention_mask, while retaining the crystal structure data of atom types, cell parameters, space group, and fractional coordinates during the serialization process. The input_ids, attention_mask, band tensor, high-symmetric path signature band_path, and fixed-length DOS vector of each material sample are encapsulated into .npz format sample files, forming a total of 12394 electronic structure standard samples, each corresponding to a material ID. All .npz format sample files are then divided according to the material ID to obtain training sample sets, validation sample sets, and test sample sets. The training sample set includes 11155 samples, the validation sample set includes 620 samples, and the test sample set includes 619 samples.
[0102] Step 2: Construct the target statistics file for the electronic structure model based on the aforementioned standard sample set of electronic structures; specifically, this includes the following steps:
[0103] Step 2.1: Determine the target structural dimensions.
[0104] Read the shape of the band tensor and the dimension of the fixed-length DOS vector in the training sample set, define the boundary between the band segment and the DOS segment, and determine the dimension of the target structure.
[0105] Step 2.2: Perform a log1p transformation on the fixed-length DOS vector.
[0106] Adding one to a fixed-length DOS vector and taking its natural logarithm yields the DOS vector after the log1p transformation. The log1p transformation is the mathematical basis for imposing a non-negative physical constraint on a fixed-length DOS vector, and its inverse transformation, the expm1 transformation, can strictly guarantee that the reconstructed output is non-negative.
[0107] Step 2.3: Data splicing and statistics.
[0108] According to the boundaries of the band segment and the DOS segment, and the dimension of the target structure, the band tensor and the DOS vector after log1p transformation are concatenated to obtain the electronic structure target vector of the training sample set. The mean and standard deviation are calculated, and the target statistics file target_stats.npz containing the mean, standard deviation, shape of the band tensor, dimension of the fixed-length DOS vector, and DOS transformation method is generated, thus completing the construction of the target statistics file of the electronic structure model.
[0109] In this embodiment, the band tensor shape is 128×16, and the band segment dimension is 2048; the fixed-length DOS vector dimension is 1024; thus, the dimension of the target structure after concatenating the band segment and the DOS segment is determined to be 3072. Adding one to the fixed-length DOS vector and taking its natural logarithm yields the log1p transformed DOS vector. Concatenating the band tensor with the log1p transformed DOS vector yields the electronic structure target vector for the training sample set. The target statistics file `target_stats.npz` is used to calculate its mean, standard deviation, band tensor shape (128×16), fixed-length DOS vector dimension (1024), and DOS transformation method (log1p transformation), completing the construction of the electronic structure model training space and target statistics.
[0110] Step 3: Construct an autoencoder. Train the autoencoder based on the target statistics file mentioned above, and verify its performance using the standard electronic structure sample set. This yields a low-dimensional electronic latent variable space, the trained encoder, and a parameter-fixed electronic structure decoder. Specifically, this includes the following steps:
[0111] Step 3.1, standardization process.
[0112] Based on the target statistics file, the electronic structure target vector in the training sample set is subjected to Z-Score normalization transformation. The band segment uses the original energy value to participate in the Z-Score normalization transformation, and the DOS segment uses the log1p transformed value to participate in the Z-Score normalization transformation, thus obtaining the normalized electronic structure target vector.
[0113] Step 3.2: Construct an autoencoder.
[0114] The autoencoder includes an encoder and a decoder; the encoder is used to compress a standardized electronic structure target vector into a fixed-dimensional low-dimensional latent electronic variable vector; the decoder is used to reconstruct a complete band tensor and a fixed-length DOS vector based on the low-dimensional latent electronic variable vector; the fixed dimension of the low-dimensional latent electronic variable vector is set to 256 dimensions.
[0115] Step 3.3: Construct the loss function for the autoencoder stage.
[0116] The stage loss functions include the band reconstruction loss function, the band slope constraint loss function, and the log1p domain DOS reconstruction loss function. The band reconstruction loss function is used to constrain the pointwise energy consistency between the predicted band and the true band. The band slope constraint loss function is used to constrain the band dispersion variation trend on the k-path. The log1p domain DOS reconstruction loss function is used to constrain the DOS prediction accuracy in the log1p transform space.
[0117] Step 3.4: Train the electronic structure manifold.
[0118] The autoencoder is trained using the standardized electronic structure target vector obtained in step 3.1 to minimize the stage loss function and obtain a low-dimensional electronic latent variable space. Then, the model performance of the autoencoder is verified using the validation sample set to obtain the trained encoder and the parameter-fixed electronic structure decoder.
[0119] In this embodiment, based on the target statistics file target_stats.npz, the electronic structure target vectors in the training sample set are subjected to Z-Score normalization transformation. Specifically, the band segment retains its original energy value for Z-Score normalization, while the DOS segment undergoes log1p transformation before normalization, resulting in a standardized electronic structure target vector. An autoencoder constructed using this standardized electronic structure target vector is trained with a batch size of 32, 80 training epochs, a learning rate of 3e-4, and a band slope constraint loss weight of 0.25. The loss on the validation sample set decreases from 0.615835 in the 0th training epoch, reaching a minimum of 0.173072 in the 71st training epoch. The model performance is optimal on the validation sample set, and the decoder parameters at this point are fixed, becoming the electronic structure decoder with fixed parameters.
[0120] Step 4: Construct a structure prediction model. Based on this model, reconstruct the electronic structure using a standard electronic structure sample set, target statistics file, trained encoder, parameter-fixed electronic structure decoder, and low-dimensional electronic latent variable space to complete the crystal electronic structure prediction. This includes the following steps:
[0121] Step 4.1: Construct a structural prediction model.
[0122] The structural prediction model consists of an embedding layer, a crystal structure encoder, a fusion module, and a multi-layer fully connected layer connected sequentially from the input to the output; the crystal structure encoder is built based on the Transformer network architecture.
[0123] Step 4.2: Construct a highly symmetric path condition vector.
[0124] The embedding layer reads input_ids, attention_mask, and high symmetric path signature band_path from the training sample set, and discretizes the high symmetric path signature band_path into a path category index to generate a high symmetric path condition vector.
[0125] Step 4.3: Obtain global characterization of the crystal structure.
[0126] The Transformer network of the crystal structure encoder processes the discrete token sequence corresponding to the CIF structure file layer by layer, and pools the hidden states of effective tokens in combination with attention_mask to obtain the global representation of the crystal structure.
[0127] Step 4.4: Feature fusion processing.
[0128] The fusion module concatenates the global representation of the crystal structure with the high-symmetry path condition vector to generate a joint representation that includes both the crystal structure and the high-symmetry path condition vector.
[0129] Step 4.5: Prediction of electronic latent variables.
[0130] The multilayer fully connected layer maps the joint representation described above into a predicted latent vector that is consistent with the low-dimensional electronic latent variable space dimension obtained in step 3.4, and inputs the standardized electronic structure target vector into the trained encoder to output the true latent vector.
[0131] Step 4.6: Electronic structure reconstruction constraints.
[0132] The predicted latent vector is input into the electronic structure decoder with solidified parameters obtained in step 3.4 to reconstruct the band tensor and fixed-length DOS vector after Z-Score normalization transformation, and the reconstruction loss is calculated using the stage loss function calculation rule.
[0133] Step 4.7: Obtain the optimal structure prediction model.
[0134] The mean squared error between the predicted latent vector and the true latent vector is calculated to obtain the latent regression loss. The latent regression loss and the reconstruction loss are weighted and combined to obtain the total loss function. Under the condition that the electronic structure decoder parameters are fixed, the parameters of the structure prediction model are updated on the training sample set based on the total loss function, and the performance is evaluated and verified by the validation sample set to finally obtain the optimal structure prediction model.
[0135] Step 4.8: Inverse transformation processing.
[0136] Read the input_ids, attention_mask, and high-symmetric path signature band_path from the test sample set, and perform discrete encoding according to the same path category index established in step 4.2 to obtain the high-symmetric path condition vector. Input the input_ids, attention_mask, and high-symmetric path condition vector into the optimal structure prediction model obtained in step 4.7 for inference to obtain the optimal prediction latent vector. Reconstruct the normalized band tensor and fixed-length DOS vector through the parameter-fixed electronic structure decoder. Perform Z-Score normalization inverse transform on the reconstructed normalized band tensor and fixed-length DOS vector based on the target statistics file. Then, further apply non-negative physical constraints to the inversely transformed fixed-length DOS vector to reconstruct the crystal electronic structure and complete the crystal electronic structure prediction.
[0137] Nonnegative physical constraints include expm1 transformation and numerical truncation; expm1 transformation is used to restore a fixed-length DOS vector from the log1p domain to the original physical quantity space to mathematically guarantee nonnegativity; numerical truncation is used to assign the negative elements in the fixed-length DOS vector after expm1 transformation to 0.
[0138] In this embodiment, the input_ids, attention_mask, and high-symmetric path signatures (band_path) from the training sample set are read. There are 57 different high-symmetric path signatures (band_path). The high-symmetric path signatures (band_path) are discretely encoded into path category indices and mapped to high-symmetric path condition vectors through an embedding layer. CrystalLLM is used as the crystal structure encoder. A Transformer network processes the discrete token sequences corresponding to the CIF structure file layer by layer. The attention_mask is used to pool the hidden states of effective tokens to obtain a global representation of the crystal structure. A fusion module consisting of linear layers, normalization layers, and nonlinear activation layers concatenates the global representation of the crystal structure with the high-symmetric path condition vectors to generate a joint representation containing both the crystal structure and the high-symmetric path condition vectors. Multiple fully connected layers map the joint representation to a predicted latent vector with the same dimension as the aforementioned low-dimensional latent variable space. Simultaneously, the standardized electronic structure target vector is input into the trained encoder, which outputs the true latent vector. The electronic structure decoder, with its input parameters fixed to predict the latent vector, reconstructs the band tensor and fixed-length DOS vector after Z-score normalization. The reconstruction loss is calculated using a stage loss function. Then, the mean squared error between the predicted and true latent vectors is calculated to obtain the latent regression loss. The latent regression loss and reconstruction loss are weighted and combined to obtain the total loss function. With the electronic structure decoder parameters fixed, the parameters of the structure prediction model are updated on the training sample set based on this total loss function. Performance is evaluated and validated using a validation sample set to obtain the optimal structure prediction model. Training uses a batch size of 2, gradient accumulation steps of 8, an equivalent batch size of 16, and a learning rate of 3e-5. During training, the validation sample set loss decreases from 1.934298 in the 0th training epoch and reaches its optimal value of 1.504126 in the 14th training epoch. Therefore, the model with the best performance on this validation sample set is selected as the optimal structure prediction model. On 619 samples in the test sample set, the CIF structure file and the high-symmetry path condition vector are input into the optimal structure prediction model for inference to obtain the optimal prediction latent vector. The normalized band tensor and the fixed-length DOS vector are reconstructed through the parameter-fixed electronic structure decoder. Based on the target statistics file, the reconstructed normalized band tensor and the fixed-length DOS vector are subjected to the inverse Z-Score normalization transformation. Then, the fixed-length DOS vector after the inverse Score normalization transformation is further subjected to non-negative physical constraints to reconstruct the crystal electronic structure and complete the crystal electronic structure prediction.The final prediction results are: mean square error 31.564887, root mean square error 4.292677, and mean absolute error 1.625782; among which, the mean absolute error of the band segment is 0.506735 eV, and the mean absolute error of the DOS segment is 3.879338.
[0139] like Figures 2 to 7 As shown, this embodiment extracts six representative material samples from the prediction results of the above 619 test samples for visualization. At the sample level, the band structure and density of states (DOS) are output simultaneously. The predicted band structure is basically consistent with the actual band structure in terms of band edge position and band dispersion trend. The predicted density of states is highly consistent with the actual density of states in terms of peak position and peak shape. This verifies that the present invention can achieve accurate prediction from CIF structure file to high-dimensional electronic structure spectrum.
[0140] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for predicting the electronic structure of crystals based on large language models and deep learning, characterized in that, Includes the following steps: Step 1: Construct a standard sample set of electronic structures; Step 2: Construct the target statistics file for the electronic structure model based on the aforementioned standard sample set of electronic structures; Step 3: Construct an autoencoder. Train the autoencoder based on the target statistics file and verify its performance using the electronic structure standard sample set to obtain a low-dimensional electronic latent variable space, the trained encoder, and the parameter-fixed electronic structure decoder. Step 4: Construct a structure prediction model. Based on this model, the electronic structure is reconstructed using a standard electronic structure sample set, a target statistics file, a trained encoder, a parameter-fixed electronic structure decoder, and a low-dimensional electronic latent variable space, thus completing the prediction of the crystal electronic structure.
2. The crystal electronic structure prediction method based on large language models and deep learning according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Collect the original crystal structure data and the original electronic structure data; The original crystal structure data includes CIF structure files; the original electronic structure data includes band structure data and density of states (DOS) data. Step 1.2: Standardize and organize the band structure data; The band structure data is resampled to a fixed number of k points along a preset high-symmetry path, and bands within a preset range from the Fermi level are selected to construct a band tensor of a fixed shape; at the same time, the high-symmetry path signature band_path is extracted and retained. Step 1.3: Perform gridding and unification processing on the density of states (DOS) data; The density of states (DOS) data is resampled to a preset fixed energy grid to generate a fixed-length DOS vector, thus unifying the DOS data of different materials to the same sampling coordinate system. Step 1.4: Perform text serialization processing on the CIF structure file; The CIF structure file is converted into a discrete token sequence using a crystal tokenizer, generating the input token number sequence input_ids and the valid token mask attention_mask, while retaining the crystal structure data such as atom type, cell parameters, space group, and fractional coordinates. Step 1.5: Generate standardized sample files; The input_ids, attention_mask, band tensor, high symmetric path signature band_path, and fixed-length DOS vector of each material sample are encapsulated into a .npz format sample file; Step 1.6: Fixed data set partitioning; All .npz format sample files are divided according to material ID to obtain a standard sample set of electronic structures, including training sample set, validation sample set and test sample set, thus completing the construction of the standard sample set of electronic structures.
3. The crystal electronic structure prediction method based on large language models and deep learning according to claim 2, characterized in that, Step 2 specifically includes: Step 2.1: Determine the target structural dimensions; Read the shape of the band tensor and the dimension of the fixed-length DOS vector in the training sample set, define the boundary between the band segment and the DOS segment, and determine the dimension of the target structure. Step 2.2: Perform a log1p transformation on the fixed-length DOS vector; Add one to the fixed-length DOS vector and take the natural logarithm to obtain the log1p transformed DOS vector. Step 2.3: Data splicing and statistics; According to the boundaries of the band segment and the DOS segment and the dimension of the target structure, the band tensor and the DOS vector after log1p transformation are concatenated to obtain the electronic structure target vector of the training sample set. The mean and standard deviation are calculated, and the target statistics file target_stats.npz containing the mean, standard deviation, shape of the band tensor, dimension of the fixed-length DOS vector, and DOS transformation method is generated, thus completing the construction of the target statistics file of the electronic structure model.
4. The crystal electronic structure prediction method based on large language models and deep learning according to claim 3, characterized in that, Step 3 specifically includes: Step 3.1, Standardization Processing; Based on the target statistics file, the electronic structure target vector in the training sample set is subjected to Z-Score normalization transformation. The band segment uses the original energy value to participate in the Z-Score normalization transformation, and the DOS segment uses the log1p transformed value to participate in the Z-Score normalization transformation, thus obtaining the normalized electronic structure target vector. Step 3.2: Construct an autoencoder; The autoencoder includes an encoder and a decoder; the encoder is used to compress a standardized electronic structure target vector into a fixed-dimensional low-dimensional latent electronic variable vector; the decoder is used to reconstruct a complete band tensor and a fixed-length DOS vector based on the low-dimensional latent electronic variable vector. Step 3.3: Construct the stage loss function for the autoencoder; The stage loss function includes a band reconstruction loss function, a band slope constraint loss function, and a log1p domain DOS reconstruction loss function. The band reconstruction loss function is used to constrain the pointwise energy consistency between the predicted band and the true band. The band slope constraint loss function is used to constrain the band dispersion variation trend on the k-path. The log1p domain DOS reconstruction loss function is used to constrain the DOS prediction accuracy in the logarithmic transformation space. Step 3.4: Train the autoencoder; The autoencoder is trained using the standardized electronic structure target vector obtained in step 3.1 to minimize the stage loss function and obtain a low-dimensional electronic latent variable space. Then, the model performance of the autoencoder is verified using the validation sample set to obtain the trained encoder and the parameter-fixed electronic structure decoder.
5. The crystal electronic structure prediction method based on large language models and deep learning according to claim 4, characterized in that, Step 4 specifically includes: Step 4.1: Construct a structural prediction model; The structural prediction model includes an embedded layer, a crystal structure encoder, a fusion module, and a multi-layer fully connected layer connected sequentially from the input end to the output end. Step 4.2: Construct a highly symmetric path condition vector; The embedding layer reads the input_ids, attention_mask and high symmetric path signature band_path from the training sample set, and discretizes the high symmetric path signature band_path into a path category index to generate a high symmetric path condition vector. Step 4.3: Obtain global characterization of the crystal structure; The Transformer network of the crystal structure encoder processes the discrete token sequence corresponding to the CIF structure file layer by layer, and pools the hidden states of the effective tokens in combination with attention_mask to obtain the global representation of the crystal structure. Step 4.4: Feature fusion processing; The fusion module concatenates the global representation of the crystal structure with the high symmetry path condition vector to generate a joint representation that includes the crystal structure and the high symmetry path condition vector. Step 4.5: Prediction of electronic latent variables; The multilayer fully connected layer maps the joint representation to a predicted latent vector that is consistent with the low-dimensional electronic latent variable space dimension obtained in step 3.4, and inputs the standardized electronic structure target vector into the trained encoder to output the true latent vector. Step 4.6: Electronic structure reconstruction constraints; The predicted latent vector is input into the electronic structure decoder with solidified parameters obtained in step 3.4 to reconstruct the band tensor and fixed-length DOS vector after Z-Score normalization transformation, and the reconstruction loss is calculated using the stage loss function calculation rule. Step 4.7: Obtain the optimal structure prediction model; The mean square error between the predicted latent vector and the true latent vector is calculated to obtain the latent regression loss; the latent regression loss and the reconstruction loss are weighted and combined to obtain the total loss function; under the condition that the electronic structure decoder parameters are fixed, the parameters of the structure prediction model are updated on the training sample set based on the total loss function, and the performance is evaluated and verified through the validation sample set to finally obtain the optimal structure prediction model. Step 4.8: Inverse transformation processing; Read the input_ids, attention_mask, and high-symmetric path signature band_path from the test sample set, and perform discrete encoding according to the same path category index established in step 4.2 to obtain the high-symmetric path condition vector. Input the input_ids, attention_mask, and high-symmetric path condition vector into the optimal structure prediction model obtained in step 4.7 for inference to obtain the optimal prediction latent vector. Reconstruct the normalized band tensor and fixed-length DOS vector through the parameter-fixed electronic structure decoder. Perform Z-Score normalization inverse transform on the reconstructed normalized band tensor and fixed-length DOS vector based on the target statistics file. Then, further apply non-negative physical constraints to the inversely transformed fixed-length DOS vector to reconstruct the crystal electronic structure and complete the crystal electronic structure prediction.
6. The crystal electronic structure prediction method based on large language models and deep learning according to claim 4, characterized in that: In step 3.2, the fixed dimension of the latent vector is set to 256 dimensions.
7. The crystal electronic structure prediction method based on large language models and deep learning according to claim 5, characterized in that: In step 4.1, the crystal structure encoder adopts CrystalLLM; the fusion module consists of a linear layer, a normalization layer and a nonlinear activation layer.
8. The method for predicting crystal electronic structure based on large language models and deep learning according to claim 2, characterized in that: In step 1.6, the training sample set, validation sample set, and test sample set are independently divided based on the material ID, ensuring that the same material sample belongs to only a single dataset and avoiding data leakage.
9. The method for predicting crystal electronic structure based on large language models and deep learning according to claim 5, characterized in that: In step 4.8, the non-negative physical constraint includes expm1 transformation and numerical truncation; the expm1 transformation is used to restore the fixed-length DOS vector from the log1p domain to the original physical quantity space to mathematically guarantee non-negativity; the numerical truncation is used to assign 0 to the negative elements in the fixed-length DOS vector after the expm1 transformation.
10. The method for predicting crystal electronic structure based on large language models and deep learning according to claim 4, characterized in that: In step 1.2, the screening of energy bands within a preset range from the Fermi level specifically involves selecting 3 to 10 nearest neighbor energy bands above and below the Fermi level to match the electronic structure characteristics of different crystals. In step 3.3, the band slope constraint loss is calculated by the gradient difference of band energy at adjacent k points to constrain the consistency of the dispersion trend between the predicted band and the real band.