Bandgap prediction and explanation method and system based on multi-head self-attention mechanism

Abstract

This application provides a method and system for bandgap prediction and interpretation based on a multi-head self-attention mechanism. The method includes: using a Transformer-based encoder to model the global dependencies of a CSV file containing the physicochemical characteristics of lithium-ion battery materials through a multi-head self-attention mechanism, determining the context embedding, and concatenating it with the physicochemical characteristics of the lithium-ion battery materials to determine training features; then using a pre-set CatBoost regressor to perform regression training on the training features to determine the Catformer model; using the Catformer model to predict the bandgap of the lithium-ion battery material to be tested, and using the SHAP analysis method to perform interpretability analysis on the bandgap prediction. This application enhances the global dependency characteristics of the material, achieves accurate bandgap prediction, and reveals the physicochemical significance of the prediction results, providing a general tool for the rapid design of lithium battery electrode materials.

Description

Technical Field

[0001] This application relates to the field of computational materials technology, and more specifically, to a method and system for bandgap prediction and interpretation based on a multi-head self-attention mechanism. Background Technology

[0002] Materials informatics, as a cutting-edge field at the intersection of materials science and artificial intelligence, has accelerated the discovery and performance prediction of new materials in recent years through machine learning models. Traditional methods for predicting material properties mainly rely on computational techniques such as density functional theory (DFT) and molecular dynamics (MD), as well as semi-empirical models based on empirical parameters, such as CALPHAD and phase-field simulations. However, these methods face significant challenges in complex crystalline systems, such as high-entropy materials, disordered solid solutions, and porous framework materials. High computational cost: Density functional theory (DFT) calculations of the energy, electronic structure and other properties of a single cell typically take hours to days, and the computational cost of dynamic simulations, such as ion migration paths, increases exponentially with the size of the system, making it difficult to apply to complex systems containing hundreds of atoms.

[0003] Feature dependencies are difficult to capture: Material properties depend not only on the local atomic environment, but also on the long-range dependencies of global structure and element combination, which are difficult to model using traditional methods.

[0004] Insufficient interpretability: Although traditional machine learning models can predict band gaps, they are difficult to reveal the specific contributions of different atoms or features to the band gap, which limits the scientific basis for material design decisions.

[0005] Therefore, developing a model that can simultaneously capture global feature dependencies, rapidly predict band gaps, and provide interpretable analysis is of great significance for the design of lithium-ion battery electrode materials. In recent years, the Transformer model, with its powerful self-attention mechanism, has demonstrated outstanding performance in capturing long-range dependencies in sequential data.

[0006] 1. The limitations of local perception and the necessity of modeling global dependencies: Traditional machine learning models, such as XGBoost and CatBoost, typically use physicochemical features, such as atomic number, electronegativity, and atomic radius, directly for modeling when predicting material band gaps. For example, in patent "CN112992290B; A perovskite band gap prediction method based on machine learning and cluster models," a database of a set number of models is constructed using the intrinsic structural parameters of perovskite cluster models as a training database for machine learning. The machine learning model is then trained using this selected training database to predict the band gap. Intrinsic parameters can include molecular volume, binding energy, ionization energy, minimum bond length, band gap value, dipole moment, ionic charge, molecular surface electrostatic potential, and ionization energy. These methods can only learn the local correlation between features and target values, ignoring the potential global dependencies between different features. For instance, the band gap is not only determined by single atomic features but also influenced by the synergistic and coupling effects of multiple elemental properties. If the model cannot capture these cross-feature dependencies, the prediction accuracy will be significantly limited. The Tansformer-based self-attention mechanism can establish weighted correlations among all input features, thereby extracting global dependency information and achieving a more comprehensive understanding and accurate prediction of the bandgap formation mechanism.

[0007] 2. The bottlenecks of static feature learning and the advantages of dynamic dependency modeling: Traditional models treat material features as independent static variables, making it difficult to reflect the complex nonlinear relationships between them. For example, in lithium-ion battery electrode materials, electron affinity, electronegativity, atomic radius, and other features often exhibit synergistic or competitive effects, and traditional algorithms struggle to automatically identify these dynamic relationships in high-dimensional spaces. The Transformer model dynamically adjusts the weight distribution among features through a self-attention mechanism, enabling the model to adaptively capture the dependence strength between key features, thereby achieving "information reconstruction" at the feature level. This mechanism effectively overcomes the limitations of traditional static feature learning, enhances the model's expressive power in multi-dimensional feature spaces, and enables more accurate and physically plausible bandgap predictions.

[0008] 3. The lack of explanation in black-box models and the transparency of attention mechanisms: While traditional deep learning models excel in predictive performance, their internal decision-making processes are often difficult to interpret. For materials science research, high-accuracy predictions alone are insufficient; understanding "why the model predicts this way" is also crucial. SHAP analysis can visualize the impact of features on prediction results and their importance values ​​during the training and data processing of machine learning algorithms.

[0009] 4. Comparative advantages of feature redundancy and dependency reconstruction Traditional feature engineering typically uses physicochemical features directly for modeling, resulting in feature redundancy and unexplicitly modeled interdependencies. These implicit correlations between features can lead to model learning bias or overfitting. Transformers, through multi-head self-attention, can model global dependencies across the feature dimension, re-encoding the original feature space. The resulting high-dimensional embedded features not only retain the original physical meaning but also contain information about the synergistic effects between features.

[0010] 5. Breakthroughs in model generalization and feature fusion learning Most models based on feature statistical relationships show a significant drop in prediction accuracy when dealing with lithium-ion battery electrode materials of different systems. This is mainly because the models cannot generalize to new types of feature combinations. The globally dependent features extracted by the Transformer capture the intrinsic coupling relationships between multi-dimensional attributes, enabling a unified mapping of the feature space. After being fused with the original physicochemical features, the model possesses both data-driven and physical prior information, thus maintaining consistent performance in bandgap prediction tasks for multi-system and multi-structure electrode materials. This fusion strategy allows the model to be independent of structural files and possess predictive capabilities and interpretability across material systems, demonstrating strong application prospects.

[0011] In summary, developing a bandgap prediction model for lithium battery electrode materials that can integrate global feature information, is interpretable, and has cross-system generalization capabilities has become a core problem that urgently needs to be solved in the field of materials informatics. Summary of the Invention

[0012] In view of the deficiencies in the prior art, the purpose of this application is to provide a method and system for bandgap prediction and interpretation based on a multi-head self-attention mechanism.

[0013] The first aspect of this application provides a method for bandgap prediction and interpretation based on a multi-head self-attention mechanism, comprising: Obtain CSV files containing the physicochemical characteristics of lithium-ion battery materials and CSV files containing their band gap values; A Transformer-based encoder is used to model the global dependencies of the physicochemical features in the CSV file through a multi-head self-attention mechanism to determine the context embedding. The context embedding is spliced ​​with the physicochemical characteristics of the lithium-ion battery material to determine the training features; Based on the training features and the CSV file containing the bandgap values, a pre-defined CatBoost regressor is used for regression training to determine the Catformer model; The Catformer model is used to predict the band gap of the lithium-ion battery material under test based on its physicochemical characteristics, thereby determining the band gap of the lithium-ion battery material under test. In the bandgap prediction process, the SHAP analysis method is used to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material to determine the interpretation of the bandgap prediction.

[0014] Optionally, the physicochemical characteristics CSV file of the lithium-ion battery material corresponds one-to-one with the band gap value CSV file.

[0015] Optionally, the method further includes: Linear projection is performed on the physicochemical characteristics in the CSV file of the lithium-ion battery material to determine a fixed-length vector representation; A learnable positional encoding is introduced into the fixed-length vector representation to determine the embedding vector.

[0016] Optionally, the step of using a Transformer-based encoder to perform global dependency modeling on the CSV file of the physicochemical features through a multi-head self-attention mechanism to determine the context embedding includes: The embedding vector is input into the multi-head self-attention layer in the Transformer-based encoder; In each autofocus head, determine the relative rate of change and normalized distance between the physicochemical characteristics; In the feature space, Gaussian radial basis and angular basis functions are used to decompose the nonlinear dependencies and determine the high-dimensional relation embedding that can smoothly express the nonlinear relationships between features. The multi-head self-attention layer obtains information interaction and dependency relationships across feature dimensions through the multi-head attention mechanism, constructs a globally perceived dependency relationship representation in the feature embedding layer, and determines the context embedding.

[0017] Optionally, the Catformer model integrates an autonomous optimization module, which is used to optimize the Catformer model based on new material data of the same format by adopting a strategy of parameter freezing and layer-by-layer unfreezing.

[0018] Optionally, determining the band gap of the lithium-ion battery material to be tested based on its physicochemical characteristics and the Catformer model includes: Obtain a CSV file containing the physicochemical characteristics of the lithium-ion battery material to be tested; The CSV file containing the physicochemical characteristics of the lithium-ion battery material to be tested is input into the Catformer model to determine the band gap of the lithium-ion battery material to be tested.

[0019] Optionally, the step of using the SHAP analysis method to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material during the regression training process, and determining the interpretation of the bandgap prediction, includes: The SHAP analysis method is used to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material, generating a feature importance ranking map and a feature heatmap to determine the interpretation of the bandgap prediction. The feature importance ranking is used to characterize the importance of the physicochemical features and the dependencies between the physicochemical features to the predicted bandgap, and the feature heatmap is used to characterize the positive and negative effects of the physicochemical features and the dependencies between the physicochemical features on the predicted bandgap.

[0020] A second aspect of this application provides a bandgap prediction and interpretation system based on a multi-head self-attention mechanism, comprising: The physicochemical characteristic acquisition module is used to acquire CSV files of the physicochemical characteristics of lithium-ion battery materials and CSV files of the band gap values. The global dependency modeling module is used to perform global dependency modeling on the CSV file of the physicochemical features using a Transformer-based encoder through a multi-head self-attention mechanism, and to determine the context embedding. The feature splicing module is used to splice the context embedding with the physicochemical features of the lithium-ion battery material to determine training features; The model regression training module is used to perform regression training using a preset CatBoost regressor based on the training features and the CSV file of the band gap values, and to determine the Catformer model. The bandgap prediction module is used to predict the bandgap of the lithium-ion battery material under test using the Catformer model, and to determine the bandgap of the lithium-ion battery material under test. An interpretability analysis module is used to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material using the SHAP analysis method during the bandgap prediction process, and to determine the interpretation of the bandgap prediction.

[0021] A third aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the methods provided in the first aspect of this application.

[0022] A fourth aspect of this application provides an electronic device comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of any of the methods provided in the first aspect of this application.

[0023] This application presents a bandgap prediction and interpretation method based on a multi-head self-attention mechanism. This method utilizes a Transformer-based encoder to model the global dependencies of the material's physicochemical characteristics, obtains contextual embeddings to enhance the model's perceptual capabilities, and concatenates the contextual embeddings with the original physicochemical features to preserve the original physical meaning and enhance the model's interpretability. A pre-defined CatBoost regressor is used to perform regression training on the concatenated training features to obtain a Catformer model. This achieves accurate prediction of bandgap properties by combining Transformer and CatBoost. Finally, SHAP analysis is used to perform interpretability analysis on the bandgap prediction, revealing the physicochemical meaning of the prediction results and providing a general tool for the rapid design of lithium-ion battery electrode materials.

[0024] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a bandgap prediction and interpretation method based on a multi-head self-attention mechanism according to an exemplary embodiment.

[0026] Figure 2 This is a schematic diagram of the MAE training curve of a Catformer model on a lithium battery electrode material dataset according to an exemplary embodiment.

[0027] Figure 3 This is a schematic diagram of the 10-fold MAE training curve of a Catformer model on a lithium battery electrode material dataset according to an exemplary embodiment.

[0028] Figure 4 This is a schematic diagram illustrating a Top 5 feature importance ranking graph generated using the SHAP method according to an exemplary embodiment.

[0029] Figure 5 This is a schematic diagram comparing the final MAE of five models—RF, MLP, SVM, KNN, and CatBoost—after combining them with Transformer, according to an exemplary embodiment.

[0030] Figure 6 This is a schematic diagram illustrating the structure of a bandgap prediction and interpretation system based on a multi-head self-attention mechanism according to an exemplary embodiment. Detailed Implementation

[0031] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0032] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0034] Figure 1 This is a flowchart illustrating a bandgap prediction and interpretation method based on a multi-head self-attention mechanism according to an exemplary embodiment.

[0035] Reference Figure 1 As shown in one embodiment of this application, a method for bandgap prediction and interpretation based on a multi-head self-attention mechanism is provided, including steps S11 to S16.

[0036] S11, obtain CSV files of the physicochemical characteristics of lithium-ion battery materials and CSV files of band gap values.

[0037] Specifically, in the CSV file containing the physicochemical characteristics of lithium-ion battery materials, each column represents a characteristic.

[0038] The band gap values ​​are stored in a separate CSV file.

[0039] S12 employs a Transformer-based encoder to model the global dependencies of the physicochemical features in the CSV file through a multi-head self-attention mechanism, thereby determining the context embedding.

[0040] S13, the context embedding is spliced ​​with the physicochemical characteristics of lithium-ion battery materials to determine the training features.

[0041] S14. Based on the CSV file containing training features and bandgap values, perform regression training using a pre-defined CatBoost regressor to determine the Catformer model.

[0042] S15. The Catformer model is used to predict the band gap of the lithium-ion battery material under test based on its physicochemical characteristics, and the band gap of the lithium-ion battery material under test is determined.

[0043] S16. In the process of bandgap prediction, the SHAP analysis method is used to perform interpretability analysis on the bandgap prediction of lithium-ion battery materials to determine the interpretation of bandgap prediction.

[0044] The embodiments described above in this application model the global dependencies of the physicochemical characteristics of materials through a multi-head self-attention mechanism based on a Transformer encoder, obtain contextual embeddings to enhance the model's perceptual capabilities, and concatenate the contextual embeddings and original physicochemical characteristics to retain physical meaning and enhance the model's interpretability. A pre-defined CatBoost regressor is used to perform regression training on the concatenated training features to obtain a Catformer model, achieving accurate prediction of bandgap properties by combining Transformer and CatBoost. The SHAP analysis method is used to perform interpretability analysis on the bandgap prediction, revealing the physicochemical meaning of the prediction results, and providing a general tool for the rapid design of lithium battery electrode materials.

[0045] In some specific embodiments of this application, the physicochemical characteristics of lithium-ion battery materials may include, but are not limited to, physicochemical parameters such as material electronegativity, atomic electronegativity, HOMO-LUMO energy level difference, Bader charge, crystal field splitting, initial band gap value, unit cell volume, ionic radius, oxidation state, conductivity, and atomic polarizability.

[0046] Based on the above physicochemical parameters, band-related indicators such as electron affinity, first ionization energy, orbital hybridization parameters, and formation energy can be obtained through high-throughput quantitative calculations. Furthermore, it can be extended to other derived characteristics describing the electronic structure of electrode materials, such as density of states integral characteristics and Fermi level position shifts.

[0047] The physicochemical parameters of the lithium-ion battery materials in this application can be obtained from experimental databases.

[0048] In some specific embodiments of this application, the physicochemical characteristics CSV file of the lithium-ion battery material corresponds one-to-one with the band gap value CSV file.

[0049] In the above embodiments of this application, by organizing the physicochemical characteristics of lithium-ion battery materials into a CSV file, with each column representing a physicochemical characteristic and the bandgap value stored separately in another CSV file, the multidimensional characteristics of different materials can be standardized and represented, thus preparing for obtaining the input of the model.

[0050] In some specific embodiments of this application, a bandgap prediction and interpretation method based on a multi-head self-attention mechanism further includes steps S17 to S18.

[0051] S17. Perform linear projection on the physicochemical features in the CSV file of the physicochemical features of lithium-ion battery materials to determine a fixed-length vector representation.

[0052] S18 introduces a learnable positional encoding to the fixed-length vector representation to determine the embedding vector.

[0053] Specifically, a learnable positional or relative positional encoding is introduced into the fixed-length vector representation. That is, a trainable vector parameter is assigned to each feature position and trained together with the backbone network parameters of the model during subsequent model training to model the positional dependency and determine the embedding vector.

[0054] Steps S17 to S18 of this application embed and positionally encode the physicochemical characteristics of lithium-ion battery materials.

[0055] The embodiments described above in this application introduce learnable positional coding or relative positional coding to preserve the relative order and semantic information of the physicochemical feature sequence of lithium-ion battery materials.

[0056] Steps S17 to S18 of this application are performed before step S12.

[0057] To model the global dependencies of physicochemical features, in some specific embodiments of this application, for S12, a Transformer-based encoder is used to model the global dependencies of the physicochemical feature CSV file through a multi-head self-attention mechanism to determine the context embedding, which can be implemented as S121 to S124.

[0058] S121, the embedded vector is input into the multi-head self-attention layer in the Transformer-based encoder.

[0059] Specifically, scaling factors, layer normalization, residual connections, and feed-forward networks are used in the attention calculation of each self-attention layer to ensure training stability and deep expression capabilities.

[0060] The Transformer-based encoder in this application adopts a deep residual and layer normalization structure, including: a multi-head attention layer, a normalization layer, a feedforward network, and a random dropout layer. It uses a cross-layer residual connection method to improve the stability of model regression training, and adopts a progressive feature aggregation strategy to expand the feature dependency receptive field layer by layer. The number of attention heads and the dimension of the hidden layer are adaptively adjusted with the network depth to achieve the capture of dependency information from local to global.

[0061] S122, determine the relative rate of change and normalized distance between physicochemical characteristics in each self-attention head.

[0062] S123, in the feature space, Gaussian radial basis and angular basis functions are used to decompose the nonlinear dependencies and determine the high-dimensional relationship embedding that can smoothly express the nonlinear relationship between features.

[0063] Specifically, high-dimensional relation embeddings that can smoothly express nonlinear relationships between features are used to enhance the self-attention module's ability to model feature dependencies. These high-dimensional relation embeddings can smoothly express complex nonlinear relationships between features.

[0064] S124, the multi-head self-attention layer obtains information interaction and dependency relationships across feature dimensions through the multi-head self-attention mechanism, constructs a globally perceived dependency relationship representation in the feature embedding layer, and determines the contextual embedding.

[0065] Specifically, steps S122 to S124 are performed in each self-attention head of each layer based on the Transformer encoder.

[0066] Based on the above steps S17 to S18 and steps S121 to S124, the specific process of determining the context embedding is as follows: For example, 1) the physicochemical features are mapped to a fixed-length vector representation through linear projection, and a learnable positional encoding or relative positional encoding is added to form an embedding vector containing feature semantics and positional information.

[0067] 2) In each self-attention head, each feature vector simultaneously calculates the correlation with all other features in the embedding vector, and aggregates the information of other features based on the correlation weight, thereby capturing global dependencies across features.

[0068] 3) The outputs of each attention head are concatenated and linearly mapped to achieve cross-head information integration and obtain the fused global dependency representation.

[0069] 4) Perform a feedforward network nonlinear transformation on the fused global dependency representation, and combine residual connections and layer normalization to further enhance the nonlinear relationship and stability between features.

[0070] 5) Repeat steps 1) to 4) above multiple layers, iteratively optimizing the dependencies between features at each layer, and finally outputting context embeddings, which contain globally aware dependency representations and can be used for bandgap prediction and interpretability analysis.

[0071] Specifically, in the multi-head self-attention layer, each self-attention head independently learns the interactions between different physicochemical features in the embedding vector, and in the final attention layer, the outputs of each attention head are weighted and aggregated to generate global dependency information of the material, forming contextual embedding.

[0072] The Transformer-based encoder learns the contribution of different physicochemical features to material properties through a self-attention learning mechanism. It can adaptively adjust the weights of interactions between different atoms based on the physicochemical features of lithium-ion battery materials acquired as training data, thereby accurately capturing the overall behavior of the material.

[0073] Context embedding preserves both local features and global interaction information.

[0074] The embodiments described above capture global feature dependencies through a multi-head self-attention mechanism based on a Transformer encoder, obtain contextual embeddings that reflect multi-dimensional feature interactions, transform local physicochemical features into global features containing feature dependencies, and improve the model's perceptual capabilities.

[0075] This application combines contextual embedding with the physicochemical characteristics of lithium-ion battery materials to determine training features, enabling the model to learn the physicochemical properties of materials and further enhancing the model's understanding of material properties and their interactions.

[0076] To construct a bandgap prediction model for lithium-ion battery electrode materials that can integrate global feature information, is interpretable, and has cross-system generalization capabilities, in some specific embodiments of this application, for S14, regression training is performed using a preset CatBoost regressor based on a CSV file of training features and bandgap values ​​to determine the Catformer model, which may include: The training features are used as input to a pre-defined CatBoost regressor, and the CSV file of the bandgap values ​​is used as output to perform regression training, thus determining the Catformer model.

[0077] Specifically, the Transformer module of the Catformer model includes multiple graph convolutional layers and attention layers. The graph convolutional layers are used to aggregate information from local neighborhoods, while the attention layers are used to capture the interactions between global atoms.

[0078] The regression training process includes forward propagation and backward propagation, and optimizes the network parameters of the Catformer model by minimizing the prediction error. The prediction error is the error between the predicted bandgap value and the CSV file containing the bandgap value.

[0079] During regression training, dropout and weight regularization are used in the attention layer and feedforward network to prevent overfitting.

[0080] For example, the weight regularization method can employ L2 regularization.

[0081] Self-supervised loss from multi-task learning or contrastive learning can also be introduced as an auxiliary task to enhance generalization ability under small sample or heterogeneous data.

[0082] In the above embodiments of this application, a preset CatBoost regressor is used to perform regression fitting on the Catformer model. The preset CatBoost regressor uses a gradient boosting-based decision tree to perform regression prediction on the band gap value of the material. It can achieve high-precision modeling under sparse data and effectively handle nonlinear relationships. It realizes the machine learning algorithm to combine physicochemical features and dependency features to gradually learn the influence of material properties and their global dependencies on the band gap value.

[0083] In some specific embodiments of this application, the Catformer model integrates an autonomous optimization module. The autonomous optimization module is used to optimize the Catformer model based on new material data of the same type and format, using a strategy of parameter freezing and layer-by-layer unfreezing.

[0084] Specifically, new material data in the same format may include, but is not limited to, material data containing the same physicochemical characteristics as the original data.

[0085] The self-optimization module only updates modules that are significantly affected by new data, avoiding the forgetting of original training knowledge and improving the stability of optimization.

[0086] In the embodiments described above, the self-optimization module can optimize the Catformer model based on new material data of the same format, thereby improving the model performance of the Catformer model and increasing the accuracy of bandgap prediction.

[0087] During the regression fitting of the Catformer model using the preset CatBoost regressor, new material data of the same format can be continuously added through the self-optimization module to achieve adaptive optimization of the Catformer model.

[0088] Figure 2This is a schematic diagram of the MAE training curve of a Catformer model on a lithium battery electrode material dataset according to an exemplary embodiment.

[0089] Reference Figure 2 As shown, MAE (Mean Absolute Error) represents the average absolute error between the predicted bandgap value and the true bandgap value.

[0090] The bandgap prediction and interpretation method based on the multi-head self-attention mechanism proposed in this application uses a pre-set CatBoost regressor to train the Catformer model. As the model is trained, the error in predicting the bandgap value becomes smaller and smaller.

[0091] Figure 3 This is a schematic diagram of the 10-fold MAE training curve of a Catformer model on a lithium battery electrode material dataset according to an exemplary embodiment.

[0092] Reference Figure 3 As shown, the Catformer model's 10-fold cross-validation results on the lithium battery electrode material dataset show little fluctuation and are generally good, demonstrating that the Catformer model has good generalization ability.

[0093] To achieve bandgap prediction for lithium-ion battery materials, in some specific embodiments of this application, for S15, the bandgap of the lithium-ion battery material to be tested is determined based on the physicochemical characteristics of the lithium-ion battery material to be tested and the Catformer model, which can be achieved using S151 to S152.

[0094] S151, Obtain a CSV file containing the physicochemical characteristics of the lithium-ion battery material to be tested.

[0095] S152, input the CSV file of the physicochemical characteristics of the lithium-ion battery material to be tested into the Catformer model to determine the band gap of the lithium-ion battery material to be tested.

[0096] The embodiments described above in this application directly use the physicochemical characteristics of lithium-ion battery materials and the Catformer model that is fitted by bandgap value regression to predict the bandgap of lithium-ion battery materials. This method can integrate global feature information and has cross-system generalization ability, providing a general tool for the rapid design of lithium battery electrode materials.

[0097] To achieve interpretability analysis in the bandgap prediction process, in some specific embodiments of this application, for 16, the SHAP analysis method is used to perform interpretability analysis on the bandgap prediction of lithium-ion battery materials during the bandgap prediction process to determine the bandgap prediction interpretation. This can be achieved by: The SHAP analysis method was used to perform interpretability analysis on the bandgap prediction process of the lithium-ion battery material under test, generating a feature importance ranking map and a feature heat map to determine the interpretation of the bandgap prediction.

[0098] Specifically, the feature importance ranking map is used to characterize the importance of physicochemical features and the dependencies between physicochemical features to the predicted band gap, and the feature heatmap is used to characterize the positive and negative effects of physicochemical features and the dependencies between physicochemical features on the predicted band gap.

[0099] Figure 4 This is a schematic diagram illustrating a Top 5 feature importance ranking graph generated using the SHAP method according to an exemplary embodiment.

[0100] For example, refer to Figure 4 As shown, the SHAP analysis plots the top five features that have the most significant impact on the prediction results when using the Catformer model to predict the bandgap of lithium-ion battery materials. These features are mean_electronegativity, mean_atomic_number, mean_covalent radius, mean ionization_energy, and mean_valence_electron_count. This demonstrates how these five features enable the bandgap of lithium-ion battery materials to reach the expected value.

[0101] For example, mean electronegativity ranks first, indicating that the mean electronegativity of a material has the greatest impact on the material's band gap value.

[0102] This application achieves interpretability of physicochemical characteristics through the SHAP analysis method.

[0103] The embodiments described above in this application employ the SHAP analysis method to achieve interpretability analysis of bandgap prediction, revealing the physicochemical significance of the prediction results and providing a general tool for the rapid design of lithium battery electrode materials.

[0104] This application provides a bandgap prediction and interpretation method based on a multi-head self-attention mechanism, which can be used for rapid electronic property prediction of systems such as lithium battery electrode materials, sodium-ion conductors, and semiconductor materials. It has universality and is applicable to bandgap prediction of various materials, such as metal oxides, sulfides, perovskites, and high-entropy materials. Only the physicochemical characteristics CSV file of the corresponding material needs to be obtained to directly predict the bandgap value and perform interpretation analysis. By adjusting the number of encoder layers, the number of attention heads, or the CatBoost hyperparameters based on Transformer, it can be flexibly adapted to different dataset sizes and feature dimensions, ensuring the universality and scalability of the model.

[0105] The Catformer model provided in this application can also be extended to various electronic structure and performance prediction tasks, including: band gap, formation energy, migration barrier, Fermi level shift, electron mobility, dielectric constant and conductivity.

[0106] This application provides a bandgap prediction and interpretation method based on a multi-head self-attention mechanism. It predicts the material bandgap using a Transformer model and performs interpretability analysis using Catboost. The material's physicochemical characteristics are used as input features to the Transformer-based model. The encoder part of the Transformer architecture encodes these characteristics, and then a Catboost regression model is established by combining the original physicochemical characteristics with the Catboost regressor. This model predicts the final bandgap value for the corresponding material. This application is not limited to any specific material and has universal applicability. It can accurately predict material bandgap values ​​and perform interpretability analysis. Only the physicochemical characteristics CSV file data of the material are needed to obtain the bandgap value.

[0107] Specifically, a CSV file containing the physicochemical features of a material is obtained. A multi-head self-attention mechanism in the encoder part based on a transformer extracts the dependencies between global features, yielding rich global information features. These features are then concatenated with the original physicochemical features to form the final training features, preserving the material's physicochemical properties. For the training features, the CatBoost regressor, currently the best-performing regressor, is used for training, directly predicting the material's bandgap value. This method achieves high-precision prediction of the electrode material's bandgap by extracting the dependencies between material features and combining them with physicochemical properties. It also provides visualization of feature importance, assisting in material performance optimization and design.

[0108] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0109] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0110] Figure 5 This is a schematic diagram comparing the final MAE of five models—RF, MLP, SVM, KNN, and CatBoost—after combining them with Transformer, according to an exemplary embodiment.

[0111] Reference Figure 5As shown, the final model MAE of five commonly used machine learning models—RF, MLP, SVM, KNN, and CatBoost—after being combined with Transformer, shows that CatBoost performs best as the prediction model. This verifies that the Catformer model trained by the bandgap prediction and interpretation method based on multi-head self-attention mechanism proposed in this application has the best prediction performance.

[0112] This application provides a bandgap prediction and interpretation method based on a multi-head attention mechanism. Through feature global dependency modeling, feature splicing, CatBoost regression and SHAP interpretability analysis, it achieves high-precision and interpretable bandgap prediction, demonstrating the application of the Catformer model in material bandgap prediction.

[0113] Figure 6 This is a schematic diagram illustrating the structure of a bandgap prediction and interpretation system based on a multi-head self-attention mechanism according to an exemplary embodiment.

[0114] Reference Figure 6 As shown in one embodiment of this application, a bandgap prediction and interpretation system 100 based on a multi-head self-attention mechanism is provided, including: a physicochemical feature acquisition module 110, a global dependency modeling module 120, a feature splicing module 130, a model regression training module 140, a bandgap prediction module 150, and an interpretability analysis module 160.

[0115] The physicochemical characteristic acquisition module 110 is used to acquire CSV files of the physicochemical characteristics of lithium-ion battery materials and CSV files of band gap values. The global dependency modeling module 120 is used to perform global dependency modeling on CSV files of physicochemical features using a Transformer-based encoder through a multi-head self-attention mechanism, and to determine the context embedding. The feature splicing module 130 is used to splice the context embedding with the physicochemical features of the lithium-ion battery material to determine the training features; The model regression training module 140 is used to perform regression training using a preset CatBoost regressor based on a CSV file of training features and bandgap values ​​to determine the Catformer model. The bandgap prediction module 150 is used to predict the bandgap of the lithium-ion battery material under test by using the Catformer model to determine the bandgap of the lithium-ion battery material under test. The interpretability analysis module 160 is used to perform interpretability analysis on the bandgap prediction of lithium-ion battery materials using the SHAP analysis method during the bandgap prediction process, and to determine the interpretation of the bandgap prediction.

[0116] The embodiments described above employ a multi-head self-attention mechanism based on a Transformer encoder to model the global dependencies of the physicochemical characteristics of materials, obtain contextual embeddings to enhance the model's perceptual capabilities, and concatenate the contextual embeddings and original physicochemical characteristics to retain physical meaning and enhance the model's interpretability. A pre-defined CatBoost regressor is used to perform regression training on the concatenated training features to obtain a Catformer model, achieving accurate prediction of bandgap properties by combining Transformer and CatBoost. The SHAP analysis method is used to perform interpretability analysis on the bandgap prediction, revealing the physicochemical meaning of the prediction results, and providing a general tool for the rapid design of lithium battery electrode materials.

[0117] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0118] Based on the same technical concept, in some specific embodiments of this application, a terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and a method that the processor can use to execute when executing the program.

[0119] Based on the same technical concept, in some specific embodiments of this application, a computer-readable storage medium is provided on which a computer program is stored, which can be used to execute a method when the program is executed by a processor.

[0120] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0121] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0122] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0123] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0124] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A method for bandgap prediction and interpretation based on a multi-head self-attention mechanism, characterized in that, include: Obtain CSV files containing the physicochemical characteristics of lithium-ion battery materials and CSV files containing their band gap values; A Transformer-based encoder is used to model the global dependencies of the physicochemical features in the CSV file through a multi-head self-attention mechanism to determine the context embedding. The context embedding is spliced ​​with the physicochemical characteristics of the lithium-ion battery material to determine the training features; Based on the training features and the CSV file containing the bandgap values, a pre-defined CatBoost regressor is used for regression training to determine the Catformer model; The Catformer model is used to predict the band gap of the lithium-ion battery material under test based on its physicochemical characteristics, thereby determining the band gap of the lithium-ion battery material under test. In the bandgap prediction process, the SHAP analysis method is used to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material to determine the interpretation of the bandgap prediction.

2. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 1, characterized in that, The CSV files containing the physicochemical characteristics of the lithium-ion battery materials correspond one-to-one with the CSV files containing the band gap values.

3. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 1, characterized in that, The method further includes: Linear projection is performed on the physicochemical characteristics in the CSV file of the lithium-ion battery material to determine a fixed-length vector representation; A learnable positional encoding is introduced into the fixed-length vector representation to determine the embedding vector.

4. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 3, characterized in that, The step involves using a Transformer-based encoder to model the global dependencies of the physicochemical features in the CSV file through a multi-head self-attention mechanism, determining the context embedding, including: The embedding vector is input into the multi-head self-attention layer in the Transformer-based encoder; In each autofocus head, determine the relative rate of change and normalized distance between the physicochemical characteristics; In the feature space, Gaussian radial basis and angular basis functions are used to decompose the nonlinear dependencies and determine the high-dimensional relation embedding that can smoothly express the nonlinear relationships between features. The multi-head self-attention layer obtains information interaction and dependency relationships across feature dimensions through the multi-head attention mechanism, constructs a globally perceived dependency relationship representation in the feature embedding layer, and determines the context embedding.

5. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 1, characterized in that, The Catformer model integrates an autonomous optimization module, which is used to optimize the Catformer model based on new material data of the same format by adopting a strategy of parameter freezing and layer-by-layer unfreezing.

6. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 1, characterized in that, The determination of the band gap of the lithium-ion battery material under test based on its physicochemical characteristics and the Catformer model includes: Obtain a CSV file containing the physicochemical characteristics of the lithium-ion battery material to be tested; The CSV file containing the physicochemical characteristics of the lithium-ion battery material to be tested is input into the Catformer model to determine the band gap of the lithium-ion battery material to be tested.

7. The bandgap prediction and interpretation method based on multi-head self-attention mechanism according to claim 1, characterized in that, The bandgap prediction process employs the SHAP analysis method to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material, determining the interpretation of the bandgap prediction, including: The SHAP analysis method is used to perform interpretability analysis on the bandgap prediction process of the lithium-ion battery material to be tested, generating a feature importance ranking map and a feature heatmap to determine the interpretation of the bandgap prediction. The feature importance ranking is used to characterize the importance of the physicochemical features and the dependencies between the physicochemical features to the predicted bandgap, and the feature heatmap is used to characterize the positive and negative effects of the physicochemical features and the dependencies between the physicochemical features on the predicted bandgap.

8. A bandgap prediction and interpretation system based on a multi-head self-attention mechanism, characterized in that, include: The physicochemical characteristic acquisition module is used to acquire CSV files of the physicochemical characteristics of lithium-ion battery materials and CSV files of the band gap values. The global dependency modeling module is used to perform global dependency modeling on the CSV file of the physicochemical features using a Transformer-based encoder through a multi-head self-attention mechanism, and to determine the context embedding. The feature splicing module is used to splice the context embedding with the physicochemical features of the lithium-ion battery material to determine training features; The model regression training module is used to perform regression training using a preset CatBoost regressor based on the training features and the CSV file of the band gap values, and to determine the Catformer model. The bandgap prediction module is used to predict the bandgap of the lithium-ion battery material under test using the Catformer model, and to determine the bandgap of the lithium-ion battery material under test. An interpretability analysis module is used to perform interpretability analysis on the bandgap prediction of the lithium-ion battery material using the SHAP analysis method during the bandgap prediction process, and to determine the interpretation of the bandgap prediction.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

Images