CNN-LSTM-Attention-based metal ion battery electrode parameter prediction method

Abstract

The invention provides a metal ion battery electrode parameter prediction method based on CNN-LSTM-Attention. The metal ion battery electrode parameter prediction method comprises the following steps: step 1, data preprocessing; step 2, model construction: a CNN-LSTM-Attention model is designed in combination with a convolutional neural network (CNN), a long short-term memory (LSTM) network and an Attention; 3, training the model: training the model by using training set data; 4, determining an optimal hyper-parameter of the model through a Bayesian optimization algorithm; step 5, final training and testing: based on the optimal hyper-parameter, re-initializing the CNN-LSTM-Attention model, and then training the model by using training set data to obtain a final model; and inputting test set data into the final model to obtain a prediction result containing the average voltage, the specific energy and the specific capacity. According to the invention, the prediction accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the fields of materials science and battery technology, and particularly relates to a method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention. Background Technology

[0002] Metal-ion batteries (MIBs), as an emerging energy storage technology, have gained widespread application and attention in electric vehicles, portable electronic devices, and large-scale energy storage systems due to their high energy density, long cycle life, and fast charge / discharge capabilities. Particularly in the electric vehicle sector, MIBs are considered a crucial material choice for next-generation energy storage systems, potentially addressing safety and cycle life issues inherent in traditional batteries. Electrode materials, as a key component of MIBs, directly impact the overall battery performance. Parameters such as average voltage, specific capacity, and specific energy are important indicators for evaluating the suitability of electrode materials, determining the battery's cycleability, coulombic efficiency, and safety.

[0003] In previous studies on battery performance prediction, feature selection typically focused on single component or structural features. The model couldn't adequately analyze the contribution of each feature to the overall performance. Therefore, finding an optimal combination of features is crucial for accurate prediction of target performance. In summary, existing technologies suffer from the following shortcomings: 1) Incomplete feature extraction: Traditional methods often struggle to fully capture the complex characteristics of electrode materials when predicting key electrochemical performance (such as average voltage and specific capacity) of metal-ion battery electrodes due to limited feature extraction capabilities. 2) Insufficient capture of time-series data dependencies: The electrochemical performance of electrode materials is often closely related to time-series data, but existing technologies often fail to effectively capture these long-term dependencies. 3) Inflexible feature importance adjustment: During prediction, the contribution of different features to the results may change over time or under varying conditions. Traditional methods often lack mechanisms for dynamically adjusting feature importance. 4) Low efficiency in model hyperparameter optimization: The selection of model hyperparameters is critical to prediction performance, but traditional optimization methods are often time-consuming and ineffective. 5) Limited applicability of prediction models: Traditional prediction models are often designed for specific types or structures of electrode materials, lacking broad applicability. 6) Unstable prediction performance: In complex and ever-changing real-world application scenarios, the stability and robustness of the prediction model are particularly important. Summary of the Invention

[0004] The purpose of this invention is to provide a method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention. This method integrates Convolutional Neural Network (CNN), Long Short-Term Memory Network (LSTM), Attention mechanism, and Bayesian optimization algorithm to form a model that efficiently predicts the key electrochemical performance of electrode materials.

[0005] The electrode parameters of a metal-ion battery include: battery voltage, specific energy, and specific capacity.

[0006] A CNN-LSTM-Attention-based method for predicting electrode parameters in metal-ion batteries is proposed. This method extracts local features from the input data using CNN, captures long-term dependencies in sequential data using LSTM, and dynamically adjusts the importance of features using an attention mechanism, thereby gaining a more accurate understanding of the electrode material's properties. Simultaneously, a Bayesian optimization algorithm is used to optimize the model's hyperparameters, ensuring optimal performance. This method significantly improves prediction accuracy. The technical approach employed is as follows:

[0007] A method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention includes the following steps:

[0008] Step 1: Data preprocessing.

[0009] Download the dataset from the Materials Project's MP API and extract features in four aspects: stoichiometry, crystal structure, electronic structure, and other battery properties.

[0010] Stoichiometric properties include the types and proportions of elements contained in the electrode, and whether the material is in a high-ionic or low-ionic state.

[0011] The characteristics of crystal structure are determined by the regular three-dimensional periodic arrangement of atoms, ions and molecules in space, such as crystal lattice systems and space group numbering.

[0012] Electronic structure properties include band gap, formation energy, and Fermi level.

[0013] Other battery properties may include ion type, stable charging and discharging in high and low ion states.

[0014] Based on these alternative features, this paper obtains two sets of input features.

[0015] The first set of features uses only stoichiometric properties, crystal structure characteristics, and ion species. These features are relatively easy to obtain and do not rely on experiments or extensive calculations. The purpose of using this set of features for model training is to develop a deep learning model that uses only simple input features. This model does not need to be based on first-principles calculations or experiments, and can make preliminary predictions of the electrochemical performance of batteries relatively independently, efficiently, and quickly. This allows for the screening of a large number of candidate materials and reduces the amount of theoretical calculations required.

[0016] The second set contains all the aforementioned candidate features. Deep learning models trained with this set of features cannot run independently because they include properties such as bandgap and stability derived from theoretical calculations. This set of features is used to provide a deep learning model that can utilize more inputs, combining theoretical calculations to fully mine and analyze available database resources for the screening and performance prediction of novel battery materials. Data cleaning and dimensionality reduction methods, including manual encoding, principal component analysis, and min-max normalization, were employed.

[0017] Step 1A: Read data from the database:

[0018] The `read_csv()` function provided by Pandas stores data as a DataFrame object, facilitating subsequent data processing, filtering, analysis, and manipulation. This allows the data to exist in a structured and easily manipulated form, providing a data foundation for subsequent steps. It provides the original dataset for subsequent data preprocessing, feature extraction, model training, and other operations, enabling various data processing as needed, such as feature filtering, target variable extraction, and data transformation.

[0019] The database stores chemical formulas that represent the elemental composition and atomic ratio of electrode materials;

[0020] Ion type refers to the type of charge carrier ions that participate in the electrochemical reaction in a metal-ion battery;

[0021] Crystal lattice refers to the framework structure formed by the periodic arrangement of atoms / ions in a crystal;

[0022] Space group, a mathematical symbol used to describe the symmetry of crystal structures;

[0023] Formation energy refers to the energy change per unit atom during the synthesis of a material from its stable constituent elements.

[0024] Band gap, in semiconductor or insulating materials, is the energy difference between the top of the valence band (the highest energy level occupied by electrons) and the bottom of the conduction band (the lowest energy level not occupied by electrons).

[0025] The Fermi level is the highest energy level occupied by electrons in a material at absolute zero.

[0026] Energy per atom refers to the total energy of the electrode material divided by the total number of atoms.

[0027] Stability refers to the structural stability of electrode materials in both high-ion (charged) and low-ion (discharged) states.

[0028] Step 1B: Convert the encoding to provide a suitable data type for subsequent models:

[0029] The categorical information in the working_ion column is converted into a numeric code using the fit_transform() method of LabelEncoder and stored in the working_ion_numeric column.

[0030] This helps the model learn category information as features, ensures data format consistency and compatibility, and provides a suitable data type for subsequent model training.

[0031] Step 1C: Normalize the target variable:

[0032] Different features and target variables may have different dimensions and ranges of values. If normalization is not performed, some features with large ranges may dominate model training, affecting the model's learning of other features and reducing model performance.

[0033] First, create a normalization object, then normalize the target variable.

[0034] Step 1D: Divide the dataset into training, validation, and test sets:

[0035] To evaluate the model's performance and generalization ability, the dataset needs to be divided into different parts.

[0036] The training set is used to train the model, enabling the model to learn patterns in the data;

[0037] The validation set is used to evaluate model performance during training and prevent overfitting;

[0038] The test set is used to evaluate the performance of the final model on unseen data.

[0039] Proper partitioning helps adjust the model's hyperparameters and evaluate its performance, ensuring that the model not only performs well on training data but also has good predictive ability on unseen data.

[0040] Normalization ensures that all data are scaled uniformly, providing a stable data foundation for subsequent model training.

[0041] Next, in order to evaluate the model's performance on unseen data, we need to split the dataset into training and test sets.

[0042] Here, we choose to use the `train_test_split` function to complete this step. This function is a commonly used tool in machine learning libraries (such as scikit-learn) for randomly splitting a dataset into training and test sets.

[0043] Step 1E: Adjust the input format to match the data with the model's input requirements:

[0044] For the training set feature data X_train, we first need to clarify the three-dimensional input format required by the CNN-LSTM-Attention model, namely [batch_size, sequence_length, input_size].

[0045] We use the reshape method to reshape X_train to the desired 3D format and correctly set its parameters. This operation ensures that the format of X_train matches the input requirements of the CNN-LSTM-Attention model.

[0046] For the test set feature data X_test, we use the same processing method as for the training set. This is because the model needs to process data in the same format as during the training phase to ensure the accuracy and reliability of the prediction results.

[0047] Similarly, we use the reshape method to reshape X_test to the [batch_size, sequence_length, input_size] format.

[0048] This step ensures that the format of X_test is consistent with X_train and meets the input requirements of the CNN-LSTM-Attention model. This lays a solid foundation for correctly inputting the test set data into the model for prediction operations.

[0049] Step 2, Model Building:

[0050] Define a CNN-LSTM-Attention network to build the model:

[0051] (1) CNN section:

[0052] Convolutional layers can automatically extract local features from input data. For local patterns and correlations that may exist in battery data, convolutional operations can transform these local information into higher-level feature representations, enhancing the model's ability to extract features from the data.

[0053] Pooling layers reduce data dimensionality through downsampling operations while preserving important local feature information, which helps reduce computational cost, prevent overfitting, and make features more representative.

[0054] Convolutional operations are performed using nn.Conv1d or nn.Conv2d to extract local features from the data. Pooling layers nn.MaxPool1d and nn.AvgPool1d are used to reduce the data dimensionality, providing more representative feature data for subsequent LSTM layers.

[0055] (2) LSTM section:

[0056] The data processed by the CNN will be used as the input to the LSTM.

[0057] Battery performance data is often time-dependent. LSTM layers can process time series data, capture long-term dependencies in the data, and learn complex patterns in time series, making them very effective for analyzing changes in battery performance over time.

[0058] The hidden_size determines the model's ability to capture the complexity of data patterns, while num_layers affects the model's ability to learn non-linear relationships. Batch_first = True conforms to common data organization habits and facilitates data processing.

[0059] By using nn.LSTM to process the feature data after CNN processing, we can learn the temporal correlation in time series data and provide richer feature information for the subsequent Attention mechanism.

[0060] (3) Attention part:

[0061] Traditional LSTM layers may give equal attention to information at different positions in a sequence, while the Attention mechanism can make the model pay more attention to the features that have an important impact on the prediction results, thus improving the model's focus on important information.

[0062] Multiple attention heads can focus on feature information from different angles, enhancing the model's expressive power and improving its ability to model complex relationships.

[0063] Using nn.MultiheadAttention to calculate the attention weights of the LSTM output and the weighted summation of the output helps the model better focus on important features and improves prediction performance.

[0064] (4) Fully connected layer:

[0065] The fully connected layer transforms the feature representation after Attention processing into specific predicted values, maps the feature space to the final prediction space, and converts the extracted and processed features into comparable prediction results.

[0066] For each sample, predicted values ​​of voltage, specific energy, and specific capacity are generated to facilitate comparison with the true values ​​and evaluate model performance.

[0067] The features output by the Attention layer are converted into specific predicted values ​​through the nn.Linear layer, so as to obtain the predicted results of battery voltage, specific energy and specific capacity, and provide a basis for model performance evaluation.

[0068] Forward propagation:

[0069] Forward propagation is the computational process of a model from input to output. Through the computation of each layer, the input data is transformed into the prediction result, realizing the forward flow of data in the model.

[0070] It processes the input data through convolution, pooling, LSTM, attention, and fully connected layers to finally obtain the predicted value, which is the core computational process of model prediction.

[0071] In the forward method, the input data is processed through each layer in sequence, and x is converted into the final prediction results voltage_pred, energy_pred and capacity_pred, which provide prediction results for subsequent loss calculation and optimization.

[0072] Step 3: Train the model:

[0073] Step 3A: Define the train_model function for model training and validation evaluation:

[0074] The `train_model` function is defined to encapsulate the training and validation evaluation process, making it convenient to call multiple times during Bayesian optimization to evaluate the model performance under different parameter combinations.

[0075] It helps to automatically evaluate the impact of different parameter combinations on model performance and facilitates the search for the optimal parameter combination.

[0076] This function is used to train the model and evaluate its performance on the validation set. It provides a tool for Bayesian optimization to evaluate the model performance under different parameter combinations, and facilitates parameter search and optimization.

[0077] Step 3B: Define the loss function and optimizer:

[0078] The statement `criterion = nn.MSELoss()` defines the loss function as Mean Squared Error Loss (MSE), and its mathematical formula is: Where n is the number of samples, and yi is the true value. This is a predicted value.

[0079] The root mean square error is the square root of the root mean square error, that is... It measures the average deviation between predicted and actual values; the smaller the value, the more accurate the model prediction.

[0080] The `optimizer = torch.optim.Adam(model.parameters(), lr = lr)` function creates an Adam optimizer. Adam is a commonly used gradient-based optimization algorithm that combines the advantages of adaptive learning rate adjustment, enabling more efficient updates to model parameters. `model.parameters()` retrieves all learnable parameters of the model (such as weight matrices and bias vectors in CNN layers, LSTM layers, Attention layers, and fully connected layers). `lr` is the learning rate parameter passed in earlier, which controls the step size for each parameter update.

[0081] Step 3C: Training.

[0082] Setting num_epochs determines the training duration; multiple iterations of the training set allow the model to fully learn the patterns and relationships in the data.

[0083] Switching the model to training mode ensures that layers such as Dropout function correctly, guaranteeing the model's proper behavior during training.

[0084] Multiple training iterations allow the model to continuously learn data features. `model.train()` ensures the model is in "training mode," enabling the Dropout layer to operate at the set dropout rate. This prevents the model from over-memorizing training set noise during training, improving generalization ability. It also provides the correct model state for subsequent forward propagation, backpropagation, and parameter updates.

[0085] Step 3D: Backpropagation and Parameter Optimization

[0086] Gradient zeroing ensures the accuracy of gradient information during each backpropagation, avoiding erroneous updates caused by gradient accumulation.

[0087] Backpropagation calculates the gradient based on the loss function and stores it in the parameters. The optimizer updates the parameters based on the gradient, enabling the model to gradually adjust the parameters to minimize the loss and improve performance.

[0088] Gradient calculation and parameter updates are implemented using optimizer.zero_grad(), loss.backward(), and optimizer.step(), continuously optimizing model parameters and gradually improving the model towards optimal performance.

[0089] Step 3E: Model evaluation on the validation set:

[0090] Evaluate model performance on the validation set to ensure the model is evaluated on data not used in training, thus avoiding overfitting.

[0091] Inverse normalization transforms the prediction results into the original data range, and uses the mean squared error to calculate the loss on the validation set, providing a more intuitive evaluation of model performance.

[0092] The model is switched to evaluation mode using model.eval(), which evaluates the model's performance on the validation set and calculates the loss on the validation set, providing a basis for adjusting the model and optimizing parameters.

[0093] Step 3F: Return the average loss:

[0094] The average loss on the validation set is a comprehensive indicator for evaluating model performance and a key basis for judging the quality of models by parametric search algorithms such as Bayesian optimization.

[0095] It reflects the model's performance on the validation set and provides an evaluation criterion for finding the optimal combination of parameters.

[0096] The average loss on the validation set is used as the return value of the train_model function, providing a quantitative metric for evaluating the performance of the Bayesian optimization algorithm and helping to find the optimal combination of parameters.

[0097] Step 4: Bayesian optimization of parameters.

[0098] Step 4A: Define the parameter space:

[0099] Define a parameter search space for Bayesian optimization, including the dimension size of the hidden state vector in the LSTM layer (hidden_size), the number of stacked LSTM layers (num_layers), the kernel size (kernel_size), the number of attention heads (num_heads), the learning rate parameter (lr), etc., to expand the search range and find better combinations of hyperparameters.

[0100] A reasonable range of parameter search helps to find hyperparameter combinations that improve model performance and generalization ability.

[0101] The pbounds dictionary specifies the range of parameters to be searched for in Bayesian optimization, guiding the optimization algorithm to find the optimal solution among different combinations of hyperparameters and improving model performance.

[0102] Step 4B: Create a Bayesian optimization object:

[0103] Create a BayesianOptimization object, use the train_model function as the objective function, and combine the parameter space and random state to search for the optimal parameter combination using the Bayesian optimization principle.

[0104] To ensure repeatability of each run, leverage the advantages of Bayesian optimization algorithms to continuously update the probability model based on existing results and search for better parameters.

[0105] Create a Bayesian optimization object and use an anonymous function call to the train_model function as the objective function to provide an optimization framework and tools for finding the optimal parameter combination.

[0106] Step 4C: Perform Bayesian optimization iterative search:

[0107] First, random exploration is conducted to gain a preliminary understanding of the parameter space. Then, based on Bayes' theorem, iterative search is performed to find a better combination of parameters so that the average loss of the model on the validation set is minimized.

[0108] By utilizing the principles of Bayesian optimization and iterative search, the parameter combinations are continuously adjusted to gradually find the optimal hyperparameter combination, thereby improving model performance.

[0109] The iterative search is performed using the optimizer.maximize() method, with initial random exploration using init_points and formal iteration using n_iter, ultimately finding the parameter combination that minimizes the average loss on the validation set.

[0110] Step 4D: Obtain the optimal parameters after optimization.

[0111] Obtain the optimal parameter combination found by Bayesian optimization to provide the best parameters for the final model training, ensuring that the final model has better performance potential.

[0112] The optimal combination of parameters can enable the final model to converge faster during training and achieve better performance.

[0113] The optimal parameters are obtained through optimizer.max['params'], converted to the appropriate data type, and used to provide the best parameters for the final model construction and training.

[0114] Step 5, Final Training and Testing:

[0115] The CNN-LSTM-Attention model is reinitialized using the optimal parameter combination obtained through Bayesian optimization.

[0116] Pass parameters such as best_hidden_size, best_num_layers, best_lr, best_kernel_size, and best_num_heads to the model's constructor to create the final optimized model.

[0117] The model is trained using the complete training set data, following the previous training process, including setting num_epochs, using the Adam optimizer, calculating loss, backpropagation, parameter updates, etc., to fully train the model and achieve optimal performance.

[0118] The trained model is evaluated on a test set by inputting the test set data into the final model to obtain prediction results. For both the prediction and actual results, a denormalization operation is performed using the corresponding MinMaxScaler object to transform them back to the original data range. Evaluation metrics (such as mean squared error, mean absolute error, etc.) are used to assess the model's performance on the test set to determine if the model meets performance requirements.

[0119] A prediction system for the average voltage, specific energy, and specific capacity of metal-ion battery electrodes based on CNN-LSTM-Attention includes a data preprocessing module, a model building module, a model training module, a model evaluation module, and a model prediction module, with each module performing its corresponding steps.

[0120] Compared with the prior art, the advantages of the present invention are:

[0121] 1. The local feature extraction capability of CNN, the time series analysis capability of LSTM, and the dynamic weight adjustment of the Attention mechanism work together to predict the electrochemical performance of electrode materials, enabling the model to understand the data more deeply and make more accurate predictions.

[0122] Furthermore, the introduction of Bayesian optimization algorithms enables the model to quickly find the optimal solution in a complex hyperparameter space, further improving prediction efficiency.

[0123] 2. This method not only boasts high-precision prediction performance but also broad applicability. It can predict key electrochemical properties of electrode materials, such as average voltage, specific capacity, and specific energy. Furthermore, it can adapt to data from electrode materials of different types and structures, providing researchers with a more flexible and efficient prediction tool. Moreover, because this method combines various advanced deep learning techniques and optimization algorithms, it exhibits high robustness and stability, maintaining consistent prediction performance across different scenarios. Attached Figure Description

[0124] Figure 1The flowchart shows the method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention.

[0125] Figure 2 for Figure 1 The specific operations of the data preprocessing module;

[0126] Figure 3 This is a schematic diagram of a method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention.

[0127] Figure 4 This is a schematic diagram of the CNN-LSTM-Attention network model. Detailed Implementation

[0128] The following will describe in more detail the CNN-LSTM-Attention-based method for predicting metal-ion battery electrode parameters of the present invention with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0129] like Figures 1-4 A method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention includes the following steps:

[0130] Step 1: Data preprocessing.

[0131] Step 1A, Data Reading:

[0132] Use the read_csv() function provided by Pandas to read CSV files from a specified path.

[0133] This function parses the file content into a DataFrame object, where each row represents a sample data and each column corresponds to a battery attribute (including feature columns and target columns, etc.). The read DataFrame object is stored in the variable data for subsequent data operations and processing.

[0134] Step 1B, Encoding Conversion:

[0135] Create an instance of LabelEncoder named label_encoder. This column contains categorical data representing different types of working ions, and is used to convert the categorical data in the working_ion column into numerical data.

[0136] The `label_encoder.fit_transform(data['working_ion'])` statement is used to perform encoding conversion on the `working_ion` column.

[0137] During the fit phase, label_encoder performs statistics and analysis on the different categories in the data['working_ion'] column.

[0138] Based on these categories, label_encoder constructs an encoding mapping rule, that is, each category will be assigned a unique numerical code.

[0139] For example, if the categories are “Li+”, “Na+”, and “K+” in sequence, they are encoded as 0, 1, and 2, respectively.

[0140] During the transform phase, label_encoder applies the previously built encoding mapping rules, replacing each string category in the data['working_ion'] column with the corresponding numeric encoding.

[0141] The converted numerical encoding will retain the category information of the original data, but represent it in numerical form, which facilitates subsequent data processing and model training.

[0142] Add a new column: After completing the encoding conversion, to maintain the correlation between the original data and the encoded data, we add the converted numeric encoding as a new column to the dataframe. The new column is named `working_ion_numeric` to clearly indicate that this is the numeric encoding form of the `working_ion` column.

[0143] Extracting feature variables: Use the data.drop() method, specify axis=1 to indicate that the operation is on the column, and list the column names to be removed (['working_ion','working_ion_numeric','average_voltage','energy_grav','capacity_grav']).

[0144] After the above steps, all remaining columns in data will constitute the feature variable X.

[0145] These columns contain various other relevant battery properties, such as material composition and structural parameters. This information is crucial for subsequent predictive modeling of the target variables (average_voltage, energy_grav, capacity_grav).

[0146] Extracting the target variable: Extracting the average voltage: Accurately select data from the "average_voltage" column of the dataset and assign it to the variable y_voltage. y_voltage represents the average voltage of the battery and is one of the key outputs that the predictive model needs to accurately predict.

[0147] Extracting specific energy: Similarly, we select data from the "energy_grav" column of the dataset and assign it to the variable y_energy. y_energy reflects the specific energy of the battery, that is, the energy that the battery can provide per unit mass, and is an important indicator for evaluating battery energy efficiency.

[0148] Extracting specific capacity: Finally, we select the data from the "capacity_grav" column and assign it to the variable y_capacity. y_capacity represents the battery's specific capacity, which is the amount of charge a battery can store per unit mass, and is a key parameter for measuring battery storage performance.

[0149] Creating a LabelEncoder object: For the string type column currently being processed, we instantiate a new LabelEncoder object, named le. LabelEncoder is a tool commonly used to convert categorical data to numeric data; it achieves this conversion by establishing a mapping from string categories to numerical values.

[0150] Constructing encoding mapping rules: Use the `le.fit_transform(X[col])` statement to perform encoding conversion on the current column. The `fit_transform` method first constructs encoding mapping rules based on the occurrence of different string values ​​in the column, that is, determines the numeric code corresponding to each different string category.

[0151] Replacing the original data: After completing the encoding mapping, the `le.fit_transform` method returns a numeric encoding array of the current column's data after conversion. We directly replace the data in the corresponding column of the original `X` with this numeric encoding array, thus completing the encoding processing for that column.

[0152] Iteratively process all string type columns: Repeat the above steps until all string type columns in the feature dataset X have been converted to numeric form.

[0153] Step 1C, Data Normalization:

[0154] Creating a normalized object: First, we define the target and object of normalization.

[0155] Specifically, we need to normalize the feature data X and the three target variables y_voltage, y_energy and y_capacity respectively.

[0156] To this end, we create four MinMaxScaler objects: scaler_X for normalizing feature data X, scaler_y_voltage for normalizing target variable y_voltage, scaler_y_energy for normalizing target variable y_energy, and scaler_y_capacity for normalizing target variable y_capacity.

[0157] Next, we will describe the normalization process in detail.

[0158] Normalization maps the value of each feature or target variable to the interval [0,1]. Its mathematical formula is: x_scaled = (x - x_min) / (x_max - x_min).

[0159] Where x is the value of the original feature or target variable, x_min and x_max are the minimum and maximum values ​​of the feature or target variable in the entire dataset, respectively, and x_scaled is the normalized value.

[0160] For the feature data X, we use the scaler_X object to perform normalization. The scaler_X calculates the normalized value of each feature according to the formula above, based on the minimum and maximum values ​​of each feature in X, thus obtaining the normalized feature data X_scaled.

[0161] Similarly, for the target variables y_voltage, y_energy, and y_capacity, we perform normalization using scaler_y_voltage, scaler_y_energy, and scaler_y_capacity objects, respectively. These scaler objects calculate the normalized values ​​based on the minimum and maximum values ​​of their respective target variables according to the normalization formula.

[0162] Feature Data Normalization: First, we define the target and object of normalization, namely the feature data X. Feature data X is an n x p matrix, where n represents the number of samples and p represents the feature dimension. Our goal is to map the value of each feature dimension in X to the interval [0,1].

[0163] Target variable normalization: First, we define the target objects for normalization, namely the target variables y_voltage, y_energy, and y_capacity. These target variables may originally exist in the form of a one-dimensional array, but MinMaxScaler requires the input data to be in the form of a two-dimensional array with the shape (n_samples, 1), where n_samples represents the number of samples.

[0164] To meet this requirement, let's take y_voltage as an example:

[0165] First, the array is converted from a one-dimensional array to a two-dimensional array using the `y_voltage.values.reshape(-1,1)` operation. Here, the `.values` attribute is used to retrieve the numerical part of `y_voltage` (if `y_voltage` is a pandas Series object), while the `.reshape(-1,1)` method is used to change the shape of the array, where -1 indicates that the size of this dimension is automatically calculated to ensure that the total number of elements remains unchanged, and 1 specifies the size of the new second dimension.

[0166] Step 1D: Divide the dataset into training, validation, and test sets.

[0167] Using the `train_test_split` function: First, we have the normalized feature data `X_scaled` and the target variables `y_voltage_scaled`, `y_energy_scaled`, and `y_capacity_scaled`. Normalization ensures that all data are scaled uniformly, providing a stable data foundation for subsequent model training.

[0168] Next, to evaluate the model's performance on unseen data, we need to split the dataset into training and test sets. Here, we choose to use the `train_test_split` function to accomplish this step. This function is a commonly used tool in machine learning libraries (such as scikit-learn) for randomly splitting datasets into training and test sets.

[0169] When calling the `train_test_split` function, we pass in the normalized feature data `X_scaled` and all target variables `y_voltage_scaled`, `y_energy_scaled`, and `y_capacity_scaled`. It's important to note that although `train_test_split` is primarily used to split the feature data and its corresponding target variables, here we performed the same split operation on each target variable to maintain data consistency. In practical applications, we might train a model separately for each target variable, thus requiring a split for each target variable.

[0170] In addition, we have specified two key parameters: test_size and random_state.

[0171] `test_size = 0.2` means we allocate 20% of the total data as the test set. This percentage is typically chosen based on experience and practice, aiming to balance the size of the training set (to ensure the model can learn sufficiently) and the size of the test set (to ensure accurate evaluation of model performance). `random_state = 42` is a random seed parameter used to control the randomness of the data partitioning. Setting this parameter ensures that the partitioning results are consistent and repeatable each time the code is run, even if the data is randomly partitioned. This is crucial for comparing results from different experiments, as it eliminates the impact of randomness on data partitioning and model performance evaluation.

[0172] Using the `train_test_split` function: First, we have training data that has been initially split and normalized, including feature data `X_train` and the corresponding target variables `y_voltage_train`, `y_energy_train`, and `y_capacity_train`. This training data provides the necessary information and foundation for subsequent model training.

[0173] However, relying solely on the training set data to train and evaluate a model is far from sufficient. This is because a model's performance on the training set is often overly optimistic and fails to accurately reflect its performance on unseen data. To more accurately assess a model's generalization ability, we need to further partition the training set into a validation set.

[0174] Here, we choose to use the `train_test_split` function to accomplish this step. Although this function is typically used to split the entire dataset into training and test sets, we cleverly utilize it here to further split the training set into a validation set. We pass in the feature data `X_train` from the training set, along with the corresponding target variables `y_voltage_train`, `y_energy_train`, and `y_capacity_train`, and set `test_size = 0.2`. This means we will extract 20% of the data from the training set as the validation set, while the remaining 80% will continue to be used as the training set.

[0175] In addition, we set the parameter `random_state=42`. This is a random seed parameter that ensures that the partitioning results are consistent and repeatable each time the code is run, even if the data is randomly partitioned. This is crucial for comparing results from different experiments, as it eliminates the impact of randomness on data partitioning and model performance evaluation.

[0176] The purpose of separating a validation set from the training set is to periodically evaluate the model's performance during training. This allows us to promptly identify potential problems during training, such as overfitting the training data. Once these problems are identified, we can adjust the model's hyperparameters or optimize the algorithm to improve its generalization ability.

[0177] Step 1E: Adjust the input format:

[0178] For the training set feature data X_train, we first need to define the required three-dimensional input format for the CNN-LSTM-Attention model, namely [batch_size, sequence_length, input_size]. Where:

[0179] Batch_size represents the number of samples fed into the model during each training iteration. It is usually determined by X_train.shape[0], which is the total number of samples contained in the feature data of the training set, i.e., the total number of training set samples (which may be adjusted according to memory and other limitations during batch processing).

[0180] sequence_length represents the length of the time series of the input data.

[0181] input_size represents the number of dimensions of the input features at each time step, which is determined by X_train.shape[1], that is, the number of original feature dimensions contained in a single sample in the training set feature data, i.e., the number of features in the feature data after preprocessing.

[0182] To reshape X_train to the desired 3D format, we use the reshape method and correctly set its parameters. This operation ensures that the format of X_train matches the input requirements of the CNN-LSTM-Attention model.

[0183] For the test set feature data X_test, we use the same processing method as for the training set. This is because the model needs to process data in the same format as during the training phase to ensure the accuracy and reliability of the prediction results.

[0184] Similarly, we use the reshape method to reshape X_test to the [batch_size, sequence_length, input_size] format.

[0185] This step ensures that the format of X_test is consistent with X_train and meets the input requirements of the CNN-LSTM-Attention model. This lays a solid foundation for correctly inputting the test set data into the model for prediction operations.

[0186] Step 2, Model Building.

[0187] Define the CNN-LSTM-Attention network:

[0188] (1) CNN section:

[0189] First, add one or more convolutional layers (nn.Conv1d or nn.Conv2d, depending on the dimension of the input data; if it is one-dimensional time series data, nn.Conv1d can be used).

[0190] in_channels represents the number of input channels, which is usually related to the feature dimensions of the input data;

[0191] out_channels represents the number of output channels, which determines the number of features extracted by the convolutional layer;

[0192] kernel_size represents the size of the convolution kernel, which determines the size of the receptive field of the convolution operation;

[0193] stride represents the stride of the convolution, which determines the step size by which the convolution kernel slides across the data; padding is used to fill the input data to ensure that the output size after the convolution operation meets expectations.

[0194] Convolutional layers can automatically extract local features from data. For local patterns or local correlations that may exist in battery data, convolutional operations can transform this local information into a higher-level feature representation.

[0195] Adding pooling layers (nn.MaxPool1d and nn.AvgPool1d) performs downsampling operations to reduce feature dimensionality. For example, `self.pool = nn.MaxPool1d(pool_size)`, where `pool_size` is the size of the pooling window. Pooling operations help reduce data dimensionality while preserving important local feature information.

[0196] (2) LSTM section:

[0197] The data processed by the CNN will be used as the input to the LSTM.

[0198] input_size is the feature dimension of the input data. This dimension will change after CNN processing and needs to be determined based on the output dimension of the CNN.

[0199] hidden_size represents the dimension of the hidden state vector in the LSTM layer. It has a significant impact on the complexity of the data patterns that the model can capture. From a mathematical point of view, the dimension of the hidden state vector at each time step is hidden_size.

[0200] num_layers represents the number of stacked LSTM layers. Increasing the number of layers can enable the model to learn more complex nonlinear relationships, but it may also lead to problems such as overfitting.

[0201] `batch_first=True` is a setting parameter that specifies the dimension order of the input data. It treats the first dimension of the input data as the batch size, which conforms to common data organization habits. Thus, the dimension order of the input data is [batch_size, sequence_length, input_size].

[0202] LSTM layers can process time-series data or data with long-term dependencies, and can effectively learn and capture time-related information in battery performance data.

[0203] (3) Attention part:

[0204] Applying an attention mechanism to the output of the LSTM layer calculates the weight distribution of the input features, enabling the model to focus more on the features that significantly influence the prediction results. For example, you can use `self.attention = nn.MultiheadAttention(embed_dim, num_heads)`.

[0205] embed_dim represents the embedding dimension of the input features, which is usually related to the output dimension of the LSTM;

[0206] num_heads represents the number of attention heads. Using multiple heads allows you to focus on feature information from different angles, improving the model's expressive power.

[0207] For the input LSTM output, the Attention weights and the weighted summation output are calculated using out,attn_weights = self.attention(query,key,value).

[0208] Here, the query, key, and value are typically set based on the output of the LSTM.

[0209] `out` is the output after Attention weighting.

[0210] attn_weights are the calculated Attention weights, which can be used in some cases to analyze important information that the model focuses on.

[0211] (4) Fully connected layer:

[0212] Finally, a fully connected layer is added to map the output after Attention processing to the final prediction dimensions, namely the three output nodes of average voltage, specific energy, and specific capacity.

[0213] These fully connected layers convert the feature representations output by the Attention layer into specific predicted values. After a linear transformation, three predicted values ​​are obtained: voltage_pred, energy_pred, and capacity_pred. Their dimensions are all [batch_size, 1], meaning that each sample corresponds to one predicted value. Finally, these three predicted values ​​are returned as the model output for subsequent comparison with the true values, loss calculation, and model performance evaluation.

[0214] Forward propagation:

[0215] In the forward method, the input data x is first processed through a CNN layer for convolution and pooling operations.

[0216] The CNN output is then passed to the LSTM layer to obtain the LSTM layer output.

[0217] The LSTM output is then passed to the Attention mechanism to calculate the Attention weights and the final feature representation.

[0218] Finally, the Attention output is passed to the fully connected layer for final prediction: voltage_pred = self.fc_voltage(out), energy_pred = self.fc_energy(out), and capacity_pred = self.fc_capacity(out).

[0219] Step 3: Train the model:

[0220] Step 3A: train_model function - model training and validation evaluation:

[0221] The `train_model` function is defined to train the model and return the average loss on the validation set. This function will be called multiple times during the Bayesian optimization process to evaluate the model's performance under different parameter combinations.

[0222] Step 3B: Define the loss function and optimizer:

[0223] The statement `criterion = nn.MSELoss()` defines the loss function as Mean Squared Error Loss (MSE), and its mathematical formula is:

[0224] Where n is the number of samples and i is the true value. This is the predicted value. The root mean square error is the square root of the mean square error, i.e. It measures the average deviation between predicted and actual values; the smaller the value, the more accurate the model prediction.

[0225] The code `optimizer = torch.optim.Adam(model.parameters(), lr = lr)` creates an Adam optimizer.

[0226] Adam is a commonly used gradient-based optimization algorithm that combines the advantages of adaptive learning rate adjustment, enabling more efficient updates to model parameters. `model.parameters()` retrieves all learnable parameters of the model (such as weight matrices and bias vectors in CNN, LSTM, Attention, and fully connected layers), and `lr` is the learning rate parameter passed in earlier, which controls the step size of each parameter update.

[0227] Step 3C:

[0228] Initialize the number of training epochs: Set num_epochs = 1000, which means that the entire training set will be traversed 1000 times, and each traversal constitutes a training epoch.

[0229] Model training mode settings: At the beginning of each training cycle, switch the model to training mode using the model.train() command to ensure that training-specific layers such as Dropout can function properly.

[0230] Forward propagation: Input the training set feature data (which has been converted into tensor form X_train_tensor) into the model. The model outputs three prediction values: voltage prediction outputs_voltage, specific energy prediction outputs_energy, and specific capacity prediction outputs_capacity. Their dimensions are all [batch_size, 1].

[0231] Loss Calculation: Based on the predefined mean squared error loss function, the loss values ​​for each target variable are calculated separately, namely loss_voltage, loss_energy, and loss_capacity. Then, these three loss values ​​are summed to obtain the total loss, which serves as a performance metric for the model in the current training cycle.

[0232] Step 3D: Backpropagation and Parameter Optimization

[0233] Gradient zeroing: Before backpropagation, the accumulated gradient information in the optimizer is cleared by optimizer.zero_grad() to prevent the accumulation of errors caused by repeated gradient calculations.

[0234] Backpropagation: Execute loss.backward() to automatically calculate the gradient of the loss with respect to the model parameters according to the chain rule. The gradient information is stored in the corresponding parameters.

[0235] Parameter update: The optimizer step() function adjusts the model parameters based on gradient information and the update rules of the optimizer (such as the Adam optimizer) in order to reduce the loss value.

[0236] Step 3E: Model evaluation on the validation set:

[0237] Model evaluation mode settings: After completing the training cycle, switch the model to evaluation mode using model.eval() to ensure that the model is evaluated in a stable state.

[0238] Validation set data preprocessing: Convert the validation set feature data into tensor form X_val_tensor and transfer it to the appropriate device (such as GPU).

[0239] Forward propagation and prediction: Input the validation set data into the model to obtain the predicted values ​​outputs_voltage, outputs_energy, and outputs_capacity.

[0240] Inverse normalization and loss calculation: For each target variable, inverse normalization is performed using the `inverse_transform` method of the corresponding `MinMaxScaler` object to restore it to the original data range. Then, the mean squared error function is used to calculate the loss values ​​`loss_voltage_val`, `loss_energy_val`, and `loss_capacity_val` on the validation set.

[0241] Step 3F: Return the average loss:

[0242] The summation and average of the loss values ​​for each objective variable on the validation set is used as the return value of the function. This average loss value represents the overall performance of the model on the validation set and is a key indicator for parametric search algorithms such as Bayesian optimization to judge the quality of a model.

[0243] Step 4: Bayesian optimization of parameters.

[0244] Step 4A: Define the parameter space:

[0245] The parameter space for Bayesian optimization is defined by the pbounds dictionary, which includes parameters related to CNN and Attention, such as kernel_size (convolutional kernel size) and num_heads (number of Attention heads).

[0246] The search range for the hidden_size parameter can be set to (32, 256), which means that Bayesian optimization will try different hidden layer size values ​​within the range of 32 to 256.

[0247] The search range for num_layers is (1,3), meaning it tries different combinations of 1 to 3 layers.

[0248] For kernel_size, a reasonable search range can be set based on data characteristics and experimental experience, such as (3,7), which means that Bayesian optimization will try different convolution kernel sizes within this range;

[0249] For num_heads, a range of (1, 8) can be set to indicate that different numbers of attention heads are being tried. In this way, Bayesian optimization will search for the optimal combination in a wider hyperparameter space to optimize model performance.

[0250] Step 4B: Create a Bayesian optimization object:

[0251] Create a BayesianOptimization object, optimizer. The f parameter specifies the objective function, which here uses an anonymous function (lambda function) that actually calls the previously defined train_model function. This function receives the training and validation set data, along with the parameters to be optimized, and returns the model's average loss on the validation set as a metric for model performance.

[0252] The pbounds parameter is the parameter space defined earlier, which tells the Bayesian optimization algorithm within which to search for parameters.

[0253] The random_state parameter is set to 42 to ensure that the initial random exploration and other operations are repeatable each time the Bayesian optimization process is run, making it easier to compare the results of different experiments.

[0254] Step 4C: Perform Bayesian optimization iterative search:

[0255] By calling the `optimizer.maximize` method, we perform an iterative search process for Bayesian optimization. During this process, we set two key parameters:

[0256] Setting the `init_points` parameter to 5 indicates that five random explorations will be performed before starting formal iterative optimization. These explorations randomly select five different parameter combinations within the parameter space and call the `train_model` function to evaluate the average loss of the model on the validation set under these parameters. These preliminary exploration results help the algorithm to understand the parameter space and provide a reference for subsequent optimization.

[0257] The `n_iter` parameter is set to 10, specifying the number of iterations for the formal optimization. After the initial 5 random explorations, the Bayesian optimization algorithm uses Bayes' theorem to predict the next parameter combination that might result in a smaller average loss, based on the existing evaluation results (the average loss on the validation set for different parameter combinations). The algorithm then evaluates this new parameter combination and continuously updates its internal probabilistic model. This process is repeated 10 times until a relatively optimal parameter combination is found that minimizes the model's average loss on the validation set.

[0258] Step 4D: Obtain the optimal parameters after optimization.

[0259] After Bayesian optimization is complete, we need to obtain the optimal parameter combination found. By accessing optimizer.max['params'], we can obtain the dictionary form of the optimal parameter combination found during the Bayesian optimization process.

[0260] Then, we can extract the optimal values ​​for hidden layer size, number of layers, learning rate, convolutional kernel size, number of attention heads, etc., and convert them into appropriate data types (such as integer types).

[0261] These optimal parameters are stored in variables such as best_hidden_size, best_num_layers, best_lr, best_kernel_size, and best_num_heads. These parameters will be used to initialize the final model for subsequent full training and testing.

[0262] Step 5, Final Training and Testing:

[0263] The CNN-LSTM-Attention model is reinitialized using the optimal parameter combination obtained through Bayesian optimization. Parameters such as best_hidden_size, best_num_layers, best_lr, best_kernel_size, and best_num_heads are passed to the model's constructor to create the final optimized model.

[0264] The model is trained using the complete training set data, following the previous training process, including setting num_epochs, using the Adam optimizer, calculating loss, backpropagation, parameter updates, etc., to fully train the model and achieve optimal performance.

[0265] The trained model is evaluated on a test set by inputting the test set data into the final model to obtain prediction results. For both the prediction and actual results, a denormalization operation is performed using the corresponding MinMaxScaler object to transform them back to the original data range. Evaluation metrics (such as mean squared error, mean absolute error, etc.) are used to assess the model's performance on the test set to determine if the model meets performance requirements.

[0266] In summary, this embodiment addresses several technical challenges in predicting the key electrochemical performance of metal-ion battery electrodes by proposing a prediction method and system combining deep learning algorithms and feature engineering. The method first downloads a dataset via the MaterialsProject MP API and carefully extracts features from four aspects: stoichiometry, crystal structure, electronic structure, and other battery properties. Two different sets of input features are then constructed to comprehensively capture the complex characteristics of the electrode materials. In the data preprocessing stage, manual encoding, principal component analysis, and min-max normalization are used to clean and reduce the dimensionality of the data. For model construction, a deep learning model is designed, combining the advantages of CNN, LSTM, and attention mechanisms to effectively capture time-series relationships and dynamically adjust feature importance. The optimal hyperparameters are determined using a Bayesian optimization algorithm to improve model performance.

[0267] The method of this invention has the advantages of comprehensive and accurate feature extraction and utilization, effective capture of time series relationships, flexible feature importance adjustment, efficient hyperparameter optimization, wide applicability, and stable and reliable prediction performance.

[0268] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention, characterized in that, Includes the following steps: Step 1: Data preprocessing; Step 2, Model Construction: Combining Convolutional Neural Network (CNN), Long Short-Term Memory Network (LSTM), and Attention mechanism, design a CNN-LSTM-Attention model; Step 3: Train the model: Train the model using the training set data; Step 4: Determine the optimal hyperparameters of the model using the Bayesian optimization algorithm; Step 5, Final Training and Testing: Based on the optimal hyperparameters, the CNN-LSTM-Attention model is reinitialized, and then the model is trained using the training set data to obtain the final model; The test set data is input into the final model to obtain prediction results including average voltage, specific energy, and specific capacity.

2. The method for predicting metal-ion battery electrode parameters based on CNN-LSTM-Attention according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1A: Read data from the database; Step 1B: Convert the encoding to provide a suitable data type for subsequent models: Step 1C: Normalize the target variable: Step 1D: Divide the dataset into training, validation, and test sets: Step 1E: Adjust the input format.

3. The method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention according to claim 1, characterized in that, The CNN-LSTM-Attention model in step 2 consists of a CNN layer, an LSTM layer, and an Attention layer connected in sequence.

4. The method for predicting metal-ion battery electrode parameters based on CNN-LSTM-Attention according to claim 3, characterized in that, The CNN layer is used to extract local features from the input data, the LSTM layer is used to capture long-term dependencies in time series data, and the Attention layer is used to dynamically adjust the importance of features.

5. The method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention according to claim 1, characterized in that, Step 3 also includes evaluating the performance of the CNN-LSTM-Attention model using validation set data and adjusting the model parameters based on the evaluation results.

6. The method for predicting electrode parameters of metal-ion batteries based on CNN-LSTM-Attention according to claim 1, characterized in that, The hyperparameters in step 4 include: the dimension of the hidden state vector in the LSTM layer (hidden_size), the number of stacked LSTM layers (num_layers), the kernel size (kernel_size), the number of attention heads (num_heads), and the learning rate parameter (lr).

Images