Lithium battery life prediction method based on Conformer-MoE model

Abstract

The invention discloses a lithium battery life prediction method based on a Conformer-MoE model, belongs to the technical field of artificial intelligence and battery management, and aims to better capture local and global features of lithium battery data. The method comprises the steps of obtaining lithium battery capacity degradation sequence data and performing preprocessing to obtain processed capacity data; segmenting the time sequence into a plurality of input-output segments using a sliding window; inputting the processed data into a Conformer module, and extracting global and local features and a dependency relationship of the lithium battery capacity degradation sequence data by using a self-attention mechanism and convolution operation; the hybrid expert module dynamically selects the most relevant sub-expert processing characteristics through a gating network, integrates the outputs of a plurality of expert sub-networks, and generates a final capacity prediction value; and training by using a proper loss function, predicting the state of the lithium battery under the current input capacity data, and calculating the remaining service life of the lithium battery based on a predicted value.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and battery management technology, and specifically relates to a method for predicting the lifespan of lithium batteries based on the Conformer-MoE model. Background Technology

[0002] Lithium-ion batteries are widely used in electric vehicles, consumer electronics, defense, aerospace, and other fields due to their advantages such as lightweight design, high energy density, high output voltage, and long cycle life. However, with continuous charge-discharge cycles, the internal resistance of lithium-ion batteries increases, leading to irreversible aging, which in turn causes capacity decay and a shortened lifespan. Therefore, accurately predicting the remaining lifespan of lithium-ion batteries is a crucial step in ensuring equipment safety and improving the reliability of equipment operation.

[0003] The remaining lifespan (RUL) of a lithium-ion battery refers to the number of charge-discharge cycles remaining from its current state until its capacity decays to the retirement threshold, typically 70%-80% of its initial rated capacity. Currently, RUL prediction methods can be mainly divided into two categories: model-based and data-driven. Model-based methods rely primarily on physical models and electrochemical principles, predicting RUL by establishing relevant mathematical models of the battery's internal reaction mechanisms, such as using a second-order RC model combined with FFRLS modeling. However, there is an inherent contradiction between prediction accuracy and model complexity. Data-driven methods, on the other hand, learn from and analyze historical data to establish relevant features to describe the battery degradation process without involving any physical characteristics. Due to this characteristic, data-driven methods are more practical and are receiving increasing attention.

[0004] Deep learning possesses a powerful ability to learn nonlinear representations from raw data. In recent years, the application of deep learning in time series analysis of battery systems has attracted much attention. Models based on recurrent neural networks, such as RNN, LSTM, and GRU, have been widely used in RUL prediction research due to their advantages in processing time series data. However, existing RNN-based models have some key limitations due to their recurrent approach to sequence modeling: limited parallelization capabilities, difficulty in capturing long-term dependencies, and a lack of attention mechanisms. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a lithium battery lifetime prediction method based on the Conformer-MoE model, which can adapt to the characteristics of different batteries and better capture the local and global features of lithium battery data.

[0006] The lithium battery lifetime prediction method based on the Conformer-MoE model includes the following steps:

[0007] Step 1: Obtain lithium battery capacity degradation sequence data and perform preprocessing to obtain processed capacity data;

[0008] Step 2: Use a sliding window to divide the time series into multiple input-output segments;

[0009] Step 3: Input the processed data into the Conformer module, and use the self-attention mechanism and convolution operation to extract the global and local features and dependencies of the lithium battery capacity degradation sequence data;

[0010] Step 4: The hybrid expert module dynamically selects the most relevant sub-experts to process features through a gating network, integrates the outputs of multiple expert sub-networks, and generates the final capacity prediction value.

[0011] Step 5: Use a suitable loss function to train the system, predict the state of the lithium battery under the current input capacity data, and calculate the remaining lifespan of the lithium battery based on the predicted value.

[0012] Furthermore, in step 1, the capacity data is normalized by mapping to the (0,1] interval, and the normalized data is then used to randomly delete noise points in the sequence using a random deactivation mask to reduce the impact of noise.

[0013] Furthermore, in step 2, a sliding window of size m is used to capture local patterns and dependencies in the time series, dividing the time series into smaller segments. The sliding window formula is as follows:

[0014]

[0015] Where, x t and y t Let t represent the network input and output, respectively, where t∈[1,2,...,nm].

[0016] Furthermore, step 3 specifically involves the following steps:

[0017] Step 3-1: The processed lithium battery capacity degradation sequence data is passed through a multi-head self-attention module. Relative position encoding is used to calculate the attention weight through linear changes in query, key, and value. Then, relying on the multi-head mechanism, different features are extracted from multiple subspaces to capture long-term dependencies in different charge-discharge cycle modes. The multi-head attention mechanism calculation formula is as follows:

[0018] MultiHead(Q,K,V)=Concat[head1,...,head h W O ,

[0019] head i =Attention(QWi Q ,KW i K VW i V ),

[0020] Where Q, K, and W are the query vector, key vector, and value vector, respectively;

[0021] Step 3-2: Feed the relationship obtained in the previous step into the convolution module. Use one-dimensional convolution to capture local patterns and short-term dependencies in the time series. The convolution operation slides in the time dimension to extract features within a short time window. Combine it with an activation function to enhance nonlinearity and stability, thereby capturing detailed changes in a single charge-discharge cycle.

[0022] Step 3-3: The modules in the above two steps are stacked alternately to form a Conformer Block. The data is processed through multiple layers of Blocks to simultaneously focus on the global and local features of the battery capacity degradation sequence data, forming a multi-scale feature representation.

[0023] Furthermore, step 4 employs a fully connected layer as the expert network, and a gated network composed of the fully connected layer and a softmax function dynamically selects the most relevant sub-experts based on the input features. Weights are assigned to each expert according to the input features, activating only a subset of experts to participate in the computation, and their outputs are integrated accordingly to generate a more accurate and comprehensive prediction result, thus obtaining a superior representation. The calculation formula for the expert network is as follows:

[0024]

[0025] Where m represents the number of experts, G(x) is the gated network, and E(x) is the sub-expert network.

[0026] To ensure sparsity and balance, the expert network model employs a noisy Top-K gating mechanism:

[0027] G(x)=Softmax(KeepTopK(H(x),k)),

[0028] H(x) i =(x·W) g ) i +StandardNormal()·softplus((x·W noise ) i ),

[0029]

[0030] Where H(x) represents the weights assigned to each sub-expert network for sample x, k is the maximum number of weights to be selected, and W...g and W noise These are the trainable weight parameters, and StandardNormal() represents the standard normal distribution.

[0031] Furthermore, in step 5, since the predicted unknown capacity value is a continuous numerical value, the mean squared error is used as the loss function for calculation. The average of the squared differences between the predicted value and the true value obtained by mapping the features extracted by the hybrid expert model to the capacity value is used to quantify the model error. The formula for calculating the mean squared error loss function is defined as follows:

[0032]

[0033] Where m is the number of predicted samples, y t For the true value, The predicted value obtained by the model is λ, which represents the regularization strength, Ω(·) refers to regularization, and θ is the model weight parameter. This loss function amplifies the penalty for larger errors through the squared term, while controlling the model complexity through the regularization term, effectively preventing overfitting.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] The lithium battery life prediction method proposed in this invention exhibits superior trend prediction capabilities under complex scenarios and diverse pattern characteristics. By utilizing the network's balanced global and local feature extraction capabilities, the network can take into account both local details and global dependencies, thereby improving the accuracy of lithium battery life prediction. Furthermore, the highly parallel computing using a self-attention mechanism significantly improves the network's training efficiency. Attached Figure Description

[0036] Figure 1 This is a flowchart of the prediction method of the present invention;

[0037] Figure 2 This is a block diagram of the Conformer model of the present invention;

[0038] Figure 3 This is a structural diagram of the hybrid expert model of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] like Figure 1 As shown, a lithium battery lifetime prediction method based on the Conformer-MoE model specifically includes the following steps:

[0041] Step 1: Obtain lithium battery capacity degradation sequence data and perform preprocessing to obtain processed capacity data. The capacity data is normalized by mapping to the (0,1] interval, and the normalized data is then used to randomly remove noise points in the sequence using a random deactivation mask to reduce the impact of noise.

[0042] Specifically, lithium battery degradation data covering different operating conditions is acquired. To mitigate the impact of varying input data distribution on the network, the data is normalized. The capacity input sequence is c = [c1, c2, ..., c...]. n ], through mapping x c =c / C0 to the interval (0,1] to achieve normalization, select a small number of data as test samples, and use the rest of the batteries as training set.

[0043] Step 2: Use a sliding window to divide the time series into multiple input-output segments.

[0044] A sliding window of size m is used to capture local patterns and dependencies in a time series, dividing the time series into smaller segments. The sliding window formula is as follows:

[0045]

[0046] Where, x t and y t Let t represent the network input and output, respectively, where t∈[1,2,...,nm].

[0047] Training data, being raw data collected from sensors, is always filled with noise, especially during the charge-discharge-regeneration phase. This severely impacts model performance and predictive accuracy. To ensure stability and robustness, a random deactivation mask is used to denoise the normalized data before inputting it into the deep neural network. Noise is reduced by randomly removing noisy points from the sequence. Then, a sliding window of size m is used to capture local patterns and dependencies in the time series x, dividing the time series into smaller segments.

[0048] Step 3: Input the processed data into the Conformer module, and use the self-attention mechanism and convolution operation to extract the global and local features and dependencies of the lithium battery capacity degradation sequence data.

[0049] A Conformer model is constructed to extract global and local features and dependencies from lithium battery capacity degradation sequence data through self-attention mechanisms and convolution operations. This is achieved through the following sub-steps:

[0050] (1) The processed lithium battery capacity degradation sequence data is passed through a multi-head self-attention module, using relative position encoding. The query, key, and value are calculated through linear projection, and then attention weights are calculated using dot product and normalized using softmax. Relying on the multi-head mechanism, Q, K, and V are projected to multiple subspaces to extract different features, thereby capturing long-term dependencies in different charge-discharge cycle modes. The multi-head attention mechanism calculation formula is as follows:

[0051] MultiHead(Q,K,V)=Concat[head1,...,head h W O ,

[0052] head i =Attention(QW i Q ,KW i K VW i V ),

[0053] Where Q, K, and W are the query vector, key vector, and value vector, respectively.

[0054] (2) The relationship obtained in the previous step is fed into the convolution module. The convolution module consists of two 1D convolution layers. The first layer expands the input feature dimension to a higher dimension, and the second layer compresses the expanded features back to the original dimension. The local patterns and short-term dependencies in the time series are captured by one-dimensional convolution. The convolution operation slides in the time dimension to extract local trends (i.e., features within a short time window). The ReLU activation function is used to enhance nonlinearity and stability, thereby capturing the detailed changes in a single charge-discharge cycle.

[0055] (3) The modules in the above two steps are stacked alternately to form a ConformerBlock. The data simultaneously focuses on the global and local features of the battery capacity degradation sequence data through multiple layers of Blocks, forming a multi-scale feature representation.

[0056] Step 4: The hybrid expert module dynamically selects the most relevant sub-experts to process features through a gating network, integrates the outputs of multiple expert sub-networks, and generates the final capacity prediction value.

[0057] like Figure 3As shown, a fully connected layer is used as the expert network, and a gating network composed of the fully connected layer and the softmax function dynamically selects the most relevant sub-experts based on the input features. The input representation first enters different sub-experts for gating calculation, and then is processed by softmax to obtain the weights assigned to each sub-expert. To ensure sparsity and balance, the KeepTopK discrete function is used to dynamically select the top k most relevant sub-experts, and the remaining values ​​are forced to infinity. Only the top k experts with the largest weights are activated to participate in the calculation, and their outputs are integrated accordingly to generate more accurate and comprehensive prediction results and obtain better representations. The calculation formula of the expert network is as follows:

[0058]

[0059] Where m represents the number of experts, G(x) is the gated network, and E(x) is the sub-expert network.

[0060] To ensure sparsity and balance, the expert network model employs a noisy Top-K gating mechanism as follows:

[0061] G(x)=Softmax(KeepTopK(H(x),k)),

[0062] H(x) i =(x·W) g ) i +StandardNormal()+softplus((x·W noise ) i ),

[0063]

[0064] Where H(x) represents the weights assigned to each sub-expert network for sample x, k is the maximum number of weights to be selected, and W... g and W noise These are the trainable weight parameters, and StandardNormal() represents the standard normal distribution.

[0065] Step 5: Use a suitable loss function to train the system, predict the state of the lithium battery under the current input capacity data, and determine the remaining number of cycles or time required for the current state to reach the preset threshold in order to calculate the remaining lifespan of the lithium battery.

[0066] Because the predicted unknown capacity values ​​are continuous numerical values, the mean squared error is used as the loss function. The average of the squared differences between the predicted and true values ​​obtained by mapping the features extracted by the hybrid expert model to the capacity values ​​is used to quantify the model error. The formula for calculating the mean squared error loss function is defined as follows:

[0067]

[0068] Where m is the number of predicted samples, y t For the true value, The predicted value obtained by the model is λ, which represents the regularization strength, Ω(·) refers to regularization, and θ is the model weight parameter. This loss function amplifies the penalty for larger errors through the squared term, while controlling the model complexity through the regularization term, effectively preventing overfitting.

[0069] The error between the true value and the predicted label value is backpropagated to update the model weights, and finally the trained life prediction model is obtained to make a judgment on the life prediction of lithium battery.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the lifetime of lithium batteries based on the Conformer-MoE model, characterized in that, Includes the following steps: Step 1: Obtain lithium battery capacity degradation sequence data and perform preprocessing to obtain processed capacity data; Step 2: Use a sliding window to divide the time series into multiple input-output segments; Step 3: Input the processed data into the Conformer module, and use the self-attention mechanism and convolution operation to extract the global and local features and dependencies of the lithium battery capacity degradation sequence data; Step 4: The hybrid expert module dynamically selects the most relevant sub-experts to process features through a gating network, integrates the outputs of multiple expert sub-networks, and generates the final capacity prediction value. Step 5: Use a suitable loss function to train the system, predict the state of the lithium battery under the current input capacity data, and calculate the remaining lifespan of the lithium battery based on the predicted value.

2. The lithium battery lifetime prediction method based on the Conformer-MoE model according to claim 1, characterized in that, In step 1, the capacity data is normalized by mapping to the (0,1] interval, and the normalized data is then used to randomly delete noise points in the sequence using a random deactivation mask to reduce the impact of noise.

3. The lithium battery lifetime prediction method based on the Conformer-MoE model according to claim 1, characterized in that, In step 2, a sliding window of size m is used to capture local patterns and dependencies in the time series, dividing the time series into smaller segments. The sliding window formula is as follows: Where, x t and y t Let t represent the network input and output, respectively, where t∈[1,2,...,nm].

4. The lithium battery lifetime prediction method based on the Conformer-MoE model according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step 3-1: The processed lithium battery capacity degradation sequence data is passed through a multi-head self-attention module. Relative position encoding is used to calculate the attention weight through linear changes in query, key, and value. Then, relying on the multi-head mechanism, different features are extracted from multiple subspaces to capture long-term dependencies in different charge-discharge cycle modes. The multi-head attention mechanism calculation formula is as follows: MultiHead(Q,K,V)=Concat[head1,...,head h ]W O , head i =Attention(QW i Q ,KW i K ,VW i V ), Where Q, K, and W are the query vector, key vector, and value vector, respectively; Step 3-2: Feed the relationship obtained in the previous step into the convolution module. Use one-dimensional convolution to capture local patterns and short-term dependencies in the time series. The convolution operation slides in the time dimension to extract features within a short time window. Combine it with an activation function to enhance nonlinearity and stability, thereby capturing detailed changes in a single charge-discharge cycle. Step 3-3: The modules in the previous two steps are stacked alternately to form a ConformerBlock. The data is processed through multiple layers of Blocks to simultaneously focus on the global and local features of the battery capacity degradation sequence data, forming a multi-scale feature representation.

5. The lithium battery lifetime prediction method based on the Conformer-MoE model according to claim 1, characterized in that, Step S4 employs a fully connected layer as the expert network. A gated network, composed of the fully connected layer and a softmax function, dynamically selects the most relevant sub-experts based on the input features. Weights are assigned to each expert according to the input features, activating only a subset of experts to participate in the computation. The outputs are then integrated to generate more accurate and comprehensive prediction results, achieving a superior representation. The calculation formula for the expert network is as follows: Where m represents the number of experts, G(x) is the gated network, and E(x) is the sub-expert network; The expert network model employs a noisy Top-K gating mechanism: G(x)=Softmax(KeepTopK(H(x),k)), H(x) i =(x·W g ) i +StandardNormal()·softplus((x·W noise ) i ), Where H(x) represents the weights assigned to each sub-expert network for sample x, k is the maximum number of weights to be selected, and W... g and W noise These are the trainable weight parameters, and StandardNormal() represents the standard normal distribution.

6. The lithium battery lifetime prediction method based on the Conformer-MoE model according to claim 1, characterized in that, In step S5, the mean squared error is used as the loss function for calculation. The average of the squared differences between the predicted value and the true value obtained by mapping the features extracted by the hybrid expert model to the capacity value is used to quantify the model error. The formula for calculating the mean squared error loss function is defined as follows: Where m is the number of predicted samples, y t For the true value, The predicted value obtained by the model is λ, which represents the regularization strength, Ω(·) refers to regularization, and θ is the model weight parameter. This loss function amplifies the penalty for larger errors through the squared term, while controlling the model complexity through the regularization term, effectively preventing overfitting.

Images