A lithium-ion battery life prediction method based on an integrated Transformer model

Abstract

The application provides a lithium-ion battery life prediction method based on an integrated Transformer model, comprising the following steps: obtaining domain knowledge of a lithium-ion battery and obtaining manual features related to battery life; inputting original measurement data into a preset convolutional neural network module E1 and a preset Transformer module E2 respectively to obtain feature vectors and encoding vectors respectively; connecting the manual features, the feature vectors and the encoding vectors to obtain training data; training a battery life prediction model using the training data, adjusting a learning rate using a cosine annealing strategy during the training process, saving model parameters at multiple different training stages to a model list, performing secondary training on the battery life prediction model by averaging the SWA parameters of the model, and obtaining a target battery life prediction model; and processing the prediction confidence interval boundary corresponding to a specified prediction starting point by quantile regression loss to obtain a lithium-ion battery life prediction result, thereby improving the accuracy of the lithium-ion battery RUL.

Description

Technical Field

[0001] This article belongs to the field of lithium-ion battery technology, specifically involving a lithium-ion battery lifetime prediction method based on an integrated Transformer model. Background Technology

[0002] Lithium-ion batteries are widely used in electric vehicles and electronic devices due to their long lifespan, low cost, and high energy density. However, lithium-ion batteries are systems with a complex aging process; their capacity decline and power degradation are not caused by a single factor, but by a combination of different processes and their interactions. During storage, aging effects such as self-discharge and increased impedance can affect the battery's calendar life. Cycle life is affected by aging effects that may occur during use, such as mechanical degradation and lithium metal plating. Therefore, there is an urgent need to develop high-performance battery management systems to accurately monitor the health status of lithium-ion batteries and provide timely status alerts, thereby minimizing potential hazards.

[0003] In existing technologies, the remaining useful life (RUL) of lithium-ion batteries is mainly predicted using data from the early cycling stages. This has significant practical implications for optimizing battery design, predicting battery life in advance, improving battery safety, and accelerating the verification of new technologies, thus providing a positive impetus for the development and application of battery technology. However, the degradation characteristic information contained in early cycling data is weak, making it a pressing problem to be solved on how to accurately predict the remaining useful life of lithium-ion batteries using this data. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this paper aims to provide a lithium-ion battery life prediction method based on an integrated Transformer model, which can improve the accuracy of predicting the remaining lifespan of lithium-ion batteries.

[0005] To solve the above-mentioned technical problems, the specific technical solution presented in this paper is as follows:

[0006] On the one hand, this paper provides a method for predicting the lifetime of lithium-ion batteries based on an integrated Transformer model, the method comprising:

[0007] Acquire domain knowledge of lithium-ion batteries, and based on this domain knowledge, obtain a set of features related to battery life through correlation analysis;

[0008] The feature set is mapped through three fully connected layers to obtain handcrafted features related to battery life.

[0009] The raw measurement data of the lithium-ion battery is obtained and input into the preset convolutional neural network module E1 and the preset Transformer module E2 respectively to obtain the feature vector and encoding vector of the raw measurement data respectively.

[0010] The handcrafted features, the feature vectors, and the encoding vectors are concatenated to obtain training data;

[0011] The battery life prediction model is trained using the training data. During the training process, the learning rate is adjusted using a cosine annealing strategy, and the model parameters of multiple different training stages are saved to the model list. The average SWA parameter of the models in the model list is calculated, and the battery life prediction model is trained a second time using the average SWA parameter to obtain the target battery life prediction model.

[0012] Based on the target battery life prediction model, the boundary of the prediction confidence interval corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result.

[0013] Furthermore, based on the domain knowledge of the lithium-ion battery, a set of features related to battery life is obtained through correlation analysis, including:

[0014] Based on the domain knowledge of the lithium-ion battery, multiple feature information is extracted;

[0015] The Spearman correlation coefficient between any two feature information is calculated using the following formula: Among them, R(X) i ),R(Y i ) represent the levels of each data point in feature information X and Y, respectively, and n is the number of samples;

[0016] Two features whose Spearman correlation coefficient exceeds a threshold are added to the candidate feature set, where the candidate feature set consists of X. domain1 express;

[0017] Recursive feature elimination based on cross-validation is used to select the optimal number of features from the candidate feature set as the feature set related to battery life, wherein the feature set is determined by X. domain2 It is indicated that its dimension is n×p, where n represents the number of battery samples and p represents the number of features in the feature set.

[0018] Furthermore, the preset convolutional neural network module includes convolutional layers, pooling layers, and fully connected layers;

[0019] The convolutional layer is used to extract features from the original policy data and generate a feature map;

[0020] The pooling layer is used to reduce the dimensionality of the feature maps extracted by the convolutional layer;

[0021] The fully connected layer is used to concatenate the pooled feature maps to obtain the feature vector of the original measurement data, denoted as: H r1 =E1(X raw ), H r1 E1 is the feature vector; E1 is the preset convolutional neural network module, X raw The initial characteristic vector of the original measurement data has dimensions n×l×m, where n represents the number of battery samples, l represents the total number of original feature statistics, and m represents the number of cycles.

[0022] Furthermore, the preset Transformer module E2 includes a denoising autoencoder, a Transformer layer, and a position encoder;

[0023] After obtaining the raw measurement data of the lithium-ion battery, the process further includes: adding Gaussian noise to the normalized input of the raw measurement data to obtain the corrupted initial feature vector.

[0024] The features learned by the denoising autoencoder from the input of superimposed noise are more robust, and the output of the DAE encoder is represented as: in Let b and b represent the output, activation function, weights, and bias of the DAE encoder, respectively. Then, to reconstruct the input vector, the latent representation is mapped back to the input space. The reconstructed input vector is represented as: Where W', b' and f(.) represent the weights, biases and mapping functions of the DAE decoder, respectively;

[0025] The position encoder is used to determine the relative or absolute position of input data within a sequence. Position encoding is represented by sine and cosine functions of different frequencies, and the calculation formula is as follows:

[0026] PE(p,2k)=sin(t / 10000 2k / d );

[0027] PE(p,2k+1)=cos(t / 10000 2k / d );

[0028] Where p represents the position of the feature in the time series data, d represents the dimension of the feature, 2k represents an even dimension, and 2k+1 represents an odd dimension;

[0029] The Transformer layer is used to process the output of the denoising autoencoder to learn the lifespan decay dependency features from the battery operation records, as shown below:

[0030] Furthermore, the Transformer layer includes an encoder and a decoder; the encoder receives the input sequence, maps it into a high-dimensional vector, and feeds it into the decoder to produce an output sequence. This paper uses only the Transformer encoder to learn the long-term dependence of life decay from battery operation records.

[0031] The encoder includes a multi-head self-attention mechanism and a feedforward network;

[0032] The multi-head self-attention mechanism is used to capture dependencies between features and ignores their distance in the sequence, as follows:

[0033] MultiHead(H l-1 )=[head1;head2;...;head h W O ;

[0034] Among them, head i =Attention(H l-1 W i Q H l-1 W i K H l-1 W i V );

[0035]

[0036] Among them, H l-1 W represents the representation layer of (l-1)-th. i Q W i K W i V The mapping weights are represented as i∈[1,h], where h represents the number of heads in the multi-head attention mechanism, Q, K, and V represent the query, key, and value, respectively, and W represents the value. O These are training weights;

[0037] Feedforward networks have two different mapping methods: linear and ReLU nonlinear. The feedforward network FFN(x) and the output H of the l-th layer of the feedforward network... l It can be represented as:

[0038] H l =FFN(MultiHead(H l-1 ));

[0039] FFN(x) = ReLU(xW1+b1)W2+b2;

[0040] Where ReLU(.) represents the mapping function, W1 and W2 represent the weights, and b1 and b2 represent the biases;

[0041] Furthermore, the battery life prediction model is trained using the training data. During the training process, a cosine annealing strategy is used to adjust the learning rate, and model parameters from multiple different training stages are saved to the model list, including:

[0042] Based on the training data, the prediction results of the battery life prediction model are calculated through forward propagation, the loss function is calculated and backpropagation is performed to update the model parameters;

[0043] The current learning rate is calculated based on the current training round number and annealing cycle to adjust the model parameters for the next round of model training;

[0044] Save the parameters of the current model to the model list.

[0045] Further, the SWA parameters of the models in the calculation model list are averaged, and the battery life prediction model is trained a second time using the averaged SWA parameters to obtain the target battery life prediction model, including:

[0046] Iterate through each model parameter in the model list and calculate the average SWA parameter value;

[0047] Set the start_swa_epoch parameter;

[0048] The starting position of the second training is determined based on the start_swa_epoch parameter.

[0049] Based on the starting position of the secondary training, the training process before start_swa_epoch is performed using ordinary model parameters, while the training afterward uses the average value of SWA parameters, until a converged target battery life prediction model is obtained.

[0050] Furthermore, based on the target battery life prediction model, the boundary of the prediction confidence interval corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result, including:

[0051] The quantile regression loss function based on the target battery life prediction model is determined as follows: Among them, y i To represent the actual remaining useful life, Let represent the remaining battery life predicted by the target battery life prediction model at a fractional level q, where q∈[0,1], and (.). + =max(0,.) means that only the positive parts are kept, and the negative parts are set to zero;

[0052] Determine multiple quantile levels Q = {0.1, 0.5, 0.9};

[0053] Based on the multiple quantile levels and the quantile regression loss function, the final objective function is obtained by using the quantile loss of all training data, expressed as: Where α represents the weighting coefficient for balancing the importance of different modules, S represents the sample size, M represents the interquartile range, and L... D This represents the reconstruction loss of the DAE module, where This is considered as a prediction of the model's remaining useful life. It can be seen as the boundary of the 80% confidence interval for the model's remaining useful life prediction;

[0054] The objective function was optimized using the AdamW optimizer to obtain the lithium-ion battery lifetime prediction results.

[0055] On the other hand, this document also provides a battery management system, the system including a lithium battery and a controller, the controller including the method described above.

[0056] Finally, this article also provides an electrical device, which includes the battery management system described above.

[0057] Using the above technical solution, this paper describes a lithium-ion battery life prediction method based on an integrated Transformer model. The method includes: acquiring domain knowledge of lithium-ion batteries; obtaining a feature set related to battery life through correlation analysis based on the domain knowledge; mapping the feature set through three fully connected layers to obtain handcrafted features related to battery life; acquiring raw measurement data of the lithium-ion battery and inputting the raw measurement data into a preset convolutional neural network module E1 and a preset Transformer module E2, respectively, to obtain feature vectors and encoding vectors of the raw measurement data; and concatenating the handcrafted features, the feature vectors, and the encoding vectors to obtain a training vector. The method involves training a battery life prediction model using the training data. During training, a cosine annealing strategy is used to adjust the learning rate, and model parameters from multiple training stages are saved to a model list. The average SWA parameters of the models in the list are calculated, and the battery life prediction model is then trained a second time using this average SWA parameter to obtain the target battery life prediction model. Based on the target battery life prediction model, quantile regression loss is used to process the prediction confidence interval boundary corresponding to the specified prediction starting point to obtain the lithium-ion battery life prediction result. The method presented in this paper can simultaneously consider features based on domain knowledge and hidden features in the original data. DAE processing is performed on the original data to achieve denoising of the input data and learning of nonlinear representations. Secondly, a novel ensemble method is proposed. This model can adaptively adjust the learning rate, increase model diversity, reduce the risk of overfitting, improve generalization ability, and provide uncertainty estimation. Finally, quantile regression is used to quantify the uncertainty of the prediction result, providing RUL point estimates and range estimates, and giving the confidence interval of the range, thereby improving the accuracy of lithium-ion battery remaining life prediction.

[0058] To make the above and other objects, features and advantages of this document more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments or prior art described herein, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this article. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 The illustration shows a step diagram of a lithium-ion battery lifetime prediction method based on an integrated Transformer model provided in the embodiments of this paper;

[0061] Figure 2 A schematic diagram of the network framework of the method provided in the embodiments of this article is shown;

[0062] Figure 3 A schematic diagram of the network framework of the Transformer module in the embodiments of this paper is shown;

[0063] Figure 4 A schematic diagram of the integrated training process in the embodiments of this paper is shown;

[0064] Figure 5 This document illustrates a schematic diagram of a lithium-ion battery lifetime prediction device based on an integrated Transformer model, as provided in an embodiment of this paper.

[0065] Figure 6 A schematic diagram of the electrical equipment provided in the embodiments of this article is shown.

[0066] Explanation of symbols in the attached drawings:

[0067] 510. Feature set acquisition module; 520. Manual feature acquisition module; 530. Encoding vector acquisition module; 540. Training data acquisition module; 550. Training module; 560. Prediction module. Detailed Implementation

[0068] The technical solutions in the embodiments described below will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments described herein, and not all of the embodiments. Based on the embodiments described herein, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this document.

[0069] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0070] Before proceeding with the embodiments, the prior art presented herein will be described:

[0071] The Transformer is an important neural network architecture that has revolutionized natural language processing and is widely used in other fields. Compared to traditional sequence models (such as recurrent neural networks and long short-term memory networks), the Transformer solves several problems when processing long sequence data, such as the difficulty in capturing long-term dependencies and limited parallel computing capabilities. Its design goal is to achieve breakthrough performance and make significant progress in sequence data processing. Traditional sequence models compute element-by-element when processing the input sequence, resulting in low computational efficiency. The self-attention mechanism, by considering the relationships between all elements of the input sequence simultaneously, achieves more efficient parallel computing, thus significantly improving model performance. The Transformer model mainly consists of an encoder and a decoder, with the self-attention mechanism being its key component. The encoder is responsible for converting the input sequence into a series of feature representations, while the decoder is used to generate the target sequence.

[0072] The main idea behind ensemble models is to achieve more accurate, stable, and better generalization performance by combining the predictions of multiple models. In the ensemble process, diverse models are typically used, which can be constructed using different algorithms, parameter settings, or feature subsets. In this way, the member models of the ensemble model can complement each other, thereby improving overall performance. Common ensemble model methods include voting, averaging, and stacking. These methods have achieved widespread success in machine learning competitions and practical applications. Properly selecting and combining multiple models can significantly improve prediction accuracy and robustness while reducing the risk of overfitting. However, building and tuning ensemble models requires certain experience and skill, and over-ensemble can also lead to overfitting problems; therefore, careful consideration and experimentation are necessary.

[0073] Methods for predicting the cycle life of lithium-ion batteries can be categorized into model-based and data-driven approaches. Model-based methods include empirical models, equivalent circuit models, and electrochemical models. Some researchers have proposed different construction schemes, such as semi-empirical models based on the Coulomb efficiency method and particle filtering algorithm to improve the prediction accuracy of remaining battery life; constructing a two-stage Wiener process model to capture the characteristics of lithium-ion batteries at different aging stages and combining it with an unscented particle filtering algorithm for remaining life prediction; and an ensemble model that integrates empirical exponents and multinomial regression models to describe the capacity degradation of lithium-ion batteries and predict their remaining usable performance, with model parameters adjusted online using particle filtering. Although model-based methods have achieved success in battery life prediction, battery degradation models significantly impact prediction performance, and accurate early prediction becomes difficult because signs of capacity degradation are not obvious in early cycles. In contrast, data-driven methods input relevant extracted battery health features into a black-box model developed based on machine learning or deep learning methods, outputting a corresponding battery life prediction. Many scholars have also proposed different construction ideas, such as a hybrid model that combines a wide learning system with a relevance vector machine to improve the accuracy and long-term predictive ability of the battery life prediction model; an online method using feedforward neural networks and importance sampling, suitable for accurate prediction of the remaining battery life in actual operation; a dual-task network structure based on bidirectional gated recursive units and multiple experts to improve the traditional prediction and health management framework, which can simultaneously assess and predict the remaining service life of industrial equipment; and a two-stage model that combines convolutional neural networks and Gaussian process regression, which is applied in a personalized manner to the early remaining life prediction of each tested lithium-ion battery, thereby improving the prediction accuracy of the model.

[0074] To address the aforementioned issues, this paper presents a lithium-ion battery lifetime prediction method based on an integrated Transformer model, which can improve the accuracy of lithium-ion battery remaining lifetime prediction. Figure 1 This document illustrates the steps of a lithium-ion battery lifetime prediction method based on an integrated Transformer model, as provided in the embodiments. While this specification provides the operational steps described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive methods. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible order. In actual system or device products, the methods shown in the embodiments or accompanying drawings can be executed sequentially or in parallel. Specifically, as shown in the embodiments or accompanying drawings... Figure 1 As shown, the method may include:

[0075] S101: Obtain domain knowledge of lithium-ion batteries, and based on the domain knowledge of lithium-ion batteries, obtain a set of features related to battery life through correlation analysis;

[0076] S102: The feature set is mapped through a three-layer fully connected layer to obtain handcrafted features related to battery life;

[0077] S103: Obtain the raw measurement data of the lithium-ion battery, and input the raw measurement data into the preset convolutional neural network module E1 and the preset Transformer module E2 respectively to obtain the feature vector and encoding vector of the raw measurement data respectively;

[0078] S104: The handcrafted features, the feature vector, and the encoding vector are concatenated to obtain training data;

[0079] S105: The battery life prediction model is trained using the training data. During the training process, the learning rate is adjusted using a cosine annealing strategy, and the model parameters of multiple different training stages are saved to the model list. The average SWA parameter of the models in the model list is calculated, and the battery life prediction model is trained a second time using the average SWA parameter to obtain the target battery life prediction model.

[0080] S106: Based on the target battery life prediction model, the boundary of the prediction confidence interval corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result.

[0081] This can be understood as, for example Figure 2The diagram shows the network framework of the method provided in the embodiments of this specification, which mainly includes three parts: feature extraction, ensemble training, and interval prediction. First, manual and automatic feature extraction are performed based on domain-knowledge-based features and raw measurement data of lithium-ion batteries. The domain-knowledge-based features are filtered using recursive feature elimination and Spearman correlation coefficient to select features with high correlation coefficients to battery life. Then, a three-layer fully connected layer is used for feature mapping to obtain the final manual features. The raw data, after min-max normalization, is simultaneously fed into a three-layer one-dimensional convolutional neural network layer and a four-layer Transformer layer with a denoising autoencoder. The features processed by the two structures are smoothly connected to obtain the automatically extracted features. The filtered manual features and automatically extracted features are merged into the final model input features, increasing the completeness of the feature extractor. Second, a novel CosineSWA ensemble training method is proposed to perform two-stage training on the model. In the initial training process, a cosine annealing strategy is used to adjust the learning rate, and multiple models from different training stages are saved. Before the second training, the average SWA parameters of these models are calculated, and the model is trained again using the average SWA parameters. Finally, quantile regression is used to quantify the uncertainty of the RUL prediction of the proposed method, thereby improving the accuracy and reliability of the remaining life prediction of lithium-ion batteries.

[0082] The embodiments in this specification aim to solve the problem of RUL range prediction in the early cycling scenario of lithium-ion battery capacity degradation, that is, how to perform accurate range prediction when the capacity degradation of lithium-ion batteries is not yet obvious. The definition of the real RUL problem is shown in equation (1):

[0083] RUL real =Cycle Failure -Cycle n (1)

[0084] Cycle Failure and Cycle n These represent the number of cycles corresponding to the lithium-ion battery's degradation to a performance failure state and the number of cycles corresponding to the prediction starting point, respectively. The model proposed in this invention uses quantile regression to quantify the uncertainty of the prediction results, providing point and range estimates of the RUL, and giving the confidence interval for that range.

[0085] The CDT-CASWA method for the RUL prediction problem can be expressed as shown in Equation (2):

[0086]

[0087] in X represents the constructed prediction model. nθ represents the input features corresponding to the prediction starting point n (n takes the values ​​100, 90, 80, 70, etc.), θ represents other relevant parameters of the model, and q represents the quantile value, which takes the value [0.1, 0.5, 0.9] in this invention.

[0088] The final optimization problem of the model can be expressed as shown in formula (3):

[0089]

[0090] Where α represents the weighting coefficient for balancing the importance of different modules, S represents the sample size, M represents the interquartile range, and L... D This represents the reconstruction loss of the DAE module. This can be viewed as the RUL point prediction of the model. This can be viewed as the 80% confidence interval boundary for the model's RUL prediction. Finally, the AdamW optimizer can be used to minimize the above objective function.

[0091] This paper proposes a method to train a model by extracting hand-crafted features from domain knowledge of lithium-ion batteries and automatically extracting features from historical usage data of lithium-ion batteries, and then fusing these features. This increases the completeness of feature extraction and improves the reliability of subsequent model training.

[0092] Furthermore, the preset convolutional neural network module includes convolutional layers, pooling layers, and fully connected layers;

[0093] The convolutional layer is used to extract features from the original policy data and generate a feature map;

[0094] The pooling layer is used to reduce the dimensionality of the feature maps extracted by the convolutional layer;

[0095] The fully connected layer is used to concatenate the pooled feature maps to obtain the feature vector of the original measurement data, denoted as: H r1 =E1(X raw ), H r1 E1 is the feature vector; E1 is the preset convolutional neural network module, X raw The initial characteristic vector of the original measurement data has dimensions n×l×m, where n represents the number of battery samples, l represents the total number of original feature statistics, and m represents the number of cycles.

[0096] In other words, the Convolutional Neural Networks (CNN) module consists of three main parts: convolutional layers, pooling layers, and fully connected layers. The main function of the convolutional layer is to extract features from the input data, which contains multiple convolutional kernels. Each convolutional kernel has corresponding weight coefficients and biases, similar to neurons in a feedforward neural network. Through convolution operations, CNN can effectively capture the spatial structure information of the input data, thereby realizing feature extraction and representation of complex data such as text. The pooling layer is used to reduce the dimensionality of the feature map, reduce the amount of computation, and enhance the robustness of the model. The fully connected layer maps the pooled feature map to the output space for the final classification or prediction task. The convolution formula is defined as shown in formula (4):

[0097]

[0098] in, W represents the output value of the i-th feature in layer (l+1). i l+1 This represents the weight matrix of the i-th convolutional kernel in layer (l+1), where "*" indicates the convolution operation, X (l) This represents the output of layer (l+1). This indicates the bias term.

[0099] The Transformer module mainly consists of a denoising autoencoder (DAE) and a Transformer layer. The module architecture is as follows: Figure 3 As shown. To maintain the robustness of the model, the original data (i.e., the original measurement data) must be processed by DAE before entering the model. DAE is based on Auto-Encoder (AE), and noise is added to the input data to ensure the noise resistance of the model. The features learned by DAE from the input with superimposed noise are more robust. The output of the DAE encoder is shown in Equation (5):

[0100]

[0101] in, b and b represent the output, activation function, weights, and bias of the DAE encoder, respectively.

[0102] Then, in order to reconstruct the input vector, the latent representation is mapped back to the input space, and the reconstructed input vector is shown in Equation (6):

[0103]

[0104] Where W', b' and f(.) represent the weights, biases and mapping functions of the DAE decoder, respectively.

[0105] Since the Transformer layer does not use an RNN structure but instead utilizes global information, it cannot directly utilize the order information of the input data. However, order information is crucial in prediction tasks involving time series data. Therefore, positional encoding is introduced into the Transformer model to store the relative or absolute position of the input data within the sequence. To achieve this, this specification uses sine and cosine functions of different frequencies to represent the positional encoding, and the calculation formulas are shown in equations (7) and (8):

[0106] PE(p,2k)=sin(t / 10000 2k / d (7)

[0107] PE(p,2k+1)=cos(t / 10000 2k / d (8)

[0108] Where p represents the position of the feature in the time series data, d represents the dimension of the feature, 2k represents an even dimension, and 2k+1 represents an odd dimension.

[0109] In a further embodiment, the Transformer layer includes an encoder and a decoder. The encoder receives the input sequence, maps it to a high-dimensional vector, and feeds it into the decoder to produce an output sequence. In this illustration, only the Transformer encoder is used to learn long-term dependencies on lifetime decay from battery running records. The Transformer encoder has two sub-layers: a multi-head self-attention mechanism and a feedforward network, which can capture dependencies between features and ignore their distance in the sequence. The multi-head attention is defined as shown in Equation (9), then the i-th attention function head i The definition is expressed as in formula (10):

[0110] MultiHead(H l-1 )=[head1;head2;...;head h W O (9)

[0111] head i =Attention(H l-1 W i Q H l-1 W i K H l-1 W i V (10)

[0112] Among them, H l-1W represents the representation layer of (l-1)-th. i Q W i K W i V The mapping weights are represented as i∈[1,h], where h represents the number of heads in the multi-head attention mechanism, Q, K, and V represent the query, key, and value, respectively, and W represents the value. o These are training weights;

[0113] The Scaled DotProduct Attention is defined as shown in Equation (11):

[0114]

[0115] Where h represents the number of heads in the multi-head attention mechanism, and d represents the dimension of the feature.

[0116] Feedforward networks have two different mapping methods: linear and ReLU nonlinear. This is based on the previous MultiHead(H) mapping method. l-1 From this, we can know that H l As shown in Equation (12), the feedforward network FFN(x) can be expressed as shown in Equation (13):

[0117] H l =FFN(MultiHead(H l-1 (12)

[0118] FFN(x)=ReLU(xW1+b1)W2+b2 (13)

[0119] Where ReLU(.) represents the mapping function, W1 and W2 represent the weights, and b1 and b2 represent the biases.

[0120] In summary, the embodiments of this specification use a one-dimensional CNN encoder (i.e., a convolutional neural network) E1 to encode these raw data into feature vectors H. r1 As shown in formula (14). Similarly, the original data is encoded into H using the DAE-Transformer encoder E2. r2 As shown in formula (15):

[0121] H r1 =E1(X raw (14)

[0122]

[0123] This allows for the acquisition of complete and comprehensive feature vectors, improving the reliability and accuracy of subsequent model training. By using CNN and DAE-Transformer models to obtain local and global degradation information, automatic feature extraction from the original degradation data can be achieved.

[0124] In some embodiments of this specification, based on domain knowledge of the lithium-ion battery, a set of features related to battery life is obtained through correlation analysis, including:

[0125] Based on the domain knowledge of the lithium-ion battery, multiple feature information is extracted;

[0126] The Spearman correlation coefficient between any two feature information is calculated using the following formula: Among them, R(X) i ),R(Y i ) represent the levels of each data point in feature information X and Y, respectively, and n is the number of samples;

[0127] Two features whose Spearman correlation coefficient exceeds a threshold are added to the candidate feature set, where the candidate feature set consists of X. domain 1 express;

[0128] Recursive feature elimination based on cross-validation is used to select the optimal number of features from the candidate feature set as the feature set related to battery life, wherein the feature set is determined by X. domain 2 It is indicated that its dimension is n×p, where n represents the number of battery samples and p represents the number of features in the feature set.

[0129] This paper can be understood as utilizing a recursive feature elimination method with cross-validation, Spearman correlation coefficient, and fully connected layers to achieve manual feature selection based on domain knowledge. Feature selection is an important problem in feature engineering, and its goal is to find the optimal subset of features. Feature selection can eliminate irrelevant or redundant features, thereby reducing the number of features, improving model accuracy, and reducing running time. The embodiments in this specification use Spearman correlation coefficient to evaluate the nonlinear degradation characteristics of lithium-ion batteries in the early stages, instead of the commonly used linear correlation Pearson coefficient. The calculation of Spearman's coefficient is shown in formula (16):

[0130]

[0131] Among them, R(X) i ),R(Y i ) are the ranks of each data point in vectors X and Y, respectively, and n is the number of samples.

[0132] Specifically, the feature selection process first uses the Spearman correlation coefficient to calculate the correlation between the domain-knowledge-based handcrafted features and the target RUL. Features with an absolute correlation value higher than 0.45 are selected as feature subset 1 (i.e., the candidate feature set). Then, X... domain 1 This is represented by the expression. Then, recursive feature elimination based on cross-validation (RFECV) is used to select the optimal number of features as feature subset 2, which is determined by X. domain 2 The feature subset 2 is then fed into network D, which consists of a series of fully connected layers, as shown in equation (17):

[0133] H d =D(X) domain 2) (17)

[0134] Among them, X domain The dimension of 2 is n×p, where n represents the number of battery samples and p represents the number of features in feature subset 2.

[0135] In the embodiments described in this specification, the hand-crafted features, the feature vectors, and the encoding vectors are concatenated to obtain training data. This can be understood as follows: after automatic feature extraction and domain-knowledge-based hand-crafted feature extraction, to prevent overfitting, dropout layers are set for the CNN encoder E1, DAE-Transformer encoder E2, and network D, respectively. Then, H... r1 H r2 H d The inputs that are connected together to form the integrated prediction model are shown in Equation (18).

[0136] H=(H r1 ||H r2 ||H d (18)

[0137] In the embodiments of this specification, the battery life prediction model is trained using the training data. During the training process, a cosine annealing strategy is used to adjust the learning rate, and model parameters from multiple different training stages are saved to a model list, including:

[0138] Based on the training data, the prediction results of the battery life prediction model are calculated through forward propagation, the loss function is calculated and backpropagation is performed to update the model parameters;

[0139] The current learning rate is calculated based on the current training round number and annealing cycle to adjust the model parameters for the next round of model training;

[0140] Save the parameters of the current model to the model list.

[0141] Further, the SWA parameters of the models in the calculation model list are averaged, and the battery life prediction model is trained a second time using the averaged SWA parameters to obtain the target battery life prediction model, including:

[0142] Iterate through each model parameter in the model list and calculate the average SWA parameter value;

[0143] Set the start_swa_epoch parameter;

[0144] The starting position of the second training is determined based on the start_swa_epoch parameter.

[0145] Based on the starting position of the secondary training, the training process before start_swa_epoch is performed using ordinary model parameters, while the training afterward uses the average value of SWA parameters, until a converged target battery life prediction model is obtained.

[0146] This invention can be understood as an improvement on the traditional Stochastic Weight Averaging (SWA) method. The SWA method improves the model's generalization ability by averaging model parameters across multiple training processes. However, simple averaging may lead to the model's inability to adapt to complex training data and ignores the uncertainty of model parameters. To address these issues, this invention proposes the Cosine-SWAI model. First, a three-layer loop is nested during the initial training process. The outer loop iterates through the models to be saved, the middle loop iterates through the training epochs of each model, and the inner loop iterates through each training step, such as: randomly selecting a mini-batch of training samples, calculating the model's prediction results through forward propagation, calculating the loss function and performing backpropagation, and updating the model's parameters. After the inner loop finishes, the current learning rate is calculated based on the current training epoch and annealing cycle. After the middle loop finishes, the parameters of the current model are saved to the model list. Second, the average SWA parameter is calculated for each model parameter in the model list. Finally, in the second training iteration, to fully utilize the training results of the ordinary model, the `start_swa_epoch` parameter was set. This parameter indicates from which epoch SWA parameter averaging begins. Training before `start_swa_epoch` continues using the ordinary model parameters, while training afterward uses SWA parameter averaging.

[0147] The purpose of this approach is to fully utilize the training results of a standard model in the early stages of training, helping the model converge to a local optimum more quickly. Then, in the later stages of training, SWA parameter averaging is used to further stabilize the model and search within a wider parameter space to achieve better generalization ability and performance. Through these improvements, the Cosine-SWAI model can better cope with uncertainties during model training, improving its performance and generalization ability. The Cosine-SWAI model integration training process provided in the embodiments of this specification is as follows: Figure 4 As shown.

[0148] In the embodiments of this specification, based on the target battery life prediction model, the boundary of the prediction confidence interval corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result. This can be understood as quantifying the uncertainty of lithium-ion battery RUL prediction, which can improve the accuracy and reliability of the prediction and provide important support and guidance for battery management, design, and manufacturing. This has significant practical implications for the development and application of battery technology. (Including:)

[0149] The embodiments in this specification utilize quantile regression to quantify the uncertainty of RUL prediction based on the CDT-CosineSWA method. The quantile regression loss is expressed by formula (19):

[0150]

[0151] Among them, y i For true RUL, Let represent the RUL predicted by the CDT-CosineSWA method at a score level q, where q∈[0,1], and (.). + =max(0,.) means that only the positive parts are kept, and the negative parts are set to zero.

[0152] In this case, when multiple quantile levels Q = {q1,...,q} are given m This invention can utilize the quantile loss of all training samples. For example, in the embodiments of this specification, Q = {0.1, 0.5, 0.9} is selected, and the quantile loss of all training samples is used. The final optimization problem can be expressed as formula (20):

[0153]

[0154] Where α represents the weighting coefficient for balancing the importance of different modules, S represents the sample size, M represents the interquartile range, and L... D This represents the reconstruction loss of the DAE module. This can be viewed as the RUL point prediction of the model. This can be viewed as the 80% confidence interval boundary for the model's RUL prediction. Finally, the AdamW optimizer can be used to minimize the above objective function.

[0155] The embodiments in this specification provide the following advantages and practical significance for early probabilistic prediction of the remaining lifespan of lithium-ion batteries:

[0156] (1) Early prediction of battery life: By predicting the lifespan of lithium-ion batteries in advance, the remaining lifespan can be estimated before the lifespan of the batteries has significantly decreased. This helps to identify potential lifespan problems early and take timely maintenance measures to extend the battery's lifespan.

[0157] (2) Optimize battery maintenance strategy: Early cycle prediction provides important information for the formulation of battery maintenance strategy. By understanding the predicted range of the battery in the early cycle stage, corresponding maintenance measures can be taken to avoid serious damage or failure of the battery in the later stage.

[0158] (3) Improve battery performance evaluation: Early interval prediction can help assess the performance differences between different battery technologies and materials. By making early predictions on multiple battery samples, high-performance batteries can be identified early, and the design and manufacturing process can be optimized.

[0159] (4) Optimize battery design and manufacturing: Understanding the battery's lifespan in the early stages helps in optimizing battery design and manufacturing. By adjusting and improving based on early predictions, battery performance and stability can be improved, thereby extending its overall lifespan.

[0160] (5) Improve battery safety: Early interval prediction helps to detect abnormalities in the battery as early as possible, such as capacity loss or power degradation. This can help to take timely safety measures and ensure battery safety.

[0161] (6) Accelerating the verification of new technologies: Early-stage prediction can help scientists and engineers understand the performance and lifespan of new lithium-ion battery technologies sooner for verification and application. This helps to quickly identify promising new technologies and promote innovation in the field of lithium-ion batteries.

[0162] Based on the methods provided above, this specification also provides a lithium-ion battery lifetime prediction device based on an integrated Transformer model, such as... Figure 5 As shown, the device includes:

[0163] The feature set acquisition module 510 is used to acquire domain knowledge of lithium-ion batteries and, based on the domain knowledge of lithium-ion batteries, obtain a feature set related to battery life through correlation analysis.

[0164] The manual feature acquisition module 520 is used to perform feature mapping on the feature set through three fully connected layers to obtain manual features related to battery life.

[0165] The encoding vector acquisition module 530 is used to acquire the raw measurement data of the lithium-ion battery, and input the raw measurement data into the preset convolutional neural network module E1 and the preset Transformer module E2 respectively to obtain the feature vector and encoding vector of the raw measurement data respectively.

[0166] The training data acquisition module 540 is used to concatenate the handcrafted features, the feature vectors, and the encoding vectors to obtain training data;

[0167] The training module 550 is used to train the battery life prediction model using the training data. During the training process, the learning rate is adjusted using a cosine annealing strategy, and the model parameters of multiple different training stages are saved to the model list. The average SWA parameter of the models in the model list is calculated, and the battery life prediction model is trained a second time using the average SWA parameter to obtain the target battery life prediction model.

[0168] The prediction module 560 is used to process the prediction confidence interval boundary corresponding to the specified prediction starting point through quantile regression loss according to the target battery life prediction model, so as to obtain the lithium-ion battery life prediction result.

[0169] The beneficial effects obtained by the above-described device are the same as those obtained by the above-described method, and will not be described in detail in the embodiments of this specification.

[0170] In another embodiment, this document also provides a battery management system including a lithium-ion battery and a controller configured with the methods described above.

[0171] This embodiment provides an electrical device, the internal structure of which can be shown in the following diagram. Figure 6 As shown, the electrical device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection.

[0172] Those skilled in the art will understand that Figure 6The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electrical equipment to which the present application is applied. Specific electrical equipment may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0173] In one embodiment, an electrical device is provided, including the battery management system described above.

[0174] In one embodiment, an electrical device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0175] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0176] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0178] It should also be understood that, in the embodiments herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated objects have an "or" relationship.

[0179] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0180] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0181] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0183] This document uses specific embodiments to illustrate the principles and implementation methods of this document. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this document. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this document. Therefore, the content of this specification should not be construed as a limitation of this document.

Claims

1. A method for predicting the lifetime of lithium-ion batteries based on an integrated Transformer model, characterized in that, The method includes: Acquire domain knowledge of lithium-ion batteries, and based on this domain knowledge, obtain a set of features related to battery life through correlation analysis; The feature set is mapped through three fully connected layers to obtain handcrafted features related to battery life. Obtain raw measurement data of the lithium-ion battery and input the raw measurement data into a preset convolutional neural network module. and the preset Transformer module In this process, the feature vector and encoding vector of the original measurement data are obtained respectively; The handcrafted features, the feature vectors, and the encoding vectors are concatenated to obtain training data; The battery life prediction model is trained using the training data. During the training process, the learning rate is adjusted using a cosine annealing strategy, and the model parameters of multiple different training stages are saved to the model list. The average SWA parameter of the models in the model list is calculated, and the battery life prediction model is trained a second time using the average SWA parameter to obtain the target battery life prediction model. Based on the target battery life prediction model, the prediction confidence interval boundary corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result.

2. The method according to claim 1, characterized in that, Based on the domain knowledge of lithium-ion batteries, a set of features related to battery life was obtained through correlation analysis, including: Based on the domain knowledge of the lithium-ion battery, multiple feature information is extracted; The Spearman correlation coefficient between any two feature information is calculated using the following formula: ,in, These are the levels of each data point in feature information X and Y, respectively. It is the sample size; Two features whose Spearman correlation coefficient exceeds a threshold are added to the candidate feature set, which consists of... express; Recursive feature elimination based on cross-validation is used to select the optimal number of features from the candidate feature set as the feature set related to battery life, wherein the feature set is composed of... It means that the dimension is , Indicates the number of battery samples. This indicates the number of features in the feature set.

3. The method according to claim 1, characterized in that, The preset convolutional neural network module includes convolutional layers, pooling layers, and fully connected layers; The convolutional layer is used to extract features from the original measurement data and generate a feature map; The pooling layer is used to reduce the dimensionality of the feature maps extracted by the convolutional layer; The fully connected layer is used to concatenate the pooled feature maps to obtain the feature vector of the original measurement data, represented as: , For feature vectors; For the pre-defined convolutional neural network module, The initial feature vector of the original measurement data has a dimension of . , Indicates the number of battery samples. This represents the total number of original features. Indicates the number of cycles.

4. The method according to claim 1, characterized in that, The preset Transformer module This includes a denoising autoencoder (DAE), a Transformer layer, and a position encoder; After obtaining the raw measurement data of the lithium-ion battery, the process further includes: adding Gaussian noise to the normalized input of the raw measurement data to obtain the corrupted initial feature vector. ; The features learned by the denoising autoencoder from the input with superimposed noise are more robust, and the output of the DAE encoder is represented as follows: , and Let represent the output, activation function, weights, and bias of the DAE encoder, respectively. Then, to reconstruct the input vector, the latent representation is mapped back to the input space. The reconstructed input vector is represented as . , and These represent the weights, biases, and mapping functions of the DAE decoder, respectively. The position encoder is used to determine the relative or absolute position of input data within a sequence. Position encoding is represented by sine and cosine functions of different frequencies, and the calculation formula is as follows: ； ； in, This indicates the location of the feature in the time series data. The dimension representing the feature. The dimension representing an even number. Indicates odd-numbered dimensions; The Transformer layer processes the output of the denoising autoencoder to learn the lifespan degradation dependency features from the battery operation records, as shown below. .

5. The method according to claim 4, characterized in that, The Transformer layer includes a stacked encoder and a decoder; the encoder receives the input sequence, maps it to a high-dimensional vector, and feeds it into the decoder to produce an output sequence. The Transformer encoder is used only to learn the long-term dependence of life decay from battery operation records. The encoder includes a multi-head self-attention mechanism and a feedforward network; The multi-head self-attention mechanism is used to capture dependencies between features and ignores their distance in the sequence, as follows: ； in, ; ； in, Indicates the mapping weights. This indicates the number of heads in the multi-head attention mechanism. These represent query, key, and value, respectively. These are training weights; The feedforward network is used to map the output of the multi-head self-attention mechanism to obtain the encoding vector of the original measurement data, as follows: ； ； in, express -th represents the layer. Indicates weight, Indicates deviation, Indicates a feedforward network. The first term of the feedforward network is represented by the second term. The output of the layer, The dimension representing the feature.

6. The method according to claim 1, characterized in that, The battery life prediction model is trained using the training data. During the training process, a cosine annealing strategy is used to adjust the learning rate, and model parameters from multiple different training stages are saved to a model list, including: Based on the training data, the prediction results of the battery life prediction model are calculated through forward propagation, the loss function is calculated and backpropagation is performed to update the model parameters; The current learning rate is calculated based on the current training round and annealing cycle to adjust the model parameters for the next round of model training; Save the parameters of the current model to the model list.

7. The method according to claim 1, characterized in that, The SWA parameters of the models in the calculation model list are averaged. The battery life prediction model is then retrained using the averaged SWA parameters to obtain the target battery life prediction model, including: Iterate through each model parameter in the model list and calculate the average SWA parameter value; Set the start_swa_epoch parameter; The starting position of the second training is determined based on the start_swa_epoch parameter. Based on the starting position of the secondary training, the training process before start_swa_epoch is performed using ordinary model parameters, while the training afterward uses the average value of SWA parameters, until a converged target battery life prediction model is obtained.

8. The method according to claim 1, characterized in that, Based on the target battery life prediction model, the prediction confidence interval boundary corresponding to the specified prediction starting point is processed by quantile regression loss to obtain the lithium-ion battery life prediction result, including: The quantile regression loss function based on the target battery life prediction model is determined as follows: ;in To represent the actual remaining useful life, This indicates that the target battery life prediction model is at a score level. The predicted remaining useful life under the given value. ,and This means that only the positive parts can be retained, and the negative parts are set to zero. Determine multiple quantile levels ； Based on the multiple quantile levels and the quantile regression loss function, the final objective function is obtained by using the quantile loss of all training data, expressed as: ,in This represents the weighting coefficients used to weigh the importance of different modules. Represents the sample size. Represents the interquartile range. This represents the refactoring loss of the DAE module. Considered as the prediction of the model's remaining useful life point, Consider it as the boundary of the 80% confidence interval for the model's remaining useful life prediction; The objective function was optimized using the AdamW optimizer to obtain the lithium-ion battery lifetime prediction results.

9. A battery management system, characterized in that, The system includes a lithium battery and a controller, the controller comprising the method of any one of claims 1 to 8.

10. An electrical appliance, characterized in that, The electrical equipment includes the battery management system as described in claim 9.

Images