A method for predicting the service life of lithium batteries for embedded devices

Abstract

A method for predicting the service life of lithium batteries for embedded devices, the steps include taking the historical voltage, current and time data of lithium-ion batteries as input features to create a prediction feature data set; preprocessing the prediction feature data set; forming a battery remaining service life time series data set from the preprocessed data set, and establishing a battery remaining service life prediction model; optimizing the incremental learning module of the model; using the Bayesian Optuna framework to optimize and dynamically adjust the weight consolidation items; optimizing the Optuna framework to improve the optimization efficiency; optimizing the information matrix Fisher used for online weight consolidation, and using random singular value decomposition for low-rank decomposition; using the priority experience replay method to optimize incremental learning; using the knowledge distillation method to optimize the priority experience replay. The model established by the present invention can maintain generalization ability in a variable use environment and more accurately reflect the actual use status of the battery.

Description

Technical Field

[0001] The invention relates to a method for predicting the service life of a lithium battery for embedded devices, and belongs to the technical field of lithium batteries. Background Art

[0002] As a leader in new energy storage technologies, lithium-ion batteries have made rapid progress in key materials, design, and fast charging technologies, significantly improving their safety and cycle life, and are widely used in various energy storage scenarios.

[0003] Lithium-ion batteries are usually used as energy supply modules. During operation, it is very important to continuously and accurately monitor the remaining useful life (RUL) of lithium-ion batteries. The performance of lithium-ion batteries decreases significantly as their capacity gradually decays. Therefore, RUL becomes a key indicator for evaluating the degree of battery aging. The capacity decay of lithium batteries often shows a relatively stable linear relationship at the beginning, but near the end of life, its decay trend will show a highly nonlinear characteristic. What is more complicated is that different charging and discharging strategies, ambient temperature changes, positive and negative electrode material selection, and battery capacity regeneration phenomena may have different effects on the capacity decay trend of the battery. This nonlinearity and complexity make it particularly difficult to accurately predict the remaining useful life of the battery. Incorrect estimation of the remaining available capacity may lead to overcharging or over-discharging of the battery, which will not only reduce the performance and life of the battery, but also may cause serious safety problems, such as explosion or fire of mobile phone batteries and electric vehicle batteries.

[0004] At present, the RUL prediction models for lithium-ion batteries mainly include models based on electrochemical mechanisms and semi-empirical models. Although these two prediction models can realize the evaluation of the remaining service life of lithium-ion batteries to a certain extent, they have the defects of being difficult to establish reliable models to accurately simulate the battery system or determine the model parameters and poor generalization during use. Based on this, the intelligent algorithm used to predict the remaining capacity of the battery has emerged as a method that is flexible in the prediction of lithium-ion battery life, and has strong generalization and real-time performance. However, the intelligent algorithms currently used for prediction are usually trained on traditional servers or clouds, and the training data used is transmitted once, which does not conform to the actual prediction application scenario. Summary of the invention

[0005] The purpose of the present invention is to provide a method for predicting the service life of lithium batteries for embedded devices, which can process data in real time and make predictions in a timely manner, thereby reducing the delay in transmitting data to the cloud; the model can be updated in a step-by-step manner, effectively avoiding the problem of overfitting the model to old data; the model established by this method can maintain generalization capabilities in a changing usage environment and more accurately reflect the actual usage status of the battery.

[0006] In order to achieve the above object, the present invention provides a method for predicting the service life of a lithium battery for an embedded device, comprising the following steps:

[0007] Step 1. Collect the voltage, current, time, discharge and charge capacity data of lithium-ion batteries under historical usage conditions and cycle times, and use the voltage, current and time data as the basis to create input features, thereby establishing a prediction feature data set;

[0008] Step 2: Preprocess the prediction feature data set established in Step 1;

[0009] Step 3, re-dividing the data in the preprocessed data set into multiple subsets to form a battery remaining life time series data set, and establishing a lithium-ion battery remaining service life prediction model based on the RALSTM model;

[0010] Step 4. Optimize the incremental learning module of the RALSTM model and calculate the weight consolidation term based on the parameter importance and the update of the model parameters;

[0011] Step 5. Optimize the regularization strength λ of the online weight consolidation used in Step 4, and use the Bayesian Optuna framework to optimize and dynamically adjust the weight consolidation term;

[0012] Step 6. Optimize the Optuna framework to improve the optimization efficiency; optimize the information matrix Fisher used for online weight consolidation, and use random singular value decomposition for low-rank decomposition;

[0013] Step 7: Use the priority experience replay method to optimize the incremental learning in Step 4;

[0014] Step 8: Use the knowledge distillation method to optimize the priority experience replay of Step 7.

[0015] Furthermore, the historical use conditions and cycle times of the lithium-ion battery in Step 1 are more than 1000 cycles at 25 degrees Celsius; the process of establishing the prediction feature data set is as follows:

[0016] Step 1-1, record the discharge time of each battery cycle discharge stage, calculate the time required for the voltage to drop from 3.6V to 3.4V during the battery discharge cycle; in the discharge stage, find and record the maximum voltage in each cycle; traverse each battery cycle, select the data of the charging stage, calculate and record the minimum voltage; in the charging stage, calculate the time required to charge from the starting voltage to 4.15V; calculate the time span during the charging stage when the current is close to the maximum value and is basically constant, and calculate the time difference according to the screened current value range as the duration of the constant current; for each cycle, calculate the total time of the entire charging stage;

[0017] Step 1-2: Calculate the difference between the current cycle number and the last cycle number of each charge and discharge cycle, which is the remaining service life of the current charge and discharge cycle, and add this feature as a prediction feature.

[0018] Furthermore, the process of preprocessing the data set in Step 2 is as follows:

[0019] Step 2-1. Use the prediction feature data set established in Step 1 to draw the corresponding box plot to observe the data distribution and possible outliers, that is, data that deviates significantly from most data points, including measurement errors, input errors, and data generated by equipment failures.

[0020] Step 2-2, record the first and third quartiles of the data as Q1 and Q3 respectively, and use the quantile function to calculate the interquartile range IQR, the formula is: IQR = Q3-Q1;

[0021] Use 1.5 times the interquartile range to set the upper and lower bounds of the outlier threshold; the upper and lower bounds are lower_bound and upper_bound respectively. Determine the outliers according to the upper and lower bounds and delete them from the data to obtain the cleaned data. The formula is:

[0022] lower_bound=Q1 -1.5*IQR

[0023] upper_bound=Q3+1.5*IQR

[0024] Step 2-3. Calculate the covariance matrix of the cleaned data in Step 2-2. The data contains the input features and the predicted RUL features. Draw the corresponding heat map according to the covariance matrix. The formula of the covariance matrix is:

[0025]

[0026] Where, X i and Y i are the values ​​of variables X and Y of the i-th sample, and are the sample means of variables X and Y respectively, and n is the number of samples;

[0027] Step 2-4: Use the preprocessed data in Step 2-2 to train the random forest regression model to obtain the importance results of all input features. By observing the contribution of each feature to the model prediction performance, combined with the covariance heat map obtained in Step 2-3, the input features that contribute little to the prediction target are eliminated.

[0028] Step 2-5: Use the PCA method to perform principal component analysis on the data set obtained after screening and eliminating the input features in Step 2-3 and Step 2-4.

[0029] Furthermore, the process of establishing the remaining service life prediction model of lithium-ion batteries based on the RALSTM model in Step 3 is as follows:

[0030] Step 3-1, group the preprocessed data set in step 2 into sub-data according to the set time interval, time t n The entire dataset is the collection of all sub-datasets:

[0031]

[0032] Step 3-2: Modify the LSTM network through incremental learning to obtain the RALSTM network model for edge devices. During the analysis process, the data set analyzed at a certain time will not be analyzed again. The necessary information of the previous sub-dataset is transferred to the current sub-dataset, that is, the last value of the cell state, the hidden state, and the predicted value used to calculate the cost function are transferred to the next sub-dataset. These values ​​continue until time step t. n The performance of the RALSTM network model is measured by the cost function, which usually calculates the error between the predicted value and the actual value, and uses the root mean square error RMSE as the cost function. The formula is:

[0033]

[0034] Where N is the total number of data points evaluated, is the model’s prediction at time step i, y i is the actual value at time step i;

[0035] The RALSTM network model in Step 3-3 and Step 3-2 includes two LSTM layers, a bidirectional LSTM layer, a fully connected layer, and an output layer. A Dropout layer is added after each LSTM layer to reduce overfitting. The loss function is set to the root mean square error. At the same time, the AdamW optimizer is used for training. The weight decay formula applied by the AdamW optimizer is as follows:

[0036] θ t+1 =θ t -u·θ t

[0037] Where u is the weight decay coefficient, and additional weight decay is performed after the gradient update, θ t represents the model parameters at time step t during the training process, θ t+1 is the model parameter at time step t+1, that is, the new parameter value after an update;

[0038] Step 3-4: Use the RALSTM network model built in Step 3-2 to train using the incremental learning method. Divide it into several sub-datasets according to the data set division method in Step 3-1. 80% of each sub-dataset is used as a training set and 20% as a test set. Establish an independent test set for the sub-dataset to facilitate the verification of the forgetting of learned knowledge in subsequent training.

[0039] Furthermore, in Step 4, the weight consolidation term is calculated based on the parameter importance and the update of the model parameters, and the steps are as follows:

[0040] Step 4-1. Using the RALSTM model established in Step 3, an additional weight consolidation term is introduced into its loss function so that the model can keep the memory of the previously learned tasks while learning new tasks. The loss function includes the relevant loss of the original task and a weight consolidation term. The formula is:

[0041]

[0042] Where L(θ; x, y) is the loss of the original task, λ is the regularization strength, which controls the balance between new and old knowledge, and F i is the Fisher information matrix corresponding to the parameter θ i The elements of θ indicate the importance of this parameter in the previous task. i is the current model parameter, θ i * is the optimal value of the parameter at the end of the previous task;

[0043] The Fisher information matrix that needs to be calculated for the weight consolidation items in Step 4-2 and Step 4-1 is used to measure the importance of parameters. The formula is:

[0044]

[0045] Where p(y|x; θ) is the probability that the model outputs y given the input x and parameter θ. is the log-likelihood function with respect to the parameter θ i The partial derivative of represents the parameter θ i The expected value of the square of the partial derivative of the log-likelihood function;

[0046] Step 4-3: According to the method of calculating the Fisher information matrix in Step 4-2, after each task is completed, the information matrix is ​​updated to reflect the importance of each parameter in the latest task. The formula is:

[0047]

[0048] Where β is the attenuation coefficient, which is used to balance the old Fisher information and the newly calculated Fisher information. is the gradient of the current task loss to the parameter, Represents the square of the gradient, which is used to estimate Fisher information;

[0049] Step 4-4: Using the loss function calculated in Step 4-1 and the Fisher information matrix calculated in Step 4-2 and Step 4-3, during each task iteration, the model parameter update takes into account the current task loss gradient and the weight consolidation term adjusted by the Fisher information matrix to adapt to the continuously changing incremental learning task.

[0050] Furthermore, the steps of using the Bayesian Optuna framework in Step 5 to optimize and dynamically adjust the weight consolidation item are as follows:

[0051] Step 5-1. Use the Optuna framework to build a probability model p(f|λ,M) to approximate the objective function f, where λ is the regularization strength λ in the weight consolidation term in Step 4-1, and M is the observed objective function value f, which is the mean square error MSE. By selecting new parameter values ​​for evaluation, a score that can be used for optimization is returned, and finally a minimum value of the objective function is found;

[0052] Step 5-2, use the Gaussian process GP as a priori representation of the initial hypothesis of the objective function. The Gaussian process is a random process defined on a continuous input space and is defined as:

[0053] f(x)~gp(m(x),k(x,x′))

[0054] Where m(x) is the mean function and k(x,x′) is the kernel function, which is used to describe the correlation between points in the input space.

[0055] Step 5-3: In each experiment, the specific value of the regularization strength λ is used to build and train the model, and then the mean square error (MSE) of the model on the training set is calculated as the objective function value, expressed as:

[0056]

[0057] Among them, y is the true target value, is the predicted value of the model under the hyperparameter λ;

[0058] Step 5-4: Use the new observation data in Step 5-3 to update the posterior distribution of the Gaussian process. During the updating process, the mean and covariance of the updated posterior are calculated to reflect the newly acquired knowledge, which can be expressed as:

[0059] m new (x) = m old (x)+K(x,A)[K(A,A)+σ 2 I] -1 (Bm old (A)

[0060] k new (x,x′)=k old (x,x′)-K(x,A)[K(A,A)+σ 2 I] -1 K(A,x′)

[0061] In the formula, m old (x) is the mean of the previous estimates at point x, A represents the set of all input points in the training data set, B is the output value corresponding to A in the training data set, K is the covariance matrix calculated by the kernel function, σ 2 is the noise term;

[0062] Step 5-5, use the expected improvement EI as the acquisition function to select the next evaluation point from the updated posterior distribution, the formula is:

[0063]

[0064] Where τ is the lowest MSE observed so far, p(y|x,D) is the predicted distribution given the regularization strength λ under the current data D;

[0065] Step 5-6: Repeat the process of Step 5-5, and iterate to find the regularization strength λ value that minimizes the objective function L(λ) to find the optimal hyperparameter configuration.

[0066] Furthermore, the steps for optimizing the Optuna framework in Step 6 are as follows:

[0067] Step 6-1. Use the median pruning method to terminate the poorly performing experiments in the optimization of the regularization strength λ of the weight consolidation term in advance. Let L(t,i) be the performance index of the i-th experiment at the t-th step. The performance index here is the mean square error loss. In each step t, calculate the median of L(t,i) of all experiments, and check whether the results of the experiments meet the pruning conditions. If they meet the conditions, terminate experiment i in advance. Otherwise, experiment i continues to the next step, which is expressed as:

[0068] M(t)=median({L(t,j)|j∈J t})

[0069] In the formula, J t is the set of all trials measured at time t, and M(t) calculates the median performance metric of all trials at step t;

[0070] L(t,i)>M(t)×(1+α)

[0071] In the formula, α is the preset tolerance rate, which indicates the percentage of performance degradation allowed. The above formula indicates the judgment condition for pruning.

[0072] Step 6-2, set the hyperparameter space of the Optuna framework to 1e3 to 5e4, 1e3 and 5e4 are the lower and upper limits of the sampling range respectively, and use a logarithmic scale to narrow the scope of optimization;

[0073] Step 6-3, use the n_jobs parameter in the study.optimize function to specify the number of threads to run in parallel. Setting n_jobs = -1 means using all available processor cores. At the same time, use the storage_url parameter to specify the database and support parallel access to ensure consistency of test results and state synchronization between different threads.

[0074] Step 6-4, calculate the Fisher information matrix F on the uncompressed data, and use the random singular value decomposition SVD to approximate F, and get the following formula:

[0075] F≈UΣV T

[0076] Where U and V are orthogonal matrices, Σ is a diagonal matrix containing singular values, and the elements on its diagonal are singular values, which represent the importance of the data on each principal component.

[0077] Furthermore, the steps of optimizing the incremental learning in Step 4 using the method of prioritizing experience replay in Step 7 are as follows:

[0078] Step 7-1, set the maximum capacity C of the experience replay cache; when the cache is full, calculate the absolute value of the model prediction error to get the priority, and remove the samples according to the rule with the lowest priority so that new samples can be added;

[0079] Step 7-2: Divide the data in the cache into K intervals according to the absolute value of the error. Each interval is called a layer. Calculate the priority of the newly added sample. The priority p i It is expressed as:

[0080] p i =|error i ∣+∈

[0081] Where, |error i ∣ is the absolute value of the error, ∈ is a constant;

[0082] Step 7-3: Using the sample priority calculation in Step 7-2, the priorities in each layer are further normalized and attenuated to adjust the priorities of samples that have not been trained for a long time, expressed as:

[0083] p i (t) = p i (t-1)·δ

[0084] Where δ is the attenuation factor, and its value range is (0,1);

[0085] Step 7-4: Use the normalized priority of each layer in Step 7-3 to perform sampling. The sampling probability P(i) of each layer is expressed as:

[0086]

[0087] Among them, γ is the priority weight adjustment parameter, L k is the set of all samples in the k-layer divided by Step7-2;

[0088] Step 7-5, extract samples from the experience replay cache according to the above probability, and train the model with new data. After training, recalculate the priority of the samples used and update them. At the same time, in order to solve the deviation caused by non-uniform sampling, calculate the importance sampling weight. The formula is:

[0089]

[0090] Among them, W is the size of the cache and μ is used to adjust the influence of the weight.

[0091] Furthermore, in Step 8, the priority experience replay of Step 7 is optimized by using the knowledge distillation method, and the data in the experience replay library is predicted to generate distillation labels, so that the model of the new task learns the distillation labels and the actual target labels. The steps include:

[0092] Step 8-1: Before each new task, train a model on the initial task or old task, also called the teacher model.

[0093] Step 8-2: Use the teacher model trained in Step 8-1 to predict the data stored in the experience replay buffer and generate distilled labels. The distilled labels represent the response of the teacher model to a specific input and contain implicit knowledge about how to perform the task.

[0094] Step 8-3, using the same input data, the model that learns knowledge from the teacher model in Step 8-1 is called the student model. This model learns to predict the output of the teacher model, that is, the distilled label, and also learns the true target label. The training loss includes the loss of predicting the true label and the loss of predicting the distilled label, which is expressed as:

[0095] L=q·L true (y,y s )+(1-q)·L distill (y t ,y s )

[0096] In the formula, the input data is x input , the true label is y true ,y t is the output of the teacher model, i.e., the distilled output, and the output of the student model is y s , L true is the loss of the student model predicting the true label, L distill is the loss of the student model predicting the distilled labels, and q is a trade-off parameter used to control the relative importance of the two losses.

[0097] The present invention collects voltage, current, time, discharge and charge capacity data under historical use conditions and cycle times of lithium-ion batteries, uses voltage, current and time data as input features, creates a prediction feature data set; preprocesses the created prediction feature data set; re-divides the data in the preprocessed data set into multiple subsets to form a battery remaining life time series data set, and establishes a lithium-ion battery remaining service life prediction model based on the RALSTM model; optimizes the incremental learning module of the RALSTM model, calculates the weight consolidation item based on the parameter importance and the update of the model parameters; optimizes the hyperparameters of online weight consolidation, uses the Bayesian Optuna framework to optimize and dynamically adjust the weight consolidation item; optimizes the Optuna framework to improve the optimization efficiency; optimizes the information matrix Fisher used in the online weight consolidation, uses random singular value decomposition for low-rank decomposition; uses the priority experience replay method to optimize incremental learning; uses the knowledge distillation method to optimize the priority experience replay. The present invention solves the problem that the model in the prior art is difficult to adapt to real-time changing data and be deployed on embedded devices. By preprocessing the charging cycle data used and screening out useful features, a resource-aware long short-term memory neural network is used to build a prediction model for the remaining service life of lithium-ion batteries. Finally, the elastic weight consolidation method is used to prevent the forgetting of old data and models, and the experience replay strategy and knowledge distillation method are integrated to balance the learning of new and old knowledge. The model is repeatedly iterated and optimized, and the model is tuned at the same time, realizing incremental learning in embedded devices and achieving the purpose of real-time prediction. The present invention achieves the generalization ability in a changing use environment, more accurately reflects the actual use status of the battery, and is particularly suitable for embedded devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a schematic diagram of the process of the present invention;

[0099] Figure 2 It is a flow chart of the incremental learning method of RALSTM based on resource awareness of the present invention;

[0100] Figure 3 It is a correlation matrix heat map based on a data set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0101] The present invention will be further described below in conjunction with the accompanying drawings.

[0102] A method for predicting the service life of a lithium battery for an embedded device comprises the following steps:

[0103] Step 1. Collect the voltage, current, time, discharge and charge capacity data of lithium-ion batteries under historical usage conditions and cycle times, and use the voltage, current and time data as the basis to create input features, thereby establishing a prediction feature data set;

[0104] Step 2: Preprocess the prediction feature dataset created in Step 1.

[0105] Step 3, re-dividing the data in the preprocessed data set into multiple subsets to form a battery remaining life time series data set, and establishing a lithium-ion battery remaining service life prediction model based on the RALSTM model;

[0106] Step 4. Optimize the incremental learning module of the RALSTM model and calculate the weight consolidation term based on the parameter importance and the update of the model parameters;

[0107] Step 5. Optimize the regularization strength λ of the online weight consolidation used in Step 4, and use the Bayesian Optuna framework to optimize and dynamically adjust the weight consolidation term;

[0108] Step 6. Optimize the Optuna framework to improve the optimization efficiency; optimize the information matrix Fisher used for online weight consolidation, and use random singular value decomposition for low-rank decomposition;

[0109] Step 7: Use the priority experience replay method to optimize the incremental learning in Step 4;

[0110] Step 8: Use the knowledge distillation method to optimize the priority experience replay of Step 7.

[0111] Example:

[0112] The experimental data comes from the Hawaii Natural Energy Institute. The data set consists of commercial 18650 batteries with graphite negative electrodes and mixed positive electrodes composed of NMC and LCO. The battery is charged and discharged at room temperature and the charging rate is 0.5C. When charged to 100%, the discharge rate is 1.5C. The charge and discharge cycles are more than 1000 times. A total of 14 groups of batteries are experimented together. This experiment takes the prediction of the remaining service life of the battery as an example. First, create a new data set with 9 columns and 15064 rows, that is, more than 1000 cycles for each test battery. Each column is: cycle index, F1: total discharge time, F2: discharge duration from 3.6V to 3.4V, F3: discharge start voltage, F4: starting voltage during charging, F5: charging time to reach 4.15V, F6: DC charging time, F7: complete charging time, RUL: target variable, that is, the number of remaining battery charging cycles;

[0113] like Figure 2 As shown in the figure, by drawing a box plot and calculating the interquartile range of the data, the data is cleaned according to the calibrated threshold of the outlier; secondly, the covariance matrix between the features based on the data set is calculated, and a heat map is drawn. At the same time, a simple random forest regression model is trained based on the data set. Combining the results of the two, the correlation and importance of the features are understood and some preset features are eliminated; finally, the principal component analysis is performed on the data to reduce the redundancy between features and improve the generalization ability of the model;

[0114] like Figure 3 As shown, by drawing a correlation matrix heat map based on the data set, it can be analyzed that the correlation between F2, F3, F5 and F6 features and RUL is relatively high, and F7 has the lowest correlation. Among them, F4 is negatively correlated with the remaining service life of the battery, and other feature variables are positively correlated. As shown in Table 1, the feature importance ranking is obtained by training the random forest regression model. Among them, F1 has the highest feature importance, and the feature importance of F7, F4 and F5 decreases in turn. Combining the correlation matrix heat map and feature importance ranking, feature F7 does not perform well in both evaluation criteria compared to other features. Therefore, feature F7 can be eliminated, and the remaining features are used to predict the remaining service life of lithium-ion batteries. The specific data samples after preprocessing are shown in Table 2;

[0115] Table 1 Feature importance ranking table

[0116] F1 F2 F6 F3 F7 F4 F5 0.8879 0.0820 0.0115 0.0079 0.0052 0.0033 0.0018

[0117] Table 2 Preprocessed data samples

[0118]

[0119]

[0120] The entire experiment was carried out on the embedded motherboard Rockchip RK3399pro, which uses CortexA72 and CortexA53 processors, and uses the Debian10 Linux operating system, version buster, and uses TensorFlow2.10.0 to carry out model training and prediction experiments, simulating incremental learning and predicting the remaining battery life on embedded devices.

[0121] First, if Figure 2As shown in the figure, a RALSTM model is built. The first layer of LSTM with 100 units is built, and then a Dropout layer with a drop rate of 0.3 is added; a second layer of LSTM with 100 units is added, and a Dropout layer with a drop rate of 0.3 is added again; a bidirectional LSTM with 50 units in each direction is added, and then a Dropout layer with a drop rate of 0.3 is added; a fully connected layer with 50 units is added; finally, an output layer with 1 unit is added to output the predicted value; the entire RALSTM model uses ReLU as the activation function, and the AdamW optimizer and mean square error loss function are used to compile the model.

[0122] Next, using the preprocessed datasets shown in Table 2, the entire dataset is divided into five equal parts to simulate incremental data. These five datasets are called Dataset1, Dataset2, Dataset3, Dataset4, and Dataset5. The last 20% of each dataset is used as a test set.

[0123] like Figure 1 As shown in the figure, the above-mentioned Dataset1 is input into the RALSTM model that has been built above. The model is used as a teacher model and performs initial training; the Fisher information matrix is ​​calculated to measure the importance of parameters; the online elastic weight consolidation module is fixed to prevent the model from forgetting previously learned knowledge in subsequent training; representative samples are selected from Dataset1 and saved to the experience playback library for review in subsequent stages.

[0124] Input Dataset2 into the student model and the teacher model; perform knowledge distillation, use the output of the teacher model as the target of the student model, and guide the student model learning through the regularization term (regularization strength is λ1); calculate the Fisher information matrix and update the online elastic weight consolidation module; select representative samples from Dataset2 and save them to the experience replay library for review in subsequent stages.

[0125] Repeat the above incremental training process, and gradually input Dataseti into the student model for training; use knowledge distillation technology to guide the student model learning through the regularization term (regularization strength is λi); continuously calculate the Fisher information matrix and update the online elastic weight consolidation module; select representative samples from each dataset and update the experience replay library.

[0126] In the above, each task is trained in a loop. The model.fit function is used in TensorFlow to train the model and update the optimal parameters and Fisher information. After each training, the current trained model is used to predict and evaluate the task and all previous tasks on the corresponding test set. Finally, the RMSE result of each task is calculated and printed out, as shown in Table 3. In the five incremental learning tasks, no catastrophic forgetting occurs. As the data increases, the model is constantly updated, and a certain degree of forgetting will occur. The degree of catastrophic forgetting can be well reduced through regularization methods and experience replay strategies. In addition, the attenuation factor is added to the model to well balance the learning of old knowledge and new knowledge.

[0127] Table 3 Model prediction results

[0128]

[0129] Through this incremental learning framework, the model can retain previously learned knowledge to the greatest extent while continuously learning new tasks, thereby reducing the impact of catastrophic forgetting; this method combines techniques such as knowledge distillation, online elastic weight consolidation, and experience replay to ensure the robustness and effectiveness of the model when facing multiple tasks.

[0130] The present invention can collect and analyze data in real time on embedded devices, balance the learning of new and old knowledge, and significantly reduce the demand for computing resources by constructing a modular incremental learning method based on resource perception. At the same time, the method uses online elastic weight consolidation and experience replay strategy combined with knowledge distillation to effectively reduce the forgetting of old knowledge in the incremental learning process. In addition, the Bayesian Optuna framework is used to optimize and dynamically adjust the hyperparameters of elastic weight consolidation, as well as the method of optimizing the information matrix and experience replay sample library used for online weight consolidation, which improves the efficiency and prediction accuracy of the model. Through experimental analysis, the effectiveness of the method of predicting the service life of lithium batteries based on resource-aware long short-term memory network and incremental learning is verified on embedded devices, so that the RMSE of lithium battery service life prediction is reduced, and the performance of the constructed RALSTM model is better than the algorithm using stochastic gradient descent (SGD) and no incremental learning, and through a variety of resource optimization methods, the running time and load on embedded devices are further reduced, and reliable and efficient prediction of the service life of lithium-ion batteries is achieved.

Claims

1. A method for predicting the service life of lithium batteries for embedded devices, characterized in that: The steps include: Step 1. Collect the voltage, current, time, discharge and charge capacity data of lithium-ion batteries under historical usage conditions and cycle times, and use the voltage, current and time data as the basis to create input features, thereby establishing a prediction feature data set; Step 2: Preprocess the prediction feature data set established in Step 1; Step 3, re-dividing the data in the preprocessed data set into multiple subsets to form a battery remaining life time series data set, and establishing a lithium-ion battery remaining service life prediction model based on the RALSTM model; Step 4. Optimize the incremental learning module of the RALSTM model and calculate the weight consolidation term based on the importance of parameters and the update of model parameters. The steps for calculating the weight consolidation term based on the importance of parameters and the update of model parameters are as follows: Step 4-1. Using the RALSTM model established in Step 3, an additional weight consolidation term is introduced into its loss function. The loss function includes the relevant loss of the original task and a weight consolidation term. The formula is: Where L(θ; x, y) is the loss of the original task, λ is the regularization strength, and F i is the Fisher information matrix corresponding to the parameter θ i The elements of θ i is the current model parameter, θ i * is the optimal value of the parameter at the end of the previous task; The Fisher information matrix that needs to be calculated for the weight consolidation items in Step 4-2 and Step 4-1 is as follows: Where p(y|x; θ) is the probability that the model outputs y given the input x and parameter θ. is the log-likelihood function with respect to the parameter θ i The partial derivative of represents the parameter θ i The expected value of the square of the partial derivative of the log-likelihood function; Step 4-3: According to the method of calculating the Fisher information matrix in Step 4-2, after each task is completed, the information matrix is ​​updated. The formula is: Where β is the attenuation coefficient, which is used to balance the old Fisher information and the newly calculated Fisher information. is the gradient of the current task loss to the parameter, Represents the square of the gradient, which is used to estimate Fisher information; Step 4-4, using the loss function calculated in Step 4-1 and the Fisher information matrix calculated in Step 4-2 and Step 4-3, in each iteration of the task, the model parameter update takes into account the current task loss gradient and the weight consolidation term adjusted by the Fisher information matrix to adapt to the continuously changing incremental learning task; Step 5. Optimize the regularization strength λ of the online weight consolidation used in Step 4, and use the Bayesian Optuna framework to optimize and dynamically adjust the weight consolidation term; Step 6. Optimize the Optuna framework to improve the optimization efficiency; optimize the information matrix Fisher used for online weight consolidation, and use random singular value decomposition for low-rank decomposition; Step 7: Use the priority experience replay method to optimize the incremental learning in Step 4; Step 8: Use the knowledge distillation method to optimize the priority experience playback of Step 7.

2. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The historical use conditions and cycle times of the lithium-ion battery in Step 1 are more than 1000 cycles at 25 degrees Celsius; the process of establishing the prediction feature data set is as follows: Step 1-1, record the discharge time of each battery cycle discharge stage, and calculate the time required for the battery voltage to drop from 3.6V to 3.4V during the battery discharge cycle; During the discharge phase, find and record the maximum voltage in each cycle; traverse each battery cycle, select the data in the charging phase, calculate and record the minimum voltage; during the charging phase, calculate the time required to charge from the starting voltage to 4.15V; Calculate the time span during which the current is close to the maximum value and is substantially constant during the charging phase, and calculate the time difference according to the screened current value range as the duration of the constant current; For each cycle, the total time of the entire charging phase was calculated; Step 1-2: Calculate the difference between the current cycle number and the last cycle number of each charge and discharge cycle, which is the remaining service life of the current charge and discharge cycle, and add it as a prediction feature.

3. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The process of preprocessing the data set in Step 2 is as follows: Step 2-1, use the prediction feature data set established in Step 1 to draw the corresponding box plot, observe the data distribution and possible outliers, that is, data that obviously deviates from most data points, including: measurement errors, input errors, and data generated by equipment failure; Step 2-2, record the first and third quartiles of the data as Q1 and Q3 respectively, and use the quantile function to calculate the interquartile range IQR, the formula is: IQR = Q3-Q1; Use 1.5 times the interquartile range to set the upper and lower bounds of the outlier threshold; the upper and lower bounds are lower_bound and upper_bound respectively. Determine the outliers according to the upper and lower bounds and delete them from the data to obtain the cleaned data. The formula is: lower_bound=Q1-1.5*IQR upper_bound=Q3+1.5*IQR Step 2-3. Calculate the covariance matrix of the cleaned data in Step 2-2. The data contains the input features and the predicted RUL features. Draw the corresponding heat map according to the covariance matrix. The formula of the covariance matrix is: In the formula, X i and Y i are the values ​​of variables X and Y of the i-th sample, and are the sample means of variables X and Y respectively, and n is the number of samples; Step 2-4: Use the preprocessed data in Step 2-2 to train the random forest regression model to obtain the importance results of all input features. By observing the contribution of each feature to the model prediction performance, combined with the covariance heat map obtained in Step 2-3, the input features that contribute little to the prediction target are eliminated. Step 2-5: Use the PCA method to perform principal component analysis on the data set obtained after screening and eliminating the input features in Step 2-3 and Step 2-4.

4. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The process of establishing the remaining service life prediction model of lithium-ion batteries based on the RALSTM model in Step 3 is as follows: Step 3-1, group the preprocessed data set in step 2 into sub-data according to the set time interval, time t n The entire dataset is the collection of all sub-datasets: Step 3-2: Modify the LSTM network through incremental learning to obtain the RALSTM network model for edge devices. During the analysis process, the data set analyzed at a certain time will not be analyzed again. The necessary information of the previous sub-dataset is transferred to the current sub-dataset, that is, the last value of the cell state, the hidden state, and the predicted value used to calculate the cost function are transferred to the next sub-dataset. These values ​​continue until time step t. n , the performance of the RALSTM network model is measured by the cost function, the error between the predicted value and the actual value is calculated, and the root mean square error RMSE is used as the cost function. The formula is: Where N is the total number of data points evaluated, is the model’s prediction at time step i, y i is the actual value at time step i; The RALSTM network model in Step 3-3 and Step 3-2 includes two LSTM layers, a bidirectional LSTM layer, a fully connected layer, and an output layer. A Dropout layer is added after each LSTM layer to reduce overfitting. The loss function is set to the root mean square error. At the same time, the AdamW optimizer is used for training. The weight decay formula applied by the AdamW optimizer is as follows: i t+1 =θ t -u·θ t Where u is the weight decay coefficient, and additional weight decay is performed after the gradient update, θ t represents the model parameters at time step t during the training process, θ t+1 is the model parameter at time step t+1, that is, the new parameter value after an update; Step 3-4: Use the RALSTM network model built in Step 3-2 to train using the incremental learning method. Divide it into several sub-datasets according to the data set division method in Step 3-1. 80% of each sub-dataset is used as a training set and 20% as a test set. Establish an independent test set for the sub-dataset to facilitate the verification of the forgetting of learned knowledge in subsequent training.

5. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The steps for optimizing and dynamically adjusting the weight consolidation terms using the Bayesian Optuna framework in Step 5 are as follows: Step 5-1. Use the Optuna framework to build a probability model p(f|λ,M) to approximate the objective function f, where λ is the regularization strength and M is the observed objective function value f, which is the mean square error MSE. By selecting new parameter values ​​for evaluation and returning a score for optimization, we can finally find the minimum value of the objective function. Step 5-2, use the Gaussian process GP as a priori representation of the initial hypothesis of the objective function. The Gaussian process is a random process defined on a continuous input space and is defined as: f(x)~gp(m(x),k(x,x′)) Where m(x) is the mean function and k(x,x′) is the kernel function, which is used to describe the correlation between points in the input space. Step 5-3: In each experiment, the specific value of the regularization strength λ is used to build and train the model, and then the mean square error (MSE) of the model on the training set is calculated as the objective function value, expressed as: Among them, y is the true target value, is the predicted value of the model under the regularization strength λ; Step 5-4: Use the new observation data in Step 5-3 to update the posterior distribution of the Gaussian process. During the updating process, the mean and covariance of the updated posterior are calculated to reflect the newly acquired knowledge, which can be expressed as: m new (x)=m old (x)+K(x,A)[K(A,A)+σ 2 I] -1 (B-m old (A)) k new (x,x′)=k old (x,x′)-K(x,A)[K(A,A)+σ 2 I] -1 K(A,x′) In the formula, m old (x) is the mean of the previous estimates at point x, A represents the set of all input points in the training dataset, B is the output value corresponding to A in the training dataset, K is the covariance matrix calculated by the kernel function, and σ 2 is the noise term; Step 5-5, use the expected improvement EI as the acquisition function to select the next evaluation point from the updated posterior distribution, the formula is: Where τ is the lowest MSE observed so far, p(y|x,D) is the predicted distribution given the regularization strength λ under the current data D; Step 5-6: Repeat the process of Step 5-5, and iterate to find the regularization strength λ value that minimizes the objective function L(λ) to find the optimal hyperparameter configuration.

6. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The steps for optimizing the Optuna framework in Step 6 are as follows: Step 6-1. Use the median pruning method to terminate the poorly performing experiments in the optimization of the regularization strength λ of the weight consolidation term in advance. Let L(t,i) be the performance index of the i-th experiment at the t-th step. The performance index here is the mean square error loss. In each step t, calculate the median of L(t,i) of all experiments, and check whether the results of the experiments meet the pruning conditions. If they meet the conditions, terminate experiment i in advance. Otherwise, experiment i continues to the next step, which is expressed as: M(t)=median({L(t,j)|j∈J t }) In the formula, J t is the set of all trials measured at time t, and M(t) calculates the median performance metric of all trials at step t; L(t,i)>M(t)×(1+α) In the formula, α is the preset tolerance rate, which indicates the percentage of performance degradation allowed. The above formula indicates the judgment condition for pruning. Step 6-2, set the hyperparameter space of the Optuna framework to 1e3 to 5e4, 1e3 and 5e4 are the lower and upper limits of the sampling range respectively, and use a logarithmic scale to narrow the scope of optimization; Step 6-3, use the n_jobs parameter in the study.optimize function to specify the number of threads to run in parallel. Setting n_jobs = -1 means using all available processor cores. At the same time, use the storage_url parameter to specify the database and support parallel access to ensure consistency of test results and state synchronization between different threads. Step 6-4, calculate the Fisher information matrix F on the uncompressed data, and use the random singular value decomposition SVD to approximate F, and get the following formula: F≈UΣV T Wherein, U and V are orthogonal matrices, Σ is a diagonal matrix and contains singular values, and the elements on its diagonal are singular values, which represent the importance of the data on each principal component.

7. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: The steps of using the priority experience replay method in Step 7 to optimize the incremental learning in Step 4 are as follows: Step 7-1, set the maximum capacity C of the experience replay cache; when the cache is full, calculate the absolute value of the model prediction error to get the priority, and remove the samples according to the rule with the lowest priority so that new samples can be added; Step 7-2: Divide the data in the cache into K intervals according to the absolute value of the error. Each interval is called a layer. Calculate the priority of the newly added sample. The priority p i It is expressed as: p i =∣error i ∣+∈ In the formula, |error i ∣ is the absolute value of the error, ∈ is a constant; Step 7-3: Using the sample priority calculation in Step 7-2, the priorities in each layer are further normalized and attenuated to adjust the priorities of samples that have not been trained for a long time, expressed as: p i (t)=p i (t-1)·δ Where δ is the attenuation factor, and its value range is (0,1); Step 7-4: Use the normalized priority of each layer in Step 7-3 to perform sampling. The sampling probability P(i) of each layer is expressed as: Among them, γ is the priority weight adjustment parameter, L k is the set of all samples in the divided k layers; Step 7-5, extract samples from the experience replay cache according to the above probability, and train the model with new data. After training, recalculate the priority of the samples used and update them. At the same time, in order to solve the deviation caused by non-uniform sampling, calculate the importance sampling weight. The formula is: Among them, W is the size of the cache and μ is used to adjust the influence of the weight.

8. The method for predicting the service life of lithium batteries for embedded devices according to claim 1, characterized in that: In the step 8, the priority experience replay of step 7 is optimized by using the knowledge distillation method, and the data in the experience replay library is predicted and distilled labels are generated, so that the model of the new task learns the distilled labels and the actual target labels. The steps include: Step 8-1: Before each new task, train a model on the initial task or old task, which is called the teacher model. Step 8-2: Use the teacher model trained in Step 8-1 to predict the data stored in the experience replay buffer and generate distilled labels. The distilled labels represent the response of the teacher model to a specific input and contain implicit knowledge about how to perform the task. Step 8-3, using the same input data, the model that learns knowledge from the teacher model in Step 8-1 is called the student model. The student model learns to predict the output of the teacher model, that is, the distilled label, and also learns the true target label. The training loss includes the loss of predicting the true label and the loss of predicting the distilled label, which is expressed as: L=q·L true (and,and s )+(1-q)·L distill (and t ,and s ) In the formula, the input data is x input , the true label is y true ,y t is the output of the teacher model, i.e., the distilled output, and the output of the student model is y s , L true is the loss of the student model predicting the true label, L distill is the loss of the student model predicting the distilled labels, and q is a trade-off parameter used to control the relative importance of the two losses.

Images