Battery RUL prediction method based on SMOE model and EIS
Through the SMOE model based on the multi-head attention mechanism, combined with multiple expert sub-models and attention mechanisms, the problem of insufficient generalization ability in battery RUL prediction is solved, and more efficient and accurate battery RUL prediction is achieved, which significantly improves the The effect of battery health management.
Patent Information
- Application Number
- CN202411412355.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional machine learning models have limited generalization capabilities in battery RUL prediction, which are prone to overfitting, and are not ideal when processing complex and heterogeneous data, resulting in insufficient accuracy in battery RUL prediction.
Using the SMOE model based on the multi-head attention mechanism, the construction of multiple expert sub-models, gated networks and attention mechanism calculation modules, combined with the multi-head attention mechanism, train and verify the model to improve the prediction accuracy of the battery RUL.
It improves the accuracy and efficiency of battery RUL prediction, can process battery EIS data more effectively, significantly reduces the amount of calculation, enhances the training and prediction efficiency of the model, and captures more complex and rich information, improving prediction accuracy.
Smart Images

Figure CN118917225A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a battery RUL prediction method, in particular to a battery RUL prediction method based on a SMOE model and EIS, belonging to the technical field of battery detection. Background Art
[0002] The remaining useful life (RUL) of a battery is one of the key indicators for measuring the health of a battery. Usually, the battery RUL can be expressed by the number of remaining cycles. The importance of battery RUL is mainly reflected in the following aspects: it is used to evaluate the health of the battery, guide the use and maintenance of the battery, provide early warning of potential battery failures, evaluate the capacity attenuation of the battery, optimize the battery management strategy, and evaluate the value of battery reuse.
[0003] Predicting the remaining useful life (RUL) of a battery plays an extremely critical role in practical applications. Electrochemical impedance spectroscopy (EIS) technology is an efficient analytical tool that can reveal the electrochemical reactions inside the battery. As the active substances in the battery are consumed and the performance decreases, the EIS curve will change accordingly. Therefore, using EIS data to predict the RUL of the battery is an effective technical approach. Typically, we use EIS data to train machine learning algorithms, such as neural networks or support vector machines, to learn and predict the changing trends of the battery RUL.
[0004] Traditional machine learning models usually rely on a single model to process the entire dataset, which often does not work well when dealing with complex and highly heterogeneous data. Such models have limited generalization capabilities, are prone to overfitting, and are not efficient when dealing with large-scale data. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a battery RUL prediction method based on the SMOE model and EIS to improve the accuracy of battery RUL prediction.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: A battery RUL prediction method based on the SMOE model and EIS includes the following steps: S1. Test and collect battery EIS and RUL data; S2. Construct a SMOE model based on a multi-head attention mechanism. The SMOE model based on a multi-head attention mechanism includes three expert sub-models, a gated network model, and three attention mechanism calculation modules. S3, training and verifying the SMOE model based on the multi-head attention mechanism; S4. Predict the battery RUL. After the SMOE model based on the multi-head attention mechanism is trained, deploy the SMOE model based on the multi-head attention mechanism on the premise that the SMOE model based on the multi-head attention mechanism works well, and use the deployed SMOE model based on the multi-head attention mechanism to predict the battery RUL.
[0007] Furthermore, the step S1 is specifically as follows: testing and collecting battery EIS data by an impedance spectrum tester, and the frequency range of the collection is 10 -2 Hz-10 5 Hz, the collected battery EIS data includes real battery EIS data real_1-real_m and imaginary battery EIS data imaginary_1-imaginary_m, m is the frequency number of battery EIS data, and SOH data is collected through the battery management system.
[0008] Furthermore, the expert sub-models are all MLP models, and each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives the input battery EIS data, the dimension of the expert sub-model middle layer is 32, and the activation function is the relu function. The expert sub-model output layer outputs a 32-dimensional vector representing the features of the input battery EIS data, and the activation function is linear.
[0009] Furthermore, the gated network model adopts a multi-layer perception model, and the gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives the input battery EIS data, the dimension of the gated network model middle layer is 32, the activation function is the relu function, and the gated network model output layer outputs a 3-dimensional vector, the activation function is softmax, and the 3-dimensional vectors correspond to the weights of the three expert sub-models respectively and the sum of the weights of the three expert sub-models is 1.
[0010] Furthermore, the query vector, key vector and value vector in the attention mechanism module are all 32-dimensional vectors, and the attention output is also a 32-dimensional vector.
[0011] Furthermore, the calculation steps of the attention mechanism calculation module are: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score iis the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)=
[0012] Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score i is the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output:
[0013] Among them, attention_output is the attention output, attention_weight i is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
[0014] Furthermore, the outputs of the three expert sub-models are output_1, output_2, and output_3, respectively. Output_1, output_2, and output_3 are all high-dimensional vectors, representing the feature representation of each sub-expert model for the input data; The three gating network models independently calculate the weight of each expert sub-model. The output of each gating network model is a 3D vector, which corresponds to the weights of the three expert sub-models respectively. The sum of the weights of the three expert sub-models is 1. The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability. Then, two weights corresponding to the two selected expert sub-models are selected from the 3D vector output by the gating network model. The outputs of the two selected expert sub-models are weighted summed with the weights corresponding to the two selected expert sub-models to obtain a comprehensive feature representation. After that, the comprehensive feature representation is connected to three attention mechanism calculation modules respectively. Each attention mechanism calculation module calculates the attention score, the attention weight and the weighted value, and outputs the calculation result, which is called the attention output. Then, the attention outputs of the three attention mechanism calculation modules are concatenated to obtain the attention output vector; Afterwards, the concatenated attention output vector is connected to a fully connected layer; Finally, the fully connected layer is connected to a linear regression layer to predict the battery RUL.
[0015] Furthermore, the step S3 is specifically as follows: implementing the SMOE model code based on the multi-head attention mechanism on the Keras platform and training, with the number of training epochs=100 and the number of batch samples batch_size=32; the optimizer for model training is the adam algorithm, and the loss function is the mean square error; The training data is divided into a training set and a test set in a ratio of 8:2. The model is trained on the training set and verified on the test set.
[0016] Compared with the prior art, the present invention has the following advantages and effects: 1. The RUL prediction method of the SMOE model based on the multi-head attention mechanism of the present invention takes advantage of its structural advantages, can more effectively process battery EIS data, and provide accurate and efficient RUL prediction, which is of great significance for battery health management; 2. The present invention adopts a combination of multiple expert models. SMOE allows different experts to learn specific aspects of the data. This multi-expert model can improve the accuracy of prediction; 3. The present invention uses a sparse selection mechanism to activate only part of the expert network, which significantly reduces the amount of calculation and improves the training and prediction efficiency of the model; 4. The present invention adopts a multi-head attention mechanism, each head can learn different features of the input sequence, which helps the model capture more complex and richer information, which is very effective in improving the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of a battery RUL prediction method based on the SMOE model and EIS of the present invention.
[0018] Figure 2 It is a structural diagram of a battery RUL prediction model based on the SMOE model and EIS of the present invention.
[0019] Figure 3 It is a structural diagram of the expert sub-model of the present invention.
[0020] Figure 4 It is a structural diagram of the gated network model of the present invention.
[0021] Figure 5 It is a structural diagram of the attention mechanism of the present invention.
[0022] Figure 6 This is a sample of battery EIS and RUL data of an embodiment of the present invention.
[0023] Figure 7 This is an example of the battery RUL prediction result of an embodiment of the present invention.
[0024] Figure 8 It is a bar chart of the battery RUL prediction results according to an embodiment of the present invention. DETAILED DESCRIPTION
[0025] In order to elaborate on the technical scheme adopted by the present invention to achieve the predetermined technical purpose, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments, and the technical means or technical features in the embodiments of the present invention can be replaced without paying creative work. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.
[0026] like Figure 1 As shown, a battery RUL prediction method based on the SMOE model and EIS of the present invention comprises the following steps: S1. Test and collect battery EIS and RUL data.
[0027] The battery EIS data was tested and collected by impedance spectroscopy test instrument, and the frequency range of the data was 10 -2 Hz-10 5 Hz, the collected battery EIS data includes real battery EIS data real_1-real_m and imaginary battery EIS data imaginary_1-imaginary_m, m is the number of frequencies of battery EIS data, and the number of frequencies is generally 5-60. SOH data is collected through the battery management system.
[0028] S2. Construct a SMOE model based on the multi-head attention mechanism, such as Figure 2 As shown in the figure, the SMOE model based on the multi-head attention mechanism contains three expert sub-models, a gating network model and three attention mechanism calculation modules.
[0029] like Figure 3As shown, the expert sub-models are all MLP models. Each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives the input battery EIS data, the dimension of the expert sub-model middle layer is 32, and the activation function is the relu function. The expert sub-model output layer outputs a 32-dimensional vector representing the features of the input battery EIS data, and the activation function is linear.
[0030] like Figure 4 As shown, the gated network model adopts a multi-layer perception model. The gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives the input battery EIS data, the dimension of the gated network model middle layer is 32, the activation function is the relu function, and the gated network model output layer outputs a 3D vector. The activation function is softmax. The 3D vectors correspond to the weights of the three expert sub-models and the sum of the weights of the three expert sub-models is 1.
[0031] The query vector, key vector, and value vector in the attention mechanism module are all 32-dimensional vectors, and the attention output is also a 32-dimensional vector.
[0032] like Figure 5 As shown, the calculation steps of the attention mechanism calculation module are: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score i is the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)=
[0033] Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score iis the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output:
[0034] Among them, attention_output is the attention output, attention_weight i is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
[0035] The outputs of the three expert sub-models are output_1, output_2, and output_3. Output_1, output_2, and output_3 are all high-dimensional vectors, representing the feature representation of each sub-expert model for the input data. The three gating network models independently calculate the weight of each expert sub-model. The output of each gating network model is a 3D vector, which corresponds to the weights of the three expert sub-models respectively. The sum of the weights of the three expert sub-models is 1. The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability. Then, two weights corresponding to the two selected expert sub-models are selected from the 3D vector output by the gating network model. The outputs of the two selected expert sub-models are weighted summed with the weights corresponding to the two selected expert sub-models to obtain a comprehensive feature representation. After that, the comprehensive feature representation is connected to three attention mechanism calculation modules respectively. Each attention mechanism calculation module calculates the attention score, the attention weight and the weighted value, and outputs the calculation result, which is called the attention output. Then, the attention outputs of the three attention mechanism calculation modules are concatenated to obtain the attention output vector; Afterwards, the concatenated attention output vector is connected to a fully connected layer; Finally, the fully connected layer is connected to a linear regression layer to predict the battery RUL.
[0036] S3. Train and verify the SMOE model based on the multi-head attention mechanism.
[0037] The SMOE model code based on the multi-head attention mechanism is implemented and trained on the Keras platform. The number of training epochs=100 and the number of batch samples batch_size=32 are used. The optimizer for model training is the adam algorithm, and the loss function is the mean square error.
[0038] The training data is divided into a training set and a test set in a ratio of 8:2. The model is trained on the training set and verified on the test set.
[0039] S4. Predict the battery RUL. After the SMOE model based on the multi-head attention mechanism is trained, deploy the SMOE model based on the multi-head attention mechanism on the premise that the SMOE model based on the multi-head attention mechanism works well, and use the deployed SMOE model based on the multi-head attention mechanism to predict the battery RUL.
[0040] The present invention tests the effect on a public data set and uses the impedance data therein to predict the battery RUL. There are 525 battery EIS and RUL data. Figure 6 As shown, EIS has 120 features, the feature numbers are f1 to f120, the first 60 features are the real data of the impedance data corresponding to the frequency from high to low, and the last 60 features are the imaginary data of the impedance data corresponding to the frequency from high to low.
[0041] like Figure 7 As shown, the mean of the test set of battery cycle number is 149.2952, the standard deviation is 105.1953, the mean of the prediction set is 149.1739, and the standard deviation is 105.1382, which shows that the data distribution of the prediction results is basically consistent with the data distribution of the real data.
[0042] The root mean square error between the prediction set and the test set of battery RUL is 2.7578, indicating that the prediction effect of the model is good.
[0043] like Figure 8 As shown in the figure, the solid line represents the true value, the dotted line represents the predicted value, and the solid square represents the error value. The numbers represent the true values. It can be seen from the figure that the prediction effect is acceptable and can be further optimized.
[0044] The RUL prediction method of the SMOE model based on the multi-head attention mechanism of the present invention utilizes its structural advantages, can process battery EIS data more effectively, and provide accurate and efficient RUL prediction, which is of great significance for battery health management; the present invention adopts a combination of multiple expert models, and SMOE allows different experts to learn specific aspects of the data. This multi-expert model can improve the accuracy of the prediction; the present invention uses a sparse selection mechanism to activate only part of the expert network, which significantly reduces the amount of calculation and improves the training and prediction efficiency of the model; the present invention adopts a multi-head attention mechanism, and each head can learn different features of the input sequence, which helps the model capture more complex and richer information, which is very effective in improving the prediction accuracy of the model.
[0045] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A battery RUL prediction method based on SMOE model and EIS, characterized in that The following steps are involved: S1. Test and collect battery EIS and RUL data; S2. Construct a SMOE model based on a multi-head attention mechanism. The SMOE model based on a multi-head attention mechanism includes three expert sub-models, a gated network model, and three attention mechanism calculation modules. S3, training and verifying the SMOE model based on the multi-head attention mechanism; S4. Predict the battery RUL. After the SMOE model based on the multi-head attention mechanism is trained, deploy the SMOE model based on the multi-head attention mechanism on the premise that the SMOE model based on the multi-head attention mechanism works well, and use the deployed SMOE model based on the multi-head attention mechanism to predict the battery RUL.
2. A battery RUL prediction method based on SMOE model and EIS according to claim 1, characterized in that: The step S1 specifically includes: testing and collecting battery EIS data through an impedance spectrum tester, and the collected frequency range is 10 -2 Hz-10 5 Hz, the collected battery EIS data includes real battery EIS data real_1-real_m and imaginary battery EIS data imaginary_1-imaginary_m, m is the frequency number of battery EIS data, and SOH data is collected through the battery management system.
3. The battery RUL prediction method based on the SMOE model and EIS according to claim 1, characterized in that: The expert sub-models are all MLP models. Each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives the input battery EIS data, the dimension of the expert sub-model middle layer is 32, and the activation function is the relu function. The expert sub-model output layer outputs a 32-dimensional vector representing the features of the input battery EIS data, and the activation function is linear.
4. A battery RUL prediction method based on SMOE model and EIS according to claim 3, characterized in that: The gated network model adopts a multi-layer perception model. The gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives the input battery EIS data, the dimension of the gated network model middle layer is 32, the activation function is the relu function, and the gated network model output layer outputs a 3-dimensional vector. The activation function is softmax. The 3-dimensional vectors correspond to the weights of the three expert sub-models respectively, and the sum of the weights of the three expert sub-models is 1.
5. A battery RUL prediction method based on SMOE model and EIS according to claim 4, characterized in that: The query vector, key vector and value vector in the attention mechanism module are all 32-dimensional vectors, and the attention output is also a 32-dimensional vector.
6. A battery RUL prediction method based on SMOE model and EIS according to claim 5, characterized in that: The calculation steps of the attention mechanism calculation module are: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score i is the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)= ; Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score i is the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output: ; Among them, attention_output is the attention output, attention_weight i is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
7. A battery RUL prediction method based on SMOE model and EIS according to claim 6, characterized in that: The outputs of the three expert sub-models are output_1, output_2, and output_3. Output_1, output_2, and output_3 are all high-dimensional vectors, representing the feature representation of each sub-expert model for the input data. The three gating network models independently calculate the weight of each expert sub-model. The output of each gating network model is a 3D vector, which corresponds to the weights of the three expert sub-models respectively. The sum of the weights of the three expert sub-models is 1. The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability. Then, two weights corresponding to the two selected expert sub-models are selected from the 3D vector output by the gating network model. The outputs of the two selected expert sub-models are weighted summed with the weights corresponding to the two selected expert sub-models to obtain a comprehensive feature representation. After that, the comprehensive feature representation is connected to three attention mechanism calculation modules respectively. Each attention mechanism calculation module calculates the attention score, the attention weight and the weighted value, and outputs the calculation result, which is called the attention output. Then, the attention outputs of the three attention mechanism calculation modules are concatenated to obtain the attention output vector; Afterwards, the concatenated attention output vector is connected to a fully connected layer; Finally, the fully connected layer is connected to a linear regression layer to predict the battery RUL.
8. The battery RUL prediction method based on the SMOE model and EIS according to claim 1, characterized in that: The step S3 is specifically as follows: implementing the SMOE model code based on the multi-head attention mechanism on the Keras platform and training, with the number of training epochs=100 and the number of batch samples batch_size=32; the optimizer for model training is the adam algorithm, and the loss function is the mean square error; The training data is divided into a training set and a test set in a ratio of 8:
2. The model is trained on the training set and verified on the test set.
Citation Information
Patent Citations
Method for predicting residual capacity of battery based on ELM (Extreme Learning Model) and battery impedance spectrum data
CN115902664A
Hybrid expert model and retrieval enhancement-based time series data prediction method and application
CN118585714A
Method for predicting RUL based on MOE model and current resistance impedance of battery
CN118625152A
Expert selection method and device, electronic equipment and storage medium
CN118690782A
Cited By
Method for predicting residual life of lithium battery of electric vehicle based on variational mode decomposition hybrid model
CN120468696A
Lithium battery residual life prediction method and system and terminal equipment
CN121454336A
Lithium battery remaining life prediction method, system and terminal device
CN121454336B
