Battery RUL prediction method based on attention mechanism SMOE model
Through the SMOE model based on attention mechanism, the problems of insufficient generalization ability and overfitting in battery RUL prediction are solved, and higher prediction accuracy and efficiency are achieved, improving the prediction accuracy of battery RUL.
Patent Information
- Application Number
- CN202411412357.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional machine learning models have limited generalization capabilities when processing complex and heterogeneous battery data, which can easily lead to overfitting and low processing efficiency, making it difficult to accurately predict the remaining service life of the battery (RUL).
Using the SMOE model based on attention mechanism, the prediction accuracy of battery RUL is improved by building multiple expert sub-models and gated network models, combining the attention mechanism calculation layer, and training and verification models are trained and checked.
The expression ability and prediction accuracy of the model are improved, the risk of overfitting is reduced, the robustness and generalization ability is improved, and the computational volume is significantly reduced through the sparse selection mechanism, which improves training and prediction efficiency.
Smart Images

Figure CN118914908A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a battery RUL prediction method, in particular to a battery RUL prediction method based on an attention mechanism SMOE model, belonging to the technical field of battery detection. Background Art
[0002] Battery RUL (Remaining Useful Life) is the remaining life of the battery, which is an important indicator for evaluating the health status of the battery. Common battery RULs include the number of remaining battery cycles. Battery RUL is an important indicator of the battery, which is mainly reflected in several aspects: evaluating the health status of the battery, guiding the use and maintenance of the battery, warning of battery failure, evaluating the battery capacity reduction, optimizing the battery management strategy and evaluating the secondary utilization value of the battery. Predicting battery RUL has high application value.
[0003] The remaining useful life (RUL) of a battery is closely related to operating data such as current, resistance, and impedance, making it a hot research area to use these parameters to estimate the RUL of a battery. Researchers usually use data on current, resistance, and impedance to train machine learning algorithms, such as neural networks or support vector machines, to capture the changing trends of battery RUL and make predictions.
[0004] However, traditional machine learning models often rely on a single model to process all data, which is inadequate when faced with complex and highly heterogeneous data sets. The generalization ability of such a single model is limited, which can easily lead to overfitting. At the same time, when processing large amounts of data, the processing efficiency of a single model is relatively low. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a battery RUL prediction method based on the attention mechanism SMOE model to improve the prediction accuracy of battery RUL.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: A battery RUL prediction method based on the attention mechanism SMOE model includes the following steps: S1. Test and collect battery current, resistance, impedance and RUL data; S2. Construct a SMOE model based on the attention mechanism. The SMOE model based on the attention mechanism includes multiple expert sub-models, a gating network model and an attention mechanism calculation layer; S3, training and verifying the SMOE model based on the attention mechanism; S4. Predict the battery RUL. After the SMOE model based on the attention mechanism is trained, the trained SMOE model is deployed on the premise that the model effect is good, and the deployed SMOE is used to predict the battery RUL.
[0007] Furthermore, the step S1 is specifically as follows: 1.1. Prepare equipment and materials. The equipment includes data acquisition equipment, data storage media and battery samples. The data acquisition equipment includes current sensors, AC impedance testers and signal recorders. 1.2. Current data collection: insert a collection resistor into the battery charging and discharging circuit, and use a current sensor to measure and record the current value in real time; 1.3. Resistance data collection: Under different charge and discharge conditions, measure the voltage across the battery and the current passing through it, and calculate the resistance according to the resistance formula R = U / I; 1.4. Impedance data collection: use an impedance tester to test the impedance value of the battery at different frequencies to obtain the impedance frequency spectrum; 1.5. RUL data collection, statistics of historical battery cycle data.
[0008] Furthermore, in step S2, the number of expert sub-models is 3, each expert sub-model is an MLP model, and each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives input battery current, resistance, impedance and RUL data, the dimension of the expert sub-model middle layer is 32, the activation function is the relu function, and the expert sub-model output layer outputs a 32-dimensional vector, and the activation function is the relu function.
[0009] Furthermore, in step S2, the gated network model adopts a multi-layer perceptron model, and the gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives input battery current, resistance, impedance and RUL data, the dimension of the gated network model middle layer is 32, the activation function is the relu function, the gated network model output layer outputs a 3D vector, the activation function is softmax, and the 3D vectors output by the gated network model output layer correspond to the weights of the three expert sub-models, and the sum of the weights of the three expert sub-models is 1.
[0010] Furthermore, the outputs of the three expert sub-models are output1, output2, and output3, respectively. Output1, output2, and output3 are all high-dimensional vectors, representing the feature representation of each sub-expert model for the input data. The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability. Then, two weights corresponding to the two selected expert sub-models are selected from the 3D vector output by the gating network model. The outputs of the two selected expert sub-models are weighted summed with the weights corresponding to the two selected expert sub-models to obtain a comprehensive feature representation. Afterwards, the comprehensive feature representation is connected to an attention mechanism computation layer.
[0011] Furthermore, the query vector, key vector and value vector in the attention mechanism calculation layer are all 32-dimensional vectors, and the output of the attention mechanism calculation layer is a 32-dimensional vector.
[0012] Furthermore, the calculation process of the attention mechanism calculation layer is: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score i is the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)=
[0013] Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score i is the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output:
[0014] Among them, attention_output is the attention output, attention_weighti is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
[0015] Furthermore, the step S3 is specifically as follows: implementing the SMOE model code based on the attention mechanism on the Keras platform and training it, with the number of training epochs=100, the number of batch samples batch_size=32, the optimizer for model training being the adam algorithm, and the loss function being the mean square error; the training data being divided into a training set and a test set in a ratio of 8:2, the model being trained on the training set, and the model being verified on the test set.
[0016] Compared with the prior art, the present invention has the following advantages and effects: 1. The SMOE model based on the attention mechanism of the present invention can improve the expression ability and prediction accuracy of the model when predicting the battery RUL, improve the robustness and generalization ability of the overall model, and reduce the risk of overfitting; 2. The present invention adopts the SMOE model, which allows different experts to learn specific aspects of the data. For example, some experts may focus on current changes, while other experts may focus on resistance patterns. This multi-expert model can improve the accuracy of predictions. 3. The present invention uses a sparse selection mechanism to activate only part of the expert network, which significantly reduces the amount of calculation and improves the training and prediction efficiency of the model; 4. The attention mechanism of the present invention enables the SMOE model to better capture the important parts of the features, which is very effective in improving the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of a battery RUL prediction method based on the attention mechanism SMOE model of the present invention.
[0018] Figure 2 It is a structural diagram of a battery RUL prediction model based on the attention mechanism SMOE model of the present invention.
[0019] Figure 3 It is a structural diagram of the expert sub-model of the present invention.
[0020] Figure 4 It is a structural diagram of the gated network model of the present invention.
[0021] Figure 5 is a schematic diagram of a collection of input data of an embodiment of the present invention Figure 1 .
[0022] Figure 6 is a schematic diagram of a collection of input data of an embodiment of the present invention Figure 2 .
[0023] Figure 7 Schematic diagram of prediction results according to an embodiment of the present invention.
[0024] Figure 8 It is a bar chart of the prediction results of the embodiment of the present invention. DETAILED DESCRIPTION
[0025] In order to elaborate on the technical scheme adopted by the present invention to achieve the predetermined technical purpose, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments, and the technical means or technical features in the embodiments of the present invention can be replaced without paying creative work. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.
[0026] like Figure 1 As shown, a battery RUL prediction method based on the attention mechanism SMOE model of the present invention comprises the following steps: S1. Test and collect battery current, resistance, impedance and RUL data.
[0027] 1.1. Prepare equipment and materials. The equipment includes data acquisition equipment, data storage media and battery samples. The data acquisition equipment includes current sensor, AC impedance tester and signal recorder.
[0028] 1.2. Current data collection: insert a collection resistor into the battery charging and discharging circuit, and use a current sensor to measure and record the current value in real time.
[0029] 1.3. Resistance data collection: Under different charge and discharge conditions, measure the voltage across the battery and the current passing through, and calculate the resistance according to the resistance formula R = U / I.
[0030] 1.4. Impedance data collection: Use an impedance tester to test the impedance value of the battery at different frequencies to obtain the impedance frequency spectrum.
[0031] 1.5. RUL data collection, statistics of historical battery cycle data.
[0032] S2. Construct a SMOE model based on the attention mechanism, such as Figure 2 As shown in the figure, the SMOE model based on the attention mechanism includes multiple expert sub-models, a gating network model and an attention mechanism calculation layer.
[0033] like Figure 3As shown, the number of expert sub-models is 3, each expert sub-model is an MLP model, and each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives input battery current, resistance, impedance and RUL data, the dimension of the expert sub-model middle layer is 32, and the activation function is the relu function. The expert sub-model output layer outputs a 32-dimensional vector, and the activation function is the relu function.
[0034] like Figure 4 As shown in the figure, the gated network model adopts a multi-layer perceptron model. The gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives the input battery current, resistance, impedance and RUL data. The dimension of the gated network model middle layer is 32, and the activation function is the relu function. The gated network model output layer outputs a 3D vector, and the activation function is softmax. The 3D vectors output by the gated network model output layer correspond to the weights of the three expert sub-models, and the sum of the weights of the three expert sub-models is 1.
[0035] The outputs of the three expert sub-models are output1, output2 and output3, respectively. Output1, output2 and output3 are all high-dimensional vectors, representing the feature representation of the input data by each sub-expert model.
[0036] The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability, and then the two weights corresponding to the two selected expert sub-models in the 3D vector output by the gating network model are selected. The outputs of the two selected expert sub-models and the weights corresponding to the two selected expert sub-models are weighted summed to obtain a comprehensive feature representation.
[0037] Afterwards, the comprehensive feature representation is connected to an attention mechanism computation layer.
[0038] The query vector, key vector, and value vector in the attention mechanism calculation layer are all 32-dimensional vectors, and the output of the attention mechanism calculation layer is a 32-dimensional vector.
[0039] The calculation process of the attention mechanism calculation layer is: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score iis the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)=
[0040] Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score i is the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output:
[0041] Among them, attention_output is the attention output, attention_weight i is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
[0042] After the attention output is a linear regression layer, the activation function is a linear function, and the output value is the battery RUL. The loss function of the model is the mean square error (MSE); the optimizer is adam; the evaluation metric is the mean absolute error (MAE), which is the sum of the absolute values of the difference between the target value and the predicted value.
[0043] S3. Train and verify the SMOE model based on the attention mechanism.
[0044] The SMOE model code based on the attention mechanism is implemented and trained on the Keras platform. The number of training epochs=100, the number of batch samples batch_size=32, the optimizer for model training is the adam algorithm, and the loss function is the mean square error; the training data is divided into a training set and a test set in a ratio of 8:2. The model is trained on the training set and verified on the test set.
[0045] S4. Predict the battery RUL. After the SMOE model based on the attention mechanism is trained, the trained SMOE model is deployed on the premise that the model effect is good, and the deployed SMOE is used to predict the battery RUL.
[0046] The present invention tests the effect on the impedance state (type=impedance) data of B0018 battery in NASA battery data set, and uses the battery current, resistance and impedance data to predict the battery RUL. The data set is divided into a training set and a test set in a ratio of 8:2, the SMOE model is trained on the training set, and the effect is tested on the test set. Figure 5 and Figure 6 As shown, the test data has 7 features and 1 target. The 7 features include sensor branch current, battery branch current, ratio of sensor branch current to battery branch current, battery impedance calculated from raw data, calibrated and smoothed battery impedance, estimated electrolyte resistance, estimated charging transfer resistance, and 1 target is the number of cycles, i.e. RUL. Among them, the sensor branch current, battery branch current, ratio of sensor branch current to battery branch current, battery impedance calculated from raw data, and calibrated and smoothed battery impedance are divided into real data and imaginary data.
[0047] like Figure 7 As shown, the mean of the test set is 139.7838, the standard deviation is 96.9046, the mean of the prediction set is 139.8015, and the standard deviation is 97.0955, which shows that the data distribution of the prediction results is basically consistent with the data distribution of the real data.
[0048] The root mean square error between the prediction set and the test set is 2.3527, and the model prediction effect is good.
[0049] like Figure 8 As shown in the figure, the dotted line represents the true value, the solid line represents the predicted value, and the solid square represents the error value. The numbers represent the true values. It can be seen from the figure that the prediction effect is good.
[0050] The SMOE model based on the attention mechanism of the present invention can improve the expression ability and prediction accuracy of the model when predicting the RUL of the battery, improve the robustness and generalization ability of the overall model, and reduce the risk of overfitting; the present invention adopts the SMOE model to allow different experts to learn specific aspects of the data. For example, some experts may focus on current changes, while other experts may focus on resistance patterns. This multi-expert model can improve the accuracy of prediction; the present invention only activates part of the expert network through a sparse selection mechanism, which significantly reduces the amount of calculation and improves the training and prediction efficiency of the model; the attention mechanism of the present invention enables the SMOE model to better capture the important parts of the features, which is very effective in improving the prediction accuracy of the model.
[0051] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A battery RUL prediction method based on the attention mechanism SMOE model, characterized in that The following steps are involved: S1. Test and collect battery current, resistance, impedance and RUL data; S2. Construct a SMOE model based on the attention mechanism. The SMOE model based on the attention mechanism includes multiple expert sub-models, a gating network model and an attention mechanism calculation layer; S3, training and verifying the SMOE model based on the attention mechanism; S4. Predict the battery RUL. After the SMOE model based on the attention mechanism is trained, the trained SMOE model is deployed on the premise that the model effect is good, and the deployed SMOE is used to predict the battery RUL.
2. According to claim 1, a battery RUL prediction method based on the attention mechanism SMOE model is characterized in that: The step S1 is specifically as follows: 1.
1. Prepare equipment and materials. The equipment includes data acquisition equipment, data storage media and battery samples. The data acquisition equipment includes current sensors, AC impedance testers and signal recorders. 1.
2. Current data collection: insert a collection resistor into the battery charging and discharging circuit, and use a current sensor to measure and record the current value in real time; 1.
3. Resistance data collection: Under different charge and discharge conditions, measure the voltage across the battery and the current passing through it, and calculate the resistance according to the resistance formula R = U / I; 1.
4. Impedance data collection: use an impedance tester to test the impedance value of the battery at different frequencies to obtain the impedance frequency spectrum; 1.
5. RUL data collection, statistics of historical battery cycle data.
3. According to claim 1, a battery RUL prediction method based on the attention mechanism SMOE model is characterized in that: In step S2, the number of expert sub-models is 3, each expert sub-model is an MLP model, and each expert sub-model includes an expert sub-model input layer, an expert sub-model middle layer and an expert sub-model output layer. The expert sub-model input layer receives input battery current, resistance, impedance and RUL data, the dimension of the expert sub-model middle layer is 32, the activation function is the relu function, and the expert sub-model output layer outputs a 32-dimensional vector, and the activation function is the relu function.
4. A battery RUL prediction method based on the attention mechanism SMOE model according to claim 3, characterized in that: In the step S2, the gated network model adopts a multi-layer perceptron model, and the gated network model includes a gated network model input layer, a gated network model middle layer and a gated network model output layer. The gated network model input layer receives input battery current, resistance, impedance and RUL data, the dimension of the gated network model middle layer is 32, the activation function is the relu function, the gated network model output layer outputs a 3D vector, the activation function is softmax, and the 3D vectors output by the gated network model output layer correspond to the weights of the three expert sub-models respectively, and the sum of the weights of the three expert sub-models is 1.
5. A battery RUL prediction method based on the attention mechanism SMOE model according to claim 4, characterized in that: The outputs of the three expert sub-models are output1, output2, and output3, respectively. Output1, output2, and output3 are all high-dimensional vectors, representing the feature representation of each sub-expert model for the input data. The top-k strategy is used to select the outputs of the two expert sub-models with the highest probability. Then, two weights corresponding to the two selected expert sub-models are selected from the 3D vector output by the gating network model. The outputs of the two selected expert sub-models are weighted summed with the weights corresponding to the two selected expert sub-models to obtain a comprehensive feature representation. Afterwards, the comprehensive feature representation is connected to an attention mechanism computation layer.
6. A battery RUL prediction method based on the attention mechanism SMOE model according to claim 5, characterized in that: The query vector, key vector and value vector in the attention mechanism calculation layer are all 32-dimensional vectors, and the output of the attention mechanism calculation layer is a 32-dimensional vector.
7. A battery RUL prediction method based on the attention mechanism SMOE model according to claim 6, characterized in that: The calculation process of the attention mechanism calculation layer is: Step 1: Calculate the attention score; First, calculate the similarity between each query vector and all key vectors: attention_score i =query·key i Among them, attention_score i is the i-th attention score, query is the query vector, key i is the i-th key vector; Step 2: Calculate attention weight; The obtained attention score is converted into a probability form through the softmax function, which is called the attention weight: attention_weights=softmax(attention_scores)= ; Among them, attention_weights is the attention weight, attention_scores is the attention score, softmax() is the softmax function, the softmax function ensures that all weights add up to 1, and attention_score i is the i-th attention score, attention_score j is the jth attention score; Step 3: Calculate the weighted value Multiply each value by its corresponding attention weight and add the results to get the final attention output: ; Among them, attention_output is the attention output, attention_weight i is the i-th attention weight, value i is the i-th value vector; The comprehensive features are used as the input data of the attention mechanism calculation layer to obtain the attention output.
8. The battery RUL prediction method based on the attention mechanism SMOE model according to claim 1 is characterized in that: The step S3 is specifically as follows: implementing the SMOE model code based on the attention mechanism on the Keras platform and training it, with the number of training epochs=100, the number of batch samples batch_size=32, the optimizer for model training using the adam algorithm, and the loss function being the mean square error; the training data is divided into a training set and a test set in a ratio of 8:2, the model is trained on the training set, and the model is verified on the test set.
Citation Information
Patent Citations
Event identification and classification method based on bidirectional recurrent neural network and attention mechanism
CN113946677A
Method for predicting RUL by using battery EIS based on multi-model combination
CN117031306A
DETR improved model-based sparse attention target detection method
CN117152416A
Image enhancement method and system based on multi-scale sparse Transform network
CN117274115A
Battery RUL prediction method based on Transform combination model
CN117347866A
Cited By
Lithium ion battery SOH-RUL joint estimation method based on Dozerformer model
CN119758097A
