Power battery low-temperature SOC estimation method based on improved Transform and adaptive transfer learning
Through the improved Transformer model and adaptive migration learning training framework, the problem of insufficient data samples in the low -temperature environment of power batteries and unusable information in the low temperature environment of power batteries, and achieved higher SOC estimation accuracy, which has important application value. Essence
Patent Information
- Application Number
- CN202411429346.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2025-01-28
AI Technical Summary
The prior art has problems such as insufficient data samples and unused implicit information in SOC estimation in low temperature environments of power batteries, resulting in insufficient accuracy of SOC estimation.
Adopting the improved Transformer model (Enformer) combined with the adaptive migration learning training framework, through multi -layer residual attention mechanisms, mode dynamic decomposition, and two -way long -term memory networks to learn valuable migration characteristics and realize the positive migration of knowledge.
It improves the accuracy of SOC estimation in the low temperature environment of power batteries, overcomes the problems of insufficient data samples and unused information, and has important application value in terms of low temperature SOC estimation of power battery and battery health management.
Smart Images
Figure CN119377598A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to power battery SOC estimation and deep learning, and in particular to a power battery low-temperature SOC estimation method based on improved Transformer and adaptive transfer learning. Background Art
[0002] Environmental pollution and the global energy crisis have accelerated the transition from traditional energy to clean energy. Especially in the automotive industry, there is an urgent need to replace traditional fossil fuels with power batteries. Lithium-ion batteries play an important role in the energy supply of electric vehicles due to their high energy conversion efficiency and long life. Battery overheating or abnormal voltage is usually associated with inaccurate state of charge (SOC) estimation. Accurate SOC estimation can provide early warnings of potential safety issues, thereby facilitating proactive maintenance. However, SOC cannot be measured directly.
[0003] Traditional SOC estimation methods mainly include open circuit voltage method, electrochemical model method, and method based on equivalent circuit model. However, these methods have large estimation errors due to their dependence on electrochemical reaction mechanism, battery material characteristics and driving conditions. At the same time, they also require in-depth understanding of domain knowledge, which leads to high costs. Machine learning methods such as Gaussian process regression and random forest regression can learn the complex relationship between battery measurement sequence and SOC, effectively overcoming these problems. However, traditional machine learning methods cannot fully capture complex potential information due to the limitations of feature extraction.
[0004] Deep learning methods can effectively capture complex nonlinear information. Methods based on convolutional neural networks (CNN), long short-term memory networks (LSTM), and a combination of the two are used for SOC estimation. Since Transformer has advantages in learning long-term series information, it is also used to improve the SOC estimation effect. However, the limited training data in the target driving style poses a challenge to the effectiveness of deep learning methods. During model training, as the network depth increases, insufficient samples will further aggravate the gradient vanishing. Existing methods also ignore the learning of hidden information such as low-temperature driving environments. Therefore, it is urgent to design an effective model to improve the SOC estimation effect.
[0005] Transfer learning based methods have been introduced to address the small sample problem in estimating SOC. Frequent speed changes due to urban driving conditions (such as traffic lights and congestion) can cause significant fluctuations in battery current and voltage. In contrast, vehicle speed remains relatively stable during highway driving. This leads to significant differences in data distribution between the source and target domains, resulting in negative transfer. A method using maximum mean dispersion (MMD) has been proposed to alleviate the problem of negative transfer in SOC estimation. However, MMD is very sensitive to changes in data distribution and it is difficult to effectively capture dynamic and potentially effective features. Therefore, it is necessary to design an adaptive transfer learning framework to promote the positive transfer of transfer knowledge to improve the accuracy of SOC estimation. Summary of the invention
[0006] In order to solve the shortcomings and deficiencies in the prior art, the purpose of the present invention is to propose a low-temperature SOC estimation method for a power battery based on an improved Transformer and adaptive transfer learning;
[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:
[0008] S1. Import the charging and discharging data of the power battery in a low temperature environment, pre-process the data, and determine the source domain and target domain according to the driving mode of the car;
[0009] S2. Design an improved Transformer (Enformer) model, which mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network;
[0010] S3. Design an adaptive transfer learning training framework, using the Enformer designed in step S2 as the basic model structure to learn valuable transfer features to achieve positive knowledge transfer;
[0011] S4. According to the model obtained in step S3, the power battery of the target driving mode in a low temperature environment is tested to estimate the SOC.
[0012] In the above step S1, the charging and discharging data of the power battery in a low temperature environment is imported, the data is preprocessed, and the source domain and the target domain are determined according to the driving mode of the car, specifically including:
[0013] S11, the voltage, current, temperature measurement data characteristics [U t ,I t ,T t ](t=1,2,..,l) constitutes the training sample X, and calculates the power battery state SOC as the prediction label Y to obtain the time series for SOC estimation;
[0014] S12, delete the data whose SOC value is not in the range of 0 to 1, delete the duplicate data and the data whose data value is NULL, and standardize the data to achieve dimensionless processing;
[0015] S13, split the samples in a sliding time window manner, set the time window length according to the relevant literature on battery SOC estimation and experimental cross-validation, that is, the sample length len is the data monitored for 45 seconds;
[0016] S14. Four driving modes, including high-speed aggressive driving (US06), high-speed fuel economy driving (HWFET), urban driving cycle (UDDS), and mixed urban and highway driving (LA92), were studied. The data of one driving mode was selected as the source domain, and the data of another driving mode was selected as the target domain for testing to achieve SOC estimation.
[0017] In the above step S2, the designed and improved Transformer (Enformer) model mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network, specifically including:
[0018] S21. Design a multi-layer residual attention mechanism to replace the traditional self-attention mechanism, and use multi-layer residual connections to alleviate the problem of gradient disappearance as the number of network layers increases;
[0019] S22. Design a pattern dynamic decomposition module to replace the normalization layer in the traditional Transformer, dynamically decompose the measurement feature sequence into trend information and implicit information, and improve the model's learning of implicit information in driving environments such as low temperatures. Specifically:
[0020] S221, mapping the features of the first and last parts of the sample to increase the sample length, so as to alleviate the loss of timing information caused by the truncation of the time window;
[0021] S222, use moving average pooling to change the added dimension to the original dimension to extract the trend information Z of the sample t ;
[0022] S223, use trend information to filter the original sample to obtain sample implicit information Z h ;
[0023] S224, trend information and implicit information are linearly learned respectively, and feature splicing is performed at the same time, and the output of the pattern dynamic decomposition is output pdd Send to the next module.
[0024] In the above step S3, the Enformer designed in step S2 is used as the basic model structure. When designing the adaptive transfer learning training framework, the following steps are specifically included:
[0025] S31, use the source domain data to train the Enformer model, and use the target domain data to fine-tune it to obtain fine-tuning features;
[0026] S32, using the target domain data to autonomously train the model Enformer* to obtain autonomous training features;
[0027] S33, adaptively concatenate the fine-tuning features and the autonomous training features, and then send them to the linear layer. When the maximum number of training rounds is reached, the trained model is obtained to achieve SOC estimation;
[0028] In the above step S4, according to the model obtained in step S3, the power battery of the target driving mode in two low temperature environments of -10°C and -20°C is tested to evaluate the effect of SOC estimation.
[0029] The beneficial effects of the present invention are: a method for estimating low-temperature SOC of a power battery based on an improved Transformer and adaptive transfer learning, which addresses the problem of insufficient sample size of battery data for a target driving mode and ignoring the correlation between implicit information such as low temperature and SOC estimation. Existing methods make it difficult to conduct relevant research on SOC estimation using machine learning. Therefore, the present invention achieves accurate SOC estimation by designing an improved Transformer model structure and optimizing the adaptive transfer learning training strategy, which has significant application value in low-temperature SOC estimation of power batteries and battery health management. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 It is a schematic diagram of the process of estimating the low-temperature SOC of the power battery of the present invention.
[0032] Figure 2 It is a schematic diagram of the overall framework of the power battery low-temperature SOC estimation based on improved Transformer and adaptive transfer learning in the present invention.
[0033] Figure 3 Schematic diagram of the training framework based on adaptive transfer learning of the present invention. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] Due to the lack of battery data for the target driving mode, the existing model ignores the impact of implicit information such as temperature on the battery state, and it is difficult to carry out relevant research with the help of deep learning methods. The present invention aims at the problem of battery low-temperature SOC estimation and proposes a power battery low-temperature SOC estimation method based on improved Transformer and adaptive transfer learning. The main steps include: importing power battery charge and discharge data in a low-temperature environment, preprocessing the data, and determining the source domain and target domain according to the vehicle driving mode; designing a multi-layer residual attention mechanism and a pattern dynamic decomposition method, and combining a bidirectional long short-term memory network to complete the design of an improved Transformer (Enformer) model; designing an adaptive transfer learning training framework to learn valuable transfer features to realize the forward transfer of knowledge; testing the power battery of the target driving mode in a low-temperature environment and performing SOC estimation;
[0036] A method for estimating low-temperature SOC of a power battery based on improved Transformer and adaptive transfer learning. The specific process is as follows: Figure 1 As shown, the implementation steps are as follows:
[0037] S1. Import the charging and discharging data of the power battery in a low temperature environment, pre-process the data, and determine the source domain and target domain according to the driving mode of the car;
[0038] S2. Design an improved Transformer (Enformer) model, which mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network;
[0039] S3. Design an adaptive transfer learning training framework, using the Enformer designed in step S2 as the basic model structure to learn valuable transfer features to achieve positive knowledge transfer;
[0040] S4. According to the model obtained in step S3, the power battery of the target driving mode in a low temperature environment is tested to estimate the SOC.
[0041] In the above step S1, the charging and discharging data of the power battery in a low temperature environment is imported, the data is preprocessed, and the source domain and the target domain are determined according to the driving mode of the car, specifically including:
[0042] S11, the voltage, current, temperature measurement data characteristics [Ut ,I t ,T t ](t=1,2,..,l) constitutes the training sample X, and calculates the power battery state SOC as the prediction label Y to obtain the time series for SOC estimation. The specific formula is as follows:
[0043]
[0044] Y=[y1 y2 … y t … y len ]=[SOC1 SOC2 … SOC t … SOC l ],
[0045] Where len represents the sample length;
[0046] S12, delete the data whose SOC value is not in the range of 0 to 1, delete the duplicate data and the data whose data value is NULL, and standardize the data to achieve dimensionless processing;
[0047] S13, split the samples in a sliding time window manner, set the time window length according to the relevant literature on battery SOC estimation and experimental cross-validation, that is, the sample length len is the data monitored for 45 seconds;
[0048] S14. Four driving modes, including high-speed aggressive driving (US06), high-speed fuel economy driving (HWFET), urban driving cycle (UDDS), and mixed urban and highway driving (LA92), were studied. The data of one driving mode was selected as the source domain, and the data of another driving mode was selected as the target domain for testing to achieve SOC estimation.
[0049] In the above step S2, the designed improved Transformer (Enformer) model mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network. Figure 2 As shown, specifically including:
[0050] S21. Design a multi-layer residual attention mechanism to replace the traditional self-attention mechanism, and use multi-layer residual connections to alleviate the problem of gradient disappearance as the number of network layers increases. Specifically:
[0051] S211, use linear transformation to transform the time point x t (U t ,I t ,T t ) is mapped to query vector Q, key vector K, value vector V, and the current time x t For each time point x1,x2,…,xt …,x l The attention score at(x t ) is calculated using the dot product formula:
[0052]
[0053] S212, combining attention mechanism and residual connection for feature learning. Multi-level residual attention is the first module, and its output O MCA The calculation formula is as follows:
[0054]
[0055] Modules 1 and O MCA The result of the residual connection is used as the input of module 2 for the next residual connection. Subsequent modules repeat similar calculations until the last module N, and the calculation formula is as follows:
[0056]
[0057] In the designed Enformer model structure, N = 4;
[0058] S22. Design a pattern dynamic decomposition module to replace the normalization layer in the traditional Transformer, dynamically decompose the measurement feature sequence into trend information and implicit information, and improve the model's learning of implicit information in driving environments such as low temperatures. Specifically:
[0059] S221. Map the features of the first and last parts of the sample to increase the sample length, so as to alleviate the loss of timing information caused by time window truncation. The details are as follows:
[0060] Different from the traditional method of first decomposing the pattern and then feeding it into the model for learning, the pattern dynamic decomposition module is the middle part of the model, which dynamically learns the trend and implicit information of the battery monitoring sequence. The dimension of the sample Z input to this module is [l×f], where f is the number of features; the head feature Z[head] and the tail feature Z[tail] of the sample Z are mapped with a kernel function size of k, and then spliced. The formula is as follows:
[0061] Padding(Z)=M k (Z[head])+Z[head+1:tail-1]+M k (Z[tail]),
[0062] Among them, M k () indicates that the mapping is performed with a kernel size of k, Z[head+1:tail-1] is the unchanged part in the middle of the sample, and the dimension of the sample after mapping is [(l-2+2k)×f];
[0063] S222, use moving average pooling to change the added dimension to the original dimension to extract the trend information Z of the sample t , the formula is as follows:
[0064] Z t =MAP(padding(Z)),
[0065] Where MAP stands for moving average pooling;
[0066] S223, use trend information to filter the original sample to obtain sample implicit information Z h , the formula is as follows:
[0067] Z h =ZZ t ;
[0068] S224, trend information and implicit information are linearly learned respectively, and feature concatenation is performed at the same time. The formula is as follows:
[0069] Output pdd =Lr(Z t )||Lr(Z h ),
[0070] Where Lr() represents linear mapping, || represents feature concatenation, and the output of the dynamic decomposition of the mode is Output pdd Send to the next module.
[0071] In the above step S3, the Enformer designed in step S2 is used as the basic model structure. When designing the adaptive transfer learning training framework, Figure 3 As shown, specifically including:
[0072] S31, use the source domain data to train the Enformer model, and use the target domain data to fine-tune it to obtain fine-tuning features;
[0073] S32, using the target domain data to autonomously train the model Enformer* to obtain autonomous training features;
[0074] S33, adaptively concatenate the fine-tuning features and the autonomous training features, and then send them to the linear layer. When the maximum number of training rounds is reached, the trained model is obtained to achieve SOC estimation, and MSE is used as the model loss. The formula is as follows:
[0075]
[0076] The pseudo code of the adaptive transfer training strategy based on Enformer is as follows:
[0077] Input: Source data Conventional driving pattern Limited sample data for target driving style Learning rate α.
[0078]
[0079]
[0080] In the above step S4, according to the model obtained in step S3, the power battery of the target driving mode in two low temperature environments of -10°C and -20°C is tested, and the effect of SOC estimation is evaluated using RMSE, MAE, and accuracy Acc. The formula is as follows:
[0081]
[0082] The present invention aims to solve the problem that the sample size of battery data for the target driving mode is insufficient and the correlation between implicit information such as low temperature and SOC estimation is ignored. Existing methods make it difficult to use machine learning to conduct research related to SOC estimation. Therefore, the present invention achieves accurate SOC estimation by designing and improving the Transformer model structure and optimizing the adaptive transfer learning training strategy, which has great application value in low-temperature SOC estimation of power batteries.
[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for estimating low-temperature SOC of a power battery based on improved Transformer and adaptive transfer learning, characterized in that: The specific steps are as follows: S1. Import the charging and discharging data of the power battery in a low temperature environment, pre-process the data, and determine the source domain and target domain according to the driving mode of the car; S2. Design an improved Transformer (Enformer) model, which mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network; S3. Design an adaptive transfer learning training framework, using the Enformer designed in step S2 as the basic model structure to learn valuable transfer features to achieve positive knowledge transfer; S4. According to the model obtained in step S3, the power battery of the target driving mode in a low temperature environment is tested to estimate the SOC.
2. According to claim 1, a method for estimating low-temperature SOC of a power battery based on improved Transformer and adaptive transfer learning is characterized in that: Step S1 imports the charging and discharging data of the power battery in a low temperature environment, pre-processes the data, and determines the source domain and the target domain according to the driving mode of the car, including the following steps: S11, taking the measured data such as current, voltage, temperature as features, and calculating the power battery state SOC as a prediction label, to obtain a time series for SOC estimation; S12, remove data outliers and standardize the data to achieve dimensionless processing; S13, performing sample segmentation in a time window sliding manner, where the time window length, i.e., the sample length, is obtained based on the relevant literature on battery SOC estimation and experimental cross-validation; S14. Four driving modes, including high-speed aggressive driving (US06), high-speed fuel economy driving (HWFET), urban driving cycle (UDDS), and mixed urban and highway driving (LA92), were studied. The data of one driving mode was selected as the source domain, and the data of another driving mode was selected as the target domain for testing to achieve SOC estimation.
3. According to claim 1, a method for estimating low-temperature SOC of a power battery based on improved Transformer and adaptive transfer learning is characterized in that: The improved Transformer (Enformer) model described in step S2 mainly includes three modules: multi-layer residual attention mechanism, pattern dynamic decomposition, and bidirectional long short-term memory network, including the following steps: S21. Design a multi-layer residual attention mechanism to replace the traditional self-attention mechanism, and use multi-layer residual connections to alleviate the problem of gradient disappearance as the number of network layers increases; S22. Design a pattern dynamic decomposition module to replace the normalization layer in the traditional Transformer, dynamically decompose the measurement feature sequence into trend information and implicit information, and improve the model's learning of implicit information in driving environments such as low temperatures; S23. Design a bidirectional long short-term memory network to replace the linear layer in the traditional Transformer to improve the temporal representation ability of the model; S24. Complete the Enformer model structure design. The order of data feature flow is multi-level residual attention module, pattern dynamic decomposition module, bidirectional long short-term memory network module, and pattern dynamic decomposition module.
4. The method according to claim 3, characterized in that The pattern dynamic decomposition module in step S22 includes the following steps: S221, mapping the features of the first and last parts of the sample to increase the sample length, so as to alleviate the loss of timing information caused by the truncation of the time window; S222, using moving average pooling to change the added dimension to the original dimension to extract trend information of the sample; S223, using the trend information to filter the original sample to obtain sample implicit information; S224, trend information and implicit information are linearly learned respectively, and feature concatenation is performed and sent to the next module.
5. According to claim 1, a method for estimating low-temperature SOC of a power battery based on improved Transformer and adaptive transfer learning is characterized in that: Designing an adaptive transfer learning training framework in step S3 includes the following steps: S31, use the source domain data to train the Enformer model, and use the target domain data to fine-tune it to obtain fine-tuning features; S32, using the target domain data to autonomously train the model Enformer* to obtain autonomous training features; S33, adaptively concatenate the fine-tuning features and the autonomous training features, and then send them to the linear layer. When the maximum number of training rounds is reached, the trained model is obtained and the SOC is estimated.