Battery life prediction method and system based on self-supervised Transform network

Abstract

The invention discloses a battery life prediction method and system based on a self-supervised Transform network. The method comprises the following steps: acquiring discharge cycle data of a battery; carrying out cross-cycle median Min-Max normalization processing on the discharge cycle data filled with the missing values to obtain target discharge cycle data; a physical property self-supervised learning transform network is constructed; according to a preset SA-Adam optimizer, optimizing parameters of the physical property self-supervised learning transformer network, and obtaining a battery life prediction model based on the self-supervised transformer network; and inputting the target discharge cycle data into the battery life prediction model based on the self-supervised Transform network, wherein the battery life prediction model based on the self-supervised Transform network outputs a corresponding battery life prediction result based on the self-supervised Transform network. Through two self-supervision tasks, the model can automatically extract physical features, and combines the physical features with a self-attention mechanism of a Transform network, thereby improving the accuracy and stability of prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of battery management, and particularly relates to a battery life prediction method and system based on a self-supervised Transformer network. Background Technique

[0002] Lithium-ion batteries are widely used in fields such as portable electronic devices, electric vehicles, and energy storage systems due to their high efficiency, safety, and reliability. Their life prediction is crucial for ensuring the safety and reliability of devices, and helps to effectively avoid equipment damage and safety accidents caused by battery failure. The life of a lithium battery is affected by various factors, including battery type, usage environment, charge and discharge rate, temperature, and current. As the number of charge and discharge cycles increases, the battery capacity gradually decreases, eventually leading to battery failure.

[0003] Currently, the battery life prediction methods based on a self-supervised Transformer network mainly include physical model methods, statistical model methods, and machine learning methods. The physical model method predicts the battery life by establishing models of the chemical and physical characteristics of the battery, which has high accuracy, but its model is complex and difficult to handle the non-linear problems inside the battery. The statistical model method establishes a statistical model by using historical data. Although it is easy to implement, the prediction accuracy is often limited by the quality and quantity of the data. Machine learning methods, such as support vector machines and neural networks, can effectively process non-linear data and have good prediction performance, but usually require a large amount of labeled data and have a high computational complexity. Summary of the Invention

[0004] The present invention provides a battery life prediction method and system based on a self-supervised Transformer network, which is used to solve the technical problems such as insufficient accuracy, poor model stability, and difficulty in processing long-time series data in the traditional methods for predicting the battery life based on a self-supervised Transformer network.

[0005] In a first aspect, the present invention provides a battery life prediction method based on a self-supervised Transformer network, including: Obtaining the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data; Performing missing value filling on the discharge cycle data according to a preset relative change rate weighted filling method, and performing cross-cycle median Min-Max normalization processing on the discharge cycle data after filling the missing values to obtain target discharge cycle data; Constructing a physical characteristic self-supervised learning transformer network; Optimize the parameters of the physical property self-supervised learning Transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network; Input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

[0006] In a second aspect, the present invention provides a battery life prediction system based on a self-supervised Transformer network, including: An acquisition module configured to acquire the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data; A preprocessing module configured to fill in the missing values of the discharge cycle data according to the preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data; A construction module configured to construct a physical property self-supervised learning Transformer network; An optimization module configured to optimize the parameters of the physical property self-supervised learning Transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network; An output module configured to input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

[0007] In a third aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the battery life prediction method based on the self-supervised Transformer network according to any embodiment of the present invention.

[0008] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program instructions are executed by a processor, the processor is enabled to execute the steps of the battery life prediction method based on the self-supervised Transformer network according to any embodiment of the present invention.

[0009] The battery life prediction method and system based on the self-supervised Transformer network of the present application introduce two self-supervised learning tasks during the training process of the model. Task 1 enables the model to accurately predict voltage changes and capture the influence of current and temperature on the battery voltage by learning the battery voltage response characteristics. Task 2 learns the attenuation characteristics of the battery capacity and guides the model to predict the capacity change of the battery according to the physical relationship among current, temperature, and capacity. Through these two self-supervised tasks, the model can automatically extract physical features and combine them with the self-attention mechanism of the Transformer, thereby improving the accuracy and stability of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 is a flowchart of a battery life prediction method based on a self-supervised Transformer network provided by an embodiment of the present invention; Figure 2 is a structural block diagram of a battery life prediction system based on a self-supervised Transformer network provided by an embodiment of the present invention; Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0013] Please refer to Figure 1 , which shows a flowchart of a battery life prediction method based on a self-supervised Transformer network of the present application.

[0014] As Figure 1 shown, the battery life prediction method based on the self-supervised Transformer network specifically includes the following steps: Step S101: Obtain the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data.

[0015] In this step, the discharge information of the battery is measured by measuring instruments or sensors, including the information of current I (unit: A), voltage V (unit: V), temperature T (unit: °C), and capacity C (unit: mAh) at different time periods. Then, the extracted physical information is preprocessed. Among them, represents the number of samples (step size) at each time step, that is, the number of data points within each time period; represents the number of cycles, that is, the number of cycles of data recording; 4 represents the data dimension at each time step, including four features: current, voltage, temperature, and capacity.

[0016] Step S102: Fill in the missing values of the discharge cycle data according to the preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data.

[0017] In this step, the expression for filling in the missing values of the discharge cycle data is: , , , , In the formula, is the filling result at time point t, is the weight at time point t - 1, is the actual data at time point t - 1, is the actual data at time point t + 1, is the weight at time point t + 1, is the relative change rate at time point t - 1, is the relative change rate at time point t + 1, is the relative change rate at time point t, is the actual data at time point t.

[0018] According to the median filtering method, the discharge cycle data after filling in the missing values is denoised to obtain the denoised discharge cycle data. Perform cross-cycle median Min-Max normalization on the discharge cycle data after median filtering. The expression is: , In the formula, is the value after normalization, is the discharge cycle data after filling missing values ​​at time point t, is the median of the minimum value of the discharge cycle data in all discharge cycles, It is the median of the maximum values ​​of battery data in all discharge cycles.

[0019] In this implementation, by preprocessing the discharge cycle data, the subsequent model can recognize and handle these special cases and avoid excessive restrictions on extreme values. At the same time, allowing out-of-range values ​​enhances the robustness of the model, enabling it to adapt to different battery working conditions, improve the accuracy and stability of the model in battery life prediction based on the self-supervised Transformer network, and effectively capture the various changing characteristics of the battery.

[0020] Step S103, constructing a physical property self-supervised learning transformer network.

[0021] The traditional transformer network usually predicts battery life as follows: after inputting battery time series data (such as voltage, current, temperature, capacity decay, etc.), the numerical value of each time step is first converted into a high-dimensional vector through the embedding layer, and the time position code is added to mark the number of cycles and the aging stage; then, the multi-layer encoder uses a multi-head self-attention mechanism to analyze the dynamic correlation between multiple variables (such as high temperature accelerates capacity decay, the impact of current fluctuations on life), stabilizes feature extraction through residual connections and layer normalization, and mines long-term dependencies (such as the potential connection between early minor degradation and later capacity drops); finally, the global features output by the encoder are mapped to the remaining life (RUL) or capacity prediction value through the fully connected layer, and the model parameters are optimized through the mean square error loss and optimizer.

[0022] In the prediction of the remaining useful life (RUL) of batteries, traditional Transformer networks mainly make predictions by learning long-term and short-term dependencies in time series data. However, the Transformer network that relies solely on the data itself may not be able to fully capture the complex relationship between the battery state and physical properties, especially when it comes to nonlinear dynamic changes such as battery voltage and capacity decay. In order to solve this problem, the present invention proposes two self-supervised learning tasks, namely voltage response feature prediction and capacity decay feature prediction. These two tasks effectively guide the Transformer network to more accurately capture the relationship between the physical laws of the battery and the battery state by introducing physical constraints such as battery current, voltage, and temperature. Compared with the traditional single Transformer network, after adding the self-supervised learning task, the network can learn the time series features while combining the physical characteristics to make more accurate battery life predictions based on the self-supervised Transformer network, thereby improving the stability, accuracy and reliability of the prediction. The input shape of the self-supervised learning task is , i.e., the data of each batch.

[0023] The first self-supervised learning task aims to help the network learn the variation law of battery voltage from historical data by guiding the self-attention mechanism of the Transformer network, better understand the relationship between current, temperature and voltage, and predict the future change trend of battery voltage. The traditional Transformer network mainly relies on the self-attention mechanism to process time series data. The first self-supervised learning task guides the network to focus on the variation law of battery voltage, making the Transformer network more accurate in processing voltage prediction, especially in the complex patterns of battery voltage changing with current and temperature. The shape of the input data is , and the expression of the first self-supervised learning task is: , In the formula, is the predicted value of the battery voltage at time point t, is the sigmoid activation function, which is used to compress the output of the network to the range of [0,1], is the battery capacity at time point t-1, is the discharge current at time point t, is the battery temperature at time point t, is the influence coefficient of temperature on battery capacity prediction, , are both normal constants, is the coefficient that controls the influence intensity of current on capacity prediction and is used to determine the influence degree of current on capacity decay, is the weight that controls the influence of temperature on battery capacity attenuation, is the bias term; : is used to capture the nonlinear relationship between current and temperature

[0024] : Through the self-attention mechanism of the Transformer network, the model learns the nonlinear influence of current and temperature on battery voltage and captures the relationship between current and temperature changing with time in long time series data, thus improving the prediction accuracy.

[0025] : It represents the impact of the changes in current and temperature over time on the battery performance. The effect of temperature on current gradually weakens, and the long-term interaction between current and temperature is reflected through integration. By accumulating historical data, the decaying impact of temperature on current is captured to assist the model in fitting the long-term state changes of the battery.

[0026] : It represents the impact of the change rates of current and temperature on the working state of the battery. The immediate effects of the changes in current and temperature are modeled to accurately capture the impact of temperature changes on the battery current, thereby improving the prediction accuracy of the model.

[0027] : It represents the impact of the changes in current and temperature over time, considering the long-term impact of temperature fluctuations on the battery state. The deep impacts of current and temperature on the battery state are captured, especially in high or low temperature environments where the impact of temperature on battery performance is more significant.

[0028] The first self-supervised learning task combines input information such as the current, voltage, and temperature of the battery, and predicts the remaining useful life (RUL) of the battery through the Transformer network and its internal self-attention mechanism and integral operation. This model attempts to capture the complex relationships between battery states and model the temporal changes in battery performance to help the model better capture the changes in battery states and improve the prediction accuracy. The output result is the predicted value of the battery voltage with the shape of , and this output represents the battery voltage prediction for each time step.

[0029] The second self-supervised learning task guides the self-supervised learning mechanism inside the Transformer network to learn the relationship between battery capacity decay and current and temperature. The traditional Transformer network uses the self-attention mechanism to process temporal data, but in the battery life prediction based on the self-supervised Transformer network, directly relying on the data itself may not be sufficient to accurately model the capacity decay law. By combining physical characteristics, the second self-supervised learning task provides a learning signal that is more in line with the actual battery characteristics for the Transformer network, helping the model better understand the dynamic changes of the battery, and thus improving the prediction accuracy of the remaining useful life (RUL). The shape of the input data is . The expression of the second self-supervised learning task is: , where is the predicted battery capacity at time point t, is the weight controlling the impact of temperature on battery capacity decay, and is the bias term.

[0030] : The relationship between the change in battery capacity and the current temperature. As the temperature increases, the battery capacity is usually negatively affected. 25 refers to the standard temperature (i.e., the reference temperature), and 25°C is a commonly used standard temperature in battery modeling to represent the battery performance at room temperature.

[0031] : The non-linear relationship between current and capacity. The relationship between the square of the current and the capacity represents the impact of high current on the battery capacity. A larger current will accelerate the capacity decline, and as the current increases, its impact gradually increases.

[0032] : Simulating the battery degradation process by considering the dynamic changes in temperature and current. The effects of temperature and current change over time. The exponential decay term simulates the phenomenon of the battery efficiency decreasing when the battery temperature increases. When the battery temperature rises, the impact of current on the battery will increase, but too high a temperature will reduce the chemical reaction rate, resulting in a gradual decrease in battery efficiency.

[0033] : The differential term of capacity with respect to current, which describes how the change in current affects the change in battery capacity. An increase in current may lead to a decline in the battery capacity, and it is used to capture the dynamic response of the battery capacity to changes in current.

[0034] The second self-supervised learning task incorporates the physical relationships between battery capacity degradation, current, and temperature into the model, guiding the Transformer network to more accurately capture the state changes of the battery capacity. By using self-supervised learning to capture the complex dynamic relationships between battery states, the prediction accuracy of the remaining useful life of the battery is improved, and the physical characteristics of the battery under different operating conditions are considered. The output result is the prediction of the battery voltage whose shape is and this output represents the prediction of the battery capacity at each time step.

[0035] It should be noted that the total loss function of self-supervised learning is trained by comparing the error between the network prediction result and the true value. To better capture the dynamic characteristics of the battery, the mean squared error (MSE) is used as the loss function. The input is the prediction result of the network and , as well as the actual voltage and the actual capacity whose shape is .

[0036] In the task of predicting the remaining useful life (RUL) of a battery, traditional supervised learning methods rely on a large amount of labeled data, while self-supervised learning guides the model to learn the dynamic behavior of the battery without direct labeling by designing tasks. To accurately capture the state changes of the battery and incorporate physical laws, the present invention proposes two self-supervised learning tasks: capacity fade prediction and voltage response prediction. First, the loss functions of the two tasks are calculated, and finally the total self-supervised learning loss function is obtained.

[0037] The expression for calculating the loss of capacity fade feature prediction is: , where, is the size of the dataset, is the predicted battery capacity at time point t, is the actual capacity at time point t, is the influence intensity of control current and temperature on capacity fade, is the time point of the battery temperature, is the time point of the battery current, is the weight for controlling the influence of temperature on battery capacity fade, is the battery capacity at time point t-1, is the natural base, is a positive constant, is the influence intensity for controlling the non-linear relationship between current and capacity, is the discharge current at time point t; The expression for calculating the loss of voltage response feature prediction is: , where, is the predicted value of the battery voltage at time point t, is the actual voltage at time point t, is the influence intensity of control current and temperature on voltage response, is the influence intensity of the differential relationship between control current and temperature on voltage prediction.

[0038] The expression for the total self-supervised learning loss function is: , where, is the total self-supervised learning loss function, is the weight of the capacity fade feature prediction loss, is the capacity fade feature prediction loss, is the weight of the voltage response feature prediction loss, is the voltage response feature prediction loss; The output of the self-supervised learning total loss function is usually the remaining useful life (RUL) prediction result obtained after passing the hidden state at the last output time step of the Transformer network through a fully connected layer after the fully connected layer.

[0039] Furthermore, the output of the Transformer network is generated from the results of the encoder layer, and the output shape is . By performing a linear transformation on the output of the last time step of the Transformer network, the final prediction result can be obtained. Specifically, after the output of the Transformer network is processed by a fully connected layer, the predicted value of the remaining useful life (RUL) of the battery is obtained. This output is usually obtained by passing the output of the last time step of the Transformer network through a linear transformation layer, providing an accurate prediction of the remaining useful life of the battery. The expression is: , In the formula, is the result representing the prediction of the remaining useful life (RUL) of the battery, is the weight matrix of the fully connected layer, and its size is , where is the hidden layer dimension of the Transformer network (i.e., the output dimension of the last encoder). Obtained through training and learning, used to map the hidden state at the last moment to the predicted value of RUL.

[0040] represents the output vector of the Transformer network at the last time step, the output of the last moment of the Transformer network. It is the feature extracted from the Transformer network, representing the hidden state of the battery at time point t. It is a representation of the historical information such as battery current, temperature, and capacity, usually a vector.

[0041] B is the bias term, which is a scalar used to perform a translation adjustment on the output of the network to ensure that the RUL value output by the model can adapt to different training data distributions.

[0042] W and b are the parameters learned by the model and optimized through training. The weight W maps the hidden state Encoder Output obtained from the Transformer network to the scalar prediction of the remaining useful life (RUL). Encoder Output is the final output of the Transformer network, which captures the state of the battery at multiple time steps, reflecting the health state, capacity decay, and other related factors of the battery.

[0043] Through the linear layer, the shape of the output is , which is the predicted remaining useful life (RUL) of the battery. Here, only the output of the last time step is taken, with the shape of B×1, and each sample corresponds to an RUL.

[0044] In this embodiment, the prediction ability of the traditional Transformer network is improved through two self-supervised learning tasks. The first self-supervised learning task guides the self-attention mechanism of the Transformer network to learn the battery voltage response characteristics, enabling it to more accurately capture the influence of current and temperature on the battery voltage; the second self-supervised learning task combines the physical characteristics of the battery to guide the network to learn the attenuation law of the battery capacity and improve the accuracy of capacity prediction. Through the design of these two tasks, the network not only relies on the data itself but also takes the physical laws of the battery as additional constraints, thereby optimizing the prediction results and enhancing the accuracy, reliability, and stability of the battery life prediction based on the self-supervised Transformer network. This way of combining physical characteristics with self-supervised learning provides a more accurate and stable battery performance prediction scheme compared to traditional methods.

[0045] Step S104, optimize the parameters of the physical characteristic self-supervised learning transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network.

[0046] In this step, the SA-Adam optimizer adjusts the optimization parameters in real time according to the performance of the network during training, thus avoiding overfitting and local optimal solution problems. Through this mechanism, the SA-Adam optimizer not only enhances the robustness of the model but also effectively improves the convergence speed and generalization ability of the battery life prediction model based on the self-supervised Transformer network, thereby improving the accuracy and stability of the prediction, especially in the face of fluctuations during the complex discharge process of the battery.

[0047] Momentum calculation (with temperature decay mechanism): , , : The momentum at the current time step t, representing the first-order moment estimate of the gradient, recording the weighted average of historical gradients and accelerating the convergence direction; : The momentum decay rate, the larger it is, the more dependent on historical momentum (smoothing the update direction); : The momentum value at the previous step (t - 1), providing historical momentum information and forming time dependence; : The weight of the current gradient. The smaller it is, the smaller the contribution of the current gradient to the momentum. : The gradient of the loss function, providing the update direction of the current parameters. : Temperature decay noise, introducing controllable randomness to prevent getting stuck in local optima. : Indicates that the noise follows a normal distribution with a mean of 0 and a variance of , dynamically adjusting the magnitude of the temperature-controlled noise of the model. : The temperature value at the current moment, affecting the amplitude of gradient update, not the physical temperature. : The temperature at the previous moment. : Temperature growth coefficient, adjusting the rate of temperature change. : Temperature decay coefficient, controlling the intensity of exponential decay. : The current time step. As time goes by, the temperature gradually decreases. : Temperature decay factor. As the training progresses, the temperature gradually decreases, simulating the process of simulated annealing. Momentum calculation (with temperature decay mechanism) is based on the traditional Adam. By introducing temperature decay noise, the model has stronger exploration ability in the initial stage of training, avoiding getting stuck in local optimal solutions, and gradually converging to the global optimum. The temperature decay factor decreases as the time step t increases, helping the optimizer to conduct extensive exploration in the initial stage and gradually reducing the gradient update amplitude in the later stage, making the optimization process more stable. This mechanism improves the ability to process complex data in battery life prediction based on self-supervised Transformer networks. Especially when facing long time series data, it can effectively improve the robustness and accuracy of the model.

[0048] Adaptive second moment estimation:

[0049] Among them, : The second moment at the current time step t, estimating the variance of the gradient, used for adaptive learning rate. : Second moment decay rate. The larger it is, the more dependent on historical variance (stable variance estimation). : The second moment value at the previous step (t - 1), providing historical variance information to form time dependence. : The weight of the current squared gradient. The smaller it is, the smaller the contribution of the current squared gradient to the learning rate. : The current squared gradient (to prevent negative gradient interference), amplifying the parameters with large gradients and shrinking the parameters with small gradients (to achieve parameter-level adaptivity); : The performance response coefficient, adjusting the influence degree of ΔQ on the second moment; : The change in the performance metric, dynamically adjusting the second moment estimation to implement the "reward - penalty" mechanism; : The base learning rate; The adaptive learning rate calculation is adjusted according to the square of the gradient and the change of model parameters. The influence of simulated annealing is corrected through and parameters.

[0050] The adaptive second moment estimation in SA - Adam is used to adjust the update amplitude of each parameter, depending on the square value of the gradient and the estimation of the second moment. Compared with the traditional Adam optimizer, SA - Adam adaptively adjusts the second moment by introducing the term (1 + γΔQ), so as to cope with the dynamic changes in the battery life prediction task based on the self - supervised Transformer network. Through the calculation of the adaptive second moment, the optimizer can adjust the parameter update according to the change of the gradient at different training stages, improving the ability to handle uncertain data, and thus enhancing the training stability and prediction accuracy of the model.

[0051] Parameter update:

[0052] , where: : The bias - corrected momentum, eliminating the momentum estimation bias in the initial stage; : The momentum at the current time step; : The momentum estimation bias correction parameter; : The bias - corrected second moment, eliminating the variance estimation bias in the initial stage; : The squared gradient at the current moment; : The variance estimation bias correction parameter; : The current parameter value, that is, the model parameter to be updated; : Model parameters at the previous moment; : Dynamic learning rate; : Smoothing constant to prevent the denominator from being zero; : Noise term; : Normalized smoothing constant to prevent the normalization term from being too large; During parameter update, the corrected momentum estimate and second - moment estimate are used to update the parameters. First, by correcting the bias of the momentum and second - moment, the influence of excessive bias at the beginning of training is avoided. Then, use to adjust the step size, combine the corrected momentum and second - moment, and perform parameter update. At this time, is introduced as a noise term to help simulate the annealing process, further enhancing the stability of the optimization process. Through this series of adjustments, SA - Adam can better converge on long - time - series data, avoid overfitting, and improve the accuracy and robustness of battery life prediction based on the self - supervised Transformer network.

[0053] Learning rate adjustment:

[0054] Where: : Learning rate at the current step; : Base learning rate; : For the initial time steps, the learning rate decays exponentially; : Current time step, after exceeding 100, the learning rate grows linearly; In the initial stage of training (t < 100), a decaying learning rate is used to help the model converge quickly, while in the later stage of training (t ≥ 100), a growing learning rate is used to improve the fitting ability of the model in the later stage. By dynamically adjusting the learning rate to update the step size, it is ensured that the model can converge and will not fall into local optimum prematurely. Adjust the size of the step in parameter update. The temperature decay mechanism makes the learning rate larger in the initial stage of training to help the model learn quickly; smaller in the later stage to help the model make fine adjustments and avoid instability caused by over - adjustment.

[0055] In this embodiment, the SA-Adam optimizer is an improvement based on the traditional Adam optimizer. It mainly optimizes the training process by introducing a temperature decay mechanism, a noise term, and an improvement in adaptive second-moment estimation. The temperature decay mechanism simulates the process of simulated annealing, providing a relatively large learning rate at the beginning of training for exploration, and gradually reducing the learning rate over time to avoid premature convergence. The introduction of the noise term increases the randomness of the optimization process, enhances the exploration ability of the model, and helps to avoid local optima. The adjustment factor of the adaptive second moment (such as (1 + γΔQ)) enables the optimizer to better adapt to the dynamic changes of the data, improving the stability of training. Compared with the traditional Adam, SA-Adam is more capable of handling data fluctuations and non-linear relationships in the battery remaining useful life (RUL) prediction task, helping to improve the accuracy and stability of the model. Through these innovations, the SA-Adam optimizer can process the complex time-series data of the battery, enhance the robustness of battery life prediction based on the self-supervised Transformer network, and provide more reliable prediction results.

[0056] Step S105: Input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

[0057] In summary, in this application, first, the input data includes information such as the voltage, current, temperature, and capacity of the battery. These data are preprocessed, including steps such as filling missing values by the relative change rate weighted filling method, median filtering for denoising, and cross-cycle median Max-Min normalization, to ensure data quality and standardize the range of features. Then, the preprocessed data is input into the Transformer network. The Transformer processes time-series data through the self-attention mechanism to capture the long-term dependencies of the battery state.

[0058] During the training process of the model, two self-supervised learning tasks are introduced. Task 1 enables the model to accurately predict voltage changes and capture the influence of current and temperature on the battery voltage by learning the battery voltage response characteristics. Task 2 learns the decay characteristics of the battery capacity and guides the model to predict the capacity change of the battery based on the physical relationship between current, temperature, and capacity. Through these two self-supervised tasks, the model can automatically extract physical features and combine them with the self-attention mechanism of the Transformer, thereby improving the accuracy and stability of the prediction.

[0059] The SA-Adam optimizer is adopted, which combines the simulated annealing mechanism (SA) to dynamically adjust the learning rate and momentum. The SA-Adam optimizer introduces an adaptive adjustment parameter process through the temperature decay mechanism and noise term, avoiding the dilemma that the traditional Adam optimizer is prone to falling into local optima and effectively improving the stability of the training process. In addition, the SA-Adam optimizer can gradually enhance the generalization ability of the model during the training process, accelerate model convergence, and reduce overfitting, ultimately improving the accuracy and robustness of battery life prediction based on the self-supervised Transformer network. It provides a more accurate and stable solution for battery life prediction based on the self-supervised Transformer network.

[0060] In summary, the method of this application proposes a battery life prediction model based on the self-supervised Transformer network with the SA-Adam optimizer. By introducing the dynamic learning mechanism and temperature decay mechanism of the SA-Adam optimizer, the model may achieve better results when processing long-time series data of batteries, especially in capturing the long-term dependence relationship of battery states, providing a relatively effective solution. The temperature decay mechanism helps the optimizer conduct a more extensive exploration in the initial stage of training and gradually converge as the training progresses, thereby potentially improving the stability and accuracy of prediction.

[0061] In addition, a physical constraint model is incorporated, enabling the model to better conform to the physical characteristics of the battery during the battery life prediction process based on the self-supervised Transformer network. Through this physical constraint, the model is expected to more accurately adapt to the dynamic changes of the battery under different operating conditions, thereby potentially enhancing the reliability and practicality of life prediction.

[0062] Please refer to Figure 2 , which shows the structural block diagram of a battery life prediction system based on the self-supervised Transformer network of this application.

[0063] As Figure 2 shown, the battery life prediction system 200 based on the self-supervised Transformer network includes an acquisition module 210, a preprocessing module 220, a construction module 230, an optimization module 240, and an output module 250.

[0064] Among them, an acquisition module 210 is configured to acquire discharge cycle data of a battery, and the discharge cycle data includes voltage data, current data, temperature data, and capacity data; a preprocessing module 220 is configured to fill missing values in the discharge cycle data according to a preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling the missing values to obtain target discharge cycle data; a construction module 230 is configured to construct a physical property self-supervised learning transformer network; an optimization module 240 is configured to optimize parameters of the physical property self-supervised learning transformer network according to a preset SA-Adam optimizer to obtain a battery life prediction model based on a self-supervised Transformer network; an output module 250 is configured to input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs a corresponding battery life prediction result based on the self-supervised Transformer network.

[0065] It should be understood that Figure 2 the modules described in Figure 1 correspond to the respective steps in the method described in the reference Figure 2 Therefore, the operations, features, and corresponding technical effects described above for the method also apply to

[0066] In some other embodiments, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored. When the program instructions are executed by a processor, the processor executes the battery life prediction method based on a self-supervised Transformer network in any of the above method embodiments; As an implementation manner, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are set as: Acquire discharge cycle data of a battery, and the discharge cycle data includes voltage data, current data, temperature data, and capacity data; Fill missing values in the discharge cycle data according to a preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling the missing values to obtain target discharge cycle data; Construct a physical property self-supervised learning transformer network; Optimize parameters of the physical property self-supervised learning transformer network according to a preset SA-Adam optimizer to obtain a battery life prediction model based on a self-supervised Transformer network; Input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

[0067] The computer-readable storage medium may include a storage program area and a storage data area. Among them, the storage program area can store an operating system and application programs required for at least one function; the storage data area can store data created according to the use of the battery life prediction system based on the self-supervised Transformer network, etc. In addition, the computer-readable storage medium may include high-speed random access memory, and may also include memories, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the computer-readable storage medium may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the battery life prediction system based on the self-supervised Transformer network through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0068] Figure 3 is a schematic structural diagram of the electronic device provided by an embodiment of the present invention, as Figure 3 shown, the device includes: a processor 310 and a memory 320. The electronic device may further include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330, and the output device 340 may be connected through a bus or other means, Figure 3 taking the connection through the bus as an example. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 320, that is, implements the battery life prediction method based on the self-supervised Transformer network in the above method embodiment. The input device 330 can receive input digital or character information, and generate key signal inputs related to the user settings and function control of the battery life prediction system based on the self-supervised Transformer network. The output device 340 may include a display device such as a display screen.

[0069] The above electronic device can execute the method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided by the embodiment of the present invention.

[0070] As an implementation manner, the above-mentioned electronic device is applied to a battery life prediction system based on a self-supervised Transformer network and is used for a client, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Obtain the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data; Fill in the missing values of the discharge cycle data according to the preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data; Construct a physical property self-supervised learning transformer network; Optimize the parameters of the physical property self-supervised learning transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network; Input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

[0071] Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0072] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A battery life prediction method based on a self-supervised Transformer network, characterized in that, Including: Obtain the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data; Fill in the missing values of the discharge cycle data according to the preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data; Construct a physical property self-supervised learning transformer network; Optimize the parameters of the physical property self-supervised learning transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network; Input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs the corresponding battery life prediction result based on the self-supervised Transformer network.

2. The battery life prediction method based on a self-supervised Transformer network according to claim 1, wherein Before performing cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data, the method further includes: Denoise the discharge cycle data after filling in the missing values according to the median filtering method to obtain the denoised discharge cycle data.

3. A battery life prediction method based on a self-supervised Transformer network according to claim 1, characterized in that, Wherein, The expression for filling in the missing values of the discharge cycle data is: ， ， ， ， In the formula, is the filling result at time point t, is the weight at time point t - 1, is the actual data at time point t - 1, is the actual data at time point t + 1, is the weight at time point t + 1, is the relative change rate at time point t - 1, is the relative change rate at time point t + 1, is the relative change rate at time point t, is the actual data at time point t.

4. A battery life prediction method based on a self-supervised Transformer network according to claim 1, characterized in that, Wherein, The expression for performing cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values is: ， Wherein, is the value after normalization, is the discharge cycle data after filling the missing values at time point t, is the median of the minimum values of the discharge cycle data in all discharge cycles, is the median of the maximum values of the battery data in all discharge cycles.

5. A battery life prediction method based on a self-supervised Transformer network according to claim 1, characterized in that, The physical property self-supervised learning transformer network includes a first self-supervised task for predicting voltage response characteristics and a second self-supervised task for predicting capacity decay characteristics; The expression of the first self-supervised task is: ， Wherein, is the predicted value of the battery voltage at time point t, is the sigmoid activation function, which is used to compress the output of the network to the range of [0,1], is the battery capacity at time point t-1, is the discharge current at time point t, is the battery temperature at time point t, is the influence coefficient of temperature on battery capacity prediction, 、 are both normal constants, is the coefficient for controlling the influence intensity of current on capacity prediction, which is used to determine the influence degree of current on capacity decay, is the weight for controlling the influence of temperature on battery capacity attenuation, is the bias term; The expression of the second self-supervised task is: ， Wherein, is the predicted battery capacity at time point t, is the weight for controlling the influence of the temperature on the battery capacity attenuation, is the bias term.

6. A battery life prediction method based on a self-supervised Transformer network according to claim 1, characterized in that, The expression of the total self-supervised learning loss function of the physical property self-supervised learning transformer network is: ， In the formula, is the total loss function of self-supervised learning, is the weight of the capacity attenuation feature prediction loss, is the capacity attenuation feature prediction loss, is the weight of the voltage response feature prediction loss, is the voltage response feature prediction loss; The expression for calculating the capacity decay characteristic prediction loss is: ， Wherein, is the size of the data set, is the predicted battery capacity at time point t, is the actual capacity at time point t, is the influence intensity of controlling current and temperature on capacity decay, is the time point of the battery temperature, is the time point of the battery current, is the weight of controlling the influence of temperature on battery capacity decay, is the battery capacity at time point t-1, is the natural base, is a positive constant, is the influence intensity of controlling the non-linear relationship between current and capacity, is the discharge current at time point t; The expression for calculating the voltage response characteristic prediction loss is: ， In the formula, is the predicted value of the battery voltage at time point t, is the actual voltage at time point t, is the influence intensity of the control current and temperature on the voltage response, is the influence intensity of the differential relationship between the control current and temperature on the voltage prediction.

7. A battery life prediction method based on a self-supervised Transformer network according to claim 1, characterized in that The optimizing the parameters of the physical property self-supervised learning transformer network according to the preset SA-Adam optimizer to obtain a battery life prediction model based on the self-supervised Transformer network includes: At the beginning stage of model training, initialize the parameters of the SA-Adam optimizer; In each training cycle, the SA-Adam optimizer calculates the gradient according to the current loss function; According to the gradient, the SA-Adam optimizer is used to correct the momentum and the second moment , and through the corrected momentum and the second moment update the parameters of the physical property self-supervised learning transformer network , to obtain a battery life prediction model based on the self-supervised Transformer network.

8. A battery life prediction system based on a self-supervised Transformer network, characterized in that, Including: An acquisition module configured to obtain the discharge cycle data of the battery, where the discharge cycle data includes voltage data, current data, temperature data, and capacity data; A preprocessing module configured to fill in the missing values of the discharge cycle data according to the preset relative change rate weighted filling method, and perform cross-cycle median Min-Max normalization processing on the discharge cycle data after filling in the missing values to obtain the target discharge cycle data; A construction module configured to construct a physical property self-supervised learning transformer network; An optimization module, configured to optimize parameters of the physical property self-supervised learning Transformer network according to a preset SA-Adam optimizer, to obtain a battery life prediction model based on the self-supervised Transformer network; An output module, configured to input the target discharge cycle data into the battery life prediction model based on the self-supervised Transformer network, and the battery life prediction model based on the self-supervised Transformer network outputs a corresponding battery life prediction result based on the self-supervised Transformer network.

9. An electronic device, characterized in that, Comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images