A Battery SOH Estimation Method and System Based on RL-PINN

Abstract

This invention discloses a battery SOH estimation method and system based on RL-PINN. The method includes: acquiring real-time monitoring data during the charge-discharge cycle of a lithium-ion battery, extracting key degradation physical features as input to a physical information time-series decoding model, and outputting an estimated SOH value and a degradation state representation; establishing a multi-dimensional physical constraint library; constructing a reinforcement learning controller based on the current SOH estimate, SOH local evolution gradient, physical constraint topology encoding, and feature statistics, and outputting a physical constraint activation vector and a physical constraint loss weight vector; inputting the physical constraint activation vector and physical constraint loss weight vector into a composite loss function to train the physical information time-series decoding model and the reinforcement learning controller, and outputting the battery SOH estimation result and life cycle stage boundary delineation information. This invention can improve the stability and physical consistency of battery SOH estimation under long-cycle and complex operating conditions.

Description

Technical Field

[0001] This invention relates to the field of battery state monitoring and battery management system technology, and in particular to a battery health state estimation method and system based on RL-PINN (fusion of reinforcement learning and physical information neural network). Background Technology

[0002] As the core energy storage unit in electric vehicles, energy storage systems, and portable electronic devices, the health status of lithium-ion batteries directly affects the safety, reliability, and lifespan of the battery system. State of Health (SOH) is a crucial indicator characterizing the battery's current usable capacity, power capability, and degree of aging. Accurately estimating battery SOH provides a basis for predicting remaining lifespan, optimizing charge / discharge strategies, providing safety warnings, and assessing secondary use within battery management systems.

[0003] Existing battery state-of-the-art (SOH) estimation methods mainly include physical model-driven methods, data-driven methods, and physical-information fusion methods. Physical model-driven methods describe the internal reaction processes of the battery through equivalent circuit models, electrochemical models, or degradation kinetic models, offering good interpretability. However, these methods have numerous model parameters, making parameter identification difficult under complex operating conditions and long-term aging. Furthermore, they are prone to model mismatch due to individual battery differences and irreversible degradation. Data-driven methods learn the mapping relationship between battery monitoring data and SOH through neural networks, recurrent neural networks, or attention models, exhibiting strong nonlinear fitting capabilities. However, their prediction results depend on the distribution of training data, lack physical constraints, and are prone to producing predictions that violate battery degradation patterns under unseen operating conditions, across battery systems, or over long lifespans.

[0004] In recent years, physical information neural networks (PENs) have been used for battery state of health (SOH) estimation. By introducing physical constraints into the loss function, the model achieves both data fitting ability and physical consistency. However, existing PSN applications in battery SOH estimation typically employ fixed physical constraints or fixed physical loss weights, failing to fully consider the significant stages and individual heterogeneity of battery degradation. For example, in the early stages of battery cycling, interfacial film formation and changes in coulombic efficiency have a significant impact on SOH degradation; in the middle stages, battery degradation is relatively stable, and SOH rate of change constraints are more suitable for describing its gradual decay process; at the end of the lifespan, mechanisms such as increased internal resistance, limited diffusion, and continuous SEI film growth may lead to nonlinear accelerated capacity decay. If the same combination of physical constraints and the same loss weights are used throughout the entire lifespan, problems of partial constraint redundancy and partial constraint insufficiency can easily arise, leading to decreased estimation accuracy of the model in long-cycle, complex operating conditions, and cross-cell applications.

[0005] Therefore, there is an urgent need for a battery SOH estimation method that can sense different life cycle stages of the battery, dynamically schedule physical constraints, and adaptively adjust the weights of physical constraint loss, so as to improve the estimation accuracy, physical consistency, and generalization ability of the model in long life cycle, complex variable operating conditions, and cross-battery system scenarios. Summary of the Invention

[0006] Purpose of the Invention: The purpose of this invention is to provide a battery SOH estimation method and system based on RL-PINN. By constructing a physical information temporal decoding model, a multi-dimensional physical constraint library, and a periodic-aware reinforcement learning controller, the model can dynamically activate or suspend PDE dynamic constraints according to the current degradation state of the battery, and adaptively adjust the physical constraint loss weights. This achieves high-precision estimation of battery SOH and dynamic division of battery life cycle stage boundaries, solving the problems of fixed physical constraints, difficulty in dynamically adjusting loss weights with degradation stages, and difficulty in adapting to the nonlinear degradation characteristics of the entire battery life cycle in existing physical information neural networks for battery SOH estimation.

[0007] Technical solution: A battery SOH estimation method based on RL-PINN, comprising the following steps:

[0008] S1, Perform feature engineering on the real-time monitoring data acquired during the charge-discharge cycle of the lithium-ion battery to obtain the key degradation physical feature vector;

[0009] S2, input the key degradation physical feature vector into the physical information time-series decoding model, extract the long-range time-series dependent features in the battery capacity decay process, and output the SOH estimate and battery degradation state characterization information of the current loop window.

[0010] S3 is a multi-dimensional physical constraint library built on basic algebraic physical constraints and PDE dynamic constraints, used to uniformly store various physical constraint losses.

[0011] S4. A state vector is constructed from the SOH estimate, the SOH local evolution gradient, the topological encoding of the currently activated physical constraints, the battery degradation state characterization information, and the statistical information of the voltage and current sequences. The physical constraint activation vector and the physical constraint loss weight vector are used as outputs to construct a reinforcement learning controller. The reinforcement learning controller dynamically activates or suspends the PDE dynamic constraints in the multi-dimensional physical constraint library according to the current degradation state of the battery, and adaptively adjusts the physical constraint loss weights corresponding to the activated physical constraints.

[0012] S5, a composite loss function is constructed from the physical constraint activation vector and the physical constraint loss weight vector to train the physical information temporal decoding model; the composite loss function selects the physical constraint residuals participating in the training according to the current battery degradation state and assigns corresponding weights;

[0013] S6, The physical information temporal decoding model and the reinforcement learning controller are jointly trained using a dual-track asynchronous optimization mechanism;

[0014] S7 inputs the key degradation physical feature vector of the battery to be estimated into the trained physical information time-series decoding model and outputs the battery SOH estimation result; and obtains the battery life cycle stage boundary division information based on the physical constraint activation vector output by the reinforcement learning controller.

[0015] Furthermore, in step S1, the real-time monitoring data includes one or more of the following: voltage, current, temperature, charging capacity, discharging capacity, and number of cycles.

[0016] The preprocessed real-time monitoring data were sorted by feature importance using the random forest algorithm to obtain key degradation physical features, including constant current charging capacity, constant voltage charging capacity, charging time constant, average voltage, voltage kurtosis, and current skewness.

[0017] The key degradation physical features are normalized to obtain the key degradation physical feature vector corresponding to the t-th loop window. :

[0018] ,

[0019] in, Indicates constant current charging capacity. Indicates constant voltage charging capacity. This represents the charging time constant. Indicates average voltage. Indicates voltage kurtosis. Indicates the current deflection.

[0020] Furthermore, the physical information temporal decoding model adopts a bidirectional long short-term memory network with a scaling dot product attention mechanism, including an input layer, a bidirectional long short-term memory network layer, an attention layer, and a fully connected output layer;

[0021] The bidirectional long short-term memory network layer is used to output the bidirectional hidden state. The forward hidden state, the reverse hidden state, and the bidirectional hidden state are represented as follows:

[0022] ,

[0023] ,

[0024] ,

[0025] in, Indicates the first The positive hidden state of a looping window. Indicates the first The reverse hidden state of a looping window. This indicates the bidirectional hidden state after splicing; This indicates a feature extraction operation in the forward time direction. This indicates a feature extraction operation performed in the reverse time direction;

[0026] The attention layer calculates the attention weights for different loop windows using the query vector, key vector, and value vector, and outputs the attention-weighted degradation representation vector. The fully connected output layer receives the attention-weighted degenerate representation vector. Output the estimated SOH value for the current loop window.

[0027] Furthermore, in step S3, the basic algebraic physical constraints include capacity decay monotonicity constraints, SOH change rate constraints, negative correlation constraints based on electrochemical impedance, and coulombic efficiency balance constraints.

[0028] The PDE kinetic constraints include one-dimensional diffusion equation constraints and SEI film growth kinetic constraints.

[0029] The one-dimensional diffusion equation constraint is used to describe the effect of restricted diffusion of active lithium ions within the electrode active material particles on the SOH variation, and to estimate the SOH value. It is considered an external characterization of changes in the concentration of available active materials in the battery;

[0030] The SEI film growth kinetics constraint is used to describe the process by which the continuous growth of the SEI film leads to the loss of active lithium and the decay of SOH.

[0031] Furthermore, in step S4, the reinforcement learning controller adopts a PPO agent based on the Actor-Critic architecture, wherein the Actor network is used to output actions according to the current state, and the Critic network is used to evaluate the state value function of the current state.

[0032] Furthermore, in step S5, the composite loss function includes a data-driven loss term and a physical constraint loss term, which are jointly adjusted by the physical constraint activation vector and the physical constraint loss weight vector output by the reinforcement learning controller. During training, the basic algebraic physical constraints remain in constant activation, while the PDE dynamics constraints are adjusted by the physical constraint activation vector output by the reinforcement learning controller. Dynamic control.

[0033] Furthermore, in step S6, the dual-track asynchronous optimization mechanism includes a continuous update process for the physical information temporal decoding model and a periodic update process for the reinforcement learning controller;

[0034] During the continuous update of the physical information temporal decoding model, the first... Key degenerate physical feature vectors of each loop window Input physical information into a time-series decoding model to obtain SOH estimates. and intermediate state information; the intermediate state information includes bidirectional hidden states. Attention weight and the attention-weighted degenerate representation vector The state vector of the reinforcement learning controller is constructed based on the SOH estimate and intermediate state information. The reinforcement learning controller is based on the state vector. Output physical constraint activation vector and physical constraint loss weight vector Construct the total loss function And update the physical information temporal decoding model parameters through backpropagation;

[0035] During the periodic update process of the reinforcement learning controller, the state vector generated by the interaction between the reinforcement learning controller and the environment is used. Action vectors Reward Value Action probabilities and state values ​​are stored in the trajectory buffer. When the number of samples accumulated in the trajectory buffer reaches a preset threshold, the advantage function is calculated using generalized advantage estimation, and the Actor network and Critic network are updated based on the PPO objective function.

[0036] A battery SOH estimation system based on RL-PINN, used to perform any of the above-mentioned RL-PINN-based battery SOH estimation methods, includes:

[0037] The data sensing and processing module is used to acquire real-time monitoring data during the charge and discharge cycle of lithium-ion batteries and extract key degradation physical characteristics.

[0038] The physical information timing decoding module is used to receive the key degradation physical features, extract long-range timing dependence features in the battery capacity decay process, and output SOH estimates and degradation state characterization information.

[0039] The physical constraint library module stores basic algebraic physical constraints and PDE dynamic constraints, and generates corresponding physical constraint residual losses.

[0040] The intelligent dynamic scheduling module is used to output the physical constraint activation vector and the physical constraint loss weight vector according to the current degradation state of the battery;

[0041] The SOH assessment and boundary output module is used to output battery SOH estimation results and life cycle stage boundary delineation information.

[0042] In the continuous update process of the physical information temporal decoding model, the dual-track asynchronous optimization mechanism constructs a composite loss function based on data-driven loss and physical constraint loss, and updates the parameters of the physical information temporal decoding model according to the composite loss function. In the periodic update process of the reinforcement learning controller, the dual-track asynchronous optimization mechanism stores the state, action, reward and next state into the trajectory buffer, and after the number of trajectory samples reaches a preset threshold, it updates the parameters of the reinforcement learning controller using generalized advantage estimation and PPO objective function.

[0043] Compared with the prior art, the significant advantages of this invention are as follows:

[0044] 1. This invention constructs a periodic-aware reinforcement learning controller, which enables physical constraints to no longer act in a fixed manner throughout the battery life cycle. Instead, it can dynamically activate or suspend PDE dynamic constraints according to the current degradation state of the battery and adaptively adjust the physical constraint loss weights, thereby improving the model's adaptability to different degradation stages.

[0045] 2. This invention combines a physical information time-series decoding model with a multi-dimensional physical constraint library, so that the SOH estimation process is simultaneously subject to the joint constraints of data-driven loss and physical constraint residuals. This can reduce the non-physical prediction bias that may occur in a pure data-driven model and improve the physical consistency of the SOH estimation results.

[0046] 3. This invention outputs physical constraint activation vectors and physical constraint loss weight vectors through a reinforcement learning controller, and inputs them into the composite loss function of the physical information temporal decoding model, forming a closed-loop mechanism of state awareness, boundary decision-making, constraint adaptation, and SOH output. It can realize the dynamic division of the boundaries of battery life cycle stages, and is suitable for long-cycle operation, complex variable load conditions, and SOH estimation scenarios under different battery systems. It has good engineering applicability and can be used for health status assessment, remaining life prediction, and safety operation and maintenance early warning in battery management systems.

[0047] 4. This invention employs a dual-track asynchronous optimization mechanism, enabling the physical information temporal decoding model to continuously learn the SOH estimation task, while simultaneously allowing the reinforcement learning controller to periodically update policy parameters based on trajectory samples. This avoids the reinforcement learning controller directly interfering with the gradient backpropagation of the main model at each step, thereby improving the stability of the joint training process. Attached Figure Description

[0048] Figure 1 This is a flowchart of the battery SOH estimation method based on RL-PINN of the present invention;

[0049] Figure 2 This is a diagram of the dual-track asynchronous optimization and physical constraint dynamic scheduling architecture of the present invention.

[0050] Figure 3This is a block diagram of the battery SOH estimation system based on RL-PINN of the present invention;

[0051] Figure 4 The figures show the battery SOH estimation results and error comparisons of the method of the present invention and the comparative model, wherein (a) is a comparison of the SOH estimation results of the method of the present invention on battery 10; (b) is a comparison of the SOH estimation effects of the method of the present invention, LSTM, GRU-BiLSTM, and KAN models on battery 10; (c) is a graph showing the SOH estimation error of the method of the present invention on battery 10; and (d) is a comparison of the SOH estimation errors of the method of the present invention, LSTM, GRU-BiLSTM, and KAN models on battery 10.

[0052] Figure 5 This is a schematic diagram of the physical constraint dynamic activation of the battery at different degradation stages in the battery life cycle of the present invention. Among them, (a) is the grayscale image of the physical constraint dynamic activation of battery 11, (b) is the grayscale image of the physical constraint dynamic activation of battery 27, and (c) is the grayscale image of the physical constraint dynamic activation of battery 36. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention.

[0054] like Figure 1 As shown, this invention provides a battery SOH estimation method based on RL-PINN. This method includes data acquisition and extraction of key degradation physical features, construction of a physical information temporal decoding model, construction of a multi-dimensional physical constraint library, construction of a period-aware reinforcement learning controller, dual-track asynchronous optimization training, and SOH estimation and lifecycle stage boundary output. In this invention, a unified definition is used. This serves as a discrete cyclic window index for the battery degradation process, used to simultaneously represent the interaction steps of the reinforcement learning controller and the temporal evolution steps of the physical information, thereby achieving aligned modeling of the reinforcement learning module and the temporal decoding model of the physical information on a unified time scale. The specific steps are as follows:

[0055] Step 1: Obtain real-time monitoring data during the charge-discharge cycle of the lithium-ion battery and extract key degradation physical characteristics;

[0056] Specifically, real-time monitoring data of the lithium-ion battery during charge-discharge cycles is acquired. This real-time monitoring data includes one or more of the following: voltage, current, temperature, charging capacity, discharging capacity, and cycle count. For the [number of cycles]... A charge-discharge cycle window is used, and the voltage, current, temperature, and capacity sequences collected within this cycle window are used as candidate input data.

[0057] Real-time monitoring data undergoes preprocessing, which includes outlier removal, missing value imputation, and data alignment. Outlier removal removes abnormal sampling points caused by transient sensor disturbances or communication anomalies; missing value imputation ensures that all monitored variables have a consistent time length within the same cyclic window; and data alignment ensures that voltage, current, temperature, charging capacity, and discharging capacity correspond within the same cyclic window.

[0058] In the feature selection stage, a random forest algorithm was used to rank the candidate input data by feature importance. The number of decision trees in the random forest algorithm was set to 100, and the Gini coefficient was used as the feature selection criterion. Based on the feature importance ranking results, six key degradation physical features related to battery capacity decay, polarization characteristics, and voltage and current distribution were selected from the candidate input data.

[0059] The six key degradation physical characteristics include constant current charging capacity, constant voltage charging capacity, charging time constant, average voltage, voltage kurtosis, and current skewness. Specifically, constant current charging capacity characterizes the change in usable battery capacity; constant voltage charging capacity characterizes the degree of battery polarization and internal resistance changes; charging time constant characterizes changes in charging kinetics; average voltage characterizes the voltage plateau changes during charge-discharge cycles; voltage kurtosis characterizes changes in voltage distribution morphology; and current skewness characterizes the degree of current distribution imbalance.

[0060] The six key degradation physical features obtained from the screening were normalized using the following formula:

[0061] (1)

[0062] in, Represents the original eigenvalues ​​before normalization. This represents the minimum value of the corresponding feature in the training set. This represents the maximum value of the corresponding feature in the training set. This represents the normalized eigenvalues.

[0063] After normalization, we obtain the first... Key degenerate physical feature vectors corresponding to each cyclic window :

[0064] (2)

[0065] in, Indicates the first The constant current charging capacity of each cycle window, Indicates the first The constant voltage charging capacity of each cycle window, Indicates the first The charging time constant for each cyclic window. Indicates the first Average voltage of each cycle window Indicates the first Voltage kurtosis of each cyclic window Indicates the first The current bias of each cyclic window. The key degraded physical feature vector. As input to the subsequent physical information temporal decoding model.

[0066] Step 2: Construct a physical information temporal decoding model;

[0067] The physical information timing decoding model is used to receive the key degraded physical feature vector obtained in step 1. It extracts long-range temporal dependence features during the battery capacity decay process, outputs the SOH estimate of the current loop window and intermediate state information for constructing the state space of the reinforcement learning controller.

[0068] In one embodiment, the physical information temporal decoding model employs a bidirectional long short-term memory network (BiLSTM-Attention model) that incorporates a scaled dot product attention mechanism. The BiLSTM-Attention model includes an input layer, a bidirectional long short-term memory network layer, an attention layer, and a fully connected output layer.

[0069] The input layer receives a sequence of key degenerate physical features within a continuous cyclic window. :

[0070] (3)

[0071] in, For the first Each cyclic window corresponds to a 6-dimensional key degenerate physical feature vector.

[0072] The bidirectional long short-term memory (LSTM) network layer includes forward LSTM units and backward LSTM units, used to extract temporal dependencies in the battery degradation sequence from the forward and backward time directions, respectively. The bidirectional LSTM network layer consists of two hidden layers, each containing 64 neurons.

[0073] For the The expression for the hidden state output to the LSTM unit in a loop window is:

[0074] (4)

[0075] The expression for the hidden state output by the inverse LSTM unit is:

[0076] (5)

[0077] Concatenate the forward hidden state and the reverse hidden state to obtain the _th_ hidden state. The bidirectional hidden state of a looping window is expressed as follows:

[0078] (6)

[0079] in, Indicates the first The positive hidden state of a looping window. Indicates the first The reverse hidden state of a looping window. This indicates the bidirectional hidden state after splicing; This indicates a feature extraction operation in the forward time direction. This indicates the feature extraction operation in the reverse time direction.

[0080] To enable the BiLSTM-Attention model to focus on key degenerate recurrent windows that significantly impact SOH estimation, a scaled dot product attention mechanism is introduced after the bidirectional long short-term memory network layer. This mechanism calculates attention weights for different recurrent windows using the query vector, key vector, and value vector.

[0081] The formula for calculating the similarity between the query vector and the key vector is:

[0082] (7)

[0083] in, Represents the query vector With key vector Similarity between them Represents the query vector. Represents the key vector. The feature dimension of the key vector. Used to scale the dot product result to avoid the value becoming too large.

[0084] The formula for calculating attention output is:

[0085] (8)

[0086] in, Represents the query matrix. Represents the key matrix, Represents a value matrix, Key matrix transpose, This represents the normalized exponential function.

[0087] By employing this scaled dot product attention mechanism, the BiLSTM-Attention model assigns different weights to different recurrent windows, resulting in an attention-weighted degenerate representation vector. .

[0088] The fully connected output layer receives the attention-weighted degenerate representation vector. It also outputs the estimated SOH value for the current loop window:

[0089] (9)

[0090] in, Indicates the first SOH estimates for each cyclic window, This represents the mapping function for the fully connected output layer.

[0091] The output of the physical information temporal decoding model includes the SOH estimate for the current loop window. And intermediate state information. The intermediate state information includes bidirectional hidden states. Attention weight and the attention-weighted degenerate representation vector .in, Used to calculate data-driven loss and construct physical constraint residuals. Used to characterize bidirectional temporal dependencies in battery degradation sequences. Used to characterize the contribution of different cyclic windows to the current SOH estimation result; It is used to comprehensively characterize the battery degradation state corresponding to the current loop window and serves as the composition information of the subsequent reinforcement learning controller state vector.

[0092] Step 3: Establish a multi-dimensional physical constraint library;

[0093] Multi-dimensional physical constraint library This system unifies the storage of various physical constraint losses during battery SOH estimation, providing a source of constraints for subsequent reinforcement learning controllers to dynamically schedule physical constraints and for constructing composite loss functions. The multi-dimensional physical constraint library includes basic algebraic physical constraints and PDE kinetic constraints, used to impose physical consistency constraints on the SOH estimates output by the physical information temporal decoding model. Specifically, basic algebraic physical constraints define the basic degradation boundary of the SOH estimation results and remain normally activated during training; PDE kinetic constraints describe the diffusion restriction and SEI film growth kinetics during battery degradation and are dynamically activated or dormant by the subsequent reinforcement learning controller based on the battery degradation state.

[0094] The fundamental algebraic physical constraints include the capacity decay monotonicity constraint, the SOH change rate constraint, the negative correlation constraint based on electrochemical impedance, and the coulombic efficiency balance constraint.

[0095] The capacity decay monotonicity constraint is used to constrain the overall non-increasing trend of battery SOH with increasing cycle number, and its residual loss can be expressed as:

[0096] (10)

[0097] in, This represents the capacity decay monotonicity constraint loss. Indicates the first SOH estimates for each cyclic window, Indicates the first SOH estimates for each cyclic window, Used to punish abnormal rises in SOH that do not conform to the monotonicity of capacity decay; This represents the average function.

[0098] SOH change rate constraint is used to limit the SOH change amplitude between adjacent cycle windows, reducing abnormal jumps caused by sensor noise or data disturbance. Its change rate can be expressed as:

[0099] (11)

[0100] in, Indicates the first The rate of change of SOH corresponding to each cycle window This represents the number of iterations between adjacent prediction points. Furthermore, the SOH rate of change constraint loss can be expressed as:

[0101] (12)

[0102] in, This represents the loss due to the constraint of the rate of change of SOH. This indicates the preset maximum allowable rate of change threshold for SOH.

[0103] The negative correlation constraint based on electrochemical impedance is used to constrain the negative correlation between the decrease in SOH and the increase in equivalent impedance. Equivalent impedance According to the first The average voltage and average current within each cyclic window are estimated as follows:

[0104] (13)

[0105] in, Indicates the first The equivalent impedance of each cyclic window Indicates the first Average voltage of each cycle window Indicates the first The average current of each cyclic window. This constraint requires that the SOH estimate and the equivalent impedance maintain a negative correlation, and its constraint loss can be expressed as:

[0106] (14)

[0107] in, This represents the negatively correlated constraint loss based on electrochemical impedance. This represents the correlation coefficient between the SOH estimated sequence and the equivalent impedance sequence. When the two are positively correlated, the corresponding residual is penalized.

[0108] Coulomb efficiency balance constraints are used to ensure that the conversion efficiency of active materials during battery charging and discharging remains within a predetermined physical range. According to the first The discharge capacity and charge capacity within each cycle window are calculated as follows:

[0109] (15)

[0110] in, Indicates the first Coulomb efficiency for a cyclic window This represents the discharge capacity of the t-th cycle window. Indicates the first The charging capacity of each cyclic window. The coulomb efficiency balance constraint requires... Located in the preset range In one embodiment, It is 0.95. The value is 1.0. Its constraint loss can be expressed as:

[0111] (16)

[0112] in, This represents the loss due to the Coulomb efficiency balance constraint.

[0113] PDE kinetic constraints include one-dimensional diffusion equation constraints and SEI film growth kinetic constraints.

[0114] One-dimensional diffusion equation constraints are used to describe the effect of restricted diffusion of active lithium ions within the electrode active material particles on the state of oxygen (SOH) change. If the estimated SOH is considered as an external characterization of the change in the concentration of available active material in the battery, then its change satisfies a simplified one-dimensional diffusion relationship:

[0115] (17)

[0116] in, This represents the effective diffusion coefficient of active lithium ions within the solid phase. The index of the cyclic window representing the battery degradation process. Representing spatial coordinates, in the battery microscopic model, it represents the radial position within the active material particles. The corresponding one-dimensional diffusion equation residual can be expressed as:

[0117] (18)

[0118] The constraint loss of the one-dimensional diffusion equation can be expressed as:

[0119] (19)

[0120] in, This represents the residual of the one-dimensional diffusion equation. This represents the constraint loss in the one-dimensional diffusion equation.

[0121] SEI film growth kinetics constraints are used to describe the process by which continuous SEI film growth leads to the loss of active lithium and subsequent SOH decay. The simplified kinetic relationship can be expressed as:

[0122] (20)

[0123] in, This represents the equivalent kinetic rate constant for SEI film growth. Represents apparent current density. This represents the absolute value of the apparent current density. The corresponding SEI film growth kinetic residual can be expressed as:

[0124] (twenty one)

[0125] The kinetic constraint loss of SEI film growth can be expressed as:

[0126] (twenty two)

[0127] in, This represents the residual in the growth kinetics of the SEI film. This represents the loss of kinetic constraint during SEI film growth.

[0128] Therefore, the multi-dimensional physical constraint library can be represented as:

[0129] (twenty three)

[0130] in, This represents a multi-dimensional physical constraint library. This represents the capacity decay monotonicity constraint loss. This represents the loss due to the constraint of the rate of change of SOH. This represents the negatively correlated constraint loss based on electrochemical impedance. This represents the loss due to the Coulomb efficiency balance constraint. This represents the constraint loss in the one-dimensional diffusion equation. This represents the loss of kinetic constraint during SEI film growth.

[0131] The basic algebraic physical constraint loss remains in constant activation during training, while the PDE dynamic constraint loss dynamically participates in the calculation of the composite loss function based on the physical constraint activation vector output by the reinforcement learning controller.

[0132] Step 4: Construct a periodic-aware reinforcement learning controller;

[0133] The periodic-aware reinforcement learning controller uses the multi-dimensional physical constraint library established in step 3. As a scheduling object, the multi-dimensional physical constraint library is used based on the current degradation state of the battery. The PDE dynamic constraints in the library are dynamically activated or dormant, and the multi-dimensional physical constraint library is also dynamically activated. The loss weights corresponding to each activated physical constraint are adaptively adjusted. The reinforcement learning controller does not directly output the battery SOH estimate, but instead outputs the physical constraint activation vector and physical constraint loss weight vector used to adjust the training process of the physical information temporal decoding model.

[0134] In one embodiment, the reinforcement learning controller employs a proximal policy optimization agent based on an Actor-Critic architecture, namely a PPO agent. The PPO agent includes an Actor network and a Critic network, where the Actor network outputs actions based on the current state, and the Critic network evaluates the state value function of the current state.

[0135] The reinforcement learning controller is built based on the Markov decision process, for the first... The state space, action space, and reward function of the reinforcement learning controller are defined as follows: (The system has a loop window.)

[0136] (a) State space;

[0137] state space Used to characterize the current degradation state of the battery. The state vector of each loop window Including the current SOH estimate, the local evolution gradient of SOH, the currently activated physical constraint topological encoding, and input feature statistics, it is represented as:

[0138] (twenty four)

[0139] in, Indicates the first The state vector of a cyclic window The output of the physical information temporal decoding model represents the first... SOH estimates for each cyclic window, This represents the local evolution gradient of SOH. This represents the degraded representation vector output by the physical information temporal decoding model. This represents the physical constraint activation vector of the previous loop window. The topological encoding of the currently activated physical constraints is derived from the physical constraint activation vector of the previous loop window. express; and Let these represent the mean and standard deviation of the voltage sequence within the current loop window, respectively. and These represent the mean and standard deviation of the current sequence within the current loop window, respectively.

[0140] The formula for calculating the local evolution gradient of SOH is as follows:

[0141] (25)

[0142] in, Indicates the first SOH estimates for each cyclic window.

[0143] (ii) Action space;

[0144] Action space It includes discrete actions and continuous actions. The Actor network adopts a dual-head output structure, including a discrete action head and a continuous action head.

[0145] The discrete action head is used to output the activation or dormancy decision of the PDE dynamic constraints, thus obtaining the physical constraint activation vector. :

[0146] (26)

[0147] in, Indicates the first Physical constraint activation vectors for each cyclic window. This represents the activation state of the constraints in the one-dimensional diffusion equation. This indicates the activation state of the SEI film growth kinetic constraint. When When, the one-dimensional diffusion equation constraints participate in the calculation of the physical constraint loss of the current loop window; when When the one-dimensional diffusion equation constraints are not included in the calculation of the physical constraint loss for the current loop window. When the SEI film growth kinetics constraints are involved in the calculation of the physical constraint loss for the current loop window; when At that time, the SEI film growth kinetic constraints are not included in the calculation of physical constraint loss in the current cycle window.

[0148] The continuous action head is used to output the physical constraint loss weight vector. :

[0149] (27)

[0150] in, Indicates the first The physical constraint loss weight vector for each cyclic window. , , , , and These represent the loss weights corresponding to the capacity decay monotonicity constraint, SOH rate of change constraint, electrochemical impedance negative correlation constraint, coulombic efficiency balance constraint, one-dimensional diffusion equation constraint, and SEI film growth kinetics constraint, respectively.

[0151] In one embodiment, the continuous motion head outputs a Gaussian distribution of the mean vector and standard deviation vector:

[0152] (28)

[0153] The physical constraint loss weight vector is obtained by sampling based on the mean vector and standard deviation vector:

[0154] (29)

[0155] in, This represents the continuous action head of the Actor network. This represents the mean vector of the head output during continuous actions. This represents the standard deviation vector of the output of the continuous action head. The mean is variance is The Gaussian distribution.

[0156] (iii) Reward function;

[0157] The reward function is used to evaluate whether the physical constraint activation vector and physical constraint loss weight vector output by the reinforcement learning controller in the current loop window are conducive to improving the accuracy and physical consistency of SOH estimation. Reward value of each loop window Represented as:

[0158] (30)

[0159] in, Indicates the first The reward value for each loop window. Indicates the first Data-driven loss for each loop window Indicates the first Physical constraint loss for each loop window, This represents the physical constraint activation complexity penalty term. This represents the physical constraint loss adjustment coefficient. This represents the complexity penalty adjustment coefficient.

[0160] Among them, the physical constraint activation complexity penalty term Represented as:

[0161] (31)

[0162] This complexity penalty term This is used to prevent the reinforcement learning controller from indiscriminately activating all PDE dynamic constraints at all lifecycle stages.

[0163] Step 5: Construct a composite loss function;

[0164] Composite loss function with multi-dimensional physical constraint library The physical constraint loss is constructed based on each physical constraint loss in the training process, and the physical constraint activation vector and physical constraint loss weight vector output by the reinforcement learning controller are used to determine the physical constraint terms and their corresponding weights participating in the training within the current loop window.

[0165] A composite loss function is used to train a physical information temporal decoding model. The composite loss function includes a data-driven loss term and a physical constraint loss term, and is jointly modulated by the physical constraint activation vector and the physical constraint loss weight vector output by the reinforcement learning controller.

[0166] The data-driven loss term measures the deviation between the SOH estimate and the true SOH label, and is expressed as:

[0167] (32)

[0168] in, Indicates the first Data-driven loss for each loop window Indicates the first SOH estimates for each cyclic window, Indicates the first The actual SOH label in a loop window.

[0169] The basic algebraic physical constraints remain in constant activation during training, while the PDE dynamics constraints are activated by the physical constraint vectors output by the reinforcement learning controller. Dynamic control. Therefore, the first Physical constraint loss of each loop window Represented as:

[0170] (33)

[0171] in, , , , , and They represent the first The constraints for capacity decay monotonicity, SOH rate of change, electrochemical impedance negative correlation, coulombic efficiency balance, one-dimensional diffusion equation, and SEI film growth kinetics are corresponding to the cyclic window. and These represent the activation states of the one-dimensional diffusion equation constraint and the SEI film growth kinetics constraint, respectively. , , , , and These represent the corresponding physical constraint loss weights.

[0172] No. The total loss function for each loop window is expressed as:

[0173] (34)

[0174] in, Indicates the first The composite loss function corresponding to each loop window is used to update the parameters of the physical information temporal decoding model. Indicates the first Data-driven loss per cyclic window; Indicates the first Physical constraint loss for each cyclic window.

[0175] Step 6: Joint training is performed using a dual-track asynchronous optimization mechanism;

[0176] like Figure 2 As shown, the dual-track asynchronous optimization mechanism includes a continuous learning process of the main network and a periodic update process of the reinforcement learning controller. The continuous learning process of the main network is used to update the parameters of the physical information temporal decoding model according to the composite loss function; the periodic update process of the reinforcement learning controller is used to update the policy parameters of the PPO agent based on the accumulated states, actions, and rewards in the trajectory buffer.

[0177] During the continuous learning process of the main network, the first... Key degenerate physical feature vectors of each loop window Input physical information into a time-series decoding model to obtain SOH estimates. and intermediate state information; construct the state vector of the reinforcement learning controller based on the SOH estimate and intermediate state information. The reinforcement learning controller is based on Output physical constraint activation vector and physical constraint loss weight vector Construct the total loss function And update the physical information timing decoding model parameters through backpropagation.

[0178] During the periodic update process of the reinforcement learning controller, the state vector generated by the interaction between the reinforcement learning controller and the environment is used. Action vectors Reward Value Action probabilities and state values ​​are stored in the trajectory buffer. When the number of accumulated samples in the trajectory buffer reaches a preset threshold, the advantage function is calculated using generalized advantage estimation, and the Actor network and Critic network are updated based on the PPO objective function.

[0179] Action vectors activation vector based on physical constraints and physical constraint loss weight vector Composition, represented as:

[0180] (35)

[0181] in, Indicates the first Action vectors for a loop window.

[0182] In generalized dominance estimation, time-series difference error Represented as:

[0183] (36)

[0184] in, Indicates the first Timing difference error of each cyclic window, Indicates the discount factor. Indicates the state of the Critic network. State value estimation, This represents the parameters of the Critic network.

[0185] The advantage function is expressed as:

[0186] (37)

[0187] in, Indicates the first The odds estimate for each cyclic window, This represents the smoothing parameter for generalized dominance estimation.

[0188] The strategy ratio of PPO is expressed as follows:

[0189] (38)

[0190] in, This represents the ratio of the probabilities of the new and old strategies. This indicates that the current policy network is in the state vector. Output action vector The probability, This indicates that the policy network was in the state vector before the update. Output action vector The probability, Indicates the current parameters of the Actor network. This represents the Actor network parameters before the update.

[0191] The objective function for PPO pruning is expressed as:

[0192] (39)

[0193] in, This represents the PPO pruning objective function. This indicates the expectation of the trajectory samples. This represents the clipping function. This indicates the PPO trimming parameters.

[0194] To prevent the policy update step from being too large, a KL divergence penalty term is further introduced, resulting in the Actor network update objective function:

[0195] (40)

[0196] in, This indicates that the Actor network updates the objective function. This represents the KL divergence penalty coefficient. This represents the KL divergence between the policy before the update and the current policy.

[0197] The loss function of the Critic network is expressed as:

[0198] (41)

[0199] in, This represents the loss function of the Critic network. Indicates the first The cumulative discount return corresponding to each cyclical window.

[0200] Through the above dual-track asynchronous optimization mechanism, the physical information temporal decoding model is continuously updated according to the composite loss function in each training step, while the reinforcement learning controller is periodically updated after accumulating a preset number of samples in the trajectory buffer, thereby realizing the joint optimization of the battery SOH estimation model and the physical constraint dynamic scheduling strategy.

[0201] Step 7: Output the SOH estimation results and lifecycle stage boundary delineation information;

[0202] After completing the joint training of the physical information temporal decoding model and the reinforcement learning controller, the key degradation physical feature vector of the battery to be estimated within the current loop window is input into the trained physical information temporal decoding model to obtain the SOH estimate for the current loop window:

[0203] (42)

[0204] in, Indicates the first SOH estimates for each cyclic window, Indicates the first The key degenerate physical feature vectors corresponding to each cyclic window This represents the physical information temporal decoding model after training. This represents the parameters of the physical information temporal decoding model.

[0205] Based on the SOH estimates from the continuous cyclic window, the battery SOH estimation sequence is obtained:

[0206] (43)

[0207] in, This represents the SOH estimation sequence of the battery over a continuous cycling window.

[0208] Simultaneously, based on the physical constraint activation vectors output by the reinforcement learning controller within the continuous cyclic window, the sequence of physical constraint activation states is obtained:

[0209] (44)

[0210] in, This represents the sequence of physical constraint activation states. Indicates the first The physical constraint activation vector corresponding to each cyclic window.

[0211] According to the physical constraint activation state sequence The changes in the physical constraint activation vector between adjacent loop windows are used to determine the boundary points of the battery life cycle stage. When the physical constraint activation vector changes between adjacent loop windows, and this change remains stable within a preset number of consecutive windows, the corresponding loop window is determined as the boundary point of the life cycle stage. The boundary point of the life cycle stage can be represented as:

[0212] (45)

[0213] in, Indicates the first Each lifecycle stage boundary point, Indicates the first Physical constraint activation vectors for each cyclic window. Indicates the first Physical constraint activation vectors for each cyclic window. This represents the threshold number of consecutive windows used to determine the stability of the active state.

[0214] Based on the aforementioned lifecycle stage boundary points, the battery lifecycle is divided into multiple degradation stages:

[0215] (46)

[0216] in, Represents the set of battery lifecycle stages. Indicates the first Each stage of life cycle degradation This indicates the number of lifecycle stages obtained from the division.

[0217] Furthermore, the battery degradation stages can be explained based on the dominant physical constraints activated in different lifecycle phases. When the coulombic efficiency balance constraint and the SEI film growth kinetics constraint are dominant, it indicates that the battery may be in the stage of interface film formation or enhanced interface side reactions; when the SOH change rate constraint is dominant, it indicates that the battery may be in a relatively stable gradual degradation stage; when the negative correlation constraint based on electrochemical impedance, the one-dimensional diffusion equation constraint, and the SEI film growth kinetics constraint are simultaneously enhanced, it indicates that the battery may enter a stage of increased internal resistance, restricted diffusion, and accelerated capacity decay.

[0218] Finally, the battery SOH estimation method output includes: the SOH estimate for the current loop window. SOH estimation sequence for continuous cyclic windows Lifecycle stage boundary point set Lifecycle stages set The output also includes the activation status of the physical constraints at each stage. These outputs can be sent to the battery management system or a host computer for battery health assessment, lifespan prediction, safety warnings, or charge / discharge strategy optimization.

[0219] Furthermore, in an optional embodiment, an evolutionary optimization algorithm based on Langevin dynamics is used to optimize the key hyperparameters in the RL-PINN framework (a fusion module consisting of a physical information temporal decoding model and a reinforcement learning controller).

[0220] Key hyperparameters include the learning rate, batch size, and hidden layer dimension of the physical information temporal decoding model, as well as the discount factor and PPO pruning coefficient in the reinforcement learning controller. A set of candidate hyperparameters can be represented as a particle's position vector:

[0221] (47)

[0222] in, Indicates the first The hyperparameter position vector of each particle The learning rate represents the time-series decoding model of physical information. Indicates batch size, Indicates the hidden layer dimension. Indicates the discount factor. This represents the PPO trimming factor.

[0223] Define the potential energy function as the root mean square error of the model on the validation set under the corresponding hyperparameter combination:

[0224] (48)

[0225] in, Indicates the first The potential energy function of a particle at its current position. ) indicates the use of The root mean square error obtained on the validation set after training.

[0226] Particle velocity and position are updated according to the Langevin dynamics equations including friction terms:

[0227] (49)

[0228] (50)

[0229] in, Indicates the first The particle in the first Speed ​​at the next iteration Indicates the first The particle in the first Speed ​​at the next iteration Indicates the first The particle in the first The position vector at the next iteration. Indicates the first The particle in the first The position vector at the next iteration. This indicates that Langevin's update step size, Represents the potential energy function at position gradient at, Indicates the coefficient of friction. Indicates the first The temperature parameters corresponding to the next iteration. This represents standard Gaussian white noise. This represents the discrete time step.

[0230] In practical calculations, if the potential energy gradient is difficult to obtain directly, it can be approximated by using finite differences between particles within the population. :

[0231] (51)

[0232] in, This indicates that another particle selected from the population is in the [number]th [phase]. The position vector at the next iteration. This represents the potential energy function value corresponding to the particle.

[0233] After each update, the root mean square error of the validation set for each particle is recalculated, and they are sorted from best to worst according to their fitness. Fitness can be expressed as:

[0234] (52)

[0235] in, Indicates the first The fitness of each particle This represents the smallest positive number that prevents the denominator from being zero.

[0236] An elite retention strategy is employed to retain the highest-fitting particles in the next iteration. The steps of particle velocity update, position update, fitness calculation, and elite retention are repeated until a preset number of iterations is reached or the root mean square error of the verification set converges. The final output is the hyperparameter combination corresponding to the highest-fitting particle.

[0237] (53)

[0238] in, This represents the optimal combination of hyperparameters obtained.

[0239] The RL-PINN framework is trained using the optimal hyperparameter combination to obtain the final physical information temporal decoding model and reinforcement learning controller used for battery SOH estimation.

[0240] like Figure 3As shown, this invention also provides a battery SOH estimation system based on RL-PINN, used to execute the aforementioned battery SOH estimation method based on RL-PINN. The battery SOH estimation system includes a data sensing and processing module, a physical information timing decoding module, a physical constraint library module, an intelligent dynamic scheduling module, a dual-track asynchronous optimization module, and an SOH evaluation and boundary output module.

[0241] The data sensing and processing module acquires real-time monitoring data during the charge-discharge cycle of lithium-ion batteries and performs preprocessing and feature engineering on the real-time monitoring data to obtain key degradation physical feature vectors. The module also performs outlier removal, missing value imputation, and data alignment on the real-time monitoring data, and uses a random forest algorithm to rank candidate features by feature importance, thereby selecting six key degradation physical features: constant current charging capacity, constant voltage charging capacity, charging time constant, average voltage, voltage kurtosis, and current skewness.

[0242] The physical information temporal decoding module is connected to the data sensing and processing module. It receives key degradation physical feature vectors and extracts long-range temporal dependency features during battery capacity decay. The physical information temporal decoding module includes an input layer, a bidirectional long short-term memory (LSTM) network layer, an attention layer, and a fully connected output layer. The bidirectional LSM network layer extracts temporal dependencies in the battery degradation sequence from both forward and reverse time directions. The attention layer assigns attention weights based on the contribution of different recurrent windows to the SOH estimation. The fully connected output layer outputs the SOH estimate for the current recurrent window. The physical information temporal decoding module also outputs bidirectional hidden states, attention weights, and attention-weighted degradation representation vectors, serving as composition information for the state space of the subsequent intelligent dynamic scheduling module.

[0243] The physical constraint library module connects to the physical information temporal decoding module to store and calculate the residual loss corresponding to multi-dimensional physical constraints. These multi-dimensional physical constraints include basic algebraic physical constraints and PDE kinetic constraints. Basic algebraic physical constraints include capacity decay monotonicity constraints, SOH rate of change constraints, negative correlation constraints based on electrochemical impedance, and coulombic efficiency balance constraints. PDE kinetic constraints include one-dimensional diffusion equation constraints and SEI film growth kinetic constraints. Basic algebraic physical constraints remain normally active during training, while PDE kinetic constraints are dynamically activated or dormant based on the physical constraint activation vector output by the intelligent dynamic scheduling module.

[0244] The intelligent dynamic scheduling module connects to the physical information timing decoding module and the physical constraint library module, and is used to output physical constraint activation vectors and physical constraint loss weight vectors based on the current degradation state of the battery. The intelligent dynamic scheduling module includes a PPO agent based on an Actor-Critic architecture. The Actor network of the PPO agent includes discrete action heads and continuous action heads; the discrete action heads are used to output activation or dormancy decisions for PDE dynamic constraints; the continuous action heads are used to output the physical constraint loss weights corresponding to each physical constraint module. The Critic network of the PPO agent is used to estimate the state value function of the current state and provide value feedback for the policy update of the Actor network.

[0245] The dual-track asynchronous optimization mechanism is used in the physical information temporal decoding module and the intelligent dynamic scheduling module to achieve continuous updates of the physical information temporal decoding model and periodic updates of the reinforcement learning controller. During the continuous update of the physical information temporal decoding model, the dual-track asynchronous optimization mechanism constructs a composite loss function based on data-driven loss and physical constraint loss, and updates the physical information temporal decoding model parameters according to the composite loss function. During the periodic update of the reinforcement learning controller, the dual-track asynchronous optimization mechanism stores the state, action, reward, and next state in the trajectory buffer, and after the number of trajectory samples reaches a preset threshold, it updates the reinforcement learning controller parameters using generalized advantage estimation and the PPO objective function.

[0246] The SOH assessment and boundary output module connects to the physical information timing decoding module and the intelligent dynamic scheduling module. Based on the physical constraint information provided by the physical constraint library module, it performs lifecycle stage analysis and outputs the SOH estimate for the current loop window, the SOH estimate sequence for consecutive loop windows, the set of lifecycle stage boundary points, the set of lifecycle stages, and the physical constraint activation status corresponding to each stage. The SOH assessment and boundary output module also sends the above output results to the battery management system or host computer for battery health status assessment, lifecycle status analysis, safety warnings, or charge / discharge strategy optimization.

[0247] In one embodiment, the battery SOH estimation system can be deployed in the edge computing unit of the battery management system, the vehicle controller, the energy storage system monitoring platform, or a cloud server. The data sensing and processing module can collect voltage, current, temperature, and capacity data from the battery management system; the SOH assessment and boundary output module can feed back the SOH estimation results and life cycle stage boundary information to the battery management system to assist in the execution of charge and discharge strategy adjustments, maintenance decisions, and safety warnings.

[0248] To verify the effectiveness of the battery SOH estimation method and system based on RL-PINN described in this invention, in one embodiment, experimental verification was performed using 10-cycle aging data of the battery. It should be understood that the following experimental verification is only intended to illustrate that this invention can achieve the corresponding technical effects and is not intended to limit the scope of protection of this invention.

[0249] In this embodiment, monitoring data such as voltage, current, temperature, capacity, and cycle number of the battery 10 during charge-discharge cycles are acquired. After preprocessing and feature engineering, six key degradation physical features are extracted: constant current charging capacity, constant voltage charging capacity, charging time constant, average voltage, voltage kurtosis, and current skewness. These six key degradation physical features are input into a trained physical information time-series decoding model, which outputs the battery SOH estimate and degradation characterization vector. Furthermore, a reinforcement learning controller outputs a physical constraint activation vector and a physical constraint loss weight vector based on the current degradation state of the battery, and constructs a composite loss function based on the physical constraint activation vector and the physical constraint loss weight vector.

[0250] like Figure 4 As shown, the SOH estimation results and errors of the method of this invention are compared with those of LSTM, GRU-BiLSTM, and KAN models. Among them, Figure 4 (a) shows the SOH estimation results of the method of the present invention on battery 10. Figure 4 (b) shows a comparison of the SOH estimation performance of the method of the present invention with that of comparative models such as LSTM, GRU-BiLSTM, and KAN. Figure 4 (c) in the figure shows the SOH estimation error of the method of the present invention. Figure 4 (d) shows a comparison of the SOH estimation error between the method of the present invention and comparative models such as LSTM, GRU-BiLSTM, and KAN.

[0251] Experimental results show that the method of this invention can dynamically adjust the activation state and loss weight of physical constraints according to the current degradation state of the battery, so that the SOH estimation results maintain good continuity and physical consistency in long-cycle processes. Compared with the comparative model that relies solely on data-driven mapping, the method of this invention, by introducing a dynamic physical constraint scheduling mechanism, can reduce non-physical estimation biases caused by data fluctuations in some cycle stages and improve the stability of the SOH estimation process.

[0252] Furthermore, the physical constraint activation vector and physical constraint loss weight vector output by the reinforcement learning controller are analyzed. For example... Figure 5 As shown, Figure 5The shades of gray in the graph represent the activation intensity or loss weight of different physical constraints within the corresponding loop window. A higher gray value indicates a greater activation intensity or loss weight for the corresponding physical constraint. The white dashed line represents the boundary of the lifecycle stage determined based on the changes in the activation state of the physical constraints. Figure 5 In the diagram, (a), (b), and (c) represent the dynamic activation thermograms of different batteries under physical constraints. Figure 5 It can be seen that in the early stage of battery cycling, the activation intensity or weight of the coulombic efficiency balance constraint and the SEI film growth kinetics constraint is relatively high, which is used to characterize the interfacial film formation and early side reaction processes; in the middle stage of battery cycling, the activation intensity or weight of the SOH change rate constraint is relatively high, which is used to characterize the relatively stable gradual degradation process; in the late stage of battery life, the activation intensity or weight of the electrochemical impedance negative correlation constraint, the one-dimensional diffusion equation constraint and the SEI film growth kinetics constraint is relatively high, which is used to characterize the internal resistance increase, diffusion restriction and capacity accelerated decay process.

[0253] As verified by the above experiments, the method of the present invention enables physical constraints to be dynamically adjusted with the changes in battery life cycle stages through a closed-loop mechanism of state perception, constraint scheduling, weight adjustment and SOH output, thereby improving the physical consistency of SOH estimation results and stability under long-term degradation scenarios.

Claims

1. A battery SOH estimation method based on RL-PINN, characterized in that, Includes the following steps: S1, Perform feature engineering on the real-time monitoring data acquired during the charge-discharge cycle of the lithium-ion battery to obtain the key degradation physical feature vector; S2, input the key degradation physical feature vector into the physical information time-series decoding model, extract the long-range time-series dependent features in the battery capacity decay process, and output the SOH estimate and battery degradation state characterization information of the current loop window. S3 is a multi-dimensional physical constraint library built on basic algebraic physical constraints and PDE dynamic constraints, used to uniformly store various physical constraint losses. S4. A state vector is constructed from the SOH estimate, the local evolution gradient of SOH, the topological encoding of the currently activated physical constraints, the battery degradation state characterization information, and the statistical information of the voltage and current sequences. The physical constraint activation vector and the physical constraint loss weight vector are used as outputs to construct a reinforcement learning controller. The reinforcement learning controller dynamically activates or suspends the PDE dynamic constraints in the multi-dimensional physical constraint library according to the current degradation state of the battery, and adaptively adjusts the physical constraint loss weights corresponding to the activated physical constraints. S5, a composite loss function is constructed from the physical constraint activation vector and the physical constraint loss weight vector to train the physical information temporal decoding model; the composite loss function selects the physical constraint residuals participating in the training according to the current battery degradation state and assigns corresponding weights; S6, The physical information temporal decoding model and the reinforcement learning controller are jointly trained using a dual-track asynchronous optimization mechanism; S7 inputs the key degradation physical feature vector of the battery to be estimated into the trained physical information time-series decoding model and outputs the battery SOH estimation result; and obtains the battery life cycle stage boundary division information based on the physical constraint activation vector output by the reinforcement learning controller.

2. The battery SOH estimation method based on RL-PINN according to claim 1, characterized in that, In step S1, the real-time monitoring data includes one or more of the following: voltage, current, temperature, charging capacity, discharging capacity, and number of cycles. The preprocessed real-time monitoring data were sorted by feature importance using the random forest algorithm to obtain key degradation physical features, including constant current charging capacity, constant voltage charging capacity, charging time constant, average voltage, voltage kurtosis, and current skewness. The key degradation physical features are normalized to obtain the key degradation physical feature vector corresponding to the t-th loop window. : ， in, Indicates constant current charging capacity. Indicates constant voltage charging capacity. Indicates the charging time constant. Indicates average voltage. Indicates voltage kurtosis. Indicates the current deflection.

3. The battery SOH estimation method based on RL-PINN according to claim 1, characterized in that, The physical information temporal decoding model adopts a bidirectional long short-term memory network with a scaling dot product attention mechanism, including an input layer, a bidirectional long short-term memory network layer, an attention layer, and a fully connected output layer; The bidirectional long short-term memory network layer is used to output the bidirectional hidden state. The forward hidden state, the reverse hidden state, and the bidirectional hidden state are represented as follows: ， ， ， in, Indicates the first The positive hidden state of a looping window. Indicates the first The reverse hidden state of a looping window. This indicates the bidirectional hidden state after splicing; This indicates a feature extraction operation in the forward time direction. This indicates a feature extraction operation performed in the reverse time direction; The attention layer calculates the attention weights for different loop windows using the query vector, key vector, and value vector, and outputs the attention-weighted degradation representation vector. The fully connected output layer receives the attention-weighted degenerate representation vector. Output the SOH estimate for the current loop window; where the degradation characterization vector is... Used to comprehensively characterize the battery degradation state corresponding to the current cycle window.

4. The battery SOH estimation method based on RL-PINN according to claim 1, characterized in that, In step S3, the basic algebraic physical constraints include capacity decay monotonicity constraints, SOH change rate constraints, negative correlation constraints based on electrochemical impedance, and coulombic efficiency balance constraints. The PDE kinetic constraints include one-dimensional diffusion equation constraints and SEI film growth kinetic constraints. The one-dimensional diffusion equation constraint is used to describe the effect of restricted diffusion of active lithium ions within the electrode active material particles on the SOH variation, and to estimate the SOH value. Considered as an external characterization of changes in the concentration of usable active materials in a battery, its expression is: ， The residuals constrained by the one-dimensional diffusion equation are expressed as: ， in, This represents the residual of the one-dimensional diffusion equation. This represents the effective diffusion coefficient of active lithium ions within the solid phase. The index of the cyclic window representing the battery degradation process. Represents spatial location coordinates; The SEI film growth kinetics constraint is used to describe the process by which continuous SEI film growth leads to the loss of active lithium and the decay of SOH, and its expression is as follows: ， The residual of the SEI film growth kinetic constraint is then expressed as: ， in, This represents the residual in the growth kinetics of the SEI film. This represents the equivalent kinetic rate constant for SEI film growth. Represents apparent current density. Represents the absolute value of the apparent current density; Therefore, the multi-dimensional physical constraint library is represented as: ， in, This represents a multi-dimensional physical constraint library. This represents the capacity decay monotonicity constraint loss. This represents the loss due to the constraint of the rate of change of SOH. This represents the negatively correlated constraint loss based on electrochemical impedance. This represents the loss due to the Coulomb efficiency balance constraint. This represents the constraint loss in the one-dimensional diffusion equation. This represents the loss of kinetic constraint during SEI film growth.

5. The battery SOH estimation method based on RL-PINN according to claim 1, characterized in that, In step S4, the reinforcement learning controller adopts a PPO agent based on the Actor-Critic architecture, wherein the Actor network is used to output actions according to the current state, and the Critic network is used to evaluate the state value function of the current state. No. The state vector of each loop window The expression is: ， in, The output of the physical information temporal decoding model represents the first... SOH estimates for each cyclic window, This represents the local evolution gradient of SOH. This represents the attention-weighted degenerate representation vector output by the attention layer; This represents the physical constraint activation vector of the previous loop window, used to represent the topology code of the currently activated physical constraints; , Let these represent the mean and standard deviation of the voltage sequence within the current loop window, respectively. , These represent the mean and standard deviation of the current sequence within the current loop window, respectively. The formula for calculating the local evolution gradient of SOH is as follows: ， in, Indicates the first SOH estimates for each cyclic window; The Actor network includes a discrete action head and a continuous action head. The discrete action head is used to output the physical constraint activation vector, and the continuous action head is used to output the physical constraint loss weight vector. The physical constraint activation vector is represented as follows: ， in, Indicates the first Physical constraint activation vectors for each cyclic window. This represents the activation state of the constraints in the one-dimensional diffusion equation. Indicates the activation state of the SEI film growth kinetic constraint; when When, the one-dimensional diffusion equation constraints participate in the calculation of the physical constraint loss of the current loop window; when When the one-dimensional diffusion equation constraints are not included in the calculation of the physical constraint loss of the current loop window; when When the SEI film growth kinetics constraints are involved in the calculation of the physical constraint loss for the current loop window; when At that time, the SEI film growth kinetic constraints are not included in the calculation of physical constraint loss in the current loop window; The physical constraint loss weight vector Represented as: ， in, , , , , , These represent the physical constraint loss weights corresponding to the capacity decay monotonicity constraint, SOH change rate constraint, negative correlation constraint based on electrochemical impedance, coulombic efficiency balance constraint, one-dimensional diffusion equation constraint, and SEI film growth kinetics constraint, respectively. The reward function is used to evaluate whether the physical constraint activation vector and physical constraint loss weight vector output by the reinforcement learning controller in the current loop window are conducive to improving the accuracy and physical consistency of SOH estimation. Reward value of each loop window Represented as: ， in, Indicates the first The reward value for each loop window. Indicates the first Data-driven loss for each loop window Indicates the first Physical constraint loss for each loop window, This represents the physical constraint loss adjustment coefficient. This represents the complexity penalty adjustment coefficient; This represents the physical constraint activation complexity penalty term. .

6. The battery SOH estimation method based on RL-PINN according to claim 5, characterized in that, In step S5, the composite loss function includes a data-driven loss term and a physical constraint loss term, and is jointly adjusted by the physical constraint activation vector and the physical constraint loss weight vector output by the reinforcement learning controller. The data-driven loss term is used to measure the deviation between the SOH estimate and the true SOH label, and is expressed as follows: ， in, Indicates the first Data-driven loss for each loop window Indicates the first SOH estimates for each cyclic window, Indicates the first The actual SOH label in a loop window; During training, the basic algebraic physical constraints remain in constant activation, while the PDE dynamic constraints are activated by the physical constraint activation vectors output by the reinforcement learning controller. Dynamic control; then the first Physical constraint loss of each loop window Represented as: ， in, , , , , and They represent the first The constraints for capacity decay monotonicity, SOH change rate, electrochemical impedance negative correlation, coulombic efficiency balance, one-dimensional diffusion equation, and SEI film growth kinetics are corresponding to the cyclic window. , , , , and These represent the corresponding physical constraint loss weights; No. The total loss function for each loop window is expressed as: ， in, This represents the total loss function used to update the parameters of the temporal decoding model for physical information.

7. The battery SOH estimation method based on RL-PINN according to claim 1, characterized in that, In step S6, the dual-track asynchronous optimization mechanism includes a continuous update process for the physical information temporal decoding model and a periodic update process for the reinforcement learning controller; During the continuous update of the physical information temporal decoding model, the first... Key degenerate physical feature vectors of each loop window Input physical information into a time-series decoding model to obtain SOH estimates. and intermediate state information; the intermediate state information includes bidirectional hidden states. Attention weight and the attention-weighted degenerate representation vector The state vector of the reinforcement learning controller is constructed based on the SOH estimate and intermediate state information. ; The reinforcement learning controller is based on the state vector Output physical constraint activation vector and physical constraint loss weight vector Construct the total loss function And update the physical information temporal decoding model parameters through backpropagation; During the periodic update process of the reinforcement learning controller, the state vector generated by the interaction between the reinforcement learning controller and the environment is used. Action vectors Reward Value Action probabilities and state values ​​are stored in the trajectory buffer. When the number of accumulated samples in the trajectory buffer reaches a preset threshold, the advantage function is calculated using generalized advantage estimation, and the Actor network and Critic network are updated based on the PPO objective function.

8. The battery SOH estimation method based on RL-PINN according to claim 7, characterized in that, The action vector activation vector based on physical constraints and physical constraint loss weight vector Composition, represented as: ， in, Indicates the first Action vectors for each loop window; In generalized dominance estimation, time-series difference error Represented as: ， in, Indicates the first Timing difference error of each cyclic window, Indicates the discount factor. This indicates that the Critic network has a state vector. State value estimation, Indicates the parameters of the Critic network; Indicates the first The reward value for each cyclic window; The advantage function is expressed as: ， in, Indicates the first The odds estimate for each cyclic window, This represents the smoothing parameter for generalized dominance estimation; The strategy ratio of PPO is expressed as follows: ， in, This represents the ratio of the probabilities of the new and old strategies. This indicates that the current policy network is in the state vector. Output action vector The probability, This indicates that the policy network was in the state vector before the update. Output action vector The probability, Indicates the current parameters of the Actor network. This represents the Actor network parameters before the update; The expression for the PPO objective function is as follows: ， in, This represents the PPO pruning objective function. This indicates the expectation of the trajectory samples. This represents the ratio of the probability of the current strategy to the probability of the old strategy in terms of actions. This represents the clipping function. This represents the PPO trimming factor.

9. The battery SOH estimation method based on RL-PINN according to claim 7, characterized in that, During training, an evolutionary optimization algorithm based on Langevin dynamics was used to optimize key hyperparameters. The key hyperparameters include the learning rate, batch size, and hidden layer dimension of the physical information temporal decoding model, as well as the discount factor and PPO pruning coefficient in the reinforcement learning controller. Represent a set of candidate hyperparameters as particle position vectors: ， in, Indicates the first The hyperparameter position vector of each particle The learning rate represents the time-series decoding model of physical information. Indicates batch size, Indicates the hidden layer dimension. Indicates the discount factor. Indicates the PPO cutting factor; And the root mean square error of the validation set is used as the potential function: ， in, Indicates the first The potential energy function of each particle. Indicates adoption The root mean square error obtained on the validation set after training; By updating the particle velocity and position using the Langevin dynamics equations, the optimal combination of hyperparameters is obtained: ， in, This represents the optimal combination of hyperparameters; The physical information temporal decoding model and reinforcement learning controller are trained using the optimal hyperparameter combination to obtain the final physical information temporal decoding model and reinforcement learning controller used for battery SOH estimation.

10. A battery SOH estimation system based on RL-PINN, used to execute the battery SOH estimation method based on RL-PINN according to any one of claims 1-9, characterized in that, include: The data sensing and processing module is used to acquire real-time monitoring data during the charge and discharge cycle of lithium-ion batteries and extract key degradation physical characteristics. The physical information timing decoding module is used to receive the key degradation physical features, extract long-range timing dependence features in the battery capacity decay process, and output SOH estimates and battery degradation state characterization information. The physical constraint library module stores basic algebraic physical constraints and PDE dynamic constraints, and generates corresponding physical constraint residual losses. The intelligent dynamic scheduling module is used to output the physical constraint activation vector and the physical constraint loss weight vector according to the current degradation state of the battery; The SOH assessment and boundary output module is used to output battery SOH estimation results and life cycle stage boundary delineation information. In the continuous update process of the physical information temporal decoding model, the dual-track asynchronous optimization mechanism constructs a composite loss function based on data-driven loss and physical constraint loss, and updates the parameters of the physical information temporal decoding model according to the composite loss function. In the periodic update process of the reinforcement learning controller, the dual-track asynchronous optimization mechanism stores the state, action, reward and next state into the trajectory buffer, and after the number of trajectory samples reaches a preset threshold, it updates the parameters of the reinforcement learning controller using generalized advantage estimation and PPO objective function.

Images