Lithium ion battery low-temperature preheating charging segmentation optimization method and system based on reinforcement learning

Abstract

The application relates to a lithium ion battery low-temperature preheating charging segmentation optimization method and system based on reinforcement learning, which comprises the following steps: establishing a thermal-electric coupling model of a battery, and establishing state transition expressions of voltage, temperature and SOC; when the battery temperature is lower than a preset switching temperature, starting a high-frequency pulse preheating optimization process based on reinforcement learning; when the battery temperature reaches the preset switching temperature, entering a heating charging process based on reinforcement learning, and adaptively switching among three modes of heating, charging and simultaneous heating and charging, so as to realize safe and rapid charging at low temperature. Through deep reinforcement learning technology, the preheating and charging stages are segmented and optimized, the parameters in the preheating and charging process are accurately controlled, rapid preheating and efficient charging are realized, battery damage is reduced, and the service life of the battery is significantly prolonged.

Description

Technical Field

[0001] This invention relates to a segmented optimization method and system for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning, belonging to the field of low-temperature battery charging technology. Background Technology

[0002] At low temperatures, batteries exhibit low usable capacity and poor charge / discharge performance, with this degradation becoming particularly pronounced below freezing. Therefore, non-destructive and rapid heating of batteries in low-temperature environments is a key approach to addressing this issue. Since battery degradation at low temperatures is a reversible process, restoring performance by raising the battery temperature to room temperature is the mainstream solution for addressing its poor low-temperature performance.

[0003] AC heating methods utilize charging currents of specific frequency and amplitude to heat the battery from the inside out, offering advantages such as rapid heating, high efficiency, and uniform heating distribution. In particular, the alternating lithium insertion and deintercalation within a single AC cycle effectively prevents lithium plating and avoids irreversible damage to battery capacity, making it a promising approach. Therefore, existing low-temperature charging methods typically employ heating followed by constant current / constant voltage charging or alternating heating and charging. While this strategy can reduce the damage to battery life caused by low-temperature charging to some extent, it is time-consuming, and maintaining the optimal charging temperature during the charging phase is difficult. Therefore, achieving segmented current optimization during low-temperature preheating and charging is crucial for rapid, lossless charging. Furthermore, achieving the fastest charging speed and minimum loss when coupling the heating and charging processes presents a challenging problem. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a reinforcement learning-based method for optimizing segmented preheating charging of lithium-ion batteries at low temperatures.

[0005] This invention establishes preheating optimization and heating-charging optimization based on deep reinforcement learning. In the preheating stage, the pulse and amplitude of the high-frequency pulsed current are used as optimization variables to maximize heat generation. In the heating-charging stage, an improved algorithm using hybrid action space solving is employed to select from three modes: "heating," "heating-charging," and "charging." The charging rate and heating loss weights are adjusted according to user needs, optimizing the parameters of the selected mode to achieve lossless fast charging. This method is expected to solve the problems of charging in low-temperature environments in existing technologies, improve charging efficiency, and extend battery life.

[0006] This invention provides a segmented optimization system for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning.

[0007] Terminology Explanation:

[0008] 1. TD3, short for Twin Delayed Deep Deterministic policy gradient algorithm, is a reinforcement learning algorithm based on DDPG (Deep Deterministic Policy Gradient). It aims to address the problem of overestimating Q-values ​​that DDPG may encounter during training. TD3 improves the algorithm's stability and performance by introducing two evaluation networks, a delayed policy update network, and target policy smoothing regularization.

[0009] 2. PDQN, or Parametrized Deep Q-Networks Learning, effectively solves the reinforcement learning problem in mixed action spaces by combining deep neural networks with parameterization techniques. It optimizes the learning process and improves decision-making efficiency and stability in complex environments.

[0010] The technical solution of this invention is as follows:

[0011] A reinforcement learning-based optimization method for segmented preheating charging of lithium-ion batteries includes:

[0012] Establish a thermoelectric coupling model for the battery, and develop state transition expressions for voltage, temperature, and SOC.

[0013] When the battery temperature is lower than the preset switching temperature, a high-frequency pulse preheating optimization process based on reinforcement learning is initiated; when the battery temperature reaches the preset switching temperature, a heating and charging process based on reinforcement learning is initiated.

[0014] According to a preferred embodiment of the present invention, the thermoelectric coupling model of the battery is as follows:

[0015]

[0016] Where m represents the mass of the battery; C p The specific heat capacity of the battery is represented by t; time is represented by Q; the heat generation rate of the battery is represented by h; the heat transfer coefficient of the battery is represented by S; the surface area of ​​the battery is represented by T; and the ambient temperature is represented by T0.

[0017] According to a preferred embodiment of the present invention, the state transition expressions for voltage, temperature, and SOC are as follows:

[0018] The voltage of a lithium-ion battery is calculated as follows:

[0019] U O =U OCV +I ac ·Z total (T,f,SOC)

[0020] Among them, UO It is the output voltage of a lithium-ion battery; U OCV It is the OCV of lithium-ion batteries; I ac It is the peak-to-peak value of the AC component; Z total The total AC impedance at the current moment is obtained through interpolation;

[0021] The heat production rate Q during the preheating stage is calculated as follows:

[0022]

[0023] Among them, Z Re is the real part of the AC impedance at the current moment, obtained by interpolation; SOC is defined as the ratio of the remaining capacity to the capacity in the fully charged state; f represents the frequency of the AC component.

[0024] Obtain the temperature T of the lithium-ion battery at time k+1. k+1 :

[0025]

[0026] Among them, T s A time interval representing temperature;

[0027] The battery's state of charge (SOC) is obtained through coulomb counting, as shown below:

[0028]

[0029] Among them, SOC int It is the initial SOC of the battery, Q bat I is the actual usable capacity of the battery, and I is the current flowing through the battery, which is positive when charging and negative when discharging.

[0030] According to a preferred embodiment of the present invention, in the high-frequency pulse preheating optimization process based on reinforcement learning, the network parameters of the reinforcement learning model are updated using a TD3 model. The TD3 model includes two evaluation networks and one action network. The evaluation network is used to evaluate the value of actions, and the action network is used to select actions. The action network outputs the optimal pulse frequency as the action based on the current voltage, temperature, and SOC state, according to the reward function. Impedance information is obtained through interpolation, and combined with the calculated heat generation rate Q, state transition is performed through the established state transition expressions for voltage, temperature, and SOC. Through continuous interaction with the environmental state, the TD3 model agent learns the optimal preheating strategy, which guides the selection of pulse frequency to achieve the predetermined preheating effect. The entire optimization process is a cyclic operation until the preset number of iterations is reached.

[0031] In a further optimized high-frequency pulse preheating process based on reinforcement learning, the state space s(t) is defined as:

[0032] s(t)={T(t)}

[0033] Where T(t) represents the battery temperature and t represents the time step;

[0034] Define the action space a(t) as:

[0035] a(t)={A(t),f(t)}

[0036] Where A(t) represents the amplitude of the high-frequency pulse current waveform, and f(t) represents the frequency of the high-frequency pulse current waveform;

[0037] Set the reward function r(t) as follows:

[0038] r(t)=ω1r temp (t)+ω2r targ (t)+ω3r volt (t)

[0039] Where ω1 to ω3 are the weights of each part of the reward or penalty; r temp (t) and r targ (t) represents the target temperature difference penalty and the heating end reward, respectively, r volt (t) represents the voltage over-limit penalty;

[0040] r temp (t) and r targ The expressions for (t) are as follows:

[0041] r temp (t)=T(t)-T target

[0042]

[0043] Among them, T target R is the preset target temperature, and R is the bonus value for the end of heating.

[0044] r volt The expression for (t) is:

[0045]

[0046] Among them, U limit U(t) is the charging cutoff voltage, and U(t) is the current battery terminal voltage.

[0047] According to a preferred embodiment of the present invention, in the reinforcement learning-based heating and charging process,

[0048] Establish the action space a(t)′ as follows:

[0049] a(t)′={(k,x k|k∈[1,2,3],x k ∈[I ac (t),f(t),I dc (t)]}

[0050] Where k represents different modes, x k Representing the action space, the correspondence between discrete actions and continuous actions is as follows:

[0051]

[0052] Where k=1 indicates the selection of the heating mode, and the associated parameter is the pulse amplitude I. ac (t) and frequency f(t); k=2 indicates the selection of heating and charging mode, and the associated parameters are pulse frequency f(t) and AC component amplitude I. ac (t) and DC component amplitude I dc (t); k=3 indicates the selection of the charging mode, and the only parameter associated with it is the DC current amplitude I. dc (t);

[0053] Define the state space s(t)′ as:

[0054] s(t)′={T(t),SOC(t)}

[0055] Where T(t) represents the battery temperature and SOC(t) represents the battery state of charge;

[0056] State transitions are performed using established state transition expressions based on voltage, temperature, and SOC.

[0057] The calculation of the heat production rate Q depends on the mode selected at the current moment:

[0058]

[0059] Among them, Z Re and Z tot The real part of the AC impedance and the total impedance at the current moment are obtained through interpolation; R i The current ohmic impedance is obtained through interpolation; T represents the battery temperature, f represents the frequency of the alternating current, and SOC refers to the battery's state of charge.

[0060] The reward function is set as follows:

[0061]

[0062] Wherein, α1 to α4 represent the weights of each part of the reward or punishment; r SOC (t) and r Qac (t) represents the penalty for the difference in target SOC and the penalty for AC heat generation, respectively.time (t) represents the charging time penalty, r volt (t) and r dclim (t) represents the voltage over-limit penalty and the DC component over-limit penalty, respectively;

[0063] r SOC (t) and r Q_ac The expressions for (t) are as follows:

[0064] r SOC (t)=SOC(t)-SOC target

[0065]

[0066] Among them, SOC target For the set SOC target value, the AC heat generation penalty is a penalty related to the AC component heat generation when selecting heating or heating-charging mode. K is the scaling factor to ensure that the SOC difference penalty and the AC heat generation penalty are on the same order of magnitude.

[0067] r time The expression for (t) is:

[0068] r time (t)=-(1-e -tT )

[0069] Where T is the maximum charging time set. As charging progresses, this reward gradually increases, allowing you to complete the charging goal faster.

[0070] r volt (t) and r dclim (t) represent the voltage over-limit penalty and the DC component over-limit penalty, respectively, and their expressions are:

[0071]

[0072]

[0073] I dc_lim (T t SOC t )=[U limit -U ocv (T t SOC t )] / R i (T t SOC t )

[0074] Among them, U limit This is the charging cutoff voltage; if the current terminal voltage is greater than the cutoff voltage, a penalty equal to the difference between the cutoff voltage and the current terminal voltage is applied; Idc_lim If the DC boundary is set and the selected heating / charging or charging mode is selected, and the DC component is greater than the DC boundary, a penalty will be imposed.

[0075] According to a preferred embodiment of the present invention, the reinforcement learning-based heating and charging process includes:

[0076] The initial state information of the current lithium battery is collected and input into the target policy network model; the target policy network model is the PDQN model.

[0077] At each time step, the PDQN model receives the current state as input and outputs the Q-value of each possible action; the agent selects an action to execute based on the Q-value and provides feedback based on the current state.

[0078] The PDQN model updates its network parameters using a gradient descent optimization algorithm based on the reward value calculated from the reward function and the new state information. This process continues as the agent interacts with the environmental state, accumulates experience, and gradually improves the heating and charging strategy through updates to the PDQN model. When the preset number of iterations is met, the PDQN model has learned a better Q-value function.

[0079] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a reinforcement learning-based method for optimizing segmented preheating charging of lithium-ion batteries at low temperatures.

[0080] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a reinforcement learning-based method for segmented optimization of low-temperature preheating charging of lithium-ion batteries.

[0081] A reinforcement learning-based segmented optimization system for low-temperature preheating charging of lithium-ion batteries includes:

[0082] The module for establishing the thermoelectric coupling model is configured to: establish the thermoelectric coupling model of the battery and establish state transition expressions for voltage, temperature, and SOC;

[0083] The preheating and charging segment optimization module is configured to: initiate a high-frequency pulse preheating optimization process based on reinforcement learning when the battery temperature is lower than the preset switching temperature; and enter a heating and charging process based on reinforcement learning when the battery temperature reaches the preset switching temperature.

[0084] The beneficial effects of this invention are as follows:

[0085] This invention proposes a solution with significant advantages and innovations in low-temperature charging technology for lithium-ion batteries, specifically including the following advantages:

[0086] 1. Improve charging efficiency and extend battery life: Through deep reinforcement learning technology, the preheating and charging stages are optimized in segments, and the parameters in the preheating and charging process are precisely controlled to achieve rapid preheating and efficient charging, while reducing battery damage and significantly extending battery life.

[0087] 2. Intelligent and widely applicable: The optimization strategy of this invention is based on deep reinforcement learning, which has strong intelligence and adaptability. The proposed strategy has the ability to adjust the charging rate and heating loss weight according to user needs, realize personalized charging, and has broad application value.

[0088] 3. Solving the problem of low-temperature charging: This invention is specifically optimized for low-temperature environments, effectively solving the problem of low charging efficiency at low temperatures and improving the reliability and safety of charging.

[0089] In summary, this invention significantly improves the charging efficiency and extends the lifespan of lithium-ion batteries through deep reinforcement learning technology, and possesses both intelligence and wide applicability. It is particularly optimized for low-temperature environments, solving the problem of low-temperature charging and providing strong support for the advancement of battery charging technology, demonstrating significant application prospects and technological advantages. Attached Figure Description

[0090] Figure 1 A flowchart illustrating the segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning.

[0091] Figure 2(a) is a schematic diagram showing the comparison of preheating pulse amplitude under different SOCs using the method of the present invention;

[0092] Figure 2(b) is a schematic diagram showing the frequency comparison results of the method of the present invention under different SOCs;

[0093] Figure 3(a) is a schematic diagram showing the comparison of battery temperatures under different switching temperatures using the method of the present invention;

[0094] Figure 3(b) is a schematic diagram showing the comparison results of SOC of the method of the present invention at different switching temperatures. Detailed Implementation

[0095] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0096] Example 1

[0097] A segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning, such as... Figure 1 As shown, it includes:

[0098] Establish a thermoelectric coupling model for the battery, and develop state transition expressions for voltage, temperature, and SOC.

[0099] When the battery temperature is lower than the preset switching temperature, the battery needs to be preheated and charged, and a high-frequency pulse preheating optimization process based on reinforcement learning is initiated; when the battery temperature reaches the preset switching temperature, a heating and charging process based on reinforcement learning is initiated.

[0100] Example 2

[0101] The difference between the reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries described in Example 1 and the method described in Example 1 is as follows:

[0102] The thermoelectric coupling model of the battery is shown below:

[0103]

[0104] Where m represents the mass of the battery; C p The specific heat capacity of the battery is represented by t; time is represented by Q; the heat generation rate of the battery is represented by h; the heat transfer coefficient of the battery is represented by S; the surface area of ​​the battery is represented by T; and the ambient temperature is represented by T0.

[0105] The state transition expressions for voltage, temperature, and SOC are shown below:

[0106] The voltage of a lithium-ion battery is calculated as follows:

[0107] U O =U OCV +I ac ·Z total (T,f,SOC)

[0108] Among them, U O It is the output voltage of a lithium-ion battery; U OCV It is the OCV of lithium-ion batteries; I ac It is the peak-to-peak value of the AC component; Z total The total AC impedance at the current moment is obtained through interpolation;

[0109] The heat production rate Q during the preheating stage is calculated as follows:

[0110]

[0111] Among them, Z Re is the real part of the AC impedance at the current moment, obtained by interpolation; SOC is defined as the ratio of the remaining capacity to the capacity in the fully charged state; f represents the frequency of the AC component.

[0112] Discretize the above formula to obtain the temperature T of the lithium-ion battery at time k+1. k+1 :

[0113]

[0114] Among them, T s A time interval representing temperature;

[0115] The battery's state of charge (SOC) is obtained through coulomb counting, as shown below:

[0116]

[0117] Among them, SOC int It is the initial SOC of the battery, Q bat I is the actual usable capacity of the battery, and I is the current flowing through the battery, which is positive when charging and negative when discharging.

[0118] In the high-frequency pulse preheating optimization process based on reinforcement learning, a TD3 (Twin Delayed Deep Deterministic policy gradient) model is used to update the network parameters of the reinforcement learning model in order to achieve the optimal preheating strategy. The TD3 model includes two evaluation networks and one action network. The evaluation network is used to evaluate the value of actions, and the action network is used to select actions. These networks use deep neural networks to approximate the state-action value function and the policy function. The action network outputs the optimal pulse frequency as the action based on the current voltage, temperature, and SOC state, according to the reward function. Impedance information is obtained through interpolation, and combined with the calculated heat generation rate Q, the state transition is performed through the established state transition expressions for voltage, temperature, and SOC. Through continuous interaction with the environmental state, the TD3 model agent learns the optimal preheating strategy, which guides the selection of pulse frequency to achieve the predetermined preheating effect. The entire optimization process is a cyclic operation until the preset number of iterations is reached.

[0119] In the high-frequency pulse preheating optimization process based on reinforcement learning, the state space s(t) is defined as:

[0120] s(t)={T(t)}

[0121] Where T(t) represents the battery temperature and t represents the time step;

[0122] Define the action space a(t) as:

[0123] a(t)={A(t),f(t)}

[0124] Where A(t) represents the amplitude of the high-frequency pulse current waveform, with a value range of [-25A, 25A]; f(t) represents the frequency of the high-frequency pulse current waveform, with a value range of [1000Hz, 10000Hz].

[0125] Set the reward function r(t) as follows:

[0126] r(t)=ω1r temp (t)+ω2r targ (t)+ω3r volt (t)

[0127] Where ω1 to ω3 are the weights of each part of the reward or penalty; r temp (t) and r targ (t) represents the target temperature difference penalty and the heating end reward, respectively, r volt (t) represents the voltage over-limit penalty;

[0128] r temp (t) and r targ The expressions for (t) are as follows:

[0129] r temp (t)=T(t)-T target

[0130]

[0131] Among them, T target The preset target temperature is represented by R, which is the heating end reward value. The target temperature difference penalty is the difference between the current temperature and the target temperature at each time step; a negative penalty means that the lower the current temperature, the greater the penalty, and vice versa. The heating end reward is a one-time reward given when the battery temperature reaches the target temperature. Both work together to achieve the fastest possible heating to the target temperature.

[0132] r volt The expression for (t) is:

[0133]

[0134] Among them, U limit U(t) is the charging cutoff voltage, and U(t) is the current battery terminal voltage. If the current terminal voltage is greater than the cutoff voltage, a voltage over-limit penalty is applied to ensure safety during the battery heating process.

[0135] In the heating and charging process based on reinforcement learning,

[0136] Establish the action space a(t)′ as follows:

[0137] a(t)′={(k,x k |k∈[1,2,3],x k ∈[I ac (t),f(t),I dc (t)]}

[0138] Where k represents different modes, x k Representing the action space, the correspondence between discrete actions and continuous actions is as follows:

[0139]

[0140] Where k=1 indicates the selection of the heating mode, and the associated parameter is the pulse amplitude I. ac (t) and frequency f(t); k=2 indicates the selection of heating and charging mode, and the associated parameters are pulse frequency f(t) and AC component amplitude I. ac (t) and DC component amplitude I dc (t); k=3 indicates the selection of the charging mode, and the only parameter associated with it is the DC current amplitude I. dc (t);

[0141] Define the state space s(t)′ as:

[0142] s(t)′={T(t),SOC(t)}

[0143] Where T(t) represents the battery temperature and SOC(t) represents the battery state of charge;

[0144] State transitions are performed using established state transition expressions based on voltage, temperature, and SOC.

[0145] The calculation of the heat production rate Q depends on the mode selected at the current moment:

[0146]

[0147] Among them, Z Re and Z tot The real part of the AC impedance and the total impedance at the current moment are obtained through interpolation; R i The current ohmic impedance is obtained through interpolation; T represents the battery temperature, f represents the frequency of the alternating current, and SOC refers to the battery's state of charge.

[0148] The reward function is set as follows:

[0149] r(t)′=α1r SOC (t)+(1-α1)r Qac (t)+α2r time (t)+α3r volt (t)+α4r dclim (t)

[0150] Wherein, α1 to α4 represent the weights of each part of the reward or punishment; r SOC (t) and r Qac (t) represents the penalty for the difference in target SOC and the penalty for AC heat generation, respectively. time (t) represents the charging time penalty, r volt (t) and r dclim(t) represents the voltage over-limit penalty and the DC component over-limit penalty, respectively;

[0151] r SOC (t) and r Qac The expressions for (t) are as follows:

[0152] r SOC (t)=SOC(t)-SOC target

[0153]

[0154] Among them, SOC target The SOC target value is set. The smaller the current SOC value, the greater the SOC difference penalty, which prompts the overall charging process to reach the SOC target value and quickly charge the battery. The AC heat generation penalty is a penalty related to the AC component heat generation when selecting heating or heating charging mode. K is a scaling factor to ensure that the SOC difference penalty and the AC heat generation penalty are on the same order of magnitude.

[0155] r time The expression for (t) is:

[0156] r time (t)=-(1-e -tT )

[0157] Where T is the maximum charging time set. As charging progresses, this reward gradually increases, allowing you to complete the charging goal faster.

[0158] r volt (t) and r dclim (t) represent the voltage over-limit penalty and the DC component over-limit penalty, respectively, and their expressions are:

[0159]

[0160]

[0161] I dc_lim (T t SOC t )=[U limit -U ocv (T t SOC t )] / R i (T t SOC t )

[0162] Among them, U limit This is the charging cutoff voltage; if the current terminal voltage is greater than the cutoff voltage, a penalty equal to the difference between the cutoff voltage and the current terminal voltage is applied; I dc_limIf the DC boundary is set and the selected heating / charging or charging mode is selected, and the DC component is greater than the DC boundary, a penalty will be imposed.

[0163] The reinforcement learning-based heating and charging process includes:

[0164] The PDQN model, which has the ability to handle mixed action spaces, is adopted.

[0165] The initial state information of the current lithium battery, such as temperature and SOC, is collected and input into the target policy network model; the target policy network model is the PDQN model.

[0166] The Target Policy Network (PDQN) model is a deep neural network that, once trained, outputs the optimal heating and charging strategy based on the current state. The PDQN model is a parameterized deep Q-network that learns to evaluate the value of performing different actions under different states. At each time step, the PDQN model receives the current state as input and outputs the Q-value for each possible action; the agent selects an action based on the Q-value and provides feedback according to the current state.

[0167] The PDQN model updates its network parameters using optimization algorithms such as gradient descent, based on the reward value calculated from the reward function and new state information. This process aims to enable the PDQN model to more accurately predict future rewards and thereby optimize its Q-value function estimation. This process continues, with the agent constantly interacting with the environmental state, accumulating experience, and gradually improving the heating and charging strategy through updates to the PDQN model. When a preset number of iterations is met, the PDQN model has learned a superior Q-value function.

[0168] Figure 2(a) shows the battery temperature comparison results of the method of the present invention under different switching temperatures; Figure 2(b) shows the SOC comparison results of the method of the present invention under different switching temperatures; it illustrates the preheating charging effect at switching temperatures of 10℃, 5℃, and 0℃ when the ambient temperature is -20℃ and the initial SOC value is 20%, and the resulting temperature changes and SOC charging curves throughout the process. As shown in the figures, the battery maintains a relatively high temperature during the heating process under different switching temperatures. Further observation reveals that higher switching temperatures not only increase the average temperature of the heating and charging phase but also accelerate the charging speed. Specifically, when the switching temperatures are set to 10℃, 5℃, and 0℃, the SOC values ​​at the end of charging are 0.5123, 0.5015, and 0.4840, respectively. Although the 0℃ switching temperature allows the heating and charging phase to begin earlier, the lower average temperature throughout the process limits the allowable charging current, resulting in a slower SOC growth rate. However, higher switching temperatures also mean that the preheating phase requires more energy, thus providing a trade-off between lower heat consumption and faster charging speed.

[0169] Figure 3(a) is a schematic diagram comparing the battery temperature results of the method of the present invention and the traditional constant current heating and charging under a 10°C switching temperature; Figure 3(b) is a schematic diagram comparing the SOC results of the method of the present invention and the traditional constant pulse current heating and charging under a 10°C switching temperature; it can be seen that the SOC growth rate of the method of the present invention is significantly higher than that of the traditional method in the same time period, and the SOC values ​​at the end of charging are 0.5123 and 0.3775, respectively, which is about 35.71% higher in terms of charging speed.

[0170] Example 3

[0171] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries as described in Embodiment 1 or 2.

[0172] Example 4

[0173] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries as described in Embodiment 1 or 2.

[0174] Example 5

[0175] A reinforcement learning-based segmented optimization system for low-temperature preheating charging of lithium-ion batteries includes:

[0176] The module for establishing the thermoelectric coupling model is configured to: establish the thermoelectric coupling model of the battery and establish state transition expressions for voltage, temperature, and SOC;

[0177] The preheating and charging segment optimization module is configured to: when the battery temperature is lower than the preset switching temperature, preheating and charging of the battery is required, and a high-frequency pulse preheating optimization process based on reinforcement learning is initiated; when the battery temperature reaches the preset switching temperature, a heating and charging process based on reinforcement learning is initiated.

Claims

1. A segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning, characterized in that, include: Establish a thermoelectric coupling model for the battery, and develop state transition expressions for voltage, temperature, and SOC. When the battery temperature is lower than the preset switching temperature, a high-frequency pulse preheating optimization process based on reinforcement learning is initiated. When the battery temperature reaches the preset switching temperature, a reinforcement learning-based heating and charging process is initiated. In the high-frequency pulse preheating optimization process based on reinforcement learning, the TD3 model is used to update the network parameters of the reinforcement learning model. The TD3 model includes two evaluation networks and one action network. The evaluation network is used to evaluate the value of actions, and the action network is used to select actions. The action network outputs based on the current voltage, temperature, and SOC state, and outputs the optimal pulse frequency as the action according to the reward function. Impedance information is obtained through interpolation, and combined with the calculated heat generation rate Q. heat The TD3 model agent performs state transitions by establishing state transition expressions for voltage, temperature, and SOC. Through continuous interaction with the environmental state, the agent learns the optimal preheating strategy. The preheating strategy guides the selection of the frequency of the high-frequency pulse current waveform to achieve the predetermined preheating effect. entire The optimization process is a cyclical operation that continues until the preset number of iterations is reached; The reinforcement learning-based heating and charging process includes: Collect the initial state information of the current lithium battery and input this initial state information into the target policy network model; The target policy network model is the PDQN model; At each time step, the PDQN model receives the current state as input and outputs the Q-factor for each possible action. value Value; the agent is based on Q. value Choose an action to perform and provide feedback based on the current state.

2. The segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning according to claim 1, characterized in that, The thermoelectric coupling model of the battery is shown below: Where m represents the mass of the battery; C p Q represents the specific heat capacity of the battery; t is the time step; Q heat The battery's heat generation rate is represented by h; the battery's heat transfer coefficient is represented by S; the battery's surface area is represented by T; the battery's temperature is represented by T0; and the ambient temperature is represented by T0.

3. The reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries according to claim 2, characterized in that, The state transition expressions for voltage, temperature, and SOC are shown below: The voltage of a lithium-ion battery is calculated as follows: Among them, U O It is the output voltage of a lithium-ion battery; U OCV It is the OCV of lithium-ion batteries; I ac It is the peak-to-peak value of the AC component; Z tot The total AC impedance at the current moment is obtained through interpolation; Heat generation rate Q during preheating stage heat The calculation is expressed as: Among them, Z Re is the real part of the AC impedance at the current moment, obtained by interpolation; SOC is defined as the ratio of the remaining capacity to the capacity in the fully charged state; f represents the frequency of the AC component. Obtain the temperature T of the lithium-ion battery at time k+1. k+1 : Among them, T s A time interval representing temperature; The battery's state of charge (SOC) is obtained through coulomb counting, as shown below: Among them, SOC int It is the initial SOC of the battery, Q bat I is the actual usable capacity of the battery, and I is the current flowing through the battery, which is positive when charging and negative when discharging.

4. The segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning according to claim 3, characterized in that, In the high-frequency pulse preheating optimization process based on reinforcement learning, the state space s(t) is defined as: Where T(t) represents the temperature of the battery; Define action space for: Where A(t) represents the amplitude of the high-frequency pulse current waveform, and f(t) represents the frequency of the high-frequency pulse current waveform; Set the reward function r(t) as follows: in, These represent the weights of the rewards or penalties for each part; r temp (t) and r targ (t) represents the target temperature difference penalty and the heating end reward, respectively, r volt (t) represents the voltage over-limit penalty; r temp (t) and r targ The expressions for (t) are as follows: Among them, T target R is the preset target temperature, and R is the bonus value for the end of heating. r volt The expression for (t) is: Among them, U limit U(t) is the charging cutoff voltage, and U(t) is the current battery terminal voltage.

5. The segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning according to claim 4, characterized in that, In the heating and charging process based on reinforcement learning, Establish action space As shown below: Where k1 represents different modes, x k1 Representing the action space, the correspondence between discrete actions and continuous actions is as follows: Where k1=1 indicates the selection of the heating mode, and the associated parameter is the AC component amplitude. and the frequency of high-frequency pulse current waveform k1=2 indicates that the heating and charging mode is selected, and the associated parameter is the frequency of the high-frequency pulse current waveform. AC component amplitude and DC component amplitude k1=3 indicates that the charging mode is selected, and the only parameter associated with it is the DC component amplitude. ; Define the state space for: Where T(t) represents the battery temperature and SOC(t) represents the battery state of charge; State transitions are performed using established state transition expressions based on voltage, temperature, and SOC. Heat production rate Q heat The calculation depends on the mode selected at the current moment: Among them, Z Re and Z tot The real part of the AC impedance and the total impedance at the current moment are obtained through interpolation; R i The ohmic impedance at the current moment is obtained through interpolation; T represents the battery temperature. The reward function is set as follows: Wherein, α1 to α4 represent the weights of each part of the reward or punishment; r SOC (t) and r Qac (t) represents the penalty for the difference in target SOC and the penalty for AC heat generation, respectively. time (t) represents the charging time penalty, r volt (t) and r dclim (t) represents the voltage over-limit penalty and the DC component over-limit penalty, respectively; r SOC (t) and r Qac The expressions for (t) are as follows: Among them, SOC target For the set SOC target value, the AC heat generation penalty is a penalty related to the AC component heat generation when selecting heating or heating-charging mode. K is the scaling factor to ensure that the SOC difference penalty and the AC heat generation penalty are on the same order of magnitude. r time The expression for (t) is: T1 is the maximum charging time set. As charging progresses, this reward gradually increases, allowing you to complete the charging goal faster. r volt (t) and These are the voltage over-limit penalty and the DC component over-limit penalty, respectively, expressed as follows: Among them, U limit This is the charging cutoff voltage; if the current terminal voltage is greater than the cutoff voltage, a penalty equal to the difference between the cutoff voltage and the current terminal voltage is applied; I dc_lim If the DC boundary is set and the selected heating / charging or charging mode is selected, and the DC component is greater than the DC boundary, a penalty will be imposed.

6. The segmented optimization method for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning-based heating and charging process includes: The PDQN model updates its network parameters using a gradient descent optimization algorithm based on the reward value calculated from the reward function and the new state information; and the heating and charging strategy is gradually improved through the updating of the PDQN model.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the reinforcement learning-based segmented optimization method for low-temperature preheating charging of lithium-ion batteries as described in any one of claims 1-6.

9. A segmented optimization system for low-temperature preheating charging of lithium-ion batteries based on reinforcement learning, characterized in that, include: The module for establishing the thermoelectric coupling model is configured to: establish the thermoelectric coupling model of the battery and establish state transition expressions for voltage, temperature, and SOC; The preheating and charging segment optimization module is configured to: initiate a high-frequency pulse preheating optimization process based on reinforcement learning when the battery temperature is lower than the preset switching temperature; and enter a heating and charging process based on reinforcement learning when the battery temperature reaches the preset switching temperature. In the high-frequency pulse preheating optimization process based on reinforcement learning, the TD3 model is used to update the network parameters of the reinforcement learning model. The TD3 model includes two evaluation networks and one action network. The evaluation network is used to evaluate the value of actions, and the action network is used to select actions. The action network outputs based on the current voltage, temperature, and SOC state, and outputs the optimal pulse frequency as the action according to the reward function. Impedance information is obtained through interpolation, and combined with the calculated heat generation rate Q. heat The TD3 model agent performs state transitions by establishing state transition expressions for voltage, temperature, and SOC. Through continuous interaction with the environmental state, the agent learns the optimal preheating strategy, which guides the selection of pulse frequency to achieve the predetermined preheating effect. entire The optimization process is a cyclical operation that continues until the preset number of iterations is reached; The reinforcement learning-based heating and charging process includes: Collect the initial state information of the current lithium battery and input this initial state information into the target policy network model; The target policy network model is the PDQN model; At each time step, the PDQN model receives the current state as input and outputs the Q-factor for each possible action. value Value; the agent is based on Q. value Choose an action to perform and provide feedback based on the current state.

Images