Lithium ion battery reinforcement learning fast charge control method based on experimental data driving

Abstract

According to the lithium ion battery reinforcement learning fast charge control method based on experimental data driving, real experimental data and an epsilon-greedy strategy are used for training an intelligent agent, the influence of different epsilon values on an optimization result is researched, and an optimization strategy of battery fast charge control is obtained. According to the method, the simulation data is used for replacing part of real data, the influence on the training result is evaluated, and the replaced data is used for training the intelligent agent, so that part of dangerous steps in the real experiment process for obtaining the training data set are avoided. Results show that the lithium ion battery reinforcement learning fast charge control method based on experimental data has advantages in the aspects of charging time, charging safety, battery aging delay and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of lithium-ion batteries and relates to lithium-ion battery fast charging control, which is a lithium-ion battery reinforcement learning fast charging control method driven by experimental data. Background Art

[0002] Based on the advocacy of green travel, the greening and electrification of automobiles have become a new trend. As the core of electric vehicles, the development of battery energy storage systems is the key to promoting the development of electric vehicles. Lithium-ion batteries are widely used in automotive power batteries because of their high operating voltage, high specific power and energy density, high energy efficiency, long cycle life, environmental friendliness, and rapidly declining costs. However, compared with traditional fuel vehicles, electric vehicles have long charging times and range anxiety problems. Therefore, solving the problem of fast battery charging is the key to promoting the development of electric vehicles. During the battery charging process, simply increasing the charging current to speed up the charging speed will cause a significant temperature rise and accelerate the aging of the battery. From the perspective of safety and economy, it is crucial to propose a safe and fast charging strategy.

[0003] Existing common charging strategies are divided into model-free strategies, model-based strategies, data-driven strategies, and learning-based strategies.

[0004] Model-free strategies, such as the constant current constant voltage charging (CCCV) currently used by mainstream electric vehicles, first charge the battery with a constant current at the beginning of charging, and switch to constant voltage charging when the threshold voltage is reached until the battery is fully charged or the charging current is less than the threshold. Other model-free strategies, such as multi-stage constant current constant voltage and pulse charging, all predetermine the charging current or voltage value, while ignoring the electrochemical reaction process inside the battery. Such strategies may cause serious safety problems during the charging process.

[0005] Model-based strategies describe the dynamic behavior of battery behavior characteristics by establishing a mathematical model for lithium-ion batteries. Model-based strategies can be divided into two categories: Equivalent Circuit Models (ECM) and Electrochemical Models (EM). ECM was originally proposed by Randles as a fast electrode reaction kinetics algorithm, which laid the foundation for the modeling of electrochemical energy storage systems. ECM is a black box model, usually combined with a thermal model, which can roughly reflect the charging and discharging process of the battery and is widely used by researchers. However, ECM cannot fully describe a series of physical and chemical phenomena occurring in lithium-ion batteries. In a complex charging process, ECM may deviate from the battery voltage, temperature, etc. Compared with ECM, EM is more like a white box model, which can truly reflect the internal physical and chemical reaction process of lithium-ion batteries during charging and discharging, improve the accuracy of the model, and gradually gain favor from researchers. The earliest EM was the DFN (Doyle–Fuller–Newman) model proposed by Newman and his collaborators. Subsequently, researchers established simplified single particle models (SPM) and enhanced single particle models (SPMe) based on the DFN model. Due to the high precision of electrochemical models, many studies on lithium-ion batteries are based on electrochemistry. However, the methods based on electrochemical models have high requirements for the accuracy of the model. The higher the accuracy, the higher the computational complexity. The single use of high-precision electrochemical models is not conducive to solving the problem.

[0006] Data-driven strategies use large amounts of historical data to learn patterns in the charging process without the need for complex battery models. This approach has advantages in dealing with complex problems such as high dimensionality, nonlinearity, and time variability, and can effectively reduce computational complexity while improving the accuracy and robustness of charging strategies. Data-driven strategies bypass the battery modeling process and directly use large amounts of existing data to train machine learning models to predict battery life. Data-driven strategies are flexible and easier to implement, but require a large number of time-consuming and energy-intensive test experiments to generate training data, and the performance of the algorithm is highly dependent on the size and number of data sets.

[0007] Learning-based strategies usually utilize existing reinforcement learning methods combined with lithium-ion battery simulation models to optimize the fast charging process. Compared with data-driven methods, the advantage of learning-based methods is that the agent can achieve the optimization goal by training and learning with relatively less data. Reinforcement learning algorithms learn by setting reward mechanisms and interacting with the environment. Even with limited information, it can approach the completeness of knowledge and effectively solve complex optimization problems. Since the training data of most current studies comes from a simulated environment, it will inevitably lead to results that deviate from the real battery. If the agent is trained directly on a real battery, improper action selection may also exceed the battery safety limit during charging, resulting in unnecessary waste and increased risk during training. Therefore, how to train the agent using real experimental data to ensure the authenticity of the results while avoiding some dangerous steps in the experimental process is also one of the problems to be solved. Summary of the invention

[0008] The problem to be solved by the present invention is to address the shortcomings of various charging strategies in the prior art and propose a lithium-ion battery reinforcement learning fast charging control method driven by experimental data. The intelligent agent is trained using real battery data to get close to the real battery, the reconstructed electrochemical model is used to verify the optimized fast charging strategy, and the data replacement strategy is used to reduce the dangerous steps in the real experiment.

[0009] The technical solution of the present invention is: a lithium-ion battery reinforcement learning fast charging control method driven by experimental data, comprising:

[0010] Step S100: Perform an open circuit voltage test (OCV), a dynamic pressure test (DST) and an urban road cycle test (UDDS) on a real battery, and record the experimental data for parameter identification of a lithium-ion battery electrochemical model;

[0011] Step S200: performing a lithium-ion battery state traversal experiment, dividing the battery charging state into different states according to the state of charge SOC and temperature, selecting different currents, and performing charging experiments in different states;

[0012] Step S300: construct a fast-charging mathematical model, and couple the reinforcement learning algorithm with the battery state traversal experiment. By setting a reward mechanism, different reward feedbacks are given to the state traversal experiment results, and the battery state, charging current and reward feedback data are stored for the training of the intelligent agent in reinforcement learning, where the battery state and charging current correspond to the actions in reinforcement learning, and the optimized fast-charging strategy for lithium-ion batteries is obtained based on the fast-charging mathematical model and the reinforcement learning results;

[0013] Step S400: Use the trained intelligent agent to simulate and verify the fast charging control in the constructed lithium-ion battery electrochemical model to evaluate the effect of the optimized fast charging strategy. After the evaluation, the trained intelligent agent is used for the fast charging control of the real lithium-ion battery.

[0014] Furthermore, in step S100, the electrochemical model of the lithium-ion battery used is the DFN model, and the model open circuit voltage is determined by the positive electrode open circuit voltage and the negative electrode open circuit voltage:

[0015] OCV=OCV p -OCV n

[0016] OCV p is the positive open circuit voltage, OCV n is the negative electrode open circuit voltage. Before parameter identification, the particle swarm optimization algorithm PSO is used to reconstruct the parameters of the positive and negative electrodes of the battery model to fit the OCV curve of the model and the actual battery. Then, the PSO algorithm and DST operating data are used to identify the battery model parameters, and the UDDS operating data is used to verify the model after parameter identification. After iterative training, the identified parameters are updated to the model to construct a lithium-ion battery electrochemical simulation model.

[0017] Furthermore, when the PSO algorithm is used to fit the parameters of the positive and negative electrodes of the reconstructed battery model, a multiple continuous fitting method is adopted. First, the OCV of the anode is fitted, and the fitting coefficients are recorded in the memory for fitting the OCV of the cathode. Then, the cathode OCV is fitted, and this process is repeated multiple times until the root mean square error meets the requirements.

[0018] Furthermore, in the lithium-ion battery state traversal experiment of step S200, the experimental temperature and the experimental range and traversal interval of the state of charge SOC are configured according to the battery model to obtain multiple states of the battery charging process. The charging current in each state is configured with a current range and a variation interval, as well as a charging time, to traverse the charging conditions of different charging currents in the state, and finally complete the charging experiment of a single battery in different states, and record the data of the experimental process, including the initial state and the termination state of each charge, the charging duration, and the charging current.

[0019] Furthermore, the SOC, temperature and current configurations in the lithium-ion battery state traversal experiment are adjusted. By reducing the set temperature and SOC intervals, the intervals between charging states are reduced to achieve more accurate experimental results. The charging current and charging time are adjusted to obtain different charging data. The current adjustment principle is to ensure that the battery state changes after each charging, that is, jumps from one state interval to the next state interval.

[0020] Further, step S300 includes: constructing a fast charging mathematical model as follows:

[0021] min J=J time +J safety

[0022]

[0023] Among them, the optimization target is the charging time loss J time and charging safety loss J safety , the constraints include that the charging current I(t), battery voltage V(t), battery temperature T(t) and battery SOC at time t are all within the specified range. The reinforcement learning algorithm is used to solve this optimization problem. The reward mechanism based on the state traversal experiment setting is:

[0024]

[0025] Among them, λ1 and λ2 are reward weights, t i is the charging time of the ith time, T i is the battery temperature at the end of the i-th charge, SOC i-1 and SOC i are the starting SOC and ending SOC of the i-th charging, respectively. When the charging time is t i If the time is less than 6 minutes, a negative reward will be given for the calculated time; if the temperature exceeds the threshold temperature, a negative reward will be given for the calculated temperature; if the battery SOC meets the requirement, a positive reward of 100 will be given; otherwise, a negative reward of -2 will be given;

[0026] The state-action-reward data is then saved for the training of the agent in reinforcement learning. The greedy algorithm ε-greedy is introduced, and the training iteration process is as follows:

[0027]

[0028] Among them, Q(s,a) is the expected reward value obtained by selecting action a in state s, s' is the next state, a' is the action selected in the next state, n is a random number that obeys the uniform distribution between [0,1], and ε is a preset value. When n is less than or equal to ε, the action of the next state that can obtain the maximum expected reward value will be selected; when n is greater than ε, the action of the next state will be randomly selected; all iterated Q(s,a) are summarized in a Q table for fast charging optimization control. When the battery charging process is in a certain state, the corresponding action is selected according to the Q table to achieve the purpose of fast charging optimization.

[0029] Furthermore, the simulation verification data of the intelligent agent trained in step S400 is used to replace part of the real traversal experimental data of step S200, the impact of the number of replacements on the experimental results is analyzed, and the optimal number of replacements is selected. The intelligent agent is trained in reinforcement learning based on the replaced experimental data, enriching the training data of the intelligent agent while avoiding some dangerous steps in the real experimental process.

[0030] The present invention also provides an electronic device, comprising a memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, it can implement the above-mentioned lithium-ion battery reinforcement learning fast charging control method driven by experimental data.

[0031] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned lithium-ion battery reinforcement learning fast charging control method driven by experimental data is implemented.

[0032] The present invention has the following beneficial effects:

[0033] 1. Compared with the existing learning-based battery fast charging strategy, the present invention couples the lithium-ion battery state traversal experiment with reinforcement learning to train the intelligent agent using experimental data from real batteries, making the optimization results closer to reality.

[0034] 2. The present invention utilizes real experimental data to reconstruct a high-precision electrochemical model of lithium-ion batteries, so that the battery model is more consistent with the actual battery, and is effectively used for verification and evaluation of optimization strategies, thereby reducing the cost of using real batteries for OCV testing, DST testing, and UDDS testing.

[0035] 3. Compared with the traditional CCCV strategy, the optimization strategy obtained by the present invention has advantages in three aspects: charging time, charging safety and delaying battery aging.

[0036] Fourth, the data replacement strategy proposed by the present invention uses simulated data to replace part of the real experimental data, and ensures the optimization control ability of the results. The intelligent agent trained by the traversal experimental data obtained by the data replacement strategy can avoid dangerous steps in actual experiments on the basis of avoiding some real experimental links, and enrich and optimize the experimental data of lithium-ion battery state traversal, and obtain the effect of the intelligent agent strategy obtained by training with real experimental data, ensuring safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The present invention is a flowchart including verification, evaluation and tuning.

[0038] Figure 2Schematic diagram of the process of the reinforcement learning fast charging control method for lithium-ion batteries of the present invention, (a) is a schematic diagram of model electrode reconstruction, (b) is a schematic diagram of the DFN model framework, and (c) is a reinforcement learning framework.

[0039] Figure 3 Schematic diagram of the experimental equipment used in the embodiment of the present invention.

[0040] Figure 4 This is a fitting diagram of the actual battery OCV after the electrode is reconstructed by the model of the embodiment of the present invention.

[0041] Figure 5 Schematic diagram of DST condition training in the parameter identification process of an embodiment of the present invention.

[0042] Figure 6 Schematic diagram of UDDS working condition verification of the parameter identification process of an embodiment of the present invention.

[0043] Figure 7 This is a schematic diagram of the state traversal experiment results after the reward mechanism is applied to the state traversal experiment results of an embodiment of the present invention.

[0044] Figure 8 This is an optimized charging current curve diagram when ε=1.0 according to an embodiment of the present invention.

[0045] Fig. 9 This is an optimized charging current curve diagram when ε=0.9 according to an embodiment of the present invention.

[0046] Fig.10 This is an optimized charging current curve diagram when ε=0.8 according to an embodiment of the present invention.

[0047] Fig.11 This is an optimized charging current curve diagram when ε=0.7 according to an embodiment of the present invention.

[0048] Fig.12 Schematic diagram of the temperature curve of the optimized charging process under different ε values ​​according to an embodiment of the present invention.

[0049] Fig.13 Schematic diagram of voltage curve of optimized charging process under different ε values ​​according to an embodiment of the present invention.

[0050] Fig.14 It is a box plot of the optimization result reward values ​​corresponding to different numbers of replacement data in an embodiment of the present invention.

[0051] Fig.15 This is a temperature comparison diagram of the charging process of the replacement data training result, the original data training result and the CCCV strategy according to an embodiment of the present invention.

[0052] Fig.16 This is a comparison diagram of the replacement data training results, the original data training results and the cycle aging of the CCCV strategy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] refer to Figure 1 The present invention is a data-driven lithium-ion battery reinforcement learning fast charging control method, comprising:

[0055] Step S100: Perform an open circuit voltage (OCV) test, a dynamic stress test (DST) and an urban Dynamometer Driving Schedule (UDDS) test on the real battery, and record the experimental data for parameter identification of the lithium-ion battery electrochemical model.

[0056] Step S200: Perform a lithium-ion battery state traversal experiment, divide the battery charging state according to the state of charge (SOC) and temperature, select different currents, and perform a charging experiment in each state.

[0057] Step S300: Construct a fast charging mathematical model, and couple the reinforcement learning algorithm with the battery state traversal experiment. By setting up a reward mechanism, different reward feedbacks are given to the state traversal experiment results, and the battery state, charging current and reward feedback data are stored for the training of the intelligent agent in reinforcement learning to achieve optimized fast charging control, where the battery state and charging current correspond to the actions in reinforcement learning.

[0058] Step S400, the trained agent is used to simulate and verify the fast charging control in the constructed lithium-ion battery electrochemical model, and the effect of optimizing the fast charging strategy is evaluated. After the evaluation, the trained agent is used for the fast charging control of the real lithium-ion battery. Compared with the traditional constant current constant voltage charging strategy (CCCV) in terms of charging time, charging safety and delaying battery aging, the present invention has a better effect.

[0059] The present invention proposes a reinforcement learning optimization fast charging control method for lithium-ion batteries driven by real experimental data. First, a series of battery experimental data, including OCV test data, DST operating data and UDDS operating data, are used to reconstruct electrodes and identify parameters of a high-precision lithium-ion battery electrochemical model. Subsequently, a state traversal experiment is reasonably set up to divide the charging process of the lithium-ion battery into multiple states, each state is charged with a specific current, and the experimental data is recorded. Then, the fast charging problem is mathematically described, converted into an optimization problem and solved using a reinforcement learning algorithm. The state traversal experimental data is combined with the defined reward feedback mechanism to construct an initialization Q table of the reinforcement learning agent, and iterative training is performed in combination with the ε-greedy strategy. Finally, part of the experimental data is replaced with simulation data, and compared with the original data training results and the CCCV strategy while ensuring the total reward value. The results show that the present invention has advantages in three aspects: charging time, charging safety and delaying battery aging.

[0060] Each step of the present invention is described in detail below.

[0061] In step S100, the lithium-ion battery electrochemical model used is the DFN model. The Doyle–Fuller–Newman (DFN) model is a basic electrochemical modeling framework widely used in the field of lithium-ion batteries. The model open circuit voltage is determined by the positive electrode open circuit voltage and the negative electrode open circuit voltage:

[0062] OCV=OCV p -OCV n

[0063] Where is the positive electrode open circuit voltage, is the negative electrode open circuit voltage. Before parameter identification, the particle swarm optimization algorithm (PSO) is used to reconstruct the parameters of the positive and negative electrodes of the battery model to fit the OCV curve of the model and the actual battery. Then, the PSO algorithm and DST operating data are used to identify the battery model parameters. And the UDDS operating data is used to verify the model after parameter identification.

[0064] In step S200, the experimental temperature and the experimental range and traversal interval of the state of charge SOC are configured according to the battery model to obtain multiple states of the battery charging process. The current range and change interval, as well as the charging time, are configured for the charging current in each state to traverse the charging conditions of different charging currents in the state. Finally, the charging experiment of a single battery in different states is completed, and the data of the experimental process is recorded, including the initial state and the termination state of each charging, the charging duration, and the charging current.

[0065] As an embodiment, in step S200, the battery for the experiment is a 92103148-3.7V soft-pack lithium-ion battery with a capacity of 18Ah. The lithium-ion battery state traversal experiment is carried out in a 25°C incubator. The lithium-ion battery charging process is divided according to SOC and temperature, where the SOC ranges from 0% to 85% at intervals of 5%, totaling 18; the temperature ranges from 25°C to 45°C at intervals of 5°C, totaling 5, so the battery charging process is divided into 90 states. The charging current selection in each state is 6 options from 0.5C to 3C with an interval of 0.5C, and each charging time is 6 minutes or the charging time to reach the cut-off voltage of 4.2V. Therefore, a single battery will undergo 540 charging experiments.

[0066] The above embodiment is an example. Other types of batteries need to adjust the experimental parameters according to the battery operating parameters. At the same time, the SOC, temperature and current values ​​in the lithium-ion battery state traversal experiment can also be adjusted and set according to actual needs, such as reducing the set temperature and SOC intervals to reduce the intervals between charging states to achieve more accurate experimental results. The charging current and charging time are adjusted to obtain different charging data. The principle of current adjustment is to ensure that the battery state changes after each charging, that is, jumps from one state interval to the next state interval.

[0067] Furthermore, in step S300, the mathematical model of the fast charging problem is constructed as follows:

[0068] min J=J time +J safety

[0069]

[0070] Among them, the optimization target is the charging time loss J time and charging safety loss J safety The constraints include that the charging current I(t), battery voltage V(t), battery temperature T(t) and battery SOC at time t are all within the specified range. The reinforcement learning algorithm is used to solve this optimization problem. The reward mechanism based on the state traversal experiment setting is:

[0071]

[0072] Among them, λ1 and λ2 are reward weights, t i is the charging time of the ith time, T i is the battery temperature at the end of the i-th charge, SOC i-1 and SOC i is the starting SOC and ending SOC of the i-th charging. iIf the time is less than 6 minutes, a negative reward is given for the corresponding calculated time; when the temperature exceeds the threshold temperature, a negative reward is given for the corresponding calculated temperature; when the battery SOC meets the requirement, a positive reward of 100 is given; otherwise, a negative reward of -2 is given. Subsequently, the state-action-reward data is saved for the training of the agent in reinforcement learning. The training iteration process of the introduced greedy algorithm (ε-greedy) is as follows:

[0073]

[0074] Among them, Q(s,a) is the expected reward value obtained by selecting action a in state s. s' is the next state, and a' is the action selected in the next state. n is a random number that obeys a uniform distribution between [0,1]. ε is a preset value. When n is less than or equal to ε, the action of the next state that can obtain the maximum expected reward value will be selected; when n is greater than ε, the action of the next state will be randomly selected. All iterated Q(s,a) are summarized in a Q table for fast charging optimization control. When the battery charging process is in a certain state, the corresponding action is selected according to the Q table to achieve the purpose of fast charging optimization.

[0075] Furthermore, in step S400, in addition to evaluating the effect of the control strategy of the trained intelligent agent, the simulation verification data of the trained intelligent agent is used to replace part of the traversal experimental data of step S200 to enrich the training data of the optimized intelligent agent. The influence of the number of replacements on the experimental results is analyzed, and the optimal number of replacements is selected to avoid dangerous steps in some real traversal experimental processes, and then the training of the intelligent agent in reinforcement learning is carried out based on the replaced experimental data. The present invention compares the optimized fast charging strategy of the intelligent agent trained by the replacement data and the optimized fast charging strategy of the intelligent agent trained by the original experimental data with the traditional CCCV strategy. The replaced strategy is superior to the CCCV strategy in terms of charging time, charging safety and delaying battery aging, and is slightly better than the optimized fast charging strategy of the intelligent agent trained by the original experimental data.

[0076] The specific experimental description is given below.

[0077] The batteries used in the experiment of the present invention are four commercially available 92103148-3.7V soft-pack lithium-ion batteries with a capacity of 18Ah and a battery type of ternary lithium-ion battery. Other experimental equipment such as Figure 3 As shown, it includes a host computer, a temperature box and a battery tester.

[0078] The lithium ion battery electrochemical model used in the present invention is the DFN model, and its structure is as follows: Figure 2As shown in (b), it mainly includes an anode, a cathode, a separator and a current collector. In the DFN model, the battery is regarded as a collection of multiple electrochemical reactors, each of which consists of multiple layers such as an electrolyte layer, an electrode layer, and a current collector layer. Each layer involves processes such as electrochemical reactions, ion diffusion, and charge transfer. The electrode material of the DFN model is composed of spherical particles with a regular porous structure, which avoids the complex structure and particle distribution of active materials in practice. In addition, due to the high conductivity of the collector, there is no obvious change in the y and z directions, and only the electrochemical reaction kinetics in the x direction and radial r need to be considered. When the battery is discharged, the lithium in the anode particles becomes lithium ions, which move to the cathode through the separator and recombine in the cathode particles.

[0079] In order to ensure the high fidelity of the DFN model, it is necessary to identify the parameters of the model so that its charge and discharge characteristics are consistent with the actual battery, thereby meeting the requirements of high-quality simulation. In order to ensure the accuracy of subsequent identification, the model electrodes need to be reconstructed before the parameter identification of the DFN model. The OCV of the battery is determined by the positive and negative electrodes, and the final open circuit voltage is the potential difference between the two electrodes. Therefore, before fitting the OCV of the battery model, it is necessary to fit the OCV of the two electrodes separately. The present invention adopts the PSO algorithm to fit the coefficients of the electrode. The algorithm has the advantages of fast convergence speed, few parameters, and simple and convenient implementation. In the fitting process, fitting the two electrodes sequentially in a single iteration will result in an excessively large root mean square error (RMSE), which cannot meet the requirements. Therefore, the present invention adopts a method of multiple continuous fitting, such as Figure 2 First, fit the anode OCV, record the fitting coefficients in the memory, and then fit the cathode OCV. Repeat this process several times until the RMSE meets the requirements. The final OCV fitting result is as follows: Figure 5 As shown, the RMSE is 0.0093V.

[0080] The DFN model has too many parameters, and it is necessary to perform sensitivity tests one by one according to the reconstructed electrode model, and select parameters with high sensitivity for identification. The training and verification conditions are DST and UDDS, respectively, and the PSO algorithm is used for identification. The fitting results of DST training are shown in Figure 2. Figure 5 As shown in Figure 2, the RMSE of the training is 0.014V. The fitting results of the UDDS working condition verification are shown in Figure 2. Figure 6 As shown in Table 1, the RMSE of the fitting is 0.023 V. After iterative training, the identified parameters are updated to the model to build a high-fidelity lithium-ion battery electrochemical simulation model. The results of parameter identification are shown in Table 1.

[0081] Table 1

[0082]

[0083] The present invention mathematically describes the fast charging problem, with the optimization objectives being charging time and charging safety, and the constraints including battery temperature, voltage and SOC during the charging process and charging current. The reinforcement learning algorithm is used to solve this fast charging optimization problem. In order to make the behavior decisions of the trained intelligent agent closer to the real battery, the present invention conducts a real lithium-ion battery state traversal experiment. The experimental setup is as follows Figure 7 As shown. The lithium-ion battery charging process is divided according to SOC and temperature, where SOC ranges from 0% to 85% with an interval of 5% for a total of 18; the temperature ranges from 25°C to 45°C with an interval of 5°C for a total of 5, so the battery charging process is divided into 90 states. The charging current selection in each state is 6 options from 0.5C to 3C with an interval of 0.5C, and each charging time is 6 minutes or the charging time to reach the cut-off voltage of 4.2V. Therefore, a single battery will undergo 540 charging experiments. Record the data of the experimental process, including the initial state and termination state of each charge, the charging duration, the charging current size and other data.

[0084] In order to use the state traversal experimental data for the training of the intelligent agent, the present invention sets up a reward mechanism to give corresponding reward feedback to the action selection in each state, such as Figure 7 As shown in the figure, the initial Q table of the reinforcement learning agent is constructed and it is iteratively trained in combination with the ε-greedy strategy. The trained agent has the ability to optimize fast charging decisions. The optimized charging current curves corresponding to different ε values ​​are shown in the figure below. Figure 8 , Fig. 9 , Fig.10 and Fig.11 As ε decreases from 1.0 to 0.7, the charging times are [2565s, 2478s, 2517s, 2706s] respectively. Fig.12 Optimized charging temperature profile and Fig.13 The analysis of the voltage curve of the optimization process shows that when ε=1.0, it corresponds to a conservative strategy, when ε=0.7, it corresponds to an aggressive strategy, and the optimal charging strategy is when ε=0.9, at which the charging time is the shortest.

[0085] In order to further explore the role of simulation experimental data, the present invention explores the results of replacing real experimental data with different amounts of simulation data, such as Fig.14 As shown in Figure 2, it can be seen that as the number of replacements increases, the mean of the reward decreases, while the variance increases. Therefore, the optimal number of replacements is 5. Fig.15 and Fig.16The comparison results of the optimized fast-charging strategy of the agent trained with 5 replaced data and the optimized fast-charging strategy of the agent trained with original experimental data and the traditional CCCV strategy are shown. It can be seen that the replaced strategy is better than the CCCV strategy in terms of charging time, charging safety and delaying battery aging, and is slightly better than the optimized fast-charging strategy of the agent trained with original experimental data. The present invention proposes a strategy of replacing part of the real experimental data with data obtained by simulation verification. On the one hand, it enriches the training data of the agent and optimizes the experimental data of the lithium-ion battery state traversal. On the other hand, it can avoid some dangerous steps in the real experiment in order to obtain more data, thereby making the exploration of the agent more sufficient. The data-driven lithium-ion battery reinforcement learning fast-charging control of the present invention has also been verified.

[0086] The present invention is based on the reinforcement learning algorithm, uses the battery state traversal experimental data to train the intelligent agent, solves the fast charging optimization problem, and uses the DFN model to verify the optimized fast charging strategy. First, the parameters of the DFN model are identified. After reconstructing the electrode, the PSO algorithm is used to train and verify the parameter identification in combination with the DST working condition and the UDDS working condition, and a simulation model that strictly matches the real battery is obtained. Subsequently, a real battery state traversal experiment is carried out, the experimental data is collected, and the initialization Q table of reinforcement learning is constructed in combination with the reward feedback mechanism. In order to avoid falling into the local optimal problem, the present invention uses the ε-greedy strategy to iterate the Q table, train the intelligent agent, and analyze the impact of different ε values ​​on the optimization results. Finally, the replacement of the simulation data for the real data is explored, and it is compared with the CCCV strategy and the original data.

[0087] In other embodiments, the present invention also provides a lithium-ion battery reinforcement learning fast charging control method driven by experimental data, comprising a memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, the above-mentioned lithium-ion battery reinforcement learning fast charging control method driven by experimental data can be implemented.

[0088] In other embodiments, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned lithium-ion battery reinforcement learning fast charging control method driven by experimental data is implemented.

[0089] The parts not involved in the present invention are the same as the prior art or are implemented by using the prior art.

[0090] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A lithium-ion battery reinforcement learning fast charging control method driven by experimental data, characterized by: include Step S100: Perform an open circuit voltage test (OCV), a dynamic pressure test (DST) and an urban road cycle test (UDDS) on a real battery, and record the experimental data for parameter identification of a lithium-ion battery electrochemical model; Step S200: performing a lithium-ion battery state traversal experiment, dividing the battery charging state into different states according to the state of charge SOC and temperature, selecting different currents, and performing charging experiments in different states; Step S300: construct a fast-charging mathematical model, and couple the reinforcement learning algorithm with the battery state traversal experiment. By setting a reward mechanism, different reward feedbacks are given to the state traversal experiment results, and the battery state, charging current and reward feedback data are stored for the training of the intelligent agent in reinforcement learning, where the battery state and charging current correspond to the actions in reinforcement learning, and the optimized fast-charging strategy for lithium-ion batteries is obtained based on the fast-charging mathematical model and the reinforcement learning results; Step S400: Use the trained intelligent agent to simulate and verify the fast charging control in the constructed lithium-ion battery electrochemical model to evaluate the effect of the optimized fast charging strategy. After the evaluation, the trained intelligent agent is used for the fast charging control of the real lithium-ion battery.

2. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 1 is characterized in that: In step S100, the lithium-ion battery electrochemical model used is the DFN model, and the model open circuit voltage is determined by the positive electrode open circuit voltage and the negative electrode open circuit voltage: OCV=OCV p -OCV n OCV p is the positive open circuit voltage, OCV n is the negative electrode open circuit voltage. Before parameter identification, the particle swarm optimization algorithm PSO is used to reconstruct the parameters of the positive and negative electrodes of the battery model to fit the OCV curve of the model and the actual battery. Then, the PSO algorithm and DST operating condition data are used to identify the battery model parameters, and the UDDS operating condition data is used to verify the model after parameter identification. After iterative training, the identified parameters are updated to the model to construct a lithium-ion battery electrochemical simulation model.

3. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 2 is characterized in that: When the PSO algorithm is used to fit the parameters of the positive and negative electrodes of the reconstructed battery model, a multiple continuous fitting method is adopted. First, the OCV of the anode is fitted, and the fitting coefficients are recorded in the memory for fitting the OCV of the cathode. Then, the cathode OCV is fitted, and this process is repeated multiple times until the root mean square error meets the requirements.

4. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 1 is characterized in that: In the lithium-ion battery state traversal experiment of step S200, the experimental temperature and the experimental range and traversal interval of the state of charge SOC are configured according to the battery model to obtain multiple states of the battery charging process. The current range and change interval, as well as the charging time, are configured for the charging current in each state to traverse the charging conditions of different charging currents in the state, and finally complete the charging experiment of a single battery in different states, and record the data of the experimental process, including the initial state and terminal state of each charging, the charging duration, and the charging current.

5. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 4 is characterized in that: The SOC, temperature and current configurations in the lithium-ion battery state traversal experiment are adjusted. By reducing the set temperature and SOC intervals, the intervals between charging states are shortened to achieve more accurate experimental results. The charging current and charging time are adjusted to obtain different charging data. The current adjustment principle is to ensure that the battery state changes after each charging, that is, jumps from one state interval to the next state interval.

6. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 1 is characterized in that: Step S300 includes: constructing a fast charging mathematical model as follows: min J=J time +J safety Among them, the optimization target is the charging time loss J time and charging safety loss J safety The constraints include that the charging current I(t), battery voltage V(t), battery temperature T(t) and battery SOC at time t are all within the specified range. The reinforcement learning algorithm is used to solve this optimization problem. The reward mechanism based on the state traversal experiment setting is: Among them, λ1 and λ2 are reward weights, t i is the charging time of the ith time, T i is the battery temperature at the end of the i-th charge, SOC i-1 and SOC i are the starting SOC and ending SOC of the i-th charging, respectively. When the charging time is t i If the time is less than 6 minutes, a negative reward will be given for the calculated time; if the temperature exceeds the threshold temperature, a negative reward will be given for the calculated temperature; if the battery SOC meets the requirement, a positive reward of 100 will be given; otherwise, a negative reward of -2 will be given; The state-action-reward data is then saved for the training of the agent in reinforcement learning. The greedy algorithm ε-greedy is introduced, and the training iteration process is as follows: Among them, Q(s,a) is the expected reward value obtained by selecting action a in state s, s' is the next state, a' is the action selected in the next state, n is a random number that obeys the uniform distribution between [0,1], and ε is a preset value. When n is less than or equal to ε, the action of the next state that can obtain the maximum expected reward value will be selected; when n is greater than ε, the action of the next state will be randomly selected; all iterated Q(s,a) are summarized in a Q table for fast charging optimization control. When the battery charging process is in a certain state, the corresponding action is selected according to the Q table to achieve the purpose of fast charging optimization.

7. The lithium-ion battery reinforcement learning fast charging control method based on experimental data drive according to claim 1 is characterized in that: The simulation verification data of the intelligent agent trained in step S400 is used to replace part of the real traversal experimental data of step S200, the influence of the number of replacements on the experimental results is analyzed, the optimal number of replacements is selected, and the intelligent agent in reinforcement learning is trained based on the replaced experimental data, thereby enriching the training data of the intelligent agent while avoiding some dangerous steps in the real experimental process.

8. An electronic device, characterized in that: It comprises a memory and a processor; wherein the memory stores a computer program, and when the program is executed by the processor, it can implement the lithium-ion battery reinforcement learning fast charging control method driven by experimental data as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fast charging control method for lithium-ion batteries driven by reinforcement learning based on experimental data as described in any one of claims 1 to 7 is implemented.

Images