Method for preventing thermal runaway of battery through reinforcement learning and battery management system

Abstract

The invention relates to a method for preventing thermal runaway of a battery through reinforcement learning. The method comprises the following steps: extracting time series features from collected battery operation states and environment detection information (S1); predicting a thermal runaway risk of the battery based at least on the extracted time series features (S2); the reinforcement learning agent determines, as action, a strategy for intervening in battery thermal runaway based on states including the predicted risk of thermal runaway, a vehicle power limiting state, a battery insulation unit state, a battery cooling system state, and a vehicle operating state (S3). The application also relates to a battery management system, a vehicle and a computer program product. According to the method and the device, the reinforcement learning agent performs dynamic decision optimization on the thermal runaway risk of the battery under various complex working conditions, and the battery safety and the vehicle performance are balanced through a reward mechanism, so that the influence on the vehicle output power and the endurance mileage is reduced as far as possible on the basis of ensuring the battery safety.

Description

Technical Field

[0001] The present application relates to the technical field of battery management, and in particular to a method for preventing battery thermal runaway through reinforcement learning, a battery management system, a vehicle including a battery management system according to the present application, and a computer program product for at least assisting in implementing the steps of the method described in the present application. Background Art

[0002] Electric vehicle batteries are at risk of thermal runaway during use, especially under extreme operating conditions such as rapid charging and discharging, high temperatures, and battery aging, where internal chain reactions can easily occur, leading to serious safety accidents. Current battery management systems are mostly limited to monitoring a single parameter, such as battery temperature or battery voltage, and employing a fixed strategy to predict thermal runaway when a set threshold is exceeded. However, the process of battery thermal runaway typically involves the coordinated changes of multiple physical parameters. Monitoring a single parameter makes it difficult to detect abnormal internal battery conditions at the initial stages of thermal runaway, and fixed strategies based on threshold judgments are also unable to address the early signs of thermal runaway under complex operating conditions.

[0003] Therefore, there is room for improvement in the current methods of preventing battery thermal runaway. Summary of the Invention

[0004] The purpose of the present application is to provide a method for preventing battery thermal runaway through reinforcement learning, a battery management system, a vehicle including the battery management system according to the present application, and a computer program product, so as to at least partially solve the problems in the prior art.

[0005] According to a first aspect of the present application, a method for preventing battery thermal runaway through reinforcement learning is provided, and the method may include:

[0006] -Time series features can be extracted from the collected battery operating status and environmental detection information;

[0007] - the thermal runaway risk of the battery can be predicted based on at least the extracted time series features; and

[0008] -The reinforcement learning agent can determine as an action a strategy for intervening in battery thermal runaway based on the state, where the state can include the predicted thermal runaway risk, vehicle power limit state, battery insulation unit state, battery cooling system state, and vehicle operating state, etc.

[0009] The core concept of this application includes at least: extracting low-dimensional time series features from high-dimensional time series data collected from multiple data sources through deep neural networks, and quickly and efficiently evaluating the thermal runaway risk of the battery based on the extracted time series features. By adjusting the parameters of the strategy network and / or value network of the reinforcement learning agent based on the predicted thermal runaway risk, vehicle power limit status, battery insulation unit status, battery cooling system status and vehicle operating status, the reinforcement learning agent can dynamically optimize the thermal runaway risk of the battery under various complex working conditions, and balance battery safety and vehicle performance through a reward mechanism, so as to minimize the impact on vehicle output power and cruising range while ensuring battery safety.

[0010] According to an optional embodiment of the present application, the reward value regarding the determined strategy can be adjusted based on the predicted thermal runaway risk and / or the vehicle power limitation state, and the strategy determined by the reinforcement learning agent can be adjusted based on the reward value, so that the reward value obtained by the reinforcement learning agent by executing the determined strategy is maximized, wherein the reward value is related to the predicted thermal runaway risk, the change in thermal runaway risk over time, and the user experience loss caused by the vehicle power limitation state, wherein the reward value is used to characterize the cumulative amount of decay of each reward value obtained by the reinforcement learning agent by executing the determined strategy over time.

[0011] According to another optional embodiment of the present application, when the predicted thermal runaway risk decreases over time, a positive reward value of the reinforcement learning agent regarding the determined strategy can be determined; when the predicted thermal runaway risk increases over time to exceed a preset alarm threshold, a larger negative reward value of the reinforcement learning agent regarding the determined strategy can be determined; when the vehicle power limitation state is that the vehicle output power is excessively limited, a smaller negative reward value of the reinforcement learning agent regarding the determined strategy can be determined.

[0012] According to another optional embodiment of the present application, the strategy for intervening in battery thermal runaway may include adjusting the battery cooling system power, adjusting the vehicle's maximum output power, activating / deactivating the battery insulation unit and / or a graded warning strategy, etc., wherein the space of the actions is particularly defined as a continuous and discrete mixed space.

[0013] According to another optional embodiment of the present application, when the predicted thermal runaway risk exceeds a preset risk threshold, a graded warning strategy adapted to the risk value range in which the predicted thermal runaway risk is located is determined based on the predicted thermal runaway risk, wherein, for example, when the predicted thermal runaway risk is greater than or equal to the first risk threshold and less than the second risk threshold, a first-level warning strategy can be determined, and prompt information about the first-level warning strategy can be sent to the back-end server; for example, when the predicted thermal runaway risk is greater than or equal to the second risk threshold and less than the third risk threshold, a second-level warning strategy can be determined, and prompt information about the second-level warning strategy can be sent to the user of the vehicle and the back-end server; for example, when the predicted thermal runaway risk is greater than or equal to the third risk threshold, a third-level warning strategy regarding the emergency mechanism can be determined, and prompt information about the third-level warning strategy can be sent to the user of the vehicle and the back-end server, and the prompt information about the third-level warning strategy includes, for example, parking suggestion information.

[0014] According to another optional embodiment of the present application, the distribution parameters of each strategy can be obtained based on the state through the strategy network of the reinforcement learning agent, wherein the distribution parameters include, for example, the mean and variance of the Gaussian distribution of each strategy, and / or the probability of discrete actions regarding each strategy, etc.

[0015] According to another optional embodiment of the present application, the value of each state can be determined based on the state through the value network of the reinforcement learning agent, wherein the value represents the cumulative reward value obtained by the reinforcement learning agent from executing the determined strategy starting from the state.

[0016] According to another optional embodiment of the present application, the parameters of the policy network can be adjusted using a clipping policy gradient term so that the policy determined by the policy network tends to be a policy with a large advantage value with an update amplitude within a predetermined range, wherein the advantage value is the difference between the reward value obtained by the reinforcement learning agent by executing the determined policy and the value of the state.

[0017] According to another optional embodiment of the present application, the target reward value can be calculated based on the reward value of the current state, the value of the next state and the discount factor, and the parameters of the value network can be adjusted based on the calculated target reward value so that the square of the deviation between the value of the next state determined by the value network and the calculated target reward value is minimized.

[0018] According to another optional embodiment of the present application, time series features can be extracted from the battery operating status and environmental detection information through a deep neural network, wherein the deep neural network includes, for example, a long short-term memory network, and / or a gated recurrent unit network, and / or a large language model based on Transformer, etc.

[0019] According to another optional embodiment of the present application, time series features about the battery cell temperature can be extracted from the battery cell temperature information through the deep learning network, wherein the time series features about the battery cell temperature include, for example, time series features about the temperature gradient at the battery cell level, and / or time series features about the temperature rise rate of the battery cell, and / or time series features about the temperature difference distribution between battery cells, etc.

[0020] According to another optional embodiment of the present application, temperature rise sequence data can be generated by the thermal conduction differential equation of the battery, and the generated temperature rise sequence data can be used to train the deep neural network, so that the time series features extracted by the deep neural network contain information about the thermal conduction process of the battery.

[0021] According to another optional embodiment of the present application, the future temperature change information of the battery can be predicted by the deep neural network based on the extracted time series features, and the thermal runaway risk of the battery can be predicted by the deep neural network based on the extracted time series features and the predicted future temperature change information.

[0022] According to another optional embodiment of the present application, model compression, pruning and / or distillation techniques can be used to simplify the model parameters of the deep neural network, and the simplified model can be deployed in the vehicle control unit through lightweight deployment technology.

[0023] According to another optional embodiment of the present application, the reinforcement learning agent can be deployed in the vehicle control unit in the form of inference acceleration.

[0024] According to another optional embodiment of the present application, the deployed deep neural network and / or reinforcement learning agent can be fine-tuned based on the collected data, wherein the collected data includes battery operating status, environmental detection information, vehicle power limit status, battery insulation unit status, battery cooling system status and / or vehicle operating status, etc.

[0025] According to another optional embodiment of the present application, the battery operating status may include battery cell voltage information, battery cell current information, battery cell temperature information, battery cell internal impedance, battery cell DC resistance, battery internal gas composition information and / or battery internal vibration and pressure information, etc.

[0026] According to another optional embodiment of the present application, the vehicle operating status may include vehicle speed information, vehicle acceleration information, and / or vehicle driving road conditions, etc.

[0027] According to another optional embodiment of the present application, the environmental detection information may include environmental temperature information, environmental humidity information and / or environmental altitude information, etc.

[0028] According to another optional embodiment of the present application, the battery cooling system status may include fan speed information and / or liquid cooling start-up status, etc.

[0029] According to a second aspect of the present application, a battery management system is provided, which may include the following components:

[0030] - a data acquisition module, which is configured to collect battery operating status and environmental detection information;

[0031] and

[0032] An on-board control unit configured to carry out the method according to the present application.

[0033] According to another optional embodiment of the present application, the battery management system may further include an information interaction module, which is configured to send prompt information about the graded warning strategy to the vehicle user and / or the back-end server.

[0034] According to another optional embodiment of the present application, the data acquisition module may also be configured to collect vehicle power limitation status, battery insulation unit status, battery cooling system status, vehicle operation status, etc.

[0035] According to a third aspect of the present application, a vehicle is provided, which may include a battery management system according to the present application.

[0036] According to a fourth aspect of the present application, a computer program product, such as a computer-readable program carrier, is provided, which contains or stores computer program instructions, and when the computer program instructions are executed by a processor, at least assists in implementing the steps of the method described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The principles, features and advantages of the present invention will be better understood by describing the present invention in more detail below with reference to the accompanying drawings.

[0038] Figure 1 A flowchart illustrating a method for preventing battery thermal runaway through reinforcement learning according to an exemplary embodiment of the present application is shown;

[0039] Figure 2 A block diagram showing a battery management system according to an exemplary embodiment of the present application; and

[0040] Figure 3 A schematic diagram of a vehicle according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by this application more clearly understood, this application will be further described in detail below with reference to the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit the scope of protection of this application.

[0042] Figure 1 A flowchart of a method for preventing battery thermal runaway through reinforcement learning according to an exemplary embodiment of the present application is shown. The following exemplary embodiments describe the method according to the present application in more detail.

[0043] like Figure 1 As shown, the method may include steps S1 to S3. In step S1, time series features may be extracted from the collected battery operating status and environmental detection information. In the current embodiment of the present application, during the battery operation process, the battery operating status may be collected by sensors arranged inside or outside the battery, and the battery operating status may include battery cell voltage information, battery cell current information, battery cell temperature information, battery cell internal impedance and / or battery cell DC resistance, etc. Optionally, the collected battery operating status may also include battery internal gas composition information detected by a gas sensor, which can reflect the decomposition products of the battery solvent, and / or battery internal vibration and pressure information detected by a vibration and pressure sensor, etc. In addition, environmental detection information about the operating environment of the battery may also be collected by an environmental sensor, which may include, for example, environmental temperature information, environmental humidity information and / or environmental altitude information, etc.

[0044] In the data preprocessing stage, the battery operating status and environmental detection information collected by these sensors under various complex working conditions (for example, including extreme temperatures, rapid acceleration of the vehicle, different aging states of the battery, etc.) can be cleaned and outliers in this information can be screened out. Considering that the collection of collected battery operating status and environmental detection information is high-dimensional time series data about the acquisition time, a deep neural network can be used to extract low-dimensional time series features h(t) from the battery operating status and environmental detection information, thereby reducing the difficulty of data processing, wherein the deep neural network includes, for example, a long short-term memory network (Long Short-Term Memory, abbreviated LSTM), and / or a gated recurrent unit network (GRU), and / or a large language model based on Transformer, etc. Since the number of model parameters of the deep neural network is very large, model compression, pruning and / or distillation techniques can be used to simplify the model parameters of the deep neural network, and the simplified model can be deployed in the on-board control unit 12 through lightweight deployment technology - such as TensorRT (high-performance inference optimization engine), ONNX (Open Neural Network Exchange, open neural network exchange format) - to achieve acceleration and simplification of the deep neural network model.

[0045] For example, a deep neural network can be used to extract time series features about battery cell temperature from battery cell temperature information, such as time series features about the temperature gradient at the battery cell level, and / or time series features about the temperature rise rate of the battery cell, and / or time series features about the temperature difference distribution between battery cells. The time series features extracted by the deep neural network implicitly contain the thermal conductivity information of the battery, including all battery cells, insulation units, and cooling units. To this end, during the training phase of the deep neural network, temperature rise series data with physical consistency can be generated using the following thermal conduction differential equation of the battery:

[0046]

[0047] Where ρ represents the density of the battery's heat transfer medium, c represents the specific heat capacity of the battery's heat transfer medium, k represents the thermal conductivity of the battery's heat transfer medium, and Q represents the battery's heat generation rate. Next, the generated temperature rise series data can be used to train the deep neural network, enabling it to learn the physical trends of battery heat conduction. The time series features h(t) extracted by the deep neural network contain information about the battery's heat conduction process. Furthermore, the extracted time series features h(t) may also include key latent variables, such as a potential battery aging factor and / or a battery internal stress index.

[0048] In step S2, the thermal runaway risk of the battery can be predicted based at least on the extracted time series feature h(t). Here, the extracted time series feature h(t) can be input into a multi-head network. A classification head model in the multi-head network can predict the thermal runaway risk of the battery based on the extracted time series feature h(t). The classification head model can be configured as a deep neural network, and its structure can be expressed, for example, by the following formula:

[0049] P_TP(t+τ)=σ(W_risk·h(t)+b_risk),

[0050] Wherein, P_TP(t+τ) represents the probability of thermal runaway risk of the battery at the future time t+τ, which is a value between 0 and 1; σ() is the Sigmoid function; W_risk and b_risk are model parameters obtained during the training phase of the classification head model.

[0051] In addition, a classification head model in the multi-head network can also predict the future temperature change information of the battery based on the extracted time series feature h(t) about the battery cell temperature, such as the predicted temperature T(t+τ) of the battery at the future time t+τ, wherein the classification head model can also be configured as a deep neural network. The predicted future temperature change information can be used as an additional influencing factor for predicting the thermal runaway risk of the battery, that is, the deep neural network can predict the thermal runaway risk of the battery based on the extracted time series feature and the predicted future temperature change information, thereby further improving the prediction accuracy of the thermal runaway risk of the battery. For example, if it is determined that the battery temperature will rise rapidly in the future time period based on the predicted future temperature change information, the thermal runaway risk of the battery in the corresponding time period increases.

[0052] In step S3, the reinforcement learning agent determines a strategy for intervening in battery thermal runaway as an action based on the state. In the context of this application, reinforcement learning is a machine learning method based on a Markov decision process that allows the reinforcement learning agent to learn the optimal strategy through trial and error in its interaction with the environment. The method is as follows: the reinforcement learning agent performs an action based on the state, which is a description of the specific situation of the constructed environment at a certain moment, and receives a reward as feedback based on the result of the action. The reward is used to guide the reinforcement learning agent to adjust its strategy for the action to maximize the long-term cumulative reward. The reinforcement learning agent can be deployed in the vehicle control unit 12 in the form of inference acceleration.

[0053] Here, the state s of the reinforcement learning agent may include the predicted thermal runaway risk, the vehicle power limit state, the battery insulation unit state, the battery cooling system state, and the vehicle operating state. Specifically, the vehicle power limit state includes the vehicle output power limit ratio, which is a ratio value from 0 to 1, wherein the smaller the ratio value, the lower the vehicle's maximum output power is limited; the battery insulation unit state indicates whether the battery insulation unit is in an activated or inactivated state; the battery cooling system state indicates the power level currently enabled by the cooling system (e.g., high, medium, low, etc.) and / or the current power value of the cooling system, such as fan speed information and / or liquid cooling start status; the vehicle operating state includes, for example, vehicle speed information, vehicle acceleration information, and / or vehicle operating mode (e.g., uphill mode, high-speed mode, congestion mode, etc.).

[0054] Based on the state s, the reinforcement learning agent can determine from a set of actions a strategy for intervening in battery thermal runaway. In the context of this application, "intervention" can be understood as taking countermeasures to prevent the battery from progressing toward a more severe thermal runaway condition; "strategy" can be defined as a coordinated control method with onboard execution units, including the battery cooling system, the battery insulation unit, and the information interaction module 13. The set of actions can include, for example, one or more of the following strategies: adjusting the battery cooling system power, i.e., adjusting the cooling power level and / or cooling power value enabled by the cooling system; adjusting the vehicle's maximum output power, i.e., adjusting the vehicle's maximum power limit ratio; activating or deactivating the battery insulation unit; and / or a graded warning strategy, which is a human-machine interaction and safety strategy with different warning levels based on the predicted risk range of thermal runaway, where the higher the warning level, the more urgent the human-machine interaction and safety strategy. For example, when the predicted thermal runaway risk exceeds a preset risk threshold, a graded warning strategy adapted to the risk value range of the predicted thermal runaway risk can be determined based on the predicted thermal runaway risk, wherein, for example, when the predicted thermal runaway risk is greater than or equal to the first risk threshold and less than the second risk threshold, a first-level warning strategy can be determined, and prompt information about the first-level warning strategy can be sent to the back-end server 2 through the information interaction module 13; for example, when the predicted thermal runaway risk is greater than or equal to the second risk threshold and less than the third risk threshold, a second-level warning strategy can be determined, and prompt information about the second-level warning strategy can be sent to both the user of the vehicle 1 and the back-end server 2 through the information interaction module 13; for example, when the predicted thermal runaway risk is greater than or equal to the third risk threshold, a third-level warning strategy regarding the emergency mechanism can be determined, and prompt information about the third-level warning strategy can be sent to both the user of the vehicle 1 and the back-end server 2 through the information interaction module 13, and the prompt information about the third-level warning strategy includes, for example, parking suggestion information.

[0055] Specifically, the policy network of the reinforcement learning agent can be used to obtain the distribution parameters of each strategy based on the state s, wherein the distribution parameters include, for example, the mean and variance of the Gaussian distribution of each strategy, and / or the probability of discrete actions for each strategy. In addition, the value network of the reinforcement learning agent can also be used to determine the value V(s) of each state s based on the input state s, wherein the value V(s) represents the expected cumulative reward value obtained by the reinforcement learning agent from executing the determined strategy starting from the state s, which can be used as an estimated score for the state s.

[0056] Here, the space of actions can in particular be defined as a continuous and discrete mixed space, for example, determining the cooling power value and / or the limiting ratio of the vehicle output power through the policy network of the reinforcement learning agent, and determining whether to activate the insulation unit and / or determine the warning level through the classification head of the learning agent.

[0057] During the training of the reinforcement learning agent, the reward value for the determined strategy can be adjusted based on the predicted thermal runaway risk and / or the vehicle power limit state. The reward value is related to the predicted thermal runaway risk, the change in thermal runaway risk over time, and the user experience loss caused by the vehicle power limit state. Therefore, the reward function needs to consider both battery safety and vehicle performance. The reward function can be set as follows:

[0058] r_t=-λ_TP·P_TP(t+τ)-λ_pwr·C_pwr(a_t)+λ_safe·Δ_risk, where r_t represents the reward value of the strategy determined at time t, P_TP(t+τ) represents the probability of thermal runaway risk of the battery at the future time t+τ, λ_TP represents the adjustable weighting factor of P_TP(t+τ), C_pwr(a_t) represents the user experience loss caused by implementing vehicle output power limitation, λ_pwr represents the adjustable weighting factor of C_pwr(a_t), Δ_risk represents the decrease in the probability of thermal runaway risk of the battery in the future time period, and λ_safe represents the adjustable weighting factor of Δ_risk. These adjustable weighting factors can be adjusted during the training process of the reinforcement learning agent, thereby adjusting the reward value for the determined strategy.

[0059] For example, if the predicted thermal runaway risk decreases over time, that is, the battery's thermal runaway risk probability P_TP(t+τ) at the future time t+τ decreases and the decrease in the battery's thermal runaway risk probability in the future time period Δ_risk is positive, a positive reward value for the reinforcement learning agent with respect to the determined strategy is determined. For another example, if the vehicle power limit state is excessive, despite the vehicle's limited power output, based on the principle of safety first, a smaller negative reward value can be determined for the reinforcement learning agent with respect to the determined strategy, i.e., a smaller penalty can be imposed. For another example, if the predicted thermal runaway risk increases over time to exceed a preset alarm threshold, i.e., the battery experiences thermal runaway without the reinforcement learning agent triggering the execution of the strategy in a timely manner, a larger negative reward value can be determined for the reinforcement learning agent with respect to the determined strategy, i.e., a larger penalty can be imposed.

[0060] Next, the strategy determined by the reinforcement learning agent can be adjusted based on the reward value to maximize the return value obtained by the reinforcement learning agent by executing the determined strategy, wherein the return value is used to represent the actual cumulative amount of the reward value obtained by the reinforcement learning agent by executing the determined strategy as it decays over time. For example, it can be represented by the following function:

[0061] J(θ)=E[∑(γ^t·r_t)],

[0062] Where θ represents the parameters of the policy network, γ represents a discount factor between 0 and 1, r_t represents the reward value at time t, γ^t represents the accumulation of reward values ​​at each moment that gradually decays over time, and J(θ) represents the return value with respect to the policy parameter θ.

[0063] During the training of a reinforcement learning agent, algorithms suitable for continuous actions and complex policies can be used to optimize the agent's policy network. These algorithms, for example, include the PRO (Proximal Policy Optimization) algorithm and / or the SAC (Soft Actor-Critic) algorithm—an algorithm based on maximum entropy reinforcement learning. Taking the PRO algorithm as an example, its objective function specifically includes a clipping policy gradient term. This clipping policy gradient term adjusts the policy network's parameters θ so that the policy determined by the policy network tends to favor the policy with a larger advantage value A, with an update amplitude within a predetermined range. The advantage value A is the difference between the reward value J obtained by the reinforcement learning agent by executing the determined policy and the value V(s) of state s. The reward value J represents the actual cumulative reward obtained by the reinforcement learning agent by executing the determined policy, while the value V(s) represents the expected cumulative reward obtained by the reinforcement learning agent by executing the determined policy starting from state s. If the advantage value A is positive, that is, the actual cumulative reward value is greater than the expected cumulative reward value, which means that the current strategy under the current state s is better than expected, then the parameter θ of the strategy network can be adjusted so that the strategy determined by the strategy network tends to the current strategy with an update amplitude within a predetermined range.

[0064] In addition, the parameters of the value network of the reinforcement learning agent can also be optimized. Specifically, the target return value R_target can be calculated based on the reward value r of the current state s, the value V(s_next) of the next state s_next, and the discount factor γ, which can be expressed, for example, by the following formula:

[0065] R_target=r+γ·V(s_next).

[0066] Next, the parameters of the value network can be adjusted based on the calculated target reward value R_target so that the square of the deviation between the value V(s_next) of the next state determined by the value network and the calculated target reward value R_target is minimized, that is, the value V(s_next) of the next state determined by the value network is made as close as possible to the calculated target reward value R_target, thereby ensuring that the advantage value calculated by the updated value network is more reliable.

[0067] It should be noted that the process of optimizing the policy network of the reinforcement learning agent and the process of optimizing the value network of the reinforcement learning agent can be executed independently of each other or simultaneously, and the updating of the policy network parameters and the updating of the value network parameters do not interfere with each other.

[0068] After deploying the trained deep neural network and reinforcement learning agent in the on-board control unit 12, the deployed deep neural network and / or reinforcement learning agent can be fine-tuned based on the collected data, including battery operating status, environmental detection information, vehicle power limit status, battery insulation unit status, battery cooling system status and / or vehicle operating status, to ensure that safe and efficient thermal runaway prevention strategy decisions can be maintained despite battery aging, climate change and / or changes in user driving habits.

[0069] According to an embodiment of the present application, low-dimensional time series features are extracted from high-dimensional time series data collected from multiple data sources through a deep neural network, and the thermal runaway risk of the battery is quickly and efficiently evaluated based on the extracted time series features. By adjusting the parameters of the strategy network and / or value network of the reinforcement learning agent based on the predicted thermal runaway risk, vehicle power limit status, battery insulation unit status, battery cooling system status and vehicle operating status, the reinforcement learning agent can perform dynamic decision-making optimization for the thermal runaway risk of the battery under various complex working conditions, and balance battery safety and vehicle performance through a reward mechanism, so as to minimize the impact on vehicle output power and cruising range while ensuring battery safety.

[0070] In addition, it should be noted that the step numbers described herein do not necessarily represent a chronological order, but are merely a reference mark. The order can be changed according to specific circumstances as long as the technical purpose of this application can be achieved.

[0071] Figure 2 1 is a block diagram of a battery management system 10 according to an exemplary embodiment of the present application.

[0072] like Figure 2 As shown, the battery management system 10 may include the following components:

[0073] - a data acquisition module 11, which is configured to collect battery operating status and environmental detection information, and optionally further configured to collect vehicle power limit status, battery insulation unit status, battery cooling system status and vehicle operating status; and

[0074] An on-board control unit 12 configured to carry out the method according to the present application.

[0075] Optionally, the battery management system 10 may further include an information interaction module 13 configured to send prompt information regarding the graded warning strategy to the user of the vehicle 1 and / or the backend server 2. Here, the information interaction module 13 may include, for example, an on-board communication device for sending prompt information regarding the graded warning strategy to the backend server 2, and an on-board voice device, instrument panel, central control display screen, and / or head-up display, etc. for sending prompt information regarding the graded warning strategy to the user of the vehicle 1.

[0076] Figure 3 A schematic diagram of a vehicle 1 according to an exemplary embodiment of the present application is shown. The vehicle 1 may include a battery management system 10 according to the present application.

[0077] It should be understood that, in this document, the expressions "first", "second", "third", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance, nor should they be understood as implicitly indicating the quantity of the indicated technical features.

[0078] If an embodiment includes an "and / or" relationship between a first feature and a second feature, it should be interpreted as follows: according to one embodiment, the embodiment has both the first feature and the second feature, and according to another embodiment, the embodiment has either only the first feature or only the second feature.

[0079] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative and not limiting, unless otherwise stated. In specific implementations, multiple features may be combined with each other, depending on actual needs, where technically feasible. Various substitutions, changes, and modifications may be contemplated without departing from the spirit and scope of the present application.

Claims

1. A method for preventing battery thermal runaway through reinforcement learning, the method comprising: Extract time series features from the collected battery operating status and environmental detection information; predicting a thermal runaway risk of the battery based at least on the extracted time series features; as well as The reinforcement learning agent determines as an action a strategy for intervening in battery thermal runaway based on states, wherein the states include a predicted thermal runaway risk, a vehicle power limit state, a battery insulation unit state, a battery cooling system state, and a vehicle operating state.

2. The method according to claim 1, wherein The reward value for the determined strategy is adjusted based on the predicted thermal runaway risk and / or the vehicle power limitation state, and the strategy determined by the reinforcement learning agent is adjusted based on the reward value, so that the return value obtained by the reinforcement learning agent by executing the determined strategy is maximized, wherein the reward value is related to the predicted thermal runaway risk, the change in the thermal runaway risk over time, and the user experience loss caused by the vehicle power limitation state, wherein the return value is used to represent the cumulative amount of decay of various reward values ​​obtained by the reinforcement learning agent by executing the determined strategy over time.

3. The method according to claim 2, wherein: Determining a positive reward value for the reinforcement learning agent with respect to the determined strategy if the predicted risk of thermal runaway decreases over time; and / or If the predicted risk of thermal runaway increases over time to exceed a preset alarm threshold, determining a larger negative reward value for the reinforcement learning agent with respect to the determined strategy; and / or When the vehicle power limitation state is that the vehicle output power is over-limited, a smaller negative reward value of the reinforcement learning agent with respect to the determined strategy is determined.

4. The method according to any one of claims 1 to 3, wherein Strategies for intervening in battery thermal runaway include adjusting battery cooling system power, adjusting vehicle maximum output power, activating / deactivating battery insulation units, and / or hierarchical warning strategies, wherein the action space is particularly defined as a continuous and discrete mixed space; and / or In the case where the predicted thermal runaway risk exceeds a preset risk threshold, a hierarchical warning strategy adapted to the risk value interval in which the predicted thermal runaway risk is located is determined based on the predicted thermal runaway risk, wherein, for example, in the case where the predicted thermal runaway risk is greater than or equal to a first risk threshold and less than a second risk threshold, a first-level warning strategy is determined, and prompt information about the first-level warning strategy is sent to the back-end server (2); for example, in the case where the predicted thermal runaway risk is greater than or equal to a second risk threshold and less than a third risk threshold, a second-level warning strategy is determined, and prompt information about the second-level warning strategy is sent to the user of the vehicle (1) and the back-end server (2); for example, in the case where the predicted thermal runaway risk is greater than or equal to a third risk threshold, a third-level warning strategy regarding an emergency mechanism is determined, and prompt information about the third-level warning strategy is sent to the user of the vehicle (1) and the back-end server (2), wherein the prompt information about the third-level warning strategy includes, for example, parking suggestion information.

5. The method according to any one of claims 1 to 4, wherein Obtaining distribution parameters of each strategy through a policy network of a reinforcement learning agent based on the state, wherein the distribution parameters include, for example, the mean and variance of a Gaussian distribution of each strategy, and / or the probability of a discrete action of each strategy; and / or The value of each state is determined based on the state through the value network of the reinforcement learning agent, wherein the value represents the cumulative reward value obtained by the reinforcement learning agent from the state to execute the determined strategy.

6. The method according to claim 5, wherein: The parameters of the policy network are adjusted using a clipping policy gradient term so that the policy determined by the policy network tends to favor a policy with a large advantage value with an update amplitude within a predetermined range, wherein the advantage value is the difference between the reward value obtained by the reinforcement learning agent by executing the determined policy and the value of the state.

7. The method according to claim 5, wherein: A target reward value is calculated based on a reward value regarding a current state, a value of a next state, and a discount factor, and parameters of the value network are adjusted based on the calculated target reward value so that the square of the deviation between the value of the next state determined by the value network and the calculated target reward value is minimized.

8. The method according to any one of claims 1 to 7, wherein Time series features are extracted from the battery operating status and environmental detection information through a deep neural network, wherein the deep neural network includes, for example, a long short-term memory network, and / or a gated recurrent unit network, and / or a large language model based on Transformer.

9. The method according to claim 8, wherein The deep learning network is used to extract time series features about the battery cell temperature from the battery cell temperature information, wherein the time series features about the battery cell temperature include, for example, time series features about the temperature gradient at the battery cell level, and / or time series features about the temperature rise rate of the battery cell, and / or time series features about the temperature difference distribution between the battery cells.

10. The method according to claim 8 or 9, wherein Temperature rise sequence data is generated by a thermal conduction differential equation of the battery, and the deep neural network is trained using the generated temperature rise sequence data, so that the time series features extracted by the deep neural network contain information about the thermal conduction process of the battery.

11. The method according to any one of claims 8 to 10, wherein Based on the extracted time series features, the future temperature change information of the battery is predicted by the deep neural network, and based on the extracted time series features and the predicted future temperature change information, the thermal runaway risk of the battery is predicted by the deep neural network.

12. The method according to any one of claims 8 to 11, wherein Simplifying the model parameters of the deep neural network using model compression, pruning and / or distillation techniques, and deploying the simplified model in an onboard control unit (12) using lightweight deployment technology; and / or The reinforcement learning agent is deployed in the vehicle control unit (12) in the form of inference acceleration; and / or Fine-tune the deployed deep neural network and / or reinforcement learning agent based on the collected data, wherein the collected data includes battery operating status, environmental detection information, vehicle power limit status, battery insulation unit status, battery cooling system status and / or vehicle operating status.

13. The method according to any one of claims 1 to 12, wherein The battery operating status includes battery cell voltage information, battery cell current information, battery cell temperature information, battery cell internal impedance, battery cell DC resistance, battery internal gas composition information and / or battery internal vibration and pressure information; and / or The vehicle operating status includes vehicle speed information, vehicle acceleration information, and / or vehicle driving road conditions; and / or The environmental detection information includes environmental temperature information, environmental humidity information and / or environmental altitude information; and / or The battery cooling system status includes fan speed information and / or liquid cooling start status.

14. A battery management system (10), comprising the following components: A data acquisition module (11) configured to acquire battery operating status and environmental detection information; and An on-board control unit (12) configured to execute the method according to any one of claims 1 to 13.

15. The battery management system (10) according to claim 14, wherein: The battery management system (10) further comprises an information interaction module (13) configured to send prompt information about the graded warning strategy to a user of the vehicle (1) and / or a back-end server (2); and / or The data acquisition module (11) is also configured to acquire vehicle power limitation status, battery insulation unit status, battery cooling system status and vehicle operation status.

16. A vehicle (1) comprising a battery management system (10) according to claim 14 or 15. 17 . A computer program product, such as a computer-readable program carrier, comprising or storing computer program instructions, which, when executed by a processor, at least assist in implementing the steps of the method according to claim 1 .

Images