Predictive thermal management control method, device and system for electric vehicle and storage medium

Abstract

The invention discloses a predictive thermal management control method, device and system for an electric vehicle, and a storage medium. The method comprises the following steps: S1, predicting working condition data in a vehicle driving process through a DQN reinforcement learning method; wherein the working condition data comprises vehicle speed and battery current; and S2, according to the vehicle speed and the battery current, the optimal temperature track of the battery is obtained through an upper-layer control strategy by utilizing a global optimal algorithm with the minimum energy loss as the target, the optimal temperature track is obtained through tracking of a lower-layer control strategy, and the thermal management system of the electric vehicle is controlled with the optimal energy consumption of the thermal management system as the target. By the adoption of the technical scheme, the battery temperature reaches the optimal state in the driving process, the energy consumption of the heat management system reaches the minimum, the energy consumption of the system is finally reduced, and the driving range of the electric vehicle is increased.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric vehicles, and particularly relates to a predictive thermal management control method, device, system, and storage medium for electric vehicles. Background Art

[0002] With the development of energy, electric vehicles are increasingly becoming a trend in today's society. To alleviate the energy crisis and reduce carbon emissions, it is of utmost importance to vigorously develop electric vehicles. At present, most of the control strategies of the thermal management system are reactive controls for the current working conditions, lacking the ability to predict future driving conditions and thus quickly respond to changing vehicle speeds to achieve energy consumption reduction of the thermal management system. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a predictive thermal management control method, device, system, and storage medium for electric vehicles.

[0004] To achieve the above object, the present invention adopts the following technical solutions: A predictive thermal management control method for an electric vehicle, comprising: Step S1, predicting the working condition data during the driving of the vehicle by the DQN reinforcement learning method; wherein, the working condition data includes: vehicle speed and battery current; Step S2, according to the vehicle speed and battery current, obtaining the optimal temperature trajectory of the battery with the minimum energy loss as the target by the upper-layer control strategy using the global optimal algorithm, and tracking the obtained optimal temperature trajectory by the lower-layer control strategy, and using an intelligent agent to perform collaborative control on the components of the electric vehicle thermal management system with the optimal energy consumption of the thermal management system as the target.

[0005] Preferably, the components of the thermal management system include: a compressor, a condenser, an evaporator, a battery, and a fan.

[0006] The present invention also provides a predictive thermal management control device for an electric vehicle, comprising: A first processing module, configured to predict the working condition data during the driving of the vehicle by the DQN reinforcement learning method; wherein, the working condition data includes: vehicle speed and battery current; A second processing module, configured to, according to the vehicle speed and battery current, obtain the optimal temperature trajectory of the battery with the minimum energy loss as the target by the upper-layer control strategy using the global optimal algorithm, and track the obtained optimal temperature trajectory by the lower-layer control strategy, and use an intelligent agent to perform collaborative control on the components of the electric vehicle thermal management system with the optimal energy consumption of the thermal management system as the target.

[0007] Preferably, the components of the thermal management system include: a compressor, a condenser, an evaporator, a battery, and a fan.

[0008] The present invention also provides a predictive thermal management control system for an electric vehicle, comprising: a memory and a processor, wherein a computer program run by the processor is stored on the memory, and the computer program executes a predictive thermal management control method for an electric vehicle when being run by the processor.

[0009] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes a predictive thermal management control method for an electric vehicle when running.

[0010] The present invention adopts a hierarchical control strategy based on the driving condition, so that the battery temperature reaches the optimal state and the energy consumption of the thermal management system is minimized during driving, ultimately reducing the system energy consumption and increasing the driving range of the electric vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0012] Figure 1 It is a flowchart of the predictive thermal management control method for an electric vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0014] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0015] Embodiment 1: As Figure 1 shown, an embodiment of the present invention provides a predictive thermal management control method for an electric vehicle, comprising: Step S1, predicting the condition data during the driving of the vehicle by the DQN reinforcement learning method; wherein, the condition data includes: vehicle speed and battery current; Step S2: According to the vehicle speed and battery current, the upper-layer control strategy uses the global optimal algorithm to obtain the optimal temperature trajectory of the battery with the goal of minimizing energy loss. The lower-layer control strategy tracks the obtained optimal temperature trajectory, and the agent is used to coordinately control the components of the electric vehicle thermal management system with the goal of minimizing the energy consumption of the thermal management system. Among them, the components of the thermal management system include: compressor, condenser, evaporator, battery, and fan. In step S2, the system current is calculated based on the vehicle speed information, and then the internal heat generation power of the battery is further deduced. The dynamic change trajectory of the battery temperature is deduced from the heat balance equation. The change in vehicle speed has a direct coupling relationship with the battery temperature, and a vehicle speed-temperature mapping function can be constructed under different working conditions to provide prediction support for the subsequent control strategy. This temperature model supports the optimal trajectory planning of the upper-layer control strategy, and the coordinated adjustment of the components of the thermal management system is realized through the lower-layer agent, achieving the goal of optimal comprehensive energy consumption. The specific steps are as follows: The driving force during vehicle driving is determined by the vehicle dynamics model, and the battery current , where is the driving power, is the battery voltage, is the vehicle speed, is the driving system efficiency, = , where , , . Then, according to the Joule heat effect, the heat generation of the battery is caused by the internal resistance: , where: is the heat generation per unit time, is the internal resistance of the battery. After that, according to the law of conservation of energy, the temperature rise is determined by the difference between heat generation and heat dissipation: . Integrating to obtain the temperature change with time: = , where is the battery temperature, is the heat dissipation power of the cooling system, is the battery mass, is the specific heat capacity of the battery. From this relationship, the battery temperature change ， can be obtained. Thus, the upper-layer control strategy can use the global optimal algorithm to obtain the optimal temperature trajectory of the battery with the goal of minimizing energy loss. The lower-layer control strategy tracks the obtained optimal temperature trajectory, and the agent is used to coordinately control the components of the electric vehicle thermal management system with the goal of minimizing the energy consumption of the thermal management system.

[0016] As an implementation manner of an embodiment of the present invention, in step S1, five elements of reinforcement learning are determined: (1), state space : The environmental state where the agent is currently located. For vehicle driving condition prediction, the state can include vehicle speed , acceleration , throttle opening , steering angle , environmental temperature , battery temperature , that is , these state variables form the input of the reinforcement learning model, and the agent needs to predict future driving conditions based on this information.

[0017] (2) Action space : Actions that the agent can choose. In driving condition prediction, actions may include adjusting the throttle opening, braking intensity, and steering angle. The agent selects the best action according to the current state to make the vehicle driving prediction more accurate and ensure the efficient operation of the power system.

[0018] (3) Reward function : The feedback obtained by the agent after executing an action. The error between the predicted vehicle speed and the actual vehicle speed is selected as the reward function , where represents the predicted acceleration, represents the weight controlling the acceleration change. The reward function is used to measure the quality of the current action. This reward function ensures that the agent can minimize the error when predicting driving conditions, while optimizing energy consumption and driving stability.

[0019] (4) Policy : The probability distribution for the agent to select an action according to the state , where is estimated by the Deep Q-Network (DQN), is the temperature parameter.

[0020] (5) Value function : Represents the expected cumulative reward after taking an action in the state .

[0021] DQN is a reinforcement learning method based on Q-learning, which combines a deep neural network (DNN) to approximate the Q-value function , and solves the difficulties of traditional Q-learning in high-dimensional state spaces. The core idea of Q-learning is to update the state-action value function , and each action selected by the agent will affect the future cumulative reward. The goal is to maximize the cumulative long-term reward.

[0022] The update formula for the Q-value function is: ; Among them, and respectively represent the state and action at time ; is the reward obtained after performing the action at time ; is the discount factor, which controls the influence of future rewards; is the learning rate, which controls the step size of Q-value update.

[0023] DQN uses a deep neural network to approximate the Q-value function, where are the parameters of the neural network. When updating the Q-value, DQN uses experience replay and a target network to improve the stability of training.

[0024] Experience replay: Each interaction of the agent generates an experience tuple , and these tuples are stored in the experience pool. A batch of samples is randomly selected to train the neural network, reducing the correlation between data.

[0025] Target network: To improve the stability of training, DQN uses a target network , and the parameters of this network are periodically copied from the online network .

[0026] The goal of updating the Q-value is:

[0027] As an implementation manner of an embodiment of the present invention, in step S1, the working condition data during the driving of the vehicle is predicted by the deep Q-learning (DQN) method, including: Initialization stage: S11. Initialize the Q-value network and the target network , and set the initial parameters and ; S12. Initialize the experience replay pool ; S13. Set the learning rate , the discount factor , and the interval for updating the target network; Training process stage: S14. Set the initial state ; S15. Select an action: Select an action according to the current policy, using the ϵ-greedy policy: ; Among them, is the exploration rate, which is used to balance exploration and exploitation; S16. Execute action: Execute action , and obtain the reward and the next state according to the feedback of the environment ; ; S17. Store experience: Store the experience tuple in the experience replay pool ; S18. Update Q value: Randomly sample a batch of experiences from the experience pool , calculate the target value , , and update the parameters of the Q value network by minimizing the loss function , and update the target network every fixed number of steps. , , S19. Update strategy: As the training progresses, update the strategy and continue to execute.

[0028] S20. Convergence verification: Finally, perform convergence and testing. The training process will continue until the model converges, that is, the Q value is stable and the desired control strategy is achieved. In the testing phase, the agent executes actions according to the learned strategy, evaluates its performance in the actual environment, adjusts the strategy according to the training results, and then continues training until the results converge.

[0029] As an implementation manner of the embodiment of the present invention, in step S2, the battery temperature is selected as the system state variable , and the battery loop cooling capacity is selected as the system control quantity

[0030] Establish a state equation for the thermal management system ; Among them, is the battery demand power at time t, is the compressor cooling work at time t, is the instantaneous change rate of the battery temperature.

[0031] Establish a corresponding objective function with the goal of global temperature control and minimum energy consumption as

[0032] Among them, , among which, is the battery heat dissipation, is the state cost coefficient in the dynamic algorithm, is the efficiency matrix of the cooling system, used to calculate the system cooling efficiency, , is the control cost coefficient in the dynamic algorithm, which restricts the values of each variable. During the entire algorithm iteration process, the following constraints need to be satisfied: .

[0033] Based on the dynamic algorithm, the hierarchical iterative algorithm decomposes complex problems into multiple levels, with each level corresponding to a sub-problem, ensuring clear dependencies between sub-problems. During the solution process, after initialization, hierarchical loop iteration is performed. During the upper-level solution process, the solution of the current lower level is fixed, the upper-level problem is solved and the upper-level variables are updated. During the lower-level solution process, according to the updated results of the upper level, the lower-level problem is optimized and the lower-level variables are solved. When the convergence condition is met, the iteration is terminated; otherwise, return to the upper level for further optimization. is the objective function of the stage, representing the optimal cost under state . The specific objective is to obtain the optimal solution of the system by minimizing this objective. is the control strategy of the stage, representing the optimal control quantity under state . At the stage, the optimal control quantity under state is the optimal control quantity obtained by minimizing the cost function : : ; Among them, represents the number of optional control strategies.

[0034] Calculate the optimal cost function of the previous stage through the recurrence relation , where: is the cost function calculated based on the state of the previous stage and the control , is the cost function of the current stage , representing the cost of the next state . In different stages, in order to ensure smooth transitions between states and optimized control strategies, additional constraints on states and controls need to be introduced, using the following formula: ; Among them, after executing the control under state , the resulting state must satisfy certain constraints to ensure compliance with the system dynamics during state transformation.

[0035] Maintain the optimal operating temperature of the battery by optimizing the control strategy. In state transformation and control strategy update, the following formula: ; Through proportional gain and the power of the temperature error control, maintain the battery temperature within the optimal temperature range.

[0036] Furthermore, use a multi-agent system to manage the air conditioning system in an electric vehicle to ensure battery thermal management is maintained at optimal energy efficiency.

[0037] The compressor agent adopts deep deterministic policy gradient (DDPG), which is suitable for non-linear systems. It obtains the required cooling capacity according to the upper-layer control algorithm , inputs the compressor speed and the power demand constraint of the current battery thermal management system, and outputs the compressor power and calculates the pressure ratios of the evaporator and condenser, interacts with the electronic expansion valve, condenser fan, and battery agent. The condenser agent adopts deep Q-network (DQN) and can be better applied to the mainstream gear control of the current condenser fan. The agent inputs the target condensation temperature and receives the high-temperature and high-pressure gas output from the compressor, and outputs the speed of the condenser fan and the condensation temperature, and cooperates with the compressor agent to adjust the heat exchange efficiency. The evaporator agent adopts the proximal policy optimization (PPO) algorithm, which is used to handle complex environments and deal with the continuous control of the expansion valve opening. Taking the temperature and humidity of the passenger compartment and the target temperature of the evaporator as inputs, it outputs the opening of the electronic expansion valve and calculates the refrigerant mass flow rate, controls the refrigerant flow, improves the evaporation efficiency, and cooperates with the fan agent to adjust the air temperature. The fan agent adopts the twin delayed deep deterministic policy gradient (TD3) algorithm, which is suitable for complex environments and can handle the continuous speed regulation of the fan. Taking the supply air temperature and the in-vehicle temperature sensor data as inputs, it outputs the air supply volume and interacts with the evaporator, etc. When increasing the wind speed, the evaporator temperature can be reduced, and when reducing the wind speed, the passenger comfort can be improved. The battery agent adopts the soft actor-critic (SAC) reinforcement learning method, which is suitable for complex battery cooling systems, is beneficial to optimizing the coolant flow control, and takes the current battery temperature 、the maximum safe temperature of the battery 、the ambient temperature as inputs, adjusts the coolant flow rate and starts battery cooling. When the battery temperature is reached, start the air conditioning system for cooling and adjust the compressor power to prevent the battery from overheating.

[0038] Agent training process: 1. The compressor agent aims to dynamically adjust the compressor power, reduce energy consumption, and ensure the stability of the battery temperature. Using the deep deterministic policy gradient (DDPG) algorithm, taking the battery temperature 、the target temperature and ambient temperature As the input state, with the power range of the compressor being 0 to 5000W as the action decision, design the reward function as Train in the established training environment. Set the initial state as the outdoor temperature and randomly initialize the compressor power. The agent explores through trial and error to learn the optimal power adjustment strategy. Train until the reward converges and verify the performance of the model under different temperature conditions. 2. The condenser agent aims to ensure that the condenser remains at the optimal temperature by adjusting the condenser fan speed. Using the condenser temperature , target temperature and ambient temperature as the state input and the fan speed as the action space, design the reward function as , adopt the deep Q-network algorithm. First, set the initial state as the initial temperature of the condenser, randomly start the fan, and the agent learns how to adjust the fan speed to optimize the condensation efficiency. Train until the reward converges and evaluate the performance under different external temperatures. 3. The evaporator agent aims to adjust the refrigerant flow rate, control the vehicle interior temperature, and maintain passenger comfort. Using the target temperature , vehicle interior humidity as the state input and the refrigerant flow rate as the action space, design the reward function as where is the refrigerant mass flow rate. Train the agent to dynamically adjust the refrigerant flow rate and find the best balance between energy conservation and comfort. Train until the temperature error is minimized and evaluate the energy conservation effect. 4. The battery agent aims to control the coolant flow rate to ensure that the battery temperature is stable between 15°C and 40°C. Using the battery temperature , target temperature , ambient temperature as the state input and the coolant flow rate as the action space, design the reward function , apply the SAC algorithm, adopt the battery thermal model, and simulate the battery temperature changes under different driving conditions. 5. The fan agent uses the current temperature of the passenger compartment , current humidity of the passenger compartment , target temperature , current fan speed and the fan energy consumption as the action space, design the reward function , while ensuring passenger comfort, minimize the fan energy consumption Train the agent to control the coolant flow rate and optimize the battery temperature.

[0039] To achieve precise tracking of the battery temperature trajectory, a deep learning method is used to optimize the control strategy, making the battery temperature close to the target temperature trajectory and minimizing the temperature error and energy consumption. First, define the system modeling and state space, and establish a model of the battery system, especially the dynamic characteristics of battery temperature management, including the temperature error in the current system state ( ) and the cooling capacity of the battery circuit as the control variable for regulating the battery temperature, and in order to capture the dynamic historical temperature information of the system 、 and the previous cooling capacity, etc., to form time series data. Through the above state variables, define the current system state , and use this information as the input of the deep learning model to help the model learn the dynamic changes of the battery temperature. Furthermore, define the action space. The agent needs to make a decision based on the current state and select an action. At this time, we choose the change in the cooling capacity of the battery circuit as the action space , where is an action representing the change in cooling capacity (such as increasing or decreasing the cooling of the battery circuit). The action space is a continuous space, so a policy network of deep reinforcement learning can be used to learn the selection of actions. After that, design the reward function. The reward function is the key in reinforcement learning, which determines the learning goal of the agent. In this problem, our goal is to minimize the temperature error and keep the energy consumption of the thermal management system at the lowest level. Therefore, we need to design a reward function that combines the temperature error and energy consumption. The cooling capacity of the battery will consume a certain amount of energy, which can be expressed as the energy consumption of the system: where is the energy consumption function, which is usually proportional to the cooling capacity . The designed reward function can be expressed as the penalty of the temperature error and the trade-off of the energy consumption: , where is the energy consumption penalty factor, is the energy consumption of the thermal management system at the current moment, is the penalty factor of the temperature error, controlling the importance of the temperature error in the reward. After that, select and train the deep learning model. Use deep reinforcement learning (DRL) to learn and optimize the control strategy. The agent continuously updates the policy network by interacting with the environment to minimize the temperature error and energy consumption. During the training process, since the action space is non-discrete and continuous, the deep Q-network (DQN) of Q-learning is used to select the optimal action by maximizing the future reward. The agent will execute the action based on the current state at each time step , obtain a new state from the environment and rewards . This process will continue and accumulate experience continuously. The reinforcement learning algorithm is selected with the goal of updating the Q-value as: where the target value , is the Q-value network, is the target network, and is the initial parameter learning rate , is the discount factor. The training process updates the neural network parameters through backpropagation, thus continuously improving the decision-making ability of the agent. The agent learns how to adjust the cooling capacity according to the current temperature error and system state, so as to minimize the temperature error and reduce energy consumption. The training process updates the neural network parameters through backpropagation, thus continuously improving the decision-making ability of the agent. After training is completed, the agent will be able to adjust the cooling capacity of the battery circuit in real time according to the current temperature error and historical state. For each time step , the agent predicts the control strategy for the next moment according to the current state , and makes the temperature approach the target trajectory by adjusting system parameters (such as cooling capacity). Specifically, the agent will calculate the temperature error according to the target trajectory and the current temperature , and then adjust according to the policy obtained through training , and continuously correct and optimize in subsequent steps. The goal of training is to minimize the total temperature error in the whole process. To achieve this, the temperature error can be weighted averaged or the errors at multiple moments can be optimized through cumulative rewards to ensure that the temperature trajectory is always within the optimal range. The following optimization goal is adopted during the training process: where,

[0040] where, is the total training time, and the optimization goal is to minimize the total temperature error during the whole process. After the training is completed, the agent will be able to control the cooling capacity of the battery circuit in real time to make the battery temperature as close as possible to the target trajectory. When the system is deployed, the model should be strictly verified and evaluated to ensure that it can exhibit good stability and adaptability under different loads and environmental conditions. The evaluation metrics can include: (1) Temperature error: Whether the temperature error remains within a reasonable range during the overall training process. (2) System energy consumption: Whether the energy consumption of the system is lower than the preset threshold and the use of the cooling capacity is optimized as much as possible. (3) System response time: The speed and accuracy of temperature adjustment to ensure that the system can quickly respond to battery temperature changes. Optimizing the battery temperature control strategy through deep reinforcement learning can not only achieve accurate temperature trajectory tracking, but also effectively reduce energy consumption and improve the stability and adaptability of the system. The ultimate goal is that through the continuous learning of the agent, the battery temperature management system can adaptively adjust in various actual environments to ensure that the battery operates within the optimal temperature range, thereby extending the battery life and improving the system efficiency.

[0041] Embodiment 2: The embodiment of the present invention further provides an anticipatory thermal management control device for an electric vehicle, including: A first processing module for predicting the operating condition data during the driving process of the vehicle through the DQN reinforcement learning method; wherein, the operating condition data includes: vehicle speed and battery current; A second processing module for, according to the vehicle speed and battery current, obtaining the optimal temperature trajectory of the battery with the minimum energy loss as the goal through the upper-layer control strategy using the global optimal algorithm, and tracking the obtained optimal temperature trajectory through the lower-layer control strategy, and using an agent to perform collaborative control on the components of the electric vehicle thermal management system with the optimal energy consumption of the thermal management system as the goal.

[0042] As an implementation manner of the embodiment of the present invention, the components of the thermal management system include: a compressor, a condenser, an evaporator, a battery, and a fan.

[0043] Embodiment 3: The present invention further provides an anticipatory thermal management control system for an electric vehicle, including: a memory and a processor, and a computer program is stored on the memory and run by the processor, and the computer program, when run by the processor, executes the anticipatory thermal management control method for an electric vehicle.

[0044] Embodiment 4: The present invention further provides a storage medium, and a computer program is stored on the storage medium, and the computer program, when running, executes the anticipatory thermal management control method for an electric vehicle.

[0045] The embodiments described above are only descriptions of the preferred embodiments of the present invention and do not limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A predictive thermal management control method for an electric vehicle, characterized in that, It includes: Step S1: Predict the operating condition data during the driving of the vehicle through the DQN reinforcement learning method; wherein, the operating condition data includes: vehicle speed and battery current; Step S2: According to the vehicle speed and battery current, use the global optimal algorithm through the upper-layer control strategy to obtain the optimal temperature trajectory of the battery with the goal of minimizing energy loss, track the obtained optimal temperature trajectory through the lower-layer control strategy, and use an intelligent agent to perform collaborative control on the components of the electric vehicle thermal management system with the goal of minimizing the energy consumption of the thermal management system.

2. The predictive thermal management control method for an electric vehicle according to claim 1, wherein The components of the thermal management system include: compressor, condenser, evaporator, battery, and fan.

3. A predictive thermal management control device for an electric vehicle, characterized in that, It includes: The first processing module is used to predict the operating condition data during the driving of the vehicle through the DQN reinforcement learning method; wherein, the operating condition data includes: vehicle speed and battery current; The second processing module is used to, according to the vehicle speed and battery current, use the global optimal algorithm through the upper-layer control strategy to obtain the optimal temperature trajectory of the battery with the goal of minimizing energy loss, track the obtained optimal temperature trajectory through the lower-layer control strategy, and use an intelligent agent to perform collaborative control on the components of the electric vehicle thermal management system with the goal of minimizing the energy consumption of the thermal management system.

4. The electric vehicle predictive thermal management control device according to claim 3, wherein, The components of the thermal management system include: compressor, condenser, evaporator, battery, and fan.

5. A predictive thermal management control system for an electric vehicle, characterized in that, It includes: A memory and a processor, where a computer program is stored on the memory and run by the processor, and the computer program, when run by the processor, executes the electric vehicle predictive thermal management control method described in any one of claims 1-2.

6. A storage medium, characterized in that, A computer program is stored on the storage medium, and the computer program, when running, executes the electric vehicle predictive thermal management control method described in any one of claims 1-2.

Images