Vehicle thermal management method and system based on black box model and deep reinforcement learning

Abstract

The application discloses a kind of whole vehicle thermal management method and system based on black box model and deep reinforcement learning, comprising the following steps: S1: obtaining the state information of automobile under complete working condition;S2: establishing automobile whole vehicle model, including air conditioner and passenger cabin coupled dynamic thermal model;S3: based on the automobile whole vehicle model established in step S2, the data collected in step S1 is used to train the automobile whole vehicle model based on black box model;S4: based on the design of whole vehicle thermal management control strategy of deep reinforcement learning algorithm, the trained black box model is used as the environment, and the intelligent agent is trained by real-time interaction with the whole vehicle thermal management control strategy under different working conditions.The application improves the whole vehicle thermal management effect.

Description

Technical Field

[0001] This invention belongs to the field of vehicle thermal management technology, specifically relating to a vehicle thermal management method and system based on a black box model and deep reinforcement learning. Background Technology

[0002] With the continuous development of the automotive industry, people have placed higher demands on passenger comfort. Therefore, research on vehicle thermal management is particularly important to improve driving and riding comfort. Firstly, during vehicle operation, the passenger compartment continuously generates and absorbs heat. Secondly, whether it's a pure electric vehicle, a hybrid vehicle, or a traditional vehicle, the power system, including the battery, motor, and engine, generates a significant amount of heat during operation. Therefore, the vehicle faces a dynamic and rapidly changing thermal environment, and the temperature of its various modules changes rapidly due to various factors. The core of the vehicle thermal management system is the automotive heating, ventilation, and air conditioning (HVAC) system. In the entire thermal management process, the HVAC system plays a crucial role in heat transfer; its energy consumption accounts for a large portion of the vehicle's total energy consumption. Furthermore, if the passenger compartment is not dynamically and accurately controlled during driving, it will lead to a decrease in thermal comfort, significantly impacting passenger health and safe driving, and negatively affecting the vehicle brand's market image. Therefore, for automobiles, the thermal management system is an important part of energy consumption. Reasonable and effective vehicle thermal management can not only provide passengers with a good driving experience, but also achieve certain energy-saving effects.

[0003] The control methods for automotive air conditioning systems have undergone many evolutions. Initially, on-off control was used, where the air conditioner turned on or off when a certain set condition was met. While simple and easy to operate, this rule-based control method suffered from poor accuracy and significant temperature fluctuations, leading to poor comfort and high energy consumption. Another classic feedback control method processes the feedback error signal to adjust the system's steady-state error and control fluctuations. Proportional-Integral-Derivative (PID) controllers, due to their efficiency and simplicity, are currently the most widely used method. However, PID control is not ideal for complex nonlinear systems and systems with frequently changing external conditions. For dynamic systems like the automotive passenger compartment, which are greatly affected by external environmental changes, relying solely on PID control with fixed parameters is insufficient.

[0004] Therefore, there is a need to develop a vehicle thermal management method and system based on black-box models and deep reinforcement learning. Summary of the Invention

[0005] The purpose of this invention is to provide a vehicle thermal management method and system based on black box model and deep reinforcement learning, which can improve the thermal management effect of the vehicle.

[0006] In a first aspect, the present invention provides a vehicle thermal management method based on a black-box model and deep reinforcement learning, comprising the following steps:

[0007] S1: Obtain the vehicle's status information under complete operating conditions;

[0008] S2: Establish a complete vehicle model, including a dynamic thermal model of the air conditioning and passenger compartment coupling;

[0009] S3: Based on the vehicle model established in step S2, perform black-box model training on the vehicle model using the data collected in step S1.

[0010] S4: Design a vehicle thermal management control strategy based on deep reinforcement learning algorithm, use the trained black box model as the environment, and interact with the vehicle thermal management control strategy in real time under different working conditions to train the intelligent agent.

[0011] Optionally, S1 includes vehicle status information, air conditioning system information, and passenger compartment system information.

[0012] Optionally, the vehicle status information includes at least one of the vehicle speed, engine speed, compressor speed, and fan speed;

[0013] The air conditioning system information includes at least one of the following: volumetric efficiency, isentropic efficiency, expansion valve flow coefficient, evaporator / condenser tube size, evaporator / condenser material specific heat, evaporator / condenser mass, evaporator / condenser air-side heat transfer coefficient, evaporator / condenser windward area, ambient temperature, and solar radiation.

[0014] The passenger compartment system information includes at least one of the following: seat heat transfer coefficient, human body heat dissipation, heat transfer area between the front end and the air, mass of the front end module / side wall / windshield, specific heat of the front end module material / windshield / roof module material / side wall module material, and convective heat transfer coefficient between the windshield / roof / side wall and the outside ambient air.

[0015] Optionally, S2 includes:

[0016] S21: Establish a dynamic thermal model of an automotive air conditioning system, which includes a compressor, condenser, evaporator, expansion valve, and blower.

[0017] S22: Establish a simplified one-dimensional dynamic thermal model of the crew cabin system;

[0018] S23: Establish an engine cooling model and a whole vehicle simulation model based on vehicle dynamics, and couple the passenger compartment system with the engine cooling system through the heater core.

[0019] Optionally, S21 specifically includes:

[0020] Compressor: A mathematical model was built in MATLAB / Simulink using the lumped parameter method to measure the refrigerant mass flow rate at the compressor outlet. The calculation formula is:

[0021]

[0022] Where, η vol It is the volumetric efficiency of the electric compressor, ρ r It is the density of refrigerant R134a at the compressor inlet, N. comp It is the rotational speed of the electric compressor, controlled by the drive motor, V d It refers to the compressor's displacement;

[0023] Expansion valve: During its dynamic process, the refrigerant mass flow rate through the expansion valve Its relationship with the pressure drop ΔP of the expansion valve is expressed by the following formula:

[0024]

[0025] Where C q A is the flow coefficient of the expansion valve. v ρ is the minimum flow area of ​​the expansion valve. v The density of the refrigerant passing through the expansion valve;

[0026] Evaporator: According to the moving boundary method, the gaseous and liquid refrigerant in the evaporator satisfy the law of conservation of mass. Therefore, for the length l of the two-phase region of the evaporator... e The change is obtained from the following formula:

[0027]

[0028]

[0029] Based on the law of conservation of energy, the change in evaporator wall temperature can be expressed as:

[0030]

[0031] Where, ρ le It is the density of the liquid refrigerant in the evaporator, h lge A is the latent heat of vaporization of the refrigerant in the evaporator. e This represents the total cross-sectional area of ​​the flat tube microchannels in the evaporator. The volume fraction of refrigeration vapor in the two-phase region is given by le, where le is the average porosity of the two-phase region of the evaporator, and h is the volume fraction of refrigeration vapor in the two-phase region. ge h le and h ie These represent the enthalpy values ​​of the gas, liquid, and inlet refrigerant in the evaporator at the current pressure, respectively. ie D is the heat transfer coefficient between the evaporator inner wall and the refrigerant in the two-phase region. ie It is the diameter of the inside of the evaporator flat tube, L. e It is the total length of the evaporator flat tube, T we T is the temperature of the evaporator wall. re It is the saturation temperature of the refrigerant at the current pressure of the evaporator, ρ. ge P is the density of refrigerant vapor in the evaporator. e The pressure in the evaporator, (C) p m) we C represents the specific heat of the evaporator material and the mass of the evaporator. p The condenser material has a specific heat capacity, m represents the mass of the evaporator, and a represents the mass of the evaporator. oe It is the heat transfer coefficient between air and the evaporator wall, A oe T represents the air-facing area of ​​the evaporator. ae It is the current temperature of the air surrounding the evaporator;

[0032] Condenser: Sub-model satisfies:

[0033]

[0034]

[0035]

[0036] Assuming there is no refrigerant leak in the air conditioning system, the total mass of refrigerant in the system remains constant. Therefore, the total mass of refrigerant in the evaporator and condenser can be considered constant, hence:

[0037]

[0038] Where, ρ ge L is the density of refrigerant vapor in the evaporator. e A is the total length of the evaporator flat tube. c ρ is the total cross-sectional area of ​​the flat tube microchannels in the condenser. lc It is the density of the liquid refrigerant in the condenser, ρ gc The vapor density in the condenser L is the volume fraction of refrigeration vapor in the two-phase region of the condenser, lc is the average porosity of the two-phase region of the condenser, and L is the volume fraction of refrigeration vapor in the two-phase region of the condenser. c It is the total length of the condenser flat tube, l c Let h be the length of the two-phase region of the condenser, Σ be a constant, and h be the length of the two-phase region of the condenser.lgc The latent heat of vaporization of the refrigerant in the condenser, h lc and h ic These represent the enthalpy values ​​of the liquid in the condenser and the inlet refrigerant, respectively, at the current pressure. ic D is the heat transfer coefficient between the condenser inner wall and the refrigerant in the two-phase region. ic The diameter of the inside of the condenser flat tube, T wc T is the condenser wall temperature. rc P is the saturation temperature of the refrigerant at the current pressure of the condenser. c C is the condensing pressure in the condenser. p The condenser material has a specific heat capacity, m represents the mass of the evaporator, and a represents the mass of the evaporator. ish It is the heat transfer coefficient between the refrigerant and the pipe wall in the superheated zone, T ic a is the temperature of the refrigerant at the condenser inlet. oc It is the heat transfer coefficient between air and the condenser wall, A oc T represents the frontal area of ​​the condenser. ac It is the temperature of the air surrounding the condenser, i.e., the ambient temperature.

[0039] Optionally, the simplified one-dimensional crew cabin system dynamic thermal model in S22 includes:

[0040] Total heat load of the vehicle passenger compartment Represented as:

[0041]

[0042] According to the law of conservation of energy, the air temperature T in the crew cabin is... a Dynamic changes can be represented as:

[0043]

[0044] in, For heat exchange of the vehicle body surface structure, For solar radiation heat load, The heat load caused by ventilation For the human body's heat load, For the heat load of machinery and instruments, The total heat load of the crew compartment, M represents the amount of cooling capacity delivered to the cabin by the air conditioning system per unit time. a C represents the air quality within the volume of a car cabin. pa It is the specific heat of air.

[0045] Optionally, establishing the engine cooling model in S23 includes:

[0046] For the engine sub-model, the engine heat dissipation Q1 is calculated using the following formula:

[0047]

[0048] In the formula: a0 is the percentage of engine heat dissipation to total fuel heat, g e The engine fuel consumption rate is obtained from the internal combustion engine performance table, where P is the engine power and H is the value. n The lower heating value of fuel;

[0049] For the radiator sub-model, when calculating the maximum heat dissipation of the radiator, multiply the engine heat dissipation Q1 by a coefficient of 1.1 to 1.25.

[0050] For the thermostat sub-model, the area of ​​the valve opening at the thermostat inlet is linearly related to the temperature of the thermostat paraffin.

[0051] For the water pump and fan sub-model, the water pump is a centrifugal water pump, and the start and stop of the fan are controlled by a temperature sensor to collect the temperature of the coolant at the radiator outlet.

[0052] Optionally, S3 specifically includes:

[0053] First, the dynamic thermal model of the air conditioning and passenger compartment coupling is input with the following parameters: vehicle speed and engine speed, blower voltage, and heater core temperature under different operating conditions. This yields data on the entire process of air conditioning energy consumption and passenger compartment temperature changes. The data collected in step S1 is randomly allocated according to k1:k2, with k1 serving as the training set. The input and output of this data are used as the input and output of the black box model, which is trained using a random forest machine learning model. Then, the other k2 data is used as the test set to validate the trained black box model, where k1 is greater than k2.

[0054] Optionally, S4 includes:

[0055] S41: State S = {T} cab T target ,RH}, where: S is the set of states containing all states, T cab For the crew cabin temperature, T target To meet the target temperature requirements of most occupants in the thermal comfort zone, RH is the relative humidity of the occupant cabin;

[0056] S42: The agent selects an action A = {FanLevel, V} based on the state S. comp A = BlowerVoltage, CoolHeatLevel}, where A contains all action sets, and FanLevel is the front cooling fan speed; V compThe compressor displacement is indicated by the passenger compartment air conditioning compressor, which is a variable displacement compressor; BlowerVoltage is the blower speed setting; coolHeatLevel is the heater core speed setting, with different speeds corresponding to different ratios of hot and cold air.

[0057] S43: After the agent takes an action, it receives a corresponding reward. The reward function R is defined as:

[0058]

[0059] Among them, E comp For compressor energy consumption, r1 = |T cab -T target | represents the absolute error between the crew cabin temperature and the target temperature, while PMV is calculated by the reference temperature generator;

[0060] S44: After taking an action, the agent receives the corresponding reward and enters the next state S'. It stores the experience samples {S,A,R,S'} in the experience pool, and stores the number of samples in the experience pool to a preset number. It randomly selects a small batch of stored experience samples from the experience pool, calculates the target value for each state, and updates the Q value of the agent based on the reward obtained after performing the action. In the deep reinforcement learning algorithm, a convolutional neural network is used to simulate the target Q function.

[0061] S45: Based on the Q-network established in S44, the deep reinforcement learning algorithm creates two copies of the neural network for both the policy network and the Q-network.

[0062] S46: After training on a small batch of data, update the parameters of the online network using the SGA / SGD algorithm, and then update the parameters of the target network using the soft update algorithm.

[0063] Secondly, the vehicle thermal management system based on black box model and deep reinforcement learning according to the present invention includes a memory and a controller. The memory stores a computer-readable program. When the computer-readable program is called by the controller, it can execute the steps of the vehicle thermal management method based on black box model and deep reinforcement learning according to the present invention.

[0064] This invention has the following advantages: It uses a machine learning-based black-box model instead of traditional physical models and deep deterministic gradient algorithms as the optimization algorithm to improve the overall vehicle thermal management effect. This invention acquires various vehicle state variable information under different complete operating conditions, then establishes a complete vehicle model based on existing data and verifies the established model through simulation. Simultaneously, to better control thermal management, a thermal coupling model of the air conditioning and passenger compartment is established to obtain more accurate vehicle heat generation information. Secondly, based on the established complete vehicle model, different vehicle driving conditions are continuously tested to obtain vehicle air conditioning energy consumption, compressor speed, fan speed, passenger compartment temperature, etc. Using the obtained data, a black-box model (a model that reflects a general direct causal relationship between relevant factors) is adopted, and random forest is used for data classification and integration. Using a trained black-box model as the current environment, a control method for vehicle thermal management based on the deep reinforcement learning algorithm (DDPG) is proposed. The compressor speed and fan speed are used as control actions, and the passenger compartment temperature and air conditioning energy consumption are used as the main components of the reward function. By continuously interacting with the black-box model, the training time can be effectively shortened and the expected control goal can be achieved, namely, to achieve a certain energy-saving effect while ensuring human comfort. Attached Figure Description

[0065] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart of the vehicle thermal management method based on black box model and deep reinforcement learning described in this embodiment;

[0067] Figure 2 This is a flowchart of the training process for the black-box model in this embodiment;

[0068] Figure 3 This is a schematic diagram of the DDPG algorithm in this embodiment;

[0069] Figure 4 This is a flowchart illustrating the process of implementing vehicle thermal management using the DDPG algorithm in this embodiment. Detailed Implementation

[0070] The present invention will now be described in detail with reference to the accompanying drawings.

[0071] like Figure 1 As shown in this embodiment, a vehicle thermal management method based on a black-box model and deep reinforcement learning includes the following steps:

[0072] S1: Obtain the vehicle's status information under complete operating conditions;

[0073] S2: Establish a complete vehicle model, including a dynamic thermal model of the air conditioning and passenger compartment coupling;

[0074] S3: Based on the vehicle model established in step S2, perform black-box model training on the vehicle model using the data collected in step S1.

[0075] S4: Design a vehicle thermal management control strategy based on deep reinforcement learning algorithm. Use the trained black box model as the environment and interact with the vehicle thermal management control strategy in real time under different working conditions in order to train the agent more accurately.

[0076] In this embodiment, S1 includes vehicle status information, air conditioning system information, and passenger compartment system information. Specifically: Vehicle status information includes at least one of vehicle speed, engine speed, compressor speed, and fan speed. Air conditioning system information includes at least one of volumetric efficiency, isentropic efficiency, expansion valve flow coefficient, evaporator / condenser tube dimensions, evaporator / condenser material specific heat, evaporator / condenser mass, evaporator / condenser air-side heat transfer coefficient, evaporator / condenser windward area, ambient temperature, and solar radiation. Passenger compartment system information includes at least one of seat heat transfer coefficient, body heat dissipation, frontal air heat transfer area, front module / sidewall / windshield mass, specific heat of front module material / windshield / roof module material / sidewall module material, and convective heat transfer coefficient between the windshield / roof / sidewall and the external ambient air. The " / " in this paragraph represents "and".

[0077] In this embodiment, S2 includes:

[0078] S21: Establish a dynamic thermal model of an automotive air conditioning system, which includes a compressor, condenser, evaporator, expansion valve, and blower.

[0079] S22: Establish a simplified one-dimensional dynamic thermal model of the crew cabin system;

[0080] S23: Establish an engine cooling model and a whole vehicle simulation model based on vehicle dynamics, and couple the passenger compartment system with the engine cooling system through the heater core.

[0081] In this embodiment, S21 specifically refers to:

[0082] Compressor: A mathematical model was built in MATLAB / Simulink using the lumped parameter method to measure the refrigerant mass flow rate at the compressor outlet. The calculation formula is:

[0083]

[0084] Where, η vol It is the volumetric efficiency of the electric compressor, ρ r It is the density of refrigerant R134a at the compressor inlet, N. comp It is the rotational speed of the electric compressor, controlled by the drive motor, V d It refers to the compressor's displacement;

[0085] Expansion valve: During its dynamic process, the refrigerant mass flow rate through the expansion valve Its relationship with the pressure drop ΔP of the expansion valve is expressed by the following formula:

[0086]

[0087] Where C q A is the flow coefficient of the expansion valve. v ρ is the minimum flow area of ​​the expansion valve. v The density of the refrigerant passing through the expansion valve;

[0088] Evaporator: According to the moving boundary method, the gaseous and liquid refrigerant in the evaporator satisfy the law of conservation of mass. Therefore, for the length l of the two-phase region of the evaporator... e The change is obtained from the following formula:

[0089]

[0090]

[0091] Based on the law of conservation of energy, the change in evaporator wall temperature can be expressed as:

[0092]

[0093] Where, ρ le It is the density of the liquid refrigerant in the evaporator, h lge A is the latent heat of vaporization of the refrigerant in the evaporator. e This represents the total cross-sectional area of ​​the flat tube microchannels in the evaporator. The volume fraction of refrigeration vapor in the two-phase region is given by le, where le is the average porosity of the two-phase region of the evaporator, and h is the volume fraction of refrigeration vapor in the two-phase region. ge h le and h ie These represent the enthalpy values ​​of the gas, liquid, and inlet refrigerant in the evaporator at the current pressure, respectively. ie D is the heat transfer coefficient between the evaporator inner wall and the refrigerant in the two-phase region. ie It is the diameter of the inside of the evaporator flat tube, L. e It is the total length of the evaporator flat tube, T we T is the temperature of the evaporator wall. reIt is the saturation temperature of the refrigerant at the current pressure of the evaporator, ρ. ge P is the density of refrigerant vapor in the evaporator. e The pressure in the evaporator, (C) p m) we C represents the specific heat of the evaporator material and the mass of the evaporator. p The condenser material has a specific heat capacity, m represents the mass of the evaporator, and a represents the mass of the evaporator. oe It is the heat transfer coefficient between air and the evaporator wall, A oe T represents the air-facing area of ​​the evaporator. ae It is the current temperature of the air surrounding the evaporator;

[0094] Condenser: Sub-model satisfies:

[0095]

[0096]

[0097]

[0098] Assuming there is no refrigerant leak in the air conditioning system, the total mass of refrigerant in the system remains constant. Therefore, the total mass of refrigerant in the evaporator and condenser can be considered constant, hence:

[0099]

[0100] Where, ρ ge L is the density of refrigerant vapor in the evaporator. e A is the total length of the evaporator flat tube. c ρ is the total cross-sectional area of ​​the flat tube microchannels in the condenser. lc It is the density of the liquid refrigerant in the condenser, ρ gc The vapor density in the condenser L is the volume fraction of refrigeration vapor in the two-phase region of the condenser, lc is the average porosity of the two-phase region of the condenser, and L is the volume fraction of refrigeration vapor in the two-phase region of the condenser. c It is the total length of the condenser flat tube, l c Let h be the length of the two-phase region of the condenser, Σ be a constant, and h be the length of the two-phase region of the condenser. lgc The latent heat of vaporization of the refrigerant in the condenser, h lc and h ic These represent the enthalpy values ​​of the liquid in the condenser and the inlet refrigerant, respectively, at the current pressure. ic D is the heat transfer coefficient between the condenser inner wall and the refrigerant in the two-phase region. ic The diameter of the inside of the condenser flat tube, T wc T is the condenser wall temperature. rc P is the saturation temperature of the refrigerant at the current pressure of the condenser. c C is the condensing pressure in the condenser.p The condenser material has a specific heat capacity, m represents the mass of the evaporator, and a represents the mass of the evaporator. ish It is the heat transfer coefficient between the refrigerant and the pipe wall in the superheated zone, T ic a is the temperature of the refrigerant at the condenser inlet. oc It is the heat transfer coefficient between air and the condenser wall, A oc T is the frontal area of ​​the condenser. ac It is the temperature of the air surrounding the condenser, i.e., the ambient temperature.

[0101] In this embodiment, the simplified one-dimensional crew cabin system dynamic thermal model in S22 includes:

[0102] Total heat load of the vehicle passenger compartment Represented as:

[0103]

[0104] According to the law of conservation of energy, the air temperature T in the crew cabin is... a Dynamic changes can be represented as:

[0105]

[0106] in,

[0107] For heat exchange of the vehicle body surface structure,

[0108] For solar radiation heat load,

[0109] The heat load caused by ventilation For the human body's heat load, For the heat load of machinery and instruments, The total heat load of the crew compartment, M represents the amount of cooling capacity delivered to the cabin by the air conditioning system per unit time. a C represents the air quality within the volume of a car cabin. pa It is the specific heat of air.

[0110] In this embodiment, establishing the engine cooling model in S23 includes:

[0111] For the engine sub-model, the engine heat dissipation Q1 is calculated using the following formula:

[0112]

[0113] In the formula: a0 is the percentage of engine heat dissipation to total fuel heat, g e The engine fuel consumption rate is obtained from the internal combustion engine performance table, where P is the engine power and H is the value.n It is the lower heating value of the fuel.

[0114] For the radiator sub-model, when calculating the maximum heat dissipation of the radiator, a coefficient of 1.1 to 1.25 should be multiplied based on the engine heat dissipation Q1 to ensure that the entire cooling system design can operate normally under the most severe conditions.

[0115] For the thermostat sub-model, the area of ​​the valve opening at the thermostat inlet is linearly related to the temperature of the thermostat paraffin.

[0116] For the water pump and fan sub-model, the water pump is a centrifugal water pump, and the start and stop of the fan are controlled by a temperature sensor to collect the temperature of the coolant at the radiator outlet.

[0117] In this embodiment, S3 specifically refers to:

[0118] First, the corresponding inputs to the dynamic thermal model of the air conditioning and passenger compartment coupling are: vehicle speed and engine speed, blower voltage, and heater core temperature under different operating conditions. This yields the corresponding data on the entire process of air conditioning energy consumption and passenger compartment temperature change. The data collected in step S1 is randomly distributed according to k1:k3 (e.g., 7:3). 70% of the data is used as the training set. The input and output of this part of the data are used as the input and output of the black box model. A random forest machine learning model is used for training. Then, the other 30% of the data is used as the test set to verify the trained black box model.

[0119] In this embodiment, the black-box model is a type of model widely used in environmental prediction. It is established based on the input-output relationship and reflects a general, direct causal relationship between relevant factors. Figure 2 This is a flowchart of the training process for a black-box model.

[0120] In this embodiment, step S4 utilizes the DDPG algorithm (i.e., deep reinforcement learning algorithm) to achieve vehicle thermal management control, incorporating passenger compartment temperature and air conditioning energy consumption as part of the reward function to ensure the human body remains within the thermal comfort zone. The principle of the DDPG algorithm is as follows: Figure 3 As shown, the DDPG model is specifically as follows:

[0121] S41: State S = {T} cab T target ,RH}, where: S is the set of states containing all states, T cab For the crew cabin temperature, T target To meet the target temperature requirements of most occupants in the thermal comfort zone, RH is the relative humidity of the occupant cabin;

[0122] S42: The agent selects an action A = {FanLevel, V} based on the state S. compA = BlowerVoltage, CoolHeatLevel}, where A contains all action sets, and FanLevel is the front cooling fan speed; V comp The compressor displacement is indicated by the passenger compartment air conditioning compressor, which is a variable displacement compressor; BlowerVoltage is the blower speed setting; coolHeatLevel is the heater core speed setting, with different speeds corresponding to different ratios of hot and cold air.

[0123] S43: After the agent takes an action, it receives a corresponding reward. The reward function R is defined as:

[0124]

[0125] Among them, E comp For compressor energy consumption, r1 = |T cab -T target | represents the absolute error between the crew cabin temperature and the target temperature, while PMV is calculated by the reference temperature generator. The PMV calculation process is as follows:

[0126]

[0127]

[0128]

[0129]

[0130] Where M is the passenger's metabolic rate, W is the effective mechanical power, and P... a The partial pressure of water vapor, t a The temperature of the crew cabin air, t r The average radiant temperature inside the driver's cab. t is the average radiation temperature. cl The surface temperature of the clothing, h c I is the convective heat transfer coefficient of the human body surface. cl For the thermal resistance of the passenger's clothing, f cl v is the surface area coefficient of clothing. ar This refers to relative wind speed.

[0131] S44: After taking an action, the agent receives the corresponding reward and enters the next state S'. It stores the experience samples {S,A,R,S'} in the experience pool, and stores the number of samples in the experience pool to a preset number. It randomly selects a small batch of stored experience samples from the experience pool, calculates the target value for each state, and updates the Q value of the agent based on the reward obtained after performing the action. In the deep reinforcement learning algorithm, a convolutional neural network is used to simulate the target Q function. Figure 4 This is a flowchart of the DDPG algorithm.

[0132] Among them, (s) i a i r i s i+1 ) represents a single transition data in a mini-batch.

[0133] Calculate the gradient of the online Q-network, where the loss of the Q-network is defined as MSE: mean squared Qerror.

[0134]

[0135] Among them, y i It can be viewed as a "tag":

[0136] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ )

[0137] Where: L represents the loss function, N represents the count, and Q(s) represents the loss function. i a i |θ Q ) indicates the current state s i Take action a i The evaluation Q-value function, where θ represents the weight parameters of the neural network, y i R represents the predicted return value. i Let Q'(s) represent the reward value at the current moment, γ represent the discount factor, and Q′(s) represent the reward value at the current moment. i+1 ,μ'(s i+1 |θ μ' )|θ Q μ'(s) represents the Q-value evaluation function after updating the neural network weight parameters. i+1 |θ μ' ) represents the policy network after updating the neural network weight parameters, s i+1 It indicates the state at the next moment.

[0138] According to the Monte Carlo method, using mini-batch data and substituting it into the policy gradient formula can provide an unbiased estimate of the expected value.

[0139] S45: Based on the Q-network established in S44, the deep reinforcement learning algorithm creates two copies of the neural network for both the policy network and the Q-network.

[0140] The policy network is:

[0141]

[0142] The Q network is:

[0143]

[0144] Where, μ(s|θ) μ ) represents the online network in the policy network, which uses the gradient update method to update the weight parameters, μ'(s|θ) μ ') represents a copy of the online network in the policy network, called the target network in the policy network, and uses the soft ipdate method to update the weight parameters. s represents the state, and θ represents the weight coefficients of the neural network; Q(s,a|θ) Q ) represents the online network in the Q network, where the gradient update method is used to update the weight parameters, Q'(s,a|θ) Q ') represents a copy of the online network in the Q network, called the target network in the Q network. The weight parameters are updated using the soft ipdate method. s represents the state, a represents the action, and θ represents the weight coefficients of the neural network.

[0145] S46: After training on a small batch of data, update the parameters of the online network using the SGA / SGD algorithm, and then update the parameters of the target network using the soft update algorithm.

[0146] Soft update is a running average algorithm, as shown below:

[0147]

[0148] Where τ is the weight of the online network when updating the target network weights, and θ is the weight of the online network weights. Q θ represents the weight parameters of the online network in the Q-network. Q ' represents the weight parameters of the target network in the Q-network; θ μ θ represents the weight parameters of the online network in the policy network. μ 'Represents the weight parameters of the target network in the policy network.

[0149] In this embodiment, a vehicle thermal management system based on a black-box model and deep reinforcement learning includes a memory and a controller. The memory stores a computer-readable program, which, when invoked by the controller, can execute the steps of the vehicle thermal management method based on a black-box model and deep reinforcement learning as described in this embodiment.

[0150] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A vehicle thermal management method based on a black-box model and deep reinforcement learning, characterized in that, Includes the following steps: S1: Obtain the vehicle's status information under complete operating conditions; S2: Establish a complete vehicle model, including a coupled dynamic thermal model of the air conditioning and passenger compartment; specifically: S21: Establish a dynamic thermal model of an automotive air conditioning system, which includes a compressor, condenser, evaporator, expansion valve, and blower. S22: Establish a simplified one-dimensional dynamic thermal model of the crew cabin system; S23: Establish an engine cooling model and a whole vehicle simulation model based on vehicle dynamics, and couple the passenger compartment system with the engine cooling system through the heater core; S3: Based on the vehicle model established in step S2, perform black-box model training on the vehicle model using the data collected in step S1. S4: Design a vehicle thermal management control strategy based on deep reinforcement learning algorithm, use the trained black box model as the environment, and interact with the vehicle thermal management control strategy in real time under different working conditions to train the intelligent agent.

2. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 1, characterized in that: The status information mentioned in step S1 includes vehicle status information, air conditioning system information, and passenger compartment system information.

3. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 2, characterized in that: The vehicle status information includes at least one of the vehicle speed, engine speed, compressor speed, and fan speed; The air conditioning system information includes at least one of the following: volumetric efficiency, isentropic efficiency, expansion valve flow coefficient, evaporator / condenser tube size, evaporator / condenser material specific heat, evaporator / condenser mass, evaporator / condenser air-side heat transfer coefficient, evaporator / condenser windward area, ambient temperature, and solar radiation. The passenger compartment system information includes at least one of the following: seat heat transfer coefficient, human body heat dissipation, heat transfer area between the front end and the air, mass of the front end module / side wall / windshield, specific heat of the front end module material / windshield / roof module material / side wall module material, and convective heat transfer coefficient between the windshield / roof / side wall and the external ambient air.

4. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 3, characterized in that: Specifically, S21 is: Compressor: A mathematical model was built in MATLAB / Simulink using the lumped parameter method to measure the refrigerant mass flow rate at the compressor outlet. The calculation formula is: in, It refers to the volumetric efficiency of the electric compressor. It is the density of refrigerant R134a at the compressor inlet. It refers to the rotational speed of the electric compressor, which is controlled by the drive motor. It refers to the compressor's displacement; Expansion valve: During its dynamic process, the refrigerant mass flow rate through the expansion valve Its expansion valve pressure drop The relationship is expressed by the following formula: in The flow coefficient of the expansion valve, This represents the minimum flow area of ​​the expansion valve. The density of the refrigerant passing through the expansion valve; Evaporator: According to the moving boundary method, the gaseous and liquid refrigerant in the evaporator satisfy the law of conservation of mass. Therefore, the length of the two-phase region of the evaporator... The change is obtained from the following formula: Based on the law of conservation of energy, the change in evaporator wall temperature can be expressed as: in, It is the density of the liquid refrigerant in the evaporator. The latent heat of vaporization of the refrigerant in the evaporator. This represents the total cross-sectional area of ​​the flat tube microchannels in the evaporator. This represents the volume fraction of refrigeration vapor in the two-phase region. The average porosity of the two-phase region of the evaporator. , and These represent the enthalpy values ​​of the gas, liquid, and inlet refrigerant in the evaporator at the current pressure. It is the heat transfer coefficient between the evaporator inner wall and the refrigerant in the two-phase region. It is the diameter inside the evaporator flat tube. It is the total length of the evaporator flat tube. The evaporator wall temperature, It is the saturation temperature of the refrigerant at the current pressure of the evaporator. The density of refrigerant vapor in the evaporator. The pressure in the evaporator, This indicates the specific heat of the evaporator material and the mass of the evaporator. Indicates the specific heat of the condenser material. Indicates the mass of the evaporator. It is the heat transfer coefficient between air and the evaporator wall. The frontal area of ​​the evaporator. It is the current temperature of the air surrounding the evaporator; Condenser: Sub-model satisfies: Assuming there is no refrigerant leak in the air conditioning system, the total mass of refrigerant in the system remains constant. Therefore, the total mass of refrigerant in the evaporator and condenser can be considered constant, hence: in, This represents the total cross-sectional area of ​​the flat tube microchannels in the condenser. It is the density of the liquid refrigerant in the condenser. The vapor density in the condenser This represents the volume fraction of refrigeration vapor in the two-phase region of the condenser. The average porosity of the two-phase region of the condenser. It is the total length of the condenser flat tube. The length of the two-phase region of the condenser. Represents a constant. The latent heat of vaporization of the refrigerant in the condenser. and These represent the enthalpy values ​​of the liquid in the condenser and the inlet refrigerant, respectively, at the current pressure. It is the heat transfer coefficient between the condenser inner wall and the refrigerant in the two-phase region. The diameter inside the condenser flat tube, The condenser wall temperature, It is the saturation temperature of the refrigerant under the current pressure of the condenser. The condensing pressure in the condenser. It is the heat transfer coefficient between the refrigerant and the pipe wall in the superheated zone. The temperature of the refrigerant at the condenser inlet. It is the heat transfer coefficient between the air and the condenser wall. The frontal area of ​​the condenser. It is the temperature of the air surrounding the condenser, i.e., the ambient temperature.

5. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 3, characterized in that: The simplified one-dimensional dynamic thermal model of the crew cabin system in S22 includes: Total heat load of the vehicle passenger compartment Represented as: According to the law of conservation of energy, the air temperature in the crew cabin is... Dynamic changes can be represented as: in, For heat exchange of the vehicle body surface structure, For solar radiation heat load, The heat load caused by ventilation For the human body's heat load, For the heat load of machinery and instruments, The total heat load of the crew compartment, The amount of cooling capacity delivered to the cabin by the air conditioning system per unit time. Air quality within the volume of a car cabin. It is the specific heat of air.

6. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 3, characterized in that: The engine cooling model established in S23 includes: For the engine sub-model, engine heat dissipation Solve using the following formula: In the formula: This represents the percentage of engine heat dissipation relative to the total heat of the fuel. The value for the engine fuel consumption rate can be found in the internal combustion engine performance table. For engine power, The lower heating value of fuel; For the radiator sub-model, when calculating the maximum heat dissipation of the radiator, the engine heat dissipation is considered. Multiply by a coefficient of 1.1 to 1.25 on top of that. For the thermostat sub-model, the area of ​​the valve opening at the thermostat inlet is linearly related to the temperature of the thermostat paraffin. For the water pump and fan sub-model, the water pump is a centrifugal water pump, and the start and stop of the fan are controlled by a temperature sensor to collect the temperature of the coolant at the radiator outlet.

7. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 1, 2, 3, 4, 5, or 6, characterized in that: Specifically, S3 is: First, the dynamic thermal model of the air conditioning and passenger compartment coupling is input with the following parameters: vehicle speed and engine speed, blower voltage, and heater core temperature under different operating conditions. This yields data on the entire process of air conditioning energy consumption and passenger compartment temperature changes. The data collected in step S1 is randomly allocated according to k1:k2, with k1 serving as the training set. The input and output of this data are used as the input and output of the black box model, which is trained using a random forest machine learning model. Then, the other k2 data is used as the test set to validate the trained black box model, where k1 is greater than k2.

8. The vehicle thermal management method based on black-box model and deep reinforcement learning according to claim 1, 2, 3, 4, 5, or 6, characterized in that: S4 includes: S41: State S = { , ,RH}, where: S is the set of states containing all states. Temperature of the crew cabin. To meet the target temperature requirements of most occupants in the thermal comfort zone, RH is the relative humidity of the occupant cabin; S42: The agent selects an action A={FanLevel, V} based on the state S. comp A = BlowerVoltage, CoolHeatLevel}, where A contains all action sets, and FanLevel is the front cooling fan speed; V comp The compressor displacement is indicated by the passenger compartment air conditioning compressor, which is a variable displacement compressor; BlowerVoltage is the blower speed setting; coolHeatLevel is the heater core speed setting, with different speeds corresponding to different ratios of hot and cold air. S43: After the agent takes an action, it receives a corresponding reward. The reward function R is defined as: in, For compressor energy consumption, =| - This represents the absolute error between the crew cabin temperature and the target temperature. It is then calculated by the reference temperature generator; S44: After taking an action, the agent receives the corresponding reward and enters the next state S'. It stores the experience samples {S,A,R,S'} in the experience pool, and stores the number of samples in the experience pool to a preset number. It randomly selects a small batch of stored experience samples from the experience pool, calculates the target value for each state, and updates the Q value of the agent based on the reward obtained after performing the action. In the deep reinforcement learning algorithm, a convolutional neural network is used to simulate the target Q function. S45: Based on the Q-network established in S44, the deep reinforcement learning algorithm creates two copies of the neural network for both the policy network and the Q-network. S46: After training on a small batch of data, update the parameters of the online network using the SGA / SGD algorithm, and then update the parameters of the target network using the soft update algorithm.

9. A vehicle thermal management system based on a black-box model and deep reinforcement learning, characterized in that: The system includes a memory and a controller. The memory stores a computer-readable program that, when invoked by the controller, can execute the steps of the vehicle thermal management method based on a black-box model and deep reinforcement learning as described in any one of claims 1 to 8.

Images