Deep learning based low temperature thermal management method and system for electric heavy-duty vehicles

Abstract

The application discloses a kind of low temperature thermal management method and system of electric heavy vehicle based on deep learning, it is related to new energy vehicle thermal management technical field, this method includes: when target vehicle is in preset low temperature working condition, the current state and multi-mode preference information of vehicle low temperature thermal management system are acquired, vehicle low temperature thermal management system includes heat pump air conditioning loop, passenger cabin air heating loop, drive motor waste heat recovery loop and power battery thermal management loop;Battery temperature tracking weight, passenger cabin temperature tracking weight and energy consumption weight are determined according to multi-mode preference information;Current state, battery temperature tracking weight, passenger cabin temperature tracking weight and energy consumption weight are input to deep reinforcement learning intelligent agent, and continuous control instruction is obtained;According to continuous control instruction, the execution components in heat pump air conditioning loop, passenger cabin air heating loop, drive motor waste heat recovery loop and power battery thermal management loop are independently adjusted, to carry out low temperature thermal management to electric heavy vehicle.

Description

Technical Field

[0001] This application relates to the field of thermal management technology for new energy vehicles, and in particular to a method and system for low-temperature thermal management of electric heavy-duty vehicles based on deep learning. Background Technology

[0002] With the rapid development of the new energy vehicle industry, pure electric heavy-duty commercial vehicles are widely used in long-haul logistics, engineering transportation, and special engineering fields due to their zero emissions and high economic efficiency. However, in extremely cold winter conditions, vehicles face the dual challenges of declining electrochemical activity of the power battery and huge heating demands on the passenger compartment. The energy consumption of the thermal management system rises sharply, severely shortening the driving range. Therefore, an efficient low-temperature thermal management method is needed.

[0003] Traditional low-temperature thermal management methods are generally based on rule-based logic control or PID control, which rely on static thresholds for on / off control. However, in the actual operation of heavy commercial vehicles in extremely cold conditions, these existing technologies have significant drawbacks. First, traditional rule-based control logic is rigid and cannot adaptively adjust according to the vehicle's current remaining battery charge (SOC) and the driver's actual needs, lacking multi-mode switching capabilities. Second, traditional threshold-based on-off control can only turn on or completely turn off the heating hardware at full power. Under the enormous thermal inertia of heavy trucks, this control method not only fails to achieve precise thermal balance of the system but also leads to severe temperature overshoot and high-frequency oscillations. Furthermore, traditional methods cannot decouple macroscopic operating conditions from microscopic execution, making it difficult to achieve an optimal balance among multiple objectives such as energy consumption and temperature rise rate.

[0004] Therefore, improving the adaptability of low-temperature thermal management in electric heavy-duty vehicles and avoiding ineffective energy loss is an urgent problem to be solved under extremely cold and variable working conditions. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a deep learning-based method for low-temperature thermal management of electric heavy-duty vehicles, the method comprising:

[0006] When the target vehicle is under a preset low temperature condition, the current status and multi-mode preference information of the vehicle's low temperature thermal management system are obtained. The vehicle's low temperature thermal management system includes a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, and a power battery thermal management circuit. The battery temperature tracking weight, the passenger cabin temperature tracking weight, and the energy consumption weight are determined based on the multi-mode preference information. The current state, the battery temperature tracking weight, the passenger compartment temperature tracking weight, and the energy consumption weight are input into the deep reinforcement learning agent to obtain continuous control commands. The actuators in the heat pump air conditioning circuit, passenger compartment heating circuit, drive motor waste heat recovery circuit, and power battery thermal management circuit are independently adjusted according to the continuous control commands to perform low-temperature thermal management on electric heavy-duty vehicles.

[0007] In one embodiment, when the target vehicle is under a preset low-temperature operating condition, acquiring the current state and multi-mode preference information of the vehicle's low-temperature thermal management system includes: Obtain the ambient temperature of the target vehicle's surroundings; When the ambient temperature is lower than a preset low temperature threshold, the target vehicle is determined to be in a preset low temperature operating condition. In response to the target vehicle being in a preset low-temperature operating condition, the current state is constructed by receiving data from the battery management system, including the power battery temperature, drive motor temperature, cooler temperature, battery state of charge, and ambient temperature. Obtain multi-mode preference information of the target vehicle, including energy-saving mode, comfort mode and balanced mode.

[0008] In one embodiment, determining the battery temperature tracking weight, the passenger compartment temperature tracking weight, and the energy consumption weight based on the multi-mode preference information includes: The current mode of the target vehicle is determined based on multi-mode preference information, wherein the current mode of the target vehicle includes energy-saving mode, comfort mode and balanced mode; When the battery state of charge is lower than a preset power threshold and the target vehicle is not in energy-saving mode, the target vehicle's current mode is changed to energy-saving mode. Based on the corrected or uncorrected target vehicle current mode, the battery temperature tracking weight, passenger compartment temperature tracking weight, and energy consumption weight are determined from a preset weight mapping relationship. The weight mapping relationship includes energy-saving mode, comfort mode, and balanced mode, and their respective weight combinations.

[0009] In one embodiment, before inputting the current state, the battery temperature tracking weight, the passenger compartment temperature tracking weight, and the energy consumption weight into the deep reinforcement learning agent to obtain continuous control commands, the method further includes: A multi-loop coupled thermal management simulation environment for electric heavy-duty vehicles is constructed. Based on the Actor online network, deterministic actions are output according to the input state, and adaptive Ornstein-Uhlenbeck noise is superimposed on the deterministic actions to obtain exploratory actions. The standard deviation of the adaptive Ornstein-Uhlenbeck noise is dynamically corrected based on the training iteration rounds, battery temperature, ambient temperature, and real-time driving power of the vehicle. Specifically, the standard deviation is reduced when the battery temperature is close to the safety threshold or when the vehicle is under heavy load and high power conditions, and the standard deviation is increased when the training iteration rounds are less than the preset round threshold. After performing the exploration action, the updated state and immediate reward are obtained. The current state, the exploration action, the immediate reward and the updated state are stored in the experience replay pool. The network parameters of the deep reinforcement learning agent are sampled from the experience replay pool to update the network parameters of the deep reinforcement learning agent. After the training meets the convergence condition, a deep reinforcement learning agent is obtained, and the adaptive Ornstein-Uhlenbeck noise is turned off.

[0010] In one embodiment, after performing the exploration action, the method further includes: Based on the deviation between the actual temperature of the power battery and the target temperature of the power battery, and the deviation between the actual temperature of the passenger compartment and the target temperature of the passenger compartment, and combined with the battery temperature tracking weight and the passenger compartment temperature tracking weight, a temperature tracking reward is obtained. Based on the heating power consumption of the warm air circuit water heater and the battery circuit water heater, and the regenerative braking recovery power allowed under the current power battery temperature boundary, an energy consumption penalty reward is obtained; wherein, the heating power consumption is penalized, and the regenerative braking recovery power is positively incentivized. The penalty for exceeding the limit is determined based on the abnormal truncation penalty and the temperature safety range penalty; A multi-objective reward function is determined based on the temperature tracking reward, the energy consumption penalty reward, and the over-limit truncation penalty, and an immediate reward is determined based on the multi-objective reward function.

[0011] In one embodiment, sampling from the experience replay pool to update the network parameters of the deep reinforcement learning agent includes: Calculate the target Q-value based on the next time step state and the target action output by the Actor target network; Construct a total loss, which includes a time-series difference mean square error term, a battery temperature over-limit penalty term, and an actuator motion smoothing penalty term. The actuator motion smoothing penalty term is generated when the difference in control actions between adjacent time moments exceeds a preset threshold. The parameters of the Critic online network are updated by minimizing the total loss through gradient descent backpropagation. The Critic online network outputs value, and the parameters of the Actor online network are updated based on maximizing the value. Based on the parameters of the Actor online network and the Critic online network, the Actor target network and the Critic target network are updated by weighted average.

[0012] In one embodiment, the step of independently adjusting the actuators in the heat pump air conditioning circuit, the passenger compartment heating circuit, the drive motor waste heat recovery circuit, and the power battery thermal management circuit according to the continuous control command includes: According to the continuous control command, a speed adjustment signal is output to the heat pump air conditioning circuit to continuously adjust the compressor speed to match the real-time heat load; According to the continuous control command, power adjustment signals are output to the water heater of the heating circuit in the passenger compartment heating circuit and the water heater of the battery circuit in the power battery thermal management circuit, respectively, to independently adjust the heating power of the two circuits, and when the heat pump air conditioning circuit outputs insufficient heat, the water heater of the heating circuit is started for auxiliary heating. According to the continuous control command, the on / off state of the battery self-circulation valve, the first heat exchange valve and the second heat exchange valve in the power battery thermal management circuit is controlled so that the power battery thermal management circuit switches between independent self-circulation mode and receiving heat from the passenger compartment heating circuit. According to the control component of the corresponding electronically controlled four-way valve in the continuous control command, the electronically controlled four-way valve in the drive motor waste heat recovery circuit is controlled to be in either a first connected state or a second connected state; in the first connected state, the drive motor waste heat recovery circuit is connected in series with the power battery thermal management circuit to transfer the waste heat of the drive motor to the power battery thermal management circuit; in the second connected state, the drive motor waste heat recovery circuit and the power battery thermal management circuit circulate independently to dissipate the heat of the drive motor to the outside.

[0013] Furthermore, to achieve the above objectives, this application also proposes a deep learning-based cryogenic thermal management system for electric heavy-duty vehicles, the deep learning-based cryogenic thermal management system for electric heavy-duty vehicles comprising: The vehicle low-temperature thermal management system includes a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, a power battery thermal management circuit, and a four-way valve. The processor is communicatively connected to the vehicle cryogenic thermal management system and is used to implement the steps of the deep learning-based cryogenic thermal management method for electric heavy-duty vehicles as described above.

[0014] In one embodiment, the four-way valve includes four ports: a, b, c, and d. Port a is connected to the motor outlet side of the drive motor waste heat recovery circuit, port b is connected to the battery self-circulation valve side of the power battery thermal management circuit, port c is connected to the thermostat outlet side of the drive motor waste heat recovery circuit, and port d is connected to the battery cold plate outlet side of the power battery thermal management circuit. When waste heat recovery is turned on, ports a and b of the four-way valve are connected, and ports c and d are connected, so that the drive motor waste heat recovery circuit and the power battery thermal management circuit are connected in series. When waste heat recovery is turned off, ports a and c of the four-way valve are connected, and ports b and d are connected, so that the drive motor waste heat recovery circuit and the power battery thermal management circuit each perform self-circulation.

[0015] In addition, to achieve the above objectives, this application also proposes an electric heavy-duty vehicle, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: (1) Obtaining the current state and multi-mode preference information of the vehicle solves the problem of rigid traditional rule control logic and inability to adapt to driver needs, and realizes dynamic coordination between driver intention and thermal management control; determining the battery temperature tracking weight, passenger compartment temperature tracking weight and energy consumption weight according to the multi-mode preference information, realizing the adaptive allocation of temperature control and energy consumption weight in different modes; since the current state and weight are input into the deep reinforcement learning agent to obtain continuous control instructions, the problem of traditional switch control being able to start and stop at full power is solved, realizing stepless adjustment and precise thermal balance control; finally, the execution components of each loop are independently adjusted according to the continuous control instructions, solving the problem of temperature overshoot and high-frequency oscillation under thermal inertia, thereby improving the adaptability of low-temperature thermal management of electric heavy vehicles under extremely cold and variable working conditions and avoiding ineffective energy loss.

[0017] (2) Anomaly truncation penalty and temperature safety range penalty were introduced in the model training stage to determine the over-limit truncation penalty and dynamically correct the standard deviation of adaptive Ornstein-Uhlenbeck noise. This solved the problem that the exploration noise parameters and empirical loss function of the passenger car thermal management model could not be adapted to the heavy truck large heat capacity system, enabling the policy network to actively avoid dangerous control actions and ensuring the safety and reliability of the deep reinforcement learning agent when facing severe disturbances in extremely cold environments. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the first embodiment of the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles in this application. Figure 2 This is a schematic diagram illustrating the interaction between the system and the intelligent agent in this application; Figure 3 This is a schematic diagram of the network model structure of the intelligent agent in this application; Figure 4 This is a schematic diagram of the structure of the vehicle's low-temperature thermal management system in this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a deep learning-based method for low-temperature thermal management of electric heavy-duty vehicles. Specifically, refer to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles according to this application. In this embodiment, the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles includes steps S10 to S50:

[0024] Step S10: When the target vehicle is under a preset low temperature condition, obtain the current status and multi-mode preference information of the vehicle's low temperature thermal management system.

[0025] It should be noted that the target vehicle can be a pure electric heavy-duty commercial vehicle, such as a heavy-duty semi-trailer or snowplow, with a large-capacity battery pack and a large passenger compartment. The preset low-temperature operating condition refers to an extremely cold working environment with an ambient temperature between 0°C and 20°C, where battery activity decreases and the passenger compartment requires significant heating. The vehicle's low-temperature thermal management system can include a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, and a power battery thermal management circuit. The current state is a multi-dimensional observation vector collected by various temperature sensors and the BMS battery management system, which can include physical quantities such as battery pack temperature, drive motor temperature, cooler temperature, battery SOC, and ambient temperature. Multi-mode preference information can be understood as the thermal management preference mode selected by the driver through the human-machine interface or the target vehicle's default mode, such as Eco mode, Comfort mode, and Balance mode, used to characterize different temperature control priorities.

[0026] Step S20: Determine the battery temperature tracking weight, the passenger cabin temperature tracking weight, and the energy consumption weight based on the multi-mode preference information.

[0027] It should be noted that the battery temperature tracking weight is a parameter in the reward function used to characterize the degree of importance attached to the battery pack temperature tracking error, and is dynamically adjusted by multi-mode preference information. The passenger compartment temperature tracking weight is a parameter in the reward function used to characterize the degree of importance attached to the passenger compartment heating temperature tracking error, and is dynamically adjusted by multi-mode preference information. The energy consumption weight is a general term for the parameters in the reward function used to penalize or incentivize system energy consumption, including power consumption penalty weight and energy recovery incentive weight.

[0028] Step S30: Input the current state, battery temperature tracking weight, crew cabin temperature tracking weight, and energy consumption weight into the deep reinforcement learning agent to obtain continuous control commands.

[0029] It should be noted that a deep reinforcement learning agent can be understood as a controller based on the DDPG (Deep Deterministic Policy Gradient) algorithm. For example, a deep reinforcement learning agent can include an online Actor network, an online Actor network, an online Critic network, and a Critic network. It learns the optimal control strategy through interaction with the environment and outputs continuous control commands. These continuous control commands are stepless adjustment signals output by the agent, such as continuous values ​​for compressor speed, water pump flow rate, and PTC heating power, replacing traditional on / off commands to achieve precise thermal balance.

[0030] Step S40: Independently adjust the actuators in the heat pump air conditioning circuit, passenger compartment heating circuit, drive motor waste heat recovery circuit, and power battery thermal management circuit according to continuous control commands to perform low-temperature thermal management on the electric heavy-duty vehicle.

[0031] It should be noted that the actuators refer to the hardware devices in each thermal management circuit that are regulated by the controller, such as compressors, electric water pumps, PTC heaters, electronic expansion valves, electronically controlled four-way valves, battery self-circulation valves, and heat exchange valves. The passenger compartment, i.e., the interior space of the vehicle's driver's cab, needs to maintain a suitable temperature to ensure driver comfort and meet defrosting and defogging requirements. The power battery, the vehicle's high-voltage battery pack, needs to be protected against lithium plating and performance degradation at low temperatures, and must be maintained within a safe operating range of 5℃ to 45℃. In low-temperature environments, by adjusting the actuators in each circuit, the passenger compartment is heated and the power battery is preheated and insulated to achieve low-temperature thermal management.

[0032] In this embodiment, the current vehicle state and multi-mode preference information are acquired, solving the problem of rigid traditional rule-based control logic and its inability to adaptively adjust according to driver needs, thus achieving dynamic coordination between driver intent and thermal management control. Based on the multi-mode preference information, battery temperature tracking weight, passenger compartment temperature tracking weight, and energy consumption weight are determined, enabling adaptive allocation of temperature control and energy consumption weights under different modes. By inputting the current state and weights into a deep reinforcement learning agent to obtain continuous control commands, the problem of traditional switch control only being able to start and stop at full power is solved, achieving stepless adjustment and precise thermal balance control. Finally, based on the continuous control commands, the execution components of each loop are independently adjusted, solving the problems of temperature overshoot and high-frequency oscillation under thermal inertia. This improves the adaptability of low-temperature thermal management in electric heavy-duty vehicles under extremely cold and variable operating conditions, avoiding ineffective energy loss.

[0033] In one embodiment, step S10 includes: acquiring the ambient temperature of the environment where the target vehicle is located; when the ambient temperature is lower than a preset low temperature threshold, determining that the target vehicle is in a preset low temperature condition; in response to the target vehicle being in a preset low temperature condition, receiving the power battery temperature, drive motor temperature, cooler temperature, battery state of charge and ambient temperature collected by the battery management system, and constructing the current state; acquiring the target vehicle's multi-mode preference information, which includes energy-saving mode, comfort mode and balanced mode.

[0034] It should be noted that the Battery Management System (BMS) is a system used to monitor and manage the power battery, and can collect parameters such as battery voltage, current, temperature, and state of charge in real time. The power battery temperature is the temperature of the cells or modules inside the power battery pack, directly reflecting the battery's thermal state. The drive motor temperature is the temperature corresponding to the heat generated by the drive motor during operation, and can be used for waste heat recovery assessment. The cooler temperature is the temperature of the cooler components in the thermal management system, reflecting the state of the heat dissipation circuit. The battery state of charge (SOC) is the percentage of the battery's current remaining charge to its total usable charge, characterizing the vehicle's driving range. The current state is a vector composed of the above multi-dimensional parameters, describing the real-time operating condition of the vehicle's thermal management system.

[0035] For example, the current state of the thermal management system can be obtained, and control actions to be executed can be generated. The agent collects the current state of the low-temperature thermal management environment of the pure electric heavy-duty commercial vehicle. , It is a multi-dimensional observation vector that includes physical quantities such as battery pack temperature, drive motor temperature, cooler temperature, battery SOC, and ambient low temperature. It is collected by various temperature sensors in the vehicle and the BMS battery management system. The complete state information is input into the Actor online network to provide a basis for strategy output.

[0036] In this embodiment, multi-mode preference information of the target vehicle is obtained, including energy-saving mode, comfort mode and balanced mode. This solves the problem that traditional rule control logic is rigid and cannot be adaptively adjusted according to the driver's personalized needs. It realizes dynamic decoupling and coordination between the driver's macro intention and the underlying thermal management control, thereby improving the system's adaptability.

[0037] In one implementation, step S20 includes: determining the current mode of the target vehicle based on multi-mode preference information, wherein the current mode of the target vehicle includes energy-saving mode, comfort mode, and balanced mode; when the battery state of charge is lower than a preset power threshold and the current mode of the target vehicle is not in energy-saving mode, correcting the current mode of the target vehicle to energy-saving mode; and determining the battery temperature tracking weight, passenger compartment temperature tracking weight, and energy consumption weight from a preset weight mapping relationship based on the corrected or uncorrected current mode of the target vehicle, wherein the weight mapping relationship includes energy-saving mode, comfort mode, and balanced mode and their respective weight combinations.

[0038] For example, the target vehicle's human-machine interface provides three thermal management preference modes for the driver to choose from: Eco, Comfort, and Balance. The system first obtains the multi-mode preference information submitted by the driver through the human-machine interface and determines the target vehicle's current mode accordingly. The system monitors the remaining state of charge (SOC) of the power battery in real time. When the battery's SOC is detected to be below a preset threshold (e.g., 20%) and the current mode is not Eco, the system automatically corrects the current mode to Eco. Based on the corrected mode, the system retrieves the corresponding weight combination from a preset weight mapping relationship. Among these, the battery temperature tracking weight... The value used to characterize the emphasis placed on battery pack temperature tracking error in the reward function is [2.0, 8.0]; the passenger compartment temperature tracking weight. The value range is [1.0, 5.0], used to characterize the emphasis placed on the crew cabin heating temperature tracking error in the reward function; energy consumption weight includes power consumption penalty weight. and energy recovery incentive weight .in The value range is [1.0, 7.0], and it is used to impose penalties on the power consumption of components such as PTC and water pump; The value range is [0.5, 4.0], which is used to introduce the energy recovered by regenerative braking as a positive incentive into the reward function.

[0039] For example, the weight mapping relationship may include: in comfort mode, the occupant cabin temperature tracking weight. When assigned a maximum value, such as 5.0, the agent prioritizes allocating power to the PTC in the warm air circuit to achieve rapid defrosting. Take a standard value, such as 5.0; Choose a smaller value, such as 2.0, to relax energy consumption constraints. Energy consumption penalty weight in energy-saving mode. Select the maximum value within the interval, such as 7.0, to maximize the constraint on system power consumption; energy recovery incentive weight. Also take a larger value, such as 4.0, to actively increase the battery temperature through the excitation algorithm, thereby unlocking regenerative braking; Then take the smaller value, such as 1.0. In balanced mode, each weight takes the median value, for example... =5.0、 =2.5、 =2.0、 =1.5, to balance energy consumption and comfort.

[0040] In this embodiment, the driver's macroscopic multi-mode preferences are dynamically transformed into specific weight parameters in the reward function of the deep reinforcement learning agent. This achieves dynamic decoupling and coordination between the underlying power intelligent optimization and the upper-level driver's macroscopic intentions, solving the problem that traditional static rules cannot adapt to extremely cold and variable working conditions and personalized needs.

[0041] In one embodiment, the steps preceding step S30 include: (1) Construct a simulation environment for multi-loop coupled thermal management of electric heavy-duty vehicles.

[0042] It should be noted that this embodiment adopts a co-simulation architecture. First, a physical environment for the thermal management of a pure electric heavy-duty commercial vehicle is built in AMESim software. This physical environment, as the dynamic base of the controlled object, not only includes the longitudinal dynamics system but also deeply integrates a multi-loop coupled vehicle low-temperature thermal management system. The simulation physical environment is equipped with a thermal management master controller, which serves as the interface between the deep reinforcement learning agent and the AMESim physical simulation environment. The master controller receives control commands from the agent and outputs them to the controlled execution components of each loop, such as the compressor, the heater loop water pump, the waste heat recovery four-way valve, the battery heat exchanger heat exchange valve, the battery self-circulation valve, and the water heater. Simultaneously, it collects signals from various temperature sensors and feeds them back to the agent. The interaction relationship is as follows: Figure 2As shown, this enables the intelligent agent to achieve closed-loop control of each component of the thermal management system.

[0043] (2) In the simulation environment of multi-loop coupled thermal management of electric heavy vehicles, based on the Actor online network, a deterministic action is output according to the input state, and an adaptive Ornstein-Uhlenbeck (OU) noise is superimposed on the deterministic action to obtain the exploratory action.

[0044] For example, the current state of the thermal management system is obtained, and control actions to be executed are generated. The agent collects the current state of the low-temperature thermal management environment of the pure electric heavy-duty commercial vehicle. and the status Online network input to the Actor network; Actor online network receiving status. Then output a confirmed action. Add noise at the same time Then it is applied to the thermal management environment; among which Add noise to the parameters of the Actor online network. The aim is to increase the randomness of actions, fully explore various control combinations under low-temperature conditions, and avoid the algorithm getting stuck in local optima during training.

[0045] An improved adaptive OU noise is adopted. Traditional DDPG uses fixed-parameter Ornstein-Uhlenbeck noise with a constant noise amplitude throughout the process. For different training stages and operating conditions of the heavy truck thermal management system, adaptive attenuation OU noise is adopted. The noise amplitude is dynamically corrected according to the battery temperature deviation, ambient temperature, and vehicle output power to avoid insufficient early exploration and excessive noise in the later stage, which may cause frequent actuator oscillation.

[0046] The basic OU noise is represented as: in, This represents the noise output at time t; Indicates the regression rate; Indicates the noise mean; Indicates the standard deviation of noise; This represents the Wiener process.

[0047] (3) Based on the training iteration rounds, battery temperature, ambient temperature and real-time driving power of the vehicle, the standard deviation of the adaptive Ornstein-Uhlenbeck noise is dynamically corrected. Specifically, the standard deviation is reduced when the battery temperature is close to the safety threshold or the vehicle is under heavy load and high power conditions, and the standard deviation is increased when the training iteration rounds are less than the preset round threshold.

[0048] For example, the standard deviation can be... For adaptive correction, the corresponding formula is: in, The initial noise standard deviation has a range of values. , can be 0.40; epoch is the number of training iterations, that is, the number of training rounds in reinforcement learning, which is a positive integer, and the total number of training rounds can be set to 1000; The iterative decay coefficient has a range of values. It can be set to 0.992 to achieve an overall decay of the noise amplitude over training rounds; Here are the working condition-related correction coefficients, which are dimensionless coefficients with a range of values. The average temperature of the battery Ambient temperature Real-time drive power of the whole vehicle This is jointly determined; when the battery temperature is far from the safety boundary and the vehicle load is low, this coefficient takes a value close to 1.0 to ensure sufficient state exploration by the algorithm; when the battery temperature is close to the safety threshold or the vehicle is under heavy load and high power conditions, this coefficient decreases to 0.10 to suppress large-scale actuator disturbances caused by exploration noise. (Exemplary normal operating conditions) Take 0.85, when the battery is near the temperature boundary or under heavy load and high power conditions. Set the value to 0.20. When the battery temperature approaches the safety threshold or the vehicle is under heavy load and high power conditions, the value will automatically decrease. To suppress control disturbances caused by random exploration and prevent thermal management actuators, such as compressors, water pumps, and valves, from jumping significantly; in the early stages of low-temperature training, such as the first 100 rounds, increase the noise amplitude to ensure sufficient state exploration.

[0049] (4) After performing the exploration action, obtain the updated state and immediate reward, store the current state, exploration action, immediate reward and updated state into the experience replay pool, and sample from the experience replay pool to update the network parameters of the deep reinforcement learning agent.

[0050] It should be noted that after performing the exploration action, the process also includes: obtaining a temperature tracking reward based on the deviation between the actual temperature of the power battery and the target temperature of the power battery, and the deviation between the actual temperature of the passenger compartment and the target temperature of the passenger compartment, combined with the battery temperature tracking weight and the passenger compartment temperature tracking weight; obtaining an energy consumption penalty reward based on the heating power consumption of the heating circuit water heater and the battery circuit water heater, and the regenerative braking recovery power allowed under the current power battery temperature boundary; wherein, a penalty is imposed on the heating power consumption, and a positive incentive is given for the regenerative braking recovery power; an over-limit truncation penalty is determined based on the abnormal truncation penalty and the temperature safety range penalty; a multi-objective reward function is determined based on the temperature tracking reward, the energy consumption penalty reward, and the over-limit truncation penalty, and an immediate reward is determined based on the multi-objective reward function.

[0051] For example, deep reinforcement learning agents optimize control strategies by maximizing cumulative rewards. Since this application involves multiple conflicting objectives for pure electric heavy-duty commercial vehicles under extremely cold conditions, such as heating rate, energy consumption minimization, and hardware lifespan protection, and requires response to multi-mode intents issued from upper layers, this embodiment constructs a multi-objective reward function. .

[0052] At time step The immediate comprehensive reward obtained by the intelligent agent It consists of the following three superimposed parts: in, As a reward for temperature tracking, As a penalty and reward for energy consumption, Penalties for boundary violations and abnormal truncation.

[0053] Among them, temperature tracking rewards The goal is to guide the agent to heat the battery and crew compartment to a target safe temperature as quickly as possible. To avoid temperature overshoot, a negative absolute error is used as a penalty, and a multi-mode weighting factor is introduced. The temperature tracking reward is represented as:

[0054] In the formula, and These are the actual temperatures of the power battery and the passenger compartment at the current moment, respectively. 20℃ can be used. 22℃ can be used as the corresponding target temperature. This is the battery temperature tracking weighting factor, used to characterize the degree of importance given to battery pack temperature tracking error in the reward function, with a value range of... Example values ; This is the crew cabin temperature tracking weighting factor, used to characterize the degree of importance given to the crew cabin heating temperature tracking error in the reward function, with a value range of... Example values These two weights are not fixed values, but are dynamically adjusted by the "multi-mode commands" issued by the human-computer interaction interface in step two. For example, in "Comfort mode," It will be assigned a maximum value, forcing the agent to prioritize power allocation to the warm air circuit PTC to achieve rapid defrosting.

[0055] Energy consumption penalty reward The aim is to constrain energy consumption under extremely cold conditions and maximize vehicle range. The energy consumption calculation includes the heating power consumption of two dual-PTC systems and incorporates the energy recovered through dynamic regenerative braking as a positive excitation. The corresponding formula is:

[0056] In the formula, and These represent the actual power consumption of the heating circuit PTC and the battery PTC at the current moment, respectively. This represents the allowable chassis regenerative braking power under the current battery temperature boundary. This value is 0 during extremely cold, frozen conditions and gradually increases with rising temperature. This is a dimensionless power consumption penalty weight used to penalize the total power consumption of components such as heat pumps, water pumps, and PTCs, thereby suppressing energy consumption; its value range is... For example, the value is 2.0; in the energy-saving mode Eco, the weight is selected as the maximum value within the range, for example, 7.0, in order to maximize the constraint on system power consumption. This is a dimensionless incentive weight for energy recovery, used to incentivize the algorithm to fully utilize waste heat from the motor and electronic control system for loop heat exchange, reducing the energy input of the heat pump and PTC. Its value range is... An example value is 1.5. This is achieved by introducing... During training, the algorithm actively learns that "heating the battery to above 0°C faster can unlock regenerative braking and thus obtain additional energy rewards," thereby realizing an advanced collaborative strategy between thermal management and chassis braking.

[0057] Deep reinforcement learning is prone to producing extreme actions (such as continuous full-load heating) during the exploration phase, which can lead to serious safety accidents in real heavy-duty truck physical systems. Therefore, a strict penalty term is designed to constrain the agent's exploration boundary, with penalties for exceeding the boundary and anomaly truncation. It can be represented as:

[0058] Among them, temperature safety range penalty This indicates that the value is 0 when the target component temperature is within a safe range; once it deviates from the safe physical range, for example... If there is a risk of thermal runaway, a nonlinear and severe penalty is imposed.

[0059] In the formula, The penalty amplification factor, dimensionless, is used to amplify the magnitude of terms that deviate from constraints, such as exceeding temperature or power consumption limits, thereby increasing the penalty intensity and strengthening the algorithm's adherence to hard constraints. Its value range is... For example, the value is 6.0.

[0060] Abnormal truncation penalty To ensure the stable operation of the co-simulation environment, an abnormal truncation is triggered when the exploration actions of the agent cause the AMESim physical model to diverge, become unsolvable, or collapse due to reasons such as a surge in fluid pressure or temperature anomalies, or when the SOC falls below the hard safety threshold.

[0061] In the formula, For a very large constant, such as Once triggered, the system immediately imposes the hefty penalty and forcibly terminates the current training episode. This mechanism forces the agent to learn during offline training to prioritize avoiding dangerous action areas that could lead to physical instability, ensuring the absolute safety and robustness of the final converged model in real-world heavy-duty truck deployments.

[0062] The network parameters of the deep reinforcement learning agent are updated by sampling from the experience replay pool, including: calculating the target Q-value based on the next time step state and the target action output by the Actor target network; constructing the total loss, which includes a temporal difference mean squared error term, a battery temperature limit violation penalty term, and an actuator action smoothing penalty term, the actuator action smoothing penalty term being generated when the difference between control actions in adjacent time steps exceeds a preset threshold; minimizing the total loss through gradient descent backpropagation and updating the parameters of the Critic online network; outputting the value from the Critic online network and updating the parameters of the Actor online network based on maximizing the value; and updating the Actor target network and the Critic target network by weighted averaging based on the parameters of the Actor online network and the Critic online network, respectively.

[0063] In this implementation method, rewards By combining battery temperature control targets with thermal management energy consumption targets, the system penalizes battery temperatures that deviate from the optimal operating range while also constraining the power consumption of water pumps and fans. The experience replay pool stores a large number of state transition samples at different times, breaking the temporal correlation between samples and effectively alleviating the overfitting problem in the reinforcement learning training process.

[0064] (5) After the training meets the convergence condition, a deep reinforcement learning agent is obtained and adaptive Ornstein-Uhlenbeck noise is turned off.

[0065] It should be noted that (2) to (5) above are the training process for deep reinforcement learning agents. Based on the above description, the complete training process can be referred to in steps S1 to S8 below.

[0066] S1. Obtain the current state of the thermal management system, generate control actions to be executed, and the agent collects the current state of the low-temperature thermal management environment of the pure electric heavy-duty commercial vehicle. and the status The online network input to the Actor network. It is a multi-dimensional observation vector that includes physical quantities such as battery pack temperature, drive motor temperature, cooler temperature, battery SOC, and ambient low temperature. It is collected by various temperature sensors in the vehicle and the BMS battery management system. The complete state information is input into the Actor online network to provide a basis for strategy output.

[0067] S2, Actor online network receiving status Then output a confirmed action. Add noise at the same time Then it is applied to the thermal management environment; among which Add noise to the parameters of the Actor online network. The aim is to increase the randomness of actions, fully explore various control combinations under low-temperature conditions, and avoid the algorithm getting stuck in local optima during training.

[0068] S3. The thermal management environment is regulated by this action, the system heat exchange state changes, and a new environmental state is output. Instant rewards from environmental feedback collected by the intelligent agent. , the dataset Store in the experience replay pool.

[0069] S4. When the number of samples stored in the experience replay pool reaches a preset threshold, the Agent randomly selects N samples from the experience replay pool. The sample sets are input into the Actor target network, the Critic online network, and the Critic target network, respectively. Random sampling disrupts the temporal order of samples, avoiding gradient correlation caused by consecutive samples and ensuring the stability of network training.

[0070] S5, the target network of the Actor network is based on each set Data output specific action values This action value is then sent to Critic's target network. These are the network parameters for the Actor target; the target action is output here. It is used only for value assessment calculations and is not output to the thermal management environment for execution, thereby completing the prediction of the action at the next moment in the Bellman equation.

[0071] S6, Critic network according to , and Calculate the target Q value The target Q-value is input into the Critic online network. This application's loss function does not simply use the mean squared error (MSE) of the policy network and value network; instead, it introduces a hard constraint penalty term specific to heavy-duty truck thermal management on top of the time-series differential loss, differing from the DDPG loss construction used in ordinary passenger vehicle thermal management. The parameters of the Critic online network are updated by minimizing the improved loss function. , The formula for the loss function is as follows:

[0072] In the formula, This is a reward discount factor, ranging from 0 to 1, used to balance current rewards with long-term future rewards; Critic target network parameters; For Critic online network parameters; Output value to the Critic target network; The output value is the value of the Critic online network; N is the batch sample size; Weighting for penalties when battery temperature exceeds limits. Smooth the penalty weights for actuator actions; Penalty for heavy-duty truck battery pack temperature exceeding limits: When the average temperature of the battery pack exceeds the safe operating range. The output is a non-zero penalty value; the greater the temperature deviates from the safe range, the higher the penalty value. When the battery temperature is within the safe range... ; This is a penalty term for smoothing the action of thermal management actuators. It is used to suppress large jumps in the actions of compressors, water pumps, and multi-way valves between adjacent time steps, and to reduce frequent impact adjustments of high-power components. A penalty is generated when the difference in action between adjacent iteration steps exceeds a preset threshold. When the action change is smooth... .

[0073] By using gradient descent backpropagation, the total loss, consisting of the TD mean square error term, the battery safety penalty term, and the actuator smoothing penalty term, is minimized to complete the online parameter update of the Critic network.

[0074] Online network output of S7 and Critic networks For an online Actor network, update the parameters of the online Actor network by maximizing the Q-value. That is, to minimize the following loss function: This loss function does not rely on the gradient information of the environment, but relies on the value output of the Critic network to guide the Actor policy update; through backpropagation of the policy gradient algorithm, the online network parameters of the Actor are updated, so that the control actions output by the Actor network can obtain higher cumulative rewards, while taking into account the battery temperature control accuracy and the energy consumption of the thermal management system.

[0075] The target networks of S8, Critic, and Actor networks are updated by weighting the parameters of their online networks and their own network parameters. The update formula is as follows: In the formula, The soft update coefficient is preferably set to 0.001, which means that the target network inherits a small number of update weights from the online network each time, and the target network parameters change smoothly, thus avoiding the problem of non-convergence in the training process caused by sudden changes in network parameters.

[0076] After completing one round of sampling training, return to step S1 to continuously collect operating condition samples of the thermal management system, iteratively executing steps S1 and S8. When the algorithm training meets the convergence condition, stop training, deploy the trained Actor online network to the vehicle controller, and directly output continuous control quantities to achieve autonomous optimization control of the thermal management system under low-temperature conditions in pure electric heavy-duty commercial vehicles. The network model structure of the deep reinforcement learning agent is as follows: Figure 3 As shown.

[0077] This embodiment addresses the problems that the exploration noise parameters and empirical loss functions of passenger car thermal management models cannot be adapted to the large heat capacity system of heavy-duty trucks, and that traditional DDPGs are prone to training oscillations, control output overshoot, and battery temperature exceeding limits, thus failing to meet the safety and energy consumption constraints of heavy-duty truck thermal management.

[0078] In one embodiment, step S40 includes, but is not limited to, one or more of the following adjustment methods: According to the continuous control command, the speed adjustment signal is output to the heat pump air conditioning circuit to continuously adjust the compressor speed to match the real-time heat load.

[0079] Specifically, such as Figure 4As shown, the heat pump air conditioning circuit serves as the core basic heat source for the entire vehicle under extremely cold conditions. This circuit mainly includes a compressor, evaporator, heat pump condenser, gas-liquid separator, electronic expansion valve, and related pipelines, forming a closed-loop refrigerant circulation path. A refrigerant-water heat exchange type heat pump condenser can be used to decouple the heat pump refrigerant side from the water circuit, facilitating simultaneous energy supply to multiple targets under extremely cold conditions. Under extremely cold low-temperature conditions, the evaporator, as the heat-absorbing component of the heat pump, absorbs low-grade heat energy from the ambient low-temperature air. The low-temperature, low-pressure gaseous refrigerant is compressed by the compressor to increase its enthalpy value, becoming a high-temperature, high-pressure gaseous refrigerant that enters the heat pump condenser. Through refrigerant-water heat exchange, the heat is released to the heating circuit water circuit, providing a basic heat source for heating the passenger compartment and preheating the power battery. The gas-liquid separator is located at the front end of the compressor suction to achieve separation of the gas and liquid refrigerant phases, preventing liquid refrigerant backflow from causing liquid slugging damage to the compressor. The electronic expansion valve controls the subcooling before the valve. The compressor is connected to the thermal management master controller, which receives instructions from the deep reinforcement learning agent to adjust the compressor speed.

[0080] According to the continuous control command, the power adjustment signal is output to the water heater of the heating circuit in the passenger compartment heating circuit and the water heater of the battery circuit in the power battery thermal management circuit, respectively, to independently adjust the heating power of the two circuits, and when the heat pump air conditioning circuit outputs insufficient heat, the water heater of the heating circuit is started for auxiliary heating.

[0081] According to continuous control commands, the on / off states of the battery self-circulation valve, the first heat exchange valve, and the second heat exchange valve in the power battery thermal management circuit are controlled so that the power battery thermal management circuit can switch between independent self-circulation mode and heat receiving mode from the passenger compartment heating circuit.

[0082] Specifically, such as Figure 4 As shown, the passenger compartment heating circuit uses coolant as the circulating medium for heat transfer. It mainly includes a heating circuit water pump, expansion tank, power battery heat exchanger, heat pump condenser, heating circuit water heater (such as a heating PTC), and a heating core located inside the passenger compartment air conditioning unit. A passenger compartment temperature sensor collects the internal temperature signal in real time. The heating circuit water pump and heating circuit water heater are controlled and connected to the main thermal management controller; the passenger compartment temperature sensor signal is also connected to the main thermal management controller. The controlled heating circuit water pump drives the flow of coolant in the circuit. The coolant flows through the heat pump condenser to absorb the heat pumped by the heat pump system. When the heat output of the heat pump is insufficient, the heating circuit water heater is activated for auxiliary heating. After heat exchange, the coolant flows sequentially through the power battery heat exchanger and the heating core, transferring heat to the battery side to preheat it, and exchanging heat with the passenger compartment air through the heating core, thus completing the heating and insulation of the passenger compartment.

[0083] In the power battery thermal management loop, the first heat exchange valve is designated as heat exchange valve 1, and the second heat exchange valve is designated as heat exchange valve 2. This loop mainly includes a battery water pump, expansion tank, battery loop water heater (battery PTC), battery cold plate, battery self-circulation valve assembly, and power battery heat exchanger. The system is equipped with a battery temperature sensor and a battery inlet water temperature sensor to monitor the battery pack and loop coolant temperature in real time. The battery self-circulation valve assembly works in conjunction with heat exchange valves 1 and 2 to achieve seamless switching between independent battery self-circulation and external heat source reception modes. The battery water pump, battery loop water heater, battery self-circulation valve assembly, heat exchange valve 1, and heat exchange valve 2 are all controlled and connected to the thermal management main controller; the signals from the battery temperature sensor and battery inlet water temperature sensor are connected to the thermal management main controller. The coolant in the circuit is driven by the battery water pump for circulation. Under extremely cold conditions, the coolant can be independently heated by the battery circuit water heater. The coolant flows through the battery cold plate to exchange heat with the power battery, keeping the power battery within the safe charging and discharging range and suppressing the risk of lithium plating in the power battery under low temperature conditions. At the same time, the battery circuit can achieve self-circulation through the battery self-circulation valve group. When the battery circuit does not need heat from the heating circuit, the battery self-circulation valve will open, and heat exchange valve 1 and heat exchange valve 2 will close. When the battery circuit needs heat from the heating circuit, the battery self-circulation valve will close, and heat exchange valve 1 and heat exchange valve 2 will open.

[0084] According to the control component of the corresponding electronically controlled four-way valve in the continuous control command, the electronically controlled four-way valve in the waste heat recovery circuit of the drive motor is controlled to be in either the first connected state or the second connected state. In the first connected state, the waste heat recovery circuit of the drive motor is connected in series with the thermal management circuit of the power battery to transfer the waste heat of the drive motor to the thermal management circuit of the power battery. In the second connected state, the waste heat recovery circuit of the drive motor and the thermal management circuit of the power battery circulate separately to dissipate the heat of the drive motor to the outside.

[0085] It should be noted that, based on the control component of the corresponding electrically controlled four-way valve in the continuous control command, the electrically controlled four-way valve in the drive motor waste heat recovery circuit can be controlled to be in either the first connected state or the second connected state. In the first connected state, the first port and the second port of the electrically controlled four-way valve are connected, and the third port and the fourth port are connected, so that the drive motor waste heat recovery circuit and the power battery thermal management circuit are connected in series to transfer the waste heat of the drive motor to the power battery thermal management circuit. In the second connected state, the first port and the third port of the electrically controlled four-way valve are connected, and the second port and the fourth port are connected, so that the drive motor waste heat recovery circuit and the power battery thermal management circuit circulate independently to dissipate the heat of the drive motor to the outside. Figure 4 As shown, the first port is represented as port a, the second port as port b, the third port as port c, and the fourth port as port d.

[0086] Specifically, the waste heat recovery circuit of the drive motor mainly includes a motor water pump, an expansion tank, a low-temperature radiator, a thermostat, a drive motor, and an electrically controlled four-way valve. A combination of a thermostat with a small circulation bypass and an electrically controlled four-way valve is configured to achieve flexible switching between motor circuit cooling and cross-circuit waste heat recovery. The thermostat is located at the inlet of the low-temperature radiator and has a small circulation bypass. It adjusts the flow rate of coolant through the low-temperature radiator according to the motor outlet water temperature. Under heavy-load conditions such as heavy-duty trucks and climbing slopes, the drive motor generates a large amount of waste heat. The coolant in the circuit absorbs the heat generated by the motor. The motor water pump and the electrically controlled four-way valve are connected to the thermal management main controller. The fluid path is switched through the electrically controlled four-way valve. It can either introduce high-temperature coolant into the low-temperature radiator to dissipate heat to the external environment, or export coolant carrying motor waste heat to other heat demand circuits, realizing cross-circuit recovery and reuse of motor waste heat. When waste heat recovery is turned on, the four-way valves ab and cd are connected, and the two circuits are connected in series. The motor waste heat recovery can be transferred to the battery circuit. When waste heat recovery is turned off, the four-way valves ac and bd are connected, and the two circuits circulate on their own.

[0087] Based on the above implementation methods, specific thermal management control also requires consideration of boundary conditions such as ambient temperature and the convective heat transfer coefficient corresponding to vehicle speed. Therefore, a thermodynamic boundary model of the passenger compartment is constructed, incorporating boundary conditions such as ambient temperature and the convective heat transfer coefficient corresponding to vehicle speed. Simultaneously, a longitudinal dynamic model of the vehicle based on the total vehicle mass, rolling resistance, and air resistance is established to reproduce the real-time power consumption of the vehicle under typical test cycles, obtaining the real-time heat generation power of each component and providing input boundaries for the multi-loop thermal management system.

[0088] The total resistance of the vehicle is expressed as: in: In the formula, Total resistance of the vehicle during driving; Indicates rolling resistance; Indicates air resistance; Indicates slope resistance; The values ​​represent acceleration drag; all drag units are in N; m represents the total mass of the vehicle, in kg; g represents the acceleration due to gravity, in _____. f represents the rolling resistance coefficient; Indicates air density, ; Indicates the air drag coefficient; A represents the frontal area, in units of... ; v represents the vehicle's speed, in units of ; Indicates the road slope angle; This represents the conversion factor for the vehicle's rotational mass. Represents vehicle acceleration, in units of .

[0089] The formula for calculating the overall vehicle drive power is: In the formula, This refers to the drive power on the motor side; For transmission system efficiency.

[0090] The real-time heat generation power of the motor is obtained by mapping the drive power to the motor efficiency: In the formula, To generate heat for the drive motor; Here, represents the motor efficiency, and represents the motor power. Motor speed The Map lookup function is used, and the power unit mentioned above is W.

[0091] The heat flow rate of convective heat transfer in the crew cabin is expressed as: In the formula, W represents the convective heat exchange between the passenger compartment and the external environment; h is the convective heat transfer coefficient that varies with vehicle speed. S represents the equivalent heat exchange area of ​​the crew cabin. ; Temperature inside the crew cabin, in K; The ambient temperature is in K.

[0092] The AMESim multi-domain coupling model constructed above, which includes heat pumps, dual water heaters, motors, batteries, and the crew cabin, together constitutes the complex dynamic physical environment of the deep reinforcement learning agent.

[0093] Based on the same inventive concept, this application also provides a deep learning-based low-temperature thermal management system for electric heavy-duty vehicles. The deep learning-based low-temperature thermal management system for electric heavy-duty vehicles includes: a vehicle low-temperature thermal management system and a processor; the vehicle low-temperature thermal management system includes a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, a power battery thermal management circuit, and a four-way valve; the processor is communicatively connected to the vehicle low-temperature thermal management system to implement the steps of the above-mentioned deep learning-based low-temperature thermal management method for electric heavy-duty vehicles.

[0094] In one embodiment, the four-way valve includes four ports: a, b, c, and d. Port a is connected to the motor outlet side of the waste heat recovery circuit of the drive motor, port b is connected to the battery self-circulation valve side of the power battery thermal management circuit, port c is connected to the thermostat outlet side of the waste heat recovery circuit of the drive motor, and port d is connected to the battery cold plate outlet side of the power battery thermal management circuit. When waste heat recovery is turned on, ports a and b of the four-way valve are connected, and ports c and d are connected, so that the waste heat recovery circuit of the drive motor and the power battery thermal management circuit are connected in series. When waste heat recovery is turned off, ports a and c of the four-way valve are connected, and ports b and d are connected, so that the waste heat recovery circuit of the drive motor and the power battery thermal management circuit each perform self-circulation.

[0095] Specifically, such as Figure 4 As shown, the heat pump air conditioning circuit includes a compressor, a gas-liquid separator, an evaporator, an electronic expansion valve, and a heat pump condenser. The compressor outlet is connected to the heat pump condenser inlet, the heat pump condenser outlet is connected in sequence to the electronic expansion valve and the evaporator, the evaporator outlet is connected to the gas-liquid separator, and the gas-liquid separator outlet flows back to the compressor.

[0096] The passenger compartment heating circuit includes a heating circuit water pump, an expansion tank, a heat pump condenser, a power battery heat exchanger, a heating circuit water heater, a heating core, the passenger compartment, and a passenger compartment temperature sensor. The heating circuit water pump outlet flows through the expansion tank to the heat pump condenser, then enters the power battery heat exchanger, then passes through the heating circuit water heater to the heating core, and finally flows back to the heating circuit water pump.

[0097] The waste heat recovery circuit of the drive motor includes a motor water pump, an expansion tank, a low-temperature radiator, a thermostat, a motor, and an electrically controlled four-way valve. The outlet of the motor water pump is divided into two paths after passing through the expansion tank. One path returns to the motor inlet through the low-temperature radiator and the thermostat, and the other path returns directly to the motor inlet. The outlet side of the motor is connected to the electrically controlled four-way valve.

[0098] The power battery thermal management circuit includes a battery water pump, an expansion tank, a battery circuit water heater, a battery cold plate, a battery, a battery temperature sensor, a battery inlet water temperature sensor, a battery self-circulation valve, heat exchange valve 1, heat exchange valve 2, a power battery heat exchanger, and an electronically controlled four-way valve. The battery water pump outlet enters the battery cold plate through the expansion tank and the battery circuit water heater. After passing through the battery, it splits into two paths, controlled by the battery self-circulation valve, heat exchange valve 1, and heat exchange valve 2, and finally flows back to the battery water pump.

[0099] The electronically controlled four-way valve has four ports: a, b, c, and d. Port a is connected to the motor outlet side of the waste heat recovery circuit of the drive motor, port b is connected to the battery self-circulation valve side of the power battery thermal management circuit, port c is connected to the thermostat outlet side of the waste heat recovery circuit of the drive motor, and port d is connected to the battery cold plate outlet side of the power battery thermal management circuit. When waste heat recovery is turned on, ports a and b of the four-way valve are connected, and ports c and d are connected, so that the waste heat recovery circuit of the drive motor and the power battery thermal management circuit are connected in series. When waste heat recovery is turned off, ports a and c of the four-way valve are connected, and ports b and d are connected, so that the two circuits are self-circulating.

[0100] It should be noted that the deep learning-based low-temperature thermal management system for electric heavy-duty vehicles provided in this application embodiment and the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned deep learning-based low-temperature thermal management method for electric heavy-duty vehicles, and the repeated parts will not be described again.

[0101] In some embodiments, the present application provides an electric heavy-duty vehicle, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the above-described deep learning-based low-temperature thermal management method for electric heavy-duty vehicles.

[0102] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0103] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0104] This application also provides a non-transitory storage medium storing a computer program. When executed by a processor, this program implements the aforementioned deep learning-based low-temperature thermal management method for electric heavy-duty vehicles. This storage medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not be assembled into that device / apparatus / system. The aforementioned non-transitory storage medium carries one or more programs, which, when executed, implement the method as described in the embodiments or implementations of this application.

[0105] According to embodiments of this application, a non-transitory storage medium can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. A computer-readable signal medium can also be any storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0107] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for low-temperature thermal management of electric heavy-duty vehicles based on deep learning, characterized in that, The method includes: When the target vehicle is under a preset low temperature condition, the current status and multi-mode preference information of the vehicle's low temperature thermal management system are obtained. The vehicle's low temperature thermal management system includes a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, and a power battery thermal management circuit. The battery temperature tracking weight, the passenger cabin temperature tracking weight, and the energy consumption weight are determined based on the multi-mode preference information. The current state, the battery temperature tracking weight, the passenger cabin temperature tracking weight, and the energy consumption weight are input into the deep reinforcement learning agent to obtain continuous control commands. The actuators in the heat pump air conditioning circuit, passenger compartment heating circuit, drive motor waste heat recovery circuit, and power battery thermal management circuit are independently adjusted according to the continuous control commands to perform low-temperature thermal management on electric heavy-duty vehicles.

2. The method as described in claim 1, characterized in that, When the target vehicle is under a preset low-temperature operating condition, the current state and multi-mode preference information of the vehicle's low-temperature thermal management system are obtained, including: Obtain the ambient temperature of the target vehicle's surroundings; When the ambient temperature is lower than a preset low temperature threshold, the target vehicle is determined to be in a preset low temperature operating condition. In response to the target vehicle being in a preset low-temperature operating condition, the current state is constructed by receiving data from the battery management system, including the power battery temperature, drive motor temperature, cooler temperature, battery state of charge, and ambient temperature. Obtain multi-mode preference information of the target vehicle, including energy-saving mode, comfort mode and balanced mode.

3. The method as described in claim 2, characterized in that, The step of determining the battery temperature tracking weight, the passenger cabin temperature tracking weight, and the energy consumption weight based on the multi-mode preference information includes: The current mode of the target vehicle is determined based on multi-mode preference information, wherein the current mode of the target vehicle includes energy-saving mode, comfort mode and balanced mode; When the battery state of charge is lower than a preset power threshold and the target vehicle is not in energy-saving mode, the target vehicle's current mode is changed to energy-saving mode. Based on the corrected or uncorrected target vehicle current mode, the battery temperature tracking weight, passenger compartment temperature tracking weight, and energy consumption weight are determined from a preset weight mapping relationship. The weight mapping relationship includes energy-saving mode, comfort mode, and balanced mode, and their respective weight combinations.

4. The method as described in claim 1, characterized in that, Before inputting the current state, the battery temperature tracking weight, the passenger compartment temperature tracking weight, and the energy consumption weight into the deep reinforcement learning agent to obtain continuous control commands, the method further includes: A multi-loop coupled thermal management simulation environment for electric heavy-duty vehicles is constructed. Based on the Actor online network, deterministic actions are output according to the input state, and adaptive Ornstein-Uhlenbeck noise is superimposed on the deterministic actions to obtain exploratory actions. The standard deviation of the adaptive Ornstein-Uhlenbeck noise is dynamically corrected based on the training iteration rounds, battery temperature, ambient temperature, and real-time driving power of the vehicle. Specifically, the standard deviation is reduced when the battery temperature is close to the safety threshold or when the vehicle is under heavy load and high power conditions, and the standard deviation is increased when the training iteration rounds are less than the preset round threshold. After performing the exploration action, the updated state and immediate reward are obtained. The current state, the exploration action, the immediate reward and the updated state are stored in the experience replay pool. The network parameters of the deep reinforcement learning agent are sampled from the experience replay pool to update the network parameters of the deep reinforcement learning agent. After the training meets the convergence condition, a deep reinforcement learning agent is obtained, and the adaptive Ornstein-Uhlenbeck noise is turned off.

5. The method as described in claim 4, characterized in that, After performing the exploration action, the process also includes: Based on the deviation between the actual temperature of the power battery and the target temperature of the power battery, and the deviation between the actual temperature of the passenger compartment and the target temperature of the passenger compartment, and combined with the battery temperature tracking weight and the passenger compartment temperature tracking weight, a temperature tracking reward is obtained. Based on the heating power consumption of the warm air circuit water heater and the battery circuit water heater, and the regenerative braking recovery power allowed under the current power battery temperature boundary, an energy consumption penalty reward is obtained; wherein, the heating power consumption is penalized, and the regenerative braking recovery power is positively incentivized. The penalty for exceeding the limit is determined based on the abnormal truncation penalty and the temperature safety range penalty; A multi-objective reward function is determined based on the temperature tracking reward, the energy consumption penalty reward, and the over-limit truncation penalty, and an immediate reward is determined based on the multi-objective reward function.

6. The method as described in claim 4, characterized in that, The step of sampling from the experience replay pool to update the network parameters of the deep reinforcement learning agent includes: Calculate the target Q-value based on the next time step state and the target action output by the Actor target network; Construct a total loss, which includes a time-series difference mean square error term, a battery temperature over-limit penalty term, and an actuator motion smoothing penalty term. The actuator motion smoothing penalty term is generated when the difference in control actions between adjacent time moments exceeds a preset threshold. The parameters of the Critic online network are updated by minimizing the total loss through gradient descent backpropagation. The Critic online network outputs value, and the parameters of the Actor online network are updated based on maximizing the value. Based on the parameters of the Actor online network and the Critic online network, the Actor target network and the Critic target network are updated by weighted average.

7. The method according to any one of claims 1 to 6, characterized in that, The actuators that independently adjust the heat pump air conditioning circuit, the passenger compartment heating circuit, the drive motor waste heat recovery circuit, and the power battery thermal management circuit according to the continuous control command include: According to the continuous control command, a speed adjustment signal is output to the heat pump air conditioning circuit to continuously adjust the compressor speed to match the real-time heat load; According to the continuous control command, power adjustment signals are output to the water heater of the heating circuit in the passenger compartment heating circuit and the water heater of the battery circuit in the power battery thermal management circuit, respectively, to independently adjust the heating power of the two circuits, and when the heat pump air conditioning circuit outputs insufficient heat, the water heater of the heating circuit is started for auxiliary heating. According to the continuous control command, the on / off state of the battery self-circulation valve, the first heat exchange valve and the second heat exchange valve in the power battery thermal management circuit is controlled so that the power battery thermal management circuit switches between independent self-circulation mode and receiving heat from the passenger compartment heating circuit. According to the control component of the corresponding electronically controlled four-way valve in the continuous control command, the electronically controlled four-way valve in the drive motor waste heat recovery circuit is controlled to be in either a first connected state or a second connected state; in the first connected state, the drive motor waste heat recovery circuit is connected in series with the power battery thermal management circuit to transfer the waste heat of the drive motor to the power battery thermal management circuit; in the second connected state, the drive motor waste heat recovery circuit and the power battery thermal management circuit circulate independently to dissipate the heat of the drive motor to the outside.

8. A cryogenic thermal management system for electric heavy-duty vehicles based on deep learning, characterized in that, The deep learning-based cryogenic thermal management system for electric heavy-duty vehicles includes a vehicle cryogenic thermal management system and a processor. The vehicle low-temperature thermal management system includes a heat pump air conditioning circuit, a passenger compartment heating circuit, a drive motor waste heat recovery circuit, a power battery thermal management circuit, and a four-way valve. The processor is communicatively connected to the vehicle cryogenic thermal management system and is used to implement the steps of the deep learning-based cryogenic thermal management method for electric heavy-duty vehicles as described in any one of claims 1 to 7.

9. The system as described in claim 8, characterized in that, The four-way valve includes four ports: a, b, c, and d. Port a is connected to the motor outlet side of the waste heat recovery circuit of the drive motor; port b is connected to the battery self-circulation valve side of the power battery thermal management circuit; port c is connected to the thermostat outlet side of the waste heat recovery circuit of the drive motor; and port d is connected to the battery cold plate outlet side of the power battery thermal management circuit. When waste heat recovery is turned on, ports a and b of the four-way valve are connected, and ports c and d are connected, so that the waste heat recovery circuit of the drive motor and the power battery thermal management circuit are connected in series. When waste heat recovery is turned off, ports a and c of the four-way valve are connected, and ports b and d are connected, so that the waste heat recovery circuit of the drive motor and the power battery thermal management circuit each perform self-circulation.

10. An electric heavy-duty vehicle, characterized in that, The electric heavy-duty vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the deep learning-based low-temperature thermal management method for electric heavy-duty vehicles as described in any one of claims 1 to 7.

Images