Energy efficiency optimization method of heat pump type heat management system

Abstract

The invention relates to the field of data processing, in particular to an energy efficiency optimization method of a heat pump type heat management system, which comprises the following steps of: acquiring state parameters such as environment temperature, target temperature, actual temperature, high and low pressure side pressure, compressor power and the like, performing control decision making based on a deep Q network (DQN) model, and adjusting the rotating speed of a compressor. The core of the method is that a self-adaptive discount factor is adopted, the factor is dynamically adjusted according to a system state index, and the system state index is calculated by combining a current thermal state deviation absolute value and a circulating pressure ratio. Through the mechanism, the DQN can dynamically adjust the strategy emphasis according to the target deviation degree and the operation intensity of the system, the multiple and dynamic change requirements of the electric vehicle for passenger comfort, battery health and endurance mileage (energy efficiency) are effectively balanced, and the comprehensive performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and particularly to an energy efficiency optimization method for a heat pump type thermal management system. Background Art

[0002] As a key component of sustainable transportation, the development of electric vehicles (EVs) is deeply influenced by factors such as driving range, charging convenience, and user experience. Among them, the heat pump type thermal management system has become an indispensable core component of modern electric vehicles due to its potential for efficient cooling and heating in different environments. This system undertakes two key tasks: one is to provide a comfortable temperature environment for the passenger compartment; the other is to precisely manage the operating temperature of the power battery pack. However, the operation of the thermal management system is the main non-driving energy consumption source of electric vehicles.

[0003] The actual operating environment of electric vehicles is extremely complex and dynamically variable. Traditional control strategies, such as logic control based on preset rules or classical PID (Proportional-Integral-Derivative) controllers, are difficult to effectively handle systems with highly nonlinear, multivariable coupling, and strong time-varying characteristics. Rule-based methods often rely on the experience of engineers and a large number of experimental calibrations, making it difficult to cover all working conditions and ensure optimality, and their control effects are often suboptimal; while PID controllers are overwhelmed when dealing with systems with multiple inputs and outputs, strong nonlinear coupling, difficult parameter tuning, and hard to balance conflicting control objectives (such as rapid cooling and low energy consumption).

[0004] In recent years, with the development of artificial intelligence technology, reinforcement learning (RL), especially deep reinforcement learning methods such as Deep Q-Network (DQN), has shown strong self-learning and optimization capabilities in complex decision-making problems. However, existing standard deep Q-network algorithms still have inherent defects when applied to scenarios such as electric vehicle thermal management that require balancing immediate needs and long-term goals. The fixed discount factor limits the flexibility of DQN strategies and their adaptability to variable scenarios, unable to achieve the optimal dynamic balance, thus limiting their potential in improving the real-world energy efficiency of electric vehicles. Summary of the Invention

[0005] In view of the problem that the above fixed discount factor limits the flexibility of the DQN strategy and its adaptability to changing scenarios, the present invention proposes an energy efficiency optimization method for a heat pump type thermal management system, including: obtaining historical state parameters of an electric vehicle heat pump type thermal management system, where the historical state parameters at least include: ambient temperature, target temperature, actual temperature, low-pressure side pressure, high-pressure side pressure, and compressor input power; training a deep Q network model based on the historical state parameters, using multiple state parameters at the same moment as the state vector of the deep Q network model; obtaining the real-time state vector of the heat pump type thermal management system, inputting the real-time state vector into the trained deep Q network model, and adjusting the operating speed of the compressor in the thermal management system according to the action instruction output by the model to optimize its energy efficiency; the deep Q network model also includes using an adaptive discount factor for training, and there is: ; where represents the maximum setting value of the discount factor; represents the minimum setting value of the discount factor; is a tuning factor; represents the system state index; the system state index is the product of the absolute value of the thermal state deviation and the cycle pressure ratio; the absolute value of the thermal state deviation is the absolute value of the difference between the current actual temperature and the target temperature; the cycle pressure ratio is the ratio of the current high-pressure side pressure to the low-pressure side pressure.

[0006] The present invention dynamically adjusts the adaptive discount factor through a deep Q network combined with a system state index jointly determined by the thermal state deviation and the cycle pressure ratio, solving the problem that the existing fixed control logic or standard reinforcement learning cannot effectively adapt to the core contradiction of dynamically balancing heat demand and energy-saving endurance under the complex and changeable working conditions of electric vehicles. This method enables the control strategy to intelligently switch the focus according to the degree of deviation of the system from the target and the operating degree, quickly respond when needed, and be energy-efficient in detail when stable. Compared with traditional methods and standard DQN, it significantly improves the adaptive ability and overall energy efficiency of the electric vehicle thermal management system.

[0007] Furthermore, the state vector of the deep Q network model is specifically: ; where represents the state vector at time ; represents the ambient temperature at time ; represents the target temperature at time ; represents the actual temperature at time ; represents the low-pressure side pressure at time ; The high-side pressure at the moment ; The compressor input power at the moment ;

[0008] Furthermore, the calculation method of the absolute value of the thermal state deviation is specifically as follows: ; where represents the absolute value of the thermal state deviation at the moment ; is the actual temperature at the moment ; is the target temperature at the moment ;

[0009] Furthermore, the calculation method of the cycle pressure ratio is specifically as follows: ; where represents the cycle pressure ratio at the moment ; is the high-side pressure at the moment ; is the low-side pressure at the moment ; represents the parameter adjustment factor.

[0010] The present invention reflects the current operating intensity or potential efficiency range of the heat pump cycle through the cycle pressure ratio. The pressure ratio is directly related to the compression work, temperature rise, and theoretical efficiency. Taking it as another key input for calculating the system state index enables subsequent adaptive adjustment to consider the current operating load of the system, rather than just the temperature deviation, improving the dimension and accuracy of adaptability.

[0011] Furthermore, the calculation method of the system state index is specifically as follows: ; where represents the system state index; represents the absolute value of the thermal state deviation at the moment ; represents the cycle pressure ratio at the moment ;

[0012] By constructing the system state index, the present invention realizes a non-linear combination method, which can effectively amplify the bad states that need to be prioritized, such as large deviations and high operating intensity, while avoiding the introduction of weights that require manual adjustment. This construction method can better reflect the system operating intensity under the joint action of multiple factors than a simple linear combination, providing a more sensitive and reasonable input for the adjustment of the adaptive discount factor.

[0013] Furthermore, the action instruction output by the deep Q-network model is defined as the adjustment amount of the compressor speed, specifically: ; wherein means reducing the operating speed of the compressor by a preset step; 0 means keeping the current speed of the compressor unchanged; means increasing the operating speed of the compressor by a preset step.

[0014] Furthermore, the reward function adopted during the training of the deep Q-network model is specifically: ; wherein represents the reward function; represents the time 's actual temperature; represents the time 's target temperature.

[0015] Furthermore, the training process of the deep Q-network model further includes: The hidden layer adopts the ReLU activation function; the learning rate is adaptively adjusted using the Adam optimizer; the action decision adopts the ε-greedy strategy.

[0016] Applying the mature technologies (ReLU, Adam, ε-greedy) in the current deep reinforcement learning field effectively improves the learning efficiency, stability of the model in dealing with the complex non-linear dynamics of the electric vehicle thermal management system, and the possibility of converging to a high-quality strategy.

[0017] Furthermore, it also includes data cleaning and data standardization of the historical state parameters.

[0018] Furthermore, the data cleaning is the median filtering algorithm; the data standardization is the Z-Score standardization algorithm By preprocessing the input data, the quality and consistency of the data are improved, and the adverse effects of noise interference and different parameter scale differences on model training are reduced. This enhances the robustness of the training model, improves the generalization ability of the model to real-world data, and ensures the reliability and stability of the final control strategy in practical applications, which is superior to directly using the original data for training.

[0019] The technical effects of the present invention are: In view of the problem of energy efficiency optimization of the thermal management system for electric vehicles, the present invention proposes an intelligent control method based on Deep Q-Network (DQN), and the key is to introduce an adaptive discount factor mechanism. Different from the prior art, this adaptive adjustment does not rely on complex external parameter estimation or artificial weights, but is driven by a system state index closely related to the physical logic of the scenario. This index is obtained by multiplying the absolute value of the real-time thermal state deviation by the pressure ratio reflecting the cycle operation intensity. The discount factor dynamically adjusted based on this index enables DQN to intelligently switch the time-scale focus of its decision-making according to the current degree of deviation of the system from the target and the degree of "effort" in operation, so as to achieve a dynamic and effective balance between meeting immediate needs such as comfort and battery health and the long-term energy efficiency goal of maximizing the driving range. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein: Figure 1 FIG. is a flowchart schematically showing an energy efficiency optimization method for a heat pump type thermal management system in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] The specific embodiments of the present invention will be described in detail below with reference to the drawings.

[0023] An embodiment of an energy efficiency optimization method for a heat pump type thermal management system: As Figure 1 shown, an energy efficiency optimization method for a heat pump type thermal management system of the present invention includes: S1. Collect and preprocess the core operation state parameters of the heat pump system of the electric vehicle.

[0024] The heat pump system of the electric vehicle generally includes components such as a compressor, a condenser, an evaporator, a throttling device, multiple heat exchangers, a coolant circulation circuit, and related valve groups. To achieve energy efficiency optimization control, the following parameters can be monitored in real time in this embodiment: Collect the ambient temperature through a temperature sensor installed on the vehicle, which is the basic condition for determining the heat exchange intensity between the vehicle and the outside world; integrate the passenger compartment temperature set by the user and the target temperature of the BMS (Battery Management System), which represents the current control target; obtain the actual temperature through the passenger compartment air temperature sensor and the internal / coolant temperature sensor of the battery pack, which corresponds to the actual measured value of the target temperature; collect the low-side pressure through the compressor suction-side pressure sensor, which reflects the operating state of the evaporator; collect the high-side pressure through the compressor discharge-side pressure sensor, which reflects the operating state of the condenser; monitor the compressor input power of the compressor through a power sensor; All the above collected data can be transmitted in real time to the central control unit of the vehicle (such as the vehicle control unit VCU or a dedicated thermal management controller) through the in-vehicle controller area network (CAN) bus. In this embodiment, the sampling frequency of all sensors can be set to 1 Hz to capture the dynamic changes in the vehicle operating state.

[0025] Furthermore, all the collected raw data needs to be preprocessed, including: Use methods such as median filtering to filter out the instantaneous noise or abnormal jumps in the sensor data to ensure the reliability of the data; then adopt Z-Score normalization to eliminate the influence of different physical dimensions and numerical range differences of temperature, pressure, power, etc. on the subsequent neural network model training, so that the model can fairly learn the importance of each input feature. The above preprocessing operations are well-known technologies, and the specific implementation methods will not be elaborated here.

[0026] S2. Define the state, action, and reward of the deep Q-network algorithm; construct the model of the deep Q-network algorithm.

[0027] Use the deep Q-network (DQN) as the core intelligent algorithm to learn the control strategy for optimizing the heat pump energy efficiency in a complex and dynamically changing electric vehicle operating environment.

[0028] The definitions of the key elements are as follows: State vector ; This vector provides the information set required for decision-making by the DQN, including environmental conditions, thermal management requirements, demand satisfaction, key thermodynamic states of the system, and the current energy consumption cost, where represents the ambient temperature at time ; represents the target temperature at time ; represents the actual temperature at time ; represents the low-side pressure at time ; represents the high-side pressure at time ; The compressor input power at the moment .

[0029] Action space is defined as the set of operations that the agent can choose to execute in the state . In this embodiment, it is set as the adjustment amount of the compressor speed: ; where means reducing the operating speed of the compressor by a preset step; 0 means keeping the current speed of the compressor unchanged; means increasing the operating speed of the compressor by a preset step. In this embodiment, the preset step can be set as the empirical value 100, with the unit of RPM.

[0030] Reward function is used to guide DQN learning and is defined as the immediate scalar signal feedback by the environment after the agent executes the action in the state . It is used to evaluate the quality of this action. That is to say, it needs to reflect the core trade-off goal of electric vehicle thermal management. Then there is: ; where represents the actual temperature at the moment ; represents the target temperature at the moment . When the actual temperature is closer to the target temperature, the absolute value of the deviation is smaller, and the reward value is larger (closer to 0), encouraging the agent to take actions that can minimize the temperature deviation.

[0031] Further construct the DQN model: First, determine its network structure: adopt a multi-layer fully connected neural network with a ReLU (Rectified Linear Unit) activation function to fit the function of state to action value (Q value) . The input layer is used to receive the state vector , with the size of the state vector dimension, which is 6 in this embodiment; the output layer corresponds to the Q values of the action set, and its size is the action space dimension, which is 3 in this embodiment.

[0032] Then determine the loss function of this network, which is the mean square value of the temporal difference (TD) error based on the Bellman equation: ; where represents the value of the loss function; represents the expectation operation function; represents the immediate reward after executing the action; represents the discount factor; represents the Q value corresponding to the optimal action in the next state Among all possible actions The maximum value among the corresponding Q-values; Represents the current state Under this condition, execute the action The expected cumulative reward after execution, and the specific Q-value update formula is as follows: ; Wherein Represents the learning rate, which can be adaptively adjusted using the Adam optimizer in this embodiment; the meanings of the remaining parameters and symbols are the same as those in the above loss function. Finally, in this embodiment, the action decision can adopt the ε-greedy strategy to balance exploration (trying new actions) and exploitation (executing known optimal actions) during training.

[0033] S3. Calculate the system state index based on the operating state of the electric vehicle, and then calculate the adaptive discount factor.

[0034] In step S2, the model construction and training of the deep Q-network are completed. The core parameter of this algorithm includes the discount factor 𝛾, which determines the influence degree of the current action on future rewards. However, the operating scenarios of electric vehicles are diverse (such as urban congestion, highway cruising, rapid charging, cold start, etc.), and the requirements for the focus of the thermal management strategy (quick response or extreme energy saving) are also different. A fixed discount factor cannot adapt to this dynamic demand. Therefore, in this embodiment, the discount factor is subsequently improved to enhance the adaptability and optimization effect of the control system.

[0035] First, quantify the most direct performance of the current system, that is, the actual temperature And the desired target The gap between them. This deviation is the fundamental reason for driving the control system to take actions, and its magnitude directly reflects the completion degree or deficiency of the current thermal management task. There is: ; Wherein (Unit: °C) represents the absolute value of the thermal state deviation at time ; (Unit: °C) is the actual measured temperature at time ; (Unit: °C) is the target set temperature at time ;

[0036] Calculate the absolute difference between the actual temperature And the desired target To obtain the absolute value of the thermal state deviation. The larger the value, the farther away from the target, and the greater the possibility or amplitude of adjustment required.

[0037] Then calculate the cycle pressure ratio, which is an indicator reflecting the current "working intensity" or "operating conditions" of the heat pump cycle. The suction and discharge pressures of the compressor ( ) are the core thermodynamic state parameters. Their ratio, i.e., the cycle pressure ratio, is thermodynamically closely related to the theoretical work required by the compressor, the temperature rise capacity of the cycle, and the efficiency.

[0038] Generally, when a greater temperature rise is needed (such as when the temperature difference between the inside and outside is large) or the performance of some components of the system (such as heat exchangers) is poor, the pressure ratio will increase. A high pressure ratio often means that the compressor operates more strenuously and the potential efficiency of the entire cycle is lower. Therefore, the cycle pressure ratio is calculated as: ; where (dimensionless) represents the cycle pressure ratio at time ; (unit: Pa) is the high-side pressure at time ; (unit: Pa) is the low-side pressure at time ; represents the tuning parameter factor, which is used to avoid the denominator being zero and can be set to the empirical value of 1e-5. The higher this ratio, the more the heat pump cycle operates in the range that requires greater compression work and lower theoretical efficiency.

[0039] Next, obtain the system state index. The adjustment requirement of the system depends not only on the degree of deviation from the target , but also on the current working intensity of the system operation. When the system is far from the target and at the same time operates under the condition of a high pressure ratio (high intensity / low potential efficiency), it indicates an unfavorable situation and urgent adjustment is needed. At this time, the algorithm needs to pay more attention. The system state index is: ; where, (unit: °C) is the calculated system state index; represents the absolute value of the thermal state deviation at time ; represents the cycle pressure ratio at time . When the temperature deviation is large and the cycle pressure ratio is high, the system state index will be amplified sharply.

[0040] Finally, determine the adaptive discount factor. According to the calculated system state index, dynamically adjust the discount factor of the DQN algorithm when evaluating future rewards . When has a high value, indicating that the system is in a challenging state (large deviation or high operating intensity), the algorithm should pay more attention to short-term effects and respond quickly; when A low value indicates that the system is in good condition (close to the target and running smoothly). The algorithm should pay more attention to long-term benefits and finely optimize energy efficiency. Then the adaptive discount factor is: ; where (dimensionless) represents the adaptive discount factor; represents the maximum value of the preset discount factor, which can be set to 0.99 in this embodiment, representing the maximum degree of emphasis on the future; represents the minimum value of the preset discount factor, which can be set to 0.5 in this embodiment, representing the minimum degree of emphasis on the future; (unit: 1 / °C) is a positive sensitivity adjustment parameter used to control the sensitivity of the discount factor to the system state index, which can be set to the empirical value 0.5 in this embodiment; represents the system state index; is the base of the natural logarithm.

[0041] When tends to 0 (ideal state), tends to 1, tends to 0, tends to , at this time, the deep Q-network balances the current and future rewards to maintain stable control of the system; when increases, tends to 0, tends to 1, tends to , at this time, the deep Q-network strengthens the role of the current reward more, so as to quickly adjust the operating speed of the compressor.

[0042] S4. Integrate the deep Q-network model with the adaptive discount factor into the thermal management control unit of the electric vehicle to complete intelligent optimization control of energy efficiency.

[0043] In step S3, the adaptive discount factor is obtained. Then, combining with the deep Q-network loss function determined in step S2, the final loss function is: ; where represents the value of the loss function; represents the expected operation function; represents the immediate reward after executing the action; represents the adaptive discount factor; represents in the next state among all possible actions the maximum value of the corresponding Q values; represents the current state Under the following conditions, perform actions The expected cumulative reward after that, and the improved Q-value update formula is as follows: ; where represents the learning rate, which can be adaptively adjusted using the Adam optimizer in this embodiment; the meanings of the remaining parameters and symbols are the same as those in the above loss function.

[0044] Apply the improved DQN model containing the calculation logic to actual control: Use the historical operation data of electric vehicles containing various real driving cycles, charging scenarios, and environmental conditions, that is, the operation data containing the parameters described in step S2. In one embodiment, it can be the time-series data of the parameters in step S2, to perform offline training on the model so that it learns different and optimal action strategies under different conditions; then deploy the trained model to the thermal management controller or vehicle control unit (VCU) of the electric vehicle.

[0045] The specific real-time control process is as follows: First, the controller collects sensor data in real time through the CAN bus to form the current state ; then the controller calculates the current system state index according to ; then input into the DQN model, and the model outputs the Q-values corresponding to each action (compressor speed adjustment); further, the controller selects the optimal action according to the Q-value ; then the controller sends control commands to actuators such as the compressor frequency converter to perform the action ; finally, the system state changes, enters the next control cycle, and repeats this closed-loop control process. ; finally, the system state changes, enters the next control cycle, and repeats this closed-loop control process.

Claims

1. A method for optimizing energy efficiency of a heat pump type thermal management system, characterized in that: The method comprises: Obtain historical state parameters of a heat pump thermal management system for an electric vehicle, wherein the historical state parameters include at least: ambient temperature, target temperature, actual temperature, low-pressure side pressure, high-pressure side pressure, and compressor input power; train a deep Q network model based on the historical state parameters, and use multiple state parameters at the same time as state vectors of the deep Q network model; Obtaining a real-time state vector of a heat pump type thermal management system, inputting the real-time state vector into a trained deep Q network model, and adjusting the operating speed of the compressor in the thermal management system according to the action instructions output by the model to optimize its energy efficiency; The deep Q network model also includes the use of an adaptive discount factor Training is performed with: ;in Indicates the maximum setting value of the discount factor; Indicates the minimum setting value of the discount factor; is the parameter adjustment factor; Indicates the system status index; The system state index is the product of the absolute value of the thermal state deviation and the circulation pressure ratio; the absolute value of the thermal state deviation is the absolute value of the difference between the current actual temperature and the target temperature; the circulation pressure ratio is the ratio of the current high-pressure side pressure to the low-pressure side pressure.

2. The energy efficiency optimization method of a heat pump type thermal management system according to claim 1, characterized in that: The state vector of the deep Q network model is specifically: ； in Indicates time The state vector of Indicates time Ambient temperature; Indicates time The target temperature; Indicates time The actual temperature; Indicates time The low pressure side pressure; Indicates time High pressure side pressure; Indicates time The compressor input power.

3. The energy efficiency optimization method of a heat pump type thermal management system according to claim 1, characterized in that: The absolute value of the thermal state deviation is calculated as follows: ； in Indicates at time The absolute value of thermal state deviation; It's time The actual temperature; It's time target temperature.

4. The energy efficiency optimization method of a heat pump type thermal management system according to claim 1, characterized in that: The circulation pressure ratio is calculated as follows: ； in Indicates at time The cycle pressure ratio; It's time High pressure side pressure; It's time The low pressure side pressure; Represents the tuning factor.

5. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 1, 3 or 4, characterized in that: The system status index is calculated as follows: ； in Indicates the system status index; Indicates at time The absolute value of thermal state deviation; Indicates time cycle pressure ratio.

6. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 1, characterized in that: The action command output by the deep Q network model is defined as the adjustment amount for the compressor speed, specifically: ； in Indicates reducing the compressor speed by a preset step; 0 means keeping the current compressor speed unchanged; Indicates increasing the compressor operating speed by a preset step.

7. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 1, characterized in that: The reward function used in the training of the deep Q network model is specifically: ； in represents the reward function; Indicates time The actual temperature; Indicates time target temperature.

8. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 1, characterized in that: The training process of the deep Q network model also includes: The hidden layer uses the ReLU activation function; the learning rate is adaptively adjusted using the Adam optimizer; and the action decision adopts the ε-greedy strategy.

9. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 1, characterized in that: It also includes data cleaning and data standardization of the historical state parameters.

10. The method for optimizing energy efficiency of a heat pump type thermal management system according to claim 9, characterized in that: The data cleaning is performed by a median filtering algorithm; the data standardization is performed by a Z-Score standardization algorithm.

Images