Defrosting self-adaptive control method for whole vehicle thermal management system based on PI-DQN and joint simulation

Abstract

The invention discloses a self-adaptive defrosting control method for a vehicle thermal management system based on PI-DQN and joint simulation, and belongs to the technical field of new energy vehicle thermal management, the core scheme comprises the following steps: (1) constructing an AMESim dynamic simulation model, and accurately simulating the frosting and defrosting physical process of a vehicle-mounted heat pump air conditioning evaporator; and (2) designing a PI-DQN model, and outputting a control action through a neural network according to a system state. And (3) establishing a joint simulation platform, and realizing real-time data interaction between the AMESim model and the Python model by using a shared memory. (4) defining a state space, an action space and a reward function; and (5) training a PI-DQN model: combining a loss function with a data fitting item and a physical constraint item to ensure that a strategy accords with a physical rule. According to the method, through deep reinforcement learning collaborative optimization of joint simulation platform and physical information enhancement, the energy efficiency and dynamic adaptability of the system are remarkably improved, and the problems of energy efficiency loss, comfort reduction and response delay in traditional defrosting control are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of new energy vehicle thermal management, and particularly relates to a dynamic control method combining an AMESim-Python joint simulation platform and a physical information enhanced deep Q network (PI-DQN), which is used for real-time optimization of the operation efficiency of a whole vehicle thermal management system (including air conditioning refrigeration, battery cooling, motor heat dissipation and other subsystems), especially optimization of defrosting control strategy in a low temperature and high humidity environment, so as to improve the performance and energy efficiency of the heat pump air conditioner of the electric vehicle in cold climate conditions. BACKGROUND

[0002] The heat pump air conditioning system of the electric vehicle plays a key role in the vehicle interior environment regulation, especially in cold climate. However, at this time, the evaporator surface is prone to frost formation, which will cause the system performance to decline, so frequent defrosting operation is needed.

[0003] The existing defrosting starting method has many problems. The traditional thermal management system relies on fixed thresholds or PID control, which is difficult to cope with complex dynamic working conditions, such as changes in vehicle speed, fluctuations in environmental temperature and humidity, etc. At present, the defrosting starting method based on time control or demand control commonly used in vehicle-mounted heat pump air conditioning systems relies on sensors to detect the frost condition of the outdoor heat exchanger, and when the detected time, temperature or humidity and other parameters reach the preset threshold, the defrosting process is automatically started. However, these thresholds are usually set by simplified heuristic rules, without considering the diversity of actual operating conditions, and cannot dynamically adapt to environmental changes, which may cause defrosting too early or too late, resulting in energy waste and system performance decline.

[0004] Specifically, the existing defrosting control method has the following problems: first, energy loss, frequent defrosting will cause the COP (coefficient of performance) to decline, and delayed defrosting will cause the evaporator to be blocked; second, multi-objective conflict, the comfort of the passenger compartment, defrosting stability and system energy consumption need to be balanced; third, lack of predictability, unable to adjust control parameters in advance based on future frost layer growth trend.

[0005] In recent years, model predictive control (MPC) and machine learning methods have been gradually applied in this field, but MPC relies on accurate physical models, and traditional machine learning lacks dynamic decision-making ability. Deep reinforcement learning (DRL) realizes strategy optimization through trial and error learning, providing a new idea for real-time control of complex systems, but its joint deployment with high-fidelity simulation models still has technical challenges.

[0006] Therefore, it is particularly important to determine the optimal defrosting starting time to maximize the performance of the heat pump, and the present application aims to solve this problem and proposes a whole vehicle thermal management system defrosting adaptive control method based on a physical information enhanced deep Q network (PI-DQN) and joint simulation. SUMMARY

[0007] In order to solve the problems in the prior art, the application provides a defrosting adaptive control method for a whole vehicle thermal management system based on PI-DQN and joint simulation, which can autonomously learn and adapt to different operating conditions, optimize the defrosting strategy, realize adaptive defrosting control of the whole vehicle thermal management system under complex working conditions, improve system efficiency and passenger comfort, and reduce energy consumption and prolong system life.

[0008] In order to achieve the above-mentioned purpose, the application adopts the following specific scheme: The defrosting adaptive control method for the whole vehicle thermal management system based on PI-DQN and joint simulation comprises the following steps: Step S1, constructing a vehicle-mounted heat pump air conditioner dynamic simulation model: based on the basic principles of thermodynamics and the heat pump cycle principle, a vehicle-mounted heat pump air conditioner dynamic simulation model containing the physical process of evaporator frosting and defrosting is constructed in AMESim; Step S2, creating a PI-DQN model: a PI-DQN model is constructed, which comprises an input layer, a hidden layer and an output layer, and is used to generate control actions according to state inputs; Step S3, establishing a joint simulation interface: a joint simulation architecture comprising an AMESim master model and a Python slave model is established, and real-time data interaction between the vehicle-mounted heat pump air conditioner dynamic simulation model and the PI-DQN model is realized through shared memory; Step S4, defining state space, action space and designing reward function: based on the Markov decision process, the state space, the action space and the reward function are defined; The state space comprises four types of features obtained in real time from the vehicle-mounted heat pump air conditioner dynamic simulation model: real-time sensor data, dynamic change rate, energy efficiency index and frost layer state feature; the frost layer state feature is calculated by processing the measured temperature of multiple positions on the evaporator surface and the preset frost-free state setting temperature and the allowable minimum temperature, and is used as the input of the PI-DQN model; The action space comprises actions for controlling the thermal management system in the vehicle-mounted heat pump air conditioner dynamic simulation model: compressor speed, electronic expansion valve opening degree, defrosting cycle length and fan speed, for controlling the vehicle-mounted heat pump air conditioner dynamic simulation model; The reward function fuses the energy efficiency index change, the passenger cabin temperature deviation and the frost layer state feature calculated based on the running state of the vehicle-mounted heat pump air conditioner dynamic simulation model, and provides environmental feedback to the PI-DQN model; Step S5, training the PI-DQN model: using PI-DQN for policy training to learn the optimal control policy; the loss function of the PI-DQN model includes a data fitting term loss function and a physical constraint term loss function; wherein the physical constraint term in the physical constraint term loss function is constructed based on the principle of energy conservation and phase change thermodynamics, and is used to constrain the relationship between the change amount of the frost coverage and the temperature difference between the evaporator outlet temperature and the dew point temperature.

[0009] Further, the PI-DQN model created in step S2 includes: Input layer: receiving the state space; At least one hidden layer: using a fully connected neural network structure; Output layer: generating control instructions of the action space.

[0010] Further, in step S3, in the joint simulation architecture: The AMESim master model is configured with a shared memory communication interface module, set to Master mode, and configured with a shared memory identifier, a sampling time, an input port number and an output port number; The Python slave model realizes data exchange with the AMESim master model through a shared memory interface class, which includes methods for initializing connection, executing data exchange and closing connection.

[0011] Further, in step S4, the real-time sensor data in the state space includes one or more of the evaporator inlet and outlet temperatures, the evaporator inlet and outlet refrigerant side pressures, the compressor speed, the ambient temperature, the ambient humidity, the battery temperature, the battery cold plate temperature, the fan speed, the condenser outlet temperature, and the passenger compartment temperature; the dynamic change rate includes one or more of the evaporator pressure change rate, the condenser temperature change rate, the compressor current change rate, the ambient humidity change rate, and the vehicle acceleration; and the energy efficiency index is the real-time coefficient of performance.

[0012] Further, in step S4, the frost state feature includes the frost coverage, and the calculation formula for calculating the frost coverage based on the evaporator surface temperature distribution is: Wherein, T evap_set is the evaporator temperature set value in the frost-free state; T evapi is the measured temperature of the i-th position on the evaporator surface, i takes 1, 2, 3, representing three different monitoring points; T evap_min is the lowest temperature allowed on the evaporator surface; ε takes 1×10 -5 .

[0013] Further, in step S4, the reward function R is specifically: R = a * ACOP - b * |T cabin -T cabinset |- g * Frost coverage Wherein, ACOP is the maximum energy conversion efficiency, T cabin represents the passenger cabin temperature detected by the sensor, T cabinset represents the target temperature set by the passenger cabin, a, b, g are weight coefficients determined by the Pareto frontier analysis.

[0014] Further, the weight coefficients a, b and g in the reward function are determined by a multi-objective optimization algorithm, specifically including: (1) the normalized energy efficiency target comfort target and frost control target are optimized by the Pareto frontier analysis; (2) the optimal weight combination is solved by non-dominated sorting, crowding calculation, simulated binary crossover and polynomial mutation iteration.

[0015] Further, the PI-DQN model adopts a double network structure and an experience replay mechanism. When training the model, the following closed-loop learning process is performed: a) action decision: the PI-DQN model adopts an e-greedy strategy to select an action according to the input state space, and acts on the dynamic simulation model constructed by S1 through the joint simulation architecture; b) reward feedback: based on the running results of the dynamic simulation model, the reward value of the reward function is calculated; c) data storage: store the state transition samples generated by the interaction to the experience replay mechanism; d) network update: when the experience replay data reaches a threshold, sample batch data, update the current Q network parameters by gradient descent algorithm using the loss function; e) target synchronization: regularly update the target Q network parameters in a soft update manner.

[0016] Further, in step S5, the formula of the loss function is: L(0) = L DQN + l L * L physics Wherein, L DQN is the data fitting item loss function, N is the number of samples in the batch, R(s, a) is the immediate reward obtained by taking action a in state s, and l is a discount factor (0 < l < 1); is the maximum estimated value of the Q value for all possible actions a' of the next state s'; Q(s, a; θ) represents the Q value of all possible actions in the current state using the parameters θ of the current main network to predict; λ L is 0.5; L physics is a physical constraint term loss function, ΔFrost_coverage is the frost layer coverage change amount, t is the tth time recorded, Frost coverage_t+1 is the frost layer coverage at the next time, Frost coverage_t is the frost layer coverage at the current time.

[0017] The PI-DQN and joint simulation-based vehicle thermal management system defrosting adaptive control method provided by the application has many significant beneficial effects, and the specific steps and model analysis are as follows:

[0018] (1) Improve system energy efficiency: the vehicle-mounted heat pump air conditioner dynamic simulation model constructed in step S1 can accurately simulate the physical process of evaporator frosting and defrosting, providing a reliable simulation basis for the optimization of subsequent control strategies. The state space defined in step S4 contains the real-time energy efficiency indicator COP, and ΔCOP is also an important part of the reward function R. Through the training of the PI-DQN model (step S5), the system can continuously learn how to maximize COP. At the same time, the physical constraint term loss function is based on the principle of energy conservation and phase change thermodynamics, and the relationship between the change amount of the frost layer coverage and the temperature difference between the evaporator outlet temperature and the dew point temperature is constrained, avoiding the energy loss caused by improper defrosting time, reducing the influence of frequent defrosting or delayed defrosting on COP, and thus improving the energy efficiency of the system.

[0019] (2) Optimize multi-objective balance: the reward function R designed in step S4 combines the energy efficiency indicator change amount, passenger compartment temperature deviation and frost layer state characteristics, and balances these three objectives through weight coefficients α, β, γ (determined through Pareto frontier analysis). In specific implementation, the NSGA-II algorithm is used to perform Pareto frontier optimization on the normalized energy efficiency target, comfort target and frost layer control target, and the optimal weight combination is solved, so that the system achieves a good balance between passenger compartment comfort, defrosting stability and system energy consumption, solving the multi-objective conflict problem.

[0020] (3) With predictability and dynamic adaptability: PI-DQN model (step S2) adopts double network structure and experience replay mechanism, through closed-loop learning process (a-e in step S5), can predict future frost layer growth trend based on historical data and real-time state. The joint simulation architecture (step S3) realizes real-time data interaction between the vehicle-mounted heat pump air conditioner dynamic simulation model and the PI-DQN model, so that the model can adjust the control parameters in time according to the dynamically changing environmental conditions (such as changes in vehicle speed, fluctuations in environmental temperature and humidity), dynamically adapt to environmental changes, and avoid the limitations of traditional fixed threshold or time control methods that cannot adapt to dynamic working conditions.

[0021] (4) Improve the reliability and accuracy of the control strategy: The calculation of the frost layer state characteristics (step S4) is based on the measured temperature and the preset temperature at multiple positions on the evaporator surface, which can accurately reflect the frosting degree of the evaporator surface and provide accurate input for the model. The loss function of the PI-DQN model includes a data fitting term and a physical constraint term. The data fitting term ensures the fitting effect of the model on historical data, and the physical constraint term ensures that the model conforms to the physical law, avoiding the "reward trap", so that the trained control strategy is more reliable and accurate. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The Markov decision process diagram in the embodiments of the present application.

[0023] Figure 2 The deep reinforcement learning principle diagram in the embodiments of the present application.

[0024] Figure 3 The physical information enhanced deep Q network (PI-DQN) flowchart in the embodiments of the present application. DETAILED DESCRIPTION

[0025] The technical solutions of the present application will be described in detail below with specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] The present application relates to a physical information enhanced deep Q network (PI-DQN) and joint simulation defrosting startup adaptive control method, which is suitable for determining the optimal defrosting startup time of the vehicle-mounted heat pump air conditioner outdoor heat exchanger of the pure electric vehicle, and the specific control method mainly includes the following steps:

[0027] S1, constructing a vehicle-mounted heat pump air conditioner dynamic simulation model

[0028] Based on the thermodynamic principle of heat pump air conditioning system, a dynamic simulation model of vehicle heat pump air conditioning system including the frosting and defrosting process of evaporator was established in AMESim. The physical equations of frost growth and melting were integrated into the evaporator module, covering the effects of frost thermal resistance, mass transfer and phase change. The model parameters include compressor efficiency, refrigerant flow, ambient temperature and humidity, etc. After calibration, the error between the simulation results and the measured data is less than 5%.

[0029] S2, create PI-DQN model

[0030] The PI-DQN model including input layer, hidden layer (fully connected neural network) and output layer was constructed to generate control actions according to state input.

[0031] S3, configure joint simulation interface

[0032] AMESim was used as the master model and Python as the slave model to design the master-slave architecture for real-time data interaction through shared memory. The DYNCOSIMSHM01 module in AMESim was used as the communication interface, and parameters such as shared memory name, port number and sampling time were configured.

[0033] AMESim master model configuration:

[0034] Insert the DYNCOSIMSHM01 module and set it to Master mode with shared memory identifier thermal_control_shm and sampling time 1 ms.

[0035] Input port (25 dimensions):

[0036] Sensor data (18 dimensions): evaporator temperature (T evap ), condenser temperature (T cond ), compressor speed (N comp ), electronic expansion valve opening (% EXV ), battery temperature (T battery ), ambient temperature (T ambient ), ambient humidity (RH ambient ), evaporator outlet temperature (T evap_out ), evaporator inlet temperature (T evap_in ), battery cold plate temperature (T battery_p ), evaporator inlet refrigerant side pressure (P refrigerant_in ), evaporator outlet refrigerant side pressure (P refrigerant_out ), fan speed (N fan ), condenser outlet temperature (T cond_out ), outdoor heat exchanger surface temperature 1 (T evap1 ), outdoor heat exchanger surface temperature 2 (T evap2), outdoor heat exchanger surface temperature 3 (T evap3 ), sensor measured passenger cabin temperature (T cabin ).

[0037] Dynamic change rate (5 dimensions): Evaporation pressure change rate Condensation temperature change rate Compressor current change rate Ambient humidity change rate Vehicle acceleration (a vehicle ).

[0038] Energy efficiency index (1 dimension): COP.

[0039] Frost layer state (1 dimension): Frost coverage rate coverage .

[0040] Output port (4-dimensional control command): Compressor speed (0% ~ 100%), electronic expansion valve opening (0 ~ 100%), defrosting period length (0 ~ 30 minutes), fan speed (0 ~ 100%).

[0041] Python model configuration:

[0042] Read sensor data through shared memory and return control command to the main model.

[0043] The python shared memory interface configuration is as follows:

[0044] Define core class:

[0045] AmeSharedmem class: Implement shared memory communication with AMESim

[0046] Key methods: init(): Initialize shared memory connection, parameters include master-slave mode identification, shared memory name, input and output port number. exchange(): Execute data exchange, input is a list of sending data, return a list of received data. close(): Close the shared memory connection. Python shared memory parameter initialization:

[0047] Master-slave collaborative process

[0048] Initialize the shared memory connection, strictly follow the order of starting the AMESim master model. During the joint simulation running, read the sensor data in real time, input the PI-DQN model fused with physical information, and return the control command.

[0049] S4. Define state space, action space and design reward function according to Markov Decision Process (MDP) framework

[0050] This step is based on Markov Decision Process (MDP) framework, the schematic diagram is as Figure 1 , including state space S, action space A, transition probability P, reward function R. Among them, the state transition probability P(s'|s,a) represents the probability of transferring to state s' under state s and action a. The goal is to learn a policy π to maximize the expected cumulative reward.

[0051] 1) State Space

[0052] Contains four types of features (total of 25 dimensions): real-time sensor data, dynamic change rate, energy efficiency indicators and frost layer state features. Specifically as shown in Table 1.

[0053] Table 1 Four types of features

[0054] Among them, the frost coverage rate Frost coverage is calculated based on the evaporator surface temperature distribution (T evap1 , T evap2 , T evap3 ), the specific formula is as follows: Among them, T evap_set is the evaporator temperature set value under frost-free state; T evapi is the measured temperature of the i-th position of the evaporator surface (i=1,2,3, representing three different monitoring points); is the minimum temperature allowed on the evaporator surface; ε is a very small positive number, which is taken as 1×10 -5 here to avoid calculation errors.

[0055] 2) Action Space

[0056] The following continuous actions are normalized to [-1,1]: Compressor speed: 0% - 100% → [-1,1] Electronic expansion valve opening: 0 - 100% → [-1,1] Fan speed: 0 - 100% → [-1,1] Defrost start period: 10 - 30 minutes → [-1,1]

[0057] 3) Design Reward Function (Reward Function) and Weight Optimization

[0058] Reward function: R = a · ACOP - b · |T cabin - T cabinset - y · Frost coverage a, b, y: weights determined by Pareto frontier analysis.

[0059] The weight coefficients a, b, y in the reward function are determined by the following multi-objective optimization method:

[0060] (1) Normalized objective function

[0061] Balance the three sub-goals in the reward function: maximum energy conversion efficiency ACOP, minimize passenger cabin temperature deviation, and minimize frost layer coverage. To eliminate dimensional differences, the objective function is normalized: where, is the energy efficiency objective, is the comfort objective, is the frost control objective. ACOP max , T max , Frost max is the maximum value observed in the simulation, T max is the maximum temperature deviation observed, Frost max is the maximum frost layer coverage observed, T cabinset indicates the target temperature set for the passenger cabin. After normalization, all target values are in the interval [0, 1], with 1 indicating the best and 0 indicating the worst.

[0062] (2) Multi-objective optimization to determine weights

[0063] Based on the normalized objective function, a trade-off between the three is established, and a multi-objective optimization algorithm is used to solve the weight distribution among the three.

[0064] Decision variables: weight coefficients a, b, y, satisfying a + b + y = 1.

[0065] Objective function: maximize comprehensive performance:

[0066] Constraints: a, b, y ≥ 0 a + b + y = 1

[0067] Use the NSGA-II algorithm to iteratively calculate the multi-objective optimization problem solution set, and the specific optimization process is as follows: a). Randomly generate N individuals (population), each individual represents a set of weight combinations (a, b, y); b). According to the objective function value (comprehensive performance index), the population is non-dominated sorted (Non-dominated Sorting), and the individuals are divided into multiple frontier levels (Frontier Level). The lower the level (such as 1) indicates the better (non-dominated solution); c). In the same front layer, the crowding distance (reflecting the distribution density of individuals in the target space) of the individual is calculated, and the individuals with sparse distribution (large crowding distance) are preferentially retained to avoid local aggregation; d). Perform "selection (Selection), crossover (Crossover), mutation (Mutation)" operations to generate a new population: Selection: select parent individuals by binary tournament, and preferentially select individuals with low front level and large crowding distance; Crossover: use simulated binary crossover (Simulated Binary Crossover, SBX) to generate offspring individuals; Mutation: Perform polynomial mutation (Polynomial Mutation) on the offspring individuals to introduce diversity; e). Merge the parent population and the offspring population, and re-perform non-dominated sorting and crowding distance calculation; f). According to the population size, select the individuals with high ranking (low front level and large crowding distance) to form a new generation population; g). Continuously iterate the above steps (b-f) until the result converges (such as reaching the maximum number of iterations or the solution set is stable); h). From the final population, select all non-dominated solutions (individuals with front level 1) to form the Pareto optimal solution set; i). According to the actual engineering requirements (such as focusing on energy efficiency first, comfort first or balance), select the corresponding weight combination from the Pareto solution set.

[0068] For example, for a typical low-temperature and high-humidity working condition (such as an environment temperature of -5°C and a relative humidity of 80%), a set of Pareto optimal solution set can be obtained through the above optimization process. The implementer can select one solution that focuses on energy efficiency, and the weight coefficients are α = 0.6, β = 0.3, and γ = 0.1.

[0069] S5, training the PI-DQN model

[0070] 1. Network architecture and training parameters

[0071] Network architecture: Input layer: 25 nodes -> Hidden layer: 128 nodes (ReLU) -> Hidden layer: 128 nodes (ReLU) -> Output layer: 4 nodes (corresponding to 4 action dimensions).

[0072] Training parameters:

[0073] Optimizer: Adam, learning rate δ = 0.001.

[0074] Discount factor λ = 0.99, exploration rate ∈ initial value is 0.5, decay by 10% every 1000 steps.

[0075] Replay memory R capacity 10000, batch size 64 (τ = 0.01).

[0076] 2. Training process

[0077] The training method of the PI-DQN model includes the following steps:

[0078] (1) Initialize network parameters

[0079] Initialize the neural network parameters θ of DQN and the target network parameters θ - . The initial value of the parameters is usually set to follow a Gaussian distribution with mean 0 and variance 0.01. Two identical networks are instantiated, including the input layer, hidden layer and output layer.

[0080] (2) Select action policy

[0081] A fully connected neural network (FCNN) is used to approximate the Q value function, which maps state-action pairs to expected returns through learning (deep reinforcement learning diagram as Figure 2 ). Initialize the initial state space s, implement the epsilon-greedy exploration strategy to balance the relationship between exploring new actions and utilizing known information, the formula is P(a|s) = ∈·rand(s, a) + (1- ∈)·argmax a Q(s, a; θ) Where: P(a|s) represents the probability of selecting action a given state space s; ∈ is a very small positive number, representing the probability that the agent randomly selects an action for exploration; rand(s, a) is a random function, representing the probability value assigned to all possible actions a given state space s; argmax a Q(s, a; θ) selects the action with the maximum Q value according to the current Q value function and neural network parameters θ. This action is considered the best action under the current policy.

[0082] (3) Interaction and data storage

[0083] By epsilon-greedy policy, the action a is selected and executed, the agent learns in the environment by actually taking action and observing the result. The next state s' is obtained by feedback from the environment, and the reward R(s, a) is calculated by the reward function. The transition sample (s, a, r, s') obtained by the agent interacting with the environment at this time step is stored in the replay memory (Replay Memory).

[0084] (4) Network parameter update

[0085] Step a), calculate the current Q value

[0086] When the amount of data in the replay memory reaches or exceeds the batch size (64), a batch of sample size data (s, a, r, s') is randomly extracted and sent into the two networks, and the two networks are used to calculate the Q value. The current value network is used for the evaluation of the current policy, and the target value network is used to provide stable Q value targets. The specific steps are as follows:

[0087] Use the parameters θ of the current neural network to predict the Q value of all possible actions in the current state: Q(s, a; θ).

[0088] Calculate the maximum Q value in the new state s' as part of the target Q value, the formula is: Where: Q target (s, a) represents the target Q value; R(s, a) is the immediate reward obtained by taking action a in state s; λ is the discount factor (0≤λ≤1); θ - is the parameter of the target network, and the parameter updating speed of the target network is slower than that of the main network.

[0089] Calculate the time difference error η between the predicted Q value and the target Q value, which is part of the loss function for training the neural network, the formula is: η = Q target (s, a) - Q(s, a; θ)

[0090] Step b), calculate the data fitting term loss function The data fitting term loss function L DQN is trained with , where N is the number of samples in the batch.

[0091] Step c), calculate the physical constraint term loss function The physical constraint term is based on the law of conservation of energy and phase change thermodynamics, which describes the relationship between the frost layer change and the evaporator-dew point temperature difference, and the formula of the physical constraint term is as follows: ΔFrost_coverage = k · (T dew -T evap_out ) where ΔFrost_coverage is the frost coverage change; k is the frost / melt rate constant, measuring the influence strength of temperature difference on frost change, with unit of % / (℃·time step), whose value needs to be determined by experiment or model calibration (e.g., 0.01 can be taken initially); T dew is the dew point temperature; a t = 17.27, b t = 237.7 (empirical constants of the Magnus-Tetens formula, which are fixed parameters for calculating the dew point temperature, applicable to air-water vapor mixtures under standard atmospheric pressure).

[0092] The physical constraint term loss function is designed based on the physical constraint term, minimizing the mean square error of actual frost change and frost change predicted by the physical model, forcing the DQN network to learn a strategy that complies with the law of conservation of energy. The specific formula is as follows: where t is the tth time recorded.

[0093] Step d), calculating the total loss function The loss function includes two parts: the data fitting term loss function and the physical constraint term fitting function. The final loss function is: L(0) = L DQN + l L · L physics where l L = 0.5, which strengthens the physical constraint, improves the physical compliance of the strategy, and avoids the "reward trap".

[0094] Step e), gradient descent to update the main network parameters

[0095] The loss function serves the policy optimization of the DRL algorithm and the parameter training of the FCNN. By fusing the physical constraint term and the data fitting term, the agent (FCNN) not only fits the environmental reward (data-driven) when learning the Q value, but also meets the thermodynamic constraints (physical compliance), which can significantly improve the performance, reliability and explainability of the whole vehicle thermal management system.

[0096] The gradient descent algorithm is used to update the parameters 0 of the neural network according to the loss function:

[0097] (5) Target network update: after updating the main network parameter θ at each step of training, the target network parameter is updated according to the following formula: θ - ←τθ+(1-τ)θ - Wherein: τ is the synchronization rate, usually a small number close to 1.

[0098] (6) Iterative training

[0099] Repeat steps (2) to (5) until the training stopping condition is met (such as reaching the preset maximum training step or performance convergence).

[0100] The above physical information enhanced deep Q network (PI-DQN) algorithm flow chart is as Figure 3 .

[0101] The joint simulation verification shows that, compared with the traditional defrosting control method based on fixed threshold, the method can realize: the system COP is improved by about 15%-20%; the passenger cabin temperature deviation is stably controlled within ±1℃; the response delay to environmental parameter (such as humidity) mutation is less than 10ms under typical low-temperature and high-humidity cycle conditions. Therefore, the method can significantly improve the system energy efficiency and dynamic adaptability by joint simulation platform and physical information enhanced deep reinforcement learning collaborative optimization, and effectively solve the problems of energy efficiency loss, comfort decline and response delay in traditional defrosting control.

[0102] The above description is only a preferred embodiment of the present application, and is not intended to limit the present application in any form. Any equivalent transformation or modification according to the essence of the present application should be covered within the protection scope of the present application.

Claims

1. The defrost adaptive control method of the vehicle thermal management system based on PI-DQN and joint simulation is characterized by: The following steps are involved: Step S1, constructing a dynamic simulation model: constructing a dynamic simulation model of a vehicle heat pump air conditioner including the physical process of evaporator frosting and defrosting in AMEsim based on the thermodynamic substrate principle and the heat pump cycle principle; Step S2: Create a PI-DQN model: Construct a PI-DQN model, which includes an input layer, a hidden layer, and an output layer, and is used to generate control actions based on state input; Step S3, establishing a joint simulation interface: establishing a joint simulation architecture including the AMESim master model and the Python slave model, and realizing real-time data interaction between the vehicle heat pump air conditioning dynamic simulation model and the PI-DQN model through shared memory; Step S4: Define the state space, action space, and design the reward function: Based on the Markov decision process, define the state space, action space, and design the reward function; The state space includes four types of features acquired in real time from a dynamic simulation model of an on-board heat pump air conditioner: real-time sensor data, dynamic rate of change, energy efficiency index, and frost layer state characteristics. The frost layer state characteristics are calculated by processing the measured temperatures at multiple locations on the evaporator surface with the preset frost-free state set temperature and the minimum allowable temperature, and serve as input to the PI-DQN model. The action space includes actions for controlling the thermal management system in the dynamic simulation model of the vehicle heat pump air conditioner: compressor speed, electronic expansion valve opening, defrost cycle duration, and fan speed, which are used to control the dynamic simulation model of the vehicle heat pump air conditioner; The reward function integrates the energy efficiency index change, passenger compartment temperature deviation, and frost state characteristics calculated based on the operating state of the vehicle heat pump air conditioning dynamic simulation model, and provides environmental feedback to the PI-DQN model; Step S5, training the PI-DQN model: using PI-DQN for strategy training to learn the optimal control strategy; the loss function of the PI-DQN model includes a data fitting term loss function and a physical constraint term loss function; wherein the physical constraint term in the physical constraint term loss function is constructed based on the principles of energy conservation and phase change thermodynamics, and is used to constrain the relationship between the change in frost layer coverage and the temperature difference between the evaporator outlet temperature and the dew point temperature.

2. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: The PI-DQN model created in step S2 includes: Input layer: receives the state space; At least one hidden layer: using a fully connected neural network structure; Output layer: generates control instructions for the action space.

3. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: In step S3, in the joint simulation architecture: The AMESim main model is configured with a shared memory communication interface module, set to Master mode, and configured with the shared memory identifier, sampling time, number of input ports, and number of output ports; The Python slave model exchanges data with the AMESim master model through a shared memory interface class, which contains methods for initializing the connection, executing data exchange, and closing the connection.

4. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: In step S4, the real-time sensor data in the state space includes one or more of evaporator inlet and outlet temperatures, evaporator inlet and outlet refrigerant side pressures, compressor speed, ambient temperature, ambient humidity, battery temperature, battery cold plate temperature, fan speed, condenser outlet temperature, and passenger compartment temperature; the dynamic change rate includes one or more of evaporation pressure change rate, condensing temperature change rate, compressor current change rate, ambient humidity change rate, and vehicle acceleration; and the energy efficiency indicator is a real-time performance coefficient.

5. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: In step S4, the frost layer state characteristics include the frost layer coverage rate. The calculation formula for calculating the frost layer coverage rate based on the evaporator surface temperature distribution is: Among them, T evap_set T is the evaporator temperature setting value in the no-frost state; evapi is the measured temperature at the i-th position on the evaporator surface, where i is 1, 2, and 3, representing three different monitoring points; T evap_min The lowest temperature allowed on the evaporator surface; ε is 1×10 -5 .

6. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 5 is characterized in that: In step S4, the reward function R is specifically: R=α·ΔCOP-β·|T cabin -T cabinset |-c·Frost coverage Among them, ΔCOP is the maximum energy conversion efficiency, T cabin Indicates the passenger compartment temperature detected by the sensor, T cabinset represents the target temperature of the passenger compartment. α, β, and γ are weight coefficients determined by Pareto front analysis.

7. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 6 is characterized in that: The weight coefficients α, β, and γ in the reward function are determined by a multi-objective optimization algorithm, specifically including: (1) Using NSGA-II algorithm to normalize the energy efficiency target Comfort target and frost control targets Perform Pareto front optimization; (2) The optimal weight combination is solved through non-dominated sorting, crowding calculation, simulated binary crossover and polynomial mutation iteration.

8. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: The PI-DQN model uses a dual-network structure and an experience replay mechanism. When training the model, the following closed-loop learning process is performed: a) Action decision: The PI-DQN model adopts an ε-greedy strategy to select actions based on the input state space and acts on the dynamic simulation model constructed by S1 through the joint simulation architecture; b) Reward feedback: Calculating the reward value of the reward function based on the running results of the dynamic simulation model; c) Data storage: storing the state transition samples generated by the interaction in the experience replay mechanism; d) Network update: When the experience replay data reaches the threshold, batch data is sampled and the current Q network parameters are updated using the gradient descent algorithm using the loss function; e) Target synchronization: Regularly update the target Q network parameters in a soft update manner.

9. The vehicle thermal management system defrost adaptive control method based on PI-DQN and joint simulation according to claim 1 is characterized in that: In step S5, the formula of the loss function is: L(θ)=L DQN +λ L ·L physics Among them, L DQN is the data fitting loss function, N is the number of samples in the batch, R(s,a) is the immediate reward obtained by taking action a in state s, and λ is the discount factor (0≤λ≤1); It is the maximum estimated value of the Q value of all possible actions a′ starting from the next state s′; Q(s,a;θ) represents the Q value of all possible actions predicted in the current state using the parameters θ of the current main network; λ L Take 0.5; L physics is the physical constraint loss function, ΔFrost_coverage is the change in frost coverage, t is the tth time of recording, Frost coverage_t+1 is the frost coverage at the next moment, Frost coverage_t is the frost coverage at the forward moment.

Images