Self-adaptive control method for multi-heat-source heat management system based on reinforcement learning model predictive control

Abstract

The invention belongs to the field of thermal management systems, and provides a multi-heat-source thermal management system self-adaptive control method based on reinforcement learning model predictive control, which comprises the steps of environmental state receiving, reward function calculation, strategy optimization, decision correction, objective function optimization and control law construction, temperature error calculation, action optimization and device regulation and control. By fusing strategy optimization of the intelligent agent and action optimization in traditional model predictive control, real-time optimization of control strategies of equipment such as a fan, a water pump and a valve in the cooling system is achieved, and adaptability of reinforcement learning and stability of model predictive control are achieved; by arranging the intelligent agent and the feedback correction module, the prediction model parameters are adjusted according to the actual state of the system, and the robustness and the adaptive capacity of the control system are improved; the control law of rolling optimization output is combined with multiple times of real-time feedback and prediction correction, and efficient temperature control of the system in dynamic change is achieved.

Description

Technical Field

[0001] The present invention relates to the field of thermal management systems, and in particular to an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control. Background Art

[0002] Existing thermal management systems usually rely on traditional feedback control methods, which are difficult to cope with the dynamic changes of complex multi-heat source systems. The control methods of existing technologies mainly rely on model predictive control, which can optimize control inputs in real time by predicting future system states. However, the model predictive control method relies heavily on the system model, and may have problems such as insufficient control accuracy or excessive calculation in multi-heat source systems. In addition, although traditional MPC has been widely used in fields such as industrial control and autonomous driving, it is difficult to adapt to sudden changes in operating conditions in complex temperature control systems, especially when controlling equipment such as water pumps and fans. The flexibility and adaptability of the control strategy are limited. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, so as to achieve real-time optimization of control strategies for equipment such as fans, water pumps and valves.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] An adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, comprising:

[0006] Receive feedback output from the cooling system to obtain environmental status;

[0007] Calculating a reward function according to the environmental state to obtain a reward value;

[0008] Inputting the environment state and the reward value into a preset intelligent agent to obtain a weight coefficient operation decision;

[0009] Inputting the weight coefficient operation decision into the feedback correction module for adjustment to obtain a correction strategy;

[0010] Construct the optimization objective function, and integrate the upper and lower limit constraints of the control input, the constraints of the control increment, and the constraints of the system output to obtain the control law of the rolling optimization output;

[0011] Calculating the error between the temperature data in the environmental state and the target temperature to obtain a temperature error;

[0012] According to the temperature error, the correction strategy is optimized by using the control law to obtain an optimized strategy;

[0013] The optimization strategy is used to regulate the components in the cooling system; the components include: a control fan, a high-temperature water pump, a low-temperature water pump, a left motor valve, a right motor valve, a generator valve, and a transmission oil cooler valve.

[0014] Preferably, the reward function is:

[0015] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump );

[0016] Wherein, r(t) is the reward value; T current is the current system temperature; T target is the target temperature; ΔT is the temperature change amplitude; P fan is the fan power consumption; P pump is the power consumption of the water pump; α is the first weight coefficient; β is the second weight coefficient; γ is the third weight coefficient; δ is the fourth weight coefficient.

[0017] Preferably, the weight coefficient operation decision is:

[0018]

[0019] Among them, the value range of j is 1 to p; p is the final prediction time; j represents the jth time; θ k+j is the fifth weight coefficient; D k+j-1 is the historical data set; T is the number of historical data; p θ (a i |s i ) represents the probability of the decision under the current state; p θ′ (a i |s i ) represents the probability of decision-making under the update state; A θ′ Indicates the advantage under the current state and operation; g() is the amplitude limit function; s t is the environmental state; a t is the operating fan; ε is the noise adjustment parameter.

[0020] Preferably, the optimization objective function is:

[0021]

[0022]

[0023] Among them, y m (k+i) is the system output; y r (k+i) is the expected output; ufan (k+i|k) is the fan control input; u pump (k+i|k) is the water pump control input; u valve (k+i|k) is the valve control input; O is the length of the prediction time domain; N is the length of the control optimization time domain.

[0024] Preferably, the expressions of the upper and lower limit constraints of the control input are:

[0025] u min ≤u(k+1|k)≤u max ;

[0026] The expression of the constraint of the control increment is:

[0027] △u min ≤△u(k+1|k)≤△u max ;

[0028] The expression of the constraint output by the system is:

[0029] y min ≤k(k+1|k)≤y max ;

[0030] Among them, u min is the output lower limit; u(k+1|k) is the control output; u max is the output upper limit; △u min is the lower limit of the output increment; △u(k+1|k) is the increment of the control output; △u max is the upper limit of the output increment; y min is the lower limit of the constraint; k(k+1|k) is the system output constraint; y max is the upper limit of the constraint.

[0031] Preferably, the control law is:

[0032]

[0033] Among them, u k+1 represents the control input at the next moment; u k Represents the control input at the current moment; a i and b i are the first constraint condition parameter and the second constraint condition parameter respectively; p i is a i The effective component in the disturbance direction p; f is the adjustment parameter.

[0034] The present invention discloses the following technical effects:

[0035] The present invention provides an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control. By integrating the strategy optimization of an intelligent agent and the action optimization in traditional model predictive control, the defect of traditional methods that only use model predictive control and result in a large dependence on the system model is solved, and real-time optimization of control strategies for equipment such as fans, water pumps and valves is achieved; by setting an intelligent agent and a feedback correction module, the problem of poor applicability of traditional model predictive control is solved, and the prediction model parameters are adjusted according to the actual state of the system; by combining the control law of rolling optimization output with multiple real-time feedback and prediction correction, the defect that traditional model predictive control is difficult to adapt to sudden changes in working conditions is solved, and the system can maintain efficient temperature control in dynamic changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A schematic diagram of an adaptive control flow of a multi-heat source thermal management system provided by an embodiment of the present invention;

[0038] Figure 2 A flow chart of adaptive control of a multi-heat source thermal management system provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of a thermal management system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The purpose of the present invention is to provide an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, so as to achieve real-time optimization of control strategies for equipment such as fans, water pumps and valves.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] Figure 1A schematic diagram of an adaptive control flow of a multi-heat source thermal management system provided by an embodiment of the present invention, Figure 2 The adaptive control flow chart of the multi-heat source thermal management system provided by the embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the present invention provides an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, comprising:

[0044] Receive feedback output from the cooling system to obtain environmental status;

[0045] Calculate the reward function according to the environment state to obtain the reward value;

[0046] Input the environment state and reward value into the preset agent to obtain the weight coefficient operation decision;

[0047] The weight coefficient operation decision is input into the feedback correction module for adjustment to obtain the correction strategy;

[0048] Construct the optimization objective function, and integrate the upper and lower limit constraints of the control input, the constraints of the control increment, and the constraints of the system output to obtain the control law of the rolling optimization output;

[0049] Calculate the error between the temperature data in the environmental state and the target temperature to obtain the temperature error;

[0050] According to the temperature error, the control law is used to optimize the correction strategy to obtain the optimized strategy;

[0051] The optimization strategy is used to regulate the components in the cooling system; the components include: control fan, high-temperature water pump, low-temperature water pump, left motor valve, right motor valve, generator valve and transmission oil cooler valve.

[0052] Furthermore, the reward function is:

[0053] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump );

[0054] Among them, r(t) is the reward value; T current is the current system temperature; T target is the target temperature; ΔT is the temperature change amplitude; P fan is the fan power consumption; P pump is the power consumption of the water pump; α is the first weight coefficient; β is the second weight coefficient; γ is the third weight coefficient; δ is the fourth weight coefficient.

[0055] Specifically, the weight coefficient operation decision is:

[0056]

[0057] Among them, the value range of j is 1 to p; p is the final prediction time; j represents the jth time; θ k+j is the fifth weight coefficient; D k+j-1 is the historical data set; T is the number of historical data; p θ (a i |s i 0 represents the probability of the decision under the current state; p θ′ (a i |s i 0 represents the probability of decision in the update state; A θ′ Indicates the advantage under the current state and operation; g() is the amplitude limit function; s t is the environmental state; a t is the operating fan; ε is the noise adjustment parameter.

[0058] Furthermore, the optimization objective function is:

[0059]

[0060] Among them, y m (k+i) is the system output; y r (k+i) is the expected output; u fan (k+i|k) is the fan control input; u pump (k+i|k) is the water pump control input; u valve (k+i|k) is the valve control input; O is the length of the prediction time domain; N is the length of the control optimization time domain.

[0061] Specifically, the expressions for the upper and lower limit constraints of the control input are:

[0062] u min ≤u(k+1|k)≤u max ;

[0063] The expression for the constraint controlling the increment is:

[0064] △u min ≤△u(k+1|k)≤△u max ;

[0065] The expression of the constraint output by the system is:

[0066] y min ≤k(k+1|k)≤y max ;

[0067] Among them, u min is the output lower limit; u(k+1|k) is the control output; umax is the output upper limit; △u min is the lower limit of the output increment; △u(k+1|k) is the increment of the control output; △u max is the upper limit of the output increment; y min is the lower limit of the constraint; k(k+1|k) is the system output constraint; y max is the upper limit of the constraint.

[0068] Furthermore, the control law is:

[0069]

[0070] Among them, u k+1 represents the control input at the next moment; u k Represents the control input at the current moment; a i and b i are the first constraint condition parameter and the second constraint condition parameter respectively; p i is a i The effective component in the disturbance direction p; f is the adjustment parameter.

[0071] refer to Figure 3 The cooling system includes: an expansion water tank, a low-temperature radiator, a high-temperature radiator, a fan, an oil heat exchanger, a first water pump, a second water pump, a third water pump, an engine, a first-stage intercooler, a second-stage intercooler, a crew cabin, a heat storage module, a first two-phase valve, a second two-phase valve, a third two-phase valve, a fourth two-phase valve, a first three-phase valve, a second three-phase valve, a third three-phase valve, a first motor controller, a second motor controller, a power generation controller, a fan controller, an active suspension oil heat exchanger, a suspension motor and controller, a left motor, a right motor, a generator, a fan motor, a transmission oil heat exchanger, a first electronically controlled proportional valve, a second electronically controlled proportional valve, a third electronically controlled proportional valve, a fourth electronically controlled proportional valve, and a fifth electronically controlled proportional valve;

[0072] The inlet of the expansion water tank is connected to the low-temperature radiator and the high-temperature radiator respectively; the outlet of the expansion water tank is connected to the low-temperature radiator and the inlet of the first three-phase valve respectively; the outlet of the first three-phase valve is connected to the inlet and outlet of the high-temperature radiator respectively; the outlet of the high-temperature radiator is also connected to the first two-phase valve, the second two-phase valve, the third two-phase valve and the first water pump respectively; the outlet of the low-temperature radiator is also connected to the inlet of the second three-phase valve; the first outlet of the second three-phase valve is connected to the crew cabin; the second outlet of the second three-phase valve is connected to the inlets of the fourth two-phase valve and the third three-phase valve respectively; the crew cabin is also connected to the first two-phase valve and the heat storage module; the third water pump is arranged in the crew cabin; the heat storage module is also connected to the second two-phase valve, the third two-phase valve, the fourth two-phase valve and the third three-phase valve respectively; the third three-phase valve is also connected to the second water pump; the first water pump is also connected to the engine and the first-stage intercooler respectively; the engine and the first The intercooler is also connected to the inlet of the first three-phase valve respectively; the second water pump is also connected to the secondary intercooler; the secondary intercooler is also connected to the first motor controller, the second motor controller, the power generation controller, the fan controller, the active suspension oil heat exchanger and the suspension motor and controller respectively; the first motor controller, the left motor and the first electronically controlled proportional valve are connected in sequence; the second motor controller, the right motor and the second electronically controlled proportional valve are connected in sequence; the power generation controller, the generator and the third electronically controlled proportional valve are connected in sequence; the fan controller, the fan motor and the fourth electronically controlled proportional valve are connected in sequence; the active suspension oil heat exchanger and the suspension motor and controller are respectively connected to the transmission oil heat exchanger; the transmission oil heat exchanger is connected to the fifth electronically controlled proportional valve; the first electronically controlled proportional valve, the second electronically controlled proportional valve, the third electronically controlled proportional valve, the fourth electronically controlled proportional valve and the fifth electronically controlled proportional valve are also respectively connected to the inlet of the oil heat exchanger; the outlet of the oil heat exchanger is connected to the low-temperature radiator.

[0073] Furthermore, the thermal management coolant flows into the system from the expansion tank, and is driven by a water pump to circulate in the low-temperature and high-temperature heat exchange circuits, passing through the engine, intercooler, oil heat exchanger, transmission oil cooler and other components, taking away the heat and dissipating it to the outside. The electronically controlled proportional valve and two-phase valve in the system are used to control the flow direction and flow distribution of the coolant to ensure reasonable distribution under different working conditions. During the coolant circulation process, the heat generated by the engine is quickly dissipated through the high-temperature heat exchanger and intercooler, and the low-temperature circuit is responsible for the heat dissipation of the left and right motors, generators and transmission oil. The entire cooling system is designed with multiple circuits in parallel to achieve effective cooling of the engine and related components. This embodiment achieves multi-parameter coordinated regulation by controlling the fan, high-temperature water pump, low-temperature water pump, left motor valve, right motor valve, generator valve and transmission oil cooler valve.

[0074] Preferably, in the RL-MPC (model predictive control based on reinforcement learning) controller, the reinforcement learning structure is mainly composed of three parts: control decision, agent and reward function. First, the state, action and reward function of the reinforcement learning agent need to be defined. The state includes the state variables of the current system (such as temperature, pressure, flow) and the predicted state of MPC (model predictive control); the action is to adjust the control strategy, such as the increment of the control input or the weight parameter in the MPC optimization; the reward function is based on the control objective, considering reducing the temperature error or minimizing energy consumption. During the training process of the reinforcement learning agent, MPC will generate multiple candidate control strategies at each time step. The agent selects one of the strategies according to the current state and executes it, and then adjusts the strategy and updates the learning parameters according to the effect (reward value). In actual implementation, the RL agent is combined with the optimization process of MPC, so that the agent can dynamically select or adjust the optimization parameters when calling the optimization algorithm, thereby improving the control effect.

[0075] Furthermore, the control method comprises the following steps:

[0076] 1) Data collection and status prediction

[0077] The system collects current temperature data, power, fan, water pump and other control variables, and predicts future temperature changes through MPC.

[0078] 2) Action Optimization

[0079] The minimum value is calculated by the optimization algorithm. The optimization objective function uses a quadratic expression to better describe the relationship between temperature error and energy consumption. The typical form of the objective function is as follows:

[0080]

[0081] This objective function not only takes into account the temperature control error, but also the control action amplitude of the fan and water pump, thereby reducing energy consumption while maintaining system stability.

[0082] Preferably, in order to obtain the optimal control sequence of the MPC controller, this embodiment calculates the minimum value of the optimization objective function of the formula and introduces a series of constraints to ensure that the control input changes smoothly and prevent the system from experiencing drastic fluctuations. The constraints are as follows:

[0083] Upper and lower limit constraints of control input: Fan speed, pump flow and valve opening need to be subject to physical restrictions and must meet certain upper and lower limit conditions. This constraint ensures that the fan and pump will not run at overspeed and the valve opening will not exceed the design range, ensuring the safe operation of the system.

[0084] Constraints on control increments: To avoid drastic changes in fan speed, pump flow, and valve opening in a short period of time, limits on control increments are set. This constraint helps achieve smooth control under fast dynamic conditions and avoids system instability due to sudden input changes.

[0085] System output constraints: The output of the control system (such as temperature, pressure, etc.) also needs to meet certain range requirements. This constraint ensures that the cooling system can maintain a safe and designed operating range in each working state to prevent problems such as overheating, overcooling or excessive pressure. The public notice is as follows:

[0086] Specifically, the constraints are announced as follows:

[0087] u min ≤u(k+1|k)≤u max

[0088] △u min ≤△u(k+1|k)≤△u max

[0089] y min ≤k(k+1|k)≤y max

[0090] Since the given quadratic objective function is a nonlinear function, it needs to be converted into a QP problem based on the numerical method for optimal value calculation. The optimal value calculation method of the QP problem based on the numerical method is implemented by the method based on the effective set method. The MPC fast optimization method based on the effective set method has the following steps:

[0091] Parameter initialization: Set the initial feasible point u for control inputs such as fan speed, pump flow and valve opening k After that, given the initial working set w k .

[0092] Iteration point u k Update calculation: Assume a disturbance variable (temperature, pressure, etc.) as the feasible point u for the next iteration k direction, and assume that the corresponding Lagrange multiplier Satisfy the formula

[0093] If the disturbance variable p is 0, then the feasible point u k It can be calculated by constructing the least squares expression corresponding to the effective set method:

[0094]

[0095] When k ≥0, then the feasible point uk This is the optimal solution. k <0, at this time, let i k ∈w k ∩I, so that the Lagrange multiplier Satisfy the formula:

[0096]

[0097] The guaranteed Lagrangian operator can be obtained by formula 10 The value of reaches the minimum complex number constraint set, then take w k+1 =w k / {i k}, that is, remove the constraint of the most negative Lagrange multiplier in the effective working set, so as to obtain the effective constraint set, and then obtain the feasible point u k .

[0098] If the disturbance variable p is not equal to 0, then the closest feasible point u can be searched near the direction of p. k + as the next iteration point u k+1 , that is, =ap, where the parameter a takes the following values:

[0099]

[0100] Indicates the current control input u i The remaining distance to the constraint boundary, Indicates the degree of influence on the constraint along the direction of the disturbance variable p.

[0101] Then we can finally get the control law output of rolling optimization output:

[0102]

[0103] Preferably, this control law can ensure that the inputs of fans, pumps, and valves are in the optimal state at every moment, and enable the system to adjust adaptively in a changing environment. Finally, the optimal control law output is applied to the control object, and the calculated optimal control law is applied to the fans, pumps, and valves in the cooling system to achieve dynamic adaptive control of the system. Through this control sequence, the system can flexibly respond to various environmental changes under different working conditions, ensure that indicators such as temperature and pressure are maintained within the ideal range, and effectively improve the energy efficiency and stability of the entire system.

[0104] 3) Reinforcement learning and feedback correction

[0105] The reinforcement learning module adjusts the control strategy according to the system status information, and corrects the prediction model in real time according to the feedback correction module. The prediction model control algorithm based on reinforcement learning is used for adaptive control of fans, pumps and valves, especially in the process of thermal load changes of power units, to achieve intelligent optimization control of the cooling system. Through the interaction between the agent and the environment, the control system can adaptively adjust the control strategy under different working conditions to ensure the stability and response speed of the system operation.

[0106] Furthermore, the principle of the prediction model based on reinforcement learning is as follows: Assume that at time t, the agent of the reinforcement learning module receives the feedback output s(t) from the cooling system, that is, the environmental state. Then the agent makes corresponding P control decisions a(t), a(t+1), ..., a(t+P-1) as the prediction control output for the next P steps. These decisions will be used to adjust the fan speed, water pump flow and valve opening to ensure that the cooling system can cope with the changing heat load.

[0107] Specifically, the following will give the implementation process steps of the control algorithm based on the reinforcement learning prediction model based on the basic principles of the reinforcement learning prediction model:

[0108] Agent obtains reward value: In order to evaluate the control effect, the agent obtains the reward value r(t) through the reward function.

[0109] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump )

[0110] In the process of heat load change, in order to ensure that the cooling system can operate safely and stably, the controller must meet the requirements of low overshoot and fast control convergence speed. Therefore, parameter x1 represents the overshoot of the controller; x E1 represents the expected overshoot of the controller; λ1 represents the adjustment value of the overshoot of the controller; x2 represents the convergence time of the controller; x E2 represents the expected convergence time of the controller; λ2 represents the adjustment value of the convergence time of the controller. When the control performance of the cooling system is close to the expected value, the reward value is close to 1 and is modified to the above description.

[0111] Furthermore, when the power unit is tracking the thermal load, if the cooling system components can achieve trajectory tracking, then the variables x1 and x2 are close to the expected overshoot and convergence time, and the reward value will be close to the maximum value 1. If the control stability of the power unit is poor and the change is slow during the thermal load change, then the variables x1 and x2 are far away from the expected overshoot and convergence time, and the reward value will be close to the minimum value 0. Therefore, the reward value can reflect the control performance of the actual control system.

[0112] Preferably, the agent uses the reinforcement learning algorithm adopted by the agent, such as the PPO (Proximal Policy Optimization) algorithm. Assume that the weighted decision strategy made by the agent to control the fan, pump and valve is a(t). The corresponding environmental state s(t) and the decision a(t) made at the past historical time T constitute the strategy trajectory sequence D. k for:

[0113] D k ={[s(1),a(1)],...,[s(T),a(T)]}

[0114] The operation decision parameters for the next moment can be expressed as:

[0115]

[0116] Where T is the number of historical data; s t is the environmental state; a t The weight decision for operating fans, pumps and valves; function A θ′ Indicates the advantage under the current state and operation; g(ε,A θ′ (s t ,a t )) function represents the amplitude limiting function, so that A θ′ The value range of is between 1-ε and 1+ε; It represents the probability ratio of making a decision in the current state to making a decision in the new state.

[0117] Furthermore, the operation decision predicted at time P in the future can be expressed as:

[0118]

[0119] …

[0120]

[0121] A θ ′(s t ,a t)g(ε,A θk+j-1 (s t ,a t )))

[0122] …

[0123]

[0124] A θ ′(s t ,a t )g(ε,A θk+p-1 (s t ,a t )))

[0125] These decisions are in turn used to adjust fan speeds, pump flows, and valve openings to ensure the cooling system remains stable under changing heat load conditions.

[0126] Furthermore, the predicted control decision parameters are input into the feedback correction module for adjustment to further optimize the control of fans, pumps and valves to ensure the overall stability and efficiency of the cooling system. This control algorithm based on reinforcement learning enables the system to intelligently adapt to complex working conditions and achieve efficient and stable operation.

[0127] 4) Actuator control

[0128] Based on the optimization results, the controller adjusts the fan speed, water pump flow and valve opening in real time to ensure the stability and energy saving of temperature control.

[0129] The beneficial effects of the present invention are as follows:

[0130] The present invention realizes real-time optimization of control strategies for fans, water pumps, valves and other equipment in the cooling system by integrating the strategy optimization of intelligent agents and the action optimization in traditional model predictive control, and has the adaptability of reinforcement learning and the stability of model predictive control; by setting up intelligent agents and feedback correction modules, it realizes adjustment of prediction model parameters according to the actual state of the system, thereby improving the robustness and adaptability of the control system; by combining the control law of rolling optimization output with multiple real-time feedback and prediction correction, it realizes efficient temperature control of the system in dynamic changes.

[0131] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0132] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, characterized in that: include: Receive feedback output from the cooling system to obtain environmental status; Calculating a reward function according to the environmental state to obtain a reward value; Inputting the environment state and the reward value into a preset intelligent agent to obtain a weight coefficient operation decision; Inputting the weight coefficient operation decision into the feedback correction module for adjustment to obtain a correction strategy; Construct the optimization objective function, and integrate the upper and lower limit constraints of the control input, the constraints of the control increment, and the constraints of the system output to obtain the control law of the rolling optimization output; Calculating the error between the temperature data in the environmental state and the target temperature to obtain a temperature error; According to the temperature error, the correction strategy is optimized by using the control law to obtain an optimized strategy; The optimization strategy is used to regulate the components in the cooling system; the components include: a control fan, a high-temperature water pump, a low-temperature water pump, a left motor valve, a right motor valve, a generator valve, and a transmission oil cooler valve.

2. According to claim 1, a multi-heat source thermal management system adaptive control method based on reinforcement learning model predictive control is characterized in that: The reward function is: r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump ); Wherein, r(t) is the reward value; T current is the current system temperature; T target is the target temperature; ΔT is the temperature change amplitude; P fan is the fan power consumption; P pump is the power consumption of the water pump; α is the first weight coefficient; β is the second weight coefficient; γ is the third weight coefficient; δ is the fourth weight coefficient.

3. The adaptive control method of a multi-heat source thermal management system based on reinforcement learning model predictive control according to claim 1, characterized in that: The weight coefficient operation decision is: Among them, the value range of j is 1 to p; p is the final prediction time; j represents the jth time; θ k+j is the fifth weight coefficient; D k+j-1 is the historical data set; T is the number of historical data; p θ (a i |s i ) represents the probability of the decision under the current state; p θ′ (a i |s i ) represents the probability of decision-making under the update state; A θ′ Indicates the advantage under the current state and operation; g() is the amplitude limit function; s t is the environmental state; a t is the operating fan; ε is the noise adjustment parameter.

4. The adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control according to claim 1, characterized in that: The optimization objective function is: Among them, y m (k+i) is the system output; y r (k+i) is the expected output; u fan (k+i|k) is the fan control input; u pump (k+i|k) is the water pump control input; u valve (k+i|k) is the valve control input; O is the length of the prediction time domain; N is the length of the control optimization time domain.

5. The adaptive control method of a multi-heat source thermal management system based on reinforcement learning model predictive control according to claim 1, characterized in that: The expressions of the upper and lower limit constraints of the control input are: you min ≤u(k+1|k)≤u max ; The expression of the control increment constraint is: △u min ≤△u(k+1|k)≤△u max ; The expression of the constraint output by the system is: y min ≤k(k+1|k)≤y max ; Among them, u min is the output lower limit; u(k+1|k) is the control output; u max is the output upper limit; △u min is the lower limit of the output increment; △u(k+1|k) is the increment of the control output; △u max is the upper limit of the output increment; y min is the lower limit of the constraint; k(k+1|k) is the system output constraint; y max is the upper limit of the constraint.

6. The adaptive control method of a multi-heat source thermal management system based on reinforcement learning model predictive control according to claim 1, characterized in that: The control law is: Among them, u k+1 represents the control input at the next moment; u k Represents the control input at the current moment; a i and b i are the first constraint condition parameter and the second constraint condition parameter respectively; p i is a i The effective component in the disturbance direction p; f is the adjustment parameter.

Images