An adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control

Abstract

The present invention belongs to the field of thermal management systems and provides an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, including: receiving environmental state, reward function calculation, strategy optimization, decision correction, optimization objective function and control law construction, temperature error calculation, action optimization, and device regulation. By integrating the strategy optimization of an intelligent agent and the action optimization in traditional model predictive control, the present invention realizes real-time optimization of the control strategy for equipment such as fans, water pumps, and valves in the cooling system, and has the adaptability of reinforcement learning and the stability of model predictive control; by setting an intelligent agent and feedback correction module, it realizes adjustment of the prediction model parameters according to the actual state of the system, improving the robustness and adaptability of the control system; by combining the control law output by rolling optimization with multiple real-time feedback and prediction correction, it realizes that the system maintains efficient temperature control in dynamic changes.

Description

Technical Field

[0001] The present invention relates to the field of thermal management systems, and in particular to an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control. Background Art

[0002] Existing thermal management systems typically rely on traditional feedback control methods, which are difficult to cope with the dynamic changes of complex multi-heat source systems. The control methods of existing technologies mainly rely on model predictive control, which can optimize control inputs in real time by predicting future system states. However, model predictive control methods are highly dependent on system models, and may cause problems such as insufficient control accuracy or excessive computational complexity in multi-heat source systems. In addition, although traditional MPC has been widely used in fields such as industrial control and autonomous driving, it is difficult to adapt to sudden changes in operating conditions in complex temperature control systems. In particular, when controlling equipment such as water pumps and fans, the flexibility and adaptability of the control strategy are limited. Summary of the Invention

[0003] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, so as to realize real-time optimization of the control strategies of equipment such as fans, water pumps and valves.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] An adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, comprising:

[0006] Receive feedback output from the cooling system to obtain environmental status;

[0007] Calculating a reward function according to the environmental state to obtain a reward value;

[0008] Inputting the environmental state and the reward value into a preset intelligent agent to obtain a weight coefficient operation decision;

[0009] Inputting the weight coefficient operation decision into the feedback correction module for adjustment to obtain a correction strategy;

[0010] Construct an optimization objective function and integrate the upper and lower limit constraints of the control input, the control increment constraints, and the system output constraints to obtain the control law of the rolling optimization output;

[0011] Calculating an error between the temperature data in the environmental state and the target temperature to obtain a temperature error;

[0012] According to the temperature error, the correction strategy is optimized using the control law to obtain an optimized strategy;

[0013] The optimization strategy is used to regulate the components in the cooling system; the components include: a control fan, a high-temperature water pump, a low-temperature water pump, a left motor valve, a right motor valve, a generator valve, and a transmission oil cooler valve.

[0014] Preferably, the reward function is:

[0015] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump );

[0016] Where r(t) is the reward value; T current is the current system temperature; T target is the target temperature; ΔT is the temperature change amplitude; P fan is the fan power consumption; P pump is the water pump power consumption; α is the first weight coefficient; β is the second weight coefficient; γ is the third weight coefficient; δ is the fourth weight coefficient.

[0017] Preferably, the weight coefficient operation decision is:

[0018]

[0019] Among them, the value range of j is 1 to p; p is the final prediction time; j represents the jth moment; θ k+j is the fifth weight coefficient; D k+j-1 is the historical data set; T is the number of historical data; p θ (a i |s i ) represents the probability of the decision under the current state; p θ′ (a i |s i ) represents the probability of decision-making under the updated state; A θ′ Indicates the advantage under the current state and operation; g() is the amplitude limit function; s t is the environmental state; a t is the operating fan; ε is the noise adjustment parameter.

[0020] Preferably, the optimization objective function is:

[0021]

[0022]

[0023] Among them, y m (k+i) is the system output; y r (k+i) is the expected output; ufan (k+i|k) is the fan control input; u pump (k+i|k) is the water pump control input; u valve (k+i|k) is the valve control input; O is the length of the prediction time domain; N is the length of the control optimization time domain.

[0024] Preferably, the upper and lower limit constraints of the control input are expressed as follows:

[0025] u min ≤u(k+1|k)≤u max ;

[0026] The expression of the control increment constraint is:

[0027] △u min ≤△u(k+1|k)≤△u max ;

[0028] The expression of the constraint output by the system is:

[0029] y min ≤k(k+1|k)≤y max ;

[0030] Among them, u min is the output lower limit; u(k+1|k) is the control output; u max is the output upper limit; △u min is the lower limit of the output increment; △u(k+1|k) is the increment of the control output; △u max is the upper limit of the output increment; y min is the lower limit of the constraint; k(k+1|k) is the system output constraint; y max is the upper limit of the constraint.

[0031] Preferably, the control law is:

[0032]

[0033] Among them, u k+1 Indicates the control input at the next moment; u k Represents the control input at the current moment; a i and b i are the first constraint condition parameter and the second constraint condition parameter respectively; p i is a i The effective component in the disturbance direction p; f is the adjustment parameter.

[0034] The present invention discloses the following technical effects:

[0035] The present invention provides an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control. By integrating the strategy optimization of an intelligent agent and the action optimization in traditional model predictive control, the method solves the defect that the traditional method only uses model predictive control, resulting in a large dependence on the system model, and realizes real-time optimization of the control strategy for equipment such as fans, water pumps and valves; by setting an intelligent agent and a feedback correction module, the method solves the problem of poor applicability of traditional model predictive control, and realizes adjustment of the prediction model parameters according to the actual state of the system; by combining the control law output by rolling optimization with multiple real-time feedback and prediction correction, the method solves the defect that traditional model predictive control is difficult to adapt to sudden changes in working conditions, and realizes that the system maintains efficient temperature control in dynamic changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of the adaptive control flow of a multi-heat source thermal management system provided by an embodiment of the present invention;

[0038] Figure 2 A flow chart of adaptive control of a multi-heat source thermal management system provided by an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of a thermal management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] The purpose of the present invention is to provide an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, so as to achieve real-time optimization of control strategies for equipment such as fans, water pumps and valves.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] Figure 1Schematic diagram of the adaptive control flow of a multi-heat source thermal management system provided by an embodiment of the present invention, Figure 2 The adaptive control flow chart of the multi-heat source thermal management system provided by the embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the present invention provides an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, comprising:

[0044] Receive feedback output from the cooling system to obtain environmental status;

[0045] Calculate the reward function according to the environment state to obtain the reward value;

[0046] Input the environment state and reward value into the preset intelligent agent to obtain the weight coefficient operation decision;

[0047] The weight coefficient operation decision is input into the feedback correction module for adjustment to obtain the correction strategy;

[0048] Construct an optimization objective function and integrate the upper and lower limit constraints of the control input, the control increment constraints, and the system output constraints to obtain the control law of the rolling optimization output;

[0049] Calculate the error between the temperature data in the ambient state and the target temperature to obtain the temperature error;

[0050] According to the temperature error, the control law is used to optimize the correction strategy to obtain the optimized strategy;

[0051] The optimization strategy is used to regulate the components in the cooling system; the components include: control fan, high-temperature water pump, low-temperature water pump, left motor valve, right motor valve, generator valve and transmission oil cooler valve.

[0052] Furthermore, the reward function is:

[0053] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump );

[0054] Among them, r(t) is the reward value; T current is the current system temperature; T target is the target temperature; ΔT is the temperature change amplitude; P fan is the fan power consumption; P pump is the water pump power consumption; α is the first weight coefficient; β is the second weight coefficient; γ is the third weight coefficient; δ is the fourth weight coefficient.

[0055] Specifically, the weight coefficient operation decision is:

[0056]

[0057] Among them, the value range of j is 1 to p; p is the final prediction time; j represents the jth moment; θ k+j is the fifth weight coefficient; D k+j-1 is the historical data set; T is the number of historical data; p θ (a i |s i 0 represents the probability of the decision under the current state; p θ′ (a i |s i 0 represents the probability of the decision under the updated state; A θ′ Indicates the advantage under the current state and operation; g() is the amplitude limit function; s t is the environmental state; a t is the operating fan; ε is the noise adjustment parameter.

[0058] Furthermore, the optimization objective function is:

[0059]

[0060] Among them, y m (k+i) is the system output; y r (k+i) is the expected output; u fan (k+i|k) is the fan control input; u pump (k+i|k) is the water pump control input; u valve (k+i|k) is the valve control input; O is the length of the prediction time domain; N is the length of the control optimization time domain.

[0061] Specifically, the expressions for the upper and lower limit constraints of the control input are:

[0062] u min ≤u(k+1|k)≤u max ;

[0063] The expression for the constraint controlling the increment is:

[0064] △u min ≤△u(k+1|k)≤△u max ;

[0065] The expression of the constraint output by the system is:

[0066] y min ≤k(k+1|k)≤y max ;

[0067] Among them, u min is the output lower limit; u(k+1|k) is the control output; umax is the output upper limit; △u min is the lower limit of the output increment; △u(k+1|k) is the increment of the control output; △u max is the upper limit of the output increment; y min is the lower limit of the constraint; k(k+1|k) is the system output constraint; y max is the upper limit of the constraint.

[0068] Furthermore, the control law is:

[0069]

[0070] Among them, u k+1 Indicates the control input at the next moment; u k Represents the control input at the current moment; a i and b i are the first constraint condition parameter and the second constraint condition parameter respectively; p i is a i The effective component in the disturbance direction p; f is the adjustment parameter.

[0071] refer to Figure 3 The cooling system includes: an expansion tank, a low-temperature radiator, a high-temperature radiator, a fan, an oil heat exchanger, a first water pump, a second water pump, a third water pump, an engine, a first-stage intercooler, a second-stage intercooler, a crew cabin, a heat storage module, a first two-phase valve, a second two-phase valve, a third two-phase valve, a fourth two-phase valve, a first three-phase valve, a second three-phase valve, a third three-phase valve, a first motor controller, a second motor controller, a power generation controller, a fan controller, an active suspension oil heat exchanger, a suspension motor and controller, a left motor, a right motor, a generator, a fan motor, a transmission oil heat exchanger, a first electronically controlled proportional valve, a second electronically controlled proportional valve, a third electronically controlled proportional valve, a fourth electronically controlled proportional valve, and a fifth electronically controlled proportional valve;

[0072] The inlet of the expansion water tank is connected to the low-temperature radiator and the high-temperature radiator respectively; the outlet of the expansion water tank is connected to the low-temperature radiator and the inlet of the first three-phase valve respectively; the outlet of the first three-phase valve is connected to the inlet and outlet of the high-temperature radiator respectively; the outlet of the high-temperature radiator is also connected to the first two-phase valve, the second two-phase valve, the third two-phase valve and the first water pump respectively; the outlet of the low-temperature radiator is also connected to the inlet of the second three-phase valve; the first outlet of the second three-phase valve is connected to the crew cabin; the second outlet of the second three-phase valve is connected to the inlets of the fourth two-phase valve and the third three-phase valve respectively; the crew cabin is also connected to the first two-phase valve and the heat storage module; the third water pump is arranged in the crew cabin; the heat storage module is also connected to the second two-phase valve, the third two-phase valve, the fourth two-phase valve and the third three-phase valve respectively; the third three-phase valve is also connected to the second water pump; the first water pump is also connected to the engine and the first-stage intercooler respectively; the engine and the first The intercooler is also connected to the inlet of the first three-phase valve respectively; the second water pump is also connected to the secondary intercooler; the secondary intercooler is also connected to the first motor controller, the second motor controller, the power generation controller, the fan controller, the active suspension oil heat exchanger and the suspension motor and controller respectively; the first motor controller, the left motor and the first electronically controlled proportional valve are connected in sequence; the second motor controller, the right motor and the second electronically controlled proportional valve are connected in sequence; the power generation controller, the generator and the third electronically controlled proportional valve are connected in sequence; the fan controller, the fan motor and the fourth electronically controlled proportional valve are connected in sequence; the active suspension oil heat exchanger and the suspension motor and controller are respectively connected to the transmission oil heat exchanger; the transmission oil heat exchanger is connected to the fifth electronically controlled proportional valve; the first electronically controlled proportional valve, the second electronically controlled proportional valve, the third electronically controlled proportional valve, the fourth electronically controlled proportional valve and the fifth electronically controlled proportional valve are also respectively connected to the inlet of the oil heat exchanger; the outlet of the oil heat exchanger is connected to the low-temperature radiator.

[0073] Furthermore, the thermal management coolant flows into the system from the expansion tank, and is driven by a water pump to circulate in the low-temperature and high-temperature heat exchange circuits, passing through the engine, intercooler, oil heat exchanger, transmission oil cooler and other components, taking away the heat and dissipating it to the outside. The electronically controlled proportional valve and two-phase valve in the system are used to control the flow direction and flow distribution of the coolant to ensure reasonable distribution under different working conditions. During the coolant circulation process, the heat generated by the engine is quickly dissipated through the high-temperature heat exchanger and intercooler, and the low-temperature circuit is responsible for the heat dissipation of the left and right motors, generators and transmission oil. The entire cooling system achieves effective cooling of the engine and related components through the parallel design of multiple circuits. This embodiment realizes multi-parameter coordinated regulation by controlling the fan, high-temperature water pump, low-temperature water pump, left motor valve, right motor valve, generator valve and transmission oil cooler valve.

[0074] Preferably, in an RL-MPC (reinforcement learning-based model predictive control) controller, the reinforcement learning structure primarily consists of three components: control decision-making, the agent, and the reward function. First, the state, actions, and reward function of the reinforcement learning agent must be defined. The state includes the current system state variables (such as temperature, pressure, and flow) and the predicted state of the MPC (model predictive control); the action is to adjust the control strategy, such as the increment of the control input or the weight parameter in the MPC optimization; and the reward function is based on the control objective, considering reducing temperature error or minimizing energy consumption. During the training process of the reinforcement learning agent, the MPC generates multiple candidate control strategies at each time step. The agent selects and executes one of these strategies based on the current state, then adjusts the strategy and updates the learning parameters based on the results (reward value). In practical implementation, the RL agent is combined with the MPC optimization process, allowing the agent to dynamically select or adjust optimization parameters when calling the optimization algorithm, thereby improving control effectiveness.

[0075] Furthermore, the control method comprises the following steps:

[0076] 1) Data collection and status prediction

[0077] The system collects current temperature data, power, fan and water pump control variables, and predicts future temperature changes through MPC.

[0078] 2) Action optimization

[0079] The minimum value is calculated through the optimization algorithm. The optimization objective function uses a quadratic expression to better describe the relationship between temperature error and energy consumption. The typical form of the objective function is as follows:

[0080]

[0081] This objective function not only considers the temperature control error, but also the control action amplitude of the fan and water pump, thereby reducing energy consumption while maintaining system stability.

[0082] Preferably, in order to obtain the optimal control sequence of the MPC controller, this embodiment performs a minimum calculation on the optimization objective function of the formula and introduces a series of constraints to ensure that the control input changes smoothly and prevent the system from experiencing severe fluctuations. The constraints are as follows:

[0083] Upper and lower limit constraints on control inputs: Fan speed, pump flow, and valve opening are subject to physical constraints and must meet certain upper and lower limits. This constraint ensures that fans and pumps do not overspeed and valve openings do not exceed the designed range, ensuring safe system operation.

[0084] Control increment constraints: To prevent drastic changes in fan speed, pump flow, and valve opening within a short period of time, control increment limits are set. This constraint helps achieve smooth control under fast dynamic conditions and avoids system instability caused by sudden input changes.

[0085] System output constraints: The output of the control system (such as temperature, pressure, etc.) must also meet certain range requirements. This constraint ensures that the cooling system can maintain a safe and designed operating range in all working conditions to prevent overheating, overcooling or excessive pressure. The public notice is as follows:

[0086] Specifically, the constraints are announced as follows:

[0087] u min ≤u(k+1|k)≤u max

[0088] △u min ≤△u(k+1|k)≤△u max

[0089] y min ≤k(k+1|k)≤y max

[0090] Since the given quadratic objective function is a nonlinear function, it needs to be converted into a numerical QP problem for optimal value calculation. The optimal value calculation method of the numerical QP problem is implemented using a method based on the active set method. The MPC fast optimization method based on the active set method has the following steps:

[0091] Parameter initialization: Set the initial feasible point u for the control inputs such as fan speed, pump flow and valve opening k After that, given the initial working set w k .

[0092] Iteration point u k Update calculation: Assume a disturbance variable (temperature, pressure...) as the feasible point u for the next iteration k direction, and assuming that the corresponding Lagrange multiplier Satisfy the formula

[0093] If the disturbance variable p is 0, then the feasible point u k It can be calculated by constructing the least squares expression corresponding to the effective set method:

[0094]

[0095] When λ k ≥0, then the feasible point uk This is the optimal solution. k <0, at this time, let i k ∈w k ∩I, so that the Lagrange multiplier Satisfies the formula:

[0096]

[0097] The guaranteed Lagrangian operator can be obtained by formula 10 The value of reaches the minimum complex number constraint set, then take w k+1 =w k / {i k}, that is, remove the constraint of the most negative Lagrange multiplier in the effective working set, thereby obtaining the effective constraint set, and then obtain the feasible point u k .

[0098] If the perturbation variable p is not equal to 0, then the closest feasible point u can be searched near the direction of p. k + as the next iteration point u k+1 , that is, =ap, where the parameter a takes the following values:

[0099]

[0100] Indicates the current control input u i The remaining distance to the constraint boundary, Indicates the degree of influence on the constraint along the direction of the disturbance variable p.

[0101] Then we can finally get the control law output of the rolling optimization output:

[0102]

[0103] Optimally, this control law ensures that the inputs to fans, pumps, and valves are always optimal, enabling the system to adapt to changing conditions. Finally, the optimal control law is applied to the controlled object, and the calculated optimal control law is applied to the fans, pumps, and valves in the cooling system, achieving dynamic adaptive control of the system. This control sequence allows the system to flexibly respond to various environmental changes under different operating conditions, ensuring that indicators such as temperature and pressure remain within ideal ranges, and effectively improving the energy efficiency and stability of the entire system.

[0104] 3) Reinforcement learning and feedback correction

[0105] The reinforcement learning module adjusts the control strategy based on system status information and, through the feedback correction module, corrects the predictive model in real time. The predictive model control algorithm based on reinforcement learning is used for adaptive control of fans, pumps, and valves, particularly to achieve intelligent optimization of the cooling system during changes in the power unit's thermal load. Through the interaction between the agent and the environment, the control system can adaptively adjust the control strategy under different operating conditions to ensure system stability and responsiveness.

[0106] The reinforcement learning-based predictive model works as follows: Assume that at time t, the reinforcement learning agent receives feedback output s(t) from the cooling system, representing the environmental state. The agent then makes P corresponding control decisions a(t), a(t+1), ..., a(t+P-1) as the predictive control outputs for the next P steps. These decisions are used to adjust fan speed, pump flow, and valve opening, ensuring that the cooling system can cope with the changing heat load.

[0107] Specifically, the following will give the implementation process steps of the control algorithm based on the reinforcement learning prediction model based on the basic principles of the reinforcement learning prediction model:

[0108] The agent obtains a reward value: In order to evaluate the control effect, the agent obtains a reward value r(t) through the reward function.

[0109] r(t)=-(α·|T current -T target |+β·|ΔT|+γ·P fan +δ·P pump )

[0110] In order to ensure the safe and stable operation of the cooling system during the heat load change, the controller must meet the requirements of low overshoot and fast control convergence speed. Therefore, the parameter x1 represents the overshoot of the controller; x E1 represents the expected overshoot of the controller; λ1 represents the adjustment value of the overshoot of the controller; x2 represents the convergence time of the controller; x E2 represents the expected convergence time of the controller; λ2 represents the adjustment value of the convergence time of the controller. When the control performance of the cooling system is close to the expected value, the reward value is close to 1 and is modified to the above description.

[0111] Furthermore, during the power plant thermal load tracking process, if the cooling system components can achieve trajectory tracking, then the variables x1 and x2 are close to the desired overshoot and convergence time, and the reward value will be close to the maximum value of 1. If the power plant thermal load is poorly controlled and the change rate is slow, then the variables x1 and x2 are far from the desired overshoot and convergence time, and the reward value will be close to the minimum value of 0. Therefore, the reward value can reflect the control performance of the actual control system.

[0112] Preferably, the agent uses the reinforcement learning algorithm adopted by the agent, such as the PPO (Proximal Policy Optimization) algorithm, assuming that the weighted decision strategy made by the agent to control the fan, pump and valve is a(t), the past historical time T, the corresponding environment state s(t) and the decision a(t) made, constitute the strategy trajectory sequence D k for:

[0113] D k ={[s(1),a(1)],...,[s(T),a(T)]}

[0114] The operation decision parameters for the next moment can be expressed as:

[0115]

[0116] Where T is the number of historical data; s t is the environmental state; a t The weight decision for operating fans, pumps and valves; function A θ′ Indicates the advantage under the current state and operation; g(ε,A θ′ (s t ,a t )) function represents the amplitude limiting function, so that A θ′ The value range of is between 1-ε and 1+ε; It represents the probability ratio of making a decision in the current state to making a decision in the new state.

[0117] Furthermore, the operational decision at the predicted future time P can be expressed as:

[0118]

[0119] …

[0120]

[0121] A θ ′(s t ,a t)g(ε,A θk+j-1 (s t ,a t )))

[0122] …

[0123]

[0124] A θ ′(s t ,a t )g(ε,A θk+p-1 (s t ,a t )))

[0125] These decisions are in turn used to adjust fan speeds, pump flow, and valve openings to ensure the cooling system remains stable under changing heat load conditions.

[0126] Furthermore, the predicted control decision parameters are fed into the feedback correction module for further adjustment to optimize the control of fans, pumps, and valves, ensuring the overall stability and efficiency of the cooling system. This reinforcement learning-based control algorithm enables the system to intelligently adapt to complex operating conditions, achieving efficient and stable operation.

[0127] 4) Actuator control

[0128] Based on the optimization results, the controller adjusts the fan speed, water pump flow and valve opening in real time to ensure the stability and energy saving of temperature control.

[0129] The beneficial effects of the present invention are as follows:

[0130] The present invention realizes real-time optimization of control strategies for fans, water pumps, valves and other equipment in the cooling system by integrating the strategy optimization of intelligent agents and the action optimization in traditional model predictive control, and has the adaptability of reinforcement learning and the stability of model predictive control; by setting up intelligent agents and feedback correction modules, it realizes adjustment of prediction model parameters according to the actual state of the system, thereby improving the robustness and adaptability of the control system; by combining the control law output by rolling optimization with multiple real-time feedback and prediction corrections, it realizes that the system maintains efficient temperature control in dynamic changes.

[0131] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0132] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, characterized in that: include: Receive feedback output from the cooling system to obtain environmental status; The reward function is calculated according to the environmental state to obtain a reward value; the reward function is: ； in, is the reward value; is the current system temperature; is the target temperature; is the temperature change amplitude; is the fan power consumption; is the power consumption of the water pump; is the first weight coefficient; is the second weight coefficient; is the third weight coefficient; is the fourth weight coefficient; The environment state and the reward value are input into a preset agent to obtain a weight coefficient operation decision; the weight coefficient operation decision is: ； in, The value range of is 1 to p; p is the final prediction time; Representative time; is the fifth weight coefficient; For historical data sets; is the number of historical data; Indicates the probability of the decision under the current state; represents the probability of the decision under the updated state; Indicates advantages in current state and operation; is the amplitude limiting function; is the environmental state; To operate the fan; Adjust parameters for noise; Inputting the weight coefficient operation decision into the feedback correction module for adjustment to obtain a correction strategy; Construct an optimization objective function and integrate the upper and lower limit constraints of the control input, the constraints of the control increment, and the constraints of the system output to obtain the control law of the rolling optimization output; the optimization objective function is: ； in, is the system output; is the expected output; It is the fan control input; It is the water pump control input; It is the valve control input; is the length of the prediction time domain; Time domain length optimized for control; The expressions of the upper and lower limit constraints of the control input are: ； The expression of the control increment constraint is: ； The expression of the constraint output by the system is: ； in, is the output lower limit; To control the output; is the output upper limit; is the lower limit of the output increment; is the increment of the control output; is the upper limit of the output increment; is the lower bound of the constraint; Output constraints for the system; is the upper limit of the constraint; The control law is: ； in, Represents the control input at the next moment; Represents the control input at the current moment; and are the first constraint condition parameter and the second constraint condition parameter respectively; yes The effective component in the perturbation direction p; To adjust the parameters; Calculating an error between the temperature data in the environmental state and the target temperature to obtain a temperature error; According to the temperature error, the correction strategy is optimized using the control law to obtain an optimized strategy; The optimization strategy is used to regulate the components in the cooling system; the components include: a control fan, a high-temperature water pump, a low-temperature water pump, a left motor valve, a right motor valve, a generator valve, and a transmission oil cooler valve.

Images