Fuel cell vehicle thermal management cooperative control method, device, equipment and medium

Abstract

This application discloses a method, device, equipment, and medium for collaborative control of thermal management in fuel cell vehicles, relating to the field of thermal management control technology for fuel cell vehicles. The method includes: acquiring the current state information of the thermally managed object, where the current state information is a multi-dimensional state vector; generating a basic control quantity for the actuator based on the current state information; inputting the multi-dimensional state vector into a reinforcement learning policy network to obtain residual adjustment actions; superimposing the basic control quantity and the residual adjustment actions, performing physical boundary limiting and rate-of-change constraint processing to generate a target control command, and issuing the target control command to the actuator of the thermally managed object. This application enables the reinforcement learning policy to be effectively trained and reliably deployed under strict constraints ensuring the safety of the thermally managed object and the stability of the co-simulation environment, significantly improving the stability, safety, and robustness of collaborative control for thermal management in fuel cell vehicles.

Description

Technical Field

[0001] This application relates to the field of thermal management control technology for fuel cell vehicles, and in particular to methods, devices, equipment and media for coordinated thermal management control of fuel cell vehicles. Background Technology

[0002] Fuel cell vehicle thermal management systems exhibit dynamic characteristics such as multiple inputs and multiple outputs, strong coupling, nonlinearity, and large hysteresis. During operation, the stack cooling circuit and the power battery cooling circuit influence each other, resulting in high thermal inertia and strict constraints. Multiple objectives, including system energy consumption, hydrogen consumption, and component lifespan, must be considered simultaneously, thus placing extremely high demands on control stability.

[0003] Currently, while control methods employing deep reinforcement learning possess autonomous learning and optimization capabilities, they generally lack safety constraint design for actual actuators and co-simulation environments. During training, the actions output by the policy network are prone to extreme exploration, leading to problems such as actuator impact, temperature exceeding limits, and abnormal solutions to the underlying physical model. These issues severely affect control stability and make it difficult to safely deploy reinforcement learning strategies in real-world systems.

[0004] Therefore, improving control stability and safety in the thermal management of fuel cell vehicles is a problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a method, device, equipment, and medium for coordinated control of thermal management in fuel cell vehicles, aiming to solve the technical problem of how to improve control stability and safety in the thermal management of fuel cell vehicles.

[0006] To achieve the above objectives, this application proposes a thermal management collaborative control method for fuel cell vehicles, the method comprising: Obtain the current state information of the thermal management object, wherein the current state information is a multi-dimensional state vector; Based on the current state information, the basic control quantities of the actuator are generated; The multidimensional state vector is input into the reinforcement learning policy network to obtain residual adjustment actions; After superimposing the basic control quantity with the residual adjustment action, physical boundary limiting processing and rate of change constraint processing are performed to generate the target control command, and the target control command is sent to the actuator of the thermal management object.

[0007] In one embodiment, the step of obtaining the current status information of the thermal management object includes: The system collects the current stack temperature, current power battery temperature, current power demand of the electronic control unit, current state of charge of the power battery, and current hydrogen consumption rate of the thermal management object. The current stack temperature, the current power battery temperature, the current power demand of the electronic control unit, the current state of charge of the power battery, and the current hydrogen consumption rate are normalized to obtain a normalized multidimensional state vector. The normalized multidimensional state vectors are combined to form the current state information of the thermal management object.

[0008] In one embodiment, the step of generating the basic control quantity of the actuator based on the current state information includes: Obtain the preset reference temperature of the fuel cell stack, the reference temperature of the power battery, and the reference state of charge; Calculate the first deviation between the current stack temperature and the stack reference temperature, and the second deviation between the current power battery temperature and the power battery reference temperature; Based on the first deviation, the second deviation, and the current power demand of the electronic control unit, a preset expert baseline control rule is invoked to calculate the basic control quantity of the actuator, which includes a water pump, a fuel cell cooling circuit compressor, and a battery cooling circuit compressor.

[0009] In one embodiment, the step of inputting the multidimensional state vector into the reinforcement learning policy network to obtain the residual adjustment action includes: The current state information is input into the reinforcement learning policy network, and the reinforcement learning policy network outputs the original action vector. Each element in the original action vector is mapped to a preset continuous interval to obtain the initial residual action; The initial residual action is multiplied by the corresponding preset scaling factor to generate the residual adjustment action.

[0010] In one embodiment, the step of superimposing the basic control quantity and the residual adjustment action, performing physical boundary limiting processing and rate of change constraint processing, generating a target control command, and issuing the target control command to the actuator of the thermal management object includes: The residual adjustment action is added to the corresponding control value in the basic control quantity to generate the undetermined control quantity; Physical boundary limiting processing is performed on the undetermined control quantity to limit it within the hardware safe operating range of the actuator, thereby obtaining the limited control quantity; A single-step rate of change constraint is applied to the control quantity after the amplitude limit, the difference between the historical target control command of the previous cycle and the control quantity after the amplitude limit is calculated, and the difference is limited to a preset smooth change range to obtain the target control command of the current cycle. The target control command for the current cycle is sent to the actuator of the thermal management object.

[0011] In one embodiment, after the step of issuing the target control command to the actuator of the thermal management object, the method further includes: In the co-simulation environment, the state of the thermal management object is updated according to the target control command, and the updated state is observed. The co-simulation environment includes a physical environment and a control environment. The immediate reward is calculated based on the updated state, wherein the immediate reward includes penalties for exceeding temperature limits, exceeding the charge state limits, and abnormal situations in the solution of the physics model. Experience data is generated based on the current status information, the residual adjustment action, the instant reward, and the updated status, and the experience data is stored in the experience replay pool. The reinforcement learning policy network is trained offline, and the offline training steps include: A physical environment is constructed based on a fuel cell stack model, a power battery model, a cooling circuit model, and an actuator dynamic model, as well as a control environment that performs joint simulation with the physical environment; The reinforcement learning policy network, value network, target value network, and experience replay pool are initialized in the control environment. Returning to the steps of updating the state of the thermal management object according to the target control command and observing the updated state, iterative training is performed on the reinforcement learning policy network and the value network; Based on the empirical data, the preset loss function is minimized, and the network parameters of the reinforcement learning policy network and the value network are updated respectively. The parameters of the value network are copied to the target value network using a soft update method. Training stops when the updated policy is verified to meet the preset safety boundary conditions in the joint simulation environment and the number of abnormal terminations is less than the preset threshold.

[0012] In one embodiment, the step of updating the network parameters of the reinforcement learning policy network and the value network respectively by minimizing a preset loss function based on the empirical data includes: The experience data in the experience replay pool is randomly sampled to obtain multiple random experience data, each of which includes the current state information, residual adjustment action, instant reward value and updated state; The updated state is input into the target value network, and the expected Q value is calculated by combining it with the instant reward value. The current state information and the residual adjustment action are input into the value network to calculate the predicted Q value; The mean square error between the predicted Q value and the expected Q value is calculated, and the mean square error is minimized by gradient descent to update the parameters of the value network. The negative correlation between the action distribution output by the reinforcement learning policy network and the Q-value output by the value network is calculated, and the negative correlation is minimized by gradient descent to update the parameters of the reinforcement learning policy network.

[0013] Furthermore, to achieve the above objectives, this application also proposes a thermal management collaborative control device for fuel cell vehicles, the thermal management collaborative control device comprising: The information acquisition module is used to acquire the current state information of the thermal management object, wherein the current state information is a multi-dimensional state vector; The basic control module is used to generate basic control quantities for the actuator based on the current state information. The residual adjustment module is used to input the multidimensional state vector into the reinforcement learning policy network to obtain residual adjustment actions. The collaborative control module is used to superimpose the basic control quantity and the residual adjustment action, perform physical boundary limiting processing and rate of change constraint processing, generate target control instructions, and send the target control instructions to the actuator of the thermal management object.

[0014] In addition, to achieve the above objectives, this application also proposes an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fuel cell vehicle thermal management cooperative control method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a non-transitory storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the fuel cell vehicle thermal management collaborative control method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: (1) Obtain the current state information of the thermal management object, which is a multi-dimensional state vector; generate the basic control quantity of the actuator based on the current state information; input the multi-dimensional state vector into the reinforcement learning policy network to obtain the residual adjustment action; after superimposing the basic control quantity and the residual adjustment action, perform physical boundary limiting processing and rate of change constraint processing to generate the target control command, and send the target control command to the actuator of the thermal management object. By introducing a two-layer control architecture that combines the safety benchmark based on the basic control quantity with the reinforcement learning residual adjustment, the basic control quantity provides a safe and reliable operation safety net for the system, ensuring that the system can still be in a controlled state in the worst case, effectively avoiding the complete loss of control or damage of the system due to the abnormal exploration of the reinforcement learning policy. On this basis, the residual adjustment action is output through the reinforcement learning policy network, enabling the system to learn and optimize the adjustment strategy on the basis of the basic control benchmark, thereby achieving autonomous learning and optimization improvement of comprehensive performance such as energy consumption and temperature control accuracy while ensuring safety. In particular, by strictly limiting the physical boundaries and rate of change of the superimposed control commands, the safety issues that may be caused by unreasonable outputs or exploratory actions of the policy network are eliminated at the final execution level. The exploratory behavior of reinforcement learning is restricted to a safe, smooth, and physically realizable range, which significantly improves the stability, safety and robustness of the control method.

[0017] (2) A physical environment is constructed based on the fuel cell stack model, power battery model, cooling circuit model, and actuator dynamic model, as well as a control environment for joint simulation with the physical environment; the reinforcement learning policy network, value network, target value network, and experience replay pool are initialized in the control environment; the steps of updating the state of the thermal management object according to the target control command and observing the updated state are returned, and the reinforcement learning policy network and value network are iteratively trained; the network parameters of the reinforcement learning policy network and value network are updated respectively based on the preset loss function minimized by the empirical data, and the parameters of the value network are copied to the target value network by means of soft update, until the updated policy is verified to meet the preset safety boundary conditions in the joint simulation environment and the number of abnormal terminations is less than the preset threshold, and then the training is stopped. When the reinforcement learning agent explores the optimization space, it must first learn to identify and avoid all dangerous areas that may cause system failure, thereby greatly improving the stability and efficiency of the training process, and finally making the policy after training convergence not only superior in performance, but also possessing an inherent safety awareness, significantly enhancing its reliability and safety in deployment in real systems. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the first embodiment of the fuel cell vehicle thermal management collaborative control method of this application; Figure 2 This is a schematic diagram showing the correspondence between the five-dimensional observation variables and the three types of actuators in this application; Figure 3 This is a flowchart of the safety action calculation process, which consists of expert baseline control, residual fine-tuning, amplitude limiting, and smoothing constraints in this application. Figure 4 This is a schematic diagram illustrating the reinforcement learning training process and anomaly truncation mechanism of this application; Figure 5 This is a schematic diagram of the module structure of the fuel cell vehicle thermal management collaborative control device of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a method for coordinated control of thermal management in fuel cell vehicles. Specifically, refer to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the fuel cell vehicle thermal management collaborative control method of this application. In this embodiment, the fuel cell vehicle thermal management collaborative control method includes steps S10 to S40:

[0024] Step S10: Obtain the current state information of the thermal management object. The current state information is a multi-dimensional state vector.

[0025] It should be noted that thermal management targets refer to the key components in a fuel cell vehicle whose temperature needs to be controlled. These mainly include the fuel cell stack, the battery pack, and the cooling circuits that dissipate heat from them, such as the stack cooling circuit and the battery cooling circuit. Current status information refers to a set of data collected from the thermal management system at a specific moment, reflecting its operating status. It includes multi-dimensional arrays, such as stack temperature, battery temperature, ambient temperature, system pressure, state of charge (SOC), and the current opening degree or speed of each actuator.

[0026] Step S20: Generate the basic control quantity of the actuator based on the current state information.

[0027] It should be noted that the basic control quantity refers to a set of actuator control instructions calculated based on classical control theory, rules, or simplified models. These instructions ensure the thermal management system operates under the most basic or safest conditions, providing a safe and functional baseline operating point for the entire control system. For example, the actuator control quantity can be directly mapped from a lookup table based on the matching relationship between the system state and preset thresholds or operating conditions. Classical control algorithms such as PID (Proportional-Integral-Derivative) and feedforward can be used, with the key objective as the setpoint. For example, using the target temperature of the fuel cell stack as the setpoint and the actual fuel cell stack temperature as the feedback value, a pre-set PID controller can calculate the basic speed of the cooling fan.

[0028] Step S30: Input the multidimensional state vector into the reinforcement learning policy network to obtain the residual adjustment action.

[0029] It should be noted that a reinforcement learning policy network is a deep neural network model that learns output actions through trial and error by interacting with the environment to maximize long-term rewards. Deep neural network models are used to learn how to fine-tune basic control variables based on the system state. Residual adjustment actions refer to the control increments output by the reinforcement learning policy network, used to fine-tune or correct the basic control variables.

[0030] Step S40: After superimposing the basic control quantity and the residual adjustment action, perform physical boundary limiting processing and rate of change constraint processing to generate the target control command, and send the target control command to the actuator of the thermal management object.

[0031] For example, physical boundary limiting processing refers to forcibly restricting the superimposed initial control commands within the physically permissible operating range of each actuator and the parameter boundaries required for the safe operation of the thermal management system. Rate of change constraint processing refers to imposing restrictions on the rate of change of the initial control commands to ensure smooth changes in the control commands, avoid impacts on actuators such as pumps and compressors, and prevent system instability due to sudden command changes.

[0032] In this embodiment, a two-layer control architecture combining a safety benchmark based on fundamental control variables and reinforcement learning residual adjustment is introduced. The fundamental control variables provide a safe and reliable operational safety net for the system, ensuring that it remains under control even in the worst-case scenario. This effectively prevents complete system loss of control or damage due to abnormal exploration by the reinforcement learning strategy. Based on this, the system can autonomously learn and optimize adjustment strategies based on the fundamental control benchmark by outputting residual adjustment actions through the reinforcement learning strategy network. This achieves autonomous learning and optimization of comprehensive performance, including energy consumption and temperature control accuracy, while ensuring safety. Specifically, by strictly limiting the physical boundary amplitude and rate of change of the superimposed control commands, potential safety issues caused by unreasonable or exploratory actions output by the strategy network are eliminated at the final execution level, restricting the exploratory behavior of reinforcement learning within a safe, smooth, and physically realizable range. This embodiment enables the reinforcement learning strategy to be effectively trained and reliably deployed under strict constraints that ensure the safety of the thermally managed object and the stability of the co-simulation environment, significantly improving the stability, safety, and robustness of the control method.

[0033] It should be noted that the embodiments and implementation methods of this application are carried out in a co-simulation environment, which includes a physical environment and a control environment. The physical environment corresponds to building a dynamic model of the fuel cell stack, power battery, ECU (Electronic Control Unit) power demand model, cooling circuit model and actuator in AMESim. The control environment corresponds to establishing a closed-loop simulation environment with AMESim in Matlab or Simulink.

[0034] In one implementation, prior to step S10, the process includes: building a fuel cell stack model, a power battery model, an electronic control unit (ECU) power demand model, a cooling circuit model, and an actuator dynamic model in AMESim software as a physical environment; establishing a closed-loop simulation environment in Matlab or Simulink that interacts with AMESim software as a control environment; and connecting the physical environment and the control environment through a co-simulation interface to form a co-simulation environment, which is used for training and validating reinforcement learning strategies.

[0035] Specifically, the fuel cell stack model is used to calculate the stack output voltage, stack output power, hydrogen consumption, and stack heat source, and is expressed as follows: in, This indicates the actual output voltage of a single fuel cell unit; Indicates the Nernst reversible voltage; Indicates the activation polarization overpotential; Indicates the ohmic polarization overpotential; This represents the concentration polarization overpotential.

[0036] in, This indicates the total output voltage of the fuel cell stack; This indicates the number of individual cells connected in series in the fuel cell stack.

[0037] in, This indicates the rate of hydrogen consumption. This represents the output current of the fuel cell stack; F represents the Faraday constant. This indicates the molar mass of hydrogen gas.

[0038] in, This indicates the output electrical power of the fuel cell stack.

[0039] in, Indicates the heating power of the fuel cell stack; This indicates the lower heating value of hydrogen.

[0040] in, Indicates the equivalent heat capacity of the fuel cell stack; Indicates the temperature of the fuel cell stack; This represents the rate of change of the fuel cell stack temperature with respect to time. This indicates the heat removed by the fuel cell stack cooling circuit. The target temperature for the fuel cell stack is preferably set in the range of 70°C to 80°C, more preferably 75°C. This temperature range is determined based on the high-efficiency operating temperature range of the fuel cell system and the lifespan requirements of the membrane electrode assembly.

[0041] The power battery model is used to obtain battery terminal voltage, SOC evolution, and heat source intensity, and is represented as follows: in, Indicates the terminal voltage of the power battery; Indicates open-circuit voltage; Indicates the current of the power battery; Indicates the internal resistance of the power battery; Indicates the battery charge / discharge rate; Indicates polarization voltage; This indicates the temperature of the power battery.

[0042] in, This indicates the state of charge of the power battery at time t; This represents the initial state of charge. Indicates the rated capacity of the power battery; Indicates coulomb efficiency; This represents the integral variable.

[0043] in, Indicates the heating power of the power battery; Indicates the equivalent heat capacity of the power battery; This indicates the heat removed by the power battery cooling circuit. The target temperature for the power battery is preferably set between 30℃ and 40℃, more preferably between 35℃ and 40℃. This parameter is determined based on the power battery's cycle life, internal resistance characteristics, and cooling system capacity. The preferred operating range for State of Charge (SOC) is set between 0.30 and 0.80 to avoid artificially improving temperature control through overcharging or excessive over-extraction.

[0044] The ECU power demand model is used to characterize the impact of vehicle driving conditions on thermal management objects. It can be imported into AMESim using the vehicle's longitudinal dynamics model, and is expressed as follows: in, This indicates the longitudinal traction resistance of the entire vehicle; Indicates air density; Indicates the air drag coefficient; Indicates the vehicle's frontal area; Indicates vehicle speed; Indicates the overall vehicle weight; Represents gravitational acceleration; Indicates the rolling resistance coefficient; Indicates the road slope angle; This represents the equivalent mass after considering the inertia of rotating components; This indicates the longitudinal acceleration of the vehicle.

[0045] in, Indicates the ECU power requirement; Indicates the efficiency of the drive motor; Indicates inverter efficiency; This indicates the efficiency of the transmission system. The upper limit is preferably set according to the maximum drive power of the target vehicle or the peak power of the motor, and in a preferred embodiment, it can be in the range of 80kW to 120kW.

[0046] The cooling circuit model includes the fuel cell stack cooling circuit and the power battery cooling circuit, which can be described using a lumped parameter heat transfer model, expressed as: in, Indicates cooling heat exchange power; Indicates the convective heat transfer coefficient; Indicates the effective heat exchange area; Indicates the temperature of the component being cooled; This indicates the temperature of the coolant.

[0047] in, Indicates the equivalent mass of the coolant; This indicates the specific heat capacity of the coolant; This indicates the amount of heat entering the coolant; This indicates the heat removed by the cooling circuit.

[0048] in, Indicates the coolant mass flow rate; This indicates the mapping relationship between the water pump opening degree and the coolant flow rate; and These represent the compressor speeds of the fuel cell stack cooling circuit and the battery cooling circuit, respectively. and These represent the mapping relationship between the drive motor torque and the compressor speed.

[0049] The dynamic model of the actuator is used to represent the response hysteresis of the actual actuator. A first-order inertial dynamic model can be established for the water pump and the motor driving the compressor, as follows:

[0050] Where k represents the discrete time; Indicates the control step size; , and These represent the target control commands for the corresponding actuators; , and These represent the inertial time constants of the corresponding actuators. The control step size is preferably 0.5s to 1.0s, and the inertial time constant can be determined based on actuator bench response tests, supplier sample data, or AMESim component library parameters.

[0051] Simulink, as the control environment, can be responsible for state preprocessing, action calculation, reward calculation, and SAC network training.

[0052] In one implementation, step S10 includes: Step S101: Collect the current stack temperature, current power battery temperature, current power demand of the electronic control unit, current state of charge of the power battery, and current hydrogen consumption rate of the thermal management object.

[0053] For example, the current state information can be represented as:

[0054] in, This represents the observation state vector at the current moment; This indicates the current temperature of the fuel cell stack. This indicates the current temperature of the power battery; This indicates the current power requirement of the ECU; This indicates the current state of charge of the power battery; This indicates the rate of hydrogen consumption at the current moment.

[0055] Step S102: Normalize the current stack temperature, current power battery temperature, current power demand of the electronic control unit, current state of charge of the power battery, and current hydrogen consumption rate to obtain a normalized multidimensional state vector.

[0056] For example, maximum-minimum normalization can be used. The upper and lower limits of the fuel cell stack temperature and the power battery temperature are determined based on the allowable operating temperature range of the target vehicle's thermal management system; the upper limit of the ECU power demand is determined based on the vehicle's maximum drive power or peak power under typical operating conditions; the upper and lower limits of the SOC operating range are determined based on the battery life protection strategy; and the normalized range of hydrogen consumption is obtained based on the fuel cell stack's rated power and statistics under typical operating conditions. By unifying the dimensions, the impact of differences in the numerical scales of different variables on the training stability of the policy network and the value network can be reduced.

[0057] Step S103: Combine the normalized multidimensional state vectors into the current state information of the thermal management object.

[0058] It should be noted that the state variables can be preprocessed by normalizing the environment to form the input to the reinforcement learning policy network. For example, normalizing a multi-dimensional state vector can be expressed as:

[0059]

[0060] in, Represents the reinforcement learning state vector; Indicates the first A normalized state variable; Indicates the first One original state variable; and These represent the minimum and maximum values ​​of the corresponding state variables, respectively.

[0061] In this embodiment, by synchronously acquiring five key multi-dimensional state information—stack temperature, power battery temperature, power demand, SOC, and hydrogen consumption rate—from the actual physical model, the control system can comprehensively and in real-time perceive the operating status of the thermally managed object and the external load demand, providing an accurate perception foundation for solving the coordinated control of multi-loop, strongly coupled systems. Furthermore, by normalizing and preprocessing the aforementioned heterogeneous state variables and combining them into a standard state vector, the problem of difficulty in convergence or instability in reinforcement learning network training caused by significant differences in the dimensions and numerical ranges of different physical quantities is solved. This provides the policy network with a scale-uniform, easily learnable input, fundamentally improving the perception accuracy and decision-making stability of the entire control system.

[0062] In one implementation, to avoid the reinforcement learning strategy directly outputting a danger control quantity in the early stages of training, a basic control quantity can be generated based on the current thermal state and power requirements. Specifically, step S20 includes: Step S201: Obtain the preset reference temperature of the fuel cell stack, the reference temperature of the power battery, and the reference state of charge.

[0063] It should be noted that the fuel cell stack reference temperature refers to the pre-set target operating temperature that the fuel cell stack needs to maintain. This temperature value is determined comprehensively based on the high-efficiency operating temperature range of the fuel cell system and the membrane electrode assembly (MEA) life requirements, with a preferred value of 75°C. The power battery reference temperature refers to the pre-set target operating temperature that the power battery needs to maintain. This temperature value is determined jointly based on the power battery's cycle life, internal resistance characteristics, and cooling system capabilities, with a preferred range of 35°C to 40°C. The reference state of charge (SOC) refers to the pre-set target operating point or reference value of the power battery's SOC. Its setting needs to consider battery life protection and should be set near the midpoint of the allowable operating range. The SOC operating range can be set to 0.30–0.80, and the reference value can be the midpoint of this range.

[0064] Step S202: Calculate the first deviation between the current stack temperature and the stack reference temperature, and the second deviation between the current power battery temperature and the power battery reference temperature.

[0065] It should be noted that the first deviation refers to the difference between the current stack temperature and the stack reference temperature, reflecting the degree of deviation between the actual stack temperature and the ideal target temperature. The second deviation refers to the difference between the current power battery temperature and the power battery reference temperature, reflecting the degree of deviation between the actual power battery temperature and the ideal target temperature.

[0066] Step S203: Based on the first deviation, the second deviation, and the current power demand of the electronic control unit, the preset expert baseline control rules are invoked to calculate and obtain the basic control quantity of the actuator, which includes a water pump, a fuel cell cooling circuit compressor, and a battery cooling circuit compressor.

[0067] It should be noted that expert baseline control rules refer to a set of deterministic control logic or calculation formulas pre-designed based on domain knowledge, such as thermodynamic principles and engineering experience. These rules directly map control commands to actuators based on the current system state, aiming to provide a fundamental control quantity that ensures the safe and stable operation of the system without the need for reinforcement learning. Actuators are physical devices that receive control commands and directly drive the cooling system, including water pumps, fuel cell stack cooling circuit compressors, and battery cooling circuit compressors. Water pumps control the coolant flow rate by adjusting their opening degree; fuel cell stack cooling circuit compressors and battery cooling circuit compressors control their speed by adjusting the torque of their drive motors, thereby regulating the cooling capacity of their respective circuits.

[0068] For example, the basic control quantity of the actuator is represented as:

[0069] in, Indicates the baseline opening of the water pump; This indicates the baseline torque of the compressor motor driven by the fuel cell stack cooling circuit; This indicates the baseline torque of the compressor motor driven by the battery cooling circuit; , , Indicates the baseline control gain coefficient of the water pump; , This represents the baseline control gain coefficient of the fuel cell stack-side drive motor; , This represents the baseline control gain coefficient of the battery-side drive motor. Indicates the reference temperature of the fuel cell stack; Indicates the reference temperature of the power battery; Indicates reference SOC; Indicates a reference threshold for power demand; , and These represent the minimum control values ​​for the corresponding actuators; This represents the amplitude limiting function.

[0070] The aforementioned baseline control coefficients are preferably determined by combining the existing rule-based control strategy calibration results, AMESim open-loop simulation results, and actuator safe operating boundaries.

[0071] In this embodiment, by pre-setting the reference temperature and SOC reference values ​​of the key controlled objects, a clear and safe control objective is established for the entire thermal management control system, providing a benchmark for solving multi-objective collaborative control problems. Furthermore, by calculating the real-time deviation between the current state and these reference values, the complex temperature and SOC control problem is transformed into an adjustment problem of the deviation, enabling the control system to clearly perceive the gap between the current operating state and the ideal state. Based on preset expert baseline control rules and by directly calculating the actuator's basic control quantities through deviation and power requirements, a deterministic safety control layer independent of reinforcement learning strategies is constructed. This solves the core safety problem that reinforcement learning strategies may output dangerous actions during the initial training phase or when facing unknown operating conditions, leading to system loss of control. This ensures that even when the intelligent strategy fails or anomalies are explored, the system can still be maintained in the most basic safe operating state by expert rules, thereby greatly improving the robustness of the entire control architecture and the feasibility of engineering deployment.

[0072] In one implementation, step S30 includes: Step S301: Input the current state information into the reinforcement learning policy network, and the reinforcement learning policy network outputs the original action vector.

[0073] It should be noted that the reinforcement learning policy network is trained using the Soft Actor Critic (SAC) algorithm, and its output is the parameters of the action distribution. Specific actions are obtained through reparameterization techniques. The original action vector refers to the action signal directly output by the reinforcement learning policy network; it is a multi-dimensional vector whose dimension corresponds to the number of actuators, with each element representing the original control intention towards one of the actuators.

[0074] Step S302: Map each element in the original action vector to a preset continuous interval to obtain the initial residual action.

[0075] It should be noted that the preset continuous interval refers to the numerical range pre-defined for each element of the original action vector, used to normalize the output of the policy network and stabilize its values. The tanh activation function can be used, with an interval of [-1, 1]. The initial residual action refers to the action signal obtained after the original action vector has been mapped and constrained within the preset continuous interval.

[0076] Step S303: Multiply the initial residual action by the corresponding preset scaling factor to generate the residual adjustment action.

[0077] It should be noted that the preset scaling factor is used to scale the dimensionless initial residual action to a coefficient that represents the magnitude of the control quantity change with actual physical meaning. Each actuator corresponds to an independent scaling factor. The residual adjustment action refers to the final adjustment amount obtained by multiplying the initial residual action by the corresponding preset scaling factor.

[0078] For example, the actuator may include the water pump opening degree, the compressor motor torque driven by the fuel cell cooling circuit, and the compressor motor torque driven by the battery cooling circuit.

[0079]

[0080] in, This represents the control command vector that is ultimately sent to the actuator; This indicates the pump opening at the current moment; This indicates the torque of the compressor motor driven by the fuel cell stack cooling circuit at the current moment; This indicates the torque of the compressor motor driven by the battery cooling circuit at the current moment.

[0081] The original action output by the reinforcement learning network is represented as follows:

[0082] in, This represents the original action vector output by the policy network at the current moment; , and These represent the residual adjustment actions corresponding to the three actuators.

[0083] In this embodiment, the original action vector output by the policy network is first mapped to a standard continuous interval, which solves the problem of unstable output range and potential for large or small outliers in reinforcement learning networks, providing a stable input basis for subsequent quantitative scaling. Furthermore, the normalized initial residual action is multiplied by a preset scaling factor to generate a residual adjustment action, solving the problem of extreme exploration actions caused by the policy network directly outputting full-range control commands. This achieves an effective balance between exploration freedom and operational safety at the control architecture level.

[0084] In one implementation, step S40 includes: Step S401: Add the residual adjustment action to the corresponding control value in the basic control quantity to generate the undetermined control quantity.

[0085] For example, the policy network outputs three-dimensional continuous residual actions, corresponding to the water pump, the fuel cell stack-side driven compressor motor, and the battery-side driven compressor motor, respectively, with the corresponding formulas as follows:

[0086] in, Indicates the first The residual action corresponding to each actuator; This indicates the target pump opening after residual adjustment; This indicates the target torque of the fuel cell stack-side drive compressor motor after residual adjustment; This indicates the target torque of the battery-side drive compressor motor after residual adjustment; , and These represent the residual scaling factors for the three actuators, respectively.

[0087] The residual scaling factor is set to 5%–20% of the actuator's full range, more preferably around 10%. This parameter is determined based on two principles: first, to retain sufficient degrees of freedom for optimization; and second, to prevent the reinforcement learning output action from straying far from the safe baseline within a single step. Through this structure, reinforcement learning no longer directly gains global control over the actuator, but can only perform local optimization near the safe baseline.

[0088] Step S402: Perform physical boundary limiting processing on the control quantity to be determined, limiting the control quantity to the safe operating range of the actuator's hardware, and obtain the limited control quantity.

[0089] For example, it can be determined whether the pending control instruction exceeds the physical operating upper and lower limits of the actuator. If the pending control instruction exceeds the physical operating upper and lower limits of the actuator, the pending control instruction is restricted to the physical operating upper and lower limits, and a limited control instruction is generated.

[0090] Step S403: Perform single-step rate of change constraint processing on the control quantity after limiting, calculate the difference between the historical target control command of the previous cycle and the control quantity after limiting, limit the difference within the preset smooth change range, and obtain the target control command of the current cycle.

[0091] For example, it can be determined whether the difference between the historical target control command of the previous control cycle and the control command after limiting exceeds the preset single-step change limit; if the difference between the historical target control command of the previous control cycle and the control command after limiting exceeds the preset single-step change limit, the difference is limited to the single-step change limit, and the target control command of the current cycle is generated.

[0092] Step S404: Send the target control command for the current cycle to the actuator of the thermal management object.

[0093] The actuators of the thermal management object may include a water pump, a compressor motor driven by the fuel cell cooling circuit, and a compressor motor driven by the battery cooling circuit. For example, three residual adjustment actions corresponding to the water pump, fuel cell side motor, and battery side motor can be superimposed one-to-one with three corresponding basic control quantities to generate three undetermined control quantities. These three undetermined control quantities are then limited to preset water pump opening range, fuel cell side motor torque range, and battery side motor torque range, respectively, to generate three intermediate control quantities. These three intermediate control quantities are then compared with their respective historical control quantities from the previous cycle and limited to preset single-step change rate ranges to generate the final water pump opening command, fuel cell side motor torque command, and battery side motor torque command.

[0094] For example, the target control quantity after residual superposition can be subject to physical boundary limits and rate of change restrictions, as follows:

[0095] in, , and These represent the actual control commands from the previous control cycle; , and These represent the actual control commands ultimately issued in the current control cycle; This indicates the maximum permissible change in pump opening in a single step; and These represent the maximum permissible variation in torque of the compressor motors on both sides in a single step.

[0096] The physical upper and lower limits of the water pump opening are preferably limited to 0-100%. The upper and lower limits of the drive compressor motor torque are determined by the motor's rated torque, the compressor's safe operating range, and the heat exchange capacity of the cooling system. The maximum single-step change is preferably determined based on the actuator response time, the cooling circuit's impact tolerance, and the drive motor's transient current constraint. The upper limit of the change rate can be set to no more than 5% to 10% of the full range per step.

[0097] It should be noted that after step S404, the following steps are also included: updating the state of the thermal management object according to the target control command in the co-simulation environment and observing the updated state. The co-simulation environment includes the physical environment and the control environment; calculating the immediate reward based on the updated state, wherein the immediate reward includes penalty terms for temperature exceeding limits, charge state exceeding limits, and abnormal situations in the physical model solution; generating empirical data based on the current state information, residual adjustment actions, immediate rewards, and the updated state, and storing the empirical data in the experience playback pool.

[0098] The step of calculating the immediate reward based on the updated state includes: calculating the first absolute deviation between the stack temperature in the updated state and the preset stack target temperature, and the second absolute deviation between the power battery temperature and the preset battery target temperature; determining whether the state of charge in the updated state is below the lower threshold or above the upper threshold, and if so, generating a state of charge out-of-bounds penalty value; when the updated state triggers any of the termination conditions of temperature out-of-bounds, state of charge out-of-bounds, or detection of physical model solution anomalies, applying a negative termination penalty value to the immediate reward value at the current moment, and terminating the current training round.

[0099] It should be noted that the reward function no longer only pursues simple temperature control, but also considers temperature deviation, SOC deviation, hydrogen consumption, actuator energy consumption, and safety status. For example, the reward function can be expressed as:

[0100]

[0101]

[0102]

[0103] in, This represents the reward value obtained by the agent at the current moment; to This represents the weighting coefficient of each penalty; Indicates the SOC out-of-bounds penalty item; Indicates the total power consumption of the actuator; Indicates the safety status indicator; This indicates a reward for stable operation within a safe range; and These represent the lower and upper limits of the SOC's allowed working range, respectively. Indicates the power consumption of the water pump; and These represent the power consumption of the compressor motors on both sides.

[0104] The reward weights are selected based on the priority of the control objectives: the weight of stack temperature control is usually greater than that of power battery temperature control; penalties for exceeding SOC limits and solving abnormalities are significantly greater than general energy consumption penalties; the hydrogen consumption weight is determined based on the overall vehicle economy objective. Parameter sources include existing rule-based control results, typical operating condition simulation statistics, and multi-round training stability tests.

[0105] It should be noted that, in order to avoid continuously collecting dangerous samples during the training process, a uniform abnormal termination condition can be set: (1) the stack temperature exceeds the allowable safe range; (2) the power battery temperature exceeds the allowable safe range; (3) the SOC exceeds the allowable working range; (4) the AMESim physical model solution diverges, has no solution, is interrupted, or crashes.

[0106]

[0107]

[0108] in, This indicates a general penalty for exceeding the limit; Indicates the penalty for solving exceptions; and This represents a preset penalty constant, and Greater than The anomaly penalty can be set to more than twice the general over-limit penalty to ensure that the policy prioritizes avoiding action regions that could lead to physical instability during training. A diagram illustrating the correspondence between the five-dimensional observation variables and the three types of actuators is shown below. Figure 2 As shown, the flowchart for calculating safe actions, consisting of expert baseline control, residual fine-tuning, amplitude limiting, and smoothing constraints, is as follows: Figure 3 As shown.

[0109] In this embodiment, when exploring the optimization space, the reinforcement learning agent must first learn to identify and avoid all dangerous areas that may cause system failure, thereby greatly improving the stability and efficiency of the training process. Ultimately, the strategy after training convergence not only has superior performance but also has an inherent safety awareness, significantly enhancing its reliability and security when deployed in real systems.

[0110] Based on the above embodiments and implementation methods, in one implementation method, the reinforcement learning policy network can also be trained offline. The offline training steps include: constructing a physical environment based on a fuel cell stack model, a power battery model, a cooling circuit model, and an actuator dynamic model, as well as a control environment that performs co-simulation with the physical environment; initializing the reinforcement learning policy network, the value network, the target value network, and the experience replay pool in the control environment; returning to the steps of updating the state of the thermal management object according to the target control command and observing the updated state, and iteratively training the reinforcement learning policy network and the value network; updating the network parameters of the reinforcement learning policy network and the value network respectively based on minimizing the preset loss function according to the empirical data, and using a soft update method to copy the parameters of the value network to the target value network, until the updated policy is verified to meet the preset safety boundary conditions in the co-simulation environment and the number of abnormal terminations is less than the preset threshold number, and then stopping the training.

[0111] The physical environment is the AMESim physical environment, and the control environment is the Matlab or Simulink control environment. It should be noted that the value network participates in gradient descent updates to learn and predict Q-values. The target value network is obtained from the value network through soft updates and is used to calculate a stable expected Q-value; it does not directly participate in gradient descent. The soft update method clearly defines the dynamic relationship between the two networks, meaning the target value network follows the updates of the value network.

[0112] It should be noted that the steps of updating the network parameters of the reinforcement learning policy network and the value network respectively based on minimizing the preset loss function using empirical data include: randomly sampling the empirical data in the experience replay pool to obtain multiple random empirical data points, where each random empirical data point includes current state information, residual adjustment action, immediate reward value, and updated state; inputting the updated state into the target value network and calculating the expected Q value in combination with the immediate reward value; inputting the current state information and residual adjustment action into the value network and calculating the predicted Q value; calculating the mean square error between the predicted Q value and the expected Q value, and minimizing the mean square error using gradient descent to update the parameters of the value network; calculating the negative correlation between the action distribution output by the reinforcement learning policy network and the Q value output by the value network, and minimizing the negative correlation using gradient descent to update the parameters of the reinforcement learning policy network.

[0113] An exemplary diagram illustrating the reinforcement learning training process and anomaly truncation mechanism is shown below. Figure 4 As shown, the reinforcement learning policy network takes a five-dimensional state as input and outputs action distribution parameters, and samples actions through a reparameterization technique; a dual-value network is used to reduce Q-value overestimation; an experience replay pool is used to store historical interaction samples; and the target network uses soft updates to improve training stability.

[0114]

[0115] in, This represents the average action output by the policy network; This represents the standard deviation of the actions output by the policy network; This represents random noise that follows a standard Gaussian distribution. This represents element-wise multiplication; This represents the hyperbolic tangent function, used to restrict the action to a finite interval.

[0116]

[0117]

[0118]

[0119] in, Indicates the first The state-action value function corresponding to each value network; Indicates the first A value network loss function; Represents the mathematical expectation; Indicates the target Q value; Indicates the reward discount factor; Represents the entropy temperature coefficient; Indicates by parameters Representational strategy network.

[0120]

[0121] in, Represents the policy network loss function; Indicates the network parameters of the strategy.

[0122]

[0123]

[0124] in, This represents an experience replay sample; This represents the target network's soft update coefficients. Training parameters can be set as follows: the empirical replay pool capacity is... Scale; warm-up step size is 3000-5000 steps; batch size is 128-256; network learning rate is The scale is as follows: the soft update coefficient is 0.005 to 0.02; and the network is updated 5 times every 10 steps.

[0125] In this embodiment, the parameters of the policy network and the value network are updated separately by minimizing a preset loss function based on empirical data. A soft update method is used to synchronously construct a deep reinforcement learning training framework for the target value network, solving the general problem of reinforcement learning training being prone to divergence and difficult to converge in continuous action spaces and complex dynamic systems. Specifically, the value network learns accurate value assessment by minimizing the mean squared error between the predicted Q-value and the expected Q-value, while the policy network optimizes decision-making by maximizing the expected Q-value. The two work together, enabling the policy to optimize long-term performance while learning autonomously.

[0126] Based on the same inventive concept, such as Figure 5 As shown, this application also provides a thermal management collaborative control device for fuel cell vehicles, which includes: Information acquisition module 10 is used to acquire the current state information of the thermal management object, wherein the current state information is a multi-dimensional state vector; The basic control module 20 is used to generate basic control quantities for the actuator based on the current state information. The residual adjustment module 30 is used to input the multidimensional state vector into the reinforcement learning policy network to obtain residual adjustment actions. The collaborative control module 40 is used to superimpose the basic control quantity and the residual adjustment action, perform physical boundary limiting processing and rate of change constraint processing, generate target control instructions, and send the target control instructions to the actuator of the thermal management object.

[0127] It should be noted that the fuel cell vehicle thermal management collaborative control device provided in this application embodiment and the fuel cell vehicle thermal management collaborative control method provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned fuel cell vehicle thermal management collaborative control method, and the repeated parts will not be described again.

[0128] In some embodiments, an electronic device provided in this application includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the above-described fuel cell vehicle thermal management collaborative control method.

[0129] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0130] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0131] This application also provides a non-transitory storage medium storing a computer program. When executed by a processor, this program implements the aforementioned fuel cell vehicle thermal management coordinated control method. This storage medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not be assembled into that device / apparatus / system. The aforementioned non-transitory storage medium carries one or more programs, which, when executed, implement the method as described in the embodiments or implementations of this application.

[0132] According to embodiments of this application, a non-transitory storage medium can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. A computer-readable signal medium can also be any storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0134] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for coordinated control of thermal management in fuel cell vehicles, characterized in that, The method includes: Obtain the current state information of the thermal management object, wherein the current state information is a multi-dimensional state vector; Based on the current state information, the basic control quantities of the actuator are generated; The multidimensional state vector is input into the reinforcement learning policy network to obtain residual adjustment actions; After superimposing the basic control quantity with the residual adjustment action, physical boundary limiting processing and rate of change constraint processing are performed to generate the target control command, and the target control command is sent to the actuator of the thermal management object.

2. The method as described in claim 1, characterized in that, The step of obtaining the current status information of the thermal management object includes: The system collects the current stack temperature, current power battery temperature, current power demand of the electronic control unit, current state of charge of the power battery, and current hydrogen consumption rate of the thermal management object. The current stack temperature, the current power battery temperature, the current power demand of the electronic control unit, the current state of charge of the power battery, and the current hydrogen consumption rate are normalized to obtain a normalized multidimensional state vector. The normalized multidimensional state vectors are combined to form the current state information of the thermal management object.

3. The method as described in claim 2, characterized in that, The step of generating the basic control quantity of the actuator based on the current state information includes: Obtain the preset reference temperature of the fuel cell stack, the reference temperature of the power battery, and the reference state of charge; Calculate the first deviation between the current stack temperature and the stack reference temperature, and the second deviation between the current power battery temperature and the power battery reference temperature; Based on the first deviation, the second deviation, and the current power demand of the electronic control unit, a preset expert baseline control rule is invoked to calculate the basic control quantity of the actuator, which includes a water pump, a fuel cell cooling circuit compressor, and a battery cooling circuit compressor.

4. The method as described in claim 1, characterized in that, The step of inputting the multidimensional state vector into the reinforcement learning policy network to obtain the residual adjustment action includes: The current state information is input into the reinforcement learning policy network, and the reinforcement learning policy network outputs the original action vector. Each element in the original action vector is mapped to a preset continuous interval to obtain the initial residual action; The initial residual action is multiplied by the corresponding preset scaling factor to generate the residual adjustment action.

5. The method as described in claim 1, characterized in that, The step of superimposing the basic control quantity with the residual adjustment action, performing physical boundary limiting and rate of change constraint processing, generating a target control command, and issuing the target control command to the actuator of the thermal management object includes: The residual adjustment action is added to the corresponding control value in the basic control quantity to generate the undetermined control quantity; Physical boundary limiting processing is performed on the undetermined control quantity to limit it within the hardware safe operating range of the actuator, thereby obtaining the limited control quantity; A single-step rate of change constraint is applied to the control quantity after the amplitude limit, the difference between the historical target control command of the previous cycle and the control quantity after the amplitude limit is calculated, and the difference is limited to a preset smooth change range to obtain the target control command of the current cycle. The target control command for the current cycle is sent to the actuator of the thermal management object.

6. The method as described in claim 1, characterized in that, After the step of issuing the target control command to the actuator of the thermally managed object, the method further includes: In the co-simulation environment, the state of the thermal management object is updated according to the target control command, and the updated state is observed. The co-simulation environment includes a physical environment and a control environment. The immediate reward is calculated based on the updated state, wherein the immediate reward includes penalties for exceeding temperature limits, exceeding the charge state limits, and abnormal situations in the solution of the physics model. Experience data is generated based on the current status information, the residual adjustment action, the instant reward, and the updated status, and the experience data is stored in the experience replay pool. The reinforcement learning policy network is trained offline, and the offline training steps include: A physical environment is constructed based on a fuel cell stack model, a power battery model, a cooling circuit model, and an actuator dynamic model, as well as a control environment that performs joint simulation with the physical environment; The reinforcement learning policy network, value network, target value network, and experience replay pool are initialized in the control environment. Returning to the steps of updating the state of the thermal management object according to the target control command and observing the updated state, iterative training is performed on the reinforcement learning policy network and the value network; Based on the empirical data, the preset loss function is minimized, and the network parameters of the reinforcement learning policy network and the value network are updated respectively. The parameters of the value network are copied to the target value network using a soft update method. Training stops when the updated policy is verified to meet the preset safety boundary conditions in the joint simulation environment and the number of abnormal terminations is less than the preset threshold.

7. The method as described in claim 6, characterized in that, The step of updating the network parameters of the reinforcement learning policy network and the value network respectively by minimizing a preset loss function based on the empirical data includes: The experience data in the experience replay pool is randomly sampled to obtain multiple random experience data, each of which includes the current state information, residual adjustment action, instant reward value and updated state; The updated state is input into the target value network, and the expected Q value is calculated by combining it with the instant reward value. The current state information and the residual adjustment action are input into the value network to calculate the predicted Q value; The mean square error between the predicted Q value and the expected Q value is calculated, and the mean square error is minimized by gradient descent to update the parameters of the value network. The negative correlation between the action distribution output by the reinforcement learning policy network and the Q-value output by the value network is calculated, and the negative correlation is minimized by gradient descent to update the parameters of the reinforcement learning policy network.

8. A thermal management collaborative control device for fuel cell vehicles, characterized in that, The fuel cell vehicle thermal management collaborative control device includes: The information acquisition module is used to acquire the current state information of the thermal management object, wherein the current state information is a multi-dimensional state vector; The basic control module is used to generate basic control quantities for the actuator based on the current state information. The residual adjustment module is used to input the multidimensional state vector into the reinforcement learning policy network to obtain residual adjustment actions. The collaborative control module is used to superimpose the basic control quantity and the residual adjustment action, perform physical boundary limiting processing and rate of change constraint processing, generate target control instructions, and send the target control instructions to the actuator of the thermal management object.

9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fuel cell vehicle thermal management cooperative control method as described in any one of claims 1 to 7.

10. A non-transitory storage medium, characterized in that, The non-transient storage medium stores a computer program, which, when executed by a processor, implements the steps of the fuel cell vehicle thermal management collaborative control method as described in any one of claims 1 to 7.

Images