Fuel cell hydrogen circulation cooling integrated system

Abstract

The invention discloses a fuel cell hydrogen circulation cooling integrated system, and relates to the technical field of fuel cells, the fuel cell hydrogen circulation cooling integrated system comprises a hydrogen circulation cooling management center, the hydrogen circulation cooling management center is in communication connection with the following modules: a multi-source state sensing module, which is used for collecting multi-dimensional state data of a hydrogen circulation-cooling system in real time, extracting state characteristics, and sending the state characteristics to a control module; and forming a state feature sequence table. The multi-dimensional state data of the hydrogen circulation-cooling system is collected, the dynamic cooperative control strategy of the reinforcement learning decision module is combined, the precision and stability of temperature control are remarkably improved, the Markov decision process model is used for predicting the instantaneous heat production change caused by load sudden change, and the stability of the system is improved. And through combined adjustment of hydrogen circulation and a cooling system, temperature fluctuation is rapidly restrained, compared with traditional PID control, the temperature overshoot is reduced under the dynamic working condition, it is ensured that the galvanic pile works within the optimal temperature range all the time, the service life of the galvanic pile is prolonged, and the energy conversion efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of fuel cell technology, in particular to a fuel cell hydrogen circulation cooling integrated system. BACKGROUND

[0002] With the global concern for environmental protection and sustainable development, traditional fuel vehicles are gradually replaced by electric vehicles. Hydrogen fuel cell vehicles, as another clean energy driven transportation tool, are gradually regarded as an effective means to cope with climate change and reduce greenhouse gas emissions. Hydrogen fuel cells not only reduce tail gas emissions, but also provide longer driving range and shorter hydrogen refueling speed, becoming one of the key technologies to solve energy and environmental problems. The core working principle of hydrogen fuel cell is to generate electricity and water through the reaction of hydrogen and oxygen. However, the operation of hydrogen fuel cell needs to work stably within a certain temperature range. In practical application, temperature management of fuel cell is particularly important.

[0003] In the prior art, traditional PID control relies on fixed parameters. Due to the lack of predictability of coupled control of hydrogen circulation and cooling system, it is difficult to quickly coordinate the action of multiple actuators when the load suddenly changes, and it is also difficult to adapt to the sudden increase of instantaneous heat generation caused by sudden change of fuel cell load, resulting in temperature overshoot or oscillation. Therefore, how to build a Markov decision process model of hydrogen circulation-cooling joint control with the goal of maximizing temperature stability, and then realize dynamic cooperative control based on reinforcement learning, is the problem to be solved by the present application. Therefore, a fuel cell hydrogen circulation cooling integrated system is proposed. SUMMARY

[0004] The present application aims to provide a fuel cell hydrogen circulation cooling integrated system to solve the problems raised in the background art.

[0005] To solve the above technical problems, the technical solution adopted by the present application is:

[0006] A fuel cell hydrogen circulation cooling integrated system comprises a hydrogen circulation cooling management center, which is communicatively connected with the following modules:

[0007] A multi-source state perception module is used to collect multi-dimensional state data of the hydrogen circulation-cooling system in real time, extract state features, and form a state feature sequence table;

[0008] A Markov decision process modeling module is used to build a Markov decision process model, define state space and action space, and build a multi-objective reward function with temperature stability as the core;

[0009] The reinforcement learning decision module trains a reinforcement learning agent based on a Markov decision process model, generates a collaborative control strategy of the hydrogen circulation-cooling system, and decomposes continuous actions output by the reinforcement learning into independent control instructions of the hydrogen circulation and the cooling system.

[0010] The hydrogen circulation execution control module is used for driving a hydrogen circulation pump frequency converter, adjusting a hydrogen recirculation rate, and controlling a hydrogen injection valve opening degree according to the independent control instruction of the hydrogen circulation output by the reinforcement learning decision module, so as to optimize hydrogen distribution in the stack and realize precise matching of hydrogen supply.

[0011] The cooling system execution control module is used for adjusting an electronic water pump rotating speed and an electric three-way valve opening degree according to the independent control instruction of the cooling system, controlling cooling liquid flow and flow direction, and driving a radiator fan PWM speed regulation, so as to strengthen convection heat exchange, eliminate overshoot phenomenon caused by inertia of the cooling system, and combine PID control and feedforward compensation.

[0012] The further improvement of the technical scheme of the present application is that the multi-source state perception module comprises a state monitoring unit and a feature extraction unit.

[0013] The state monitoring unit is used for collecting state data of the hydrogen circulation system and the cooling system, and pre-processing the state data to form a comprehensive data set.

[0014] The feature extraction unit is used for performing feature analysis on the collected multi-dimensional state data, extracting state features related to hydrogen circulation and cooling from the multi-dimensional state data, and integrating the state features to form a state feature sequence table, wherein the state features include a hydrogen supply fluctuation rate, a stack temperature gradient, a hydrogen-cooling coupling efficiency and a latent heat utilization rate of phase change material.

[0015] The further improvement of the technical scheme of the present application is that in the state monitoring unit, the process of forming the comprehensive data set is:

[0016] The state data of the hydrogen circulation system and the cooling system are collected in real time through a sensor network deployed in the hydrogen circulation system and the cooling system of the fuel cell, wherein the state data of the hydrogen circulation system include a hydrogen circulation pump rotating speed, a hydrogen inlet pressure / flow rate, an internal hydrogen partial pressure of the stack and a water vapor concentration, and the hydrogen supply fluctuation caused by load mutation is captured, and the state data of the cooling system include a cooling liquid temperature / flow rate, a radiator fan rotating speed, a stack surface temperature distribution and a phase change state of phase change material (PCM).

[0017] The original state data are pre-processed, including filtering and denoising, outlier rejection and time sequence alignment, so as to eliminate sensor drift and interference, and a sliding window method is used to extract short-time statistical features to form a standardized time series data set.

[0018] The pretreated state data is classified according to the hydrogen circulation system and the cooling system, and a comprehensive data set is formed.

[0019] The further improvement of the technical scheme of the present application is that, in the feature extraction unit, the process of forming the state feature sequence list is:

[0020] The state data of the hydrogen circulation system and the cooling system collected is time-synchronized, communication delay and sampling period difference are eliminated, and the data stream is divided through a sliding window, so that the state data of each subsystem is in a unified time scale;

[0021] The state data of the hydrogen circulation system and the cooling system after pretreatment is analyzed for features, and state features are extracted, wherein, for the hydrogen circulation system, the hydrogen supply fluctuation rate is calculated, for the cooling system, the temperature gradient of the stack and the latent heat utilization rate of the phase change material are quantified, and at the same time, through the collaborative analysis of the hydrogen utilization rate and the cooling power, the hydrogen-cooling coupling efficiency feature is extracted;

[0022] The extracted hydrogen supply fluctuation rate, stack temperature gradient, hydrogen-cooling coupling efficiency and latent heat utilization rate of the phase change material are normalized to eliminate dimensional differences, and are integrated into a state feature sequence list according to the time dimension, each record containing a time stamp and four types of feature values.

[0023] The further improvement of the technical scheme of the present application is that: the Markov decision process modeling module includes a state space definition unit and a reward function design unit;

[0024] The state space definition unit is used to abstract the dynamic coupling relationship of the hydrogen circulation-cooling system into a Markov decision process model, and map the state features into a discretized state vector, while defining the state space and the action space.

[0025] The reward function design unit is used to construct a multi-objective reward function with temperature stability as the core, including a temperature fluctuation suppression term, a system energy consumption optimization term and an actuator action smoothing term, and a weighted summation method is used to balance the contradictory objectives.

[0026] The further improvement of the technical scheme of the present application is that: the state space definition unit specifically includes:

[0027] The state feature sequence list covering the state features of the hydrogen circulation system and the cooling system is taken as input, the dynamic dependence between variables is quantified through covariance analysis and mutual information calculation, a state transition probability matrix is constructed, a coupled dynamics framework of Markov decision process (MDP) is formed, and it is ensured that the state evolution conforms to the physical constraints;

[0028] Adopting an adaptive binning algorithm for state features, based on feature distribution density and key working condition threshold to divide discrete intervals, mapping continuous values to discrete state symbols, while retaining the dynamic sensitivity of original state features, generating a discretized state vector to reduce model complexity;

[0029] Taking the discrete state vector as the basis to construct a state space, defining the action space as the discrete control instructions of hydrogen circulating pump speed regulation and cooling liquid flow control, combining with system safety boundary constraints to generate an effective action set, ensuring that state transition and action execution meet the real-time and robustness requirements of fuel cell operation.

[0030] The further improvement of the technical scheme of the present application is that the reward function design unit specifically comprises:

[0031] According to the control requirements of the fuel cell system, the multi-objective reward function is decomposed into three reward items, namely the temperature fluctuation suppression item (quantifying thermal stability through stack temperature variance), the system energy consumption optimization item (normalizing hydrogen circulating pump and cooling liquid flow energy consumption), and the actuator action smoothing item (punishing the mutation amplitude of adjacent actions), each reward item is monotonically related to the corresponding target;

[0032] The reward items are normalized to eliminate dimensional differences, the temperature fluctuation suppression item is normalized based on the maximum variance of historical data, the system energy consumption optimization item is calibrated according to the rated power, and the actuator action smoothing item is linearly scaled through the action range, and the weights are dynamically allocated by using the analytic hierarchy process, wherein the weight proportion of the temperature fluctuation suppression item is the highest (≥ 50%), and the weights of the system energy consumption optimization item and the actuator action smoothing item are adaptively adjusted according to the working condition;

[0033] A hard constraint penalty item is introduced, which is forced to be negative when the temperature is out of limit or violates the safety boundary, and then the normalized reward items are weighted and summed according to the weights to construct a total reward function, which is output to the reinforcement learning decision module.

[0034] The further improvement of the technical scheme of the present application is that the reinforcement learning decision module specifically comprises:

[0035] Based on the constructed Markov Decision Process (MDP) model, the reinforcement learning agent is trained, and the policy network parameters are optimized through environmental interaction, so that the reinforcement learning agent learns to generate collaborative control actions that meet the multi-objective constraints, and performs state to continuous action mapping;

[0036] The continuous collaborative actions output by the reinforcement learning agent are mapped to independent control instructions of the hydrogen circulation and cooling system through an action decomposition mechanism, the decomposition rules are designed based on the coupling relationship of the system, and the physical feasibility of the actions is ensured through weighted allocation, while the fidelity of the original collaborative target after decomposition is maintained, avoiding performance degradation caused by action segmentation;

[0037] The independent control instruction is input into the fuel cell system, state feedback is collected in real time, and instant rewards are calculated, the experience replay buffer of the Markov decision process is updated, the reinforcement learning strategy weight is dynamically adjusted based on new data, the action decomposition rule is optimized to adapt to working condition changes, and the robustness and adaptability of the hydrogen circulation-cooling system collaborative control are continuously improved.

[0038] Further improvement of the technical scheme of the present application is that the hydrogen circulation execution control module specifically comprises:

[0039] The hydrogen circulation execution control module receives the hydrogen circulation independent control instruction output by the reinforcement learning decision module, accurately analyzes the control instruction, and determines the hydrogen circulation pump frequency converter drive parameter and hydrogen injection valve opening degree set value.

[0040] According to the analyzed control instruction, the hydrogen circulation pump frequency converter is driven, the running frequency is adjusted according to the set parameter, the hydrogen recirculation rate is changed, and the hydrogen injection valve is controlled so that its opening degree reaches the set value, and the hydrogen distribution in the stack is optimized.

[0041] During the adjustment process, the hydrogen circulation related parameters are continuously monitored, the hydrogen supply matching condition is evaluated, and the actual effect is fed back to the reinforcement learning decision module so as to adjust the strategy and realize the continuous and accurate matching of hydrogen supply.

[0042] Further improvement of the technical scheme of the present application is that the cooling system execution control module specifically comprises:

[0043] The control instruction output by the reinforcement learning module is received, the electronic water pump target rotating speed, the electric three-way valve opening degree and the radiator fan PWM duty cycle are analyzed, and dynamic clipping is performed according to the equipment physical limit, so that the control instruction is within the safe operation range, and hardware overload or mechanical damage is avoided.

[0044] The water pump rotating speed and the valve opening degree are adjusted based on the PID algorithm, the change of cooling demand is predicted combining with the feedforward compensation model, the control amount is adjusted in advance to offset the system inertia, the fan rotating speed is driven through the PWM signal, the radiator convection heat exchange efficiency is strengthened, and the coolant flow distribution and temperature uniformity are simultaneously optimized.

[0045] The key parameters of the coolant temperature, flow and pressure are monitored in real time, the deviation is calculated by comparing the target value, if overshoot or response lag is detected, the PID parameter adaptive adjustment or feedforward-feedback composite control strategy is adopted, the actuator output is fine-tuned, the cooling system is ensured to quickly and stably converge to the target state, and the execution effect data is fed back to the reinforcement learning decision module.

[0046] Due to the adoption of the above technical scheme, the present application has the following technical progress compared with the prior art:

[0047] 1. This invention provides an integrated hydrogen cycle cooling system for fuel cells. By collecting multi-dimensional state data of the hydrogen cycle-cooling system and combining it with a dynamic collaborative control strategy of a reinforcement learning decision module, the accuracy and stability of temperature control are significantly improved. The Markov decision process model is used to predict instantaneous heat generation changes caused by load mutations, and temperature fluctuations are quickly suppressed through joint regulation of the hydrogen cycle and cooling system. Compared with traditional PID control, the temperature overshoot is reduced under dynamic operating conditions, ensuring that the fuel cell stack always operates within the optimal temperature range, extending its service life and improving energy conversion efficiency.

[0048] 2. This invention provides an integrated hydrogen circulation cooling system for fuel cells. By constructing a multi-objective reward function with temperature stability as the core, the system ensures thermal management performance while optimizing the energy consumption of the hydrogen circulation pump and cooling system. Furthermore, the reinforcement learning agent dynamically adjusts the hydrogen circulation rate, coolant flow rate, and fan speed to reduce ineffective power consumption and lower overall parasitic power. In addition, the quantitative evaluation of the latent heat utilization rate of phase change materials further improves the heat recovery efficiency and significantly enhances the overall energy efficiency. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0050] Figure 1 This is a schematic diagram of the working process of a fuel cell hydrogen cycle cooling integrated system according to the present invention;

[0051] Figure 2 This is a data flow diagram of the system functional modules of a fuel cell hydrogen cycle cooling integrated system according to the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1, such as Figure 1 , Figure 2 As shown, the present invention provides an integrated hydrogen cycle cooling system for fuel cells, including a hydrogen cycle cooling management center, which is communicatively connected to the following modules:

[0054] a multi-source state perception module for collecting multi-dimensional state data of the hydrogen circulation-cooling system in real time and extracting state features to form a state feature sequence table, the multi-source state perception module including a state monitoring unit and a feature extraction unit;

[0055] The state monitoring unit is configured to collect state data of the hydrogen circulation system and the cooling system, wherein the state data of the hydrogen circulation system includes hydrogen circulation pump speed, hydrogen inlet pressure / flow rate, hydrogen partial pressure inside the stack, water vapor concentration, and the like, and captures hydrogen supply fluctuations caused by load mutations, and the state data of the cooling system includes coolant temperature / flow rate, radiator fan speed, stack surface temperature distribution, and phase change material (PCM) phase change state, and the like, and performs preprocessing to form a comprehensive data set. The state data of the hydrogen circulation system and the cooling system is collected in real time through a sensor network deployed in the hydrogen circulation system and the cooling system of the fuel cell, wherein the state data of the hydrogen circulation system includes hydrogen circulation pump speed, hydrogen inlet pressure / flow rate, hydrogen partial pressure inside the stack, water vapor concentration, and the like, and captures hydrogen supply fluctuations caused by load mutations, and the state data of the cooling system includes coolant temperature / flow rate, radiator fan speed, stack surface temperature distribution, and phase change material (PCM) phase change state, and the like. The original state data is preprocessed, including filtering and denoising, outlier rejection, and time series alignment, to eliminate sensor drift and interference, and a sliding window method is used to extract short-time statistical features to form a standardized time series data set. The preprocessed state data is classified according to the hydrogen circulation system and the cooling system to form a comprehensive data set.

[0056] The specific working content of the state monitoring unit is: through the deployment of the distributed sensor network in the fuel cell hydrogen circulation system and the cooling system, real-time acquisition of full-dimensional state data is realized, for the hydrogen circulation system, dynamic parameters such as hydrogen circulation pump rotating speed, hydrogen inlet pressure and flow rate are monitored, in order to capture the instantaneous fluctuation of hydrogen supply caused by load mutation, at the same time, through the hydrogen partial pressure and water vapor concentration sensor inside the stack, the hydrogen utilization efficiency and the humidity change inside the stack are quantified, on the cooling system side, the sensor network covers key indicators such as cooling liquid temperature, flow rate and radiator fan rotating speed, and real-time reflects the thermodynamic state of the cooling medium, the stack surface temperature distribution is obtained through the array type temperature sensor, combined with the phase change state monitoring of the phase change material (PCM), the dynamic response atlas of the cooling system to the stack thermal management is constructed; the collected original state data is subjected to multi-stage pretreatment to eliminate noise and interference, an adaptive filtering algorithm is used to denoise the sensor signal, to suppress the random fluctuation caused by electromagnetic interference and mechanical vibration, based on statistical threshold, abnormal values are eliminated to avoid data distortion caused by sensor failure or transient impact, through the time sequence alignment algorithm, the time stamps of multi-sensor data are synchronized, to eliminate the time sequence misalignment caused by communication delay or sampling period difference, after pretreatment, the data enters the short-time feature extraction stage: statistical features are calculated by using the sliding window method, including mean, variance, maximum / minimum difference and slope, to quantify the instantaneous change trend of the state parameter, at the same time, combined with Fourier transform or wavelet analysis, frequency domain features are extracted to capture periodic fluctuation patterns, to form a standardized time series data set; the pretreated data needs to be classified according to the subsystem, and fused into a comprehensive data set according to the time dimension;

[0057] The feature extraction unit is used for feature analysis on the collected multi-dimensional state data, to extract hydrogen circulation and cooling related state features, and to integrate them into a state feature sequence list, wherein the state features include hydrogen supply fluctuation rate, stack temperature gradient, hydrogen-cooling coupling efficiency and latent heat utilization rate of phase change material, the collected state data of the hydrogen circulation system and the cooling system are time sequence synchronized to eliminate communication delay and sampling period difference, and the data stream is divided by using the sliding window, so that the state data of each subsystem is under the unified time scale, the pretreated state data of the hydrogen circulation system and the cooling system are subjected to feature analysis, to extract state features, wherein for the hydrogen circulation system, the hydrogen supply fluctuation rate is calculated, for the cooling system, the stack temperature gradient and the latent heat utilization rate of the phase change material are quantified, at the same time, through the collaborative analysis of hydrogen utilization rate and cooling power, the hydrogen-cooling coupling efficiency feature is extracted, the extracted hydrogen supply fluctuation rate, stack temperature gradient, hydrogen-cooling coupling efficiency and latent heat utilization rate of phase change material are normalized to eliminate dimension difference, and integrated into a state feature sequence list according to the time dimension, each record contains a time stamp and four types of feature values;

[0058] The feature extraction unit specifically works as follows: the state data of the hydrogen circulation system and the cooling system are time-synchronized to eliminate time sequence misalignment caused by communication delay and sampling period difference. Since the sampling periods and communication protocols of the sensors of the two types of subsystems are different, the time deviation needs to be eliminated through hardware clock synchronization (IEEE 1588 protocol) and software interpolation algorithm. The hardware synchronization ensures that the clock error of all sensors is less than 1 μs, and the software layer re-samples the low-frequency signals to a high-frequency time reference by using cubic spline interpolation, so as to realize time stamp alignment. The data stream under the unified time scale is divided by using the sliding window method, the window length is set according to the dynamic characteristics of the system, and the step length is 50% of the window length, so as to ensure that the feature extraction covers the local dynamic change while retaining the time sequence continuity. On the basis of time synchronization, the state features of the hydrogen circulation system and the cooling system are calculated respectively. For the hydrogen circulation system, the dynamic fluctuation of hydrogen supply is quantified, the standard deviation and the mean value of the hydrogen inlet flow in the sliding window are calculated, and the hydrogen supply fluctuation rate is obtained, which reflects the instantaneous supply instability degree caused by load mutation. For the cooling system, the temperature gradient of the battery is quantified by the ratio of the maximum temperature difference to the average temperature of the surface temperature sensor array, which represents the uniformity of heat distribution. The latent heat utilization rate of the phase change material (PCM) is calculated by combining the phase change temperature threshold and the temperature-time integral, which evaluates the actual contribution of the PCM in thermal management. The hydrogen-cooling coupling efficiency feature is extracted by covariance analysis of the hydrogen utilization rate (calculated based on hydrogen partial pressure and flow) and the cooling power (calculated based on cooling liquid temperature difference and flow), which reveals the dynamic synergistic relationship between hydrogen supply and thermal management. The hydrogen supply fluctuation rate, the battery temperature gradient, the hydrogen-cooling coupling efficiency and the latent heat utilization rate of the phase change material are normalized. The state features of the hydrogen circulation system are normalized by Min-Max normalization, which is mapped to the [0, 1] interval, and the relative change range is retained. The state features of the cooling system are dynamically normalized in combination with the design boundary, which highlights the sensitivity in key operating conditions. The binary feature (PCM phase change state) is converted to a numerical form by one-hot encoding. After normalization, the four types of feature values are integrated in the time dimension to generate a state feature sequence table, each record containing a unified time stamp and a corresponding feature value vector.

[0059] The Markov decision process modeling module is configured to construct a Markov decision process model, define a state space and an action space, and construct a multi-objective reward function with temperature stability as the core. The Markov decision process modeling module includes a state space definition unit and a reward function design unit.

[0060] The state space defining unit is configured to abstract the dynamic coupling relationship of the hydrogen circulation-cooling system into a Markov decision process model, map state features into a discretized state vector, define state space and action space, take a state feature sequence table covering state features of the hydrogen circulation system and the cooling system as input, quantize dynamic dependency between variables through covariance analysis and mutual information calculation, construct a state transition probability matrix, form a coupled dynamics framework of Markov decision process (MDP), ensure that state evolution conforms to physical constraints, adopt an adaptive binning algorithm for state features, divide discrete intervals based on feature distribution density and key working condition thresholds, map continuous values into discrete state symbols while retaining dynamic sensitivity of original state features, generate a discretized state vector to reduce model complexity, construct state space based on the discretized state vector, define the action space as discrete control instructions of hydrogen circulation pump speed regulation and cooling liquid flow control, generate an effective action set in combination with system safety boundary constraints, and ensure that state transition and action execution meet real-time and robustness requirements of fuel cell operation.

[0061] The state space definition unit specifically works as follows: integrating the state feature sequence table of the hydrogen circulation system and the cooling system, quantifying the dynamic dependency between variables through covariance analysis and mutual information, wherein the covariance analysis focuses on linear correlation, calculates the covariance matrix between features and extracts principal components to reveal the collaborative change trend of hydrogen supply volatility and cooling power, and the mutual information measures the nonlinear correlation strength through KL divergence to capture the implicit coupling relationship between the temperature gradient of the stack and the latent heat utilization rate of the PCM, and then constructs a state transition probability matrix to define the conditional probability distribution of the system transitioning from the current state to the next state, forming a coupled dynamics framework of Markov Decision Process (MDP); the state space definition unit uses an adaptive binning algorithm to discretize the state features, reducing the complexity of the continuous state space, and the adaptive binning algorithm dynamically divides the intervals based on feature distribution density and key operating condition thresholds: for high-density areas, equal frequency binning is used to retain local dynamic details, and for low-density areas, forced segmentation is performed in combination with design boundaries (the maximum tolerance temperature of the stack) to ensure the sensitivity of key operating conditions. The binning results are verified for interval independence through chi-square test to avoid information loss caused by excessive discretization. Finally, each continuous state feature is mapped to a discrete state symbol to generate a discretized state vector; based on the discretized state vector, a finite state set is constructed, and the action space is defined as a discrete instruction set for hydrogen circulation pump speed adjustment and cooling liquid flow control, wherein the action design of the action space follows the fuel cell control logic: the hydrogen circulation pump speed is divided into low, medium and high three gears to meet different hydrogen supply requirements; the cooling liquid flow control adopts three-level regulation of minimum flow, rated flow and maximum flow to match the stack heat load change, and at the same time, the safety boundary constraints of the integrated system, i.e. the lower limit of hydrogen concentration and the upper limit of cooling liquid pressure, are integrated to filter invalid action combinations and generate an effective action set;

[0062] The reward function design unit is configured to construct a multi-objective reward function with temperature stability as the core, including a temperature fluctuation suppression term, a system energy consumption optimization term, and an actuator action smoothing term. A weighted summation method is used to balance the contradictory objectives. According to the control requirements of the fuel cell system, the multi-objective reward function is decomposed into three reward terms, namely the temperature fluctuation suppression term (quantifying thermal stability through stack temperature variance), the system energy consumption optimization term (normalizing hydrogen circulation pump and coolant flow energy consumption), and the actuator action smoothing term (penalizing the mutation amplitude of actions at adjacent time points). Each reward term is monotonically related to the corresponding objective. The temperature fluctuation suppression term uses a negative exponential function to enhance sensitivity, the system energy consumption optimization term uses an inverse form to avoid numerical overflow, and the actuator action smoothing term uses action difference absolute value integration. Each reward term is normalized to eliminate dimensional differences. The temperature fluctuation suppression term is normalized based on historical data statistics maximum variance, the system energy consumption optimization term is calibrated according to the rated power, and the actuator action smoothing term is linearly scaled by the action range. The weights are dynamically allocated using the analytic hierarchy process. The temperature fluctuation suppression term has the highest weight proportion (≥ 50%), and the weights of the system energy consumption optimization term and the actuator action smoothing term are adjusted adaptively according to the working conditions. A hard constraint penalty term is introduced, which is forced to be negative when the temperature exceeds the limit or violates the safety boundary. Then, the normalized reward terms are weighted and summed according to the weights to construct the total reward function, which is output to the reinforcement learning decision module.

[0063] The reward function design unit specifically works as follows: the control target of the fuel cell system is divided into three independent reward items, corresponding to thermal management stability, energy efficiency optimization and actuator life protection, i.e. temperature fluctuation suppression item, system energy consumption optimization item and actuator action smoothing item. The temperature fluctuation suppression item maps the variance of the stack surface temperature sequence by using a negative exponential function, so that the reward is highly sensitive to temperature fluctuations, and a slight deviation can trigger significant punishment. The system energy consumption optimization item is based on the real-time power consumption of the hydrogen circulating pump and the cooling system, and is inversely normalized based on the rated power to avoid numerical truncation problems in high energy consumption conditions. The actuator action smoothing item suppresses mechanical wear caused by frequent action switching by integrating the absolute value of the difference between adjacent control instructions. Each reward item has a monotonic relationship with the target, ensuring consistency of the optimization direction. Each reward item is normalized to eliminate dimensional differences. The temperature fluctuation suppression item is linearly scaled to the [0, 1] interval based on the historical maximum temperature variance. The system energy consumption optimization item is directly calibrated with the rated power as the upper limit. The actuator action smoothing item is linearly normalized according to the physical range of the actuator. The weight distribution uses the analytic hierarchy process, prioritizing temperature stability (fixed weight ≥ 50%), and the remaining weight is dynamically adjusted according to the working condition: increasing the weight of the system energy consumption optimization item under high load, and focusing on action smoothness in steady state conditions. A hard constraint penalty item is introduced to ensure system safety. When the stack temperature exceeds the design threshold or the hydrogen concentration / cooling liquid pressure violates the safety boundary, a maximum negative reward is directly assigned. This penalty item has the highest priority and overrides all other reward items. Finally, the total reward function integrates each normalized item by weighted summation, and outputs the total reward to the reinforcement learning decision module as a direct feedback signal for policy optimization.

[0064] The reinforcement learning decision module trains a reinforcement learning agent based on a Markov decision process model, generates a coordinated control strategy for the hydrogen circulation-cooling system, and decomposes the continuous action output by the reinforcement learning into independent control instructions for the hydrogen circulation and the cooling system.

[0065] The hydrogen circulation execution control module is used to drive the hydrogen circulation pump frequency converter, adjust the hydrogen recirculation rate, and control the hydrogen injection valve opening degree according to the independent control instructions for the hydrogen circulation output by the reinforcement learning decision module, so as to optimize the hydrogen distribution inside the stack and achieve precise matching of hydrogen supply.

[0066] The cooling system execution control module is used to adjust the electronic water pump speed and the electric three-way valve opening degree according to the independent control instructions for the cooling system, control the cooling liquid flow and flow direction, and drive the radiator fan PWM speed regulation to strengthen the convection heat transfer, and eliminate the overshoot phenomenon caused by the inertia of the cooling system through PID control and feedforward compensation.

[0067] In embodiment 2, as Figure 1 , Figure 2As shown, on the basis of embodiment 1, the application provides a technical solution: preferably, the reinforcement learning decision module specifically comprises:

[0068] Based on the constructed Markov decision process (MDP) model, the reinforcement learning agent is trained, the policy network parameters are optimized through environment interaction, the collaborative control action meeting the multi-objective constraints is learned and generated, the mapping from state to continuous action is performed, the continuous collaborative action output by the reinforcement learning agent is mapped into independent control instructions of the hydrogen circulation and cooling system through the action decomposition mechanism, the decomposition rules are designed based on the coupling relationship of the system, the physical feasibility of the action is ensured through weighted allocation, the loyalty of the decomposed instructions to the original collaborative target is maintained, the performance degradation caused by action segmentation is avoided, the independent control instructions are input into the fuel cell system for execution, the state feedback is collected in real time and the immediate reward is calculated, the experience replay buffer of the Markov decision process is updated, the reinforcement learning strategy weight is dynamically adjusted based on the new data, the action decomposition rule is optimized to adapt to the working condition change, and the robustness and adaptability of the hydrogen circulation-cooling system collaborative control are continuously improved;

[0069] The specific work of the reinforcement learning decision module is: based on the constructed Markov decision process (MDP) model, the policy network parameters are optimized through environmental interaction, and the mapping relationship from the system state to the continuous control action is learned, wherein the state includes key parameters such as stack temperature, hydrogen concentration and cooling liquid pressure, and the action space is a continuous collaborative control instruction. During the training process, the reinforcement learning agent interacts with the fuel cell system environment through the exploration-exploitation mechanism, collects state transition samples and calculates the immediate reward, breaks the data correlation using the experience replay mechanism, updates the policy network weight using the gradient descent algorithm, the policy network adopts a deep neural network structure, which fits the complex state-action relationship through nonlinear transformation, and outputs the continuous collaborative action that meets the multi-objective constraints; the continuous collaborative action output by the policy network is mapped to the independent control instructions of the hydrogen circulation system and the cooling system through the action decomposition mechanism, the decomposition rule is designed based on the coupling relationship of the system, considering the physical constraints of the influence weight of the hydrogen circulation pump speed on the stack temperature and the contribution proportion of the cooling liquid flow to the heat dissipation efficiency, and using the weighted allocation method to ensure the physical feasibility of the decomposed instructions, at the same time, the decomposition process needs to maintain the loyalty to the original collaborative goal, and the loyalty constraint term is introduced to prevent performance degradation caused by action segmentation, in addition, the decomposition rule supports dynamic adjustment, which can optimize the weight allocation in real time according to the working condition change, ensuring that the independent instructions can still effectively collaborate in complex scenarios; after the independent control instructions are input into the fuel cell system for execution, real-time state feedback is collected, and the immediate reward is calculated based on the preset reward function to evaluate the degree of satisfaction of the current action to the multi-objective constraints, the new data is stored in the experience replay buffer for updating the state transition probability distribution of the Markov decision process model, and the dynamic adjustment of the policy network weight is triggered, during the optimization process, the priority of each target in the reward function is dynamically adjusted according to the working condition characteristics through the meta-learning mechanism, to ensure the robustness of the policy, and the optimization of the action decomposition rule is realized through the reinforcement learning subtask, taking the collaborative effect of the decomposed instructions as the feedback signal, iteratively updating the rule parameters, and finally forming an efficient collaborative control strategy that adapts to all working conditions;

[0070] The hydrogen circulation execution control module specifically includes:

[0071] The hydrogen circulation execution control module receives the hydrogen circulation independent control instruction output by the reinforcement learning decision module, accurately analyzes the control instruction, and determines the hydrogen circulation pump frequency converter drive parameter and hydrogen injection valve opening degree set value. According to the analyzed control instruction, the hydrogen circulation pump frequency converter is driven, the running frequency is adjusted according to the set parameter, the hydrogen recirculation rate is changed, and the hydrogen injection valve is controlled to make its opening degree reach the set value, so as to optimize the hydrogen distribution in the stack. During the adjustment process, the hydrogen circulation related parameters are continuously monitored, the hydrogen supply matching condition is evaluated, and the actual effect is fed back to the reinforcement learning decision module, so as to adjust the strategy and realize the continuous and accurate matching of hydrogen supply;

[0072] The hydrogen circulation execution control module specifically performs the following work: the hydrogen circulation execution control module receives the hydrogen circulation independent control instruction output by the reinforcement learning decision module, converts it into executable bottom-layer equipment control parameters, the control instruction includes the target rotating speed of the hydrogen circulation pump and the target opening of the hydrogen injection valve, dynamically calibrates according to the current state of the system to ensure the physical feasibility of the instruction, and the analysis process needs to consider the safe operation range of the equipment, for example, the rotating speed of the hydrogen circulation pump needs to be limited within the rated range, while the opening of the injection valve needs to be optimized by combining the hydrogen flow demand, after the analysis is completed, specific frequency converter driving signals and valve position control instructions are generated and sent to the execution mechanism; after completing the instruction analysis, the hydrogen circulation pump is driven by the frequency converter, and its rotating speed is adjusted according to the set frequency, so as to change the hydrogen recirculation rate, ensure that the anode hydrogen supply matches the load demand of the stack, at the same time, the opening of the hydrogen injection valve is adjusted in real time according to the analysis value, the hydrogen distribution in the stack flow channel is optimized, and local starvation is avoided, during the adjustment process, key parameters such as the actual rotating speed of the circulation pump, the feedback opening of the injection valve, the hydrogen flow and pressure fluctuation are monitored in real time to ensure the execution accuracy, if a deviation is detected between the actual value and the set value, a feedforward-feedback composite control strategy is used for dynamic compensation, the frequency converter output is fine-tuned through the PID algorithm or the valve position instruction is corrected in advance through the predictive control to reduce the influence of the response lag; after the adjustment is completed, the hydrogen supply effect is comprehensively evaluated, the data of the hydrogen circulation efficiency, the hydrogen concentration distribution in the stack and the anode pressure fluctuation are collected through the sensor network, the actual effect of the current control instruction is quantified, the evaluation indexes include the hydrogen utilization rate, the supply stability and the synergistic effect with the cooling system, the monitoring data are compared and analyzed with the target value, an execution effect report is generated and fed back to the reinforcement learning decision module as a key basis for strategy optimization, if it is found that there is a persistent mismatch between the hydrogen supply and demand, i.e., the flow fluctuation is out of limit or the concentration is uneven, real-time alarm is triggered and the decision module is suggested to adjust the control strategy;

[0073] The cooling system execution control module specifically includes:

[0074] The control instruction output by the receiving reinforcement learning module is parsed to analyze the target rotating speed of the electronic water pump, the opening of the electric three-way valve and the PWM duty cycle of the radiator fan, and the control instruction is dynamically clipped according to the physical limit of the equipment, so that the control instruction is in the safe operation interval, avoiding hardware overload or mechanical damage. The water pump rotating speed and the valve opening are adjusted based on the PID algorithm, the change of the cooling demand is predicted based on the feedforward compensation model, the control quantity is adjusted in advance to offset the system inertia, the fan rotating speed is driven through the PWM signal, the convection heat exchange efficiency of the radiator is strengthened, the cooling liquid flow distribution and the temperature uniformity are simultaneously optimized, the key parameters of the cooling liquid temperature, flow and pressure are monitored in real time, the deviation is calculated by comparing the target value, if the overshoot or response lag is detected, the PID parameter adaptive adjustment or the feedforward-feedback composite control strategy is adopted, the actuator output is fine-tuned, the cooling system is ensured to quickly and stably converge to the target state, and the execution effect data is generated and fed back to the reinforcement learning decision module;

[0075] The specific working content of the cooling system execution control module is that the cooling system execution control module receives the continuous control instructions output by the reinforcement learning decision module, including the electronic water pump target rotating speed, the electric three-way valve opening degree and the radiator fan PWM duty cycle. In the analysis process, the instructions are physically checked for feasibility, and are dynamically cropped in combination with the safe operation range of the equipment. The rotating speed of the electronic water pump needs to be limited within the rated range to avoid mechanical wear caused by overspeed. The opening degree of the electric three-way valve is optimized according to the cooling liquid flow demand to prevent pressure from suddenly changing due to extreme opening degree. The fan PWM duty cycle is adjusted in combination with the radiator heat load curve to ensure that the convective heat exchange efficiency is maximized. The cropped control instructions need to meet the dynamic constraints of the cooling system, that is, the flow-pressure balance relationship and the temperature gradient limit. At the same time, the slope limiting algorithm is used to smooth the instruction mutation to avoid frequent action of the actuator. After the analysis is completed, the corresponding frequency converter driving signal, valve position control instruction and PWM waveform are generated and sent to the lower layer execution mechanism. Based on the analyzed control instructions, the cooling system execution control module adjusts the rotating speed of the electronic water pump and the opening degree of the three-way valve by using the PID algorithm to realize accurate control of the cooling liquid flow. A feedforward compensation model is introduced to predict the change of cooling demand according to the heat generation rate of the battery pack and to adjust the control quantity in advance. The radiator fan is driven by the PWM signal, and the duty cycle is nonlinearly mapped with the heat load to strengthen the convective heat exchange efficiency. At the same time, the three-way valve proportionally distributes the cooling liquid flow direction to optimize the uniformity of the battery pack temperature. During the execution process, the inlet and outlet temperatures, flow and pressure parameters of the cooling liquid are monitored in real time, and the control output is dynamically corrected through feedback loop. If overshoot or response lag is detected, the PID parameter adaptive adjustment or the feedforward-feedback composite control strategy is triggered to ensure that the system quickly and stably converges to the target state. The cooling system execution control module collects the key parameters after execution in real time, including the cooling liquid temperature drop gradient, flow stability and pressure fluctuation rate, quantifies the actual effect of the current control instruction, calculates the deviation by comparing the target value, evaluates the cooling power, temperature uniformity and energy consumption indicators, and generates an abnormal alarm and feeds back to the reinforcement learning decision module if the deviation continuously exceeds the limit to trigger strategy optimization. The execution effect data is uploaded in a standardized format, including the control instruction, actual output and environmental state variable, to provide high-confidence samples for strategy iteration. At the same time, the module supports dynamic adjustment of the control mode, switches to the preset safety strategy in the case of hardware failure or extreme working conditions to ensure the robustness of the system.

[0076] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A fuel cell hydrogen circulation cooling integrated system comprising a hydrogen circulation cooling management center, characterized in that, The hydrogen circulation cooling management center is communicatively connected with the following modules: A multi-source state perception module, configured to collect multi-dimensional state data of the hydrogen circulation-cooling system in real time, and extract state features to form a state feature sequence list; A Markov decision process modeling module, configured to construct a Markov decision process model, define a state space and an action space, and construct a multi-objective reward function with temperature stability as the core; A reinforcement learning decision module, configured to train a reinforcement learning agent based on the Markov decision process model, generate a collaborative control strategy for the hydrogen circulation-cooling system, and decompose continuous actions output by the reinforcement learning into independent control instructions for the hydrogen circulation system and the cooling system; A hydrogen circulation execution control module, configured to drive a hydrogen circulation pump frequency converter and control a hydrogen injection valve opening degree according to the independent control instructions for the hydrogen circulation system output by the reinforcement learning decision module; A cooling system execution control module, configured to adjust an electronic water pump speed and an electric three-way valve opening degree, and drive a radiator fan PWM speed regulation according to the independent control instructions for the cooling system.

2. The hydrogen circulation cooling integrated system of a fuel cell according to claim 1, characterized in that: The multi-source state perception module includes a state monitoring unit and a feature extraction unit; The state monitoring unit is configured to collect state data of the hydrogen circulation system and the cooling system, and perform preprocessing to form a comprehensive data set; The feature extraction unit is configured to perform feature analysis on the collected multi-dimensional state data, extract state features related to hydrogen circulation cooling, and integrate to form a state feature sequence list, wherein the state features include hydrogen supply fluctuation rate, stack temperature gradient, hydrogen-cooling coupling efficiency, and latent heat utilization rate of phase change material.

3. The hydrogen circulation cooling integrated system of a fuel cell according to claim 2, characterized in that: In the state monitoring unit, the process of forming a comprehensive data set is as follows: Through the sensor network deployed in the hydrogen circulation system and the cooling system of the fuel cell, the state data of the hydrogen circulation system and the cooling system are collected in real time, wherein the state data of the hydrogen circulation system includes hydrogen circulation pump speed, hydrogen inlet pressure / flow, internal hydrogen partial pressure of the stack, and water vapor concentration, the hydrogen supply fluctuation caused by load mutation is captured, and the state data of the cooling system includes cooling liquid temperature / flow, radiator fan speed, stack surface temperature distribution, and phase change state of phase change material; The original state data is preprocessed, including filtering and denoising, outlier rejection, and time series alignment, and the sliding window method is used to extract short-time statistical features to form a standardized time series data set; The preprocessed state data is classified according to the hydrogen circulation system and the cooling system to form a comprehensive data set.

4. The hydrogen circulation cooling integrated system of a fuel cell according to claim 3, characterized in that: In the feature extraction unit, the process of forming a state feature sequence list is as follows: The collected state data of the hydrogen circulation system and the cooling system are time-synchronized, and the data stream is divided by the sliding window method, so that the state data of each subsystem is in a unified time scale; The preprocessed state data of the hydrogen circulation system and the cooling system are analyzed to extract state features, wherein for the hydrogen circulation system, the hydrogen supply fluctuation rate is calculated, for the cooling system, the stack temperature gradient and the latent heat utilization rate of the phase change material are quantified, and simultaneously, the hydrogen-cooling coupling efficiency feature is extracted through the collaborative analysis of hydrogen utilization rate and cooling power; The extracted hydrogen supply fluctuation rate, stack temperature gradient, hydrogen-cold coupling efficiency and latent heat utilization rate of the phase change material are normalized and integrated into a state feature sequence table according to a time dimension, and each record contains a time stamp and four types of feature values.

5. The hydrogen circulation cooling integrated system of a fuel cell according to claim 2, characterized in that: The Markov decision process modeling module includes a state space definition unit and a reward function design unit. The state space definition unit is configured to abstract the dynamic coupling relationship of the hydrogen circulation-cooling system into a Markov decision process model, map the state features to a discretized state vector, and define the state space and the action space. The reward function design unit is configured to construct a multi-objective reward function with temperature stability as the core, including a temperature fluctuation suppression term, a system energy consumption optimization term and an actuator action smoothing term.

6. The hydrogen circulation cooling integrated system of a fuel cell according to claim 5, characterized in that: The state space definition unit specifically includes: The state feature sequence table covering the state features of the hydrogen circulation system and the cooling system is taken as input, the dynamic dependence between variables is quantified through covariance analysis and mutual information calculation, a state transition probability matrix is constructed, and a coupled dynamics framework of the Markov decision process is formed; An adaptive binning algorithm is used for the state features, and based on feature distribution density and key working condition threshold, discrete intervals are divided, continuous values are mapped to discrete state symbols, and a discretized state vector is generated; The discretized state vector is used to construct the state space, the action space is defined as the discrete control instructions of the hydrogen circulation pump speed regulation and the cooling liquid flow control, and the effective action set is generated in combination with the system safety boundary constraint.

7. The hydrogen circulation cooling integrated system of a fuel cell according to claim 5, characterized in that: The reward function design unit specifically includes: According to the control requirements of the fuel cell system, the multi-objective reward function is decomposed into three reward terms, namely the temperature fluctuation suppression term, the system energy consumption optimization term and the actuator action smoothing term, each reward term is monotonically related to the corresponding target; Each reward term is normalized, the temperature fluctuation suppression term is normalized based on the maximum variance of historical data, the system energy consumption optimization term is calibrated according to the rated power, the actuator action smoothing term is linearly scaled through the action range, and the weights are dynamically allocated by using the analytic hierarchy process; A hard constraint penalty term is introduced, the reward is forced to be negative when the temperature exceeds the limit or violates the safety boundary, and then the normalized reward terms are weighted and summed according to the weights to construct the total reward function, which is output to the reinforcement learning decision module.

8. The hydrogen circulation cooling integrated system of a fuel cell according to claim 5, characterized in that: The reinforcement learning decision module specifically includes: Based on the constructed Markov decision process model, a reinforcement learning agent is trained, the policy network parameters are optimized through environmental interaction, the agent learns to generate collaborative control actions that meet the multi-objective constraints, and the state is mapped to continuous actions; The continuous collaborative actions output by the reinforcement learning agent are mapped to independent control instructions of the hydrogen circulation and cooling systems through an action decomposition mechanism, the decomposition rules are designed based on the system coupling relationship, and the fidelity of the decomposed instructions to the original collaborative target is maintained; The independent control instructions are input to the fuel cell system for execution, state feedback is collected in real time and the immediate reward is calculated, the experience replay buffer of the Markov decision process is updated, the reinforcement learning strategy weights are dynamically adjusted based on the new data, and the action decomposition rules are optimized to adapt to the working condition changes.

9. The hydrogen circulation cooling integrated system of a fuel cell according to claim 8, characterized in that: The hydrogen circulation execution control module specifically includes: The hydrogen circulation execution control module receives the hydrogen circulation independent control instruction output by the reinforcement learning decision module, accurately analyzes the control instruction, and determines the hydrogen circulation pump frequency converter drive parameter and the hydrogen injection valve opening degree set value; According to the analyzed control instruction, the hydrogen circulation pump frequency converter is driven, the running frequency is adjusted according to the set parameter, the hydrogen recirculation rate is changed, and the hydrogen injection valve is controlled so that the opening degree reaches the set value; During the adjustment process, the hydrogen circulation related parameters are continuously monitored, the hydrogen supply matching condition is evaluated, and the actual effect is fed back to the reinforcement learning decision module.

10. The hydrogen circulation cooling integrated system of a fuel cell according to claim 8, characterized in that: The cooling system execution control module specifically includes: Receiving the control instruction output by the reinforcement learning module, analyzing the electronic water pump target rotating speed, the electric three-way valve opening degree and the radiator fan PWM duty cycle, dynamically clipping according to the equipment physical limit, so that the control instruction is within the safe operation interval; Adjusting the water pump rotating speed and the valve opening degree based on the PID algorithm, combining the feedforward compensation model to predict the cooling demand change, adjusting the control quantity in advance to offset the system inertia, driving the fan rotating speed through the PWM signal, strengthening the radiator convection heat exchange efficiency, and synchronously optimizing the cooling liquid flow distribution and temperature uniformity; Real-time monitoring of the key parameters of the cooling liquid temperature, flow and pressure, comparing the target value to calculate the deviation, if overshoot or response lag is detected, using the PID parameter adaptive adjustment or the feedforward-feedback composite control strategy to fine-tune the actuator output, and generating the execution effect data feedback to the reinforcement learning decision module.

Images