Ship fuel cell cooling system energy efficiency management method based on deep reinforcement learning

Abstract

The invention discloses a ship fuel cell cooling system energy efficiency management method based on deep reinforcement learning, and relates to the technical field of ship energy management, and the method comprises the steps: constructing a digital simulation model of a ship fuel cell cooling system, which is used for representing the dynamic characteristics of the cooling system under different working conditions; constructing a multi-time scale prediction and thermal load prediction model, and respectively outputting cooling system state and working condition prediction parameters and thermal load prediction parameters under different time scales; a reinforcement learning framework based on the dual deep Q network is built, a state space, an action space and a reward function are defined, and the state space comprises current state parameters of the cooling system and output parameters of all prediction models; the action space comprises seawater pump rotating speed, fresh water pump rotating speed and regulating valve opening; and an optimal control strategy is generated based on a training result of deep reinforcement learning, and global collaborative optimization operation of execution equipment of the cooling system is realized, so that the system energy efficiency and environmental adaptability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship energy management, and in particular to an energy efficiency management method for a ship fuel cell cooling system based on deep reinforcement learning. Background Art

[0002] With the International Maritime Organization (IMO) enforcing increasingly stringent emission standards for ships, fuel cells are rapidly being adopted as clean energy power systems in the marine sector. The significant heat generated by fuel cells during operation requires precise temperature control to ensure safe and efficient operation. However, traditional marine cooling systems, which primarily rely on PID or simple rule-based control, lack energy consumption optimization, response speed, and adaptability, making it difficult to achieve optimal energy efficiency management of cooling systems.

[0003] Existing energy efficiency management methods for marine fuel cell cooling systems suffer from key issues, including a lack of overall coordinated optimization of system components, rigid control strategies that are difficult to adapt to environmental and load changes, a lack of forward-looking planning for energy efficiency management, and limited system model accuracy. These limitations prevent the system from accurately characterizing dynamic characteristics under complex operating conditions and from proactively adjusting operating parameters based on future operating conditions. Therefore, there is an urgent need to develop an intelligent, adaptive energy efficiency management method for marine fuel cell cooling systems that maximizes energy efficiency and optimizes control accuracy. Summary of the Invention

[0004] To address these challenges and meet these technical needs, the inventors have proposed a deep reinforcement learning-based energy efficiency management method for marine fuel cell cooling systems. This method aims to overcome existing technical bottlenecks such as low energy efficiency, insufficient control accuracy, and poor environmental adaptability. The technical solutions of this invention are as follows:

[0005] (1) Build a digital simulation model of the ship fuel cell cooling system to accurately characterize the dynamic characteristics of the cooling system under different operating conditions;

[0006] (2) Constructing a multi-time scale prediction and control structure to output cooling system status and operating condition prediction parameters at different time scales can enhance the cooling system's prediction capability and response adaptability to environmental changes;

[0007] (3) Develop an adaptive heat load prediction model based on long short-term memory network and attention mechanism to output heat load prediction parameters, which can improve the system's response speed and prediction accuracy to changes in working conditions;

[0008] (4) Build a reinforcement learning framework based on dual deep Q-network, define the state space, action space and reward function, and build a system decision-making mechanism;

[0009] (5) Generate the optimal control strategy based on the training results of deep reinforcement learning to achieve global coordinated optimization operation of each execution device in the cooling system to maximize the system energy efficiency.

[0010] A further technical solution is that the step (1) specifically comprises:

[0011] To accurately describe the dynamic characteristics of the cooling system, this paper constructs a digital simulation model of a marine fuel cell cooling system. This model includes fluid dynamics models of the seawater and freshwater circulation subsystems, describing the characteristic curves of the seawater and freshwater pumps, the relationship between the resistance characteristics of the seawater and freshwater pipelines and the flow rate of the regulating valves, energy consumption calculations for the seawater and freshwater sides, and heat transfer characteristics of the heat exchangers. The model also includes a fuel cell thermodynamic model to characterize the thermal load characteristics of the fuel cell at different power levels. Model parameters are identified and updated online using measured data to ensure high-precision performance under various operating conditions.

[0012] A further technical solution is that the step (2) specifically includes:

[0013] In order to improve the adaptability of the control system to environmental changes, the present invention proposes a multi-time scale prediction and control architecture, which includes the following three hierarchical structures: a short-term prediction layer, which predicts the cooling system state changes in the future short-term preset period (such as 5-15 minutes) based on the current state parameters of the cooling system, for real-time control decision optimization; a medium-term prediction layer, which predicts the cooling system operating condition change trend in the future medium-term preset period (such as 1-4 hours) based on meteorological data, for early adjustment and optimization of the control strategy; a long-term prediction layer, which predicts the cooling system operating condition change trend in the future long-term preset period (such as 24 hours) based on historical navigation data, for energy efficiency planning and resource optimization allocation.

[0014] The multi-timescale control strategy is expressed as:

[0015]

[0016] Among them, π() is the policy function, s t is the current state parameter of the cooling system collected, They are the state / operating condition prediction vector parameters output by the short-term, medium-term and long-term prediction layers respectively.

[0017] Its further technical solution is that the step (3) specifically includes:

[0018] Fuel cell thermal load prediction is crucial to the accuracy of cooling system control. This paper uses a long short-term memory (LSTM) network structure combined with an attention mechanism to construct an adaptive thermal load prediction model:

[0019] Q t=LSTM(P t ,P t-1 ,…,P t-n ,T env ,v ship )

[0020] Among them, Q t is the predicted heat load value at time t, P t 、P t-n are the fuel cell power at time t and time tn, T env is the ambient temperature at time t, v ship is the ship speed at time t. By introducing the attention mechanism, the importance weights of data at different time steps are automatically learned to improve the prediction accuracy.

[0021] The adaptive heat load prediction model is trained using supervised learning, and the loss function is the mean square error (MSE):

[0022]

[0023] Among them, Q i is the actual heat load, To predict heat load, N is the number of samples.

[0024] To adapt to changes in system characteristics, the present invention also adds a multi-time scale prediction and automatic update mechanism for the heat load prediction model:

[0025] Regular updates: For example, update the model parameters of the multi-time scale prediction and heat load prediction models every 100 hours of operation;

[0026] Trigger update: When the prediction error of the multi-time scale prediction and heat load prediction models exceeds a set threshold (such as 5%), the model parameter update is triggered.

[0027] A further technical solution is that the step (4) specifically includes:

[0028] This paper formalizes the energy efficiency management problem of a ship fuel cell cooling system as follows: Minimize the total energy consumption of the system by collaboratively controlling the seawater pump speed, freshwater pump speed, and regulating valve opening, while ensuring that the fuel cell temperature remains within a safe range. This problem is formulated as a Markov decision process (MDP), specifically including:

[0029] State space S: contains the collected current state parameters of the cooling system and the predicted state parameters of the cooling system at different time scales, such as the fuel cell temperature T fc , fuel cell power P fc , sea water temperature T sw , fresh water temperature T fw , seawater flow F sw, freshwater flow F fw ; Also includes the working condition prediction parameter C env And other system variables.

[0030] Action space A: Contains the seawater pump speed n sw , fresh water pump speed n fw , regulating valve opening θ and other control variables. To improve computational efficiency, the present invention discretizes the continuous action space into a finite action set, and divides each control variable into several discrete levels.

[0031] Reward function R: As the core of the deep reinforcement learning algorithm, the reward function designed in this paper comprehensively considers the two goals of minimizing energy consumption and temperature control accuracy, and is designed as follows:

[0032] R=w1(-P total )+w2(-|T fc -T set |)+w3(-ΔU)+w4(-σ T )

[0033] Among them, P total =P sw +P fw is the total energy consumption of the cooling system, P sw is the energy consumption P on the seawater side sw , P fw is the energy consumption on the freshwater side; T fc is the measured temperature of the fuel cell, T set Set the temperature for the fuel cell; ΔU=|a i -a i-1 | is the control action a i The rate of change is used to suppress drastic changes in control actions; is the standard deviation of temperature fluctuation, and w1~w4 are weight coefficients.

[0034] In addition, the present invention also sets the cooling system safety constraints to ensure that all control actions are strictly executed within the safety range, effectively guaranteeing the stability and reliability of the system. The safety constraints include: sw , fresh water pump speed n fw , regulating valve opening θ, fuel cell measured temperature T fc Set the corresponding safety constraint range and impose additional penalties on the output of the reward function when it exceeds the set safety range, such as:

[0035]

[0036] This paper uses a dual deep Q-network (DDQN) algorithm to achieve intelligent control of the cooling system. By introducing a mechanism to separate the target network and the main network, the DDQN algorithm effectively alleviates the problem of overestimation of Q values ​​and improves learning stability. When calculating the Q value of the target network, the main network is used to select the optimal action for the next state s′ of the cooling system:

[0037] a′ * =argmax a′ Q(s′,a′;φ)

[0038] The target network is then used to evaluate the Q value corresponding to the optimal action:

[0039] Q(s′,a′ * ;φ - )

[0040] The final target value is:

[0041] r+γQ(s′,a′ * ;φ - )

[0042] The loss function of the DDQN algorithm is defined as:

[0043] L(φ)=Ε (s,a,r,s′)～D [(r+γQ(s′,argmax a′ Q(s′,a′;φ);φ - )-Q(s,a;φ)) 2 ]

[0044] Among them, φ is the parameter of the main network, φ - are the parameters of the target network, and the main network and the target network keep the parameters synchronized through the soft update mechanism to improve the training stability; r is the sum of the reward function and the additional penalty, γ is the discount factor; D is the experience replay buffer, which is used to store and sample the transfer samples (s, a, r, s′).

[0045] A further technical solution is that the step (5) specifically includes:

[0046] Based on the training results of deep reinforcement learning, the present invention generates the optimal strategy function π * (s). For any given cooling system state s, the optimal control action a=π is generated by the optimal strategy function * (s), and decompose it into control instructions for each execution device, including the seawater pump speed n sw , fresh water pump speed n fw The cooling system accurately sends control instructions to each execution device via the industrial standard Modbus TCP protocol, achieving global coordinated and optimized operation of the cooling system.

[0047] The beneficial technical effects of the present invention are:

[0048] In the present invention, a complete system digital simulation model including fluid mechanics, thermodynamics and energy conversion is constructed to provide an accurate basis for control optimization. The designed multi-time scale prediction and heat load prediction model can meet the immediate control needs and long-term energy efficiency optimization goals at the same time, thereby improving the adaptability of the system. Deep reinforcement learning control technology is adopted to adapt to the complex and changeable working conditions of the system through autonomous learning of the optimal control strategy, so as to achieve continuous optimization of energy efficiency. When designing the reward function, the minimization of balanced energy consumption, temperature control accuracy, system stability and temperature uniformity are comprehensively considered to achieve multi-objective optimization. Based on the synergistic effect of the above-mentioned parts, the global collaborative optimization control of each execution equipment of the cooling system is realized, thereby improving the energy efficiency and environmental adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a structural diagram of the ship fuel cell cooling system provided in this application.

[0050] Figure 2 This is the overall framework diagram of the energy efficiency management method for the ship fuel cell cooling system provided in this application.

[0051] Figure 3 This is the structure diagram of the deep reinforcement learning controller provided by this application.

[0052] Figure 4 This is a comparison chart of cumulative energy consumption under different control strategies provided in this application. DETAILED DESCRIPTION

[0053] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0054] In one embodiment of the present application, a method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning is proposed, wherein the ship fuel cell cooling system includes a seawater circulation subsystem and a freshwater circulation subsystem, such as Figure 1 As shown. The seawater circulation subsystem includes a variable frequency seawater pump (1), seawater pipeline (2), and the seawater side of the plate heat exchanger (3). The freshwater circulation subsystem includes a variable frequency freshwater pump (4), freshwater pipeline (5), the freshwater side of the plate heat exchanger (3), an electric control valve (6), and a fuel cell cooling circuit (7). The system is equipped with multiple temperature sensors (T1-T6), flow sensors (F1-F6), pressure sensors (P1-P6), and energy consumption monitoring devices (E1-E3). The energy efficiency management intelligent control system establishes real-time communication connections with various sensors and actuators through industrial Ethernet to achieve comprehensive monitoring and precise control of the system status.

[0055] The energy efficiency management method of the present invention adopts a layered architecture design, such as Figure 2 As shown in the figure, the framework is divided into data acquisition layer, digital simulation model layer, prediction layer, deep reinforcement learning control layer and execution layer. The following describes the specific implementation content of each layer:

[0056] (1) Construct a digital simulation model of the ship fuel cell cooling system at the digital simulation model layer, including the fluid dynamics model of the seawater circulation subsystem, the fluid dynamics model of the freshwater circulation subsystem, and the fuel cell thermodynamic model. Among them:

[0057] The fluid dynamics model of the seawater circulation subsystem includes:

[0058] The seawater pump characteristic model describes the relationship between the pump head, flow rate and speed:

[0059]

[0060] Among them, H sw is the seawater pump head (m), n sw is the relative speed of the seawater pump (%), F sw is the seawater flow rate (m 3 / h), a1, b1, c1 are characteristic coefficients, which are determined by measured data.

[0061] The seawater pipeline characteristic model describes the relationship between pipeline resistance and flow rate:

[0062]

[0063] Among them, H pipe,sw is the seawater pipeline resistance (m), k sw is the seawater pipeline resistance coefficient, k v is the regulating valve resistance coefficient, and θ is the regulating valve opening (%).

[0064] Calculation of energy consumption on the seawater side:

[0065]

[0066] Among them, P sw is the seawater pump power (kW), ρ sw is the density of seawater (kg / m 3 ), g is the acceleration due to gravity (m / s 2 ), η sw is the seawater pump efficiency.

[0067] Similarly, the fluid dynamics model of the freshwater circulation subsystem includes:

[0068] Fresh water pump characteristic model:

[0069]

[0070] Fresh water pipeline characteristic model:

[0071]

[0072] Calculation of energy consumption on the fresh water side:

[0073]

[0074] Among them, H fw is the fresh water pump head (m), n fw is the relative speed of the fresh water pump (%), F fw is the freshwater flow (m 3 / h), H pipe,fw is the fresh water pipeline resistance (m), k fw is the resistance coefficient of the fresh water pipeline.

[0075] The heat transfer process of the plate heat exchanger follows the following relationship:

[0076] Q=U·A·LMTD

[0077]

[0078] Where Q is the heat transfer (kW), U is the total heat transfer coefficient (kW / m 2 K), A is the heat exchange area (m 2 ), LMTD is the logarithmic mean temperature difference (K). The overall heat transfer coefficient U is related to the fluid flow rate:

[0079]

[0080] Among them, h sw and h fw are the convective heat transfer coefficients on the seawater side and the freshwater side, respectively; δ is the thickness of the heat transfer wall; λ is the thermal conductivity of the heat transfer wall; C1 and C2 are empirical coefficients.

[0081] Fuel cell thermodynamic model:

[0082]

[0083] Among them, Q fc is the fuel cell heat load (kW), P fc is the fuel cell output power (kW), η fc is the fuel cell electrical efficiency. The fuel cell temperature changes dynamically:

[0084]

[0085] Among them, C fc is the heat capacity of the fuel cell (kJ / K), T fcis the fuel cell temperature (℃), c p is the specific heat capacity of fresh water (kJ / (kg·K)), Q fc,fw is the fuel cell cooling water flow rate (m 3 / h), T fc,in and T fc,out are the cooling water inlet and outlet temperatures (℃) respectively.

[0086] The simulation parameter settings of the digital simulation model of the ship fuel cell cooling system are shown in Table 1. The simulation operating condition design takes into account the fuel cell power variation, including constant power and random fluctuation conditions.

[0087] Table 1 Simulation parameter settings of ship fuel cell cooling system

[0088]

[0089] (2) Constructing multi-time scale prediction and control at the prediction layer

[0090] This invention adopts a multi-timescale prediction architecture to adapt to the dynamic characteristics of the system at different timescales. Both short-term and long-term predictions utilize an LSTM network with an integrated attention mechanism. Short-term predictions are based on the system's current state parameters, predicting system state changes over the next 15 minutes with a sampling interval of 30 seconds. Medium-term predictions are based on meteorological data, predicting operating condition trends over the next four hours with a sampling interval of 15 minutes. Long-term predictions are based on historical navigation data, predicting operating condition trends over the next 24 hours with a sampling interval of 1 hour. This multi-timescale prediction architecture can simultaneously meet both immediate control requirements and long-term energy efficiency optimization goals, improving system adaptability.

[0091] (3) Constructing adaptive heat load prediction in the prediction layer

[0092] The present invention uses a deep learning method to achieve accurate prediction of fuel cell thermal load. The LSTM network structure is as follows: the input layer is [P t ,P t-1 ,…,P t-n ,T env ,v ship ], the two-layer LSTM has 64 and 32 units respectively, and uses the self-attention mechanism to enhance the learning ability of key time steps, and outputs the predicted heat load Q t The model training uses the Adam optimizer with a learning rate of 0.001, a batch size of 64, and a training cycle of 200. To ensure the prediction accuracy, the system has designed an automatic update mechanism: when the prediction error exceeds 5% and lasts for 3 time steps, it will automatically trigger the multi-time scale prediction and adaptive heat load prediction model update process and retrain the prediction model. The heat load prediction module is Figure 2The core component in the prediction layer provides critical feedforward information for the deep reinforcement learning controller.

[0093] (4) The core of the present invention is to design an intelligent controller based on deep reinforcement learning in the deep reinforcement learning control layer, and its structure is as follows Figure 3 As shown in Figure 2, the controller consists of an input layer (state space), a main network (Q network), a target network, an experience replay buffer, and an output layer (action space).

[0094] State space definition: s = [T fc ,P fc ,t sw,in ,T sw,out ,F fw,in ,T fw,out ,F sw ,F fw ,Q t ,C env ]

[0095] Action space definition (discretization): a=[n sw ,n fw ,θ],n sw ∈{30%,40%,…,100%},n fw ∈{30%, 40%,…, 100%}, θ∈{10%, 20%,…, 100%}.

[0096] Reward function design:

[0097]

[0098] The reward function comprehensively considers four aspects: energy consumption minimization, temperature control accuracy, motion control stability and temperature uniformity. The weights of each item reflect the relative importance of the optimization objectives.

[0099] The present invention adopts the DDQN algorithm, and the network structure is as follows: the state vector s dimension is used as input, the hidden layer has 128 and 64 neurons respectively, ReLU activation, and the output layer is the action space dimension. The experience replay buffer size is set to 10000, the batch size is 64, and the discount factor γ = 0.95. Figure 3 As shown in Figure 2, the main network and the target network keep parameters synchronized through a soft update mechanism to improve training stability.

[0100] (5) Realizing intelligent collaborative control at the execution layer

[0101] Based on the current state s, through the optimal strategy function π * (s) Calculate the optimal control action a; resolve a into specific control instructions: n sw ,n fw,θ; The control instructions are sent to each execution device through the Modbus TCP protocol. In addition, the system has set up a safety protection mechanism to ensure that 30% ≤ n sw ,n fw ≤100%, 10%≤θ≤100%, 60℃≤T fc ≤80℃.

[0102] (6) Control performance evaluation indicators

[0103] In order to comprehensively evaluate the control performance, the following indicators are defined:

[0104] Energy efficiency index: Cooling system energy consumption rate (CER)

[0105]

[0106] Among them, E cool is the total energy consumption of the cooling system (kWh), through the total energy consumption of the cooling system P total Multiply by the working hours to get E fc is the total power generation of the fuel cell (kWh).

[0107] Temperature control indicators include:

[0108] Mean Temperature Deviation (MTD):

[0109] Temperature Fluctuation (TVF):

[0110] Overtemperature ratio (OTR): OTR = ∑(T fc >80℃) / N×100%.

[0111] Control stability index: Control action change rate (ACR):

[0112] (7) Comparison of algorithms

[0113] To verify the superiority of the proposed method, the following algorithms are selected for comparison: (1) PID control: traditional three-loop PID control; (2) model predictive control (MPC): predictive control based on linearized model; (3) single DQN algorithm: DQN without dual Q network structure; (4) DDQN multi-time scale control method proposed in this application.

[0114] Table 2 compares the performance of each algorithm under constant fuel cell power conditions, maintaining a constant 400 kW. The DDQN method outperforms other methods in both energy efficiency and temperature control, reducing the CER by 15.2% and the MTD by 62.5% compared to the PID control strategy.

[0115] Table 2 Comparison of performance under constant power conditions

[0116]

[0117] To simulate the power demand fluctuations during actual navigation, a random fluctuation condition was designed, where the power fluctuated between 150 and 500 kW. Table 3 compares the performance of the various algorithms under this random power condition. Under this complex and fluctuating condition, the DDQN method demonstrates a significant advantage, achieving an 18.7% reduction in CER and a 69.4% reduction in MTD compared to the PID control strategy.

[0118] Table 3 Random power condition performance comparison

[0119]

[0120] Figure 4 The cumulative energy consumption of each control method during 24 hours of operation is compared. The data shows that the DDQN method saves 17.3% of the cooling system energy consumption during the whole day, which is equivalent to saving about 42kWh of electricity.

[0121] The above description is only a preferred embodiment of the present application, and the present invention is not limited to the above embodiment. It is understood that other improvements and variations directly derived or imagined by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included in the scope of protection of the present invention.

Claims

1. A method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning, characterized in that: The method comprises: Construct a digital simulation model of a ship fuel cell cooling system to characterize the dynamic characteristics of the cooling system under different operating conditions; Construct multi-time scale prediction and heat load prediction models to output cooling system status and operating condition prediction parameters at different time scales, as well as heat load prediction parameters; A reinforcement learning framework based on a dual deep Q network was built to define a state space, an action space, and a reward function. The state space included the collected current state parameters of the cooling system and the output parameters of the multi-time-scale prediction and heat load prediction models; the action space included the seawater pump speed, the freshwater pump speed, and the regulating valve opening. Based on the training results of deep reinforcement learning, an optimal control strategy is generated to achieve global coordinated optimization operation of each execution device of the cooling system to maximize the system energy efficiency.

2. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: The digital simulation model includes: The fluid dynamics model of the seawater and freshwater circulation subsystem is used to describe the seawater and freshwater pump characteristic curves, the relationship between the seawater and freshwater pipeline resistance characteristics and the regulating valve flow rate, the energy consumption calculation of the seawater and freshwater sides, and the heat transfer characteristics of the heat exchanger; Fuel cell thermodynamic model, used to characterize the heat load characteristics of fuel cells at different powers.

3. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: Build multi-timescale forecast models, including: A short-term prediction layer, based on the current state parameters of the cooling system, predicts the state changes of the cooling system in the short-term preset period in the future, so as to optimize the control decision in real time; The medium-term prediction layer predicts the cooling system operating condition change trend in the future medium-term preset period based on meteorological data, so as to adjust and optimize the control strategy in advance; The long-term prediction layer predicts the cooling system operating condition change trend in the future long-term preset period based on historical navigation data, which is used for energy efficiency planning and resource optimization allocation.

4. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: Build a heat load prediction model, including: The long short-term memory network structure is used, combined with the attention mechanism to automatically learn the importance weights of data at different time steps, and an adaptive heat load prediction model is constructed as follows: Q t =LSTM(P t ,P t-1 ,…,P t-n ,T env ,v ship ) Among them, Q t is the predicted heat load value at time t, P t 、P t-n are the fuel cell power at time t and time tn, T env is the ambient temperature at time t, v ship is the ship speed at time t; The adaptive heat load prediction model is trained using a supervised learning method, and the loss function is the mean square error (MSE).

5. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: The method further includes adding an automatic update mechanism for the multi-time scale prediction and heat load prediction models: regularly updating model parameters of the multi-time-scale prediction and heat load prediction models; When the prediction error of the multi-time-scale prediction and heat load prediction model exceeds a set threshold, the model parameter update is triggered.

6. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: The reward function comprehensively considers the two goals of minimizing energy consumption and temperature control accuracy, and is designed as follows: R=w1(-P total )+w2(-|T fc -T set |)+w3(-ΔU)+w4(-σ T ) Among them, P total is the total energy consumption of the cooling system, represented by the energy consumption on the seawater side P sw and freshwater side energy consumption P fw The sum of T fc is the measured temperature of the fuel cell, T set is the set temperature of the fuel cell; ΔU is the control action change rate, σ T is the standard deviation of temperature fluctuation, and w1~w4 are weight coefficients.

7. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: The reinforcement learning framework based on the dual deep Q network is constructed, including: The dual deep Q network introduces a separation mechanism between the target network and the main network. When calculating the Q value of the target network, the main network is used to select the optimal action for the next state s′ of the cooling system: a′ * =argmax a′ Q(s′,a′;φ) The target network is used to evaluate the Q value corresponding to the optimal action: Q(s′,a′ * ;φ - ) The target value of the dual deep Q network is: r+γQ(s′,a′ * ;φ - ) The loss function of the dual deep Q network is defined as: L(φ)=E (s,a,r,s′)～D [(r+γQ(s′,argmax a′ Q(s′,a′;φ);φ - )-Q(s,a;φ)) 2 ] Among them, φ is the parameter of the main network, φ - are the parameters of the target network; r is the sum of the reward function and the additional penalty, γ is the discount factor; D is the experience replay buffer, which is used to store and sample the transfer samples (s, a, r, s′).

8. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 7 is characterized in that: The main network and the target network maintain parameter synchronization through a soft update mechanism to improve training stability.

9. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to claim 1 is characterized in that: The training results based on deep reinforcement learning generate an optimal control strategy to achieve global coordinated optimization operation of each device in the cooling system, including: For any given cooling system state s, the optimal control action a = π is generated by the optimal strategy function * (s); Decomposing the optimal control action a into control instructions for each execution device, including the seawater pump speed, the freshwater pump speed, and the regulating valve opening; The control instructions are sent to each execution device to achieve global coordinated optimization operation of the cooling system.

10. The method for energy efficiency management of a ship fuel cell cooling system based on deep reinforcement learning according to any one of claims 1 to 9, characterized in that: The method further includes setting cooling system safety constraints: Corresponding safety constraint ranges are set for the seawater pump speed, the freshwater pump speed, the regulating valve opening, and the measured temperature of the fuel cell, and when the set safety ranges are exceeded, additional penalties are imposed on the output of the reward function.

Images