Cloud edge collaboration-based multi-stack fuel cell grid active support method and system

Abstract

The application provides a kind of based on cloud edge coordination's multi-pile fuel cell power grid active support method and system, it is related to fuel cell technical field, including: receiving power generation power demand, determine day-ahead optimal value in combination with day-ahead hydrogen supply attribute;Power prediction model is trained and power prediction model parameter is uploaded to cloud node, receives target model parameter issued by cloud node and carries out parameter update;Based on the physical state of each resource type and power prediction value, a standardized state vector is constructed, a unified action vector is output through a local strategy model, a decision reward value is calculated based on system operating parameters and day-ahead optimal value, and the local strategy model is updated based on the soft actor-critic algorithm and the model parameters are uploaded;Receive the global strategy model parameter issued, adjust the local strategy model parameter based on the difference between the average decision reward of the global strategy model and the local strategy model.The scheme of the application can achieve comprehensive optimization of multiple indicators while protecting data privacy.

Description

Technical Field

[0001] This application relates to the field of fuel cell technology, and in particular to a method and system for active grid support of multi-stack fuel cells based on cloud-edge collaboration. Background Technology

[0002] With the advancement of the "dual carbon" goal, the high proportion of renewable energy connected to the grid leads to a decrease in grid inertia and an increase in frequency regulation pressure. Therefore, multi-stack fuel cell power generation systems that provide active support to the grid through grid operation play an increasingly important role in the new power system.

[0003] However, in the process of achieving efficient, stable, and proactive grid support for multi-stack fuel cell power generation systems, existing technologies are still unable to effectively achieve the optimal balance between hydrogen consumption economy, stack health maintenance, and robustness of proactive grid support while protecting data privacy. Summary of the Invention

[0004] In view of this, this application proposes a method and system for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration.

[0005] In a first aspect, this application provides a cloud-edge collaborative active support method for multi-stack fuel cell grids, applied to edge-side nodes, the method comprising: Receive the power generation demand sent by the cloud node and determine the day-ahead optimal value based on the day-ahead hydrogen supply attributes; A power prediction model is trained based on time-series data of historical stack output, power generation demand and hydrogen consumption. The power prediction model parameters are uploaded to the cloud node, and the target model parameters are received and updated after aggregation and iteration by the cloud node. The energy management problem at the edge is modeled as a Markov decision process. The physical states of various resource types collected at the receiving end are combined with the output of the power prediction model to construct a standardized state vector. Based on the current and historical standardized state vectors, a unified action vector is output through the local strategy model to generate active support decisions for the power grid. The decision reward value is calculated based on the system operating parameters and the day-ahead best value. The local strategy model is updated based on the soft actor-commentator algorithm, and the parameters of the local strategy model are uploaded to the cloud node. Receive global strategy model parameters from cloud nodes, and adjust local strategy model parameters based on the difference in average decision reward obtained when making decisions between the global strategy model and the local strategy model.

[0006] In one embodiment, the process of receiving the power generation demand sent by the cloud node and determining the day-ahead optimal value based on the day-ahead hydrogen supply attributes includes: With the goal of minimizing overall operating costs, a stage cost function for dynamic programming is constructed based on the defined state variables and decision variables. Based on the stage cost function, the day-ahead hydrogen supply attributes, the power generation demand, the state variables, and the decision variables, under operational constraints, the Bellman equation is used to recursively solve the problem from the last moment of the scheduling cycle to obtain the day-ahead optimal value.

[0007] In one embodiment, the physical states of each resource type collected by the receiving end are combined with the output of the power prediction model to construct a standardized state vector, including: Based on the physical state of each resource type and the output results of the power prediction model, construct the state vector of energy production type resources and the state vector of energy storage type resources; By using a learnable embedding function, discrete temporal features and scene categories are mapped to a continuous low-dimensional vector space, and the time-scene joint embedding code is extracted and output. The state vector of the production capacity resource or the state vector of the energy storage resource are concatenated with the time period-scenario joint embedding code to obtain the standardized state vector of the corresponding resource.

[0008] In one embodiment, the local policy model includes a local shared encoder and a resource-dedicated decision head network; the step of generating proactive grid support decisions by outputting a unified action vector through the local policy model based on current and historical standardized state vectors includes: The local shared encoder encodes the current and historical standardized state vectors to extract shared feature vectors. The shared feature vector and the resource type embedding vector are input into the resource-specific decision head network set in parallel, and a unified action vector is output. The unified action vector is mapped to form an active support decision for the power grid through a preset action mapping function. The active support decision for the power grid includes the output power and start-stop control commands of the fuel cell stack, as well as the charging and discharging power and target state of charge commands of the battery energy storage system.

[0009] In one embodiment, the day-ahead hydrogen supply attribute includes the day-ahead hydrogen price; the calculation of the decision reward value based on system operating parameters and the day-ahead optimal value includes: Based on the aforementioned day-ahead optimal value and the optimal estimate under the current local policy model, potential energy reward shaping is performed, and the potential energy reward shaping term is obtained by solving for it. The economic incentive is calculated based on the direct hydrogen consumption of the fuel cell stack, the equivalent hydrogen consumption of the battery energy storage system, and the day-ahead hydrogen price; the equipment health incentive is calculated based on the equipment loss cost and the health consistency cost of multiple fuel cell stacks; and the instruction tracking incentive is calculated based on the deviation between the actual output power and the power generation demand. The potential energy reward shaping item, economic reward, equipment health reward, and instruction tracking reward are weighted and integrated according to preset weights to obtain the local decision reward value.

[0010] In one embodiment, updating the local policy model based on the soft actor-critic algorithm includes: The enhanced state is obtained by sampling state, action, reward, and next state samples from the local experience pool and splicing the power prediction value. Based on the enhanced state, a Bellman objective is constructed, the critic network loss is calculated, and the critic network parameters are updated using gradient descent. The policy loss is calculated with the goal of maximizing expected return and policy entropy. The actor network parameters are updated by gradient descent. After multiple iterations, the parameters of the local policy model are updated.

[0011] In one embodiment, adjusting the local policy model parameters based on the difference in the average decision reward obtained when making decisions based on the global policy model and the local policy model includes: The active support decisions for the power grid are generated using both a global strategy model and a local strategy model, and multiple sets of decision rewards are statistically analyzed and the average decision reward is calculated. Based on the difference between the average values ​​of the two sets of decision rewards and a preset temperature parameter, an adaptive fusion coefficient is calculated using the Sigmoid function, wherein the temperature parameter is used to adjust the sensitivity of the difference. The global policy model parameters and the local policy model parameters are weighted and fused based on the adaptive fusion coefficients to obtain personalized local policy model parameters.

[0012] Secondly, this application also provides a method for active support of multi-stack fuel cell grids based on cloud-edge collaboration, applied to cloud nodes, the method comprising: The power generation demand is distributed to each edge node, and the global strategy model is initialized. Receive the power prediction model parameters of each edge node and perform aggregation and iteration according to the federated learning framework, and then send the obtained target model parameters to each edge node. Obtain the local policy model parameters of each edge node, perform cross-system model parameter aggregation for the same resource type, and obtain the federated policy model; The global strategy model is trained by using the federated strategy model as the teacher model to perform cross-resource type knowledge distillation, and the trained global strategy model parameters are distributed to all edge nodes.

[0013] In one embodiment, the step of performing cross-resource type knowledge distillation using the federated policy model as the teacher model to train the global policy model, and distributing the trained global policy model parameters to all edge nodes, includes: Define the global strategy model as the student model and initialize the parameters of the global strategy model to the average value of the parameters of each teacher model; State vector samples are sampled from a cloud node shared dataset, wherein the cloud node shared dataset is formed by local experience pool data selectively uploaded by each edge node; The global policy model is trained based on the target loss function and the state vector samples to obtain the global policy model. The target loss function includes a distillation loss sub-function and a uniformity loss sub-function.

[0014] Thirdly, this application also provides a multi-stack fuel cell grid active support system based on cloud-edge collaboration, the system including edge-side nodes and cloud-side nodes; The edge-side node is used to execute the cloud-edge collaborative active support method for multi-stack fuel cell grids as described in the first aspect; The cloud node is used to execute the cloud-edge collaborative active support method for multi-stack fuel cell grids as described in the second aspect.

[0015] The proposed cloud-edge collaborative active grid support method for multi-stack fuel cell grids has the following advantages over related technologies: 1. The cloud-edge collaborative multi-stack fuel cell grid active support method of this application adopts a data interaction mode of local training model on the edge side and only transmitting parameters in the cloud. Since the original data such as historical output, hydrogen consumption time series data and physical status of end-side equipment do not leave the edge-side node, and the cloud only completes the aggregation, distillation and distribution of model parameters, the transmission of original sensitive data can be completely avoided, and data privacy protection can be reliably achieved.

[0016] 2. Edge nodes construct standardized state vectors based on the physical state of each resource type and the output of power prediction models. Based on the current and historical standardized state vectors, a unified action vector is output through the local policy model to generate proactive grid support decisions. Decision reward values ​​are calculated based on system operating parameters and the day-ahead best values. The local policy model is updated using a soft actor-commentator algorithm, allowing for continuous iterative optimization to simultaneously consider hydrogen consumption economy and fuel cell stack health. Furthermore, cloud-edge federated learning is used to improve power prediction accuracy, and the model parameters are adaptively adjusted based on the reward difference between the global and local policy models. This further enhances the stability and robustness of the grid's proactive support capabilities, ultimately achieving a comprehensive optimization of hydrogen consumption economy, fuel cell stack health maintenance, and grid proactive support robustness. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for active grid support of multi-stack fuel cells based on cloud-edge collaboration in one embodiment of this application. Figure 2 This is a flowchart illustrating a multi-stack fuel cell grid active support method based on cloud-edge collaboration in another embodiment of this application. Figure 3 This is a schematic diagram of the structure of a multi-stack fuel cell grid active support system based on cloud-edge collaboration in one embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] In some embodiments, such as Figure 1 As shown, this application provides an active support method for multi-stack fuel cell grids based on cloud-edge collaboration, which is applied to edge nodes. The active support method for multi-stack fuel cell grids based on cloud-edge collaboration includes the following steps S101 to S104.

[0021] S101: Receives the power generation demand sent by the cloud node and determines the optimal value for the day-ahead based on the day-ahead hydrogen supply attributes.

[0022] For example, the cloud-edge collaborative federated reinforcement learning-based active support method for multi-stacking fuel cell power grids provided in this embodiment can be applied to a distribution network area composed of Ck multi-stacking fuel cell power generation systems (also known as power generation systems), where each multi-stacking fuel cell power generation system k contains The system comprises a fuel cell stack and a battery energy storage system. The multi-stack fuel cell power generation system includes an Energy Management Controller (EMC), a hydrogen tank, a fuel cell stack (FCS), a DC / DC converter, a battery energy storage system (BESS), and a DC / AC converter. The overall architecture of the embodiments provided in this specification can adopt a three-layer "cloud-edge-device" structure. Cloud nodes can be deployed in power grid dispatch centers, distribution substations, etc., responsible for cross-system energy dispatch collaborative optimization, power generation demand formulation, global model aggregation, and cross-type knowledge distillation. The communication cycle Tcloud can be 1 hour. Edge nodes are typically deployed on the energy management controller of the multi-stack fuel cell power generation system, responsible for local reinforcement learning training, federated fuel cell power prediction, and generation of proactive grid support decisions. The communication cycle Tedge can be 15 minutes, and one edge node corresponds to one multi-stack fuel cell power generation system. Device nodes are typically deployed on the controllers of the fuel cell stack and battery energy storage system, responsible for data acquisition and execution of proactive grid support decisions.

[0023] The day-ahead hydrogen supply attribute can be either hydrogen price or hydrogen supply quantity. For ease of explanation, the following description of the day-ahead hydrogen supply attribute will use hydrogen price as an example. The edge nodes perform offline dynamic programming (DP) based on the day-ahead hydrogen price to obtain the day-ahead optimal value.

[0024] It is understandable that edge nodes receive the power generation demand transmitted by cloud nodes to clarify the power output requirements of the grid for multi-stack fuel cell power generation systems. Then, combined with the relevant attributes of hydrogen supply on the day, the grid output target and hydrogen energy usage conditions are comprehensively optimized to finally determine the day-ahead optimal value.

[0025] S102: Train a power prediction model based on time-series data of historical stack output, power generation demand and hydrogen consumption, upload the power prediction model parameters to the cloud node, receive the target model parameters after aggregation and iteration by the cloud node and update the parameters.

[0026] In applications, edge devices collect and upload power output and hydrogen consumption data of the fuel cell stack to their respective edge nodes. When training the power prediction model, the multi-stack fuel cell power generation system first initializes the power prediction model parameters. Then, the future output power of the fuel cell stack is modeled using the LSTM prediction model defined by the following formula:

[0027] Specifically, for time t, the model input is a historical window of length L, containing the stack output over the past L time steps. Power generation demand and hydrogen consumption The power generation demand is distributed from the cloud to the edge, and this time-series data is fed into a parameter. LSTM networks, outputting the future h =1,…, H Step power prediction value During training, each multi-stack fuel cell power generation system will use the predicted values ​​of the same fuel cell stack. Actual fuel cell output at the same time Calculate the mean square error loss using the following formula. : Then, the power prediction model parameters are updated through backpropagation and gradient descent to obtain... , This represents the power prediction model parameters for edge node k, including trainable parameters and hyperparameters in the LSTM network. The trainable parameters include the weight matrix. ( ), bias vector ( Output weight matrix and output bias vector Among them, the weight matrix Including the forget gate weight matrix Input gate weight matrix Candidate memory unit weight matrix and output gate weight matrix Bias vector Including the forget gate bias vector Input gate bias vector Candidate memory cell bias vector Output gate bias vector Hyperparameters include the number of LSTM layers (2 layers in this example) and the dimension of the hidden layers. (64-dimensional in this embodiment), Input feature dimensions And the predicted step size H.

[0028] The cloud aggregates the power prediction model parameters from each edge side according to the federated learning framework to obtain the power prediction model for this round of federated fuel cell power prediction. The model parameters of the federated fuel cell power prediction model are the target model parameters.

[0029] Specifically, after local training is completed, the edge nodes will Local data sample size and mean square error loss Uploaded to cloud nodes. The cloud nodes aggregate the data according to the following formula to obtain the current round of federal fuel cell prediction model. :

[0030] in, It is a very small positive number to prevent division by zero errors; round=1, ..., , This indicates the number of training iterations for the prediction model. After training is complete, the cloud node ultimately outputs the parameters of the federated fuel cell prediction model. And feedback is sent to the edge side.

[0031] It is understandable that edge nodes train their power prediction models locally based on multi-dimensional time-series operational data comprised of locally stored historical fuel cell output data, cloud-based power generation demands, and actual hydrogen consumption. This highly correlated time-series data allows for the full understanding of the operational patterns between fuel cell output, grid dispatch requirements, and hydrogen consumption, ensuring the accuracy of power prediction results. After model training, only the parameters of the power prediction model are uploaded to the cloud nodes; the original operational time-series data is not transmitted externally, effectively protecting the privacy and security of system operational data. The cloud nodes aggregate and iteratively optimize the model parameters uploaded by each edge node according to federated learning rules, forming globally optimal target model parameters. Upon receiving these target model parameters, the edge nodes update their existing power prediction models, achieving global collaborative iteration of the model and ensuring that the power prediction model continuously adapts to overall grid dispatch and actual on-site operating conditions.

[0032] S103: The energy management problem at the edge is modeled as a Markov decision process. The physical states of various resource types are collected at the receiving end. A standardized state vector is constructed by combining the output of the power prediction model. Based on the current and historical standardized state vectors, a unified action vector is output through the local policy model to generate active support decisions for the power grid. The decision reward value is calculated based on the system operating parameters and the day-ahead best value. The local policy model is updated based on the soft actor-commentator algorithm. The parameters of the local policy model are then uploaded to the cloud node.

[0033] The resource types include power generation resources and energy storage resources. Fuel cell stacks only generate electricity and are considered power generation resources; battery energy storage systems can both generate electricity and charge batteries and are considered energy storage resources. The local policy model refers to a policy neural network that edge nodes iteratively train on a local experience pool using the Soft Actor-Critic (SAC) algorithm. It includes a local shared encoder and a resource-specific decision head, representing the personalized decision-making knowledge of the power generation system.

[0034] It is understandable that modeling the energy management problem of a multi-stack fuel cell power generation system as a Markov decision process can adapt to the dynamic application scenarios of the system. Edge nodes receive real-time physical state data of various resources such as production capacity and energy storage from end-side devices, and integrate this data with the output of the local power prediction model to construct a standardized state vector, comprehensively gathering multi-dimensional system operation information. Based on the current and historical standardized state vectors, the local strategy model outputs a unified action vector, which is then transformed into corresponding grid active support decisions to complete grid output response tasks. Simultaneously, the decision reward value is calculated by combining real-time system operating parameters with the previously determined day-ahead optimal value. The day-ahead optimal value is used to anchor optimization goals such as hydrogen consumption economy and stack health operation, thereby evaluating the comprehensive benefits of the decision scheme. Subsequently, a soft actor-critic reinforcement learning algorithm is used to iteratively update the local strategy model based on the decision reward value, continuously optimizing the decision logic and output effect. Finally, only the locally shared encoder parameters and resource-specific decision head network parameters, which have undergone iterative optimization, are uploaded to the cloud node. Throughout the process, the original device state data and system operation data are processed at the edge, with only model parameters interacting. This effectively protects the privacy and security of the power generation system's operation data while achieving cloud-edge collaborative model optimization.

[0035] S104: Receives global policy model parameters from the cloud node and adjusts the local policy model parameters based on the difference in the average decision reward obtained when making decisions using the global policy model and the local policy model. The global policy model refers to the policy model obtained after cross-resource type knowledge distillation in the cloud. It has a similar structure to the local policy model, containing a global shared encoder and multiple resource-specific decision head networks. The global policy model parameters refer to all trainable parameters in the global policy model. The global shared encoder is responsible for extracting general environmental representations independent of resource type.

[0036] It is understandable that after the edge nodes obtain the global strategy model parameters from the cloud, they use the global strategy model and the local strategy model to carry out decision-making. They statistically calculate the average decision reward for the two models, judge the performance gap between the local model and the global optimal model based on the difference between the two average values, and optimize and adjust the parameters of the local strategy model accordingly. This allows the local decision model to integrate global scheduling experience and on-site operating characteristics, continuously improve the decision quality, and strengthen the support capability of the multi-stack fuel cell system for the power grid.

[0037] The aforementioned cloud-edge collaborative multi-stack fuel cell grid active support method employs a data interaction mode where the edge side trains the model locally and the cloud only transmits parameters. Since raw data such as historical output, hydrogen consumption time-series data, and the physical state of end-side equipment do not leave the edge nodes, and the cloud only aggregates, distills, and distributes model parameters, the transmission of raw sensitive data can be completely avoided, reliably protecting data privacy. Edge nodes construct standardized state vectors based on the physical state of each resource type and the output of the power prediction model. Based on the current and historical standardized state vectors, a unified action vector is output through the local policy model to generate grid active support decisions. Decision reward values ​​are calculated based on system operating parameters and the current day-ahead best value. The local policy model is updated using a soft actor-critic algorithm, allowing for continuous iterative optimization to simultaneously consider hydrogen consumption economy and stack health. Furthermore, cloud-edge federated learning improves power prediction accuracy, and the model parameters are adaptively adjusted based on the reward difference between the global and local policy models, further enhancing the stability and robustness of grid active support capabilities. Ultimately, this achieves a comprehensive optimization of hydrogen consumption economy, stack health maintenance, and grid active support robustness.

[0038] In some embodiments, receiving the power generation demand sent by the cloud node and determining the day-ahead optimal value in combination with the day-ahead hydrogen supply attributes includes: constructing a stage cost function for dynamic programming based on defined state variables and decision variables with the goal of minimizing overall operating costs; and, based on the stage cost function, day-ahead hydrogen supply attributes, power generation demand, state variables, and decision variables, recursively solving the Bellman equation from the last moment of the scheduling cycle under operating constraints to obtain the day-ahead optimal value.

[0039] In this application, the input parameters and operational constraints for the day-ahead dynamic programming solution are first determined. The input parameters include the day-ahead hydrogen price. and the power generation demand distributed from the cloud The operating constraints include power constraints on the fuel cell stack, power constraints and state of charge constraints on the battery energy storage system (BESS), and power balance constraints, as shown below: in, This indicates that the i-th fuel cell stack is at time k. r The output power, i=1,2,…, Indicates the sequence number of the fuel cell stack. For the efficiency of the DC / DC converter connected to the fuel cell stack, For battery energy storage systems at all times The synthesis of charging and discharging power. For the cloud at all times The power generation demand issued to the multi-stack fuel cell power generation system k , The lower and upper limits of the charging and discharging power of the battery energy storage system. , Let be the lower and upper limits of the output power of the i-th fuel cell stack. , These represent the lower and upper limits of the state of charge (SOC) of a battery energy storage system, respectively.

[0040] Then, define the state variables and decision variables for solving the current-day dynamic programming problem, where the state variables are... Set as The decision variables are: the current state of charge of the battery energy storage system and the real-time power of each fuel cell stack; Set as This refers to the synthesis of charging and discharging power of the battery energy storage system and the reference power of each fuel cell stack.

[0041] Construct the stage cost function for the current-day dynamic programming solution with the objective of minimizing the overall cost of system operation. The stage cost function consists of the sum of the lifespan cost of the fuel cell stack, the lifespan cost of the battery energy storage system, the hydrogen consumption cost of the power generation system, and the health consistency cost of multiple fuel cell stacks, as shown below: in, , These represent the lifespan costs of the fuel cell stack and the battery energy storage system, respectively, characterizing the operational losses of the fuel cell stack and the battery energy storage system. The current price of hydrogen; For the power generation system at time Hydrogen consumption during operation.

[0042] It should be noted that the performance of the fuel cell stack under given operating conditions is considered to degrade linearly over time. The actual operating state is broken down into several typical operating conditions, including load variation cycles, start-stop cycles, idling conditions, and high-power load conditions. Each typical operating condition is assigned a performance degradation rate c1, c2, c3, c4, and the corresponding number of cycles or operating time for that rate. These are then accumulated, and the accumulated result is multiplied by an acceleration factor. Later used as a degradation indicator for fuel cell stacks (FCS) : in, and These represent the number of cycles for the load change cycle and the start / stop cycle, respectively. , The operating times are for idling and high-power load conditions, respectively. The degradation index of the fuel cell stack (FCS) is used. Multiply by fuel cell stack price factor As molecules, the performance degradation threshold of a fuel cell stack (FCS) from its initial state to the end of its lifespan. As the denominator, we obtain the lifespan cost of the fuel cell stack. :

[0043] Among them, coefficients c1, c2, c3, and c4 were obtained through experimental calibration; acceleration coefficient This is a comprehensive correction factor that adjusts for the impact of actual operating conditions on the degradation rate of the fuel cell stack, obtained through experiments or from fuel cell manufacturers; fuel cell stack price factor. The unit life cost of FCS is converted into a cost item with the same dimensions as hydrogen consumption cost, which is determined by the purchase cost and life of the fuel cell.

[0044] Taking the lithium iron phosphate (LiFePO4) battery solution in the battery energy storage system as an example, based on capacity loss... Calculate the life-cycle cost of a battery energy storage system for: in, This is the unit capacity price coefficient for battery energy storage systems, which transforms the unit capacity loss of battery energy storage systems into a cost item with the same dimensions as hydrogen consumption costs. This is the capacity failure threshold. It represents the incremental capacity loss during a specific charge / discharge phase. It is the long-term capacity decay trend of BESS (Balance of Life Loss) under a given temperature trajectory and charge / discharge behavior, obtained from numerous LiFePO4 battery cycle life tests. The corresponding formula is: in, As the pre-factor, For activation energy, The gas constant is... Absolute temperature For the capacity of charge and discharge cycles, For the newly added charging and discharging capacity in the current stage, It is a power-law factor.

[0045] Multi-stack fuel cell health consistency cost It is through the coefficient of variation With weighting coefficients The weighting coefficients obtained after multiplication The dimensionless coefficient of variation The economic penalty term, converted to the same dimension as hydrogen consumption cost, determines the weight of the health consistency objective relative to the hydrogen consumption economic objective in the overall optimization. For example, Coefficient of variation It is achieved by predicting the standard deviation of the remaining lifespan of the fuel cell stack. Divide by the average predicted remaining life of the fuel cell stack The resulting formula is: , in, and Let $\frac{i}{i}$ be the predicted remaining lifetime of the $i$-th fuel cell stack and the average predicted lifetime of all fuel cell stacks within the power generation system, respectively. The corresponding calculation formula is as follows: in, and Let be the remaining health status and the decommissioning threshold health status of the i-th fuel cell stack, respectively. The decay rate per unit time is the total decay of the i-th fuel cell stack within the current time window. Divide by the total duration corresponding to the current time window The resulting formula is: Finally, the Bellman equations are used to solve the problem recursively from the last moment of the scheduling cycle. The Bellman equations are as follows: in, for Moment State The optimal value function, Let be the state transition function, representing the current decision. The system state under the action changes from State at the next moment The transition relationship is determined. The optimal value function for the current day is finally obtained by recursively solving the Bellman equation. .

[0046] In some embodiments, the physical states of each resource type collected by the receiving end are combined with the output results of the power prediction model to construct a standardized state vector, including: constructing a power generation resource state vector and an energy storage resource state vector based on the physical states of each resource type and the output results of the power prediction model; mapping discrete time features and scenario categories to a continuous low-dimensional vector space through a learnable embedding function, extracting and outputting a time-segment joint embedding code; and concatenating the power generation resource state vector or the energy storage resource state vector with the time-segment joint embedding code to obtain the standardized state vector of the corresponding resource.

[0047] Specifically, by selecting core states that have a direct impact on the power grid's proactive support decisions, a state space is constructed, and a state vector for production-type resources is generated. This includes normalized output intensity, performance retention rate, power forecast, and parameters related to active grid support capabilities; energy storage resource state vectors. This includes real-time state of charge, normalized charge / discharge power, performance retention rate, target state of charge, and normalized power deficit. It's worth noting that incorporating the predicted output power of the fuel cell stack into the state vector of production-type resources allows the policy network to perceive future power demand trends and possible operating states of the fuel cell stack during current decision-making, thereby enabling better long-term scheduling decisions.

[0048] The state vector of production capacity resources is constructed as follows: in, This represents the upper limit of the output power of the fuel cell stack. This is the ratio of the output power of fuel cell stack i to its upper limit, reflecting the current output intensity of a single stack. After normalization, it can be adapted to fuel cell stacks with different rated power, improving versatility. This represents the current performance retention rate of the fuel cell stack. These are the predicted output power values ​​of fuel cell stack i for the next 1 hour and 2 hours, respectively, derived from the output results of the power prediction model. It is the power generation demand sent from the cloud to the power generation system k. It is the ratio of the output of fuel cell stack i to the power generation demand, reflecting the active support capability of a single stack for the power grid; It is the time code of the current moment within the day. Indicates the current day of the week. Incorporating these into the state variables of production capacity resources can characterize the periodic patterns of grid load, power generation demand, and fuel cell stack operation modes in the intraday and time domains, thereby improving the adaptability of the strategy to different time periods and weekly scenarios.

[0049] The state vector of the energy storage resource is constructed as follows: in, This refers to the real-time state of charge of the battery energy storage system. This represents the normalized charge and discharge power of the battery energy storage system; positive values ​​indicate charging, and negative values ​​indicate discharging. This represents the current performance retention rate of the battery energy storage system. The target state of charge for the battery energy storage system is used to guide the battery energy storage system to recover to a safe and lifespan-friendly range in a timely manner after supporting the power grid. To normalize the power deficit, a positive value indicates insufficient output from the FCS, requiring the BESS to discharge for compensation; a negative value indicates surplus output from the FCS, allowing the BESS to recharge and store energy, which guides the BESS's charging and discharging decisions. This reflects the difference between the total output of the fuel cell stack and the power generation demand in current multi-stack fuel cell power generation systems.

[0050] Then, through a shared embedding function It can map discrete temporal features and scene categories to a continuous low-dimensional vector space. This transforms the data into learnable continuous vectors, enhancing the model's generalization ability across different scenarios. Ultimately, it allows for the extraction of the joint embedding encoding of the output time period and scene. The corresponding formula is as follows: in, It represents hourly time periods to capture the peak-valley patterns of the power grid within a day; It represents a daily time period to capture the periodic differences in power grid load patterns; It represents the four seasons and captures seasonal load changes in the power grid; for dimensionality; For embedding functions Model parameters.

[0051] Finally, the state vectors of production capacity resources or energy storage resources are concatenated with time-segment joint embedding codes to obtain the standardized state vectors of the corresponding resources: Embedding learnable time periods and scenarios into vectors enhances generalization ability and avoids the drawback of a single time variable failing to capture the coupling patterns of multi-dimensional scenarios (such as the grid load being much higher on summer weekends than on winter weekdays), enabling the model to spontaneously learn under what circumstances ( For a specific state ( How to react.

[0052] In some embodiments, based on current and historical standardized state vectors, a unified action vector is output through a local policy model to generate proactive grid support decisions. This includes: encoding the current and historical standardized state vectors using a local shared encoder to extract shared feature vectors; embedding the shared feature vectors and resource types into a parallel resource-specific decision head network to output a unified action vector; and mapping the unified action vector to form proactive grid support decisions using a preset action mapping function. The proactive grid support decisions include the output power and start-stop control commands of the fuel cell stack, and the charging / discharging power and target state of charge commands of the battery energy storage system. The unified action vector contains action components representing power control and action components representing mode selection.

[0053] In this application, considering the heterogeneous resources at the edge, the local strategy model architecture is designed as follows: in, It is a resource indicator. Indicates the state of edge node k. s Take action below a The probability distribution. The local policy model structure includes a local shared encoder and a resource-specific decision head. The local shared encoder is responsible for processing the original state vectors of heterogeneous resources. s Mapping to a unified hidden feature space yields shared feature vectors. h This enables the policy network to process different types of resources using the same set of parameters; the resource-specific decision head refers to the independent sub-network corresponding to each resource type in the local policy model, which outputs a unified action vector a for that resource type based on the shared feature vector h and the resource type embedding vector zᵣ.

[0054] First, through local shared encoder For the standardized state vector Encode and extract shared feature vectors : in, They are shared feature vectors; It is a local shared encoder. The parameters include the embedding layer parameters (input projection matrix). and its bias Multi-head attention layer parameters (projection matrices corresponding to query vector Q, key vector K, and value vector V) , , and its bias Output projection matrix and its bias Feedforward network parameters (weight matrix of two fully connected layers) , and bias , ) and layer normalization parameters (scaling factors for each layer) and offset factor The local shared encoder can employ a Transformer-style multi-head attention mechanism to normalize the state at time t. As the query vector Q, As the key vector K and value vector V, the correlation weights between the current state and each historical state are calculated through a multi-head attention mechanism. Then, the standardized historical states are weighted and fused based on these weights, and finally, a shared feature vector that integrates historical operating patterns and temporal regularities is output. The calculation formula is as follows:

[0055] Where t is the real-time decision time at the edge. It is the shared feature vector at time t. For the standardized state at time t, This represents an ordered sequence of all normalized state vectors from time tL to time t-1, a historical state sequence of length L; Q is the query vector, K is the key vector, and V is the value vector. This is achieved through a local shared encoder. Generate shared feature vectors This approach eliminates heterogeneity between different resource types, enabling local policy models to make unified decisions. Furthermore, by calculating the attention weights between the current and historical states using a multi-head attention mechanism, it extracts similar operational patterns from the past that are relevant to the current scenario. This addresses the problems of traditional reinforcement learning, such as making decisions solely based on the current state, lacking historical pattern awareness, and being prone to short-sighted scheduling. Additionally, setting the attention key vector K and value vector V to the same historical state sequence is a general, lightweight design for edge training in reinforcement learning, eliminating the need for additional calculations of the K / V mapping differences and significantly reducing the computational load at the edge.

[0056] After that, As a standardized state Part of the resource type embedding vector After concatenation, the data is input into a dedicated resource decision head, enabling the local policy model to automatically focus on historical operational trends most relevant to the current scheduling scenario, thereby improving the time-series awareness and long-term decision rationality of energy decisions. Among these, the resource type embedding vector... This refers to mapping discrete resource type identifiers (energy-producing resources G or energy-storage resources S) through an embedding layer to obtain a continuous dense vector, and then inputting it into a resource-specific decision head network. This allows the network to distinguish the control logic of different resource types parametrically, without having to build a completely independent network for each type of resource.

[0057] Based on this, a semantically consistent two-dimensional unified action space is designed, in which the action outputs of each resource type satisfy the constraints of this space. The specific expression is as follows: Among them, action component α1 represents power control, adjusting the power output ratio of energy-producing resources and the charging and discharging power of energy storage resources; action component α2 represents mode selection, determining the start-up and shutdown operations of energy-producing resources and the target state of charge of energy storage resources. The two-dimensional action vector is generated as follows:

[0058] in, It is a resource-specific decision-making head network. These are the trainable parameters of the resource-specific decision head network, including all weight matrices of the three fully connected layers. and bias vector ,in, , respectively, represent the first layer, second layer, and output layer network.

[0059] Finally, using preset action mapping functions, the unified action vector is mapped to the output power and start / stop control commands of the fuel cell stack, and the charge / discharge power and target state of charge commands of the battery energy storage system. The preset action mapping functions include mapping functions corresponding to the fuel cell stack and mapping functions corresponding to the battery energy storage system.

[0060] The mapping function corresponding to the FCS output power is: The mapping function corresponding to the BESS charging and discharging power is: in, Indicates charging; Indicates discharge. At this time, BESS neither charges nor discharges.

[0061] The mapping function corresponding to the start / stop control command of FCS is: The shutdown mode indicates that the fuel cell stack is completely shut down; the operation mode indicates that the fuel cell stack is operating according to the output power adjusted by the action component α1.

[0062] The mapping function corresponding to the target state of charge of BESS is: In some embodiments, the decision reward value is calculated based on system operating parameters and the current day-ahead best value, including: shaping the potential energy reward based on the current day-ahead best value and the optimal estimate under the current local strategy model, and solving for the potential energy reward shaping term; calculating the economic reward based on the direct hydrogen consumption of the fuel cell stack, the equivalent hydrogen consumption of the battery energy storage system, and the current day-ahead hydrogen price; calculating the equipment health reward based on the equipment loss cost and the health consistency cost of multiple fuel cell stacks; calculating the command tracking reward based on the deviation between the actual output power and the power generation demand; and weighting and fusing the potential energy reward shaping term, the economic reward, the equipment health reward, and the command tracking reward according to preset weights to obtain the local decision reward value.

[0063] In this embodiment, the day-ahead hydrogen supply attribute includes the day-ahead hydrogen price. The formula for calculating the decision reward value is as follows:

[0064] in, , These are the weighting coefficients for device health rewards and command tracking rewards, for example, , It is an economic reward. It's a device health reward. It is an instruction tracking reward. It is a DP-guided potential energy reward shaping item.

[0065] In applications, the economic incentive is determined by multiplying the sum of the direct hydrogen consumption of the fuel cell stack and the equivalent hydrogen consumption of the battery energy storage system by the unit price of hydrogen. The objective of designing the economic incentive is to minimize the hydrogen consumption cost and the equivalent operating cost of the system. The calculation formula is as follows: in, and These are the sampling intervals. The corresponding direct hydrogen consumption of the fuel cell stack and the equivalent hydrogen consumption of the battery energy storage system. This refers to the price per unit of hydrogen. The formula for calculating the direct hydrogen consumption of a fuel cell stack is:

[0066] in, The lower heating value of hydrogen. The equivalent electrical efficiency of the fuel cell stack. This represents the output power of the i-th fuel cell stack.

[0067] The formula for calculating the equivalent hydrogen consumption of a battery energy storage system is as follows: in, and These refer to the charging efficiency and discharging efficiency of the battery energy storage system, respectively. , These are the charging power and discharging power of the battery energy storage system, respectively. The charging and discharging process of the battery energy storage system itself does not directly consume hydrogen, but it associates the energy conversion between electrical energy and hydrogen through equivalent logic. The charging process of the battery energy storage system is regarded as "consuming hydrogen to store electrical energy", and the discharging process is regarded as "recovering the electrical energy corresponding to hydrogen".

[0068] Equipment health rewards are determined based on the degradation indicators of the fuel cell stack, the lifetime loss cost calculated from the capacity loss of the battery energy storage system, and the health consistency cost of multiple fuel cell stacks, avoiding drastic changes in operating conditions. The corresponding formula is: in, For equipment health penalty coefficient, , These are the lifespan costs of fuel cell stacks and battery energy storage systems, respectively. Cost of health consistency across multiple fuel cell stacks.

[0069] Command tracking rewards are determined based on the deviation between the actual output power of the edge node and the power generation demand, and the corresponding formula is: in, This is the instruction tracking deviation penalty coefficient. To meet power generation demand, This refers to the actual output power. The corresponding formula for calculating the actual output power is:

[0070] The potential energy reward shaping term is determined based on the difference between the day-ahead optimal value obtained in S101 and the optimal estimate under the current local policy network. It is a potential energy reward shaping method guided by dynamic programming. The optimal value is calculated offline using dynamic programming, and the potential energy function is designed as follows:

[0071] in, This is the discount factor, which can be between 0.95 and 0.99. For state The potential energy function under the given conditions, s This is the current state. s ' represents the next state, i.e., the state where the action is performed. The state that was later transitioned to To control the weighting coefficient of the potential energy shaping intensity, it is determined through ablation experiments; for example, in this embodiment, it is set to 0.6. The optimal value function is obtained through offline computation using dynamic programming (DP). For the current local policy network The optimal value estimation function under the current local policy network. The optimal value estimation function is calculated according to the following formula:

[0072] in, This is the output value of the Critic network in the current local policy network. This is the entropy coefficient.

[0073] Finally, the economic reward, equipment health reward, instruction tracking reward, and potential energy reward shaping items are weighted and summed to obtain the local decision reward value. The corresponding weight coefficients can be determined according to the actual needs and scenarios, and are not subject to further restrictions here.

[0074] In some embodiments, updating the local policy model based on the soft actor-critic algorithm includes: sampling state, action, reward, and next state samples from the local experience pool, and concatenating power prediction values ​​to obtain an augmented state; constructing a Bellman objective based on the augmented state, calculating the critic network loss, and updating the critic network parameters using gradient descent; calculating the policy loss with the objective of maximizing expected reward and policy entropy, updating the actor network parameters using gradient descent, and repeating this process for multiple iterations to complete the parameter update of the local policy model.

[0075] In the application, from the local experience pool sampling Standardized state Compared with short-term fuel cell power forecasts The splicing results in an enhanced state. , Standardize the state splicing to generate enhanced state This approach uses short-term fuel cell power forecasts as global features, enabling energy storage resource decisions to be based on future fuel cell output trends, thus achieving forward-looking and collaborative scheduling of multiple resources.

[0076] Then, update the Critic network so that its output... Approximate the target Q value y. Specifically, construct the Bellman objective:

[0077] in, The local decision-making reward value. This is the discount factor, which can be 0.99 in this embodiment; For the target Critic network, For state The next state, For the next action, The entropy coefficient, This is the current policy network.

[0078] Using mean squared error as the loss function: Gradient descent is used to update the parameters of the Critic network. : ,in The learning rate for the Critic network. , Indicates an enhanced state Take action below a The expected return is calculated. After the loss converges, the next-state value is calculated using the objective Critic network, and an entropy-based Bellman objective y is constructed, which is minimized by the mean square error. The difference between y and y allows the Critic network to estimate state value more accurately.

[0079] The optimization objective of the policy network is to maximize the weighted sum of expected reward and policy entropy: in, This is the policy entropy weight coefficient, ranging from 0.01 to 0.2; the Actor network parameters are updated by minimizing the policy loss through gradient descent, with the corresponding formula being: Actor network parameters, i.e., current policy network parameters. The calculation formula is: in, The learning rate of the Actor network. The above training strategy enables the policy network to output optimal, stable, and exploratory action vectors. This process is repeated cyclically. After the above iterative training process, the updated parameters are output. , Complete the update of the local strategy model.

[0080] In some embodiments, adjusting the local strategy model parameters based on the difference in the average decision reward obtained when making decisions using the global strategy model and the local strategy model includes: generating active grid support decisions using both the global and local strategy models, statistically analyzing multiple sets of decision rewards and calculating the average decision reward; calculating an adaptive fusion coefficient using the Sigmoid function based on the difference between the two sets of average decision rewards and a preset temperature parameter; and performing a weighted fusion of the global strategy model parameters and the local strategy model parameters according to the adaptive fusion coefficient to obtain personalized local strategy model parameters. The temperature parameter is used to adjust the sensitivity to the difference.

[0081] Among them, the adaptive fusion coefficient The calculation formula is as follows: in, The average decision reward obtained when using the global policy model to make decisions for edge node k; The average decision reward obtained when making a decision using the local policy model for edge node k; This is a temperature parameter used to adjust the sensitivity to differences; This is the Sigmoid function, which maps the output to the (0,1) interval.

[0082] Edge node k receives the global policy model Then, an adaptive personalization fusion mechanism is used to calculate the parameters of the personalized local strategy model, and the corresponding formula is as follows: in, These are the parameters of the local policy model obtained after personalized fusion, which can be used for the next round of local policy model training.

[0083] It is understandable that adaptive personalized fusion can dynamically balance the weight between global general knowledge and local individual characteristics. When the performance of the global strategy model is better than that of the local strategy model at a certain edge node, the adaptive fusion coefficient increases so that the edge node benefits more from the mature decision-making model shared across the system. Conversely, when the local strategy model is more adapted to individual differences, the adaptive fusion coefficient decreases to retain the advantages of the local strategy model.

[0084] Based on the same inventive concept, such as Figure 2 As shown, this application also provides a cloud-edge collaborative active support method for multi-stack fuel cell grids, which is applied to cloud nodes. The cloud-edge collaborative active support method for multi-stack fuel cell grids includes the following steps S201 to S204.

[0085] S201: Distribute the power generation demand to each edge node and initialize the global strategy model.

[0086] In this process, cloud nodes formulate and distribute power generation requirements for each edge node based on grid load forecasts and / or aggregated operational status information reported by each power generation system. Aggregated operational status information refers to macroscopic summary information reported by each edge node to the cloud, excluding detailed operational data of individual devices within the power generation system. This summary includes total power generation, total power generation cost, and the health consistency of internal system devices. For example, after obtaining grid load forecasts and aggregated operational status information, the cloud, aiming for the lowest overall cost and device health consistency, formulates power generation requirements for each edge node while meeting the capacity constraints of each power generation system. Initializing the global strategy model involves initializing the global strategy model parameters for each resource type. Share encoder parameters globally And distribute it to all edge nodes.

[0087] S202: Receive the power prediction model parameters of each edge node and perform aggregation iteration according to the federated learning framework, and then send the obtained target model parameters to each edge node.

[0088] The specific implementation scheme of S202 has been described in the aforementioned embodiments and will not be repeated here.

[0089] S203: Obtain the local policy model parameters of each edge node, perform cross-system model parameter aggregation of the same resource type, and obtain the federated policy model.

[0090] In the application, cloud nodes perform federated weighted aggregation of local policy models for similar resources from different multi-stack fuel cell power generation systems to obtain the federated policy model parameters for that type of resource. The corresponding formula is: in, The parameters for the federated policy model corresponding to the resource type r obtained after aggregation include the parameters of the federated shared encoder and the parameters of the resource-specific decision head network. It is a power generation system k The number of samples of resource type r. It is the aggregate weight of edge node k. It is an edge-side node k The number of samples of resource type r in the middle. This represents its recent average reward value, and τ is a temperature coefficient used to adjust the sensitivity of "performance differences" to the weights. It can be understood that federated weighted aggregation considers both local policy rewards, allowing high-quality policies to be exponentially amplified and rapidly shared within the federated policy model, and sample size, ensuring that when the federated policy model encounters a local policy model with comparable policy rewards, it trusts the model with richer data. Although the two types of resources have different physical characteristics, they share similar hydrogen price response patterns and time-series decision-making logic; therefore, the same processing logic can be used.

[0091] S204: Perform cross-resource type knowledge distillation using the federated policy model as the teacher model, train the global policy model, and distribute the trained global policy model parameters to all edge nodes.

[0092] It is understandable that the cloud will use the federated policy model obtained by integrating the model capabilities of various edge nodes as the teacher model. It will use cross-resource type knowledge distillation technology to complete knowledge transfer and train the global policy model. This will enable the global policy model to fully learn the mature decision-making logic under different resource types and the operating experience of each edge node, forming a global decision-making capability that can adapt to the overall scheduling of the entire network and take into account the operating characteristics of multiple types of resources. After the global policy model is trained, the cloud will uniformly distribute the finalized global policy model parameters, including the global shared encoder parameters and the resource-specific decision head network parameters, to all edge nodes. This will provide global guidance for each edge node to optimize its local decision model. This process only transmits model parameters between the cloud and the edge. The original operating data always remains on the edge side, which ensures data privacy and security while realizing the collaborative optimization of the entire network model.

[0093] It should be noted that the cloud-edge collaborative multi-stack fuel cell grid active support method provided in this application embodiment is based on the same inventive concept as the cloud-edge collaborative multi-stack fuel cell grid active support method provided in the previous embodiment. Therefore, the beneficial effects of this embodiment relative to related technologies can be referred to the description of the foregoing embodiment, and repeated parts will not be repeated.

[0094] In some embodiments, cross-resource type knowledge distillation is performed using a federated policy model as the teacher model to train a global policy model, and the trained global policy model parameters are distributed to all edge nodes. This includes: defining the global policy model as a student model and initializing the global policy model parameters to the average value of the parameters of each teacher model; sampling state vector samples from a shared dataset of cloud nodes; and training the global policy model based on the target loss function and the state vector samples to obtain the global policy model. The shared dataset of cloud nodes is formed by selectively uploading local experience pool data from each edge node; the target loss function includes a distillation loss sub-function and a uniformity loss sub-function.

[0095] In the application, after acquiring the federated policy model, knowledge distillation is initiated in the cloud, and a global policy model is trained using a contrastive learning method. This enables it to extract general response pattern features across resource types. First, the federated policy model is used as the teacher model in knowledge distillation, and the global policy model is used as the student model. The parameters of the global policy model are initialized to the average value of the teacher model parameters, as shown in the following formula:

[0096] in, Indicates resource type r The global decision head network takes the state features encoded by the globally shared encoder as input. The output is an action. a Distribution; These are the teacher model parameters. It should be noted that the local policy model parameters uploaded to the cloud by each edge node are federated and weighted in the first stage to obtain the federated policy model. The federated policy model has a similar structure to the local policy model, consisting of a federated shared encoder network and multiple resource-specific decision head networks. The federated shared encoder and the productivity-oriented decision head network together form the productivity-oriented teacher model, and the federated shared encoder and the energy storage-oriented decision head network together form the energy storage-oriented teacher model.

[0097] Then, the dataset is shared from the cloud nodes. Independent state vector samples are sampled in the middle, and the dataset is shared by cloud nodes. Each edge node selectively draws from its local experience pool. Upload; calculate the square of the Euclidean distance between the output of each teacher model and the output of the global policy model as a distillation loss sub-function, forcing the output of the global policy model to approximate the output of the teacher model, and use the Gaussian kernel function to calculate the uniformity loss of the distance between different samples in the shared representation space as a uniformity loss sub-function to prevent feature collapse.

[0098] Distillation loss subfunction for; in, It's about the state. s The mathematical expectation; It is the square of the Euclidean distance; It is a balance coefficient used to adjust for uniformity loss. The weights in the total loss were determined experimentally. It is the output of the teacher model. It is the output of the global strategy model.

[0099] And design uniformity loss subfunction This prevents feature representation collapse by increasing the distance between different samples in the shared representation space, ensuring sufficient diversity in the encoding space and preventing all samples in the shared representation space from collapsing to the same point. The corresponding formula is:

[0100] in, s , s ' indicates from a shared dataset Two independently sampled samples from different states; It is a Gaussian kernel function; Represents two independent and identically distributed states s and s The joint expectation of ′.

[0101] The global policy model is updated by minimizing the weighted sum of distillation loss and uniformity loss through backpropagation. This enables it to make decisions across resource types: In this embodiment, by comparing and learning to extract the decision-making capabilities of different types of resources in supporting the power grid, and designing uniformity loss to prevent representation collapse, knowledge collaboration and model optimization across resource types can be achieved.

[0102] In some embodiments, such as Figure 3 As shown, this application also provides a multi-stack fuel cell grid active support system based on cloud-edge collaboration, which includes an edge node 31 and a cloud node 32.

[0103] Edge node 31 and cloud node 32 can be edge node and cloud node of any of the above schemes, and execute the active support method for multi-stack fuel cell grid based on cloud-edge collaboration of any of the above schemes.

[0104] It should be noted that the cloud-edge collaborative multi-stack fuel cell grid active support system provided in this application embodiment and the cloud-edge collaborative multi-stack fuel cell grid active support method provided in this application embodiment are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned cloud-edge collaborative multi-stack fuel cell grid active support method, and the repeated parts will not be described again.

[0105] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described active support method for multi-stack fuel cell grids based on cloud-edge collaboration.

[0106] This application also provides a non-transitory computer storage medium storing a computer program thereon. When executed by a processor, this program implements the aforementioned cloud-edge collaborative multi-stack fuel cell grid active support method described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0108] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application, even if such combinations or combinations are not explicitly described in this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration, characterized in that, Applied to edge-side nodes, the method includes: Receive the power generation demand sent by the cloud node and determine the day-ahead optimal value based on the day-ahead hydrogen supply attributes; A power prediction model is trained based on time-series data of historical stack output, power generation demand and hydrogen consumption. The power prediction model parameters are uploaded to the cloud node, and the target model parameters are received and updated after aggregation and iteration by the cloud node. The energy management problem at the edge is modeled as a Markov decision process. The physical states of various resource types collected at the receiving end are combined with the output of the power prediction model to construct a standardized state vector. Based on the current and historical standardized state vectors, a unified action vector is output through the local strategy model to generate active support decisions for the power grid. The decision reward value is calculated based on the system operating parameters and the day-ahead best value. The local strategy model is updated based on the soft actor-commentator algorithm, and the parameters of the local strategy model are uploaded to the cloud node. Receive global strategy model parameters from cloud nodes, and adjust local strategy model parameters based on the difference in average decision reward obtained when making decisions between the global strategy model and the local strategy model.

2. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 1, characterized in that, The power generation demand sent by the cloud node is used to determine the optimal value for the day-ahead hydrogen supply, including: With the goal of minimizing overall operating costs, a stage cost function for dynamic programming is constructed based on the defined state variables and decision variables. Based on the stage cost function, the day-ahead hydrogen supply attributes, the power generation demand, the state variables, and the decision variables, under operational constraints, the Bellman equation is used to recursively solve the problem from the last moment of the scheduling cycle to obtain the day-ahead optimal value.

3. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 1, characterized in that, The physical states of each resource type collected at the receiving end are combined with the output of the power prediction model to construct a standardized state vector, including: Based on the physical state of each resource type and the output results of the power prediction model, construct the state vector of energy production type resources and the state vector of energy storage type resources; By using a learnable embedding function, discrete temporal features and scene categories are mapped to a continuous low-dimensional vector space, and the time-scene joint embedding code is extracted and output. The state vector of the production capacity resource or the state vector of the energy storage resource are concatenated with the time period-scenario joint embedding code to obtain the standardized state vector of the corresponding resource.

4. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 1, characterized in that, The local policy model includes a local shared encoder and a resource-specific decision head network; The process of generating proactive power grid support decisions by outputting a unified action vector through a local policy model based on current and historical standardized state vectors includes: The local shared encoder encodes the current and historical standardized state vectors to extract shared feature vectors. The shared feature vector and the resource type embedding vector are input into the resource-specific decision head network set in parallel, and a unified action vector is output. The unified action vector is mapped to form an active support decision for the power grid through a preset action mapping function. The active support decision for the power grid includes the output power and start-stop control commands of the fuel cell stack, as well as the charging and discharging power and target state of charge commands of the battery energy storage system.

5. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 1, characterized in that, The day-ahead hydrogen supply attribute includes the day-ahead hydrogen price; the calculation of the decision reward value based on system operating parameters and the day-ahead optimal value includes: Based on the aforementioned day-ahead optimal value and the optimal estimate under the current local policy model, potential energy reward shaping is performed, and the potential energy reward shaping term is obtained by solving for it. The economic incentive is calculated based on the direct hydrogen consumption of the fuel cell stack, the equivalent hydrogen consumption of the battery energy storage system, and the day-ahead hydrogen price; the equipment health incentive is calculated based on the equipment loss cost and the health consistency cost of multiple fuel cell stacks; and the instruction tracking incentive is calculated based on the deviation between the actual output power and the power generation demand. The potential energy reward shaping item, economic reward, equipment health reward, and instruction tracking reward are weighted and integrated according to preset weights to obtain the local decision reward value.

6. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 1, characterized in that, The local policy model updated based on the soft actor-critic algorithm includes: The enhanced state is obtained by sampling state, action, reward, and next state samples from the local experience pool and splicing the power prediction value. Based on the enhanced state, a Bellman objective is constructed, the critic network loss is calculated, and the critic network parameters are updated using gradient descent. The policy loss is calculated with the goal of maximizing expected return and policy entropy. The actor network parameters are updated by gradient descent. After multiple iterations, the parameters of the local policy model are updated.

7. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in any one of claims 1 to 6, characterized in that, The adjustment of local policy model parameters based on the difference in average decision rewards obtained when making decisions based on the global policy model and the local policy model includes: The active support decisions for the power grid are generated using both a global strategy model and a local strategy model, and multiple sets of decision rewards are statistically analyzed and the average decision reward is calculated. Based on the difference between the average values ​​of the two sets of decision rewards and a preset temperature parameter, an adaptive fusion coefficient is calculated using the Sigmoid function, wherein the temperature parameter is used to adjust the sensitivity of the difference. The global policy model parameters and the local policy model parameters are weighted and fused based on the adaptive fusion coefficients to obtain personalized local policy model parameters.

8. A method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration, characterized in that, Applied to cloud nodes, the method includes: The power generation demand is distributed to each edge node, and the global strategy model is initialized. Receive the power prediction model parameters of each edge node and perform aggregation and iteration according to the federated learning framework, and then send the obtained target model parameters to each edge node. Obtain the local policy model parameters of each edge node, perform cross-system model parameter aggregation for the same resource type, and obtain the federated policy model; The global strategy model is trained by using the federated strategy model as the teacher model to perform cross-resource type knowledge distillation, and the trained global strategy model parameters are distributed to all edge nodes.

9. The method for active grid support of multi-stack fuel cell power grids based on cloud-edge collaboration as described in claim 8, characterized in that, The process of performing cross-resource type knowledge distillation using the federated policy model as the teacher model to train the global policy model, and then distributing the trained global policy model parameters to all edge nodes, includes: Define the global strategy model as the student model and initialize the parameters of the global strategy model to the average value of the parameters of each teacher model; State vector samples are sampled from a cloud node shared dataset, wherein the cloud node shared dataset is formed by local experience pool data selectively uploaded by each edge node; The global policy model is trained based on the target loss function and the state vector samples to obtain the global policy model. The target loss function includes a distillation loss sub-function and a uniformity loss sub-function.

10. A multi-stack fuel cell grid active support system based on cloud-edge collaboration, characterized in that, The system includes edge nodes and cloud nodes; The edge-side node is used to execute the cloud-edge collaborative active support method for multi-stack fuel cell grids as described in any one of claims 1 to 7; The cloud node is used to execute the cloud-edge collaborative active support method for multi-stack fuel cell grids as described in claim 8 or 9.

Images