Electric hydrogen alcohol ammonia system hierarchical constraint multi-agent scheduling method, system, device and medium

Abstract

The application discloses an electric-hydrogen-olivine system layered constraint multi-agent scheduling method, system, device and medium, relates to the intelligent scheduling control technical field of comprehensive energy system, and comprises the following steps: constructing a coupled system physical operation model to represent power and constraints, establishing a layered multi-agent control framework composed of a system-level coordination agent and multiple device-level execution agents, determining reinforcement learning specifications, constructing a layered constraint strategy optimization model embedded with self-attention feature fusion, using self-attention to generate fusion features for layered strategy mapping, and introducing a strategy update constraint and value evaluation network, training a global network using centralized training and decentralized execution, adjusting the constraint violation degree influence of dual variables dynamically, and outputting control actions online by each execution agent after training. The application improves new energy consumption, hydrogen storage safety and carbon emission satisfaction rate, reduces electric-hydrogen balance deviation, and guarantees stable and low-carbon operation of the coupled system.

Description

Technical Field

[0001] This invention relates to the field of intelligent scheduling and control technology for integrated energy systems, specifically to a hierarchical constrained multi-agent scheduling method, system, equipment, and medium for an electric hydrogen, alcohol, and ammonia system. Background Technology

[0002] With the continuous increase in the penetration rate of renewable energy power generation, the electro-hydrogen-methanol-ammonia coupled system, as an important technical path to achieve new energy consumption, industrial decarbonization, and multi-energy complementarity, has become a research hotspot in the field of integrated energy systems. The system uses electrolytic hydrogen production as its hub, deeply coupling the power grid with chemical processes such as hydrogen storage, fuel cell power generation, methanol synthesis, and ammonia synthesis. Its operation exhibits a strong nonlinear interaction between electrical power, hydrogen flow, and chemical material flow, and is subject to various physical constraints such as equipment capacity limits, ramp-up rates, energy storage ranges, and carbon emission quotas. Driven by both fluctuations in new energy output and changes in energy demand, how to coordinate and optimize continuous control variables such as power dispatch, hydrogen production capacity, hydrogen storage charging and discharging, and the load rate of methanol and ammonia units in real time is the core challenge for achieving stable and low-carbon operation of the system.

[0003] Existing centralized optimization scheduling methods typically rely on complete system models and accurate source-load prediction information. However, the solution complexity increases dramatically in high-dimensional continuous action spaces, making it difficult to meet the real-time requirements of online rolling decision-making. Deep reinforcement learning methods based on single agents model the control problem of the entire coupled system as a single-entity decision-making process. However, single-policy networks cannot fully express the interaction relationships between multiple subsystems that are physically independent but mutually constrained in operation. During training, problems such as local action infeasibility or global constraint violation accumulation can easily occur. Although planar multi-agent architectures distribute control to each device unit, they lack system-level coordination mechanisms. When agents output continuous control quantities, action conflicts are prone to occur, leading to increased power or material flow balance deviations. Furthermore, there is a lack of effective means to handle global constraints such as carbon emission accumulation constraints that are coupled across time periods. Summary of the Invention

[0004] In view of the above-mentioned existing problems, the present invention provides a hierarchical constraint multi-agent scheduling method, system, device and medium for an electrohydrogen alcohol ammonia system, in order to solve the problems of local infeasibility or global constraint violation accumulation and increased power or material flow balance deviation in the prior art.

[0005] To address the aforementioned technical issues, a hierarchical constrained multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system is proposed, including: A physical operation model of the coupled system is constructed to characterize system power and constraints, and a hierarchical multi-agent control framework is established, including a system-level coordinating agent and multiple device-level execution agents. Based on the physical operation model and the hierarchical multi-agent control framework, reinforcement learning task specifications are determined, and a hierarchical constraint policy optimization model with embedded self-attention interaction feature fusion is constructed to generate fused state features and perform hierarchical policy mapping. At the same time, a policy update constraint mechanism and a value evaluation network are introduced. A centralized training and distributed execution framework is used to train the global network, and the policy is updated according to the constraint violation degree through a dual variable dynamic adjustment mechanism. The online output control actions of each device-level execution agent are used after training.

[0006] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system described in this invention, the hierarchical multi-agent control framework includes a global state as the input of the system-level coordinating agent and an output system-level running reference vector. The reference vector includes reference settings for the level of new energy consumption, hydrogen storage, chemical material storage, chemical plant load, and remaining carbon emission quotas. The process of defining the reinforcement learning task specifications includes defining the state space, action space, performance reward signal, and constraint violation evaluation function for reinforcement learning. The state space includes the global state and the local state of each device-level executive agent.

[0007] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system described in this invention, the generation of fusion state features includes: self-attention interaction feature fusion for each device-level execution agent constructing query mapping, key mapping, and value mapping; calculating the attention weight of the current agent to the state information of the remaining agents; and using the attention weight to perform a weighted summation of each value mapping to obtain the fusion state features.

[0008] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system described in this invention, the hierarchical policy mapping includes: taking the global state as the input of the system-level policy network and outputting a system-level running reference vector; taking the fused state features, local states, and the system-level running reference vector of the corresponding agent as the input of the device-level policy network and outputting device-level control actions. The policy update constraint mechanism includes constructing a policy optimization objective by limiting the magnitude of the difference in probability distribution between the outputs of the old and new policies and combining the advantage function estimate.

[0009] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system described in this invention, the value assessment network includes a system-level value assessment network and a device-level value assessment network. The system-level value assessment network takes the global state and the system-level operating reference vector as input and outputs the system-level state-action value. The device-level value assessment network takes the fused state characteristics, local state, and control actions of the corresponding agent as input and outputs the device-level state-action value.

[0010] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system described in this invention, the method of training the global network using a centralized training and distributed execution framework includes: during the centralized training phase, collecting the global state, all device-level control actions, and the true values ​​of constraint violations; updating the system-level value assessment network using system-level reward estimation; updating each device-level value assessment network using device-level reward estimation; updating the constraint violation assessment network using the true values ​​of constraint violations; and updating the parameters of the system-level policy network and each device-level policy network based on the value estimates and constraint violation predictions output by each network. During the distributed execution phase, the system-level coordinating agent acquires the global state online and generates a system-level operational reference vector. Each device-level execution agent generates device-level control actions using only the local observed state, the system-level operational reference vector, and the interaction features of neighboring devices captured through a self-attention mechanism.

[0011] As a preferred embodiment of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system described in this invention, the dual variable dynamic adjustment mechanism includes updating the dual variables associated with each device-level execution agent during training based on the deviation between the average constraint violation degree of the sampling trajectory and the preset violation threshold. The online output control actions of the executing agents include: each device-level executing agent periodically acquiring local observation states, performing self-attention interaction feature fusion, updating the fused state features, and outputting control commands for the corresponding links through a device-level policy network combined with a system-level running reference vector.

[0012] The beneficial effects of this preferred technical solution are as follows: by adopting hierarchical coordination, self-attention fusion and dual constraint adjustment, it overcomes the problems of high-dimensional action conflict and physical constraint violation in the electricity-hydrogen-alcohol-ammonia system, improves the consumption of new energy, hydrogen storage safety and carbon emission satisfaction rate, reduces the electricity-hydrogen balance deviation, and ensures the stable and low-carbon operation of the coupled system.

[0013] As a preferred embodiment of the hierarchical constraint multi-agent scheduling system for the electrohydrogen alcohol ammonia system described in this invention, it is characterized by comprising a physical operation modeling module, a hierarchical multi-agent control framework construction module, a self-attention interaction feature fusion module, a strategy optimization and constraint evaluation network module, and a centralized training, decentralized execution, and dual adjustment module.

[0014] The physical operation modeling module is used to establish mathematical descriptions of power balance constraints, material conservation constraints, equipment operating boundaries, dynamic storage constraints, and carbon emission limits for the electrical energy, hydrogen energy, alcohol-ammonia chemical and energy storage links in the electro-hydrogen-ethanol-ammonia coupling system. It provides a simulation environment and state transition basis that reflects the multi-energy flow coupling characteristics and various physical constraints for scheduling control.

[0015] The hierarchical multi-agent control framework construction module is used to decompose the high-dimensional continuous control problem into two layers of decision-making by having system-level coordinating agents generate cross-link collaborative operation reference vectors based on the global state, and multiple device-level execution agents generate device control actions based on the local state and reference vectors.

[0016] The self-attention interaction feature fusion module is used to generate fusion state features that integrate multi-device interaction information for each device-level execution agent, enabling the agent to adaptively perceive the state influence of other devices when outputting actions.

[0017] The policy optimization and constraint evaluation network module is used to perform hierarchical policy mapping using a policy network, estimate state-action value using a value evaluation network, predict the degree of constraint violation using a constraint violation evaluation network, and achieve stable learning of the collaborative scheduling policy under constraints by combining with the policy update constraint mechanism.

[0018] The centralized training, distributed execution, and dual adjustment module is used to centrally update the network parameters using global information during the training phase, and to adjust the influence of constraint violation on policy updates by dynamically adjusting the dual variables. During the execution phase, each agent independently outputs actions based only on local information, taking into account both information sufficiency and online response capability.

[0019] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for hierarchical constraint multi-agent scheduling of an electrohydrogen alcohol ammonia system.

[0020] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a method for hierarchical constraint multi-agent scheduling of an electrohydrogen alcohol ammonia system.

[0021] The beneficial effects of this invention are as follows: By establishing a physical operation model encompassing electrical energy, hydrogen energy, alcohol-ammonia chemical industry, and energy storage, this invention characterizes constraints such as power balance, material conservation, equipment capacity and ramp-up, inventory range, and carbon emission accumulation under multi-energy flow coupling, providing a precise state transition basis for online decision-making in complex systems. It combines a system-level coordinating agent with multiple device-level executing agents to form a hierarchical control framework, dividing the high-dimensional continuous action space into two layers: operation reference generation and device instruction execution, reducing system coordination difficulty and minimizing action conflicts. Furthermore, it introduces a self-attention mechanism to dynamically generate feature representations covering different device interaction information for each executing agent, enhancing the control over multiple aspects such as new energy, hydrogen production, hydrogen storage, and alcohol-ammonia loads. The system possesses adaptive sensing capabilities for coupling relationships. During training, it utilizes a constraint violation assessment network and a Lagrange duality update mechanism to map all physical constraints onto a set of learnable violation indices. The penalty intensity is dynamically adjusted based on constraint satisfaction to prevent the generation of infeasible strategies. The system employs centralized training and distributed execution, using global information to fully optimize network parameters while balancing information sufficiency and online response speed. This significantly enhances the capacity for renewable energy absorption, reduces the imbalance between electricity and hydrogen flow, and improves the retention rate of hydrogen storage safety zones, the accuracy of alcohol and ammonia production tracking, and the satisfaction rate of carbon emission constraints and equipment ramp-up constraints. This provides technical support for achieving stable, low-carbon, and coordinated operation of the electricity-hydrogen-alcohol-ammonia coupled system with a high proportion of renewable energy access. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the overall process of a hierarchical constraint multi-agent scheduling method for an electrohydrogen alcohol ammonia system, as provided in one embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the global network training process using a centralized training and decentralized execution framework in a hierarchical constraint multi-agent scheduling method for an electrohydrogen alcohol ammonia system provided in an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the wind power, photovoltaic, and electrical load time series curves of a hierarchical constraint multi-agent scheduling method for an electrohydrogen-ethanol-ammonia system provided in an embodiment of the present invention.

[0026] Figure 4 This diagram illustrates the comparison of new energy absorption rates of different control methods for the hierarchical constraint multi-agent scheduling method of the electrohydrogen-ethanol-ammonia system provided in one embodiment of the present invention.

[0027] Figure 5 This diagram illustrates a comparison of constraint satisfaction rates among different control methods for a hierarchical constraint multi-agent scheduling method for an electrohydrogen alcohol ammonia system, as provided in an embodiment of the present invention.

[0028] Figure 6 The graph shows the trend of the average technical performance enhancement signal and the average constraint violation during the training process of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0030] Example 1, referring to Figure 1 As an embodiment of the present invention, a hierarchical constrained multi-agent scheduling method for an electrohydrogen alcohol ammonia system is provided, comprising: S100: Construct a physical operation model of the coupled system, characterize the system power and constraints, and establish a hierarchical multi-agent control framework that includes a system-level coordinating agent and multiple device-level execution agents.

[0031] S200: Based on the physical operation model and hierarchical multi-agent control framework, the specification of reinforcement learning tasks is determined, and a hierarchical constraint policy optimization model with embedded self-attention interaction feature fusion is constructed to generate fused state features, perform hierarchical policy mapping, and introduce a policy update constraint mechanism and a value evaluation network.

[0032] S300: The global network is trained using a centralized training and distributed execution framework, and updated according to the constraint violation adjustment strategy through a dual variable dynamic adjustment mechanism. After training, each device-level execution agent outputs control actions online.

[0033] It should be noted that this invention solves the problems of difficult coordination of high-dimensional continuous actions and difficulty in satisfying physical constraints in the electricity-hydrogen-alcohol-ammonia coupling system by constructing a complete physical operation model, a hierarchical multi-agent control framework, self-attention interaction feature fusion, constraint violation evaluation network, and dual variable dynamic adjustment mechanism, and by centralized training and decentralized execution. It significantly improves the renewable energy absorption rate, hydrogen storage safety range maintenance rate, and carbon emission constraint satisfaction rate, and reduces the balance deviation of electricity and hydrogen flow. It provides reliable scheduling technology support for the stable and low-carbon operation of the coupling system under high proportion of renewable energy access.

[0034] Example 2, refer to Figure 1 and Figure 2 This is a second embodiment of the present invention, which provides a hierarchical constrained multi-agent scheduling method for an electrohydrogen alcohol-ammonia system, comprising: In step S100, the construction of the physical operation model of the coupled system includes, in order to achieve precise control of the electro-hydrogen-ethanol-ammonia coupled system, the construction of the physical operation model of the electro-hydrogen-ethanol-ammonia coupled system is required. This model is also the foundation that all subsequent intelligent agents must have for decision-making and training. At the same time, it quantifies the flow and conversion relationship between electrical energy, hydrogen energy, and chemical materials (methanol, ammonia) within the system, defines the operating boundaries of all equipment and the physical constraints that must be met, including the electrical energy subsystem model, the hydrogen energy subsystem model, the methanol synthesis subsystem model, the ammonia synthesis subsystem model, the energy storage and material storage subsystem model, and the carbon emission constraint model.

[0035] S101: Construct a power subsystem model to describe the system's power balance constraints and the operating boundaries of various power sources and loads. The key power balance constraint formula is expressed as follows: in, Let be the output of the k-th controllable unit at time t. For the actual power consumption of the w-th wind farm, This represents the total number of controllable generating units. The total number of wind farms The total number of photovoltaic power stations. For the total number of fuel cells, The actual power consumption of the s-th photovoltaic power station at time t. Let f be the output power of the f-th fuel cell at time t. Let t be the system electrical load demand. Let be the input power of the g-th electrolytic cell at time t. This represents the total number of electrolytic cells. Let be the power consumption of the c-th chemical unit (such as a methanol synthesis or ammonia synthesis unit) at time t. This represents the total number of chemical engineering units. Let t be the equivalent power loss of the system at time t.

[0036] It should be noted that the remaining constraints include the fact that the output of controllable units must meet their own upper and lower limits and maximum ramp rate constraints; the actual output of wind power and photovoltaic power cannot exceed their own available output limit and cannot be lower than 0, all of which are described in the form of inequalities.

[0037] It should also be noted that the maximum climbing rate constraint is expressed as: in, This represents the maximum ramp rate for the controllable unit. Let be the output of the kth controllable unit at time t-1.

[0038] S102: The hydrogen energy subsystem model includes hydrogen production via electrolyzer, hydrogen storage via hydrogen storage tank, and power generation via fuel cell.

[0039] Among them, the hydrogen production capacity of the electrolyzer is determined by the input power and the electro-hydrogen conversion coefficient; the dynamic storage capacity of the hydrogen storage tank is obtained by adding the hydrogen filling amount to the storage capacity at the previous moment, subtracting the hydrogen release amount and self-loss; and the output power of the fuel cell is determined by the hydrogen consumption and conversion efficiency.

[0040] It should be noted that the calculation formula for hydrogen production by electrolyzer is expressed as follows: in, Let be the hydrogen production rate of the g-th electrolyzer at time t. The electro-hydrogen conversion coefficient, For the input power of the electrolytic cell, For auxiliary equipment power, This is the scheduling time interval.

[0041] Meanwhile, the input power of the electrolyzer, the storage capacity of the hydrogen storage tank, and the hydrogen charging and discharging flow rate must all be maintained within their respective safe upper and lower limits.

[0042] S103: The methanol synthesis subsystem model and the ammonia synthesis subsystem model include the following: the output of methanol and ammonia is determined by the rated capacity and current load rate of the unit, while the hydrogen consumption is proportional to the output (determined by the hydrogen consumption coefficient per unit output). The load rate of the unit must be between the specified upper and lower limits, and the load change between adjacent times cannot exceed its maximum ramp rate.

[0043] S104: The energy storage and material storage subsystem model includes: the storage capacity of methanol and ammonia changes over time, and the storage capacity must be maintained between the minimum and maximum safety limits. The dynamic change formula for the hydrogen storage tank is expressed as: in, Let be the amount of hydrogen stored in the r-th hydrogen storage tank at the end of time t. Let be the amount of hydrogen stored in the r-th hydrogen storage tank at the end of time t-1. Let be the hydrogen filling efficiency of the r-th hydrogen storage tank, dimensionless. Let be the hydrogen charging flow rate of the r-th hydrogen storage tank at time t. Let be the hydrogen release flow rate of the r-th hydrogen storage tank at time t. Let be the hydrogen release efficiency of the r-th hydrogen storage tank, which is dimensionless. Let be the self-loss coefficient of the r-th hydrogen storage tank.

[0044] S105: The carbon emission constraint model includes the following: the total carbon emissions of the system are obtained by adding up the carbon emissions from the controllable unit power generation, methanol production and synthetic ammonia production, and the total emissions within a scheduling cycle shall not exceed the prescribed upper limit.

[0045] It should be noted that the formula for calculating the system's total carbon emissions is as follows: in, Let be the total carbon dioxide emissions produced by the system within a time step Δt at time t. Let be the carbon emission intensity of the k-th controllable unit (such as a thermal power generating unit), representing the carbon emissions generated per ton of methanol produced. The carbon emission intensity of the methanol synthesis process. Let be the methanol production at time t. The carbon emission intensity of the ammonia synthesis process represents the amount of carbon emissions generated per ton of ammonia produced. Let be the ammonia production at time t.

[0046] In step S100, the hierarchical multi-agent control framework includes constructing a hierarchical multi-agent control framework after establishing a complete physical model, reducing the complexity and coordination of the problem. The core of the hierarchical multi-agent control framework is to decompose the entire scheduling task into the planning of the target on two boards and the execution of the actions below, specifically including steps S111~S113: S111: The input to the system-level coordinating agent is the global state, which consists of the scheduling time, predicted renewable energy output, predicted energy demand, energy status of energy storage devices, chemical material storage capacity, chemical plant load rate, cumulative carbon emissions, and vectors representing the current relaxation levels of various constraints. in, This represents the global state of the system-level coordinating agent at time t. Let t be the available wind power output. Let t be the available photovoltaic output. Let be the amount of hydrogen stored at time t. Let be the methanol reserves at time t. Let be the ammonia reserves at time t. Let t be the loading rate of the methanol synthesis unit. Let t be the loading rate of the ammonia synthesis unit. This represents the cumulative carbon emissions from the start of the scheduling period to time t. The constraint margin vector at time t reflects the degree of relaxation or safe distance of various physical constraints (such as upper and lower limits of reserves, power limits, etc.) at the current time.

[0047] S112: The system-level coordinating agent outputs a system-level operation reference vector, which includes reference settings for the level of new energy consumption, hydrogen storage, chemical material storage, chemical plant load, and remaining carbon emission quotas.

[0048] It should be noted that the system-level runtime reference vector is represented as follows: in, This is the system-level operational reference vector output by the system-level coordinating agent at time t. This is a reference value for renewable energy consumption, representing the system-level recommended renewable energy consumption target. This is a reference quantity for hydrogen storage, representing the system-level recommended target hydrogen storage capacity. This is a reference figure for methanol reserves, representing the system-level recommended target for methanol storage tank capacity. This is a reference quantity for ammonia storage, representing the system-level recommended ammonia storage tank capacity target. This is a reference load for the methanol synthesis unit, representing the system-level recommended target load rate for the methanol unit. This is a reference load for the ammonia synthesis unit, representing the system-level recommended target load rate for the ammonia synthesis unit. This serves as a reference for carbon emission margin, representing a system-level recommendation for the use of remaining carbon emission allowances.

[0049] S113: The local state, action vector, and state transition relationship of the device-level execution agent are as follows: in, Let be the local state of the i-th device-level execution agent at time t. This represents the operating state of the i-th device itself. The observation status of adjacent devices, Let be the constraint margin of the i-th device, representing the degree of relaxation of the current device at its current distance from the constraint boundary. Let t be the total action vector output by all device-level executing agents. The input power to the electrolyzer controls the hydrogen production rate. To control the hydrogen filling rate of the hydrogen storage tank. To control the hydrogen release rate from the hydrogen storage tank. To control the output power of the fuel cell and the amount of hydrogen generated, This is the adjustment amount for the load rate of the methanol synthesis unit. This is the adjustment amount for the load rate of the ammonia synthesis unit. To control the external supply rate of methanol, To control the external supply rate of ammonia, Let be the system state vector at time t+1. Let be the system state vector at time t. The state transition function is determined by the combined factors of electrical energy balance, hydrogen flow balance, material balance, equipment capacity, and ramp-up constraints. The external disturbance vector includes uncertainties such as disturbances in available power output from new energy sources, electrical load disturbances, and equipment efficiency disturbances.

[0050] It should be noted that the agent, by fusing its own local state, generates specific device-level control actions after the system-level coordination agent sends a running reference vector and obtains features from neighboring devices through a self-attention mechanism. For example, in hydrogen production, the agent outputs the power setpoint of the electrolyzer, and the set of actions of all devices forms the total action vector. When this action acts on the system, the system state is updated to the next time step according to the state transition function determined by the physical model (such as power balance and hydrogen flow balance). In addition, the system is currently subject to random disturbances (such as renewable energy output, electrical load, and equipment efficiency). In step S200, determining the reinforcement learning task specifications includes determining the state space, action space, performance reward signal, and constraint violation evaluation function. S201: Determining the state space and action space includes: the state space, composed of the system-level global state and the local states of each device level as defined in steps S111 and S112, provides all the necessary information for the entire system (e.g., new energy sources, loads, energy storage, materials, constraint margins); the action space is the set A of control actions output by all device-level executing agents. t It includes continuous variables such as electrolyzer power, hydrogen charging and discharging flow rate, fuel cell power, and load rate adjustment of alcohol-ammonia unit.

[0051] S202: The performance reward signal, i.e., the technical performance enhancement signal, includes, in order to guide the agent to learn the optimal strategy, the following indicators were designed: new energy absorption rate, power balance deviation, hydrogen flow balance deviation, alcohol and ammonia production tracking deviation, inventory safety margin, and carbon emission margin. The formula for calculating the new energy absorption rate is as follows: in, Let be the renewable energy absorption rate at time t, reflecting the actual utilization rate of renewable energy power generation. Let w be the available output of the w-th wind farm at time t, which is the theoretically maximum wind power it can generate. Let be the available output of the s-th photovoltaic power station at time t, which is the theoretically maximum photovoltaic power it can generate. It is a very small positive number (10) -6 ( ), to prevent the denominator from being zero.

[0052] The power balance deviation and hydrogen flow balance deviation are expressed as follows: in, Let be the absolute value of the power balance deviation at time t, used to penalize imbalances in power supply and demand. Let be the absolute value of the hydrogen flow balance deviation at time t, used to penalize hydrogen supply and demand imbalances. This represents the amount of hydrogen consumed in the methanol synthesis process at time t. Let be the amount of hydrogen consumed in the ammonia synthesis process at time t. Let f be the hydrogen consumption of the f-th fuel cell at time t. The formulas for calculating the tracking deviation of alcohol and ammonia production, inventory safety margin, and carbon emission margin are as follows: in, The tracking deviation of methanol and ammonia production at time t reflects the difference between the actual and planned production of methanol and ammonia. Let be the actual methanol production at time t. Let t be the planned output of methanol production at time t. The actual ammonia production at time t. Let t be the planned output of synthetic ammonia production at time t. Let be the inventory safety margin at time t, taking the minimum value among all inventory indicators to reflect the most dangerous storage state. and These are the minimum and maximum safe limits for hydrogen storage capacity. and These are the minimum and maximum safe limits for methanol reserves. and These are the minimum and maximum safe limits for ammonia reserves. Let be the carbon emission margin at time t, reflecting the sufficiency of remaining carbon emission allowances. The upper limit of carbon emissions allowed during the scheduling cycle. For time indexing.

[0053] It should be noted that, taking into account the above indicators, the system-level technical performance enhancement signal is obtained by weighted summation of the above indicators, minus a penalty term consisting of the total system constraint violation rate, as expressed by the formula: in, This is the system-level technical performance enhancement signal at time t. , , , , , and These are the weighting coefficients for the indicators, used to balance the importance of different objectives. Let be the total constraint violation value at time t. This is a signal to enhance the device-level technical performance of the i-th device-level execution agent at time t. For the balance deviation associated with the i-th device, This represents the operating reference deviation of the i-th device, which is the deviation between the actual output of device i and the corresponding reference value in the system-level reference vector. The penalty for the i-th device's ramp deviation is the degree to which the rate of change of the i-th device's output exceeds the maximum allowable ramp rate. Let i be the constraint violation degree corresponding to the i-th device. The safety margin for the i-th device reflects the degree of relaxation of the current operating point of device i from the constraint boundary. , , , and These are the weighting coefficients for each indicator.

[0054] S203: Determine the constraint violation evaluation function. To ensure that control actions meet the requirements of all physical constraints, a constraint violation function is given. This function is a weighted sum of the squared violations of all inequality constraints (such as equipment capacity and ramp rate) and a weighted sum of the absolute violations of all equality constraints (such as power balance equations). The formula is expressed as: in, For the q-th inequality constraint, For the p-th equality constraint, When the inequality constraint is violated, i.e. When the constraint is satisfied, the square is taken as the penalty value. When the time is right, the penalty is 0. This is the absolute violation of the equality constraint, that is, the absolute value of the difference between the two sides of the equation. Let q be the violation weight coefficient of the inequality constraint. is the violation weight coefficient of the p-th equality constraint.

[0055] Furthermore, in step S200, the introduction of the policy update constraint mechanism and value evaluation network includes building a deep reinforcement learning model, namely a hierarchical constraint multi-agent proximal policy optimization model, so that the agents can learn the optimal cooperative scheduling policy based on the reward signal, specifically including steps S211~S215: S211: Self-attention interaction feature fusion: Each device-level execution agent constructs query mapping, key mapping, and value mapping, calculates the attention weight of the current agent to the state information of the remaining agents, and uses the attention weights to perform a weighted summation of each value mapping to obtain the fused state features. The formula for calculating the attention weights is expressed as: in, For the first The attention weights of each device-level execution agent to the state features of the j-th agent. For the first The query vector of each agent Let be the transpose of the key vector of the j-th agent. Scaling factor Let be the dimension of the key vector. This represents the total number of device-level execution agents in the system.

[0056] S212: The hierarchical policy mapping includes a system-level policy network that takes the global state as input and outputs a system-level operational reference vector; and a device-level policy network that takes the fused state features, local states, and system-level operational reference vector of the corresponding agent as input and outputs device-level control actions. It should be noted that the formula for the fusion state characteristics of the computational agent is expressed as: in, Let j be the value vector of the j-th agent. For the first The fusion state characteristics of each device-level execution agent.

[0057] S213: The policy update constraint mechanism constructs a policy optimization objective by limiting the difference in probability distribution between the outputs of the old and new policies and combining it with the advantage function estimate.

[0058] To ensure training stability, a pruning mechanism from Proximal Policy Optimization (PPO) is used as a policy update constraint mechanism. (Device-level...) The optimization objective of an agent is to update the policy network by limiting the update magnitude of the new and old policies to avoid policy collapse. The core idea is to prune the objective function when the probability ratio of the new and old policies exceeds a predetermined range, thereby limiting the step size of a single update. The formula is expressed as: in, For the first The optimization objective of the shearing strategy for each device-level execution agent. For the first Parameters of a device-level policy network. Let $\mathematical expectation$ represent the average of the sampled data over time step $t$. The probability ratio between the old and new strategies. For the first Old parameters for a device-level policy network. For action, Take action under the new strategy (the strategy currently awaiting update). The probability, This represents the probability of taking the same action under the old strategy (the strategy before the update). For the first The estimated advantage function of an agent at time t. For shearing function, The shear coefficient is set to 0.20 in this embodiment.

[0059] S214: The constraint violation evaluation network takes the global state and all device-level control actions as inputs and outputs the predicted constraint violation degree. It predicts the possible constraint violation degree under the current policy and uses the constraint violation degree for dynamic adjustment of the dual variable.

[0060] S215: The value assessment network includes a system-level value assessment network and a device-level value assessment network. The system-level value assessment network takes the global state and the system-level operational reference vector as input and outputs the system-level state-action value. The device-level value assessment network takes the fused state features, local state and control actions of the corresponding agent as input and outputs the device-level state-action value to guide the update of the policy network.

[0061] like Figure 2 In step S300, the training of the global network using a centralized training and distributed execution framework includes steps S301 to S303: S301: During the intensive training phase, collect the global state, all device-level control actions, and the true values ​​of constraint violations. Update the system-level value assessment network using the system-level reward estimate, update each device-level value assessment network using the device-level reward estimate, update the constraint violation assessment network using the true values ​​of constraint violations, and update the parameters of the system-level policy network and each device-level policy network based on the value estimate and constraint violation prediction values ​​output by each network.

[0062] It should be noted that the strategy optimization objective is based on the combined outputs of the value assessment network and the constraint violation assessment network. Each policy network is updated, and the reward signal calculated during the update guides the policy network update through the dominance function.

[0063] S302: The dual variable dynamic adjustment mechanism updates the dual variable associated with each device-level execution agent during training based on the deviation between the average constraint violation degree of the sampled trajectory and the preset violation threshold, and uses the dual variable to adjust the influence weight of constraint violation degree on the policy update target.

[0064] It should be noted that, in order to achieve a balance between rewards (pursuing high performance) and constraints (ensuring feasibility), a dynamic adjustment mechanism for dual variables is introduced. In this embodiment, it is the Lagrange dual update method, which maintains a Lagrange multiplier for each agent. The update method is expressed as follows: in, For the multiplier in the nth update, For the (n+1)th update, For learning rate, The average constraint violation rate of the sampled trajectory. The default violation threshold is used to adjust the penalty weight of constraint violation in the reward function during policy updates. When the average constraint violation exceeds the threshold, the multiplier increases, which strengthens the impact of the penalty term and makes the policy more conservative.

[0065] It should also be noted that training iterations continue until the convergence condition is met, i.e., the average technical performance enhancement signal changes very little over two consecutive training rounds and the average constraint violation is below a preset threshold, at which point updates cease; the model training convergence condition is expressed as: in, This is a signal to enhance the average technical performance of the e-th training round. This is a signal to enhance the average technical performance of the (e-1)th training round. To enhance the signal change threshold, Let e ​​be the average constraint violation rate during the e-th training round. The threshold for overall constraint violation.

[0066] S303: During the distributed execution phase, the system-level coordinating agent acquires the global state online and generates the system-level operating reference vector. Each device-level executing agent independently generates device-level control actions using only its own local observation state, the system-level operating reference vector, and the interaction features of neighboring devices captured through the self-attention mechanism, without relying on the complete action information and global state of the remaining devices.

[0067] Furthermore, in step S300, the online output control actions of each device-level execution agent after training include multiple device-level execution agents corresponding to the power dispatching, hydrogen production, hydrogen storage, fuel cell power generation, methanol synthesis, ammonia synthesis and material storage links respectively. In the distributed execution stage, each device-level execution agent periodically acquires the local observation state, performs self-attention interaction feature fusion to update the fused state features, and outputs the control instructions of the corresponding link through the device-level policy network combined with the system-level operation reference vector.

[0068] Among them, after the control command is applied to the physical operation model, the system state is updated according to the state transition relationship determined by power balance, material flow balance and equipment capacity and adjustment rate limit, forming a closed-loop rolling optimization scheduling control.

[0069] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0070] Example 3, referring to Figures 3-6 This is the third embodiment of the present invention, which provides a hierarchical constraint multi-agent scheduling method for an electrohydrogen alcohol ammonia system. To verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0071] This embodiment uses a simplified and improved IEEE 14-node system as the power network, and sets up wind power, photovoltaic, electrolyzer, hydrogen storage tank, fuel cell, methanol synthesis unit and ammonia synthesis unit. The scheduling cycle is 30 days, with one time step per hour, for a total of 720 time steps. The parameter settings are shown in Table 1. Table 1 Parameter Setting Table

[0072] The parameter settings for the methanol synthesis unit and the ammonia synthesis unit, as well as the model training parameter settings, are shown in Tables 2 and 3. Table 2 Parameter Setting Table for Methanol Synthesis Unit and Ammonia Synthesis Unit

[0073] Table 3 Model Training Parameter Settings

[0074] Before training begins, the system-level policy network, device-level policy networks, system-level value network, device-level value network, constraint violation evaluation network, and Lagrange multipliers are initialized. Each training round starts from the typical daily initial state of the system. At each scheduling time step, the system-level coordinating agent generates a system-level running reference vector based on the current global state. Each device-level executing agent generates its own device-level control action based on its local state, the received system-level running reference vector, and the fused state features calculated through a self-attention mechanism.

[0075] After all agents have performed their actions, the system state is updated to the next time step according to the state transition function. At the same time, the reward value and constraint violation degree of the current step are calculated according to the technical performance reinforcement signal formula and the constraint violation degree function, respectively. The generalized advantage estimation method is used to calculate the advantage function estimate and the system-level and device-level reward estimates. Each policy network is updated according to the policy optimization objective formula, the system-level and device-level value networks are updated according to the value network loss function formula, the constraint violation evaluation network is updated according to the constraint violation evaluation network loss function formula, and the Lagrange multipliers associated with each device-level agent are updated according to the Lagrange multiplier update formula.

[0076] The above process is repeated at each time step until all time steps of a training round are completed. When the average change in technical performance enhancement signal between consecutive training rounds is less than a preset threshold and the average constraint violation is lower than a preset upper limit, the model is determined to have converged and training is stopped. If it has not converged, the next round of training continues until the convergence condition is met or the preset maximum number of training rounds is reached.

[0077] Furthermore, this embodiment also demonstrates the simulation verification process of the method in the coordinated scheduling and control of an electricity-hydrogen-methanol-ammonia coupled system. Taking a typical coupled system including wind power, photovoltaic, electrolyzer, hydrogen storage tank, fuel cell, methanol synthesis unit, and ammonia synthesis unit as the object, the implementation method is as follows: First, parameters such as system installed capacity, equipment rated power, hydrogen storage capacity, methanol and ammonia storage range, unit load rate range, ramping constraints, and carbon emission constraints are set; then, the available output of wind power and photovoltaic, electrical load, equipment efficiency disturbance, and methanol and ammonia production reference curves are generated by superimposing typical daily curves with random disturbances; finally, comparative simulations are performed using the rule-based scheduling method, the single-agent PPO method, the planar MAPPO method, and the method of this invention, respectively.

[0078] Figure 3The figures show the time-series curves of wind power, photovoltaic (PV) power, and electrical load in the simulation example. The available output of wind power and PV power is generated by normalizing the diurnal variation curve and the random fluctuation curve, respectively, and then superimposed with disturbances within the range of -8% and 8%. The electrical load is generated by superimposing the morning and evening peak load curves with disturbances within the range of -5% and 5%. The methanol and synthetic ammonia production tasks are generated using intraday smoothed reference curves to ensure that the reference load change at adjacent times does not exceed the equipment ramp-up range. The equipment efficiency is superimposed with an operating disturbance of no more than 3% near the rated value. All four control methods are tested under the same 30-day disturbance sequence, and the network parameters are not updated during the testing phase.

[0079] Figure 4 The results show a comparison of the renewable energy absorption rates using different control methods. Figure 5 The figure shows a comparison of constraint satisfaction rates for different control methods. Figure 6 The table shows the changing trends of the average technical performance enhancement signal and average constraint violation during the training process of the method of this invention. The evaluation indicators include the new energy absorption rate, the average absolute deviation of power balance, the average absolute deviation of hydrogen flow balance, the retention rate of the hydrogen storage safety range, the tracking error of methanol production, the tracking error of synthetic ammonia production, the carbon emission constraint satisfaction rate, the equipment ramp-up constraint satisfaction rate, and the strategy convergence rounds, as shown in Table 4. Table 4. Index data for different methods

[0080] The implementation results show that the method of this invention can generate a stable collaborative scheduling and control strategy under conditions of fluctuating new energy output and coupled operation of multiple devices. Compared with the rule-based scheduling method, the single-agent PPO method, and the planar MAPPO method, the method of this invention performs better in terms of new energy absorption rate, power balance deviation, hydrogen flow balance deviation, hydrogen storage safety range retention rate, methanol and ammonia production tracking error, carbon emission constraint satisfaction rate, and equipment ramp-up constraint satisfaction rate. Specifically, the new energy absorption rate of the method of this invention reaches 97.1%, the average absolute deviation of power balance is reduced to 2.6MW, the average absolute deviation of hydrogen flow balance is reduced to 0.15t / h, the hydrogen storage safety range retention rate reaches 98.8%, the methanol and synthetic ammonia production tracking errors are reduced to 1.9% and 2.2%, respectively, and the carbon emission constraint satisfaction rate reaches 99.6%, proving the effectiveness of the hierarchical constraint multi-agent near-end strategy optimization model for collaborative scheduling and control of the electricity-hydrogen-methanol-ammonia coupled system.

[0081] To further illustrate the roles of the self-attention mechanism, the constraint violation dynamic adjustment mechanism, and the system-level coordinating agent in this invention, the following ablation methods are set up: removing the self-attention mechanism, removing the constraint violation dynamic adjustment mechanism, and removing the system-level coordinating agent. The indices for different mechanisms are shown in the table below: Table 5 Indicators for Different Mechanisms

[0082] The results show that the self-attention mechanism can improve the representation ability of multi-device interaction relationships, the constraint violation dynamic adjustment mechanism can improve the satisfaction rate of multiple physical constraints, and the system-level coordinated agent can improve the consistency of multi-device actions and the overall operational stability.

[0083] In the early stages of training, the average technical performance enhancement signal fluctuated significantly and the constraint violation rate was high because the actions of the various device-level execution agents had not yet formed a stable coordination. As the number of training rounds increased, the system-level coordinating agent gradually learned the operating reference vector suitable for the current output and storage status of new energy sources, and the device-level execution agents gradually formed a stable response to the operating reference vector. Around the 430th round, the average technical performance enhancement signal tended to stabilize, and the average constraint violation rate dropped to below 0.05.

[0084] Table 6. Training Results (Illustrated)

[0085] The training results in Table 6 demonstrate that the method of this invention, through hierarchical policy structure, self-attention fusion, and dynamic adjustment of constraint violation, can achieve stable convergence in a complex continuous action space and generate a cooperative scheduling control strategy that satisfies multiple physical constraints.

[0086] Example 4 is the fourth embodiment of the present invention. This embodiment provides a hierarchical constraint multi-agent scheduling system for an electrohydrogen alcohol ammonia system, including a physical operation modeling module, a hierarchical multi-agent control framework construction module, a self-attention interaction feature fusion module, a policy optimization and constraint evaluation network module, and a centralized training, distributed execution, and dual adjustment module.

[0087] The physical operation modeling module is used to establish mathematical descriptions of power balance constraints, material conservation constraints, equipment operating boundaries, dynamic storage constraints, and carbon emission limits for the electrical energy, hydrogen energy, alcohol-ammonia chemical and energy storage links in the electro-hydrogen-ethanol-ammonia coupling system. It provides a simulation environment and state transition basis that reflects the multi-energy flow coupling characteristics and various physical constraints for scheduling control.

[0088] The hierarchical multi-agent control framework construction module is used to decompose the high-dimensional continuous control problem into two layers of decision-making by having system-level coordinating agents generate cross-link collaborative operation reference vectors based on the global state, and multiple device-level execution agents generate device control actions based on the local state and reference vectors.

[0089] The self-attention interaction feature fusion module is used to generate fusion state features that integrate multi-device interaction information for each device-level execution agent, enabling the agent to adaptively perceive the state influence of other devices when outputting actions.

[0090] The policy optimization and constraint evaluation network module is used to perform hierarchical policy mapping using a policy network, estimate state-action value using a value evaluation network, predict the degree of constraint violation using a constraint violation evaluation network, and achieve stable learning of the collaborative scheduling policy under constraints by combining with the policy update constraint mechanism.

[0091] The centralized training, distributed execution, and dual adjustment module is used to centrally update the network parameters using global information during the training phase, and to adjust the influence of constraint violation on policy updates by dynamically adjusting the dual variables. During the execution phase, each agent independently outputs actions based only on local information, taking into account both information sufficiency and online response capability.

[0092] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0093] Example 5, the fifth embodiment of the present invention, differs from the previous four embodiments in that: If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0095] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0096] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

Claims

1. A hierarchical constraint multi-agent scheduling method for an electrohydrogen-ethanol-ammonia system, characterized in that: include, Construct a physical operation model of the coupled system, characterize the system power and constraints, and establish a hierarchical multi-agent control framework that includes a system-level coordinating agent and multiple device-level execution agents; Based on the physical operation model and the hierarchical multi-agent control framework, the reinforcement learning task specifications are determined, and a hierarchical constraint policy optimization model with embedded self-attention interaction feature fusion is constructed to generate fused state features and perform hierarchical policy mapping. At the same time, a policy update constraint mechanism and a value evaluation network are introduced. A centralized training and distributed execution framework is used to train the global network, and the network is updated according to the constraint violation adjustment strategy through a dual variable dynamic adjustment mechanism. The trained device-level execution agents are used to output control actions online.

2. The hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system as described in claim 1, characterized in that: The hierarchical multi-agent control framework includes a global state as the input of the system-level coordinating agent and an output system-level operational reference vector. The reference vector includes reference settings for the level of new energy consumption, hydrogen storage, chemical material storage, chemical plant load, and remaining carbon emission quotas. The process of defining the reinforcement learning task specifications includes defining the state space, action space, performance reward signal, and constraint violation evaluation function for reinforcement learning. The state space includes the global state and the local state of each device-level executive agent.

3. The hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system as described in claim 2, characterized in that: The generation of fused state features includes: self-attention interaction feature fusion for each device-level execution agent constructing query mapping, key mapping and value mapping, calculating the attention weight of the current agent to the state information of the remaining agents, and using the attention weight to perform a weighted summation of each value mapping to obtain the fused state features.

4. The hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system as described in claim 3, characterized in that: The hierarchical policy mapping includes taking the global state as the input to the system-level policy network and outputting a system-level operational reference vector, and taking the fused state features, local state, and the system-level operational reference vector of the corresponding agent as the input to the device-level policy network and outputting a device-level control action. The policy update constraint mechanism includes constructing a policy optimization objective by limiting the magnitude of the difference in probability distribution between the outputs of the old and new policies and combining the advantage function estimate.

5. The hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system as described in claim 4, characterized in that: The value assessment network includes a system-level value assessment network and a device-level value assessment network. The system-level value assessment network takes the global state and the system-level operational reference vector as input and outputs the system-level state-action value. The device-level value assessment network takes the fused state features, local states and control actions of the corresponding intelligent agent as input and outputs the device-level state-action value.

6. The hierarchical constraint multi-agent scheduling method for the electrohydrogen-ethanol-ammonia system as described in claim 5, characterized in that: The training of the global network using the centralized training and distributed execution framework includes the following steps: In the centralized training phase, the global state, all device-level control actions, and the true values ​​of constraint violations are collected; the system-level value assessment network is updated using system-level reward estimation; each device-level value assessment network is updated using device-level reward estimation; the constraint violation assessment network is updated using the true values ​​of constraint violations; and the parameters of the system-level policy network and each device-level policy network are updated based on the value estimates and constraint violation predictions output by each network. During the distributed execution phase, the system-level coordinating agent acquires the global state online and generates a system-level operational reference vector. Each device-level execution agent generates device-level control actions using only the local observed state, the system-level operational reference vector, and the interaction features of neighboring devices captured through a self-attention mechanism.

7. The hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol-ammonia system as described in claim 6, characterized in that: The dual variable dynamic adjustment mechanism includes updating the dual variables associated with each device-level execution agent during training based on the deviation between the average constraint violation degree of the sampling trajectory and the preset violation threshold. The online output control actions of the executing agent include: each device-level executing agent periodically acquiring local observation states, performing self-attention interaction feature fusion, updating the fused state features, and outputting control commands for the corresponding links through the device-level policy network combined with the system-level running reference vector; In this process, after the control command is applied to the physical operation model, the system state is updated according to the state transition relationship, forming a closed-loop rolling optimization scheduling control.

8. A hierarchical constrained multi-agent scheduling system for an electrohydrogen-ethanol-ammonia system, employing the hierarchical constrained multi-agent scheduling method for an electrohydrogen-ethanol-ammonia system as described in any one of claims 1 to 7, characterized in that, It includes a physical operation modeling module, a hierarchical multi-agent control framework construction module, a self-attention interaction feature fusion module, a policy optimization and constraint evaluation network module, and a centralized training, distributed execution, and dual adjustment module; The physical operation modeling module is used to establish mathematical descriptions of power balance constraints, material conservation constraints, equipment operating boundaries, dynamic storage constraints, and carbon emission limits for electrical energy, hydrogen energy, alcohol and ammonia chemical processes, and energy storage in the electro-hydrogen-ethanol-ammonia coupling system. This provides a simulation environment and state transition basis that reflects the multi-energy flow coupling characteristics and various physical constraints for scheduling and control. The hierarchical multi-agent control framework construction module is used to decompose the high-dimensional continuous control problem into two layers of decision-making by having a system-level coordinating agent generate a cross-link collaborative operation reference vector based on the global state, and multiple device-level execution agents generate device control actions based on the local state and the reference vector. The self-attention interaction feature fusion module is used to generate fusion state features that integrate multi-device interaction information for each device-level execution agent, so that the agent can adaptively perceive the state influence of other devices when outputting actions. The policy optimization and constraint evaluation network module is used to perform hierarchical policy mapping using a policy network, estimate state-action value using a value evaluation network, predict the degree of constraint violation using a constraint violation evaluation network, and achieve stable learning of the cooperative scheduling policy under constraints by combining with the policy update constraint mechanism. The centralized training, distributed execution, and dual adjustment module is used to centrally update the network parameters using global information during the training phase, and to adjust the influence of constraint violation on policy updates by dynamically adjusting the dual variables. During the execution phase, each agent independently outputs actions based only on local information, taking into account both information sufficiency and online response capability.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the hierarchical constraint multi-agent scheduling method for the electrohydrogen alcohol ammonia system as described in any one of claims 1 to 7.

Images