Natural gas pipeline network scheduling optimization method and apparatus based on multiple agents

Abstract

A natural gas pipeline network scheduling optimization method and apparatus based on multiple agents. The method comprises: constructing a physical model of a natural gas pipeline network, the physical model being used for representing nodes involved in the natural gas pipeline network, pipelines between the nodes, and constraint conditions; constructing a reward policy for a target agent in a reinforcement learning process by taking at least one of the following as an optimization objective: minimizing operating costs of the natural gas pipeline network, maximizing pipeline linepack, or minimizing gas demand fluctuations, wherein the target agent at least comprises a gas source, a compressor, and a gas storage reservoir in the nodes; on the basis of behavior rules for the target agent and the constraint conditions, performing industrial chain scheduling simulation on the physical model; and performing reinforcement learning and adjustment on behaviors of the target agent in the industrial chain scheduling simulation process on the basis of the reward policy, so as to obtain optimized scheduling information of the natural gas pipeline network. The method enables accurate simulation to obtain optimized scheduling information of a natural gas pipeline network, thereby promoting scientific scheduling and cost reduction and efficiency improvement of a natural gas supply chain.

Description

A method and apparatus for optimizing natural gas pipeline network scheduling based on multi-agent systems

[0001] Related applications

[0002] This application claims priority to Chinese Patent Application No. 202411909793.9, filed on December 24, 2024, and incorporates the entire contents of the aforementioned patent application as part of this application. Technical Field

[0003] This disclosure relates to the field of natural gas pipeline technology, and in particular to a method and apparatus for optimizing natural gas pipeline scheduling based on multi-agent systems. Background Technology

[0004] As energy demand becomes increasingly widespread, natural gas pipeline networks are being laid on a large scale. This brings with it high costs associated with the supporting equipment, dispatching, and maintenance of these pipelines.

[0005] Because the user groups of natural gas vary greatly in characteristics, their demand for natural gas differs while their requirements for a stable supply are very consistent. Therefore, imbalances between supply and demand are common. Furthermore, ensuring timely supply can lead to increased storage costs due to large-scale gas storage or over-exploitation. Consequently, promoting a stable natural gas supply presents significant technical challenges and substantial economic costs when scheduling the natural gas pipeline network supply chain.

[0006] In view of this, the present disclosure aims to provide a method and apparatus for optimizing natural gas pipeline network scheduling based on multi-agent systems. Summary of the Invention

[0007] To address the aforementioned problems in the prior art, the purpose of this disclosure is to provide a method and apparatus for optimizing natural gas pipeline network scheduling based on multi-agent systems, thereby solving the problems of difficult and costly supply chain scheduling of natural gas pipeline networks in the prior art.

[0008] To solve the above-mentioned technical problems, the specific technical solutions of this disclosure are as follows:

[0009] In a first aspect, embodiments of this disclosure provide a method for optimizing natural gas pipeline network scheduling based on multi-agent systems, including:

[0010] Construct a physical model of the natural gas pipeline network. The physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between the nodes, and the constraints.

[0011] Using at least one of the following as optimization objectives—lowest operating cost of natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—a reward strategy is constructed for the target agent during the reinforcement learning process. The target agent includes at least the gas source, compressor, and gas storage in the nodes.

[0012] Based on the behavioral rules and constraints of the target intelligent agent, a supply chain scheduling simulation is performed on the physical model in which the target intelligent agent is located.

[0013] Based on the reward strategy, the behavior of the target agent in the industrial chain scheduling simulation process is reinforced and adjusted to obtain the optimized scheduling information of the natural gas pipeline network.

[0014] Specifically, the constraints include at least the constraints on gas source and gas demand, natural gas transportation, pipeline pressure, compressor, and gas storage.

[0015] Furthermore, the constraints on gas supply and demand include at least the constraints that the gas supply and demand must satisfy.

[0016] The constraints for natural gas transportation include at least: a first constraint condition that must be satisfied between the transport flow rate and related parameters based on the physical characteristics of the pipeline during the natural gas transportation process, and a second constraint condition that must be satisfied between the inflow and outflow flow rates at the nodes.

[0017] Pipeline pressure constraints include at least the following: the actual distribution pressure at the node is higher than the user contract pressure, and the actual operating pressure of the corresponding pipe section at the node is lower than the maximum allowable operating pressure of the pipeline.

[0018] The compressor constraints include at least the following: the third constraint that the output power of the compressor in the node must satisfy with respect to the variable compression work and the mass flow rate of natural gas; the fourth constraint that the output power of the compressor must satisfy with respect to the rated power; the fifth constraint that the power of the compressor at different speeds varies with the flow rate of the transported gas; and the sixth constraint that the compressor is in a stable operating state.

[0019] The constraints of a gas storage facility include at least the following: the actual gas extraction flow rate of the gas storage facility is between the minimum and maximum gas extraction flow rates, and the actual change in the gas storage flow rate within an adjacent time period is lower than the maximum change in the gas storage flow rate.

[0020] Specifically, taking at least one of the following as optimization objectives—lowest operating cost of natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—as the objective, a reward strategy for the target agent during reinforcement learning is constructed, further including:

[0021] When the optimization objective is to minimize operating costs, the scheduling adjustments of gas sources, compressors, and gas storage facilities that move towards reducing operating costs during the industry chain scheduling simulation are identified as the first reward behavior; and the scheduling adjustments of gas sources, compressors, and gas storage facilities that move towards increasing operating costs during the industry chain scheduling simulation are identified as the first penalty behavior.

[0022] When the optimization objective is to maximize pipeline inventory, the scheduling adjustment behaviors of gas source, compressor, and gas storage in the process of supply chain scheduling simulation that change towards increasing pipeline inventory are identified as the second reward behavior; and the scheduling adjustment behaviors of gas source, compressor, and gas storage in the process of supply chain scheduling simulation that change towards decreasing pipeline inventory are identified as the second penalty behavior.

[0023] When the optimization objective is to minimize gas consumption fluctuations, the scheduling adjustments of gas sources, compressors, and gas storage facilities in the supply chain scheduling simulation that aim to reduce gas consumption fluctuations are identified as the third reward behavior; and the scheduling adjustments of gas sources, compressors, and gas storage facilities in the supply chain scheduling simulation that aim to increase gas consumption fluctuations are identified as the third penalty behavior.

[0024] Specifically, taking at least one of the following as optimization objectives—lowest operating cost of natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—as the objective, a reward strategy for the target agent during reinforcement learning is constructed, further including:

[0025] When any two of the following are taken as optimization objectives, namely the lowest operating cost, the largest pipeline inventory, and the smallest gas consumption fluctuation, one of the two is determined as the primary optimization objective and the other of the two is determined as the auxiliary optimization objective.

[0026] Solving for the main objective yields the optimal solution set.

[0027] Based on the set of optimal solutions, the optimal solution corresponding to the auxiliary optimization objective is obtained and used as the optimal solution that simultaneously satisfies the main optimization objective and the auxiliary optimization objective;

[0028] Among them, the scheduling adjustment behaviors of gas source, compressor and gas storage in the process of industrial chain scheduling simulation that move towards the direction of achieving the main optimization goal are identified as the fourth reward behavior; and the scheduling adjustment behaviors of gas source, compressor and gas storage in the process of industrial chain scheduling simulation that move away from the direction of achieving the main optimization goal are identified as the fourth penalty behavior.

[0029] The scheduling adjustments made by the gas source, compressor, and gas storage facility in the supply chain scheduling simulation that move towards achieving the optimization goal are defined as the fifth reward behavior; the scheduling adjustments made by the gas source, compressor, and gas storage facility in the supply chain scheduling simulation that deviate from achieving the optimization goal are defined as the fifth penalty behavior.

[0030] Furthermore, reinforcement learning and adjustment are performed on the target agent during the supply chain scheduling simulation based on a reward strategy, including:

[0031] In the process of supply chain scheduling simulation, the goal is to maximize the total reward value of all target agents or to exceed a set threshold, and reinforcement learning is carried out for each target agent.

[0032] Specifically, adjusting the behavior of the target intelligent agent includes:

[0033] Adjust the gas source pressure and / or gas supply volume; and / or

[0034] Adjust the compressor's operating parameters, which include at least one or a combination of discharge pressure, output power, and operating speed; and / or

[0035] Adjust the gas extraction volume of the gas storage facility.

[0036] Secondly, embodiments of this disclosure provide a multi-agent-based natural gas pipeline network scheduling optimization device, comprising:

[0037] The pipeline network model building module is used to build a physical model of the natural gas pipeline network. The physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between the nodes, and the constraints.

[0038] The reward strategy construction module is used to construct the reward strategy for the target agent in the reinforcement learning process, with the optimization objective being at least one of the following: the lowest operating cost of the natural gas pipeline network, the largest pipeline inventory, or the smallest fluctuation in gas consumption. The target agent includes at least the gas source, compressor, and gas storage in the node.

[0039] The scheduling simulation module is used to simulate supply chain scheduling for the physical model in which the target intelligent agent is located, based on the behavioral rules and constraints of the target intelligent agent.

[0040] The scheduling information determination module is used to perform reinforcement learning and adjustment on the behavior of the target intelligent agent in the industry chain scheduling simulation process based on the reward strategy, so as to obtain the optimized scheduling information of the natural gas pipeline network.

[0041] Thirdly, embodiments of this disclosure provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method provided by the above-described technical solution.

[0042] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method provided by the above-described technical solution.

[0043] Fifthly, embodiments of this disclosure provide a computer program product, including at least one instruction or at least one program segment, wherein the at least one instruction or at least one program segment is loaded and executed by a processor to implement the method provided by the above-described technical solution.

[0044] By adopting the above technical solution, the natural gas pipeline network scheduling optimization method and apparatus based on multi-agent provided in this disclosure considers the physical transportation characteristics of natural gas in the pipeline network during the construction of the physical model of the natural gas pipeline network, and sets the reward strategy for each target agent in the subsequent interactive simulation based on multi-agent with at least one of the following as the optimization objective: the lowest operating cost of the natural gas pipeline network, the largest pipeline inventory, or the smallest gas consumption fluctuation. This can accurately simulate the optimized natural gas pipeline network scheduling information, achieving at least one objective such as saving pipeline network operating costs, ensuring the natural gas inventory of the pipeline network, and reducing gas consumption fluctuation, thereby promoting the scientific scheduling and cost reduction and efficiency improvement of the natural gas supply chain.

[0045] To make the above and other objects, features and advantages of the embodiments of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 illustrates the steps of a multi-agent-based natural gas pipeline network scheduling optimization method provided in an embodiment of this disclosure.

[0048] Figure 2 is a schematic diagram of the natural gas pipeline network;

[0049] Figure 3 is a graph showing the relationship between the gas storage facility and the gas supply volume of the gas source after optimization according to the multi-agent-based natural gas pipeline network scheduling optimization method provided in the embodiments of this disclosure.

[0050] Figure 4 shows a schematic diagram of a multi-agent-based natural gas pipeline network scheduling optimization device provided in an embodiment of this disclosure;

[0051] Figure 5 shows a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0052] Explanation of symbols in the attached diagram: 41. Pipeline network model construction module; 42. Reward strategy construction module; 43. Scheduling simulation module; 44. Scheduling information determination module; 502. Computer equipment; 504. Processor; 506. Memory; 508. Drive mechanism; 510. Input / output module; 512. Input device; 514. Output device; 516. Presentation device; 518. Graphical user interface; 520. Network interface; 522. Communication link; 524. Communication bus. Detailed Implementation

[0053] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0054] It should be noted that the terms "first," "second," etc., used in this disclosure, the claims, and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0055] To address the aforementioned problems, this disclosure provides a method and apparatus for optimizing natural gas pipeline network scheduling based on multi-agent systems, which can solve the problems of difficult and costly supply chain scheduling of natural gas pipeline networks in the prior art. Figure 1 is a schematic diagram of the steps of a method for optimizing natural gas pipeline network scheduling based on multi-agent systems provided in this disclosure. This disclosure provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-creative labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is only one of many possible execution orders and does not represent the only execution order. In actual system or device products, the method can be executed in the order shown in the embodiments or figures or in parallel. Specifically, as shown in Figure 1, the method may include:

[0056] S110: Construct a physical model of the natural gas pipeline network. The physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between nodes, and the constraints.

[0057] Figure 2 shows a schematic diagram of the natural gas pipeline network. As shown in Figure 2, the natural gas pipeline network in this embodiment includes multiple nodes, such as gas sources, compressor stations (mainly including compressors, gas purification process systems, compressor drive devices, cooling systems, metering and pressure regulating systems, etc.), gas storage facilities, and gas demand terminals; it also includes pipelines connecting the various nodes.

[0058] In Figure 2, CX, CZ, WX, and DQ represent four gas demand terminals, whose natural gas demand changes over time. The natural gas demand of other demand terminals remains constant and does not change over time. Referring to Figure 2, in one specific embodiment, the natural gas pipeline network may include two gas sources, one gas storage facility, 31 gas demand terminals, and two compressor stations. The pipeline is 553 km long, with a diameter between 406.4 mm and 1016 mm, and an operating pressure between 6 MPa and 10 MPa. The term "between" includes endpoint values.

[0059] S120: Using at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory (the amount of natural gas stored in the pipeline), or minimum gas consumption fluctuation—construct a reward strategy for the target agent during reinforcement learning. The target agent includes at least the gas source, compressor, and gas storage unit among the nodes. Gas consumption fluctuation refers to the degree to which the natural gas demand or actual consumption at one or more gas demand ends in the natural gas pipeline network deviates drastically from its average level within a specified time period.

[0060] Multi-agent simulation technology is a method for studying complex systems by simulating the interactive behaviors of multiple autonomous, reactive, social, and proactive agents using computers. It offers advantages such as strong experimental capabilities, scalability, and visualization. Through modeling, simulation platforms, programming languages, and algorithms, multi-agent simulation technology can better understand group behavior and the overall properties of systems.

[0061] Modeling a multi-agent system is a systematic process involving the following key steps:

[0062] Requirements Analysis: In this initial stage, it is essential to first clarify the problem that the multi-agent system aims to solve and the goals it seeks to achieve.

[0063] Agent design: This involves designing the agents that constitute the system. This includes determining the type of agent, such as rule-based, model-based, or hybrid agents, and determining their number. Agent attributes include their capabilities, goals, knowledge base, and decision-making strategies. Furthermore, it is necessary to construct behavioral models of the agents, including how they perceive their environment, make decisions based on their perceptions, and execute actions.

[0064] Environmental modeling: The environment is where multi-agent systems interact, therefore it needs to be modeled. This includes defining the structure and state of the environment, simulating dynamic changes within the environment (such as time evolution, state transitions, and external events), and considering how agents interact with the environment and how the environment influences the agents' decisions and actions.

[0065] Collaboration and Coordination Mechanism Design: A core characteristic of multi-agent systems is the collaboration and coordination among agents. This requires designers to formulate rules and strategies for agent collaboration, ensuring that the coordination mechanism guarantees that agents can pursue their individual goals while also maintaining the overall goals of the system.

[0066] During the research and development process, it was discovered that if the physical characteristics of natural gas transported through pipelines are ignored in order to simplify parameters when applying multi-agent simulation technology, it will lead to significant simulation errors and results that do not match reality. Therefore, in order to improve the accuracy of simulation results, this embodiment constructs a physical model of the natural gas pipeline network, taking into account the physical transport characteristics of natural gas in the pipeline network during the construction process. In the subsequent interactive simulation based on multi-agents, the reward strategy for each target agent is set with at least one of the following optimization objectives: the lowest operating cost of the natural gas pipeline network, the maximum pipeline inventory, or the minimum gas consumption fluctuation. This helps to simulate and optimize more accurate natural gas pipeline network scheduling information, achieving at least one objective such as saving pipeline network operating costs, ensuring the natural gas inventory in the pipeline network, and reducing gas consumption fluctuation, thereby promoting the scientific scheduling and cost reduction and efficiency improvement of the natural gas supply chain.

[0067] Since the gas source, compressor station (mainly compressor) and gas storage are the nodes with decision-making capabilities and adjustable in the natural gas supply chain among the many nodes such as gas source, compressor station, gas storage, and gas demand side, the gas source, compressor and gas storage in the above nodes are identified as target intelligent agents, and simulation is performed based on these target intelligent agents.

[0068] S130: Based on the behavioral rules and constraints of the target intelligent agent, perform supply chain scheduling simulation on the physical model where the target intelligent agent is located.

[0069] S140: Based on the reward strategy, the behavior of the target agent in the industry chain scheduling simulation process is reinforced and adjusted to obtain the optimized scheduling information of the natural gas pipeline network.

[0070] The target intelligent agent follows the aforementioned constraints to perform reinforcement learning. During the reinforcement learning process, it makes behavioral decisions based on a reward strategy, thereby using the learned decision results as simulated and optimized natural gas pipeline network scheduling information. The supply chain scheduling simulation refers to driving the target intelligent agent (gas source, compressor, and gas storage) to interact according to its preset or learned behavioral rules, based on the physical model of the natural gas pipeline network, under an operating environment jointly defined by constraints on gas source and demand, natural gas transportation, pipeline pressure, compressors, and gas storage. This dynamically deduces the process of changes in state parameters such as pressure, flow rate, and inventory in the pipeline network system over time. This process provides a digital simulation platform for reinforcement learning algorithms to interact with the simulation environment, obtain state feedback, and evaluate the effects of actions. In the embodiments of this disclosure, the target intelligent agent is modeled as an intelligent agent with autonomous decision-making capabilities, where the gas source, compressor, and gas storage, whose operating parameters can be adjusted, are modeled as such.

[0071] This disclosure provides a multi-agent-based natural gas pipeline network scheduling optimization method. It applies multi-agent simulation technology to the scheduling optimization scenario of a natural gas pipeline network. Based on a constructed physical model of the natural gas pipeline network, it refines the target agents with decision-making capabilities within the network. Reward strategies for each target agent are set according to the optimization objectives. Then, based on the behavioral rules followed by each target agent, the constraints and reward strategies corresponding to interactions between agents and with the pipeline, the physical model is used to simulate supply chain scheduling, thereby obtaining simulated and optimized natural gas pipeline network scheduling information. By considering the physical transport characteristics of natural gas within the pipeline network during the construction of the physical model, and by setting reward strategies for each target agent based on at least one of the following optimization objectives—lowest operating cost, maximum pipeline inventory, or minimal gas consumption fluctuation—during the subsequent multi-agent interactive simulation, the optimized natural gas pipeline network scheduling information can be accurately simulated. This achieves at least one objective, such as saving pipeline operating costs, ensuring natural gas inventory, and reducing gas consumption fluctuations, thus promoting scientific scheduling and cost reduction / efficiency improvement in the natural gas supply chain.

[0072] Specifically, the constraints in step S110 include at least: gas source and gas demand constraints, natural gas transportation constraints, pipeline pressure constraints, compressor constraints, and gas storage constraints.

[0073] The constraints on gas supply and demand include at least the constraints that the gas supply volume and demand must meet. The gas supply volume can be adjusted within a preset range. The boundary condition for the gas supply is pressure control. For example, taking the schematic diagram of the natural gas pipeline network shown in Figure 2 as an example, the supply pressures of gas source 1 and gas source 2 are 8MPa and 10MPa, respectively; each demand end uses flow control mode. User demand depends on the user end. The natural gas demand of the four demand ends CX, CZ, WX, and DQ changes over time, while the natural gas demand of other demand ends is a fixed value and does not change over time.

[0074] The constraints for natural gas transportation include at least: a first constraint condition that must be satisfied between the transport flow rate and related parameters based on the physical characteristics of the pipeline during the natural gas transportation process, and a second constraint condition that must be satisfied between the inflow and outflow flow rates at the nodes.

[0075] Considering that the flow rate of a natural gas pipeline is related to the pipeline diameter, pipeline length, natural gas properties, average transport temperature, pressure drop characteristics, etc., the flow rate between two nodes in a natural gas pipeline is expressed as follows:

[0076] Among them, Q ij The flow between node i and node j, in Nm 3 / s(Nm 3 It is called standard cubic, which is the displacement under standard conditions, in Nm. 3 / s is standard cubic meters per second; C0 is an empirical constant, which can take a value of 0.00138; p i and p j D represents the pressure at node i and node j, respectively, in Pa (Pa). ij μ represents the inner diameter of the pipe between node i and node j, in meters (m). ij Z is the friction coefficient of the natural gas pipeline; Z is the compressibility factor of natural gas; Δ * T represents the specific gravity of natural gas. ij The average temperature of natural gas during transportation is expressed in Kelvin (K); L ij This refers to the length of the natural gas pipeline, expressed in meters (m).

[0077] Therefore, equation (1) can be used as an example of the first constraint condition.

[0078] Among them, the proportion of natural gas Δ * It can be calculated using the following formula: S g =ΣS i y i (3)

[0079] Among them, S a S represents the molar mass of air, expressed in g / mol (grams per mole); g S represents the molar mass of natural gas, expressed in g / mol. i y represents the molar mass of the i-th component in natural gas, expressed in g / mol; i This represents the proportion of the i-th component in natural gas.

[0080] In addition, it is necessary to ensure that the inflow and outflow of each node remain balanced. The flow balance constraint of the node (i.e., the second constraint condition) can be expressed as: m i,in -m i,out =0 (4)

[0081] Where, m i,in This represents the mass flow rate of natural gas flowing into the i-th node, expressed in kg / s (kilograms per second); m i,out This represents the mass flow rate of natural gas flowing out of the i-th node, in kg / s.

[0082] Pipeline pressure constraints must include at least the following: the actual distribution pressure at the node is higher than the user contract pressure, and the actual operating pressure of the corresponding pipe section at the node is lower than the maximum allowable operating pressure of the pipeline.

[0083] Wherein, the actual transmission pressure of the user node is higher than the user contract pressure, expressed by the following expression: p i,contact ≤p i,deliver (5)

[0084] Where, p i,contact p represents the user contract pressure of the i-th node, in MPa (megapascals); i,deliver This represents the actual distribution pressure of the i-th node, in MPa.

[0085] The actual operating pressure of the pipe section corresponding to the node is lower than the maximum allowable operating pressure of the pipeline, expressed by the following expression: pi,operation≤p i,desgin (6)

[0086] Where pi,operation represents the actual operating pressure of the pipe segment at the i-th node, in MPa; p i,desgin This represents the design pressure of the pipe segment containing the i-th node, which is the maximum allowable operating pressure of the pipeline, in MPa.

[0087] The compressor constraints include at least the following: the third constraint that the compressor output power and variable compression work and natural gas mass flow rate must satisfy in the node; the fourth constraint that the compressor output power and rated power must satisfy; the fifth constraint that the compressor power at different speeds varies with the transport gas flow rate; and the sixth constraint that the compressor is in a stable operating state (including speed and working pressure, limit power, etc.).

[0088] Taking centrifugal compressors as an example, centrifugal compressors can be driven by either electric or gas. Centrifugal compressors are powered by the natural gas flowing through them to overcome energy losses along the way.

[0089] The required relationship between the compressor's output power and the variable compression work and natural gas mass flow rate (i.e., an example of the third constraint) is shown in the following formula (7):

[0090] Among them, PW t This indicates the compressor's output power, measured in kW (kilowatts); H t G represents the variable compression work (also known as energy head) of the compressor at time t, with units of kJ / kg (kilojoules per kilogram); t The value of T represents the mass flow rate of natural gas through the compressor at time t, in kg / s; η represents the adiabatic efficiency of the compressor; Z represents the compressibility factor; R represents the universal gas constant, in kJ / (kg·K) (representing kilojoules per kilogram per Kelvin); T in,t p represents the compressor inlet temperature at time t, in Kelvin (K); k represents the adiabatic index; out,t p represents the compressor outlet pressure at time t, in MPa. in,t This represents the compressor inlet pressure at time t, in MPa.

[0091] The fourth constraint that the compressor's output power and rated power must satisfy is: the compressor's output power must not exceed its rated power, expressed as follows: PW t ≤PW e (9)

[0092] Among them, PW e This indicates the compressor's rated power.

[0093] The performance of a compressor under different operating conditions can be described by its discharge pressure, power, flow rate, and speed. By analyzing the pressure ratio (the ratio between the compressor's outlet pressure and inlet pressure) - flow rate and efficiency - flow rate relationship, the functional relationship between power and flow rate at different speeds can be obtained, which is the fifth constraint condition.

[0094] For example, the relationship between the compressor's variable compression power (energy head) and the compressor inlet flow rate and compressor speed can be expressed as follows:

[0095] Where, N R This indicates the compressor speed, expressed in r / min (revolutions per minute); Q r This indicates the compressor inlet flow rate, in cubic meters per second (m³). 3 / s (cubic meters per second), where c1, c2, and c3 are coefficients.

[0096] In some embodiments, the compressor's stable operating state is subject to constraints including speed, operating pressure, and maximum power. For example, in addition to the compressor speed being limited by the maximum and minimum speeds, the compressor's maximum permissible operating pressure and the drive motor's maximum power should also be controlled to ensure that the compressor's operating state is within the stable operating range, i.e., the sixth constraint condition is as follows: N R,min <N R <N R,max (11)

[0097] Where, N R,min This refers to the compressor's minimum speed, expressed in r / min; N R,max Q represents the compressor's maximum speed, expressed in r / min. r,low This indicates the minimum flow rate through the compressor, measured in cubic meters per second (m³). 3 / s;Q r,up This indicates the maximum flow rate through the compressor, measured in meters per second (m³). 3 / s;Q ac This indicates the actual flow rate through the compressor, in meters (m³). 3 / s.

[0098] Furthermore, the constraints of the gas storage facility include at least the following: the actual gas extraction flow rate of the gas storage facility is between the minimum gas extraction flow rate and the maximum gas extraction flow rate, and the actual change in the gas storage facility flow rate within adjacent time periods is lower than the maximum change in the gas storage facility flow rate.

[0099] For example, when extracting gas from a gas storage facility, constraints are imposed on the gas extraction flow rate and the rate of change of the gas extraction flow rate, which can be represented by the following expression: Q UGS,min ≤Q UGS,t ≤Q UGS,max (13) abs(Q UGS,t -Q UGS,t+1 )≤δ UGS (14)

[0100] Among them, QUGS,min Q UGS,max These correspond to the minimum and maximum gas extraction flow rates of the gas storage facility, respectively, in cubic meters per second (m³). 3 / h (cubic meters per hour); abs indicates absolute value operation; Q UGS,t Q UGS,t+1 These correspond to the gas extraction flow rates of the gas storage facility at time t and time t+1, respectively; Q UGS,t -Q UGS,t+1 This represents the change in gas storage flow rate within adjacent time intervals, in meters (m³). 3 / h; δ UGS This represents the maximum change in gas storage flow rate between two adjacent time intervals, expressed in meters per second (m³). 3 / h.

[0101] In this embodiment of the disclosure, adjusting the behavior of the target intelligent agent includes:

[0102] Adjust the gas source pressure and / or gas supply volume; and / or

[0103] Adjust the compressor's operating parameters, which include at least one or a combination of discharge pressure, output power, and operating speed; and / or

[0104] Adjust the gas extraction volume of the gas storage facility.

[0105] Therefore, the behavioral rules for the target intelligent agent can include:

[0106] The behavioral rules corresponding to the gas source may include one or a combination of several of the following: the range of change of gas source pressure and / or gas source supply volume, the timing of change, and the triggering conditions for change.

[0107] The behavior rules corresponding to the compressor may include one or a combination of several of the following: the adjustment range of operating parameters, the timing of adjustment, and the triggering conditions for adjustment.

[0108] Furthermore, the behavioral rules of a gas storage facility may include one or a combination of several of the following: the range of gas extraction volume variation, the timing of variation, and the triggering conditions for adjusting the gas extraction volume.

[0109] Furthermore, in a specific embodiment, step S120, which uses at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—to construct a reward strategy for the target agent during reinforcement learning, may include:

[0110] When the optimization objective is to minimize operating costs, the scheduling adjustments of gas sources, compressors, and gas storage facilities that move towards reducing operating costs during the industry chain scheduling simulation are identified as the first reward behavior; and the scheduling adjustments of gas sources, compressors, and gas storage facilities that move towards increasing operating costs during the industry chain scheduling simulation are identified as the first penalty behavior.

[0111] When the optimization objective is to maximize pipeline inventory, the scheduling adjustment behaviors of gas source, compressor, and gas storage in the process of supply chain scheduling simulation that change towards increasing pipeline inventory are identified as the second reward behavior; and the scheduling adjustment behaviors of gas source, compressor, and gas storage in the process of supply chain scheduling simulation that change towards decreasing pipeline inventory are identified as the second penalty behavior.

[0112] When the optimization objective is to minimize gas consumption fluctuations, the scheduling adjustments of gas sources, compressors, and gas storage facilities in the supply chain scheduling simulation that aim to reduce gas consumption fluctuations are identified as the third reward behavior; and the scheduling adjustments of gas sources, compressors, and gas storage facilities in the supply chain scheduling simulation that aim to increase gas consumption fluctuations are identified as the third penalty behavior.

[0113] Furthermore, during the supply chain scheduling simulation, the goal is to maximize the total reward value of all target agents or to exceed a set threshold, thereby enabling reinforcement learning for each target agent.

[0114] The convergence condition of reinforcement learning can be that the sum of rewards of all target agents is maximized; or, the sum of the above reward values ​​exceeds a set threshold, which can be regarded as the end of reinforcement learning, and the decision results of each target agent obtained by reinforcement learning are used as the simulated and optimized natural gas pipeline network scheduling information.

[0115] By taking a single objective as the optimization objective, defining the scheduling and adjustment behaviors that promote the achievement of the single objective as reward behaviors, and defining the scheduling and adjustment behaviors that hinder the achievement of the single objective as punishment behaviors, the target agent learns in the direction of promoting the achievement of the single objective during the reinforcement learning process, thus obtaining optimized decision results.

[0116] Furthermore, in another specific embodiment, step S120, which uses at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—to construct a reward strategy for the target agent during reinforcement learning, may include:

[0117] When any two of the following are taken as optimization objectives, namely the lowest operating cost, the largest pipeline inventory, and the smallest gas consumption fluctuation, one of the two is determined as the primary optimization objective and the other of the two is determined as the auxiliary optimization objective.

[0118] Solving for the main objective yields the optimal solution set.

[0119] Based on the set of optimal solutions, the optimal solution corresponding to the auxiliary optimization objective is obtained and used as the optimal solution that simultaneously satisfies the main optimization objective and the auxiliary optimization objective; that is, within the set of optimal solutions, the optimal solution corresponding to the auxiliary optimization objective is then obtained to obtain the optimal solution that satisfies the above main optimization objective and the auxiliary optimization objective.

[0120] Among them, the scheduling adjustment behaviors of gas source, compressor and gas storage in the process of industrial chain scheduling simulation that move towards the direction of achieving the main optimization goal are identified as the fourth reward behavior; and the scheduling adjustment behaviors of gas source, compressor and gas storage in the process of industrial chain scheduling simulation that move away from the direction of achieving the main optimization goal are identified as the fourth penalty behavior.

[0121] The scheduling adjustments made by the gas source, compressor, and gas storage facility in the supply chain scheduling simulation that move towards achieving the optimization goal are defined as the fifth reward behavior; the scheduling adjustments made by the gas source, compressor, and gas storage facility in the supply chain scheduling simulation that deviate from achieving the optimization goal are defined as the fifth penalty behavior.

[0122] Correspondingly, in the above-mentioned industrial chain scheduling simulation process, the goal is to maximize the total reward value of all target agents or to exceed a set threshold, and reinforcement learning is carried out for each target agent.

[0123] In the above embodiments, when two optimization objectives are combined as a comprehensive optimization objective, the logic for achieving the objective is implemented in a hierarchical manner. One objective is the primary optimization objective, and the other is the secondary optimization objective. Within the optimal solution set of the primary optimization objective, the optimal solution corresponding to the secondary optimization objective is then solved. The scheduling and adjustment behaviors that promote the achievement of each individual objective are identified as reward behaviors, and the scheduling and adjustment behaviors that hinder the achievement of each individual objective are identified as penalty behaviors. This allows the target agent to learn in the direction of promoting the achievement of the comprehensive optimization objective during the reinforcement learning process, thereby obtaining the optimized decision result.

[0124] In this embodiment, when downstream users increase their gas consumption while the gas supply remains constant, the natural gas stored in the pipeline will be consumed first. When the pipeline storage drops to a certain level, it will cause a decrease in downstream distribution pressure, making it impossible to meet the production needs of enterprise units. The natural gas pipeline network scheduling scheme provided in this embodiment is used for peak shaving. Its key lies in developing a supply chain operation plan to improve the reliability of the system's gas supply, taking into account fluctuations in user demand, changes in compressor operating status, and changes in gas supply capacity. The pipeline gas storage capacity directly determines the reliability of the system's gas supply.

[0125] The gas storage capacity of the pipeline is calculated as follows:

[0126] Where, p pj p represents the average pressure of the pipe section. s This indicates the starting pressure of the pipe section, in MPa; p e This indicates the pressure at the end of the pipe section, in MPa.

[0127] V represents the volume of natural gas in the pipeline section under standard conditions, in cubic meters (m³). 3 V pipe This indicates the spatial volume of a pipe section, in meters (m). 3 Z0 represents the compressibility factor of natural gas under standard conditions, with a value of 1; T0 represents the temperature under standard conditions, for example, 273.15K; p0 represents the pressure under standard conditions, in MPa; Z1 represents the actual compressibility factor of natural gas under the current operating conditions; T1 represents the average temperature of the pipeline section, in K.

[0128] In some embodiments, the daily operation and management of natural gas pipeline networks involves multiple aspects, including flow parameter monitoring, regular equipment maintenance and repair, and operation plan development. In peak shaving optimization, the energy consumption costs of compressor station operation and the gas extraction costs of gas storage facilities are key aspects of pipeline system operation and management.

[0129] For example, the calculation formulas for compressor energy consumption costs and gas storage facility gas extraction costs are as follows:

[0130] Where C1 represents the compressor energy consumption; P t,n The power of the nth compressor at time t is expressed in kW; N represents the total number of compressors; T represents the total operating time of the compressors; e c This indicates the unit electricity price for the compressor, expressed in yuan / (kW·h);

[0131] C2 represents the gas extraction fee for the gas storage facility; Q UGS,t This represents the gas extraction flow rate of the gas storage facility at time t, in m³ / s. 3 / h;e g This indicates the unit extraction cost of the gas storage facility, expressed in yuan / m³. 3 .

[0132] In this embodiment of the disclosure, the optimization objectives regarding pipeline storage are represented by equations (15) and (16), and the optimization objectives regarding operation and maintenance costs are represented by equations (17) and (18). This enables the gas source, compressor, and gas storage to perform reinforcement learning according to their respective reward strategies and with the objective of achieving at least one of the following: the lowest operating cost of the natural gas pipeline network, the highest pipeline storage, or the lowest gas consumption fluctuation. This results in natural gas pipeline network scheduling information that conforms to the constraints of the physical model and is optimized through simulation.

[0133] Figure 3 shows the relationship between the gas storage facility and the gas supply volume over time after optimization using the multi-agent-based natural gas pipeline network scheduling optimization method provided in this embodiment. Before optimization, the gas storage facility extracts gas at a fixed rate, and the compressor outlet pressure is fixed. As shown in Figure 3, after optimization, the gas extraction rate of the gas storage facility is dynamically adjusted over time, and the gas supply volume of the gas source is also dynamically adjusted over time, with the two working in tandem. In this embodiment, the maximum system pipeline storage and the minimum fluctuation in user gas consumption are used as the comprehensive optimization objectives, and multiple target agents are represented using the deep Q-matrix algorithm.

[0134] As shown in Figure 3, the peak periods for both gas supply and storage tank extraction occur between the 8th and 15th hours. Between the 18th and 24th hours, the gas supply decreases while storage tank extraction gradually increases. This is because the gas supply control mode is pressure control; when the gas supply flow rate decreases, the supply-demand imbalance further widens. The increased storage tank extraction flow rate then makes the storage tank the primary means of peak shaving.

[0135] Table 1 shows a comparison between the results of scheduling optimization using the multi-agent-based natural gas pipeline network scheduling optimization method provided in this disclosure and the results before scheduling optimization:

[0136] Table 1

[0137] Before optimization, the gas storage facility was 100×10⁴ m². 3 The gas sampling rate is fixed at a constant flow rate of / h, and the compressor outlet pressure is fixed at 12MPa. The comparison results in Table 1 show that the optimized pipeline system operating cost decreased from 2.1128 million yuan to 2.0063 million yuan, and the average pipeline inventory increased to 3362.8 × 10⁴ m³. 3 User pressure fluctuations decreased from 3.5% to 2.7%. Therefore, the multi-agent-based natural gas pipeline network scheduling optimization method provided in this disclosure not only reduces pipeline system operating costs but also increases the average pipeline inventory, effectively ensuring the safety and reliability of gas supply for users.

[0138] Based on the above-described multi-agent-based natural gas pipeline network scheduling optimization method, this disclosure also provides a multi-agent-based natural gas pipeline network scheduling optimization device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc., using the method of this disclosure, combined with necessary implementation hardware. Based on the same innovative concept, the devices in one or more embodiments provided by this disclosure are as follows. Since the implementation schemes and methods for solving the problem are similar, the implementation of specific devices in this disclosure can refer to the implementation of the aforementioned method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0139] As shown in Figure 4, the natural gas pipeline network scheduling optimization device based on multi-agent provided in this embodiment includes:

[0140] Pipeline model construction module 41 is used to construct a physical model of a natural gas pipeline network. The physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between nodes, and the constraints.

[0141] The reward strategy construction module 42 is used to construct the reward strategy of the target agent in the reinforcement learning process with at least one of the following as optimization objectives: the lowest operating cost of the natural gas pipeline network, the largest pipeline inventory, or the smallest fluctuation in gas consumption. The target agent includes at least the gas source, compressor, and gas storage in the node.

[0142] The scheduling simulation module 43 is used to perform supply chain scheduling simulation on the physical model where the target intelligent agent is located, based on the behavioral rules and constraints of the target intelligent agent.

[0143] The scheduling information determination module 44 is used to perform reinforcement learning and adjustment on the behavior of the target intelligent agent in the industry chain scheduling simulation process based on the reward strategy, so as to obtain the optimized scheduling information of the natural gas pipeline network.

[0144] The beneficial effects obtained by the apparatus provided in this disclosure are consistent with the beneficial effects obtained by the method described above, and will not be repeated here.

[0145] As shown in Figure 5, a computer device is provided according to an embodiment of this disclosure. The multi-agent-based natural gas pipeline network scheduling optimization device in this disclosure can be the computer device in this embodiment, executing the methods described above. The computer device 502 may include one or more processors 504, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 502 may also include any memory 506 for storing any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 506 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 502. In one case, when the processor 504 executes associated instructions stored in any memory or combination of memories, the computer device 502 can perform any operation of the associated instructions. Computer device 502 also includes one or more drive mechanisms 508 for interacting with any memory, such as hard disk drive mechanism, optical disk drive mechanism, etc.

[0146] Computer device 502 may also include an input / output module 510 (I / O) for receiving various inputs (via input device 512) and providing various outputs (via output device 514). A specific output mechanism may include a presentation device 516 and an associated graphical user interface (GUI) 518. In other embodiments, the input / output module 510 (I / O), input device 512, and output device 514 may be omitted, and the device may function solely as a computer device within a network. Computer device 502 may also include one or more network interfaces 520 for exchanging data with other devices via one or more communication links 522. One or more communication buses 524 couple the components described above together.

[0147] Communication link 522 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 522 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0148] Corresponding to the method shown in FIG1, this disclosure also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the above method.

[0149] This disclosure also provides a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to perform the method shown in FIG1.

[0150] This disclosure also provides a computer program product, including at least one instruction or at least one program segment, wherein the at least one instruction or at least one program segment is loaded and executed by a processor to implement the method shown in FIG1.

[0151] It should be understood that in the various embodiments of this disclosure, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0152] It should also be understood that, in the embodiments of this disclosure, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0153] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this disclosure can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0154] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0155] In the embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this disclosure, depending on actual needs.

[0157] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] This disclosure uses specific embodiments to illustrate the principles and implementation methods of this disclosure. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this disclosure should not be construed as a limitation of this disclosure.

Claims

1. A multi-agent based natural gas pipeline network dispatch optimization method, characterized in that, The method includes: Construct a physical model of the natural gas pipeline network, wherein the physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between the nodes, and the constraints. Using at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—a reward strategy is constructed for the target agent during the reinforcement learning process. The target agent includes at least the gas source, compressor, and gas storage tank in the node. Based on the behavioral rules and constraints of the target intelligent agent, a supply chain scheduling simulation is performed on the physical model in which the target intelligent agent is located. Based on the reward strategy, the behavior of the target agent in the supply chain scheduling simulation process is reinforced and adjusted to obtain the optimized scheduling information of the natural gas pipeline network.

2. The method of claim 1, wherein, The constraints include at least the constraints on gas source and gas demand, natural gas transportation, pipeline pressure, compressor, and gas storage.

3. The method according to claim 2, characterized in that, The constraints on gas source and gas demand include at least the constraints that the gas supply and gas demand must satisfy. The natural gas transportation constraints include at least: a first constraint condition that needs to be satisfied between the transport flow rate and related parameters based on the transportation physical characteristics of the pipeline during the natural gas transportation process, and a second constraint condition that needs to be satisfied between the inflow flow rate and outflow flow rate of the node. The pipeline pressure constraints include at least the following: the actual distribution pressure of the node is higher than the user contract pressure, and the actual operating pressure of the pipe section corresponding to the node is lower than the maximum allowable operating pressure of the pipeline. The compressor constraints include at least the following: a third constraint that the output power of the compressor in the node must satisfy in relation to the variable compression work and the mass flow rate of natural gas; a fourth constraint that the output power of the compressor must satisfy in relation to the rated power; a fifth constraint that the power of the compressor at different speeds varies with the flow rate of the transported gas; and a sixth constraint that the compressor is in a stable operating state. The constraints of the gas storage facility include at least the following: the actual gas extraction flow rate of the gas storage facility is between the minimum gas extraction flow rate and the maximum gas extraction flow rate, and the actual change value of the gas storage facility flow rate in adjacent time periods is lower than the maximum change value of the gas storage facility flow rate.

4. The method of claim 1, wherein, Using at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—a reward strategy is constructed for the target agent during reinforcement learning, further including: When the lowest operating cost is taken as the optimization objective, the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage facility that change towards reducing operating costs during the industry chain scheduling simulation are determined as the first reward behavior; and the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage facility that change towards increasing operating costs during the industry chain scheduling simulation are determined as the first penalty behavior. When the maximum pipeline inventory is taken as the optimization objective, the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage tank that change towards increasing pipeline inventory during the industry chain scheduling simulation are determined as the second reward behavior; and the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage tank that change towards decreasing pipeline inventory during the industry chain scheduling simulation are determined as the second penalty behavior. When the goal is to minimize gas consumption fluctuations, the scheduling adjustments of the gas source, the compressor, and the gas storage facility in the supply chain scheduling simulation that aim to reduce gas consumption fluctuations are identified as the third reward behavior; and the scheduling adjustments of the gas source, the compressor, and the gas storage facility in the supply chain scheduling simulation that aim to increase gas consumption fluctuations are identified as the third penalty behavior.

5. The method of claim 1, wherein, Using at least one of the following as optimization objectives—lowest operating cost of the natural gas pipeline network, maximum pipeline inventory, or minimum gas consumption fluctuation—a reward strategy is constructed for the target agent during reinforcement learning, further including: When any two of the following are taken as optimization objectives: the lowest operating cost, the largest pipeline inventory, and the smallest gas consumption fluctuation, one of the two is determined as the primary optimization objective and the other of the two is determined as the auxiliary optimization objective. The optimal solution set for the main optimization objective is obtained by solving the problem. Based on the set of optimal solutions, the optimal solution corresponding to the auxiliary optimization objective is obtained and used as the optimal solution that simultaneously satisfies the main optimization objective and the auxiliary optimization objective; Specifically, the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage tank that move towards the main optimization goal during the industry chain scheduling simulation are defined as the fourth reward behavior; and the scheduling adjustment behaviors of the gas source, the compressor, and the gas storage tank that deviate from the main optimization goal during the industry chain scheduling simulation are defined as the fourth penalty behavior. The scheduling adjustments made by the gas source, the compressor, and the gas storage facility in the supply chain scheduling simulation that move towards achieving the auxiliary optimization goal are defined as the fifth reward behavior; the scheduling adjustments made by the gas source, the compressor, and the gas storage facility in the supply chain scheduling simulation that deviate from achieving the auxiliary optimization goal are defined as the fifth penalty behavior.

6. The method according to claim 4 or 5, characterized in that, The target agent in the supply chain scheduling simulation process is reinforced and adjusted based on the reward strategy, further including: In the supply chain scheduling simulation process, the goal is to maximize the total reward value of all target agents or to exceed a set threshold, and reinforcement learning is performed on each target agent.

7. The method of claim 1, wherein, Adjusting the behavior of the target agent includes: Adjust the gas source pressure and / or gas supply volume; and / or The operating parameters of the compressor are adjusted, and the operating parameters include at least one or a combination of several of the following: discharge pressure, output power, and operating speed; and / or The gas extraction rate of the gas storage facility is adjusted.

8. A multi-agent based natural gas network dispatch optimization apparatus, characterized by, The device includes: The pipeline network model construction module is used to construct a physical model of the natural gas pipeline network. The physical model is used to represent the nodes involved in the natural gas pipeline network, the pipelines between the nodes, and the constraints. The reward strategy construction module is used to construct a reward strategy for the target agent during the reinforcement learning process, with at least one of the following as optimization objectives: the lowest operating cost of the natural gas pipeline network, the largest pipeline inventory, or the smallest gas consumption fluctuation. The target agent includes at least the gas source, compressor, and gas storage tank in the node. The scheduling simulation module is used to perform supply chain scheduling simulation on the physical model where the target intelligent agent is located, based on the behavioral rules of the target intelligent agent and the constraints. The scheduling information determination module is used to perform reinforcement learning and adjustment on the behavior of the target intelligent agent in the industry chain scheduling simulation process based on the reward strategy, so as to obtain the optimized scheduling information of the natural gas pipeline network.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

11. A computer program product, characterised in that, It includes at least one instruction or at least one program segment, said at least one instruction or said at least one program segment being loaded and executed by a processor to implement the method as claimed in any one of claims 1 to 7.

Images