An intelligent control method for a natural gas unmanned station and related components

Abstract

The application discloses a kind of natural gas unmanned station yard intelligent control method and related components, it is related to natural gas field, including: obtaining the historical control strategy of natural gas station, the fluid information and safety performance information corresponding to historical control strategy;According to historical control strategy, fluid information and safety performance information, first model is trained;According to historical control strategy and historical control strategy corresponding fluid information, second model is trained;Current cycle fluid information is input to second model, and the multiple control strategies output by second model are obtained;First model can determine the safety performance information of each strategy, second model can determine the corresponding multiple strategies according to fluid information, finally according to the safety performance information of multiple control strategies corresponding to first model, and from it select the lowest safety performance information as the control strategy of next cycle, can more accurately automatically control natural gas station.

Description

Technical Field

[0001] This invention relates to the field of natural gas, and in particular to an intelligent control method and related components for unmanned natural gas stations. Background Technology

[0002] As the natural gas market continues to develop and competition intensifies, the operating costs of natural gas pipelines are rising, placing increasing cost pressure on pipeline operators. Natural gas stations account for the primary portion of these operating costs. Currently, the vast majority of natural gas stations are manually operated, with staff manually adjusting pressure and flow control components. This process demands a high level of skill from operators, and the lengthy adjustment process indirectly increases pipeline operating costs. Furthermore, manual control of pressure and flow control components in natural gas stations is not sufficiently accurate and cannot achieve the desired control objectives. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent control method and related components for unmanned natural gas stations, which obtains safety performance information corresponding to multiple control strategies, and selects the one with the lowest safety performance information as the control strategy for the next cycle, thereby enabling more accurate automatic control of the natural gas station.

[0004] To solve the above-mentioned technical problems, the present invention provides an intelligent control method for unmanned natural gas stations, comprising:

[0005] The system acquires historical control strategies of the natural gas station, corresponding fluid information, and safety performance information. The control strategies involve adjusting the opening of various pressure and flow regulating elements within the natural gas station. The fluid information includes one or more combinations of the flow rate, pressure, and temperature of natural gas in the pipelines within the natural gas station. The safety performance information includes one or more combinations of the maximum flow rate, maximum pressure, and maximum pressure drop rate of natural gas in the pipelines within the natural gas station. The pressure and flow regulating elements include ball valves and regulating valves.

[0006] The first model is trained based on the historical control strategy, the fluid information, and the safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy.

[0007] The second model is trained based on the historical control strategies and the fluid information corresponding to the historical control strategies. The second model is used to output multiple control strategies based on the input fluid information.

[0008] The fluid information of the current cycle is input into the second model to obtain various control strategies output by the second model;

[0009] Based on the first model, the safety performance information corresponding to the various control strategies is obtained, and the one with the lowest safety performance information is selected as the control strategy for the next cycle.

[0010] On the other hand, the first model is a generative adversarial network, which includes generative networks and discriminative networks;

[0011] The first model is trained based on the historical control strategy, the fluid information, and the safety performance information, including:

[0012] The historical control strategy is input into the generative network to obtain the fluid information and safety performance information output by the generative network.

[0013] When the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range, the training of the first model is considered to have ended.

[0014] On the other hand, when the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range, the training of the first model is determined to be complete, including:

[0015] Construct the loss function of the generative network and the loss function of the discriminative network;

[0016] The relationship of the loss function of the generative network is as follows: The loss function of the discriminative network is L G Let L be the loss function of the generative network. D Let D(G(z) be the loss function of the discriminative network. i )) represents the probability that the discriminator determines the i-th data output by the generator to be the true value, G(z) i Let be the i-th data output by the generator, and D(x) be the i-th data. i ) represents the probability that the discriminator determines the i-th actual data point to be the true value, n is the number of data points input to the generative network, and z i Let x be the i-th input data of the generative network. i This is the i-th actual data, which includes real fluid information and safety performance information;

[0017] The training of the first model is considered complete when the function values ​​of the loss function of the integral network and the loss function of the discriminative network reach their minimum values.

[0018] On the other hand, the second model is based on the TD3 algorithm and includes an Actor network and a Critic network.

[0019] The second model is trained based on the first model, the historical control strategy, and the fluid information corresponding to the historical control strategy. The second model is used to output a control strategy based on the input fluid information, including:

[0020] Set the initial weight bias θ of the Actor network and the initial weight bias ω of the Critic network;

[0021] The fluid information corresponding to the historical control strategy is input into the Actor network to obtain the control strategy output by the Actor network;

[0022] The output control strategy is input into the first model to obtain the fluid information, safety performance information and reward value corresponding to the output control strategy of the first model. The reward value is negatively correlated with the safety performance information.

[0023] If the difference between the security performance information and the target security performance information is less than a preset difference value, then the training of the second model is determined to be completed.

[0024] On the other hand, after inputting the output control strategy into the first model to obtain the fluid information, safety performance information, and reward value corresponding to the output control strategy output by the first model, the method further includes:

[0025] If the difference between the security performance information and the target security performance information is not less than a preset difference, then the initial weight bias of the Actor network is updated from θ to θ′ according to θ′←τθ+(1-τ)θ′, and the initial weight bias of the Critic network is updated from ω to ω′ according to ω′←τω+(1-τ)ω′.

[0026] Where τ is the soft update coefficient.

[0027] On the other hand, after selecting the control strategy with the lowest safety performance information from the various control strategies as the control strategy for the next cycle, the following steps are taken:

[0028] The opening degree of the regulating valve and the conduction state of the ball valve are controlled according to the control strategy, wherein the conduction state includes being open or closed.

[0029] On the other hand, the control strategy with the lowest safety performance information is selected from the various control strategies as the control strategy for the next cycle, including:

[0030] Based on the reward value formula Determine the reward value for each of the control strategies, wherein the reward value is negatively correlated with the security performance information;

[0031] Select the control strategy with the highest reward value from among the multiple control strategies as the control strategy for the next cycle;

[0032] Where u is the control target. The actual control result after applying the control strategy is shown. The safety state evaluation function is the actual control result after applying the control strategy, where ε and M are both constants.

[0033] To address the aforementioned technical problems, the present invention also provides an intelligent control system for unmanned natural gas stations, comprising:

[0034] The historical data acquisition unit is used to acquire the historical control strategies of the natural gas station, the fluid information and safety performance information corresponding to the historical control strategies, the control strategies being the adjustment of the opening degree of various pressure and flow regulating elements in the natural gas station, the fluid information including one or more combinations of the flow rate, pressure and temperature of natural gas in the pipeline of the natural gas station, and the safety performance information including one or more combinations of the maximum flow rate, maximum pressure and maximum pressure drop rate of natural gas in the pipeline of the natural gas station, and the pressure and flow regulating elements including ball valves and regulating valves;

[0035] The first training unit is used to train the first model based on the historical control strategy, the fluid information and the safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy.

[0036] The second training unit is used to train the second model based on the historical control strategy and the fluid information corresponding to the historical control strategy. The second model is used to output multiple control strategies based on the input fluid information.

[0037] The control strategy determination unit is used to input the fluid information of the current cycle into the second model to obtain a variety of control strategies output by the second model;

[0038] The control strategy selection unit is used to obtain the safety performance information corresponding to the multiple control strategies based on the first model, and select the one with the lowest safety performance information as the control strategy for the next cycle.

[0039] To address the aforementioned technical problems, the present invention also provides an intelligent control device for unmanned natural gas stations, comprising:

[0040] Memory, used to store computer programs;

[0041] A processor is used to execute the computer program to implement the steps of the intelligent control method for unmanned natural gas stations as described above.

[0042] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the intelligent control method for the unmanned natural gas station described above.

[0043] This invention discloses an intelligent control method and related components for unmanned natural gas stations, relating to the natural gas field. The method includes: acquiring historical control strategies of the natural gas station, corresponding fluid information, and safety performance information; training a first model based on the historical control strategies, fluid information, and safety performance information; training a second model based on the historical control strategies and corresponding fluid information; inputting the current period's fluid information into the second model to obtain multiple control strategies output by the second model; the first model can determine the safety performance information of each strategy, and the second model can determine the corresponding multiple strategies based on the fluid information. Finally, the first model obtains the safety performance information corresponding to multiple control strategies, and selects the one with the lowest safety performance information as the control strategy for the next period, enabling more accurate automatic control of the natural gas station. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart of an intelligent control method for an unmanned natural gas station provided by the present invention;

[0046] Figure 2 A schematic diagram of the structure of a natural gas station provided by the present invention;

[0047] Figure 3 A schematic diagram illustrating the training of a first model provided by the present invention;

[0048] Figure 4 A schematic diagram illustrating the training of a second model provided by the present invention;

[0049] Figure 5 This invention provides a schematic diagram illustrating the training of the first model and the second model during actual use.

[0050] Figure 6A schematic diagram of the structure of an intelligent control system for an unmanned natural gas station provided by the present invention;

[0051] Figure 7 This is a schematic diagram of the structure of an intelligent control device for an unmanned natural gas station provided by the present invention. Detailed Implementation

[0052] The core of this invention is to provide an intelligent control method and related components for unmanned natural gas stations, which obtains safety performance information corresponding to multiple control strategies, and selects the one with the lowest safety performance information as the control strategy for the next cycle, thereby enabling more accurate automatic control of the natural gas station.

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Figure 1 The flowchart of an intelligent control method for an unmanned natural gas station provided by the present invention includes:

[0055] S11: Obtain the historical control strategy of the natural gas station, the fluid information and safety performance information corresponding to the historical control strategy. The control strategy is to adjust the opening degree of each pressure and flow regulating element in the natural gas station. The fluid information includes one or more combinations of the flow rate, pressure and temperature of natural gas in the pipeline of the natural gas station. The safety performance information includes one or more combinations of the maximum flow rate, maximum pressure and maximum pressure drop rate of natural gas in the pipeline of the natural gas station. The pressure and flow regulating elements include ball valves and regulating valves.

[0056] S12: Train the first model based on historical control strategies, fluid information and safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy.

[0057] S13: The second model is trained based on the historical control strategies and the corresponding fluid information. The second model is used to output multiple control strategies based on the input fluid information.

[0058] S14: Input the fluid information of the current cycle into the second model to obtain various control strategies output by the second model;

[0059] S15: Obtain the safety performance information corresponding to multiple control strategies based on the first model, and select the one with the lowest safety performance information as the control strategy for the next cycle.

[0060] As the natural gas market continues to develop and competition intensifies, the operating costs of natural gas pipelines are rising, placing increasing cost pressure on pipeline operators. Natural gas stations account for the primary portion of these operating costs. Currently, the vast majority of natural gas stations are manually operated, with staff manually adjusting pressure and flow control components. This process demands a high level of skill from operators, and the lengthy adjustment process indirectly increases pipeline operating costs.

[0061] To meet operational needs, unmanned natural gas stations require flow control to achieve target values ​​while minimizing pressure fluctuations. Currently, flow control at unmanned natural gas stations faces the following challenges: 1. Natural gas flow exhibits strong nonlinearity, making its state particularly unpredictable in complex pipeline topologies. 2. The control process must consider safety performance indicators to prevent alarms, such as maximum pressure alarms during regulation and flow meter overspeed alarms. 3. For the same control objective, multiple combinations of control strategies can be employed.

[0062] Because the nonlinear response of valve actuators and flow states results in a nonlinear relationship between the overall input and output of the station, the control of unmanned natural gas stations exhibits significant nonlinear characteristics. Therefore, this method includes: a first model, which generates fluid information and safety performance information within the station based on control strategies. The fluid information refers to relevant information about the flow, transmission, and storage processes of natural gas within the station, including: natural gas flow rate, pressure, and temperature. The safety performance information refers to the safety constraints that need to be considered to ensure the safe operation of unmanned station equipment during the flow state control process, including: maximum flow rate during flow changes, and maximum pressure drop rate during pressure changes. A second model generates multiple control strategies based on the station state and control objectives. Each control strategy produces corresponding control results and safety performance in the generative adversarial model. The control strategies refer to the combination schemes of process switching and regulating valve positions within the natural gas station. The process switching is the implementation of different station functions through different opening and closing combinations of ball valves in the station's pipeline network. The parallel valve group consists of multiple parallel regulating valves with different pipe diameters, which jointly regulate the flow state of the same pipeline.

[0063] This application employs automatic control to operate natural gas pipelines, specifically by automatically calculating the opening degrees of pressure and flow regulating elements within the natural gas station. A first model and a second model are constructed. First, the first model is trained using historical control strategies, corresponding fluid information, and safety performance information as training data. The trained first model can then obtain the corresponding fluid and safety performance information based on the input control strategy. Specifically, the historical control strategies, their corresponding fluid and safety performance information, and other relevant data are the historical operating data of the current natural gas station, which can be obtained from the controller's historical records; this application does not impose further limitations on this. The first model can then be used to determine which of several control strategies is optimal for the next cycle. The trained first model, along with the historical control strategies and their corresponding fluid information, is used as training data to train the second model. The second model's input data is the fluid information, and its output is multiple control strategies corresponding to that fluid information. In practical use, multiple control strategies are first determined based on the second model, and then the first model is used to determine which specific control strategy is selected for the next cycle.

[0064] The main function of an unmanned natural gas station is, under the premise of fully automated remote control of the natural gas station, to generate and push control schemes for pressure and flow regulating elements (such as gas regulating valves) within the station. It controls the gas flow state by changing the valve position, and the gas flow state includes temperature, pressure, and flow rate. The topology of the natural gas station to which this invention is applicable is as follows: Figure 2 As shown, the station switches the station process by opening and closing ball valves and regulates the gas flow state by regulating valves.

[0065] When determining the control strategy for the next cycle in the current cycle, the fluid state of the current cycle is input into the second model to obtain multiple control strategies output by the second model. These multiple control strategies are then input into the first model to obtain the safety performance information output by the first model. Since the safety performance information includes one or more combinations of the maximum flow rate, maximum pressure, and maximum pressure drop rate of natural gas, it is safer to select the one with the lowest safety performance information as the control strategy for the next cycle.

[0066] This invention discloses an intelligent control method and related components for unmanned natural gas stations, relating to the natural gas field. The method includes: acquiring historical control strategies of the natural gas station, corresponding fluid information, and safety performance information; training a first model based on the historical control strategies, fluid information, and safety performance information; training a second model based on the historical control strategies and corresponding fluid information; inputting the current period's fluid information into the second model to obtain multiple control strategies output by the second model; the first model can determine the safety performance information of each strategy, and the second model can determine the corresponding multiple strategies based on the fluid information. Finally, the first model obtains the safety performance information corresponding to multiple control strategies, and selects the one with the lowest safety performance information as the control strategy for the next period, enabling more accurate automatic control of the natural gas station.

[0067] Based on the above embodiments:

[0068] Figure 3 A schematic diagram illustrating the training of a first model provided by the present invention;

[0069] In some embodiments, the first model is a generative adversarial network, which includes generative networks and discriminative networks;

[0070] The first model is trained based on historical control strategies, fluid information, and safety performance information, including:

[0071] Historical control strategies are input into the generative network to obtain fluid information and safety performance information output by the generative network.

[0072] The training of the first model ends when the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range.

[0073] Generative Adversarial Networks (GANs) aim to learn the distribution patterns of complex nonlinear data and generate similar data distributions under these patterns. This invention utilizes this feature of GANs to enable them to learn the data characteristics of natural gas flow within a station, thereby acquiring fluid information and safety performance information. Essentially, a GAN uses two neural networks that compete with each other by designing different objective functions, achieving a balance in their training. More specifically, it comprises two networks: a generative (G) network and a discriminative (D) network. The purpose of the G network is to output fluid information after applying a control strategy and safety performance information during station state transitions, given a control strategy and station boundary conditions. The G network maps the control strategy obtained manually or through reinforcement learning to the data space G (G is a deep fully connected neural network), normalizes it to obtain interpretable result parameters, and the G network is the generator in the diagram. The D network is a discriminator used to estimate the probability that the input fluid information and safety performance information samples come from real data rather than data generated by the G network. If the samples come from real training data, the probability output by the D network is close to 1; otherwise, the probability output is close to 0. The samples referred to are instrument feedback data from flow meters, thermometers, pressure gauges, regulating valves, and ball valves within the natural gas field.

[0074] The goal of applying GANs to unmanned sites is to generate realistic and reliable state data through the G network. Therefore, the main objectives of training GANs are: 1) the D network can maximize the ability to correctly identify the data generated by the G network and the real data; 2) the G network can maximize the probability that the D network regards the data it generates as real data.

[0075] The specific GAN training process is as follows:

[0076] Dataset Acquisition: Data sample set 1 consists of measurement data from the actual SCADA system of the calibration station. To prevent the dataset from being too small, measurement data obtained from high-precision simulations were added as a supplement. Data sample set 2 consists of state data generated by the G network, and its dimensionality is consistent with data sample set 1.

[0077] For the G network, the inputs are a random inner boundary (i.e., the control scheme) and a random outer boundary (i.e., the simulation boundary, i.e., the station inlet and outlet pressures) after noise has been added. The output of the G network is the generated data of the control target (flow rate) and the user-defined safety parameters (maximum pressure differential flow rate). Sample data 2 is the valve opening strategy generated by the G network, and its combination with the G network inputs constitutes the complete calibration station data.

[0078] For the D network, the input is either sample 1 or sample 2. The purpose of the D network is to output the accuracy of the output vector of the G network. For calibration station control, under the same operating conditions and for the same target flow rate, there are multiple valve control strategies to make the actual flow rate approximate the target flow rate. Therefore, directly comparing the measured valve opening and the calculated valve opening cannot measure the control effect. To ensure the consistency of the final flow control target, the D network is further advanced, with the aim of outputting the accuracy of the flow rate at the calibration station under the control strategy generated by the G network.

[0079] In some embodiments, when the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range, the training of the first model is determined to be complete, including:

[0080] Constructing loss functions for generative networks and discriminative networks;

[0081] The relationship of the loss function of generative networks is as follows: The loss function of a discriminative network is L G Let L be the loss function of the generative network. D Let D(G(z) be the loss function of the discriminative network. i )) represents the probability that the discriminator determines the i-th data output by the generator to be the true value, G(z) i Let be the i-th data output by the generator, and D(x) be the i-th data. i ) represents the probability that the discriminator determines the i-th actual data point to be the true value, n is the number of data points input to the generative network, and z i Let x be the i-th input data of the generative network. i This is the i-th actual data, which includes real fluid information and safety performance information;

[0082] The training of the first model ends when the function values ​​of the loss functions of the adaptive network and the discriminative network both reach their minimum values.

[0083] The optimization objectives of GAN training based on the loss function are as follows: first, to enable the generator G to generate realistic fluid data; and second, to enable the discriminator D to better distinguish between real and generated samples. The parameters of the G and D networks are updated using the Adam algorithm to progressively approximate these two objectives.

[0084] The design objectives of G and D networks are as follows: the G network aims to obtain the optimal control strategy, while the D network aims to distinguish the difference between the flow rate under the G network's control strategy and the target flow rate. Generally, for control systems with significant nonlinear characteristics, such as those at natural gas stations, more complex networks are needed to adapt to this characteristic. As the network deepens, more information and richer features can be acquired. However, existing research shows that the optimization effect gradually decreases with increasing network depth (wide residual networks) because traditional neural networks input features through layer-by-layer convolution. As the number of hidden layers increases, gradients vanish or explode, leading to performance degradation. Residual networks reduce the training difficulty of deep networks by adding residual modules to traditional neural networks. Based on the characteristics of different networks, this invention selects appropriate models. For the G network, a deep fully connected neural network suitable for fitting and prediction problems is chosen; for the D network, a residual neural network is selected.

[0085] Figure 4 A schematic diagram illustrating the training of a second model provided by the present invention;

[0086] In some embodiments, the second model is established based on the TD3 algorithm, and the second model includes an Actor network and a Critic network.

[0087] The second model is trained based on historical control strategies and the corresponding fluid information. The second model outputs a control strategy based on the input fluid information, including:

[0088] Set the initial weight bias θ for the Actor network and the initial weight bias ω for the Critic network;

[0089] The fluid information corresponding to the historical control strategy is input into the Actor network to obtain the control strategy output by the Actor network;

[0090] The output control strategy is input into the first model to obtain the fluid information, safety performance information and reward value corresponding to the output control strategy of the first model. The reward value and safety performance information are negatively correlated.

[0091] If the difference between the safety performance information and the target safety performance information is less than a preset difference, then the training of the second model is considered complete.

[0092] After training, the reinforcement learning controller can generate valve position strategies based on the control objective and the station status. Its core design is based on a two-step control method, and its control flowchart is shown in the figure. At the beginning of the flow control process, the measurement data from the field instruments and the control objective are input as the environmental state to the reinforcement learning controller (TD3 algorithm) to generate an initial control strategy, which includes multiple control processes. The station flow is controlled within a specific range. When the calibration process stabilizes, it is determined whether the calibration requirements (relative error less than 5%) are met. If so, the control process ends; otherwise, a reward function is used to compensate for all control errors. The second control step compensates for the control error by inputting the control error and the current state of the controlled object. This process continues until the calibration requirements are met.

[0093] Due to safety risks and data acquisition time constraints, actual natural gas stations cannot meet the training logic requirements of reinforcement learning controllers. This invention addresses these needs by incorporating a memory bank that communicates with the GAN. The controller's training process includes memory bank establishment, sampling training, reward transfer, parameter updates, and memory bank updates.

[0094] The memory bank is built gradually during training. In the early stages of training, actions are randomly generated using a random number generation algorithm, and states are generated using the G function in the GAN. The reward is calculated based on the state, and a set of state, action, and reward is used as an entry in the memory bank for subsequent training sampling. When the entries in the memory bank meet the minimum training requirements, the random number generation algorithm is switched to the actual control algorithm for training.

[0095] Sampling training is the training mode adopted in the early stage of control algorithm training. In the early stage of training, the network parameters within the control algorithm are initialized. By extracting a large number of entries from the memory bank, the network is trained with the lowest reward as the target, and the control algorithm parameters are updated to complete the pre-training.

[0096] The reward transfer process considers the design of the reward function. The control strategy generated by the control algorithm serves as the input to the G network, generating the final station traffic and custom security state parameters under this strategy. The accuracy and security performance are evaluated through the reward function to derive the optimized objective function. The reward function is designed as follows: the first term considers the error between the target state and the actual state, and the second term considers the target's security state.

[0097] In some embodiments, after inputting the output control strategy into the first model to obtain the fluid information, safety performance information, and reward value corresponding to the output control strategy of the first model, the method further includes:

[0098] If the difference between the security performance information and the target security performance information is not less than the preset difference, then the initial weight bias of the Actor network is updated from θ to θ′ according to θ′←τθ+(1-τ)θ′, and the initial weight bias of the Critic network is updated from ω to ω′ according to ω′←τω+(1-τ)ω′.

[0099] Where τ is the soft update coefficient.

[0100] The TD3 algorithm is a deep deterministic policy gradient algorithm, where "deep" represents deep learning, meaning this algorithm combines deep learning and reinforcement learning. It is also a deterministic policy algorithm based on the Actor-Critic algorithm. Both the Actor and Critic in this algorithm use deep neural networks to establish approximate functions. The adaptability of the TD3 algorithm to unmanned station control is as follows:

[0101] 1. Use two Critic networks. Two networks are used to estimate the action value function, both evaluating the input actions. The smaller value is then used as the update target through the min() function, allowing for stable evaluation of multiple generated policies.

[0102] 2. Use soft updates. Instead of direct copying, update network parameters using the method θ′←τθ+(1-τ)θ′.

[0103] 3. Using policy noise. A greedy policy is used to incorporate exploration noise during the exploration process. This noise is equivalent to considering the noise generated by the instruments in the unmanned station itself, which makes the value function update smoother and increases the algorithm's stability.

[0104] 4. Use delayed learning. The Critic network updates more frequently than the Actor network, making the Actor training more stable.

[0105] The TD3 algorithm explores a small range around deterministic behavior during the learning phase by adding a noise function to the given unmanned station control policy. In addition, to improve the policy guidance level of the Critic network, the algorithm copies the parameters of both the Actor and Critic networks to calculate the expected value of the unmanned station state after applying the control policy. The network using the copied parameters is defined as the target network, and the copied parameters in the target network are updated in small increments each time. The Actor and Critic networks with the other set of parameters are used to generate the actual interaction behavior and calculate the corresponding policy gradient. This set of parameters is updated more quickly, once after each learning iteration. The purpose of this dual-parameter setup is to reduce non-convergence caused by guidance from similar data. The specific roles of the four networks are as follows:

[0106] Actor Network: It is mainly responsible for updating the policy network parameters θ, and also responsible for generating the current control policy action based on the current fluid state of the unmanned station or the state generated by the G network, and interacting with the G network to generate the station state state after applying the control policy, and generating the reward through the reward function;

[0107] The Critic network is responsible for updating the value network parameters w and calculating the current expected Q value. The target Q value qi = R + γQ'(S',A',w'), with a decay factor γ.

[0108] The Target Actor network is responsible for selecting the optimal next control policy, action, based on the next station state (sampled from the experience replay pool). The network parameters θ' are periodically copied from the corresponding parameters in the Actor network.

[0109] The Target Critic network is responsible for calculating the Q'(S',A',w') part of the target Q-value. The network parameter w' is periodically copied from w.

[0110] The specific implementation of the TD3 algorithm is as follows:

[0111] Input: The four networks mentioned above, with corresponding parameters θ, θ′, w, and w′, including a decay factor γ, a soft update coefficient τ, the number of samples for batch gradient descent n, and the update frequency of the target Q-network parameters K. Let the maximum number of iterations be x.

[0112] Output: The optimal Actor network has parameters θ, and the Critic network has parameters w.

[0113] (1) Randomly initialize θ and w, copy w' to w, and copy θ' to θ. Clear the experience replay set M;

[0114] (2) Iterate the following steps x times:

[0115] a) Take the initial state of the station as the first state sequence and obtain its feature vector φ;

[0116] b) The Actor currently generates the first control policy action based on the initial state of the station.

[0117] c) After obtaining the control policy action, input it into the G network to obtain the new site state state' and reward value after applying the control policy, and determine whether the current state is the final state;

[0118] d) Place all variables generated during training into memory bank M;

[0119] e) Use the new station state' as the current station state';

[0120] f) Sample n samples from memory M and calculate the Q value using the following formula:

[0121] Q j =R j (Final state);

[0122] (Non-final state);

[0123] In the formula, S is the current state, A is the current action, and R is the current reward.

[0124] g) Use the mean squared error loss function to update all parameters w of the current Critic network through backpropagation of the neural network's gradient:

[0125] h) Using the calculation formula, update all parameters θ of the current Actor network through gradient backpropagation of the neural network:

[0126] i) If x%K=1, that is, K steps after the last network update, then update the Critic target network and the Actor target network. The purpose of this step is to ensure training stability through delayed learning: θ′←τθ+(1-τ)θ′, ω′←τω+(1-τ)ω′;

[0127] j) If S' reaches the termination state, the current round of iteration is complete; otherwise, proceed to step b to continue iteration.

[0128] In some embodiments, after selecting the control strategy with the lowest safety performance information from multiple control strategies as the control strategy for the next cycle, the process includes:

[0129] The control strategy controls the opening degree of the regulating valve and the conduction state of the ball valve, which includes being open or closed.

[0130] The regulating valve is a controller that can adjust the opening degree, specifically selectable from 0-100%, determined according to the specific opening degree specified in the control strategy. The ball valve only has two control modes: on and off, determined by the specific on or off settings in the control strategy.

[0131] The main function of an unmanned natural gas station is, under the premise of fully automated remote control of the natural gas station, to generate and push control schemes for pressure and flow regulating elements (such as gas regulating valves) within the station. It controls the gas flow state by changing valve positions, and the gas flow state includes temperature, pressure, and flow rate. The topology of the natural gas station to which this invention is applicable is as follows: Figure 2As shown, the station switches the station process by opening and closing ball valves and regulates the gas flow state by regulating valves.

[0132] In some embodiments, selecting the control strategy with the lowest safety performance information from a variety of control strategies as the control strategy for the next cycle includes:

[0133] Based on the reward value formula The reward value for each control strategy is determined, and the reward value is negatively correlated with the safety performance information;

[0134] Select the control strategy with the highest reward value from multiple control strategies as the control strategy for the next cycle;

[0135] Where u is the control target. The actual control result after applying the control strategy. Let ε and M be the safety state evaluation function of the actual control results after applying the control strategy, where ε and M are both constants.

[0136] Where ε is a very small number to prevent the denominator from being 0, and M is a very large number.

[0137] Figure 5 This invention provides a schematic diagram illustrating the training of the first model and the second model during actual use.

[0138] When deployed in actual stations, this method can synchronously read the station's measured data in real time from the SCADA system. After outlier handling, null value imputation, and noise filtering, the measured data undergoes feature extraction to obtain the current state. When a control objective is issued, multiple control schemes are generated through reinforcement learning. These control schemes are independent of each other and can be verified by a generator to return the final reward value. The control scheme with the highest reward value is the optimal control scheme.

[0139] If the final control result is outside the allowable error range, it indicates that the operating conditions of the station have changed unpredictably. The GAN will perform synchronous training based on the measured data to make the TD3 control scheme verification process more accurate. At the same time, the control results will be collected into the TD3 memory for synchronous training.

[0140] Figure 5 The present invention provides a structural schematic diagram of an intelligent control system for an unmanned natural gas station, which includes:

[0141] The historical data acquisition unit 31 is used to acquire the historical control strategy of the natural gas station, the fluid information and safety performance information corresponding to the historical control strategy. The control strategy is to adjust the opening degree of each pressure and flow regulating element in the natural gas station. The fluid information includes one or more combinations of the flow rate, pressure and temperature of natural gas in the pipeline in the natural gas station. The safety performance information includes one or more combinations of the maximum flow rate, maximum pressure and maximum pressure drop rate of natural gas in the pipeline in the natural gas station. The pressure and flow regulating elements include ball valves and regulating valves.

[0142] The first training unit 32 is used to train the first model based on historical control strategies, fluid information and safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy.

[0143] The second training unit 33 is used to train the second model based on historical control strategies and the fluid information corresponding to the historical control strategies. The second model is used to output multiple control strategies based on the input fluid information.

[0144] The control strategy determination unit 34 is used to input the fluid information of the current cycle into the second model to obtain a variety of control strategies output by the second model;

[0145] The control strategy selection unit 35 is used to obtain safety performance information corresponding to multiple control strategies based on the first model, and select the one with the lowest safety performance information as the control strategy for the next cycle.

[0146] Based on the above embodiments:

[0147] The first model is a generative adversarial network, which includes generative networks and discriminative networks;

[0148] The first training unit 32 is specifically used to input historical control strategies into the generative network to obtain fluid information and safety performance information output by the generative network.

[0149] The first training termination determination unit is used to determine the end of the first model training when the difference between the fluid information and safety performance information output by the discriminative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range.

[0150] Loss function building unit, used to construct the loss function for generative networks and the loss function for discriminative networks;

[0151] The relationship of the loss function of generative networks is as follows: The loss function of a discriminative network is L G Let L be the loss function of the generative network. DLet D(G(z) be the loss function of the discriminative network. i )) represents the probability that the discriminator determines the generator output to be the true value, D(x) i ) represents the probability that the discriminator determines the actual data to be the true value, n is the number of model input parameters, i is the calculation of the i-th parameter, z is the input of the G network, and x i Provides accurate fluid information and safety performance information;

[0152] The first training termination determination unit is specifically used to determine the end of the first model training when the function values ​​of the loss functions of the synthetic network and the discriminative network reach their minimum values.

[0153] The second model is based on the TD3 algorithm and includes an Actor network and a Critic network.

[0154] The parameter setting unit is used to set the initial weight bias θ of the Actor network and the initial weight bias ω of the Critic network.

[0155] The second training unit 33 is specifically used to input the fluid information corresponding to the historical control strategy into the Actor network to obtain the control strategy output by the Actor network.

[0156] The output control strategy is input into the first model to obtain the fluid information, safety performance information and reward value corresponding to the output control strategy of the first model. The reward value and safety performance information are negatively correlated.

[0157] The second training termination determination unit is used to determine the end of the second model training if the difference between the safety performance information and the target safety performance information is less than a preset difference value.

[0158] The update unit is used to update the initial weight bias θ of the Actor network to θ′ according to θ′←τθ+(1-τ)θ′ and the initial weight bias ω of the Critic network to ω′ according to ω′←τω+(1-τ)ω′ if the difference between the security performance information and the target security performance information is not less than a preset difference.

[0159] Where τ is the soft update coefficient.

[0160] Pressure and flow regulating elements include ball valves and regulating valves;

[0161] The control unit is used to control the opening degree of the regulating valve and the conduction state of the ball valve according to the control strategy. The conduction state includes being on or off.

[0162] Control strategy selection unit 35 is specifically used to select strategies based on reward value formulas. The reward value for each control strategy is determined, and the reward value is negatively correlated with the safety performance information;

[0163] The control strategy selection unit 35 is specifically used to select the control strategy with the highest reward value from multiple control strategies as the control strategy for the next cycle.

[0164] Where u is the control target. The actual control result after applying the control strategy. Let ε and M be the safety state evaluation function of the actual control results after applying the control strategy, where ε and M are both constants.

[0165] The intelligent control system for the unmanned natural gas station provided in this application is described in the above embodiments and will not be repeated here.

[0166] Figure 6 This is a schematic diagram of the structure of an intelligent control device for an unmanned natural gas station provided by the present invention. The intelligent control device for the unmanned natural gas station includes:

[0167] Memory 41 is used to store computer programs;

[0168] The processor 42 is used to execute computer programs to implement the steps of the intelligent control method for the above-mentioned unmanned natural gas station.

[0169] The description of the intelligent control device for the unmanned natural gas station provided in this application is similar to the above embodiments and will not be repeated here.

[0170] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described intelligent control method for unmanned natural gas stations.

[0171] The description of the computer-readable storage medium provided in this application is given in the above embodiments and will not be repeated here.

[0172] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0174] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent control method for unmanned natural gas stations, characterized in that, include: The system acquires historical control strategies of the natural gas station, corresponding fluid information, and safety performance information. The control strategies involve adjusting the opening of various pressure and flow regulating elements within the natural gas station. The fluid information includes one or more combinations of the flow rate, pressure, and temperature of natural gas in the pipelines within the natural gas station. The safety performance information includes one or more combinations of the maximum flow rate, maximum pressure, and maximum pressure drop rate of natural gas in the pipelines within the natural gas station. The pressure and flow regulating elements include ball valves and regulating valves. The first model is trained based on the historical control strategy, the fluid information, and the safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy. The second model is trained based on the historical control strategies and the fluid information corresponding to the historical control strategies. The second model is used to output multiple control strategies based on the input fluid information. The fluid information of the current cycle is input into the second model to obtain various control strategies output by the second model; Based on the first model, the safety performance information corresponding to the various control strategies is obtained, and the one with the lowest safety performance information is selected as the control strategy for the next cycle. The first model is a generative adversarial network, which includes generative networks and discriminative networks; The first model is trained based on the historical control strategy, the fluid information, and the safety performance information, including: The historical control strategy is input into the generative network to obtain the fluid information and safety performance information output by the generative network. When the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range, the training of the first model is determined to be completed. The second model is based on the TD3 algorithm and includes an Actor network and a Critic network. The second model is trained based on the historical control strategies and the corresponding fluid information. The second model is used to output a control strategy based on the input fluid information, including: Set the initial weight bias θ of the Actor network, and the initial weight bias of the Critic network. ; The fluid information corresponding to the historical control strategy is input into the Actor network to obtain the control strategy output by the Actor network; The output control strategy is input into the first model to obtain the fluid information, safety performance information and reward value corresponding to the output control strategy of the first model. The reward value is negatively correlated with the safety performance information. If the difference between the security performance information and the target security performance information is less than a preset difference value, then the training of the second model is determined to be completed. Selecting the control strategy with the lowest safety performance information from among the various control strategies as the control strategy for the next cycle includes: Based on the reward value formula Determine the reward value for each of the control strategies, wherein the reward value is negatively correlated with the security performance information; Select the control strategy with the highest reward value from among the multiple control strategies as the control strategy for the next cycle; Where u is the control target. The actual control result after applying the control strategy is shown. The safety state evaluation function is the actual control result after applying the aforementioned control strategy. Both M and M are constants.

2. The intelligent control method for unmanned natural gas stations as described in claim 1, characterized in that, When the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range, the training of the first model is determined to be complete, including: Construct the loss function of the generative network and the loss function of the discriminative network; The relationship of the loss function of the generative network is as follows: The loss function of the discriminative network is L G Let L be the loss function of the generative network. D Let D(G(z) be the loss function of the discriminative network. i G(z) represents the probability that the discriminator determines the i-th data output by the generator to be the true value. i ) represents the i-th data output by the generator, D(x) i ) represents the probability that the discriminator determines the i-th actual data point to be the true value, n is the number of data points input to the generative network, and z i Let x be the i-th input data of the generative network. i This is the i-th actual data, which includes real fluid information and safety performance information; The training of the first model is considered complete when the function values ​​of the loss function of the integral network and the loss function of the discriminative network reach their minimum values.

3. The intelligent control method for unmanned natural gas stations as described in claim 1, characterized in that, After inputting the output control strategy into the first model to obtain the fluid information, safety performance information, and reward value corresponding to the output control strategy of the first model, the method further includes: If the difference between the security performance information and the target security performance information is not less than a preset difference value, then according to The initial weight bias of the Actor network is updated from θ to... ,according to The initial weight bias of the Critic network is changed by... Updated to ; Where τ is the soft update coefficient.

4. The intelligent control method for unmanned natural gas stations as described in claim 1, characterized in that, After selecting the control strategy with the lowest safety performance information from among the various control strategies as the control strategy for the next cycle, the following is included: The opening degree of the regulating valve and the conduction state of the ball valve are controlled according to the control strategy, wherein the conduction state includes being open or closed.

5. An intelligent control system for an unmanned natural gas station, characterized in that, include: The historical data acquisition unit is used to acquire the historical control strategies of the natural gas station, the fluid information and safety performance information corresponding to the historical control strategies, the control strategies being the adjustment of the opening degree of various pressure and flow regulating elements in the natural gas station, the fluid information including one or more combinations of the flow rate, pressure and temperature of natural gas in the pipeline of the natural gas station, and the safety performance information including one or more combinations of the maximum flow rate, maximum pressure and maximum pressure drop rate of natural gas in the pipeline of the natural gas station, and the pressure and flow regulating elements including ball valves and regulating valves; The first training unit is used to train the first model based on the historical control strategy, the fluid information and the safety performance information. The first model is used to output fluid information and safety performance information based on the input control strategy. The second training unit is used to train the second model based on the historical control strategy and the fluid information corresponding to the historical control strategy. The second model is used to output multiple control strategies based on the input fluid information. The control strategy determination unit is used to input the fluid information of the current cycle into the second model to obtain a variety of control strategies output by the second model; The control strategy selection unit is used to obtain the safety performance information corresponding to the multiple control strategies according to the first model, and select the one with the lowest safety performance information as the control strategy for the next cycle. The first model is a generative adversarial network, which includes generative networks and discriminative networks; The first training unit (32) is specifically used to input the historical control strategy into the generative network to obtain the fluid information and safety performance information output by the generative network. The first training end determination unit is used to determine the end of the first model training when the discriminative network determines that the difference between the fluid information and safety performance information output by the generative network and the fluid information and safety performance information corresponding to the historical control strategy is within a preset range. The second model is based on the TD3 algorithm and includes an Actor network and a Critic network. The parameter setting unit is used to set the initial weight bias θ of the Actor network and the initial weight bias of the Critic network. ; The second training unit (33) is specifically used to input the fluid information corresponding to the historical control strategy into the Actor network to obtain the control strategy output by the Actor network; The output control strategy is input into the first model to obtain the fluid information, safety performance information and reward value corresponding to the output control strategy of the first model. The reward value is negatively correlated with the safety performance information. The second training end determination unit is used to determine that the training of the second model has ended if the difference between the security performance information and the target security performance information is less than a preset difference value. The control strategy selection unit (35) is specifically used to select the reward value based on the formula. Determine the reward value for each of the control strategies, wherein the reward value is negatively correlated with the security performance information; The control strategy selection unit (35) is specifically used to select the control strategy with the highest reward value from a plurality of control strategies as the control strategy for the next period. Where u is the control target. The actual control result after applying the control strategy is shown. The safety state evaluation function is the actual control result after applying the aforementioned control strategy. Both M and M are constants.

6. An intelligent control device for an unmanned natural gas station, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the intelligent control method for unmanned natural gas stations as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent control method for unmanned natural gas stations as described in any one of claims 1 to 4.

Images