A gas meter valve control system and method

Abstract

The application discloses a gas meter valve control system and method, constructs an intelligent adjustment model according to user gas use history records and gas pipeline arrangement positions, detects user gas delivery pipelines, gas daily use amounts and gas uses by using the intelligent adjustment model, automatically adjusts gas valves according to user living habits, takes corresponding solutions according to different abnormal conditions, and notifies the user to exchange information with kitchen monitoring, user gas electromagnetic valves and community gas main valves in abnormal use, so as to timely find and eliminate the safety hazards of the gas pipeline. The gas meter valve control method is more scientific, safe and intelligent than traditional methods, greatly improves the management efficiency, reduces the misjudgment conditions and improves the user experience. The application further discloses a gas meter valve control system.

Description

Technical Field

[0001] This invention relates to the field of gas collection and detection, and more specifically, to a gas meter valve control system and method. Background Technology

[0002] Natural gas boasts advantages such as cleanliness, high calorific value, simple storage, and ease of use, making it an indispensable energy source for countless households. The safety of natural gas usage is paramount for users. However, in reality, gas companies typically organize gas safety inspectors to periodically manage the safety of all residential gas users within a community. These inspectors use specialized equipment to collect and test gas equipment for safety, record and track subsequent operations, resulting in low efficiency, a high risk of errors, and poor timeliness. Especially in the event of a gas leak, fires and explosions can easily occur. Existing technologies include simple leak detection with alarms and automatic gas shut-off functions. However, with the development of artificial intelligence and the Internet of Things, how to improve a scientific, controllable, and intelligent gas meter and valve control system based on pipeline layout, while simultaneously controlling detection costs, has become a pressing issue. Summary of the Invention

[0003] The purpose of this invention is to provide a gas meter valve control system and method to solve the above-mentioned technical problems.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A gas meter valve control method includes the following steps:

[0006] S1: Obtain the user's gas usage history and the location of the user's gas pipeline;

[0007] S2: Construct an intelligent adjustment model based on the acquired historical records and pipeline location information;

[0008] S3: Utilizes intelligent regulation models to monitor users' real-time gas delivery pipelines, daily usage, and gas application.

[0009] S4: Automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas application data.

[0010] Preferably, step S1 specifically includes:

[0011] S11: Obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records;

[0012] S12: Obtain the location of gas equipment in the gas pipeline network through the cloud platform, and obtain pressure, flow and temperature parameter data from pipeline sensors.

[0013] Preferably, step S2 specifically includes:

[0014] Based on the acquired historical records, main pipeline location parameters of the community, and user sub-pipeline location parameters, a multi-agent distributed reinforcement learning IDQN intelligent regulation model is constructed.

[0015] The training process of the multi-agent distributed reinforcement learning IDQN intelligent regulation model specifically includes:

[0016] S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community.

[0017] S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions am and ar according to the greedy exploration strategy;

[0018] S23: Execute am and ar in the network environment, read the reward r obtained after the execution action, and read the new state s' after the execution action;

[0019] S23: Will<s,am,r,s’> The experience replay pool stored in the main pipeline agent will<s,ar,r,s’> Experience replay pool stored in the main pipeline agent;

[0020] S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents.

[0021] S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks.

[0022] S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

[0023] Step S4 specifically includes:

[0024] S41: Adjust the gas meter valve opening based on the detection data;

[0025] S42: Detect abnormal situations. When an abnormal situation occurs during the day, the user can exchange information with the kitchen monitoring system via the Internet of Things by controlling the user's gas solenoid valve and the main gas valve of the community.

[0026] A gas meter valve control system includes the following modules:

[0027] The information acquisition module is used to acquire the user's gas usage history and the location of the user's gas pipeline.

[0028] The model building module is used to build an intelligent adjustment model based on the acquired historical records and pipeline location information;

[0029] The intelligent monitoring module is used to monitor the user's real-time gas delivery pipeline, daily usage, and gas application using an intelligent adjustment model;

[0030] The intelligent adjustment module is used to automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas application data.

[0031] Preferably, the information acquisition module specifically includes:

[0032] The statistical analysis module is used to obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records;

[0033] The parameter acquisition module is used to obtain the arrangement position of gas equipment in the gas pipeline network through the cloud platform and to obtain pressure, flow and temperature parameter data from pipeline sensors.

[0034] Preferably, the model building module specifically includes:

[0035] Based on the acquired historical records, main pipeline of the community, and user sub-pipeline location parameters, a multi-agent distributed reinforcement learning IDQN intelligent regulation model is constructed.

[0036] Preferably, the training process of the above-mentioned multi-agent distributed reinforcement learning IDQN intelligent regulation model specifically includes:

[0037] S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community.

[0038] S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions am and ar according to the greedy exploration strategy;

[0039] S23: Execute am and ar in the network environment, read the reward r obtained after the execution action, and read the new state s' after the execution action;

[0040] S23: Will<s,am,r,s’> The experience replay pool stored in the main pipeline agent will<s,ar,r,s’> Experience replay pool stored in the main pipeline agent;

[0041] S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents.

[0042] S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks.

[0043] S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

[0044] Preferably, the above-mentioned intelligent adjustment module specifically includes:

[0045] The meter valve adjustment module is used to adjust the opening of the gas meter valve based on monitoring data.

[0046] The anomaly handling module is used to detect abnormal situations. When an abnormality occurs during daily use, the user can exchange information with the kitchen monitoring system via the Internet of Things, controlling the user's gas solenoid valve and the community's main gas valve.

[0047] Compared with existing technologies, this invention provides a gas meter valve control system and method. It adopts a multi-agent distributed reinforcement learning IDQN intelligent regulation model to achieve rapid and intelligent detection of gas meter valves, which is more scientific, safe and intelligent, greatly improves management efficiency, reduces misjudgments and improves user experience. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the method flow disclosed in this invention;

[0049] Figure 2 This is a diagram of the cloud platform architecture for gas safety monitoring disclosed in this invention;

[0050] Figure 3 This is a schematic diagram of the IDQN joint control strategy model disclosed in this invention;

[0051] Figure 4 This is a diagram showing the training results of the IDQN joint control strategy model disclosed in this invention. Detailed Implementation

[0052] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0053] A gas meter valve control method includes the following steps:

[0054] S1: Obtain the user's gas usage history and the location of the user's gas pipeline;

[0055] S2: Construct an intelligent adjustment model based on the acquired historical records and pipeline location information;

[0056] S3: Utilizes intelligent regulation models to monitor users' real-time gas delivery pipelines, daily usage, and gas application.

[0057] S4: Automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas application data.

[0058] Preferably, step S1 specifically includes:

[0059] S11: Obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records;

[0060] S12: Obtain the location of gas equipment in the gas pipeline network through the cloud platform, and obtain pressure, flow and temperature parameter data from pipeline sensors.

[0061] Typically, indoor gas appliances consist of gas pipelines, gas valves, gas meters, gas water heaters, and gas stoves. Indoor gas leaks can be caused by two main factors: gas facility equipment and user habits.

[0062] The gas safety monitoring cloud platform hardware includes several components: alarms, a gas safety monitoring server, a cloud platform, AI gas safety valves, and thermal imaging dry-burn monitors. The gas safety monitoring server is connected to the cloud platform via the internet. Users can use WeChat and official accounts to check the equipment's operating status and alarm status, allowing for immediate dispatch of emergency response upon receiving alarm information on their mobile phones. See the appendix for details. Figure 2 .

[0063] The gas safety monitoring cloud platform includes a data layer, which focuses on the pressure information and flammable gas content information of gas pipelines. The platform's data covers geographic information and spatial data of gas pipeline equipment, including service data such as gas pipeline pressure and flammable gas content information collected from the user end, as well as monitoring data. Based on the above data, the cloud platform statistically analyzes the user's gas usage habits.

[0064] In addition, the data layer also acquires pressure, flow, and temperature parameters of the main pipeline and user branch pipelines in the community through pipeline sensors.

[0065] Preferably, step S2 specifically includes:

[0066] Based on the acquired historical records, main pipeline of the community, and user sub-pipeline location parameters, a multi-agent distributed reinforcement learning IDQN intelligent adjustment model is constructed.

[0067] Reinforcement learning is a method in machine learning that refers to an individual learning to perform a task through repeated interaction with a dynamic environment. Through continuous trial and error, the individual's ultimate goal is to obtain an optimal strategy to achieve the goal. Reinforcement learning uses the concept of constraint from psychology; after an individual responds to a stimulus, its behavior is controlled by giving it rewards or punishments. In reinforcement learning, the individual is called the agent, the received stimulus is called the state, and the entity providing the stimulus and reward is called the environment. The interaction between the agent and the environment can be described mathematically as a Markov Decision Process (MDP), which consists of the following elements:

[0068] 1. S is the set of state spaces.

[0069] 2.A is the set of action spaces.

[0070] 3. Pa is the transition probability function, representing the probability of transitioning to state s′ by taking action a in state s.

[0071] 4. Ra is the reward value obtained by taking action a in state s to transition to state s′.

[0072] Q-Learning is an algorithm that uses Temporal Difference (TD) learning for optimization. Temporal Difference learning combines Monte Carlo methods and dynamic programming, and this method does not need to wait until the end of each generation to update. Its update equation is as follows (1):

[0073] Q t+1 (s,a)=(s,a)+[r t+1 +[maxQ t (s′,a′)]′-Q t (s,a)] (1)

[0074] a∈A

[0075] Where s is the current state

[0076] 'a' represents the current action.

[0077] s′ represents the state at the next time step.

[0078] a′ represents the action in the next time step.

[0079] α is the learning rate, which determines the number of steps in each update. It can be set according to previous literature or adjusted based on training results.

[0080] The Q-function is updated by calculating the difference between the estimated Q-value and the target Q-value. However, since the actual Q-function value is unknown, the immediate reward rt+1 obtained from the agent's interaction with the environment and the expected future reward are used as substitutes. Through continuous updates, the error between the prediction and the target Q-value gradually converges, thus learning the optimal policy. The calculation steps of Q-Learning are as follows:

[0081] 1: Initialize Q(s, a) with random values.

[0082] 2: In each training round, at each time step, select the action based on the environmental state and use ∈-greedy.

[0083] Strategy selection action.

[0084] 3: Collect rewards and update Q(s, a) using the updated equation.

[0085] 4: Repeat steps 2 and 3 until all training rounds are completed.

[0086] In step 2, a greedy strategy (∈-greedy) is used to aid learning. This is because if the agent consistently executes the action with the highest current Q-value during the learning process, it might ignore other untried actions that could yield greater rewards. This situation is known as the "exploration-exploitation dilemma." Exploration refers to the agent randomly trying different actions in hopes of obtaining a larger reward, while exploitation refers to the agent choosing the action with the highest current Q-value. Q-Learning addresses this dilemma through a greedy strategy. A greedy strategy uses a random probability epsilon (∈) to determine whether the agent will execute the optimal action according to the strategy. It selects the optimal action with a probability of 1-∈, and takes random action with a probability of ∈. In practice, ∈ usually decreases as the number of training rounds increases, allowing the agent to try more different actions in the early stages of training and to execute the action with the highest Q-value more frequently in the later stages.

[0087] In a deep Q-network, the state is input from the input layer. After calculation using the weight parameters of neurons in the hidden layers, the expected Q-values ​​for each action are output. The agent then selects the action with the highest Q-value to execute. After the action is executed, the loss value is calculated based on the target value and the predicted value, and the network weights are updated using gradient descent to minimize the loss value. The loss function used is as follows:

[0088] Los = (y i -(s,a;θ)) 2 (2)

[0089] y i =r+γmax a ′(s′,a′;θ)(3)

[0090] Where s is the current state

[0091] s' represents the next state

[0092] 'a' represents the current action.

[0093] a' indicates the next action

[0094] r represents the reward.

[0095] γ is the discount factor

[0096] θ is the neural network parameter

[0097] In practical applications, neural networks struggle to update to the optimal policy using a single set of loss values ​​between the target and predicted values. Therefore, when interacting with the environment, the agent records the current state s, the action a, the reward r obtained after the action, and the updated action s'.<s,a,r,s’> Each transfer experience is recorded and stored in the replay buffer. When the network parameters need to be updated, some transfer experiences are drawn from the replay buffer as samples for training. This method is called experience replay. The random sampling of transfer experiences in experience replay can effectively reduce the correlation between samples to avoid overfitting. Users can also customize the storage limit of the experience replay buffer, deleting older transfer experiences and storing new ones as training progresses.

[0098] From equations (2) and (3) above, it can be seen that the weight parameter θ is used to estimate the target value and the predicted value. This situation may cause the target value and the predicted value to diverge and make training difficult to converge. In order to solve this problem, the idea of ​​a target network is proposed. First, the network is divided into an actual network and a target network. During the training process, the parameters of the target network are frozen. The target network generates the target value and the actual network generates the predicted value to calculate the loss value. The actual network is updated according to the loss value. After several time steps, the weight parameters of the actual network are copied to the target network. The loss function of this method is adjusted as shown in equation (4).

[0099] Los = (y i -(s,a;θ)) 2 (4)

[0100] y i =r+γmaxa′(s′,a′;θ′) (5)

[0101] Where θ is the actual network parameter

[0102] θ′ is the target network parameter

[0103] The Deep Q-Network (DQN) with added experience replay and target network is shown in the following flowchart:

[0104] 1: Initialize θ and θ′ with random values.

[0105] 2: Input the current state s into the actual network and output the predicted Q value of each action.

[0106] 3: Perform action a with a greedy strategy and obtain reward r and the next state s'.

[0107] 4: Transfer experience<s,a,r,s’> Store in the experience replay pool.

[0108] 5: Sample and transfer experience from the experience replay pool, and update the actual network parameters θ through the loss function.

[0109] 6: After every n steps, copy the actual network parameters θ to the target network parameters θ′.

[0110] 7: Repeat the above steps until the set total number of rounds ends.

[0111] A multi-agent system refers to a group of autonomous individuals sharing the same environment who can interact with each other. They perceive environmental changes and make decisions. Many real-world problems, such as robotic tasks requiring teamwork and traffic control, can be solved using multi-agent systems. Because agents in a multi-agent learning environment can interact with the environment simultaneously, and the actions of one agent can affect others, multi-agent learning is much more complex than single-agent learning. If we assume that all agents can observe the true state of the entire environment rather than a local state, we can build a single-agent model to represent the multi-agent system. However, this approach causes the action space of the single-agent model to grow exponentially with the number of agents, making it impractical for practical training. Therefore, a distributed learning strategy is chosen for training.

[0112] This invention establishes a multi-agent distributed reinforcement learning IDQN intelligent regulation model, including a cell main pipeline model and a user sub-pipeline model. The cell main pipeline model and the user sub-pipeline model are two independent agents, each with its own independent strategy, and both operate simultaneously in the same simulated pipeline network environment. See the appendix for details. Figure 3The simulated pipeline collects information from data collection points within the pipeline and transmits the necessary training information to the model via the COM interface. Upon receiving the input state, the model combines the fingerprint of another agent as a decision-making basis, selects an action as the output, and executes it again through the COM interface into the pipeline. COM stands for Component Object Model, allowing users to control the pipeline externally through this interface.

[0113] This invention uses pipeline simulation software as a training platform, employs Python programming language to build a multi-agent reinforcement learning model, and interacts with the pipeline environment via a COM interface. The main pipeline agents and user sub-pipeline agents are trained separately, but simultaneously use the same environment for decision-making. The three most important elements in reinforcement learning are the settings of state, action, and reward, as well as the architecture of the neural network-like system. Since the state, action, reward, and network architecture settings used in the main pipeline and sub-pipeline models established in this invention are consistent, they are described here simultaneously.

[0114] State: The DQN learning method used in this invention uses a neural network as an approximation function of the Q function, which can effectively relax the restrictions on the dimension of the model input. Therefore, pressure, flow rate, and temperature are chosen as environmental states. This invention uses a fully connected layer as the Q function of DQN, so the state information needs to be converted into a one-dimensional input model. After obtaining the state information of the neural network, since pressure, flow rate, and temperature indicators have different scales, they are normalized before being input into the neural network to adjust the values ​​to between -1 and 1.

[0115] Actions: Instrumentation management achieves control objectives by adjusting the opening of valves. Most studies using reinforcement learning control methods choose discrete instrumentation rates as the instrumentation policy. The purpose of this invention is to improve the control logic of existing instrumentation policies, considering feasibility and the flexibility of adjusting the instrumentation rate. The action settings of the instrumentation model in this study are consistent with the current selection of discrete instrumentation rates, and a decision is made every 5 minutes. However, since the cycle time of each instrumentation rate is different, and the 5-minute setting may prevent some actions from completing a full cycle, the simulation software defaults to switching to a new instrumentation rate after the current phase is completed.

[0116] Rewards: Rewards are part of the objective value in reinforcement learning, so the setting of rewards has a great impact on the learning of agents. Different reward settings will result in different strategies of agents after training. The research purpose of this invention is to solve the intelligent safety management of gas pipeline networks. Therefore, this invention readjusts the reward definition. Considering the supply side, if the gas consumption in the pipeline within a unit time period is kept within a safe range compared with the historical gas consumption of the community or user, it can be considered intelligent and safe. Therefore, the flow rate in the pipeline within a unit time period is selected as the model reward setting, so that the model objective is to keep the flow rate at that location within a predetermined threshold compared with the historical gas consumption.

[0117] Neural Network Architecture: The state is first normalized to between -1 and 1 before being sent to the input layer. Then, it passes through three fully connected layers to form the output layer. Because this study employs a competitive network architecture and distributed reinforcement learning, the output layer is divided into a state-value stream and an action-dominance stream. Combining the results from both sides, the output layer shows the state-action value distribution for each action. In this distributed reinforcement learning approach, the number of atoms is set to 9, so the state-value stream is divided into 9 approximate state-value distributions, and the action-dominance stream also divides each action into 9 approximate action-dominance distributions. Therefore, the final output will be 72-dimensional. Distributed reinforcement learning changes the neural network output to the distribution of Q-values ​​for each action. First, the maximum value vmax and minimum value vmin of the Q-value distribution are defined, along with the number of atoms to divide the distribution. For example, vmax is 10, vmin is -10, and the number of atoms is 7. The loss function is then calculated by comparing the cross-entropy between the distribution of the actual network output and the distribution of the target network output. Overall, distributed reinforcement learning changes the model output to the distribution of Q-values ​​for each action, allowing the model to obtain more useful information during training, thereby making the training process more stable.

[0118] The hardware and software configurations used in this invention are shown in Tables 1-2 below.

[0119] Table 1. Hyperparameters of Reinforcement Learning Models

[0120]

[0121]

[0122] Table 2. Equipment and Software Configuration

[0123]

[0124] The training process of the multi-agent distributed reinforcement learning IDQN intelligent regulation model includes:

[0125] S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community.

[0126] S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions am and ar according to the greedy exploration strategy;

[0127] S23: Execute am and ar in the network environment, read the reward r obtained after the execution action, and read the new state s' after the execution action;

[0128] S23: Will<s,am,r,s’> The experience replay pool stored in the main pipeline agent will<s,ar,r,s’> Experience replay pool stored in the main pipeline agent;

[0129] S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents.

[0130] S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks.

[0131] S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

[0132] To facilitate understanding, the training process of the model is described in detail using pseudocode-like methods. Pseudocode is a way of describing algorithms; it is not an actual programming language, but rather uses the syntax of various programming languages ​​or natural language to help us better express the function of the algorithm.

[0133] The overall training process and algorithm flow of the multi-agent deep reinforcement learning model are described in detail below:

[0134] 1:global_steps=1.

[0135] 2: for episode in total_episode(450 episodes) do

[0136] 3: Reset the random seed and delete old simulation records

[0137] 4: Start warming up

[0138] 5: Start the main pipeline and branch pipeline control signs

[0139] 6: while current time < total simulation time (14400 seconds) do

[0140] 7: Read the current environment state s

[0141] 8: Input 's' into the main pipeline and sub-pipeline agents, and extract actions 'am' and 'ar' according to a greedy exploration strategy.

[0142] 9: Execute am and ar in the pipeline environment.

[0143] 10: Read the reward r obtained after executing the action.

[0144] 11: Read the new state s' after the action is executed

[0145] 12: will<s,am,r,s’> Experience replay pool stored in the main pipeline agent

[0146] 13: will<s,ar,r,s’> Experience replay pool stored in the pipeline agent

[0147] 14:if global_steps>pretrain_steps and global_steps%train_steps=0do

[0148] 15: Training main pipeline and branch pipeline agents

[0149] 16:if global_steps>pretrain_steps and global_steps%update_steps=0do

[0150] 17: Copy the network parameters of the main pipeline and branch pipelines to their respective target networks.

[0151] 18:global_steps+=1

[0152] 19:end for

[0153] The above is the overall training process of the model of this invention. The following describes the process of updating the network parameters for training one agent at a time, also in the form of virtual code.

[0154] The specific process of updating network parameters is as follows:

[0155] 1: Use the priority experience replay method to extract batch_size groups from the experience replay pool.<s,a,r,s′>

[0156] 2: Input s' into the current Q-network (Q) and extract the action a* with the largest output Q value.

[0157] 3: Input s' into the target Q network (Q') and record the Q' value Q′(a*) of action a*.

[0158] 4: Estimating the distribution: Input s into the current Q-network (Q) to obtain the Q-value distribution of action a.

[0159] 5: Input s' into the target Q-network (Q') to obtain the Q' value distribution of action a*.

[0160] 6: Target Distribution: The target distribution is obtained by scaling, translating, and projecting the Q' value distribution of actions r and a*.

[0161] 7: Loss = Sample priority * Cross-entropy of estimated and target distributions

[0162] 8: Update network parameters using gradient descent based on the loss value.

[0163] After training for 450 rounds using the above process, observe whether the cumulative loss function of the model converges in each round and whether the cumulative reward in each round increases significantly. The cumulative loss function values ​​for the main pipeline agent and the sub-pipeline agents are as follows: Figure 4 It can be observed that the losses of both agents begin to decrease after 100 rounds, and the loss function values ​​gradually converge after 200 rounds.

[0164] Step S4 specifically includes:

[0165] S41: Adjust the gas meter valve opening based on the detection data;

[0166] S42: Detect abnormal situations. When an abnormal situation occurs, the user can control the user's gas solenoid valve and the community's main gas valve to exchange information through the Internet of Things (IoT) linkage with the kitchen monitoring system.

[0167] Specifically, this valve control method can adapt and adjust itself according to changes in input parameters, autonomously reason based on acquired information to select the combination of working-stage pipelines and regulating valves, decide the combination of opening degrees of multi-channel flow regulating valves, and continuously learn and optimize the model through feedback strategies.

[0168] The cloud platform automatically retrieves and analyzes data on users' gas usage habits, compares and contrasts them with historical gas usage data for the same period, analyzes the differences between the data on gas usage time and volume and the historical data, classifies the corresponding safety levels based on the differences, and takes corresponding solutions.

[0169] Preferably, based on the difference compared with a preset threshold, abnormal situations are detected. When the situation is at the first level of safety, the user's gas solenoid valve is controlled through the Internet of Things-linked kitchen monitoring system, and the user is alerted. When the situation is at the second level of safety, the user's gas solenoid valve is shut off, and the user is contacted urgently and the community management personnel are notified to check whether there is any abnormality in the community. When the situation is at the third level of safety, the community's main gas valve is controlled, and the community management personnel are notified to dispatch gas company safety inspectors to conduct on-site repairs.

[0170] A gas meter valve control system includes the following modules:

[0171] The information acquisition module is used to acquire the user's gas usage history and the location of the user's gas pipeline.

[0172] The model building module is used to build an intelligent adjustment model based on the acquired historical records and pipeline location information;

[0173] The intelligent monitoring module is used to monitor the user's real-time gas delivery pipeline, daily usage, and gas application using an intelligent adjustment model;

[0174] The intelligent adjustment module is used to automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas application data.

[0175] Preferably, the information acquisition module specifically includes:

[0176] The statistical analysis module is used to obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records;

[0177] The parameter acquisition module is used to obtain the arrangement position of gas equipment in the gas pipeline network through the cloud platform and to obtain pressure, flow and temperature parameter data from pipeline sensors.

[0178] Typically, indoor gas appliances consist of gas pipelines, gas valves, gas meters, gas water heaters, and gas stoves. Indoor gas leaks can be caused by two main factors: gas facility equipment and user habits.

[0179] The gas safety monitoring cloud platform hardware includes several components: alarms, a gas safety monitoring server, a cloud platform, AI gas safety valves, and thermal imaging dry-burn monitors. The gas safety monitoring server is connected to the cloud platform via the internet. Users can use WeChat and official accounts to check the equipment's operating status and alarm status, allowing for immediate dispatch of emergency response upon receiving alarm information on their mobile phones. See the appendix for details. Figure 2 .

[0180] The gas safety monitoring cloud platform includes a data layer, which focuses on the pressure information and flammable gas content information of gas pipelines. The platform's data covers geographic information and spatial data of gas pipeline equipment, including service data such as gas pipeline pressure and flammable gas content information collected from the user end, as well as monitoring data. Based on the above data, the cloud platform statistically analyzes the user's gas usage habits.

[0181] In addition, the data layer also acquires pressure, flow, and temperature parameters of the main pipeline and user branch pipelines in the community through pipeline sensors.

[0182] Preferably, the model building module specifically includes:

[0183] The IDQN building module constructs a multi-agent distributed reinforcement learning IDQN intelligent adjustment model based on the acquired historical records and pipeline position parameter information.

[0184] A multi-agent system refers to a group of autonomous individuals sharing the same environment and capable of interacting with each other. They perceive environmental changes and make decisions. Many real-world problems, such as robotic tasks requiring teamwork and traffic control, can be solved using multi-agent systems. Because agents in a multi-agent learning environment can interact with the environment simultaneously, and the actions of one agent may affect others, multi-agent learning is much more complex than single-agent learning. If we assume that all agents can observe the true state of the entire environment rather than a local state, we can build a single-agent model to represent the multi-agent system. However, this approach causes the action space of the single-agent model to grow exponentially with the number of agents, making it impractical for practical training. Therefore, a distributed learning strategy is chosen for training.

[0185] This invention establishes a multi-agent distributed reinforcement learning IDQN intelligent regulation model, including a cell main pipeline model and a user sub-pipeline model. The cell main pipeline model and the user sub-pipeline model are two independent agents, each with its own independent strategy, and both operate simultaneously in the same simulated pipeline network environment. See the appendix for details. Figure 3 The simulated pipeline collects information from data collection points within the pipeline and transmits the necessary training information to the model via the COM interface. Upon receiving the input state, the model combines the fingerprint of another agent as a decision-making basis, selects an action as the output, and executes it again through the COM interface into the pipeline. COM stands for Component Object Model, allowing users to control the pipeline externally through this interface.

[0186] This invention uses pipeline simulation software as a training platform, employs Python programming language to build a multi-agent reinforcement learning model, and interacts with the pipeline environment via a COM interface. The main pipeline agents and user sub-pipeline agents are trained separately, but simultaneously use the same environment for decision-making. The three most important elements in reinforcement learning are the settings of state, action, and reward, as well as the architecture of the neural network-like system. Since the state, action, reward, and network architecture settings used in the main pipeline and sub-pipeline models established in this invention are consistent, they are described here simultaneously.

[0187] State: The DQN learning method used in this invention uses a neural network as an approximation function of the Q function, which can effectively relax the restrictions on the dimension of the model input. Therefore, pressure, flow rate, and temperature are chosen as environmental states. This invention uses a fully connected layer as the Q function of DQN, so the state information needs to be converted into a one-dimensional input model. After obtaining the state information of the neural network, since pressure, flow rate, and temperature indicators have different scales, they are normalized before being input into the neural network to adjust the values ​​to between -1 and 1.

[0188] Actions: Instrumentation management achieves control objectives by adjusting the opening of valves. Most studies using reinforcement learning control methods choose discrete instrumentation rates as the instrumentation policy. The purpose of this invention is to improve the control logic of existing instrumentation policies, considering feasibility and the flexibility of adjusting the instrumentation rate. The action settings of the instrumentation model in this study are consistent with the current selection of discrete instrumentation rates, and a decision is made every 5 minutes. However, since the cycle time of each instrumentation rate is different, and the 5-minute setting may prevent some actions from completing a full cycle, the simulation software defaults to switching to a new instrumentation rate after the current phase is completed.

[0189] Rewards: Rewards are part of the objective value in reinforcement learning, so the setting of rewards has a great impact on the learning of agents. Different reward settings will result in different strategies of agents after training. The research purpose of this invention is to solve the intelligent safety management of gas pipeline networks. Therefore, this invention readjusts the reward definition. Considering the supply side, if the gas consumption in the pipeline within a unit time period is kept within a safe range compared with the historical gas consumption of the community or user, it can be considered intelligent and safe. Therefore, the flow rate in the pipeline within a unit time period is selected as the model reward setting, so that the model objective is to keep the flow rate at that location within a predetermined threshold compared with the historical gas consumption.

[0190] Neural Network Architecture: The state is first normalized to between -1 and 1 before being sent to the input layer. Then, it passes through three fully connected layers to form the output layer. Because this study employs a competitive network architecture and distributed reinforcement learning, the output layer is divided into a state-value stream and an action-dominance stream. Combining the results from both sides, the output layer shows the state-action value distribution for each action. In this distributed reinforcement learning approach, the number of atoms is set to 9, so the state-value stream is divided into 9 approximate state-value distributions, and the action-dominance stream also divides each action into 9 approximate action-dominance distributions. Therefore, the final output will be 72-dimensional. Distributed reinforcement learning changes the neural network output to the distribution of Q-values ​​for each action. First, the maximum value vmax and minimum value vmin of the Q-value distribution are defined, along with the number of atoms to divide the distribution. For example, vmax is 10, vmin is -10, and the number of atoms is 7. The loss function is then calculated by comparing the cross-entropy between the distribution of the actual network output and the distribution of the target network output. Overall, distributed reinforcement learning transforms the model output into the distribution of Q-values ​​for each action, enabling the model to acquire more useful information during training and thus making the training process more stable. The hardware and software configurations used in this study are shown in Table 1-2.

[0191] Preferably, the training process of the above-mentioned IDQN intelligent adjustment model specifically includes:

[0192] S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community.

[0193] S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions am and ar according to the greedy exploration strategy;

[0194] S23: Execute am and ar in the network environment, read the reward r obtained after the execution action, and read the new state s' after the execution action;

[0195] S23: Will<s,am,r,s’> The experience replay pool stored in the main pipeline agent will<s,ar,r,s’> Experience replay pool stored in the main pipeline agent;

[0196] S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents.

[0197] S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks.

[0198] S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

[0199] The training process of the multi-agent distributed reinforcement learning IDQN intelligent regulation model specifically includes:

[0200] S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community.

[0201] S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions am and ar according to the greedy exploration strategy;

[0202] S23: Execute am and ar in the network environment, read the reward r obtained after the execution action, and read the new state s' after the execution action;

[0203] S23: Will<s,am,r,s’> The experience replay pool stored in the main pipeline agent will<s,ar,r,s’> Experience replay pool stored in the main pipeline agent;

[0204] S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents.

[0205] S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks.

[0206] S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

[0207] To facilitate understanding, the device and software configuration details the model training process in a manner similar to pseudocode. Pseudocode is a way of describing algorithms; it is not an actual programming language, but rather uses the syntax of various programming languages ​​or natural language to help us better express the functionality of the algorithm.

[0208] The overall training process and algorithm flow of the multi-agent deep reinforcement learning model are described in detail below:

[0209] 1:global_steps=1.

[0210] 2: for episode in total_episode(450 episodes) do

[0211] 3: Reset the random seed and delete old simulation records

[0212] 4: Start warming up

[0213] 5: Start the main pipeline and branch pipeline control signs

[0214] 6: while current time < total simulation time (14400 seconds) do

[0215] 7: Read the current environment state s

[0216] 8: Input 's' into the main pipeline and sub-pipeline agents, and extract actions 'am' and 'ar' according to a greedy exploration strategy.

[0217] 9: Execute am and ar in the pipeline environment.

[0218] 10: Read the reward r obtained after executing the action.

[0219] 11: Read the new state s' after the action is executed

[0220] 12: will<s,am,r,s’> Experience replay pool stored in the main pipeline agent

[0221] 13: will<s,ar,r,s’> Experience replay pool stored in the pipeline agent

[0222] 14:if global_steps>pretrain_steps and global_steps%train_steps=0do

[0223] 15: Training main pipeline and branch pipeline agents

[0224] 16:if global_steps>pretrain_steps and global_steps%update_steps=0do

[0225] 17: Copy the network parameters of the main pipeline and branch pipelines to their respective target networks.

[0226] 18:global_steps+=1

[0227] 19:end for

[0228] The above is the overall training process of the model of this invention. The following describes the process of updating the network parameters for training one agent at a time, also in the form of virtual code.

[0229] The specific process of updating network parameters is as follows:

[0230] 1: Use the priority experience replay method to extract batch_size groups from the experience replay pool.<s,a,r,s′>

[0231] 2: Input s' into the current Q-network (Q) and extract the action a* with the largest output Q value.

[0232] 3: Input s' into the target Q network (Q') and record the Q' value Q′(a*) of action a*.

[0233] 4: Estimating the distribution: Input s into the current Q-network (Q) to obtain the Q-value distribution of action a.

[0234] 5: Input s' into the target Q-network (Q') to obtain the Q' value distribution of action a*.

[0235] 6: Target Distribution: The target distribution is obtained by scaling, translating, and projecting the Q' value distribution of actions r and a*.

[0236] 7: Loss = Sample priority * Cross-entropy of estimated and target distributions

[0237] 8: Update network parameters using gradient descent based on the loss value.

[0238] After training for 450 rounds using the above process, observe whether the cumulative loss function of the model converges in each round and whether the cumulative reward in each round increases significantly. The cumulative loss function values ​​for the main pipeline agent and the sub-pipeline agents are as follows: Figure 4 It can be observed that the losses of both agents begin to decrease after 100 rounds, and the loss function values ​​gradually converge after 200 rounds.

[0239] Preferably, the above-mentioned intelligent adjustment module specifically includes:

[0240] The meter valve adjustment module is used to adjust the opening of the gas meter valve based on monitoring data.

[0241] The anomaly handling module is used to detect abnormal situations. When an anomaly occurs, the user can exchange information with the kitchen monitoring system via the Internet of Things by controlling the user's gas solenoid valve and the main gas valve of the community.

[0242] Specifically, this valve control method can adapt and adjust itself according to changes in input parameters, autonomously reason based on acquired information to select the combination of working-stage pipelines and regulating valves, decide the combination of opening degrees of multi-channel flow regulating valves, and continuously learn and optimize the model through feedback strategies.

[0243] The cloud platform automatically retrieves and analyzes data on users' gas usage habits, compares and contrasts them with historical gas usage data for the same period, analyzes the differences between the data on gas usage time and volume and the historical data, classifies the corresponding safety levels based on the differences, and takes corresponding solutions.

[0244] Preferably, based on the difference compared with a preset threshold, abnormal situations are detected. When the situation is at the first level of safety, the user's gas solenoid valve is controlled through the Internet of Things-linked kitchen monitoring system, and the user is alerted. When the situation is at the second level of safety, the user's gas solenoid valve is shut off, and the user is contacted urgently and the community management personnel are notified to check whether there is any abnormality in the community. When the situation is at the third level of safety, the community's main gas valve is controlled, and the community management personnel are notified to dispatch gas company safety inspectors to conduct on-site repairs.

[0245] The results above show that the present invention provides a gas meter valve control system and method. By constructing a multi-agent distributed reinforcement learning IDQN intelligent adjustment model, the gas meter valve can be detected quickly and intelligently, which is more scientific, safe and intelligent, greatly improving management efficiency, reducing misjudgments and improving user experience.

[0246] Those skilled in the art will recognize that the units of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware.

[0247] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0248] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and other division methods may exist in actual implementation. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0249] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A gas meter valve control method, comprising the following steps: S1: Obtain the user's gas usage history and the location of the user's gas pipeline; S2: Construct an intelligent adjustment model based on the acquired historical records and pipeline location information; S3: Utilizes intelligent regulation models to monitor users' real-time gas delivery pipelines, daily usage, and gas application. S4: Automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas application data; Step S2 specifically includes: Based on the acquired historical records, main pipeline of the community, and user sub-pipeline location parameters, a multi-agent distributed reinforcement learning IDQN intelligent regulation model is constructed. The training process of this multi-agent distributed reinforcement learning IDQN intelligent regulation model specifically includes: S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community. S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions according to the greedy exploration strategy. , ; S23: Will , Execute in the network environment, read the reward r obtained after the action is executed, and read the new state s' after the action is executed; S23: Will < , , , >Store the experience replay pool to the main pipeline agent, and < , , , >Store the experience replay pool to the main pipeline agent; S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents. S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks. S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

2. The gas meter valve control method according to claim 1, characterized in that, Step S1 specifically includes: S11: Obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records; S12: Obtain the location of gas equipment in the gas pipeline network through the cloud platform, and obtain pressure, flow and temperature parameter data from pipeline sensors.

3. The gas meter valve control method according to claim 1, characterized in that, Step S4 specifically includes: S41: Adjust the gas meter valve opening based on the detection data; S42: Detect abnormal situations. When an abnormal situation occurs, the user can control the user's gas solenoid valve and the community's main gas valve to exchange information through the Internet of Things (IoT) linkage with the kitchen monitoring system.

4. A gas meter valve control system, comprising the following modules: The information acquisition module is used to acquire the user's gas usage history and the location of the user's gas pipeline. The model building module is used to build an intelligent adjustment model based on the acquired historical records and pipeline location information; The intelligent monitoring module is used to monitor the user's real-time gas delivery pipeline, daily usage, and gas application using an intelligent adjustment model; The intelligent adjustment module is used to automatically adjust the gas meter valve based on the monitored gas transmission pipeline, daily usage, and gas usage data. The model building module specifically includes: Based on the acquired historical records, main pipeline of the community, and user sub-pipeline location parameters, a multi-agent distributed reinforcement learning IDQN intelligent regulation model is constructed. The training process of the multi-agent distributed reinforcement learning IDQN intelligent regulation model specifically includes: S21: Reset the random seed of the simulation software, delete old simulation records, and activate the control flags for the main pipeline and user sub-pipelines in the community. S22: Read the current environment state s, input s into the main pipeline and sub-pipeline agents, and extract actions according to the greedy exploration strategy. , ; S23: Will , Execute in the network environment, read the reward r obtained after the action is executed, and read the new state s' after the action is executed; S23: Will < , , , >Store the experience replay pool to the main pipeline agent, and < , , , >Store the experience replay pool to the main pipeline agent; S24: Determine whether the set training conditions are met. If they are met, train the main pipeline and sub-pipeline agents. S25: Determine whether the set update conditions are met. If they are met, copy the network parameters of the main pipeline and sub-pipeline agents to their respective target networks. S26: Determine whether the set termination condition is met. If not, return to step S21 to continue the iteration.

5. The gas meter valve control system according to claim 4, characterized in that, The information acquisition module specifically includes: The statistical analysis module is used to obtain users' historical records through the cloud platform and perform statistical analysis based on user usage records; The parameter acquisition module is used to obtain the arrangement position of gas equipment in the gas pipeline network through the cloud platform and to obtain pressure, flow and temperature parameter data from pipeline sensors.

6. The gas meter valve control system according to claim 4, characterized in that, The intelligent adjustment module specifically includes: The meter valve adjustment module is used to adjust the opening of the gas meter valve based on monitoring data. The anomaly handling module is used to detect abnormal situations. When an anomaly occurs, the user can exchange information with the kitchen monitoring system via the Internet of Things, controlling the user's gas solenoid valve and the community's main gas valve.

Images