Gas sensing leakage risk intelligent control method based on reinforcement learning

Abstract

The invention discloses a reinforcement learning-based gas sensing leakage risk intelligent control method, which comprises the following steps of S1, collecting data of a gas sensor, and performing preprocessing; s2, extracting local spatial features through convolution operation, and performing feature weight adjustment in combination with a CBAM attention mechanism; s3, inputting the state vector sequence into an IQN network, and constructing an action value distribution function based on quantile estimation; s4, selecting an action corresponding to the maximum quantile index, and controlling ventilation equipment, a pipeline valve or an alarm device; s5, binding the feedback state information with the state vector sequence, and carrying out batch gradient updating on the IQN network by adopting a segmented Huber loss function; and S6, repeatedly executing the whole process until the action value distribution function is converged and stable output is kept. According to the invention, high-precision identification and self-adaptive control of gas leakage are realized, and the response efficiency and stability are improved.

Description

Technical Field

[0001] The present invention relates to the field of gas sensing and environmental perception technology, and in particular to a method for intelligent control of gas leakage risk based on reinforcement learning. Background Art

[0002] In the context of highly integrated industrial production and urban infrastructure, the widespread use of flammable, toxic, and explosive gases has made the risk of gas leaks an increasingly critical factor affecting public safety and environmental health. Traditional gas leak monitoring and control methods typically rely on fixed threshold judgment models. That is, when the sensor detects that the concentration of a certain type of gas exceeds a preset standard, the system triggers an alarm signal and controls the actuator to perform a response operation, such as starting a ventilation device or closing a solenoid valve. Although this method has certain practicality in simple scenarios, its emergency response accuracy and strategy robustness are easily challenged when faced with multi-source sensor input, complex environmental disturbances, and the uncertainty of the leakage process. There are problems such as false alarms, missed alarms, slow response, and irrational allocation of control resources.

[0003] Some current studies have attempted to enhance the gas leak detection process by introducing intelligent algorithms, such as using support vector machines, decision trees, or simple neural network structures to classify or regress gas concentration trends. These methods primarily focus on the "leak" detection level, lack in-depth modeling of the dynamic evolution of leaks under complex working conditions, are unable to effectively capture the nonlinear correlations between multidimensional environmental variables, and struggle to support linkage control and multi-device collaborative response. At the same time, most of these methods use a supervised learning paradigm and rely heavily on a large number of labeled samples for training, making it extremely difficult to obtain complete, high-quality leakage data in high-risk industrial scenarios. Furthermore, existing research models are mostly static strategy generation, lacking online learning and strategy adaptation capabilities. When faced with sudden changes in environmental conditions or equipment status, the model's generalization ability significantly decreases, leading to failure of control effects and even inducing secondary risks.

[0004] On the other hand, in recent years, reinforcement learning technology has demonstrated significant advantages in complex decision-making and control tasks, particularly in scenarios such as autonomous driving, intelligent manufacturing, and energy scheduling. Through repeated interactions between an agent and its environment, it autonomously learns optimal strategies to maximize its objectives. However, in the field of gas leak risk control, the application of reinforcement learning is still in its infancy. Existing solutions are mostly limited to theoretical simulations or experimental research on single action outputs. They lack the ability to model high-dimensional state spaces and fail to integrate industrial-grade sensor data to build stable and efficient policy optimization mechanisms.

[0005] Furthermore, existing reinforcement learning models often use traditional Q-value estimation, which fails to effectively model the uncertainty of action outcomes. This leads to insufficient control strategy stability. This is particularly true in the face of extreme leaks or sudden environmental disturbances, where system outputs exhibit jitter, latency, and increased uncertainty. Currently, few methods incorporate probability distribution modeling to quantify and rank the risk expectations of actions, making it difficult to provide a reliable strategic basis for industrial safety control.

[0006] At the perception level, existing methods also have limited utilization of sensor data. Most control logic triggers control commands based solely on a single gas concentration value, failing to incorporate multi-dimensional information such as temperature, humidity, air pressure, and wind speed to extract spatiotemporal features. This results in a single state representation and insufficient environmental understanding. Furthermore, the introduction of attention mechanisms is relatively rare, making it difficult to automatically focus on key feature dimensions and spatial regions in high-dimensional inputs, thus affecting the accuracy and responsiveness of the overall control effect.

[0007] Therefore, how to provide an intelligent control method for gas leakage risk based on reinforcement learning is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0008] One purpose of the present invention is to propose an intelligent control method for gas leakage risk based on reinforcement learning. The present invention integrates convolutional feature extraction, CBAM attention mechanism, quantile distribution modeling and IQN reinforcement learning network, constructs a dynamic state representation and action strategy optimization mechanism for multi-source gas sensor data, and describes in detail the entire process of realizing intelligent identification of gas leaks, risk response and strategy self-update in complex industrial scenarios. It has the advantages of high response accuracy, strong control strategy stability and excellent environmental adaptability.

[0009] According to an embodiment of the present invention, a method for intelligently controlling gas leakage risk based on reinforcement learning includes the following steps:

[0010] S1, collect gas sensor data and perform preprocessing;

[0011] S2. Based on the preprocessed aura data, local spatial features are extracted through convolution operations, and the feature weights are adjusted in combination with the CBAM attention mechanism to generate a state vector sequence with spatial position embedding;

[0012] S3. Input the state vector sequence into the IQN network to construct an action value distribution function based on quantile value estimation. The IQN network consists of a state encoding module, a quantile sampling module, and an action sorting module.

[0013] S4. Select the action corresponding to the maximum quantile index as the response operation in the gas leakage scenario, and control the ventilation equipment, pipeline valve or alarm device, and collect feedback status information after the action is executed;

[0014] S5. Bind the feedback state information to the state vector sequence and write it into the error-sorted experience buffer pool. Use the piecewise Huber loss function to perform batch gradient updates on the IQN network.

[0015] S6. Repeat the state construction, action selection, feedback collection and IQN network update process until the action value distribution function converges and maintains a stable output.

[0016] Optionally, the gas sensor data includes gas concentration, temperature, humidity, wind speed, air pressure parameters and sampling timestamp.

[0017] Optionally, the preprocessing includes data cleaning, missing value filling, outlier removal, time series reconstruction and data standardization.

[0018] Optionally, the feedback status information includes the gas concentration change value, temperature and humidity change value, wind speed change value, air pressure change value and execution time delay after the action is executed, and together with the action type and execution device number, constitutes a feedback record vector, which is used to construct sample entries in the experience cache pool.

[0019] Optionally, the S2 specifically includes:

[0020] S21. Define the preprocessed gas sensor data as a tensor matrix Where m represents the length of the time series, n represents the number of sensors, and c represents the feature dimension corresponding to each sampling point. The feature dimension consists of gas concentration, temperature, humidity, wind speed, air pressure, and sampling timestamp.

[0021] S22, perform a two-dimensional convolution operation on the tensor matrix D, set the convolution kernel size to k×k, and set the step size to s to obtain the local feature matrix in represents the output size of the time dimension after convolution, represents the output size of the spatial dimension after convolution, f is the number of convolution kernels, Indicates rounding down;

[0022] S23. Apply the local feature matrix F to the CBAM structure that combines the channel attention mechanism and the spatial attention mechanism. The channel attention weight vector is denoted as α, and the spatial attention weight matrix is denoted as β. The two are applied jointly to F to obtain the weighted feature matrix F′:

[0023] F′ i,j,k =Fi,j,k ·σ(α k )·tanh(β i,j );

[0024] Among them, σ(·) represents the Sigmoid function, tanh(·) represents the hyperbolic tangent function, α k represents the importance weight of the kth channel, β i,j represents the response weight of the spatial position of row i and column j, F i,j,k represents the local feature element of the i-th row, j-th column, and k-th channel, F′ i,j,k Represents the weighted feature element of the i-th row, j-th column, and k-th channel;

[0025] S24. Perform spatial position embedding mapping on the weighted feature matrix F′, and fuse it with the weighted feature matrix through the position encoding function PE(t) to obtain the state vector sequence S.

[0026] Optionally, the state vector sequence S is expressed as follows:

[0027]

[0028] Among them, s t represents the state vector at time t, T represents the total length of the state vector sequence, δ t,i,j is the spatial position selection function, where the value is 1 if the position (i, j) belongs to the perception area corresponding to time step t, otherwise it is 0, λ is the position embedding weight coefficient, sin(·) represents the hyperbolic sine function, cos(·) represents the hyperbolic cosine function, ω k Represents the kth position encoding frequency factor, which is used to construct the frequency domain feature embedding term in the time series. satisfy

[0029] Optionally, the S3 specifically includes:

[0030] S31. Input the state vector sequence S into the IQN network for encoding, and use the fully connected mapping to construct the state representation tensor H:

[0031]

[0032] Among them, H t,j represents the state representation of the jth hidden dimension at time step t, Represents the weight matrix from the state vector to the latent space, is the bias vector of the jth hidden dimension, ReLU(·) is the rectified linear unit function, f is the number of convolution kernels, s t,k Represents the state vector corresponding to the k-th convolution kernel at time t;

[0033] S32. Construct a quantile value sample set in the quantile sampling structure of the IQN network τ i Perform embedding mapping to generate the quantile tensor Ψ;

[0034] S33. Double interactive mapping is performed between the state representation tensor H and the quantile tensor Ψ to construct the action value distribution function:

[0035]

[0036] Among them, Q t,a,i represents the action value estimate at time step t, action a, and quantile index i, is the action mapping weight matrix, is the action bias vector, d represents the embedding dimension of the state representation tensor, and tanh(·) represents the hyperbolic tangent function;

[0037] S34. In the action ranking structure of the IQN network, all action value estimates are averaged to obtain a quantile mean vector, which is used to sort the action priorities.

[0038] Optionally, the embedded mapping process expression is as follows:

[0039] Ψ i,j =cos(πj·τ i )+sin(πj·τ i );

[0040] Among them, i,j Represents the quantile τ i Positional encoding value in the jth embedding dimension, τ i ∈(0,1) represents the i-th quantile, π is the constant pi, and N represents the number of sampled quantile values;

[0041] Optionally, the S4 specifically includes:

[0042] S41. Select the action index a with the largest quantile value from the quantile mean vector * , as the response operation in the gas leakage scenario, and mapping the action to the actual control instruction μ:

[0043]

[0044] Among them, e represents the device number, Represents action a at time step t * The quantile mean, μ e represents the control signal strength of device number e, and γa*,e∈{0,1} represents action a *Whether to act on device e, η is the nonlinear enhancement coefficient, which controls the slope of the response curve;

[0045] S42, controlling ventilation equipment, pipeline valves or alarm devices according to actual control instructions, and collecting feedback status information after the action is executed, the structure is as follows:

[0046]

[0047] Among them, v represents the feedback indicator number, R v Represents the same type of feedback status information of the vth indicator after the action is executed, represents the observed value of the v-th indicator before the action is executed, represents the observation of the vth indicator after the action is executed, τ represents the delay time of the current control step, ρ v is the sensitivity factor of the vth index to time delay, δ t represents the actual time delay from the action being sent to the feedback being collected, and log2(·) represents the logarithmic function;

[0048] S43. Encode all feedback status information to form a feedback record vector R, which serves as a component element of storage entries in the experience buffer pool.

[0049] Optionally, the S5 specifically includes:

[0050] S51, bind the feedback record vector R with the state vector sequence S, construct a sample tuple, write it into the error-sorted experience buffer pool, and assign an initial priority weight ω to each sample i :

[0051]

[0052] in, represents the mean value of the same type of feedback indicators in the i-th historical sample, Represents action a at time step t * The quantile mean of represents the historical action value estimate at time step t, κ represents the error fusion coefficient;

[0053] S52, from the experience cache pool according to weight ω i Sorting is performed, and the top K samples are selected using the Top-K sampling method to form a batch set for gradient calculation and parameter optimization of the IQN network;

[0054] S53, use the segmented Huber loss function to perform error calculation and back propagation update on the IQN network, and set the target quantile value to Define the loss function under sample b and quantile i as

[0055]

[0056] in, Action a in the IQN network (b) The valuation at the i-th quantile, δ is the Huber loss segmentation threshold;

[0057] S54, with For the loss function, batch gradient updates of IQN network parameters are performed to continuously optimize the estimation accuracy of the action value distribution function.

[0058] The beneficial effects of the present invention are:

[0059] First, by introducing convolution operations and the CBAM attention mechanism, the present invention can fully extract local spatial features and key position weights in multi-dimensional sensor data such as gas concentration, temperature, humidity, wind speed, and air pressure, and construct a state vector sequence with spatial embedding capabilities. Compared with the traditional method based only on threshold judgment or simple feature input, it greatly improves the perception accuracy and state expression ability of leakage behavior in complex environments.

[0060] Secondly, this paper employs an IQN reinforcement learning network based on quantile distribution estimation, constructing the action-value function distribution using quantile sampling, and combining piecewise Huber loss with a prioritized experience cache mechanism for policy training and optimization. This overcomes the issues of policy jitter and instability in traditional reinforcement learning under extreme conditions. Through the maximum quantile index decision-making mechanism, it effectively prioritizes and outputs high-risk control actions, ensuring the policy's robustness and generalization capabilities in variable leakage environments.

[0061] Finally, the present invention collects feedback status information after the action is executed and writes this feedback and action execution records together into an experience buffer pool, which is used to drive iterative updates of the reinforcement learning model, thus achieving an adaptive control process from perception and decision-making to feedback closed-loop optimization. This mechanism enables the model to continuously learn and dynamically adjust its strategy, continuously accumulating experience over long-term operation, thereby effectively improving the real-time response efficiency, safety assurance capabilities, and intelligent evolution level of the gas leakage control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0063] Figure 1 This is a flow chart of a gas leakage risk intelligent control method based on reinforcement learning proposed by the present invention;

[0064] Figure 2This is a schematic diagram of feature extraction and weight adjustment of a reinforcement learning-based intelligent control method for gas leakage risk proposed by the present invention;

[0065] Figure 3 This is an IQN network input and output flow chart of the intelligent control method for gas leakage risk based on reinforcement learning proposed by the present invention. DETAILED DESCRIPTION

[0066] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0067] refer to Figure 1-3 , an intelligent control method for gas leakage risk based on reinforcement learning, comprising the following steps:

[0068] S1, collect gas sensor data and perform preprocessing;

[0069] S2. Based on the preprocessed aura data, local spatial features are extracted through convolution operations, and the feature weights are adjusted in combination with the CBAM attention mechanism to generate a state vector sequence with spatial position embedding;

[0070] S3. Input the state vector sequence into the IQN network to construct an action value distribution function based on quantile value estimation. The IQN network consists of a state encoding module, a quantile sampling module, and an action sorting module.

[0071] S4. Select the action corresponding to the maximum quantile index as the response operation in the gas leakage scenario, and control the ventilation equipment, pipeline valve or alarm device, and collect feedback status information after the action is executed;

[0072] S5. Bind the feedback state information to the state vector sequence and write it into the error-sorted experience buffer pool. Use the piecewise Huber loss function to perform batch gradient updates on the IQN network.

[0073] S6. Repeat the state construction, action selection, feedback collection and IQN network update process until the action value distribution function converges and maintains a stable output.

[0074] The present invention realizes intelligent recognition and response to gas leakage scenarios by constructing a complete reinforcement learning control process from sensor data acquisition, preprocessing, state construction, strategy estimation, action execution to feedback update. It has the ability of full-process closed-loop control, adaptive optimization and stable strategy output, which significantly improves the real-time and robustness of risk control.

[0075] In this embodiment, the gas sensor data includes gas concentration, temperature, humidity, wind speed, air pressure parameters and sampling timestamp.

[0076] The present invention introduces multiple environmental parameters such as temperature, humidity, wind speed, air pressure and sampling timestamp into the original gas sensing data, making the state representation more comprehensive and dimensional, which helps to improve the strategy's perception accuracy and generalization ability of leakage risks under different working conditions.

[0077] In this embodiment, the preprocessing includes data cleaning, missing value filling, outlier removal, time series reconstruction and data standardization.

[0078] The present invention designs a preprocessing process of data cleaning, missing value filling, outlier removal, time series reconstruction and standardization, which ensures the quality of input data while enhancing the stability and accuracy of model training, laying a solid foundation for the construction of reinforcement learning strategies.

[0079] In this embodiment, the feedback status information includes the gas concentration change value, temperature and humidity change value, wind speed change value, air pressure change value and execution time delay after the action is executed, and together with the action type and execution device number, it constitutes a feedback record vector, which is used to construct sample entries in the experience cache pool.

[0080] The present invention records the changes in gas concentration, environmental parameters and execution delay after the execution of an action, constructs a high-dimensional feedback vector, and comprehensively quantifies the impact of the action results, thereby improving the representativeness of experience samples in training and the effectiveness of strategy updates.

[0081] In this embodiment, S2 specifically includes:

[0082] S21. Define the preprocessed gas sensor data as a tensor matrix Where m represents the length of the time series, n represents the number of sensors, and c represents the feature dimension corresponding to each sampling point. The feature dimension consists of gas concentration, temperature, humidity, wind speed, air pressure, and sampling timestamp.

[0083] S22, perform a two-dimensional convolution operation on the tensor matrix D, set the convolution kernel size to k×k, and set the step size to s to obtain the local feature matrix in represents the output size of the time dimension after convolution, represents the output size of the spatial dimension after convolution, f is the number of convolution kernels, Indicates rounding down;

[0084] S23. Apply the local feature matrix F to the CBAM structure that combines the channel attention mechanism and the spatial attention mechanism. The channel attention weight vector is denoted as α, and the spatial attention weight matrix is denoted as β. The two are applied jointly to F to obtain the weighted feature matrix F′:

[0085] F′ i,j,k =F i,j,k ·σ(α k )·tanh(β i,j );

[0086] Among them, σ(·) represents the Sigmoid function, tanh(·) represents the hyperbolic tangent function, α k represents the importance weight of the kth channel, β i,j represents the response weight of the spatial position of row i and column j, F i,j,k represents the local feature element of the i-th row, j-th column, and k-th channel, F′ i,j,k Represents the weighted feature element of the i-th row, j-th column, and k-th channel;

[0087] S24. Perform spatial position embedding mapping on the weighted feature matrix F′, and fuse it with the weighted feature matrix through the position encoding function PE(t) to obtain the state vector sequence S.

[0088] The present invention combines convolution operation with CBAM attention mechanism to perform spatial feature extraction and key area weighted processing on sensor time series data, which improves the model's perception of local spatial changes and important feature dimensions, and provides a guarantee for the efficient construction of state vectors.

[0089] In this embodiment, the state vector sequence S is expressed as follows:

[0090]

[0091] Among them, s t represents the state vector at time t, T represents the total length of the state vector sequence, δ t,i,j is the spatial position selection function, where the value is 1 if the position (i, j) belongs to the perception area corresponding to time step t, otherwise it is 0, λ is the position embedding weight coefficient, sin(·) represents the hyperbolic sine function, cos(·) represents the hyperbolic cosine function, ω k Represents the kth position encoding frequency factor, which is used to construct the frequency domain feature embedding term in the time series. satisfy

[0092] The present invention constructs a state vector sequence that integrates spatial embedding and position encoding, which can express the perception area and frequency domain characteristics at different time steps, enhances the state modeling's ability to represent dynamic spatial changes, and is conducive to the reinforcement learning model to capture the leakage evolution process.

[0093] In this embodiment, S3 specifically includes:

[0094] S31. Input the state vector sequence S into the IQN network for encoding, and use the fully connected mapping to construct the state representation tensor H:

[0095]

[0096] Among them, H t,j represents the state representation of the jth hidden dimension at time step t, Represents the weight matrix from the state vector to the latent space, is the bias vector of the jth hidden dimension, ReLU(·) is the rectified linear unit function, f is the number of convolution kernels, s t,k Represents the state vector corresponding to the k-th convolution kernel at time t;

[0097] S32. Construct a quantile value sample set in the quantile sampling structure of the IQN network τ i Perform embedding mapping to generate the quantile tensor Ψ;

[0098] S33. Double interactive mapping is performed between the state representation tensor H and the quantile tensor Ψ to construct the action value distribution function:

[0099]

[0100] Among them, Q t,a,i represents the action value estimate at time step t, action a, and quantile index i, is the action mapping weight matrix, is the action bias vector, d represents the embedding dimension of the state representation tensor, and tanh(·) represents the hyperbolic tangent function;

[0101] S34. In the action ranking structure of the IQN network, all action value estimates are averaged to obtain a quantile mean vector, which is used to sort the action priorities.

[0102] The present invention introduces the IQN network structure and realizes the construction of action value distribution function through state coding, quantile sampling and action interaction mapping. It not only provides the uncertainty modeling capability of action output, but also improves the stability and risk avoidance capability of strategy selection.

[0103] In this embodiment, the embedded mapping process is expressed as follows:

[0104] Ψ i,j =cos(πj·τ i )+sin(πj·τ i );

[0105] Among them, i,j Represents the quantile τ i Positional encoding value in the jth embedding dimension, τ i ∈(0,1) represents the i-th quantile, π is the constant pi, and N represents the number of sampled quantile values;

[0106] The present invention utilizes the triangular coding embedding mechanism of quantile values for high-dimensional information mapping, which enables the reinforcement learning model to have fine-grained action value estimation capabilities, significantly enhancing the model's distribution expression capabilities and strategy resolution accuracy.

[0107] In this embodiment, the S4 specifically includes:

[0108] S41. Select the action index a with the largest quantile value from the quantile mean vector * , as the response operation in the gas leakage scenario, and mapping the action to the actual control instruction μ:

[0109]

[0110] Among them, e represents the device number, Represents action a at time step t * The quantile mean, μ e represents the control signal strength of device number e, and γa*,e∈{0,1} represents action a * Whether to act on device e, η is the nonlinear enhancement coefficient, which controls the slope of the response curve;

[0111] S42, controlling ventilation equipment, pipeline valves or alarm devices according to actual control instructions, and collecting feedback status information after the action is executed, the structure is as follows:

[0112]

[0113] Among them, v represents the feedback indicator number, R v Represents the same type of feedback status information of the vth indicator after the action is executed, represents the observed value of the v-th indicator before the action is executed, represents the observation of the vth indicator after the action is executed, τ represents the delay time of the current control step, ρ v is the sensitivity factor of the vth index to time delay, δ t represents the actual time delay from the action being sent to the feedback being collected, and log2(·) represents the logarithmic function;

[0114] S43. Encode all feedback status information to form a feedback record vector R, which serves as a component element of storage entries in the experience buffer pool.

[0115] The present invention realizes refined action response and equipment control through the strategy selection method of maximum quantile index, combined with actual control instruction mapping and feedback information construction, effectively improving the system's response efficiency and risk intervention accuracy.

[0116] In this embodiment, the S5 specifically includes:

[0117] S51, bind the feedback record vector R with the state vector sequence S, construct a sample tuple, write it into the error-sorted experience buffer pool, and assign an initial priority weight ω to each sample i :

[0118]

[0119] in, represents the mean value of the same type of feedback indicators in the i-th historical sample, Represents action a at time step t * The quantile mean of represents the historical action value estimate at time step t, κ represents the error fusion coefficient;

[0120] S52, from the experience cache pool according to weight ω i Sorting is performed, and the top K samples are selected using the Top-K sampling method to form a batch set for gradient calculation and parameter optimization of the IQN network;

[0121] S53, use the segmented Huber loss function to perform error calculation and back propagation update on the IQN network, and set the target quantile value to Define the loss function under sample b and quantile i as

[0122]

[0123] in, Action a in the IQN network (b) The valuation at the i-th quantile, δ is the Huber loss segmentation threshold;

[0124] S54, with For the loss function, batch gradient updates of IQN network parameters are performed to continuously optimize the estimation accuracy of the action value distribution function.

[0125] The present invention uses an empirical caching mechanism based on error sorting and a piecewise Huber loss optimization algorithm to perform high-confidence sample training and robustness enhancement updates on the IQN network, achieving a stable and reliable strategy optimization path and improving the credibility and generalization ability of the control strategy in actual industrial scenarios.

[0126] Example 1:

[0127] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the intelligent control system for gas leakage in the hazardous materials storage area of a chemical park. There are many types of high-risk gas storage tanks and pipelines in this area, such as propane, ammonia, methanol, etc. In actual operation, small-scale gas leakage incidents are very likely to occur due to pipeline aging, loose interfaces, sudden temperature changes, etc. The traditional leakage monitoring method adopts a fixed threshold + single-point alarm model, which is insensitive to environmental disturbances, and the information between multiple sensors is fragmented, and a collaborative judgment cannot be formed, resulting in frequent alarms but low accuracy, and the false alarm rate remains above 22% for a long time. At the same time, in some key ventilation areas, the valve startup lag exceeds 4 seconds, there is a significant delay in risk response, and the feedback response mechanism of on-site operators is "sluggish" and "false alarm" is serious.

[0128] To address the above issues, the present invention is deployed in this scenario. The sensing layer is equipped with five types of sensor nodes: gas concentration, temperature, humidity, wind speed, and air pressure. A total of 32 sensor points are deployed around the storage area and at the pipeline interface, with a sampling frequency of 2 times per second. First, the system accesses the raw sensor data and performs preprocessing such as data cleaning and standardization to form a continuous time series tensor. In the state modeling stage, the convolutional structure is combined with the CBAM attention mechanism to extract spatial features from the sensor matrix and generate a state vector sequence. The IQN network is used to perform quantile sampling and action value distribution modeling. After the control action is output, the system will apply the selected response instructions to the automatic fan, pipeline solenoid valve, and local alarm, and record the execution results in real time, constructing a feedback vector for policy update.

[0129] After 30 days of continuous operation, a total of 142 gas leakage-related risk events were recorded, of which 110 automatic response actions were triggered, and 106 risk diffusion events were effectively controlled, with a control success rate of 96.4%, an increase of 14.3% over the original traditional control mechanism (82.1%). The system response delay was reduced from the original average of 3.97 seconds to 1.21 seconds, an overall reduction of 69.5%. In scenarios where the risk level is higher than level 3 (5-level scale), this system can trigger the joint linkage of fans and valves within 2 seconds to block the gas diffusion path and restore the leakage gas concentration to below the warning threshold within 15 seconds. In terms of strategy stability, with the increase in the number of training rounds, the action percentile mean error of the IQN network converged from the initial 0.184 to 0.029, and the control strategy was basically stable without jitter or repeated switching.

[0130] To further analyze the control system's performance, we compared some key indicators from a high-concentration leak trigger scenario, as shown in the table below. The table records key data such as gas concentration changes before and after the leak trigger, action response delay, and control success rate.

[0131] Table 1 Statistics of key operating indicators of the reinforcement learning gas sensing control system

[0132]

[0133]

[0134] In summary, the present invention demonstrates excellent state recognition capabilities, strategy control accuracy, and response timeliness in complex gas leakage risk scenarios. It not only effectively improves the intelligence level of leakage control, but also significantly reduces safety risks and manual intervention costs, and has good engineering promotion and practical application value.

[0135] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for intelligent control of gas leakage risk based on reinforcement learning, characterized in that: The steps include: S1, collect gas sensor data and perform preprocessing; S2. Based on the preprocessed aura data, local spatial features are extracted through convolution operations, and the feature weights are adjusted in combination with the CBAM attention mechanism to generate a state vector sequence with spatial position embedding; S3. Input the state vector sequence into the IQN network to construct an action value distribution function based on quantile value estimation. The IQN network consists of a state encoding module, a quantile sampling module, and an action sorting module. S4. Select the action corresponding to the maximum quantile index as the response operation in the gas leakage scenario, and control the ventilation equipment, pipeline valve or alarm device, and collect feedback status information after the action is executed; S5. Bind the feedback state information to the state vector sequence and write it into the error-sorted experience buffer pool. Use the piecewise Huber loss function to perform batch gradient updates on the IQN network. S6. Repeat the state construction, action selection, feedback collection and IQN network update process until the action value distribution function converges and maintains a stable output.

2. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The gas sensor data includes gas concentration, temperature, humidity, wind speed, air pressure parameters and sampling timestamp.

3. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The preprocessing includes data cleaning, missing value filling, outlier removal, time series reconstruction and data standardization.

4. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The feedback status information includes the gas concentration change value, temperature and humidity change value, wind speed change value, air pressure change value and execution time delay after the action is executed, and together with the action type and execution device number, it constitutes a feedback record vector, which is used to construct sample entries in the experience cache pool.

5. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The S2 specifically includes: S21. Define the preprocessed gas sensor data as a tensor matrix Where m represents the length of the time series, n represents the number of sensors, and c represents the feature dimension corresponding to each sampling point. The feature dimension consists of gas concentration, temperature, humidity, wind speed, air pressure, and sampling timestamp. S22, perform a two-dimensional convolution operation on the tensor matrix D, set the convolution kernel size to k×k, and set the step size to s to obtain the local feature matrix in represents the output size of the time dimension after convolution, represents the output size of the spatial dimension after convolution, f is the number of convolution kernels, Indicates rounding down; S23. Apply the local feature matrix F to the CBAM structure that combines the channel attention mechanism and the spatial attention mechanism. The channel attention weight vector is denoted as α, and the spatial attention weight matrix is denoted as β. The two are applied jointly to F to obtain the weighted feature matrix F′: F′ i,j,k =F i,j,k ·σ(α k )·tanh(β i,j ); Among them, σ(·) represents the Sigmoid function, tanh(·) represents the hyperbolic tangent function, α k represents the importance weight of the kth channel, β i,j represents the response weight of the spatial position of row i and column j, F i,j,k represents the local feature element of the i-th row, j-th column, and k-th channel, F′ i,j,k Represents the weighted feature element of the i-th row, j-th column, and k-th channel; S24. Perform spatial position embedding mapping on the weighted feature matrix F′, and fuse it with the weighted feature matrix through the position encoding function PE(t) to obtain the state vector sequence S.

6. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 5, characterized in that: The state vector sequence S is expressed as follows: Among them, s t represents the state vector at time t, T represents the total length of the state vector sequence, δ t,i,j is the spatial position selection function, where the value is 1 if the position (i, j) belongs to the perception area corresponding to time step t, otherwise it is 0, λ is the position embedding weight coefficient, sin(·) represents the hyperbolic sine function, cos(·) represents the hyperbolic cosine function, ω k Represents the kth position encoding frequency factor, which is used to construct the frequency domain feature embedding term in the time series. satisfy 7. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The S3 specifically includes: S31. Input the state vector sequence S into the IQN network for encoding, and use the fully connected mapping to construct the state representation tensor H: Among them, H t,j represents the state representation of the jth hidden dimension at time step t, Represents the weight matrix from the state vector to the latent space, is the bias vector of the jth hidden dimension, ReLU(·) is the rectified linear unit function, f is the number of convolution kernels, s t,k Represents the state vector corresponding to the k-th convolution kernel at time t; S32. Construct a quantile value sample set in the quantile sampling structure of the IQN network τ i Perform embedding mapping to generate the quantile tensor Ψ; S33. Double interactive mapping is performed between the state representation tensor H and the quantile tensor Ψ to construct the action value distribution function: Among them, Q t,a,i represents the action value estimate at time step t, action a, and quantile index i, is the action mapping weight matrix, is the action bias vector, d represents the embedding dimension of the state representation tensor, and tanh(·) represents the hyperbolic tangent function; S34. In the action ranking structure of the IQN network, all action value estimates are averaged to obtain a quantile mean vector, which is used to sort the action priorities.

8. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 7, characterized in that: The embedded mapping process expression is as follows: P i,j =cos(πj·τ i )+sin(πj·τ i ); Among them, i,j Represents the quantile τ i Positional encoding value in the jth embedding dimension, τ i ∈(0,1) represents the i-th quantile, π is the constant pi, and N represents the number of sampled quantile values.

9. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The S4 specifically includes: S41. Select the action index a with the largest quantile value from the quantile mean vector * , as the response operation in the gas leakage scenario, and mapping the action to the actual control instruction μ: Among them, e represents the device number, Represents action a at time step t * The quantile mean, μ e represents the control signal strength of device number e, and γa*,e∈{0,1} represents action a * Whether to act on device e, η is the nonlinear enhancement coefficient, which controls the slope of the response curve; S42, controlling ventilation equipment, pipeline valves or alarm devices according to actual control instructions, and collecting feedback status information after the action is executed, the structure is as follows: Among them, v represents the feedback indicator number, R v Represents the same type of feedback status information of the vth indicator after the action is executed, represents the observed value of the v-th indicator before the action is executed, represents the observation of the vth indicator after the action is executed, τ represents the delay time of the current control step, ρ v is the sensitivity factor of the vth index to time delay, δ t represents the actual time delay from the action being sent to the feedback being collected, and log2(·) represents the logarithmic function; S43. Encode all feedback status information to form a feedback record vector R, which serves as a component element of storage entries in the experience buffer pool.

10. The method for intelligent control of gas leakage risk based on reinforcement learning according to claim 1, characterized in that: The S5 specifically includes: S51, bind the feedback record vector R with the state vector sequence S, construct a sample tuple, write it into the error-sorted experience buffer pool, and assign an initial priority weight ω to each sample i : in, represents the mean value of the same type of feedback indicators in the i-th historical sample, Represents action a at time step t * The quantile mean of represents the historical action value estimate at time step t, κ represents the error fusion coefficient; S52, from the experience cache pool according to weight ω i Sorting is performed, and the top K samples are selected using the Top-K sampling method to form a batch set for gradient calculation and parameter optimization of the IQN network; S53, use the segmented Huber loss function to perform error calculation and back propagation update on the IQN network, and set the target quantile value to Define the loss function under sample b and quantile i as in, Action a in the IQN network (b) The valuation at the i-th quantile, δ is the Huber loss segmentation threshold; S54, with For the loss function, batch gradient updates of IQN network parameters are performed to continuously optimize the estimation accuracy of the action value distribution function.

Images