A gas leakage detection method based on deep learning and optical flow method

Abstract

The present invention belongs to the technical field of alkane gas leakage detection, and specifically relates to a gas leakage detection method based on deep learning and optical flow method. The present invention comprises: using a Gaussian plume model to simulate the image data of real gas leakage and produce a data set; using a deep learning network YOLOv8‑seg model for training; using a mid-infrared camera to monitor the area where leakage may occur; when a leakage occurs, detecting and segmenting the leakage gas area to generate a binary segmentation mask of the leakage gas; using the optical flow method to analyze the movement direction of the gas plume in the binary segmentation mask. To solve the problem that it is difficult to find the leakage point in the existing gas detection means, the present invention can more accurately track the diffusion trajectory of the leaked gas and achieve rapid tracing.

Description

Technical Field

[0001] The present invention relates to the technical field of gas monitoring, and in particular to a gas leakage detection method based on deep learning and optical flow method. Background Art

[0002] With the rapid development of the industrial and energy sectors, the use and production of various types of gases are also increasing. Gas leakage accidents occur from time to time, especially in high-risk industries such as petrochemicals, natural gas transportation and storage. Dangerous gas leaks not only cause economic losses, but also pose a major threat to the environment and public safety. Traditional gas leak detection methods mainly rely on manual detection and fixed sensors, which have many shortcomings in practical applications. Manual detection is inefficient and relies on manual inspections, making it difficult to detect leaks in a timely manner. Fixed sensors are limited by their installation location and number, with limited coverage, making it difficult to fully monitor the entire system. At the same time, some gases are colorless and odorless, making them difficult to detect by conventional means, further increasing the difficulty of leak detection. Therefore, the development of an efficient and accurate gas leak detection technology has become an urgent need in the field of industrial safety. Summary of the invention

[0003] The present invention provides a gas leakage detection method based on deep learning and optical flow method, which solves the problem of difficulty in finding leakage points in existing gas detection methods.

[0004] A gas leakage detection method based on deep learning and optical flow method comprises the following steps:

[0005] S1 uses the Gaussian plume model to simulate the image data of real gas leakage and create a data set:

[0006] ,

[0007] in Represents three-dimensional coordinates The leakage gas concentration at is the effective height of the leak source, Q is the release rate, represents the average wind speed, is the diffusion coefficient in the lateral direction, is the diffusion coefficient in the vertical direction;

[0008] S2 uses the deep learning network YOLOv8-seg model for training, extracts spatial features, and segments the gas diffusion area;

[0009] The S3 uses a mid-infrared camera to monitor the area in need of monitoring and capture images when a leak occurs:

[0010] S4 inputs the image when the leakage occurs into the deep learning network YOLOv8-seg model, detects and segments the leaking gas area, and generates a binary segmentation mask of the leaking gas;

[0011] S5 uses the optical flow method to analyze the movement direction of the gas plume in the binary segmentation mask, and traces the diffusion path of the gas by calculating the optical flow field between multiple consecutive image frames, and infers the initial position of the leak backwards.

[0012] Furthermore, after capturing the image when leakage occurs in step S3, preprocessing is performed to enhance the image quality. The preprocessing formula is:

[0013] ,

[0014] in, is the original image, is the filtered image, is the standard deviation of the filter, is the two-dimensional coordinate of the center point of the filter in the image, are the two-dimensional coordinates in the image.

[0015] Furthermore, the optical flow method is used to analyze the movement direction of the gas plume in the binary segmentation mask, including: calculating the a gradient in the horizontal and vertical directions of the image as:

[0016] ,

[0017] ,

[0018] is the horizontal a gradient, is the vertical a gradient, which represents the brightness change rate of the image in the horizontal and vertical directions respectively. The motion state is derived by combining the horizontal a gradient and vertical a gradient of the image using the optical flow constraint equation. The optical flow constraint equation is expressed as:

[0019] ,

[0020] in, Indicates that the image is The velocity component in the direction, Indicates that the image is Directional velocity component.

[0021] Furthermore, by calculating the optical flow field between multiple consecutive image frames, the diffusion path of the gas is tracked and the initial location of the leak is inferred backwards, including:

[0022] Set the objective function;

[0023] By minimizing the objective function, the optical flow field is obtained;

[0024] Calculate the optical flow field for each pixel to obtain the optical flow vector field;

[0025] Performing threshold processing on the optical flow vector field to obtain the optical flow vector field after threshold processing;

[0026] According to the optical flow vector field after threshold processing, the movement trajectory of the gas is tracked and the source point of the gas is determined.

[0027] Furthermore, the objective function adopted is:

[0028] ,

[0029] is the weight of the smoothness term in the optical flow method, is the objective function of the optical flow method, represents the error of the optical flow constraint equation, It is the smoothness of the optical flow field, which controls the smoothness of the velocity between adjacent pixels;

[0030] By minimizing the objective function, we get the optical flow field, that is, the image in The velocity component in the direction and Directional velocity component.

[0031] Furthermore, the optical flow vector field is calculated as:

[0032] ,

[0033] To represent the pixel The optical flow vector field at , that is, the displacement of the pixel point on the image plane between the two frames.

[0034] Furthermore, the threshold processing is:

[0035] ,

[0036] in, is the optical flow vector field after threshold processing. is the threshold of the optical flow vector, that is, when the size of the optical flow vector exceeds a certain threshold , the optical flow vector is retained, otherwise it is set to zero.

[0037] Furthermore, the traceability points of the gas include:

[0038] ,

[0039] in, Indicates the location of the leak. Indicates the coordinates of the pixel points traced in the traceability analysis. Indicates at location The optical flow vector at is used to find the source of the gas leak, that is, the point with the smallest motion vector.

[0040] The present invention has the following beneficial effects:

[0041] Traditional sensors are often point-type or line-type detections, which can only cover a limited spatial range and are prone to missed detection or misjudgment. The present invention uses a mid-infrared camera through its wide field of view and imaging capabilities to achieve comprehensive monitoring of the surface source area, can generate infrared images in real time, and output them in the form of a video stream, providing operators with an intuitive and dynamic display of the leakage situation. The present invention can accurately segment the leaking gas area at the pixel level, and can handle complex image scenes, such as multi-source leakage, interference from obstructions, etc., to provide clear and accurate input data for subsequent source tracing detection. In the source tracing detection stage, the present invention innovatively combines the optical flow method with deep learning technology. By training a deep learning model to optimize the results of the optical flow calculation, the diffusion trajectory of the leaking gas can be more accurately tracked to achieve rapid source tracing. The realization of the present invention marks an important step in the automation and intelligent direction of alkane gas leakage detection. Automated monitoring and real-time analysis reduce manual intervention, improve detection efficiency, and reduce personnel safety risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 A schematic flow chart of a gas leakage detection method based on deep learning and optical flow method provided in an embodiment of the present invention;

[0044] Figure 2 This is a model principle diagram of a gas leakage detection method based on deep learning and optical flow method of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only part of the embodiments involved in the present invention. All non-innovative embodiments based on this embodiment by other researchers in the field belong to the protection scope of the present invention. Figure 1-2 , the specific working mode of the present invention is further described as follows:

[0046] See also Figure 1 As shown, a gas leakage detection method based on deep learning and optical flow method includes:

[0047] S1 uses the Gaussian plume model to generate gas leakage images in various scenarios and create data sets:

[0048] ,

[0049] in Represented in three-dimensional coordinates is the leakage gas concentration , is the effective height of the leak source ( ), Q is the release rate , Indicates average wind speed , is the diffusion coefficient in the lateral direction , is the diffusion coefficient in the vertical direction .

[0050] S2 uses the deep learning network YOLOv8-seg model for training, extracts spatial features, and segments the gas diffusion area; the process of using the deep learning network YOLOv8-seg model for training is to establish a gas leak detection model. Through training, it can automatically learn and extract spatial features in the image, thereby improving the accuracy and robustness of detection.

[0051] The S3 uses a mid-infrared camera to monitor areas where leaks may occur, capturing images of leaks as they occur:

[0052] S4 inputs the image when the leakage occurs into the deep learning network YOLOv8-seg model to detect and segment the leaking gas area and generate a binary segmentation mask of the leaking gas.

[0053] S5 uses the optical flow method to analyze the movement direction of the gas plume in a two-dimensional segmentation mask. By calculating the optical flow field between multiple consecutive image frames, it tracks the diffusion path of the gas and infers the initial location of the leak backwards.

[0054] After capturing the image when the leakage occurs in step S3, preprocessing is performed to enhance the image quality:

[0055] ,

[0056] in, is the original image, is the filtered image, is the standard deviation of the filter, is the two-dimensional coordinate of the center point of the filter in the image, is the two-dimensional coordinate in the image;

[0057] Calculate the horizontal and vertical gradients of the image:

[0058] ,

[0059] ,

[0060] is the horizontal a gradient, is the vertical a gradient, which represents the brightness change rate of the image in the horizontal and vertical directions respectively. The optical flow constraint equation is used to deduce the motion state through the horizontal a gradient and vertical a gradient of the image. The optical flow constraint equation is expressed as:

[0061] ,

[0062] in, Indicates that the image is The velocity component in the direction, Indicates that the image is Directional velocity component.

[0063] In step S4, the deep learning network YOLOv8-seg model is used to extract the deep features of the input image, and the input image is a grayscale image. , the deep learning network YOLOv8-seg model generates a binary segmentation mask ,in Indicates Whether the pixel belongs to the gas leakage area. Binary segmentation mask It can be expressed as:

[0064] ,

[0065] in, is the partition function.

[0066] Since there is only one category (leaking gas), the mask confidence It indicates that the deep learning network YOLOv8-seg model predicts the probability of each pixel being a gas leak area. The calculation of mask confidence is simplified to a two-classification problem: ,

[0067] in, Represents the probability. Since there is only one category of leaking gas, the classification loss is simplified to a binary classification problem. The binary cross entropy loss is used to measure the difference between the model's prediction and the actual category. If a region contains leaking gas, the classification function is defined as:

[0068] ,

[0069] in, is the true label of the smoke category. If an area contains leaking gas, then ,otherwise . is the probability that the model predicts that it belongs to the gas category. For leaking gas, Represents the true category label.

[0070] Bounding box prediction of the deep learning network YOLOv8-seg model for single-category gas leak detection. Bounding box regression loss function Combined with intersection-and-union , center point distance and aspect ratio. The real bounding box , the predicted bounding box . Regression box loss Defined as:

[0071] ,

[0072] Bounding Box Regression Loss Function Expressed as:

[0073] ,

[0074] in, is the intersection-over-union ratio of the true bounding box and the predicted bounding box, is the Euclidean distance between the center points of the two boxes, is the diagonal length of the smallest enclosing box that can contain both boxes, and Used to correct aspect ratio.

[0075] Since there is only one category of leaking gas, the segmentation loss function directly measures the difference between the predicted binary segmentation mask and the true mask. The segmentation loss function is defined as:

[0076] ,

[0077] in, is the total number of pixels, represents the true label, is the real mask The value of a pixel (0 or 1), is a binary segmentation mask, i.e., the predicted The probability that a pixel belongs to the leakage area.

[0078] The total loss function is defined as:

[0079] ,

[0080] in, is the classification loss function, is the regression box loss function, is the segmentation loss function. and is a weight hyperparameter used to balance the losses of each part.

[0081] After model training, the infrared image is processed and analyzed to determine the characteristic points of gas leakage. Characteristic points refer to key pixel points in the image that can represent the location of gas leakage. By identifying characteristic points, the gas leakage area can be preliminarily located.

[0082] The collected grayscale image is input into the deep learning network YOLOv8-seg model. The grayscale image first passes through the backbone network of the deep learning network YOLOv8-seg model. The main task of the backbone network is to extract features related to the morphology of the leaked gas, such as the shape of the gas plume, concentration changes, etc. The backbone network uses the C2f (Cross-convolution with 2 filters) module. The C2f module uses two convolution kernels of different sizes (for example, 1x1 and 3x3) to replace the traditional 3x3 convolution kernel, which increases the network's receptive field and reduces the amount of calculation. Then it passes through the neck network. For gas leak detection, the neck network helps to integrate multi-scale information, so that the deep learning network YOLOv8-seg model can detect gas leakage areas of different scales, whether they are small leaks or larger diffused gas masses. Finally, the detection head is the output part of the deep learning network YOLOv8-seg model, which is responsible for predicting the specific location and morphology of the leaked gas and accurately marking the outline of the leaked gas through pixel segmentation.

[0083] S5 Step S5 specifically includes:

[0084] Set the objective function;

[0085] By minimizing the objective function, the optical flow field is obtained;

[0086] Calculate the optical flow field for each pixel to obtain the optical flow vector field;

[0087] Performing threshold processing on the optical flow vector field to obtain the optical flow vector field after threshold processing;

[0088] According to the optical flow vector field after threshold processing, the movement trajectory of the gas is tracked and the source point of the gas is determined.

[0089] The present invention uses dense optical flow method to calculate the pixel motion vector between the previous and next frames to obtain the motion field. The objective function of sampling is:

[0090] ,

[0091] is the weight of the smoothness term in the optical flow method, is the objective function of the optical flow method, represents the error of the optical flow constraint equation, It is the smoothness of the optical flow field, which controls the smoothness of the velocity between adjacent pixels;

[0092] By minimizing the objective function, we get the optical flow field, that is, the image in Velocity component in direction and Velocity component in direction .

[0093] Calculate the optical flow field for each pixel to obtain the optical flow vector field:

[0094] ,

[0095] To represent the pixel The optical flow vector field at , which is a two-dimensional vector, contains horizontal velocity components and vertical velocity components. The optical flow vector field represents the displacement or movement direction and magnitude of the pixel point on the image plane between two frames.

[0096] Thresholding:

[0097] ,

[0098] in, is the optical flow vector field after threshold processing. is the threshold of the motion vector, that is, when the size of the optical flow vector exceeds a certain threshold , keep the vector, otherwise set it to zero. This is used to exclude small noise or insignificant motion.

[0099] The optical flow method is used to analyze the infrared image sequence. By analyzing the pixel movement in the image, the movement trajectory of the gas is tracked to determine the source point of the gas:

[0100] ,

[0101] in, Indicates the location of the leak. Indicates the coordinates of the pixel points traced in the traceability analysis. Indicates at location The optical flow vector at , which is used to find the source of the gas leak (i.e. the point with the smallest motion vector). Maybe in all In the example, a certain position in the optical flow field obtained by analyzing the motion field is the point in the entire optical flow field that is most likely to point to the leakage source.

[0102] See also Figure 2 As shown in the figure, the gas leakage area segmented by the deep learning network YOLOv8-seg model and the result of optical flow tracing are combined to determine that the leakage has indeed occurred and trigger an alarm. At this time, a warning signal will be sent to the operator and further emergency measures will be taken.

Claims

1. A gas leakage detection method based on deep learning and optical flow method, characterized in that: The following steps are involved: S1 uses the Gaussian plume model to simulate the image data of real gas leakage and create a data set: ， in Represents three-dimensional coordinates The leakage gas concentration at is the effective height of the leak source, Q is the release rate, represents the average wind speed, is the diffusion coefficient in the lateral direction, is the diffusion coefficient in the vertical direction; S2 uses the deep learning network YOLOv8-seg model for training, extracts spatial features, and segments the gas diffusion area; The S3 uses a mid-infrared camera to monitor the area in need of monitoring and capture images when a leak occurs: S4 inputs the image when the leakage occurs into the deep learning network YOLOv8-seg model, detects and segments the leaking gas area, and generates a binary segmentation mask of the leaking gas; S5 uses the optical flow method to analyze the movement direction of the gas plume in the binary segmentation mask, and traces the diffusion path of the gas by calculating the optical flow field between multiple consecutive image frames, and infers the initial position of the leak backwards; The optical flow method is used to analyze the movement direction of the gas plume in the binary segmentation mask, including: calculating the a gradient in the horizontal and vertical directions of the image as: ， ， is the filtered image, is the horizontal a gradient, is the vertical a gradient, which represents the brightness change rate of the image in the horizontal and vertical directions respectively. The motion state is derived by combining the horizontal a gradient and vertical a gradient of the image using the optical flow constraint equation. The optical flow constraint equation is expressed as: ， in, Indicates that the image is The velocity component in the direction, Indicates that the image is Directional velocity component; By calculating the optical flow field between multiple consecutive image frames, the diffusion path of the gas is tracked and the initial location of the leak is inferred backwards, including: Set the objective function; By minimizing the objective function, the optical flow field is obtained; Calculate the optical flow field for each pixel to obtain the optical flow vector field; Performing threshold processing on the optical flow vector field to obtain the optical flow vector field after threshold processing; According to the optical flow vector field after threshold processing, the movement trajectory of the gas is tracked and the source point of the gas is determined; The objective function used is: ， is the weight of the smoothness term in the optical flow method, is the objective function of the optical flow method, represents the error of the optical flow constraint equation, It is the smoothness of the optical flow field, which controls the smoothness of the velocity between adjacent pixels; By minimizing the objective function, we get the optical flow field, that is, the image in The velocity component in the direction and Directional velocity component; The optical flow vector field is calculated as: ， To represent the pixel The optical flow vector field at , that is, the displacement of the pixel point on the image plane between the two frames; The threshold processing is: ， in, is the optical flow vector field after threshold processing. is the threshold of the optical flow vector, that is, when the size of the optical flow vector exceeds a certain threshold , the optical flow vector is retained, otherwise it is set to zero.

2. The gas leakage detection method based on deep learning and optical flow method according to claim 1 is characterized in that: After capturing the image when leakage occurs in step S3, preprocessing is performed to enhance the image quality. The preprocessing formula is: ， in, is the original image, is the filtered image, is the standard deviation of the filter, is the two-dimensional coordinate of the center point of the filter in the image, are the two-dimensional coordinates in the image.

3. The gas leakage detection method based on deep learning and optical flow method according to claim 1, characterized in that: The traceability points for determining the gas include: ， in, Indicates the location of the leak. Indicates the coordinates of the pixel points traced in the traceability analysis. Indicates at location The optical flow vector at is used to find the source of the gas leak, that is, the point with the smallest motion vector.

Images