An infrared natural gas leakage video detection method based on TSDNet

Abstract

The present application belongs to the field of gas leakage monitoring, and specifically provides an infrared natural gas leakage video detection method based on TSDNet, to solve the problem of low accuracy in existing gas leakage detection. The present application proposes an infrared natural gas leakage detection model based on TSDNet network, which can more accurately extract the leaked gas area, especially for weak gas, with the advantages of small measurement error, fast response speed, high sensitivity, accurate and reliable measurement results, and can realize high-precision, non-contact detection of gas leakage, and further realize real-time early warning. In addition, the present application uses the simulation software Ansys Fluent to simulate the method of obtaining natural gas leakage video, to obtain a large amount of accurate segmentation data, solving the problems of high difficulty in data acquisition and low accuracy in label making in the prior art. At the same time, the pre-training model trained by the simulation data is migrated to the real data for secondary training, further improving the detection performance of the model.

Description

Technical Field

[0001] This invention belongs to the field of gas leak monitoring, specifically providing an infrared natural gas leak video detection method based on TSDNet. Background Technology

[0002] In recent years, industrial emissions and accidental leaks have had an increasingly serious impact on air quality and public health. As industrial development places higher demands on safety, real-time monitoring of safety hazards has attracted widespread attention in the oil and gas industry. Flammable and explosive gas leaks are a major cause of safety accidents; therefore, it is crucial to effectively detect and warn of gas leaks in their early stages.

[0003] However, substances like natural gas are undetectable under normal lighting conditions and require infrared imaging for effective monitoring. Currently, detecting gases in infrared images faces unique challenges, especially when gas features are weak and easily masked by environmental noise. Traditional methods rely mainly on manual feature extraction and color analysis, which are effective in simple scenarios but often fail under complex dynamic conditions of smoke diffusion. In these situations, weak or diffuse smoke patterns can easily evade human detection and image processing techniques, resulting in low leak detection accuracy and response delays. Furthermore, these methods struggle to handle the transient and weak characteristics of gas plumes, further leading to severe delays or false alarms in leak detection. Summary of the Invention

[0004] The purpose of this invention is to provide an infrared natural gas leak video detection method based on TSDNet, which solves the problem of low accuracy in gas leak detection in the prior art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for detecting infrared natural gas leaks using TSDNet-based video, characterized by the following steps:

[0007] Step 1: Use Ansys Fluent fluid simulation tool to perform video simulation of natural gas leakage, obtain raw simulation video data, and create labeled images based on the raw simulation video data. At the same time, perform background fusion on the raw simulation video data to finally form a simulation data training set.

[0008] Step 2: Construct an infrared natural gas leak video detection network (TSDNet). The infrared natural gas leak video detection network (TSDNet) includes an encoder and a decoder. The encoder is composed of four convolutional combination modules connected together. The input of the first convolutional combination module is video sample data. The output features of the four convolutional combination modules are labeled as Y1, Y2, Y3, and Y4 respectively. The output features Y2, Y3, and Y4 are concatenated to obtain the video feature tensor Y. The video feature tensor Y is used as the output of the encoder and sent to the decoder.

[0009] The decoder consists of a first linear transformation layer, a matrix factorization module, a second linear transformation layer, and an output layer connected together.

[0010] Step 3: Construct the loss function and pre-train the infrared natural gas leak video detection network using the simulation data training set. After training, the infrared natural gas leak video detection pre-trained model is obtained.

[0011] Step 4: Collect real video data of natural gas leaks and use Labelme to annotate at the pixel level to obtain labels, thereby forming a real data training set; use the same loss function to retrain the infrared natural gas leak video detection pre-trained model, and obtain the infrared natural gas leak video detection model after training is completed.

[0012] Step 5: Input the video to be detected into the infrared natural gas leak video detection model, and the model will output the detection results.

[0013] Furthermore, in step 1, the specific process of video simulation of the natural gas leak is as follows:

[0014] In the Ansys Fluent fluid simulation tool, the Species Transport model was used to simulate natural gas diffusion, and the k-omega model was used to simulate near-wall turbulence. The natural gas composition was set as methane (CH4): ethane (C2H6): propane (C3H8): nitrogen (N2) = 0.86: 0.09: 0.03: 0.02. The natural gas leakage velocity at the leak point was set to vary randomly. Thus, the natural gas leakage simulation was completed, and the original simulation video data of the natural gas leakage was obtained.

[0015] Furthermore, in step 1, the specific process of creating the label image is as follows: for each frame of original simulation image data, the pixel value of the background area is marked as 0, and the pixel value of the natural gas area is marked as 1, thereby generating the corresponding single-channel label image.

[0016] Furthermore, in step 1, the specific process of background blending is as follows:

[0017] First, the original simulation image data is color-inverted to obtain a smoke image. For each pixel, the current pixel value is subtracted from 255 to obtain the color-inverted pixel value. In this process, white parts become black and black parts become white. Parts with higher gas concentration are closer to white, and parts with lower gas concentration are closer to the background color.

[0018] Then, a mask is created based on the smoke image to obtain a grayscale mask; the smoke image is converted into a grayscale image (L mode), which is the grayscale mask; the grayscale mask is generated using the color-inverted smoke image to control the transparency of the image. Each pixel value in the grayscale mask represents the transparency, and the lower the pixel value, the higher the transparency.

[0019] Finally, a grayscale mask is used to blend the smoke image with the application background image to generate a sample image.

[0020] Furthermore, in step 2, the matrix decomposition module employs Tucker decomposition.

[0021] Furthermore, in step 2, the four convolutional combination modules in the encoder are each composed of multiple 3D convolutional modules (3DMSCAN) connected together, and the number of 3D convolutional modules in the four convolutional combination modules is 3, 3, 5, and 2 respectively.

[0022] The 3D convolutional module includes: a first batch normalization module (BN), an attention mechanism module, a second batch normalization module (BN), and a downsampling module (FFN); wherein, the feature input of the 3D convolutional module is labeled as y0, the feature y0 is passed through the first batch normalization module and the attention mechanism module in sequence to obtain feature y1, feature y1 is added to feature y0 to obtain feature y2, feature y2 is passed through the second batch normalization module and the downsampling module in sequence to obtain feature y3, feature y3 is added to feature y2 to obtain feature y4, and feature y4 is used as the output of the 3D convolutional module.

[0023] Furthermore, in step 2, the attention mechanism module includes seven convolutional units, wherein the feature input of the attention mechanism module is labeled y. 01 The first convolutional unit uses Conv(1×1×1), and the feature y 01 The feature y is obtained by sequentially passing it through the first convolutional unit and GELU. 02 The second convolutional unit uses Conv(3×5×5,d), and the feature y 02 After the second convolutional unit, the feature y is obtained. 03The third convolutional unit uses Conv(3×1×7,d)+Conv(3×7×1,d), the fourth convolutional unit uses Conv(3×1×11,d)+Conv(3×11×1,d), and the fifth convolutional unit uses Conv(3×1×21,d)+Conv(3×21×1,d). The feature y... 03 After passing through the third, fourth, and fifth convolutional units respectively, the features y are obtained sequentially. 04 , feature y 05 , feature y 06 , feature y 04 , feature y 05 , feature y 06 With feature y 03 Adding them together gives the feature y 07 The sixth convolutional unit uses Conv(1×1×1), and the feature y 07 After the sixth convolutional unit, the feature y is obtained. 08 , feature y 08 With feature y 02 Multiplication yields the characteristic y 09 The seventh convolutional unit uses Conv(1×1×1), and the feature y 09 The output of the attention mechanism module is obtained after passing through the seventh convolutional unit;

[0024] The structure of the downsampling module is: Conv(1×1×1)+Conv(1×1×1,d)+GELU+Conv(1×1×1);

[0025] Conv(a×b×c) represents a convolutional layer with kernel size a×b×c, Conv(a×b×c,d) represents a depthwise separable convolutional layer with kernel size a×b×c, and GELU represents a non-linear activation function.

[0026] Furthermore, in step 3, the loss function is expressed as:

[0027] L total =β×L ce +γ×L te +δ×L sp

[0028] Among them, L total L represents the total loss function. ce L represents the cross-entropy loss function. te L represents the time consistency loss function. sp Let represent the spatial consistency loss function; β, γ, and δ represent the weighting coefficients of the corresponding loss functions, and β + γ + δ = 1;

[0029] The time consistency loss function is expressed as:

[0030]

[0031] diff i (x,y)=Pixel i+1 (x,y)-Pixel i (x,y)

[0032] Where N represents the total number of frames in the model's output video sequence, and H and W represent the height and width of each frame, respectively; Pixel i (x,y) and Pixel i+1 (x,y) represents the feature value of the i-th frame and the (i+1)-th frame in the output video sequence of the model at the same pixel position (x,y);

[0033] The spatial consistency loss function is expressed as:

[0034]

[0035] The cross-entropy loss function is:

[0036]

[0037] Where C represents the total number of categories. p represents the model's predicted probability that the pixel at pixel position (x, y) in the i-th frame of the output video sequence belongs to the j-th class. i,j (x,y) represents the label value of the pixel at pixel position (x,y) in the i-th frame of the input video sequence, indicating that it belongs to the j-th class.

[0038] Based on the above technical solution, the beneficial effects of the present invention are as follows:

[0039] This invention provides an infrared natural gas leak detection method based on TSDNet. Compared to other methods, it can more accurately extract the leaking gas area, especially for weak gases. It boasts advantages such as small measurement error, fast response speed, high sensitivity, and accurate and reliable measurement results, enabling high-precision, non-contact detection of gas leaks and thus achieving real-time early warning. Furthermore, this invention uses Ansys Fluent simulation software to obtain a large amount of accurate segmentation data from natural gas leak videos, solving problems such as high data acquisition difficulty and low label accuracy in existing technologies. Simultaneously, this invention transfers the pre-trained model trained on simulation data to real data for secondary training, further improving the model's detection performance. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the infrared natural gas leak video detection method based on TSDNet in this invention.

[0041] Figure 2 This is a schematic diagram of the structure of the infrared natural gas leak video detection network (TSDNet) in this invention.

[0042] Figure 3 This is a schematic diagram of the structure of the 3D convolution module (3DMSCAN) in this invention.

[0043] Figure 4 This is a schematic diagram illustrating the principle of synthesizing the simulation data training set in this invention.

[0044] Figure 5 This is a test result diagram of the infrared natural gas leak video detection pre-trained model on the test set in an embodiment of the present invention.

[0045] Figure 6 This is a test result diagram of the infrared natural gas leak video detection model in a real-world dataset, as shown in this embodiment of the invention. Detailed Implementation

[0046] To make the above-mentioned objectives, technical solutions and beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0047] This embodiment provides an infrared video detection method for natural gas leaks based on TSDNet, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0048] Step 1: Use Ansys Fluent fluid simulation tool to perform video simulation of natural gas leakage, obtain raw simulation video data, and create labeled images based on the raw simulation video data. At the same time, perform background fusion on the raw simulation video data to finally form a simulation data training set.

[0049] Specifically, using Ansys Fluent fluid simulation tools to simulate gas leakage mainly includes preprocessing, setting simulation parameters, running the simulation, and post-processing; in this embodiment, it is specifically as follows:

[0050] Step 1.1: Collect raw simulation video data;

[0051] In the Ansys Fluent fluid simulation tool, the Species Transport model was used to simulate natural gas diffusion, and the k-omega model was used to simulate near-wall turbulence. The natural gas composition was set as methane (CH4): ethane (C2H6): propane (C3H8): nitrogen (N2) = 0.86: 0.09: 0.03: 0.02. The natural gas leakage velocity at the leak point was set to vary randomly. Thus, the natural gas leakage simulation was completed, and the original simulation video data of the natural gas leakage was obtained.

[0052] Step 1.2: Create label images based on the original simulated video data;

[0053] For the original simulation video data generated in step 1.1, the background is pure white. Therefore, the white pixels in each frame can be directly set as background pixels, and the remaining pixels can be set as natural gas region pixels. The specific process is as follows: For each frame of original simulation image data, the pixel value of the background region is marked as 0, and the pixel value of the natural gas region is marked as 1, generating a corresponding single-channel label image. This single-channel label image is more in line with the needs of natural gas detection and can effectively distinguish between the background and gas regions, providing accurate annotations for subsequent data analysis and model training.

[0054] Step 1.3: Perform background fusion on the original simulation video data to obtain an image sample set;

[0055] For each frame of the original simulation image data, it is combined with the application background image to generate a sample image, such as... Figure 4 As shown; specifically:

[0056] First, the original simulation image data is color-inverted to obtain a smoke image. For each pixel, the current pixel value is subtracted from 255 to obtain the color-inverted pixel value. In this process, white parts become black and black parts become white. Parts with higher gas concentration are closer to white, and parts with lower gas concentration are closer to the background color.

[0057] Then, a mask is created based on the smoke image to obtain a grayscale mask; the smoke image is converted into a grayscale image (L mode), which is the grayscale mask; the grayscale mask is generated using the color-inverted smoke image to control the transparency of the image. Each pixel value in the grayscale mask represents the transparency, and the lower the pixel value, the higher the transparency.

[0058] Finally, the smoke image is blended with the background image using a grayscale mask to generate an image with an infrared natural gas effect, which is the image sample; the blending process is represented as follows:

[0059]

[0060] Where (x,y) represents pixel coordinates, B represents the blended image, m represents the grayscale mask, s represents the smoke image, and b represents the background image; that is: when the mask value is 255, the pixel values ​​of the blended image are completely the pixel values ​​of the smoke image; when the mask value is 0, the pixel values ​​of the blended image are completely the pixel values ​​of the background image; in other cases, the pixel values ​​of the blended image are a linear combination of the two.

[0061] Step 1.4: Create a simulation data training set based on the image sample set;

[0062] The image sample set was frame-sampling at 3-frame intervals to obtain 8320 natural gas leak video frames. Then, 260 video sample data were obtained by assembling every 32 consecutive frames into one video data set. The video sample data was divided into a training set and a test set. In this embodiment, the training set includes 208 video data sets and the test set includes 52 video data sets.

[0063] Step 2: Construct an infrared natural gas leak video detection network (TSDNet);

[0064] The infrared natural gas leak video detection network (TSDNet) is as follows: Figure 2 As shown, it includes an encoder and a decoder;

[0065] The encoder is composed of four convolutional combination modules connected together. The input of the first convolutional combination module is video sample data. The output features of the four convolutional combination modules are labeled as Y1, Y2, Y3, and Y4 respectively. The output features Y2, Y3, and Y4 are concatenated to obtain the video feature tensor Y. The video feature tensor Y is used as the output of the encoder and sent to the decoder.

[0066] The decoder adopts a Hamburger structure, which consists of a first linear transform layer, a matrix factorization module, a second linear transform layer, and an output layer. The matrix factorization module uses Tucker decomposition, which can more effectively process high-dimensional data. The input of the first linear transform layer is the video feature tensor Y, and the output layer outputs the segmentation result.

[0067] Furthermore, in the encoder, the four convolutional combination modules are respectively composed of different numbers of 3D convolutional modules (3DMSCAN); in this embodiment, the structural parameters of the four convolutional combination modules are shown in Table 1.

[0068] Table 1

[0069]

[0070] Furthermore, the 3D convolution module (3DMSCAN) is as follows: Figure 3 As shown, it includes: a first batch normalization module (BN), an attention mechanism module, a second batch normalization module (BN), and a downsampling module (FFN); wherein, the feature input of the 3D convolution module is labeled y0, the feature y0 is passed through the first batch normalization module and the attention mechanism module in sequence to obtain feature y1, feature y1 is added to feature y0 to obtain feature y2, feature y2 is passed through the second batch normalization module and the downsampling module in sequence to obtain feature y3, feature y3 is added to feature y2 to obtain feature y4, and feature y4 is used as the output of the 3D convolution module;

[0071] Furthermore, the attention mechanism module includes seven convolutional units, wherein the feature input of the attention mechanism module is labeled y. 01 The first convolutional unit uses Conv(1×1×1), and the feature y 01 The feature y is obtained by sequentially passing it through the first convolutional unit and GELU. 02 The second convolutional unit uses Conv(3×5×5,d), and the feature y 02 After the second convolutional unit, the feature y is obtained. 03 The third convolutional unit uses Conv(3×1×7,d)+Conv(3×7×1,d), the fourth convolutional unit uses Conv(3×1×11,d)+Conv(3×11×1,d), and the fifth convolutional unit uses Conv(3×1×21,d)+Conv(3×21×1,d). The feature y... 03 After passing through the third, fourth, and fifth convolutional units respectively, the features y are obtained sequentially. 04 , feature y 05 , feature y 06 , feature y 04 , feature y 05 , feature y 06 With feature y 03 Adding them together gives the feature y 07 The sixth convolutional unit uses Conv(1×1×1), and the feature y 07 After the sixth convolutional unit, the feature y is obtained. 08 , feature y 08 With feature y 02 Multiplication yields the characteristic y 09 The seventh convolutional unit uses Conv(1×1×1), and the feature y 09 The output of the attention mechanism module is obtained after passing through the seventh convolutional unit; Conv(a×b×c) represents a convolutional layer with a kernel size of a×b×c, Conv(a×b×c,d) represents a depthwise separable convolutional layer with a kernel size of a×b×c, and GELU represents a non-linear activation function.

[0072] The structure of the downsampling module is: Conv(1×1×1)+Conv(1×1×1,d)+GELU+Conv(1×1×1), where Conv(a×b×c) represents a convolutional layer with kernel size a×b×c, Conv(a×b×c,d) represents a depthwise separable convolutional layer with kernel size a×b×c, and GELU represents a non-linear activation function;

[0073] The encoder employs a 3D convolutional attention mechanism module, which performs 3D convolution operations in the spatial and temporal dimensions using convolution kernels of different sizes to aggregate multi-scale features and enhance the input feature map through dot product operations on the feature matrix. The shape of the video feature tensor generated by the encoder is represented as B×C×T×H×W, where B is the batch size, C is the number of channels, T is the temporal depth (number of video frames), and H and W are the spatial dimensions (height and width).

[0074] Step 3: Construct the loss function and pre-train the infrared natural gas leak video detection network using the simulation data training set. After training, the infrared natural gas leak video detection pre-trained model is obtained.

[0075] The loss function is expressed as:

[0076] L total =β×L ce +γ×L te +δ×L sp

[0077] Among them, L total L represents the total loss function. ce L represents the cross-entropy loss function. te L represents the time consistency loss function. sp Let represent the spatial consistency loss function; β, γ, and δ represent the weighting coefficients of the corresponding loss functions, and β+γ+δ=1. In this embodiment, β, γ, and δ are set to 0.8, 0.1, and 0.1, respectively.

[0078] The temporal consistency loss function is used to calculate the consistency loss between consecutive frames in a video sequence, i.e., the temporal differences between consecutive frames; using Pixel... i (x,y) and Pixel i+1 (x,y) represents the feature value at the same pixel position (x,y) between the i-th frame and the (i+1)-th frame in the output video sequence. The difference between consecutive frames is represented as diff. i (x,y), specifically:

[0079] diffi i (x,y)=Pixel i+1 (x,y)-Pixel i (x,y)

[0080] Then the time consistency loss function L te Represented as:

[0081]

[0082] Where N represents the total number of frames in the video sequence, and H and W represent the height and width of each frame, respectively;

[0083] The spatial consistency loss function aims to ensure that the outputs of spatially adjacent pixels in a video frame remain consistent, specifically expressed as:

[0084]

[0085] The cross-entropy loss function is:

[0086]

[0087] Where C represents the total number of categories. p represents the model's predicted probability that the pixel at pixel position (x, y) in the i-th frame of the output video sequence belongs to the j-th class. i,j (x,y) represents the label value of the pixel at pixel position (x,y) in the i-th frame of the input video sequence, which belongs to the j-th class.

[0088] Step 4: Collect real video data of natural gas leaks and use Labelme to annotate at the pixel level to obtain labels, thereby forming a real data training set; use the same loss function to retrain the infrared natural gas leak video detection pre-trained model, and obtain the infrared natural gas leak video detection model after training is completed.

[0089] Step 5: Input the video to be detected into the infrared natural gas leak video detection model, and the model will output the detection results.

[0090] Based on the aforementioned pre-trained infrared natural gas leak video detection model, this embodiment uses a test set for testing, and the test results are as follows: Figure 5 As shown in Table 2;

[0091] Table 2

[0092]

[0093] Furthermore, based on the aforementioned infrared natural gas leak video detection model, its generalization ability was verified using a dataset from a real-world application scenario, with the following results: Figure 6 As shown, the specific values ​​are shown in Table 3:

[0094] Table 3

[0095]

[0096] As can be seen from the above, this invention provides an infrared natural gas leak video detection method based on TSDNet. The TSDNet network can accurately identify weak gas leak signals and effectively cope with complex backgrounds and environmental noise. Test results show that the mIoU reaches 87.17% and the F1-score is 93.09% on the test set, while on the dataset of real application scenarios, the mIoU reaches 70.73% and the F1-score is 80.09%, demonstrating the high accuracy and adaptability of the model. At the same time, this invention can realize real-time detection and early warning, effectively preventing safety accidents that may be caused by natural gas leaks. Furthermore, this invention utilizes a large amount of accurately segmented data generated by the Ansys Fluent simulation tool, solving the problems of difficult data acquisition and low labeling accuracy in traditional methods. The pre-trained model based on the simulation data is then trained on real data, further improving the model performance. In addition, infrared imaging technology also allows the detection process to be performed without contact with the monitored gas, increasing the safety and convenience of operation.

[0097] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A method for detecting infrared natural gas leaks using TSDNet-based video, characterized in that, Includes the following steps: Step 1: Use Ansys Fluent fluid simulation tool to perform video simulation of natural gas leakage, obtain raw simulation video data, and create labeled images based on the raw simulation video data. At the same time, perform background fusion on the raw simulation video data to finally form a simulation data training set. Step 2: Construct the infrared natural gas leak video detection network TSDNet, which includes an encoder and a decoder. The encoder consists of four convolutional combination modules connected together. The input of the first convolutional combination module is video sample data. The output features of the four convolutional combination modules are labeled as Y1, Y2, Y3, and Y4, respectively. The output features Y2, Y3, and Y4 are concatenated to obtain the video feature tensor Y. The video feature tensor Y is used as the output of the encoder and sent to the decoder. The decoder consists of a first linear transformation layer, a matrix factorization module, a second linear transformation layer, and an output layer connected together. The matrix decomposition module employs Tucker decomposition. The encoder consists of four convolutional combination modules, each composed of multiple 3D convolutional modules connected together. The number of 3D convolutional modules in the four convolutional combination modules are 3, 3, 5, and 2 respectively. The 3D convolution module includes: a first batch normalization module, an attention mechanism module, a second batch normalization module, and a downsampling module; wherein, the feature input of the 3D convolution module is labeled as y0, feature y0 is passed through the first batch normalization module and the attention mechanism module to obtain feature y1, feature y1 is added to feature y0 to obtain feature y2, feature y2 is passed through the second batch normalization module and the downsampling module to obtain feature y3, feature y3 is added to feature y2 to obtain feature y4, and feature y4 is used as the output of the 3D convolution module; Step 3: Construct the loss function and pre-train the infrared natural gas leak video detection network using the simulation data training set. After training, the infrared natural gas leak video detection pre-trained model is obtained. Step 4: Collect real video data of natural gas leaks and use Labelme to annotate at the pixel level to obtain labels, thereby forming a real data training set; use the same loss function to retrain the infrared natural gas leak video detection pre-trained model, and obtain the infrared natural gas leak video detection model after training is completed. Step 5: Input the video to be detected into the infrared natural gas leak video detection model, and the model will output the detection results.

2. The infrared natural gas leak video detection method based on TSDNet according to claim 1, characterized in that, In step 1, the specific process of video simulation of natural gas leakage is as follows: In the Ansys Fluent fluid simulation tool, the Species Transport model was used to simulate natural gas diffusion, and the k-omega model was used to simulate near-wall turbulence. The natural gas composition was set as methane: ethane: propane: nitrogen = 0.86: 0.09: 0.03: 0.02, and the natural gas leakage velocity at the leak point was set to vary randomly. Thus, the natural gas leakage simulation was completed, and the original simulation video data of the natural gas leakage was obtained.

3. The infrared natural gas leak video detection method based on TSDNet according to claim 1, characterized in that, In step 1, the specific process of creating the label image is as follows: for each frame of original simulation image data, the pixel value of the background area is marked as 0, and the pixel value of the natural gas area is marked as 1, thus generating the corresponding single-channel label image.

4. The infrared natural gas leak video detection method based on TSDNet according to claim 1, characterized in that, In step 1, the specific process of background blending is as follows: First, the original simulation image data is color-inverted to obtain a smoke image. For each pixel, the current pixel value is subtracted from 255 to obtain the color-inverted pixel value. In this process, white parts become black and black parts become white. Parts with higher gas concentration are closer to white, and parts with lower gas concentration are closer to the background color. Then, a mask is created based on the smoke image to obtain a grayscale mask; Converting a smoke image to a grayscale image is called a grayscale mask. A grayscale mask is generated using the color-inverted smoke image to control the image's transparency. Each pixel value in the grayscale mask represents transparency, with lower pixel values ​​indicating higher transparency. Finally, a grayscale mask is used to blend the smoke image with the application background image to generate a sample image.

5. The infrared natural gas leak video detection method based on TSDNet according to claim 1, characterized in that, In step 2, the attention mechanism module includes seven convolutional units, wherein the feature input of the attention mechanism module is labeled y. 01 The first convolutional unit uses Conv(1×1×1), and the feature y 01 The feature y is obtained by sequentially passing it through the first convolutional unit and GELU. 02 The second convolutional unit uses Conv(3×5×5,d), and the feature y 02 After the second convolutional unit, the feature y is obtained. 03 The third convolutional unit uses Conv(3×1×7,d)+Conv(3×7×1,d), the fourth convolutional unit uses Conv(3×1×11,d)+Conv(3×11×1,d), and the fifth convolutional unit uses Conv(3×1×21,d)+Conv(3×21×1,d). The feature y... 03 After passing through the third, fourth, and fifth convolutional units respectively, the features y are obtained sequentially. 04 , feature y 05 , feature y 06 , feature y 04 , feature y 05 , feature y 06 With feature y 03 Adding them together gives the feature y 07 The sixth convolutional unit uses Conv(1×1×1), and the feature y 07 After the sixth convolutional unit, the feature y is obtained. 08 , feature y 08 With feature y 02 Multiplication yields the characteristic y 09 The seventh convolutional unit uses Conv(1×1×1), and the feature y 09 The output of the attention mechanism module is obtained after passing through the seventh convolutional unit; The structure of the downsampling module is: Conv(1×1×1)+Conv(1×1×1,d)+GELU+Conv(1×1×1); Conv(a×b×c) represents a convolutional layer with kernel size a×b×c, Conv(a×b×c,d) represents a depthwise separable convolutional layer with kernel size a×b×c, and GELU represents a non-linear activation function.

6. The infrared natural gas leak video detection method based on TSDNet according to claim 1, characterized in that, In step 3, the loss function is expressed as: in, Represents the total loss function. Represents the cross-entropy loss function. Represents the time consistency loss function. Represents the spatial consistency loss function; Let represent the weighting coefficients of the corresponding loss functions, and ; The time consistency loss function is expressed as: Where N represents the total number of frames in the model's output video sequence, and H and W represent the height and width of each frame, respectively; and This represents the feature value of the i-th frame and the (i+1)-th frame in the video sequence output by the model at the same pixel position (x, y); The spatial consistency loss function is expressed as: The cross-entropy loss function is: in, The total number of categories, This represents the model's predicted probability that the pixel at pixel position (x, y) in the i-th frame of the output video sequence belongs to the j-th class. This represents the label value of the pixel at pixel position (x,y) in the i-th frame of the input video sequence, indicating that it belongs to the j-th class.

Images