A method for detecting a leaking gas from an infrared video

Abstract

The application discloses a kind of gas leakage detection methods for infrared video, comprising the following steps: obtaining infrared video;Infrared video is smoothed using Gaussian filter;The first frame in the processed infrared video is taken as background image, and the remaining frame images are subtracted from the background image respectively, to obtain difference image and be converted into binary image respectively;Delete connected region, then determine candidate gas region;Randomly distributed points are generated inside and boundary of candidate gas region, to determine the center of candidate gas region;The center of candidate gas region is connected with the boundary point of candidate gas region respectively, to calculate the difference between the pixel gray value of adjacent two points on the corresponding frame image I i of connecting line, and the leakage gas region in all binary images is obtained by the sum of difference value.The gas leakage detection method for infrared video of the application is convenient to use, reduces cost, realizes rapid detection, and guarantees detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and gas detection, and particularly relates to a leakage gas detection method for infrared video. BACKGROUND

[0002] Gas leakage refers to the leakage of colorless, odorless and invisible gases in the air, such as hydrogen, methane, nitrogen and oxygen. These gas leakages are difficult to be detected by people, and can easily spread and accumulate in the air, forming dangerous situations such as explosion and poisoning, which can bring great threat and harm to personnel and environment. The significance of leakage gas detection is to timely discover, locate and handle gas leakage, and avoid potential harm and loss.

[0003] The existing leakage gas detection methods for infrared video mainly include inter-frame difference method, optical flow method and machine learning. The inter-frame difference method determines the leakage gas region according to a threshold value, but the detection effect of this method is affected by the size of the threshold value, which can easily lead to false detection or missed detection, thereby affecting the accuracy of gas leakage detection. The optical flow method can be used to detect the motion trajectory of the leakage gas, but it may cause false detection when the light intensity changes greatly or there is complex background interference. At the same time, the calculation amount of this method is large, and the requirement for computing resources is high. The use of machine learning for leakage gas detection requires a large amount of data for training to establish a leakage gas detection model. If the data quantity is insufficient or the data quality is not high, the model effect may be poor. In addition, the target and background contrast of infrared image is low, and the signal-to-noise ratio is low, the edge is fuzzy, and other characteristics also increase the detection difficulty. SUMMARY

[0004] In view of the above defects of the prior art, the technical problems to be solved by the present application are the deficiencies of the existing leakage gas detection methods, including overcoming the influence of improper threshold value selection, large light intensity change, background interference and other factors on the detection effect, and the problem of requiring a large amount of data for training. Therefore, the present application provides a leakage gas detection method for infrared video, which has the advantages of easy use, rapidity, high precision and low cost.

[0005] To achieve the above purpose, the present application provides a leakage gas detection method for infrared video, comprising the following steps:

[0006] Step 1, acquiring infrared video of a monitoring scene by using an infrared thermal imager;

[0007] Step 2, performing Gaussian filtering on each frame image I i Step 3, performing smoothing processing on the image I

[0008] Step 3, take the first frame of the smoothed infrared video as the background image, and subtract each of the remaining frames from the background image to obtain a difference image;

[0009] Step 4, convert each difference image into a binary image according to the threshold T1;

[0010] Step 5, delete the connected regions in the binary image whose area is less than the threshold T2 or whose aspect ratio exceeds the threshold T3;

[0011] Step 6, calculate the intersection-over-union between the connected region C i in the current frame and the connected region C i+1 in the corresponding position of the next frame, if the intersection-over-union is greater than the threshold T4, then C i is taken as a candidate gas region;

[0012] Step 7, generate randomly distributed points inside and on the boundary of the candidate gas region, generate a minimum spanning tree according to these points, and further obtain the tree barycenter, which is taken as the center of the candidate gas region;

[0013] Step 8, draw a line between the center of the candidate gas region and each boundary point of the candidate gas region, calculate the difference between the pixel gray values of the adjacent two points on the line in the corresponding frame image I i , if the difference is greater than 0, then mark it as 1, otherwise mark it as 0, and add the 0 or 1 values on the line to obtain ∑ i , add all the ∑ i obtained on all lines, if the sum is greater than the threshold T5, then mark the candidate gas region as a leakage gas region, here the threshold T5 can be taken as 0.7 times the sum of the number of adjacent point pairs on all lines;

[0014] Step 9, traverse all binary images in the infrared video, including all binary images between the second frame and the second-to-last frame, and repeat steps 5-8 to obtain the leakage gas regions in all binary images.

[0015] Further, the infrared video of the monitoring scene is obtained by using the infrared thermal imager, specifically including:

[0016] First, prepare the infrared thermal imager device and set it to video capture mode;

[0017] Second, aim the infrared thermal imager at the scene to be monitored, and start the device to begin capturing infrared video;

[0018] Then, connect and transmit the captured infrared video to the computer through the data interface of the device, and save the captured infrared video to the local hard disk of the computer for subsequent use.

[0019] Further, for each frame image I in the infrared video i The smoothing is performed by using Gaussian filtering, specifically including:

[0020] Firstly, for each frame image I in the infrared video i is converted into a gray image;

[0021] Secondly, a Gaussian kernel G is defined

[0022] Then, the center of G is aligned to each pixel point in the image, and the pixel values corresponding to each position in G are weighted and averaged according to the weights of the positions, i.e. the smoothed pixel values.

[0023] Further, each difference image is converted into a binary image according to a threshold T1, specifically including:

[0024] For each difference image, the absolute value of the difference value is calculated, and the pixel points with the absolute value of the difference value less than the threshold T1 are set to 0, and the remaining pixel points are set to 1, so as to convert the difference image into a binary image; wherein the pixel points with a value of 1 in the binary image are taken as foreground pixel points, and the pixel points with a value of 0 are taken as background pixel points.

[0025] Further, the connected regions in the binary image with an area less than a threshold T2 or an aspect ratio exceeding a threshold T3 are deleted, specifically including:

[0026] Firstly, starting from a foreground pixel point in the binary image, adjacent foreground pixel points are scanned along eight directions, and are marked as the same connected region, and adjacent unmarked pixel points are added to the queue to continue to expand the connected region until the queue is empty, and the process is repeated until the entire image is traversed, to obtain all connected regions in the binary image;

[0027] Secondly, the number of all pixel points in each connected region in the binary image is counted, i.e. the area of the connected region, and the connected regions with an area less than the threshold T2 are deleted, i.e. the pixel values in these connected regions with smaller areas are set to 0;

[0028] Then, for each connected region in the binary image, the minimum and maximum values of the horizontal coordinates of all points in the region are calculated, as well as the minimum and maximum values of the vertical coordinates of all points in the region, to determine the minimum bounding box of the region according to the above four values, and the aspect ratio of the minimum bounding box is taken as the aspect ratio of the connected region, and if the aspect ratio exceeds the threshold T3, the connected region is deleted, i.e. the pixel values in the region are set to 0.

[0029] Further, the connected region C i in the current frame is calculated i+1the intersection-over-union between C i As a candidate gas region, specifically includes:

[0030] First, according to the connected region C i in the current frame, find the corresponding position in the next frame, if there is no connected region C i+1 in the corresponding position, record the intersection-over-union as 0;

[0031] Secondly, if there is a connected region C i+1 in the corresponding position in the next frame, calculate , that is, respectively count the number of intersection pixels and the number of union pixels between C i and C i+1 , and calculate the ratio of the two;

[0032] If the intersection-over-union IoU is greater than a threshold T4, C i is taken as a candidate gas region.

[0033] Further, randomly distributed points are generated inside and on the boundary of the candidate gas region, and a minimum spanning tree is generated according to these points, and a tree barycenter is further obtained, which is taken as the center of the candidate gas region, specifically including:

[0034] First, traverse each pixel inside the candidate gas region, if the adjacent pixels of the pixel are all in the candidate gas region, the pixel is an internal point, otherwise it is a boundary point, at the same time, all internal points are listed in an internal point pixel column with a length of N i , and all boundary points are listed in a boundary point pixel column with a length of N j ;

[0035] Next, 0.3N i random numbers in the interval [1, N i ] are generated, and the pixels at the corresponding positions in the internal point pixel column are obtained according to the random numbers, so as to obtain the randomly distributed points inside the candidate gas region;

[0036] Then, 0.3N j random numbers in the interval [1, N j ] are generated, and the pixels at the corresponding positions in the boundary point pixel column are obtained according to the random numbers, so as to obtain the randomly distributed points on the boundary of the candidate gas region;

[0037] Finally, according to the randomly distributed points inside and on the boundary of the candidate gas region, a minimum spanning tree is generated by using the KRUSKAL algorithm, and each node in the minimum spanning tree is traversed, the number of nodes contained in the subtree with the node as the root node is calculated, and a node that makes the maximum subtree in all subtrees minimum is found, which is the tree barycenter of the minimum spanning tree, and the tree barycenter is taken as the center of the candidate gas region.

[0038] Further, the KRUSKAL algorithm is used to generate the minimum spanning tree, and the steps are as follows:

[0039] All edges are sorted in non-decreasing order of weight;

[0040] The smallest edge is selected, and it is checked whether it forms a cycle with the spanning tree formed so far, if not, the edge is included, otherwise, it is discarded;

[0041] The step above is repeated until all other edges in the spanning tree.

[0042] Further, the center of the candidate gas region is connected with the boundary points of the candidate gas region respectively, and the difference between the pixel gray values of the adjacent two points on the connecting line in the corresponding frame image I i is calculated, if the difference is greater than 0, it is recorded as 1, otherwise, it is recorded as 0, and the sum of the 0 or 1 values on the connecting line is obtained ∑ i , all the ∑ i obtained on all the connecting lines are added, if the addition result is greater than the threshold T5, the candidate gas region is marked as a leakage gas region, here, the threshold T5 can be 0.7 times the sum of the number of adjacent point pairs on all the connecting lines, and specifically includes:

[0043] First, the center of the candidate gas region is connected with the boundary points of the candidate gas region one by one, and the difference between the pixel gray values of the adjacent two points on the connecting line in the corresponding frame image I i is calculated, if the difference is greater than 0, it is recorded as 1, otherwise, it is recorded as 0, here, the frame image I i refers to the i-th frame gray image in the infrared video processed by step 2;

[0044] Then, the sum of the 0 or 1 values on the connecting line is obtained ∑ i , all the ∑ i obtained on all the connecting lines are added, if the addition result is greater than the threshold T5, the candidate gas region is marked as a leakage gas region, here, the threshold T5 can be 0.7 times the sum of the number of adjacent point pairs on all the connecting lines.

[0045] Further, each frame image in the infrared video is a three-channel color image I i including red R, green G and blue B, I i is converted into a gray image I Gray , which is completed by I Gray =R*0.299+G*0.587+B*0.114.

[0046] Technical effects

[0047] The leakage gas detection method for infrared video of the present application obtains a binary image through inter-frame difference, and marks a leakage gas area according to the binary image, thereby avoiding a large amount of data marking and training process, being convenient to use, reducing cost, and realizing rapid detection; candidate gas areas are selected by analyzing the area and shape of a connected area and the shape change between adjacent frames, thereby ensuring detection accuracy; the leakage gas area is determined according to the pixel gray scale distribution between the center point and the boundary point of the candidate gas area, thereby further ensuring detection accuracy.

[0048] The concept, specific structure and generated technical effects of the present application will be further described below in combination with the drawings, so as to fully understand the purposes, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0050] Figure 2 is a schematic diagram of connected areas and minimum bounding boxes of a current frame and a next frame of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0051] Figure 3 is a schematic diagram of intersection and union ratio of connected areas of a current frame and a next frame of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0052] Figure 4 is a schematic diagram of a candidate gas area of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0053] Figure 5 is a schematic diagram of a minimum spanning tree generated according to 12 points of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0054] Figure 6 is a schematic diagram of leakage gas detection results of 12 continuous frames of images of a leakage gas detection method for infrared video of a preferred embodiment of the present application;

[0055] Figure 7 is a schematic diagram of leakage gas detection results of 12 continuous frames of images containing interference factors of a leakage gas detection method for infrared video of a preferred embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the technical problems, technical solutions and beneficial effects of the present application clearer, the present application will be further described in detail below in conjunction with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0057] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular procedures, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0058] As shown in Figure 1 , the embodiment of the present application provides a kind of gas detection method for infrared video, comprising the following steps:

[0059] Step 1, infrared video of monitoring scene is obtained using infrared thermal imager, and infrared video is transmitted to computer by corresponding data interface;

[0060] Step 2, each frame image I i In the infrared video is smoothed using Gaussian filter;

[0061] Step 3, the first frame in the infrared video after smoothing is taken as background image, and the remaining frame images are respectively subtracted from the background image, to obtain difference image;

[0062] Step 4, each difference image is converted into binary image according to threshold T1;

[0063] Step 5, delete the connected region in binary image whose area is less than threshold T2, or whose aspect ratio exceeds threshold T3;

[0064] Step 6, the intersection-over-union between connected region C i In the current frame and the connected region C i+1 Corresponding position of next frame is calculated, if the intersection-over-union is greater than threshold T4, then C i It is taken as candidate gas region;

[0065] Step 7, random distribution points are generated inside and on the boundary of candidate gas region, minimum spanning tree is generated according to these points, further to obtain tree barycenter, and the tree barycenter is taken as the center of the candidate gas region;

[0066] Step 8, the center of the candidate gas region is connected with the boundary point of the candidate gas region respectively, and the intersection-over-union between adjacent two points on the connecting line in corresponding frame image I iThe difference between the pixel gray values on the line, if the difference is greater than 0, record as 1, otherwise record as 0, add the 0 or 1 values on the line to obtain ∑ i Add all the ∑ obtained on the line i If the addition result is greater than the threshold T5, mark the candidate gas region as a leakage gas region, here, the threshold T5 can take 0.7 times the sum of the number of adjacent point pairs on all lines;

[0067] Step 9, traverse all binary images in the infrared video, including all binary images between the 2nd frame and the 2nd last frame (the 1st last frame is the last frame), repeat steps 5-8 to obtain the leakage gas region in all binary images.

[0068] The specific steps of a leakage gas detection method for infrared video according to the present application will be illustrated as follows.

[0069] Step 1, use an infrared thermal imager to obtain an infrared video of a monitoring scene, and transmit the infrared video to a computer through a corresponding data interface, specifically including:

[0070] First, prepare the infrared thermal imager device and set it to video capture mode;

[0071] Second, aim the infrared thermal imager at the scene to be monitored, and start the device to begin capturing infrared video;

[0072] Then, connect and transmit the captured infrared video to the computer through the data interface of the device, and save the captured infrared video to the local hard disk of the computer for subsequent use.

[0073] When selecting the scene to be monitored, attention should be paid to avoid factors that may affect the performance of the infrared thermal imager, such as strong light and reflection. When transmitting the infrared video to the computer, use the correct data line to connect the computer. Different devices may require different types of data lines, such as USB, HDMI, etc.

[0074] Step 2, for each frame image I i smoothed using Gaussian filtering, specifically including:

[0075] First, convert each frame image I i in the infrared video to a grayscale image;

[0076] Second, define a Gaussian kernel

[0077] Then, align the center of G to each pixel point in the image, and perform weighted average on the corresponding pixel values according to the weights of each position in G, which is the smoothed pixel value.

[0078] Here, each frame image in the infrared video is a three-channel color image I including red R, green G and blue B i , I is converted into a gray image I i Gray I can be converted by I Gray = R*0.299 + G*0.587 + B*0.114.

[0079] I is obtained Gray , and each pixel in I Gray is weightedly averaged with the weight of the corresponding position in the Gaussian kernel. For example, the eight neighborhood pixels of P0 are:

[0080]

[0081] The value of P0 after Gaussian smoothing is equal to 0.0778*P1 + 0.1233*P2 + 0.0778*P3 + 0.1233*P4 + 0.1946*P0 + 0.1233*P5 + 0.0778*P6 + 0.1233*P7 + 0.0778*P8.

[0082] Step 3, the first frame in the infrared video after smoothing is taken as a background image, and each of the remaining frames is subtracted from the background image to obtain a difference image, which specifically includes:

[0083] First, the first frame image in the infrared video after smoothing is taken as a background image.

[0084] Secondly, each of the remaining frames in the infrared video is subtracted from the background image to obtain a difference image.

[0085] For example, the background image is represented by I back , and for the convenience of description, it is assumed that the background image is represented by a 5*5 matrix:

[0086]

[0087] The second frame in the infrared video is represented by I2, and for the convenience of description, it is assumed that the second frame image is represented by a 5*5 matrix:

[0088]

[0089] The difference between I back and I2 is calculated to obtain a difference image I Δ :

[0090]

[0091] ​For the 3rd frame, the 4th frame, and so on, of the infrared video, respectively subtract the background image I back to obtain the difference image.

[0092] Step 4, convert each difference image into a binary image according to the threshold T1, specifically including:

[0093] For each difference image, set the pixel points with the absolute value of the difference value less than the threshold T1 to 0, and set the remaining pixel points to 1, thereby converting the difference image into a binary image; the pixel points with a value of 1 in the binary image are taken as foreground pixel points, and the pixel points with a value of 0 are taken as background pixel points.

[0094] For example, according to the above I Δ , the absolute value is equal to:

[0095]

[0096] The threshold T1 can be generally defined as 8, and the pixel points with an absolute value less than the threshold 8 in |I Δ | are set to 0, and the remaining pixel points are set to 1, thereby obtaining the binary image I B .

[0097]

[0098] For the 3rd frame, the 4th frame, and so on, of the infrared video, respectively obtain the binary image according to the above method.

[0099] Step 5, delete the connected regions in the binary image with an area less than the threshold T2 or an aspect ratio exceeding the threshold T3, specifically including:

[0100] First, start from a foreground pixel point in the binary image, scan the adjacent foreground pixel points in 8 directions, and mark them as the same connected region, then add the adjacent unmarked pixel points to the queue and continue to expand the connected region until the queue is empty, repeat the process until the entire image is traversed, and obtain all the connected regions in the binary image;

[0101] Secondly, count the number of all pixel points in each connected region in the binary image, which is the area of the connected region, and delete the connected regions with an area less than the threshold T2, that is, set the pixel values in these connected regions with smaller areas to 0;

[0102] Next, for each connected region in the binary image, calculate the minimum and maximum values ​​of the x-coordinates of all points in the region, as well as the minimum and maximum values ​​of the y-coordinates of all points in the region. Based on the above four values, determine the minimum bounding box of the region. The ratio of the long side to the short side of the minimum bounding box is taken as the aspect ratio of the connected region. If the aspect ratio exceeds the threshold T3, the connected region is deleted, that is, the pixel values ​​in the region are set to 0.

[0103] For example, for We obtained 3 connected components (marked with boxes):

[0104]

[0105] The threshold T2 can generally be set to 100. Then the areas of the three connected regions mentioned above are equal to 1, 2, and 1 respectively, all less than the threshold 100. Therefore, the pixel values ​​in these connected regions are set to 0, resulting in I. B :

[0106]

[0107] For example, if a binary image contains a connected region C, and the minimum and maximum x-coordinates of all points in C are found to be 100 and 200 respectively, and the minimum and maximum y-coordinates of all points in C are found to be 50 and 60 respectively, then the minimum bounding box of C is obtained. The coordinates of the four vertices of this minimum bounding box are (100, 50), (100, 60), (200, 50), and (200, 60). The long side of this minimum bounding box is 200 - 100 = 100, and the short side is 60 - 50 = 10. The aspect ratio of the long side to the short side is 100 / 10 = 10. The threshold T3 can generally be set to 5. If the aspect ratio of 10 exceeds the threshold 5, then the connected region is deleted, that is, the pixel values ​​in the region are set to 0.

[0108] Step 6: Calculate the connected component C in the current frame. i Connected region C at the position corresponding to the next frame i+1 The cross-union ratio (CUC) between them; if the CUC is greater than the threshold T4, then C will be... i The candidate gas regions specifically include:

[0109] First, based on the connected region C in the current frame i Find the corresponding position in the next frame using the smallest bounding box. If there is no connected region C at that position... i+1 If the intersection and union ratio is 0, then the intersection and union ratio will be denoted as 0.

[0110] Secondly, if a connected component C exists at the corresponding position in the next frame. i+1 Then calculate That is, to count C separately i With Ci+1 Calculate the number of intersecting pixels and the number of merged pixels, and then calculate the ratio between the two.

[0111] If the Intersection over Union (IoU) is greater than the threshold T4, then C will be... i As a candidate gas region.

[0112] like Figure 2 As shown, the connected region C in the current frame is represented by a solid black dot. i And the connected region C in the next frame i+1 C i and C i+1 The smallest bounding boxes are represented by squares. According to C i Given the smallest bounding box, find a connected component C at the corresponding position in the next frame. i+1 .like Figure 3 As shown, C i With C i+1 The number of intersecting pixels is equal to 15, C i With C i+1 The number of inter-pixel unions is 30, therefore, IoU = 15 / 30 = 0.5. The threshold T4 can generally be set to 0.4, resulting in an IoU greater than the threshold 0.4, thus C... i As a candidate gas region.

[0113] Step 7: Generate randomly distributed points within and at the boundaries of the candidate gas region. Use these points to generate a minimum spanning tree, and further obtain the tree centroid. Use the tree centroid as the center of the candidate gas region. Specifically, this includes:

[0114] First, iterate through each pixel within the candidate gas region. If all adjacent pixels of a pixel are within the candidate gas region, then that pixel is an interior point; otherwise, it is a boundary point. Simultaneously, list all interior points into a column of length N. i The internal pixel column is calculated, and all boundary points are listed in a column of length N. j The boundary point pixel column;

[0115] Next, 0.3N is generated. i A number in [1, N] i Random numbers within the interval are used to obtain pixels at corresponding positions in the internal pixel column, thereby obtaining randomly distributed points within the candidate gas region;

[0116] Then, 0.3N is generated. j A number in [1, N] j Random numbers within the interval are used to obtain pixels at corresponding positions in the boundary point pixel column, thereby obtaining randomly distributed points at the boundary of the candidate gas region.

[0117] Finally, according to the randomly distributed points inside and on the boundary of the candidate gas region, a minimum spanning tree is generated by using the KRUSKAL algorithm, and each node in the minimum spanning tree is traversed to calculate the number of nodes contained in the sub-tree with the node as the root node, and a node that makes the largest sub-tree in all sub-trees minimum is found as the tree center of the minimum spanning tree, and the tree center is taken as the center of the candidate gas region.

[0118] For example, Figure 4 A candidate gas region is shown in which the points in the candidate gas region are represented by 1-41. In the region, the upper, lower, left, and right adjacent four pixels of pixel 1 are not in the candidate gas region, so pixel 1 is a boundary point, and the upper, lower, left, and right four pixels of pixel 5 are in the candidate gas region, so pixel 5 is an internal point, and the remaining points are determined as boundary points and internal points according to the above method, Figure 4 The boundary points are represented by gray dots and the internal points are represented by black dots. At the same time, all internal points are listed in a length N i = 17 internal point pixel column (5, 9, 10, 11, 17, 18, 19, 20, 21, 22, 23, 26, 27, 28, 29, 30, 33), and all boundary points are listed in a length N j = 24 boundary point pixel column (1, 2, 3, 4, 6, 7, 8, 12, 13, 14, 15, 16, 24, 25, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41);

[0119] Then, 0.3N i ( = 0.3 * 17 ≈ 5) random numbers in the interval [1, 17] are generated, and suppose the random numbers are 3, 6, 7, 12, and 14, then the pixels (10, 18, 19, 26, 28) in the internal point pixel column corresponding to the positions of the random numbers are obtained;

[0120] Then, 0.3N j ( = 0.3 * 24 ≈ 7) random numbers in the interval [1, 24] are generated, and suppose the random numbers are 1, 3, 7, 8, 10, 22, and 24, then the pixels (1, 3, 8, 12, 14, 39, 41) in the boundary point pixel column corresponding to the positions of the random numbers are obtained;

[0121] Finally, according to the 12 randomly distributed points (10, 18, 19, 26, 28, 1, 3, 8, 12, 14, 39, 41) inside and on the boundary of the candidate gas region, a minimum spanning tree is generated by using the KRUSKAL algorithm, and the steps of generating a minimum spanning tree with 12 nodes by using the KRUSKAL algorithm are as follows:

[0122] (1) Sort all edges in non-decreasing order of weight;

[0123] (2) select the smallest edge, check whether it forms a loop with the spanning tree formed so far, if not, include the edge, otherwise, discard it;

[0124] (3) repeat step (2) until there are 11 edges in the spanning tree.

[0125] Figure 5 The minimum spanning tree generated according to the above 12 points. Traverse each node in the minimum spanning tree, calculate the number of nodes contained in the subtree with the node as the root node, find a node that makes the maximum subtree in all subtrees minimum, which is the tree center of the minimum spanning tree, here, the maximum subtree of node 19 contains 6 nodes, which is the minimum value among all nodes, so node 19 is the tree center, and node 19 is taken as the center of the candidate gas region.

[0126] Step 8, connecting the center of the candidate gas region with the boundary points of the candidate gas region respectively, calculating the difference between the pixel gray values of adjacent points on the connecting line in the corresponding frame image I i , if the difference is greater than 0, mark 1, otherwise mark 0, and add the 0 or 1 values on the connecting line to obtain ∑ i , add all the ∑ i obtained on all connecting lines, if the addition result is greater than threshold T5, mark the candidate gas region as a leakage gas region, here, threshold T5 can take 0.7 times the sum of the number of adjacent point pairs on all connecting lines, which specifically includes:

[0127] First, connect the center of the candidate gas region with the boundary points of the candidate gas region one by one, calculate the difference between the pixel gray values of adjacent points on the connecting line in the corresponding frame image I i , if the difference is greater than 0, mark 1, otherwise mark 0, here, frame image I i refers to the i-th gray scale image in the infrared video processed by smoothing in step 2;

[0128] Then, add the 0 or 1 values on the connecting line to obtain ∑ i , add all the ∑ i obtained on all connecting lines, if the addition result is greater than threshold T5, mark the candidate gas region as a leakage gas region, here, threshold T5 can take 0.7 times the sum of the number of adjacent point pairs on all connecting lines.

[0129] For example, connect the center node 19 of the candidate gas region with the boundary points (1, 3, 8, 12, 14, 39, 41) of the candidate gas region one by one, calculate the difference between the pixel gray values of adjacent points on the connecting line in the corresponding frame image I i , here, frame image I iThis refers to the i-th grayscale image in the smoothed infrared video described in step 2;

[0130] like Figure 5 As shown, taking the candidate gas region center node 19 and boundary point 14 as an example, assuming the line connecting 19 and 14 passes through the following nodes: 18, 17, 16, 15, then calculate the difference in pixel grayscale values ​​of these adjacent points in the i-th frame of the grayscale image. Assuming the pixel grayscale values ​​of (19, 18, 17, 16, 15, 14) at the corresponding positions in the i-th frame of the grayscale image are 132, 131, 125, 126, 125, 128 respectively, then ∑ i =1+1+0+1+0=3, and the number of adjacent point pairs on the line connecting 19 to 14 is equal to 5. Then calculate the ∑ on the line connecting 19 to the other boundary points 1, 3, 8, 12, 39, and 14 respectively. i And the number of adjacent point pairs, and the ∑ obtained from all the connections. i If the sum is greater than the threshold T5, the candidate gas region is marked as a leaking gas region. Here, the threshold T5 can be 0.7 times the sum of the number of adjacent point pairs on all the connecting lines.

[0131] Step 9: Traverse all binary images in the infrared video, including all binary images from the 2nd frame to the 2nd to 3rd frame (the 3rd to 3rd frame is the last frame), and repeat steps 5-8 to obtain the leaking gas area in all binary images.

[0132] This invention was used to detect an infrared video of a natural gas leak. Figure 6 The video displays 12 consecutive frames, where black areas with white borders represent detected gas leaks. From... Figure 6 As can be seen, the leaked gas areas detected by the present invention are concentrated near the gas source, indicating that the present invention can accurately locate the area where the leaked gas is located.

[0133] In addition, we extracted 12 consecutive frames containing moving figures from infrared video to test the anti-interference capability of this invention. Figure 7 The test results. From Figure 7 It is evident that the present invention can still accurately locate the area where the leaking gas is located even when there is interference from moving objects.

[0134] The present application provides a kind of gas leakage detection method for infrared video, only one infrared thermal imager is needed to obtain the infrared video of monitoring scene, subsequent operations are completed through infrared video, without a large number of data marking and training process, convenient to use, reduce the cost, realize fast detection;At the same time, by analyzing the area and shape of the connected region, and the shape change between adjacent frames, the candidate gas region is selected, and the detection accuracy is guaranteed;For candidate gas region, the present application fully considers the pixel gray distribution between the center point of candidate gas region and the region boundary point to determine the leakage gas region, and further guarantees the detection accuracy.

[0135] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A method for detecting leaked gas using infrared video, characterized in that, Includes the following steps: Step 1: Acquire infrared video of the monitored scene using an infrared thermal imager; specifically including the following steps: First, prepare the infrared thermal imager and set it to video capture mode; Next, point the infrared thermal imager at the scene to be monitored and start the device to collect infrared video. Next, the acquired infrared video is connected to and transmitted to the computer via the device's data interface, and the acquired infrared video is saved to the computer's local hard drive for later use; Step 2: For each frame of the infrared video I i Smoothing is performed using Gaussian filtering; specifically, the following steps are included: First, I extract each frame of the infrared video. i Convert to grayscale image I Gray Each frame in the infrared video is a three-channel color image including red (R), green (G), and blue (B). i , will I i Convert to grayscale image I Gray Through I Gray =R*0.299+G*0.587+B*0.114 complete; Secondly, define the Gaussian kernel G = ; Next, the center of G is aligned to each pixel in the image, and the corresponding pixel values ​​are weighted and averaged according to the weight of each position in G, which is the smoothed pixel value. Step 3: Take the first frame of the smoothed infrared video as the background image, and subtract the background image from each of the remaining frames to obtain the difference image. Step 4: Convert each difference image into a binary image according to the threshold T1; specifically including: For each difference image, calculate the absolute value of the difference value, set the pixels whose absolute value of the difference value is less than the threshold T1 to 0, and set the remaining pixels to 1, thereby converting the difference image into a binary image; in the binary image, the pixels with a value of 1 are used as foreground pixels, and the pixels with a value of 0 are used as background pixels. Step 5: Delete connected regions in the binary image whose area is smaller than threshold T2 or whose aspect ratio exceeds threshold T3; specifically, this includes the following steps: First, starting from a foreground pixel in the binary image, scan the adjacent foreground pixels along 8 directions and mark them as the same connected region. Then add the adjacent unmarked pixels to the queue and continue to expand the connected region until the queue is empty. Repeat this process until the entire image is traversed to obtain all connected regions in the binary image. Secondly, count the number of all pixels in each connected region of the binary image, which is the area of ​​the connected region. Delete connected regions with an area smaller than the threshold T2, that is, set the pixel value of these smaller connected regions to 0. Next, for each connected region in the binary image, calculate the minimum and maximum values ​​of the x-coordinates of all points in the region, as well as the minimum and maximum values ​​of the y-coordinates of all points in the region. Based on the above four values, determine the minimum bounding box of the region. Use the ratio of the long side to the short side of the minimum bounding box as the aspect ratio of the connected region. If the aspect ratio exceeds the threshold T3, delete the connected region, that is, set the pixel values ​​in the region to 0. Step 6: Calculate the connected component C in the current frame. i Connected region C at the position corresponding to the next frame i+1 The cross-union ratio (CUC) between them; if the CUC is greater than the threshold T4, then C will be... i As a candidate gas region, the specific steps include: First, based on the connected region C in the current frame i Find the corresponding position in the next frame using the smallest bounding box. If there is no connected region C at that position... i+1 If the intersection and union ratio is 0, then the intersection and union ratio will be recorded as 0. Secondly, if a connected component C exists at the corresponding position in the next frame. i+1 Then calculate That is, to count C separately i With C i+1 Calculate the number of intersecting pixels and the number of merged pixels, and then calculate the ratio between the two. If the Intersection over Union (IoU) is greater than the threshold T4, then C will be... i As a candidate gas region; Step 7: Generate randomly distributed points within and at the boundaries of the candidate gas region. Use these points to generate a minimum spanning tree, and further obtain the tree centroid. Use the tree centroid as the center of the candidate gas region. This specifically includes the following steps: First, iterate through each pixel within the candidate gas region. If all adjacent pixels of a pixel are within the candidate gas region, then that pixel is an interior point; otherwise, it is a boundary point. Simultaneously, list all interior points into a column of length N. i The internal pixel column is calculated, and all boundary points are listed in a column of length N. j The boundary point pixel column; Next, 0.3N is generated. i A number in [1, N] i Random numbers within the interval are used to obtain pixels at corresponding positions in the internal pixel column, thereby obtaining randomly distributed points within the candidate gas region; Then, 0.3N is generated. j A number in [1, N] j Random numbers within the interval are used to obtain pixels at corresponding positions in the boundary point pixel column, thereby obtaining randomly distributed points at the boundary of the candidate gas region. Finally, based on the randomly distributed points inside and at the boundary of the candidate gas region, the KRUSKAL algorithm is used to generate a minimum spanning tree. Each node in the minimum spanning tree is traversed, and the number of nodes in the subtree rooted at that node is calculated. The node that minimizes the largest subtree among all subtrees is the centroid of the minimum spanning tree, and the centroid is used as the center of the candidate gas region. Step 8: Connect the center of the candidate gas region to the boundary point of the candidate gas region, and calculate the distance between two adjacent points on the connecting line in the corresponding frame image I. i The difference between pixel grayscale values ​​is recorded as 1 if the difference is greater than 0, and 0 otherwise. The 0 or 1 values ​​on the connecting line are added together to obtain ∑. i ∑ obtained from all the connections i The results are summed. If the sum is greater than the threshold T5, the candidate gas region is marked as a leaking gas region. Here, the threshold T5 can be 0.7 times the sum of the number of adjacent point pairs on all connecting lines. Specifically, it includes: First, connect the center of the candidate gas region to the boundary points of the candidate gas region one by one, and calculate the distance between adjacent points on the connecting line in the corresponding frame image I. i The difference in pixel grayscale values ​​is recorded as 1 if the difference is greater than 0, and 0 otherwise. Here, frame image I... i This refers to the i-th grayscale image in the smoothed infrared video described in step 2; Next, add the 0 or 1 values ​​on the connecting lines to get ∑. i ∑ obtained from all the connections i If the sum is greater than the threshold T5, the candidate gas region is marked as a leaking gas region. Here, the threshold T5 can be 0.7 times the sum of the number of adjacent point pairs on all the connecting lines. Step 9: Traverse all binary images in the infrared video, including all binary images from the second frame to the second-to-last frame, and repeat steps 5-8 to obtain the leaking gas area in all binary images.

2. The method for detecting leaked gas using infrared video as described in claim 1, characterized in that, The minimum spanning tree is generated using the Kruskal algorithm, and the steps are as follows: Sort all edges in non-decreasing order of their weights; Select the smallest edge and check if it forms a cycle with the spanning tree formed so far. If it does not form a cycle, include the edge; otherwise, discard it. Repeat the previous step until the remaining edges in the tree are generated.

Images