Visual detection method and system for mist gas leakage based on visible light sensing equipment

Abstract

The present invention discloses a method and system for visually detecting mist gas leaks based on visible light sensing equipment. The method utilizes the visible light sensing equipment to obtain a global image of the leakage scene in real time, and also obtains an image of the suspected leakage area. The image data within the area is preprocessed to obtain a grayscale image and an RGB image. The RGB image is subjected to mist gas leak transmittance and mist gas leak consistency feature extraction to obtain high-dimensional transmittance features and high-dimensional consistency features of the RGB image. The pre-grayscale image is subjected to mist gas leak uniformity and HOG feature extraction to obtain high-dimensional uniformity features and high-dimensional HOG features of the grayscale image. The four extracted feature sets are respectively introduced into four SVM classification models. Confidence is introduced into the classification models to optimize weights in real time. An adaptive weighted voting algorithm is designed to integrate the prediction results of the four base classifiers and output the final gas leakage status. The present invention is low-cost, highly versatile, and improves the accuracy of mist gas leakage status determination.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a method and system for visually detecting mist gas leakage based on visible light sensing equipment. Background Art

[0002] Gas leak safety awareness and prevention measures are widely valued across various sectors, and real-time monitoring of leaks is equally crucial. In modern industry, toxic gas leaks from chemical plants pose not only a threat to workers but also potentially serious impacts on nearby residents and ecosystems. In aviation, leaks from the refueling drogue not only cause visual disturbances to pilots but can also pose a significant safety hazard if they enter the engine. Therefore, providing timely and accurate information on gas leak status can effectively reduce the likelihood of repeated dangerous incidents.

[0003] Currently, to address sudden safety issues such as toxic gas leaks during storage, some researchers are using laser gas detectors combined with visible light sensing equipment to achieve full-coverage gas leak detection in the area being tested. Some researchers have proposed using various gas sensors to build wireless sensor networks to monitor the status of compressed toxic gas leaks in warehouses in real time. Some institutions have used thermal imaging principles, combined with scale-invariant feature transformation and support vector machines, to achieve real-time detection of cloud-like gas leaks. Furthermore, methods such as ultrasound and optical flow are being used to locate gas leaks and determine their status. Regarding oil and gas leaks during aerial refueling, some researchers have proposed hardware improvements, such as improving the material properties of the fuel hose and refueling drogue, and optimizing the mechanical structure between them, to reduce the probability of oil and gas leaks. However, oil leaks are inevitable, and this method cannot effectively determine the status of online oil and gas leaks in real time. Therefore, a method for detecting oil and gas leaks based on infrared thermal imaging has been proposed. By installing an oil and gas leak detection device embedded with an infrared detector on the refueling aircraft wing, regional gas monitoring can be performed within the refueling area to determine if there is a leak.

[0004] However, in actual applications, both methods based on sensor networks and infrared equipment to detect gas leakage are relatively complex and difficult to implement. They lack economy and are easily affected by external factors such as temperature. In contrast, the method of using visible light sensing equipment such as warehouse monitoring equipment and cameras inside the refueling cabin to detect leaked gas is easy to implement and has strong environmental adaptability. Summary of the Invention

[0005] Purpose of the invention: The present invention proposes a visual detection method and system for mist gas leakage based on visible light sensing equipment, which can be used for the detection of various atomized gases and can improve the accuracy of mist gas leakage status judgment.

[0006] Technical solution: The present invention provides a method for visually detecting mist gas leaks based on a visible light sensing device, which specifically includes the following steps:

[0007] (1) Using visible light sensing equipment to obtain a global image of the leakage scene in real time;

[0008] (2) extracting the suspected leakage area from the global image to obtain an image of the suspected leakage area;

[0009] (3) Preprocessing the image data in the area to obtain preprocessed grayscale image and RGB image;

[0010] (4) Extracting the transmittance feature of mist leakage from the RGB image to obtain the high-dimensional transmittance feature of the RGB image;

[0011] (5) Extracting the consistency feature of mist leakage from the RGB image to obtain the high-dimensional consistency feature of the RGB image;

[0012] (6) Extracting the uniformity feature of mist leakage from the grayscale image to obtain the high-dimensional uniformity feature of the grayscale image;

[0013] (7) Extracting the HOG feature of mist leakage from the grayscale image to obtain the high-dimensional HOG feature of the grayscale image;

[0014] (8) The four sets of extracted features are respectively introduced into four SVM classification models to obtain four prediction results of the corresponding features;

[0015] (9) Introduce confidence real-time optimization weights into the classification model, design an adaptive weighted voting algorithm, integrate the prediction results of the four base classifiers, and output the final gas leakage status.

[0016] Furthermore, the implementation process of step (3) is as follows:

[0017] The image in the area is scaled normalized, and the edge detection algorithm is used to detect the edges of the grayscale image to highlight the contour information in the image; the coordinates of the non-zero pixel points in the image are obtained, and these coordinates are fitted into an ellipse using the ellipse fitting algorithm; finally, an elliptical mask is created on the grayscale image and RGB image respectively to extract the corresponding elliptical area.

[0018] Furthermore, the implementation process of step (4) is as follows:

[0019] For each pixel (x, y) in the leakage image, its dark channel value Dark(x, y) is:

[0020] Dark(x,y)=min{R(x,y),G(x,y),B(x,y)} (1)

[0021] Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y);

[0022] A certain RGB block in the image is H, with a size of M×N. The dark channel block is constructed by calculating the dark channel values ​​of all pixels in the block and estimating the block transmittance H T :

[0023]

[0024] Among them, W is a weight parameter used to control the smoothness of transmittance; H Al To estimate the atmospheric light, sort all pixel values ​​in the dark channel block in ascending order and take the maximum value of the first W% for estimation:

[0025] H Al =max{Dark(x,y)},(x,y)∈M×N×W% (3)

[0026] Use the boundary clipping method to limit the range of the calculated transmittance of all blocks:

[0027]

[0028] in, is the block transmittance that meets the specifications; the transmittance of each block of the input image is calculated, that is, the transmittance values ​​corresponding to M blocks are obtained, and they are compressed into a 1×M feature vector, that is, the M-dimensional transmittance feature describing the input image is obtained.

[0029] Furthermore, the implementation process of step (5) is as follows:

[0030] For a block H of size M×N in the leakage image, its RGB mean H avg for:

[0031]

[0032] Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y) in the block respectively;

[0033] Calculate the RGB mean of the 8 adjacent blocks of block H and form a mean array V avg :

[0034] V avg ={V1,V2,…,V i},i=1,2,…,8 (6)

[0035] Among them, V1, V2, V iis the RGB mean of adjacent blocks;

[0036] To obtain the distance array H that represents the degree of color gamut change between block H and adjacent blocks dist , respectively calculate H avg With V avg The Euclidean distance of each element in is:

[0037] H dist ={D1,D2,…,D i},i=1,2,…,8 (7)

[0038] Among them, the difference D i The calculation formula is as follows:

[0039] D i =||H avg -V i ||,i=1,2,…,8 (8)

[0040] Based on the distance array H dist , calculate the compatibility feature H of block H cons. :

[0041]

[0042] in, It is the largest element in the distance array, representing the maximum difference in color gamut; It is the smallest element in the distance array, representing the minimum difference in the color gamut. For each block matching feature of the input image, the matching values ​​corresponding to M blocks are obtained, and they are compressed into a 1×M feature vector to obtain the M-dimensional matching feature describing the input image.

[0043] Furthermore, the implementation process of step (6) is as follows:

[0044] For a grayscale block I in the leakage image, the size is M×N, and the grayscale average value of block I is calculated.

[0045]

[0046] Where I(x,y) represents the pixel value at coordinate (x,y) in the block;

[0047] For each pixel in I, calculate its uniformity factor in turn, and accumulate the average value to get the block uniformity

[0048]

[0049] in, is the uniformity factor of a single pixel:

[0050]

[0051] The uniformity of each block of the input image is calculated to obtain the uniformity values ​​corresponding to M blocks, which are compressed into a 1×M feature vector to obtain the M-dimensional uniformity feature describing the input image.

[0052] Furthermore, the implementation process of step (7) is as follows:

[0053] For the pixel point (x, y), first pass the gradient operator [-1, 0, 1] and [-1, 0, 1] in the horizontal and vertical directions respectively. T Perform convolution operation on the original image to obtain the gradient component G in the corresponding direction x (x,y) and G y (x,y), and then calculate the gradient magnitude G(x,y) and gradient direction θ(x,y) at the pixel:

[0054]

[0055] The 180° gradient direction is divided into M blocks, each corresponding to an angular interval. The gradient information of the corresponding interval is counted to form a gradient direction histogram. Specifically, for a certain cell unit, its gradient direction and gradient amplitude are first calculated, and then the gradient amplitude is mapped to the angular interval corresponding to the gradient direction. The gradient direction histogram containing M amplitudes is obtained, which is also the M-dimensional feature vector corresponding to the cell unit.

[0056] Adjacent cell units are grouped into a block, and multiple cell units in the block together constitute the features of the block, generating the HOG feature vector of the entire image.

[0057] Furthermore, in step (9), the confidence level is introduced into the classification model to optimize the weights in real time, and the adaptive weighted voting algorithm is designed and implemented as follows:

[0058] Confidence represents the degree of trust the classifier has in the prediction result, and is measured by the decision function distance DFD. The decision function distance represents the distance between the sample point and the hyperplane. The support vector machine determines the category to which the sample point belongs based on its size. The larger the distance, the higher the probability of being classified into that category, and the more trust is given to that category. Specifically, in the decision function, the sign function classifies the input sample according to the size of the decision function distance. The confidence of the classification model is:

[0059]

[0060] Among them, C i is the current confidence of the i-th classification model; x is the input sample, xi is the support vector, α i is the Lagrange multiplier of the support vector, y i is the category label of the support vector, rho is the bias term, K(x i ,x) is the kernel function.

[0061] The present invention provides a visual detection system for mist gas leakage based on a visible light sensing device, comprising:

[0062] Image acquisition module, which acquires the global image of the leakage scene in real time based on visible light sensing equipment;

[0063] Suspected leakage area extraction module, extracts the suspected leakage area and segments the area image;

[0064] The data preprocessing module takes the image of the suspected leak area as input and uses the structural characteristics of the leaking gas container to further locate the background target area under different channels;

[0065] A mist gas leakage multi-feature extraction module is used to extract several mist gas leakage features in the target area and generate corresponding mist gas leakage feature vectors;

[0066] The gas leakage status determination module is used to take multiple mist gas leakage feature vectors as input, combine the trained support vector machine classification model and adaptive weighted voting ensemble learning algorithm, determine the gas leakage status and output the results.

[0067] Furthermore, the mist gas leakage characteristics include transmittance characteristics, conformity characteristics, uniformity characteristics and HOG characteristics.

[0068] Furthermore, the support vector machine introduces a confidence real-time optimization weight, so that the weight can change dynamically with the degree of confidence of the classification model for different input samples.

[0069] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention can judge the gas leakage status in the suspected leakage area by relying solely on visible light sensing equipment, is low-cost and highly versatile, and is applicable to various scenarios; the present invention utilizes the motion characteristics of gas leakage and introduces transmittance, conformity, uniformity and HOG features in combination with the structural characteristics of the leakage surface to improve the generalization ability of the gas leakage detection model and is applicable to the detection of various atomized gases; the present invention takes into account the strong correlation between features and designs a multi-classifier training strategy, which integrates different classifier perspectives to reduce overfitting and thus improve model performance, and in response to the problem that weighted voting cannot adjust weights for different samples, introduces confidence to optimize weights in real time, thereby improving the accuracy of the judgment of the mist gas leakage status. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of a visual detection method for mist gas leaks based on visible light sensing equipment;

[0071] Figure 2 It is a schematic diagram of the application scenario;

[0072] Figure 3 Schematic diagram of image preprocessing effect; (a) is a schematic diagram of image preprocessing effect using a gas canister as an example; (b) is a schematic diagram of image preprocessing effect using a cone sleeve as an example;

[0073] Figure 4 The images of each stage are taken as an example of oil leakage of the cone sleeve; among them, (a) is the original image of oil and gas leakage; (b) is the original image without leakage; (c) is the effect diagram of oil and gas leakage transmittance feature extraction; (d) is the effect diagram of non-leakage transmittance feature extraction; (e) is the effect diagram of oil and gas leakage consistency feature extraction; (f) is the effect diagram of non-leakage consistency feature extraction; (g) is the effect diagram of oil and gas leakage uniformity feature extraction; (h) is the effect diagram of non-leakage uniformity feature extraction; (i) is the effect diagram of oil and gas leakage HOG feature extraction; (j) is the effect diagram of non-leakage HOG feature extraction;

[0074] Figure 5 1. It is a schematic diagram of the confusion matrix of the classification results of the present invention on the drogue oil and gas leakage verification set;

[0075] Figure 6 This is a diagram of the confusion matrix of the classification results of multiple SVMs combined with the ordinary weighted voting method on the cone sleeve oil and gas leakage verification set. DETAILED DESCRIPTION

[0076] The present invention will be further described in detail below with reference to the accompanying drawings.

[0077] like Figure 1 As shown, the present invention proposes a visual detection method for mist gas leakage based on a visible light sensing device, which specifically includes the following steps:

[0078] Step 1: Use visible light sensor equipment to obtain a global image of the leakage scene in real time.

[0079] like Figure 2 As shown in the figure, taking two typical leakage scenarios as examples, namely, leakage of compressed poison gas in storage tanks and leakage of oil and gas from aerial refueling drogues, the global image of the leakage scene is obtained in real time using visible light sensing equipment such as monitoring equipment in the warehouse and airborne cameras in the receiving aircraft cabin.

[0080] Step 2: Extract the suspected leakage area from the global image and obtain an image of the suspected leakage area;

[0081] Use target detection algorithms to locate suspected leak areas, and then use image segmentation techniques to extract images of these areas. Target detection algorithms can be traditional machine learning methods, such as decision trees, random forests, and AdaBoost, or deep learning methods, such as GhostNet, MobileNet series, ShuffleNet series, YOLO series, and R-CNN series.

[0082] Step 3: Preprocess the image data in the area to obtain a preprocessed image.

[0083] First, the input region image is normalized and resized to 320×320. Next, an edge detection algorithm is used to detect edges in the grayscale image, highlighting the contours within the image. The coordinates of non-zero pixels in the image are obtained and an ellipse is fitted to these coordinates using an ellipse fitting algorithm. Finally, an elliptical mask is created on both the grayscale and RGB images to extract the corresponding elliptical region.

[0084] By utilizing the structural characteristics of the leaking gas container, the circular area of ​​the leaking surface of the container is further located under different channels. Taking the leakage of canned toxic gas and the leakage of cone sleeve oil and gas as examples, the image preprocessing effect of the present invention is as follows: Figure 3 As shown in (a) and (b), the segmented elliptical area image is the image required after preprocessing.

[0085] Step 4: Extract the transmittance feature of the mist gas leakage from the RGB image to obtain the high-dimensional transmittance feature of the RGB image; the high-dimensional transmittance feature reflects the blurring degree of the gas leakage area by extracting the information of each channel of the leakage image.

[0086] Transmittance feature calculation: For each pixel (x, y) in the leakage image, its dark channel value Dark(x, y):

[0087] Dark(x,y)=min{R(x,y),G(x,y),B(x,y)} (1)

[0088] Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y).

[0089] Assume that a certain RGB block in the image is H and its size is M×N. The dark channel block is constructed by calculating the dark channel values ​​of all pixels in the block and estimating the block transmittance H. T :

[0090]

[0091] Among them, W is a weight parameter used to control the smoothness of transmittance; HAl The atmospheric light estimation value is generally sorted in ascending order for all pixel values ​​in the dark channel block, and the maximum value of the first W% is taken for estimation. The calculation formula is as follows:

[0092] H Al =max{Dark(x,y)},(x,y)∈M×N×W% (3)

[0093] Considering that the transmittance value should be between 0 and 1, where 0 represents that the light is completely absorbed and 1 represents that the light is not absorbed or scattered. To ensure the validity of the transmittance results, the boundary clipping method is used to limit the range of the calculated transmittance of all blocks. The calculation formula is:

[0094]

[0095] in, is the block transmittance that meets the specification.

[0096] Transmittance feature extraction: For an input image of size 320×320, it is divided into M blocks. The processing steps for one of the blocks are as follows:

[0097] First, separate the RGB channels of the block and calculate the minimum value of each channel according to formula (1) to generate a dark channel image. Then, stretch the obtained dark channel image into a row and sort it in ascending order. Combined with formula (3), the weight parameter W is set and the maximum value of the first W% after sorting is selected as the atmospheric light estimation value. Then, based on the atmospheric light estimation value, calculate the initial block transmittance according to formula (2). Finally, as shown in formula (4), use the min and max functions in OpenCV to limit the range of the initial block transmittance to between 0 and 1 to obtain the final block transmittance that meets the specifications. Among them, the weight parameter W is generally taken as 0.95.

[0098] Repeating the above processing steps for each block of the input image can obtain the transmittance values ​​corresponding to M blocks, and compressing them into a 1×M feature vector to obtain the M-dimensional transmittance feature describing the input image.

[0099] Step 5: Extract the consistency feature of the mist gas leakage from the RGB image to obtain the high-dimensional consistency feature of the RGB image; the consistency combines the maximum and minimum differences of the color gamut based on the degree of color gamut change, reflecting more realistic color gamut change information of the gas leakage area.

[0100] Compatibility feature calculation: For a block H of size M×N in the leakage image, its RGB mean H avg :

[0101]

[0102] Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y) in the block respectively.

[0103] Calculate the RGB mean of the 8 adjacent blocks of block H and form a mean array V avg , of the form:

[0104] V avg ={V1,V2,…,V i},i=1,2,…,8 (6)

[0105] Among them, V1, V2, V i is the RGB mean of adjacent blocks.

[0106] To obtain the distance array H that represents the degree of color gamut change between block H and adjacent blocks dist , respectively calculate H avg With V avg The Euclidean distance of each element in is:

[0107] H dist ={D1,D2,…,D i},i=1,2,…,8 (7)

[0108] Among them, the difference D i The calculation formula is as follows:

[0109] D i =||H avg -V i ||,i=1,2,…,8 (8)

[0110] Based on the above distance array H dist , and finally calculate the compatibility feature H of block H cons ., the calculation formula is

[0111]

[0112] in, It is the largest element in the distance array, representing the maximum difference in color gamut; It is the smallest element in the distance array, representing the minimum difference in color gamut.

[0113] Conformity feature extraction: Similar to the transmittance feature, for an input image of size 320×320, it is divided into M blocks. The processing steps for one of the blocks are as follows:

[0114] First, according to formula (5), for a given block and its eight adjacent blocks, the average values ​​of their RGB channels are calculated, and the average values ​​of the eight adjacent blocks are combined into a neighborhood mean array, as shown in formula (7). Next, the Euclidean distance between the block mean and each element in the neighborhood mean array is calculated using formula (8), forming a distance array that represents the color gamut consistency, as shown in formula (7). Furthermore, in order to comprehensively consider the maximum and minimum differences in the color gamut, the maximum and minimum elements are extracted from the distance array, and the square of their difference is calculated in combination with formula (9) to obtain the initial consistency feature of the block.

[0115] Repeating the above processing steps for each block of the input image can obtain the corresponding consistency values ​​of M blocks, and compressing them into a 1×M feature vector to obtain the M-dimensional consistency feature describing the input image.

[0116] In addition, to avoid numerical overflow and improve the generalization of the model, a normalization method is used to map all calculated block consistency values ​​to the target range between 0 and 1.

[0117] Step 6: Extract the uniformity feature of the mist gas leakage from the grayscale image to obtain the high-dimensional uniformity feature of the grayscale image; the uniformity is used to characterize the smoothness of the leakage image at the grayscale level. By utilizing the grayscale mean information in a specific background target area, it reflects the smoothness difference when there is gas leakage in the area.

[0118] Uniformity feature calculation: For a grayscale block I in the leakage image with a size of M×N, first calculate the grayscale average value of block I

[0119]

[0120] Where I(x,y) represents the pixel value at coordinate (x,y) in the block.

[0121] Then, for each pixel in I, calculate its uniformity factor in turn, and accumulate the average value to get the block uniformity The calculation formula is

[0122]

[0123] in, is the uniformity factor of a single pixel, and the calculation formula is as follows:

[0124]

[0125] Uniformity feature extraction: Similar to the transmittance feature, for an input image of size 320×320, it is divided into M blocks. The processing steps for one of the blocks are as follows:

[0126] First, the grayscale average value of a given block is calculated using Equation (10). Next, the square of the difference between each pixel value in the block and the grayscale average value of the block is calculated using Equation (11) to obtain the uniformity factor corresponding to each pixel. Finally, the uniformity factors of all pixels in the block are accumulated and averaged using Equation (12) to obtain the initial uniformity feature of the block.

[0127] Repeat the above processing steps for each block of the input image to obtain the uniformity values ​​corresponding to M blocks. Compress them into a 1×M feature vector to obtain the M-dimensional uniformity feature describing the input image.

[0128] Similar to the consistency feature, in order to avoid numerical overflow and improve the generalization of the model, a normalization method is used to map the calculated uniformity values ​​of all blocks to the target range between 0 and 1.

[0129] Step 7: Extract the HOG features of the mist gas leakage from the grayscale image to obtain the high-dimensional HOG features of the grayscale image. The HOG feature is essentially a statistical analysis of the gradient information in the local area of ​​the leakage image, reflecting the edge and texture information of the gas leakage area.

[0130] The specific implementation process of the HOG feature extraction algorithm is as follows:

[0131] Calculate the gradient. For the pixel point (x, y), first pass the gradient operator [-1, 0, 1] and [-1, 0, 1] in the horizontal and vertical directions respectively. T Perform convolution operation on the original image to obtain the gradient component G in the corresponding direction x (x,y) and G y (x, y), and then calculate the gradient amplitude G(x, y) and gradient direction θ(x, y) at the pixel point. The calculation formula is

[0132]

[0133] Construct a gradient direction histogram. Divide the 180° gradient direction into M blocks, each block corresponds to an angular interval, and count the gradient information of the corresponding intervals to form a gradient direction histogram. Specifically, for a certain cell unit, first calculate its gradient direction and gradient amplitude, and then map the gradient amplitude to the angular interval corresponding to the gradient direction, and you can get a gradient direction histogram containing M amplitudes, that is, the M-dimensional feature vector corresponding to the cell unit. Since the selection of cell units of different sizes will affect the dimension of the feature and the recognition effect, after multiple screening and verification, N×N cell units can be selected to classify and identify the leakage image adjusted to 64×64.

[0134] Normalize and generate the HOG feature vector. Because the gradient of a cell unit is easily affected by external factors and is not robust enough, adjacent cells are usually grouped into a block. Multiple cells within the block together constitute the block's features, and normalization is performed within the block to reduce the impact of illumination changes. The size of the pixel block generally needs to be larger than the cell unit to achieve a better normalization effect. Therefore, a 2N×2N pixel block containing 4 cell units can be selected. The entire image can be traversed with a step size of N. The number of required movements is (64 / N-1)×(64 / N-1) times, and the HOG feature vector of the entire image is finally generated.

[0135] Figure 4 The images of each stage are taken as an example of oil leakage from the cone sleeve; among them, (a) is the original image of oil and gas leakage; (b) is the original image without leakage; (c) is the effect diagram of oil and gas leakage transmittance feature extraction; (d) is the effect diagram of non-leakage transmittance feature extraction; (e) is the effect diagram of oil and gas leakage consistency feature extraction; (f) is the effect diagram of non-leakage consistency feature extraction; (g) is the effect diagram of oil and gas leakage uniformity feature extraction; (h) is the effect diagram of non-leakage uniformity feature extraction; (i) is the effect diagram of oil and gas leakage HOG feature extraction; (j) is the effect diagram of non-leakage HOG feature extraction.

[0136] Step 8: Import the four sets of extracted features into four SVM classification models respectively to obtain four prediction results of the corresponding features;

[0137] (1) Obtain gas leakage images and non-leakage images to create a mist gas leakage sample set.

[0138] After obtaining real warehouse toxic gas leakage or aerial refueling video data from the Internet, we divided it into two categories: leakage data and non-leakage data. Considering that the extremely small amount of leaked gas has a negligible impact on the accident, we manually removed samples with poor quality containing such parts, and used editing software or target detection algorithms to frame the suspected leakage area. The suspected leakage area was saved and a mist gas leakage sample set containing 3,600 images was produced.

[0139] (2) Preprocessing the leaked and non-leaked images in the sample set to obtain preprocessed grayscale images and RGB images;

[0140] The image preprocessing here is consistent with the preprocessing process in step 3, and finally the elliptical area in the input sample is extracted.

[0141] (3) Label the preprocessed grayscale and RGB images, where the leaked image is marked as 1 and the non-leaked image is marked as -1, and randomly divide them into training set, test set and validation set;

[0142] The preprocessed grayscale and RGB image data of the leaked area are labeled and classified, with leaked images marked as 1 and non-leaked images marked as -1. A sample set with labeled information is then generated based on the labeled image data and randomly divided into non-overlapping training, test, and validation sets in a specific ratio.

[0143] (4) Extract four groups of mist leakage features from the preprocessed image and obtain four groups of feature vectors.

[0144] The feature extraction of mist gas leakage here is consistent with the feature extraction method in steps 4 to 7, and finally four sets of corresponding feature vectors are obtained.

[0145] (5) The four groups of mist gas leakage feature vectors and corresponding label information in the training set are respectively imported into different SVM classifiers for training to obtain four SVM classification models.

[0146] The features introduced in steps 4 through 7 are highly correlated with each other, so this paper employs a multi-classifier training strategy. Taking canned toxic gas leaks and drogue oil and gas leaks as examples, to obtain a high-performance classification model, the penalty coefficient, kernel function, and corresponding kernel function parameters were optimized using a grid search and cross-validation approach. The final training parameters for each classifier are shown in Tables 1 and 2.

[0147] Table 1 Training parameters of each SVM classifier (canned poison gas)

[0148]

[0149] Table 2 Training parameters of each SVM classifier (cone oil and gas)

[0150]

[0151] Based on the above optimal training parameters, the four sets of features and their corresponding label information are respectively imported into four SVM classifiers for training, and finally four SVM classification models are obtained.

[0152] Step 9: Use the adaptive weighted voting algorithm to integrate the prediction results of the four base classifiers and output the final gas leakage status.

[0153] The four groups of mist gas leakage feature vectors and corresponding label information in the test set were input into four SVM classification models respectively. The classification accuracy of each model was statistically obtained, and the initial weights of weighted voting were designed based on this.

[0154] First, use the given classification model to predict the category of the test set and assign it a predicted label. Next, repeatedly compare the predicted label with the actual label value and accumulate the number of correct predictions. Finally, calculate the ratio of the number of correct predictions to the total number of test samples to obtain the classification accuracy A of the classification model. The classification accuracy corresponding to each feature classification model was calculated according to the above steps, and the results are shown in Table 3.

[0155] For the i-th classification model, its corresponding initial weight w i :

[0156]

[0157] Among them, A(i) is the classification accuracy of the i-th classification model.

[0158] At this point, the initial weights w1, w2, w3, and w4 corresponding to the four classification models can be calculated, and the results are shown in Table 3.

[0159] Table 3 Classification accuracy of each SVM model and the corresponding initial weights (canned poison gas / drogue oil and gas)

[0160]

[0161] Confidence is introduced from the classification model to optimize weights in real time, and an adaptive weighted voting algorithm is designed.

[0162] In order to give full play to the role of decision function distance in characterizing the degree of confidence of the classifier, the present invention takes the absolute value of the decision function distance and performs nonlinear mapping on the Sigmoid function to obtain the confidence of the classification model:

[0163]

[0164] Among them, C i is the current confidence of the i-th classification model, x is the input sample, x i is the support vector, α i is the Lagrange multiplier of the support vector, y i is the category label of the support vector, rho is the bias term, K(x i ,x) is the kernel function. The confidence value is within the definition of probability; when the decision function distance is 0, that is, the sample point is exactly on the hyperplane, the confidence value is mapped to 0.5, which is consistent with the uncertainty in this case in the binary classification problem.

[0165] In addition, the calculation of the new weights must ensure that the sum of the weights is constant and reflect the impact of the confidence optimization on the voting results. The calculation formula is as follows:

[0166]

[0167] Among them, w i ′ is the new weight corresponding to the i-th classification model.

[0168] The present invention also provides a visual detection system for mist gas leakage based on a visible light sensing device, comprising:

[0169] The image acquisition module acquires the global image of the leakage scene in real time based on the visible light sensing device.

[0170] The suspected leakage area extraction module extracts the suspected leakage area and segments the area image.

[0171] The data preprocessing module takes the image of the suspected leakage area as input and uses the structural characteristics of the leaking gas container to further locate the background target area under different channels.

[0172] The mist gas leakage multi-feature extraction module is used to extract the mist gas leakage emissivity feature, consistency feature, uniformity feature and HOG feature in the target area and generate the corresponding mist gas leakage feature vector.

[0173] The gas leakage status determination module takes multiple mist leakage feature vectors as input and combines them with a trained support vector machine classification model and an adaptive weighted voting ensemble learning algorithm to determine the gas leakage status and output the result. The support vector machine incorporates confidence level to optimize weights in real time, allowing the weights to dynamically change based on the classification model's confidence level for different input samples.

[0174] In order to verify the effectiveness of the method of the present invention, the accuracy of different algorithms in judging the gas leakage status on the validation set was compared.

[0175] ① The mist gas leakage feature vector and corresponding label information are input into four SVM classification models. The ordinary weighted voting algorithm and the adaptive weighted voting algorithm are used to integrate the prediction results of all models and calculate the judgment accuracy.

[0176] ② The same mist leakage feature vector and corresponding label information are input into the random forest classification model and AdaBoost classification model respectively to calculate the judgment accuracy.

[0177] Experiments were carried out on canned toxic gas leakage and cone sleeve oil and gas leakage respectively. The final experimental results obtained according to the above steps are shown in Table 4.

[0178] Table 4 Judgment accuracy of different classification methods (canned poison gas / cone oil gas)

[0179]

[0180] From the data in the table, we can see that in the application of two typical leakage scenarios, the method of the present invention improves by 4.32% and 4.89% respectively compared with the method of combining multiple SVMs with ordinary weighted voting. In addition, compared with the state judgment based on random forest and AdaBoost algorithms, the performance is also more outstanding, with a significant improvement in accuracy. Figure 5 and Figure 6 From the confusion matrix, it can be seen that the method of the present invention has a very significant effect on the judgment of both leakage and non-leakage states.

[0181] The present invention is designed to utilize, but is not limited to, the aforementioned four features. In subsequent research, the present invention will further utilize the continuous variable of leakage volume, using methods such as robust regression, ridge regression, and lasso regression to fit leakage volume curves and construct a gas leakage concentration model. This will assign a corresponding score to each frame of leaked gas during real-time detection, enabling a more refined method for determining mist gas leakage status.

[0182] The present invention is based on a multi-feature extraction method of a mist gas-container combination model and a leakage status determination method based on adaptive weighted voting. The key technical problem to be solved is to construct a mist gas-container combination model by utilizing the obvious relative position relationship between the container and the leaking gas when a suspected leakage area has been segmented, and to introduce transmittance, consistency, uniformity and HOG features to construct pixel-level features of the leaking gas. Based on the strong correlation between features and combined with the excellent small sample classification ability of machine learning, a multi-SVM learning strategy is used during training, and an adaptive weighted voting ensemble learning algorithm is used during detection to integrate the prediction results of all classification models, thereby achieving stable and reliable gas leakage status determination.

[0183] The technical solutions provided by the embodiments of the present invention are described in detail above. Specific examples are used in this patent to illustrate the principles and implementation methods of the embodiments of the present invention. The above embodiments and features in the embodiments can be combined with each other unless there is a conflict. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A visual detection method for mist gas leakage based on visible light sensing equipment, characterized in that: The following steps are involved: (1) Using visible light sensing equipment to obtain a global image of the leakage scene in real time; (2) extracting the suspected leakage area from the global image to obtain an image of the suspected leakage area; (3) Preprocessing the image data in the area to obtain preprocessed grayscale image and RGB image; (4) Extracting the transmittance feature of mist leakage from the RGB image to obtain the high-dimensional transmittance feature of the RGB image; (5) Extracting the consistency feature of mist leakage from the RGB image to obtain the high-dimensional consistency feature of the RGB image; (6) Extracting the uniformity feature of mist leakage from the grayscale image to obtain the high-dimensional uniformity feature of the grayscale image; (7) Extracting the HOG feature of mist leakage from the grayscale image to obtain the high-dimensional HOG feature of the grayscale image; (8) The four sets of extracted features are respectively introduced into four SVM classification models to obtain four prediction results of the corresponding features; (9) Introducing confidence in real-time optimization weights into the classification model, designing an adaptive weighted voting algorithm, integrating the prediction results of the four base classifiers, and outputting the final gas leakage status; The implementation process of step (4) is as follows: For each pixel (x, y) in the leakage image, its dark channel value Dark(x, y) is: Dark(x,y)=min{R(x,y),G(x,y),B(x,y)} (1) Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y); A certain RGB block in the image is H, with a size of M×N. The dark channel block is constructed by calculating the dark channel values ​​of all pixels in the block and estimating the block transmittance H T : Among them, W is a weight parameter used to control the smoothness of transmittance; H Al To estimate the atmospheric light, sort all pixel values ​​in the dark channel block in ascending order and take the maximum value of the first W% for estimation: H Al =max{Dark(x,y)},(x,y)∈M×N×W% (3) Use the boundary clipping method to limit the range of the calculated transmittance of all blocks: in, is the block transmittance that meets the specifications; the transmittance of each block of the input image is calculated, that is, the transmittance values ​​corresponding to M blocks are obtained, and they are compressed into a 1×M feature vector, that is, the M-dimensional transmittance feature describing the input image is obtained; The implementation process of step (5) is as follows: For a block H of size M×N in the leakage image, its RGB mean H avg for: Among them, R(x,y), G(x,y), and B(x,y) are the RGB values ​​of the pixel point (x,y) in the block respectively; Calculate the RGB mean of the 8 adjacent blocks of block H and form a mean array V avg : V avg ={V1,V2,…,V i },i=1,2,…,8 (6) Among them, V1, V2, V i is the RGB mean of adjacent blocks; To obtain the distance array H that represents the degree of color gamut change between block H and adjacent blocks dist , respectively find H avg With V avg The Euclidean distance of each element in is: H dist ={D1,D2,…,D i },i=1,2,…,8 (7) Among them, the difference D i The calculation formula is as follows: D i =||H avg -V i ||,i=1,2,…,8 (8) Based on the distance array H dist , calculate the compatibility feature H of block H cons. : in, It is the largest element in the distance array, representing the maximum difference in color gamut; It is the smallest element in the distance array, representing the minimum difference in the color gamut. For each block matching feature of the input image, the matching values ​​corresponding to M blocks are obtained, and they are compressed into a 1×M feature vector to obtain the M-dimensional matching feature describing the input image.

2. The method for visual detection of mist gas leakage based on visible light sensing equipment according to claim 1, characterized in that: The implementation process of step (3) is as follows: The image in the area is scaled normalized, and the edge detection algorithm is used to detect the edges of the grayscale image to highlight the contour information in the image; the coordinates of the non-zero pixel points in the image are obtained, and these coordinates are fitted into an ellipse using the ellipse fitting algorithm; finally, an elliptical mask is created on the grayscale image and RGB image respectively to extract the corresponding elliptical area.

3. The method for visually detecting mist gas leakage based on visible light sensing equipment according to claim 1, characterized in that: The implementation process of step (6) is as follows: For a grayscale block I in the leakage image, the size is M×N, and the grayscale average value of block I is calculated. Where I(x,y) represents the pixel value at coordinate (x,y) in the block; For each pixel in I, calculate its uniformity factor in turn, and accumulate the average value to get the block uniformity in, is the uniformity factor of a single pixel: The uniformity of each block of the input image is calculated to obtain the uniformity values ​​corresponding to M blocks, which are compressed into a 1×M feature vector to obtain the M-dimensional uniformity feature describing the input image.

4. The method for visually detecting mist gas leakage based on visible light sensing equipment according to claim 1, characterized in that: The implementation process of step (7) is as follows: For the pixel point (x, y), first pass the gradient operator [-1, 0, 1] and [-1, 0, 1] in the horizontal and vertical directions respectively. T Perform convolution operation on the original image to obtain the gradient components Gx(x,y) and Gy(x,y) in the corresponding directions, and then calculate the gradient amplitude G(x,y) and gradient direction θ(x,y) at the pixel point: The 180° gradient direction is divided into M blocks, each corresponding to an angular interval. The gradient information of the corresponding interval is counted to form a gradient direction histogram. Specifically, for a certain cell unit, its gradient direction and gradient amplitude are first calculated, and then the gradient amplitude is mapped to the angular interval corresponding to the gradient direction. The gradient direction histogram containing M amplitudes is obtained, which is also the M-dimensional feature vector corresponding to the cell unit. Adjacent cell units are grouped into a block, and multiple cell units in the block together constitute the features of the block, generating the HOG feature vector of the entire image.

5. The visual detection method for mist gas leakage based on visible light sensing equipment according to claim 1 is characterized in that: Step (9) introduces the confidence level into the classification model to optimize the weights in real time, and designs an adaptive weighted voting algorithm to implement the process as follows: Confidence represents the degree of trust the classifier has in the prediction result, and is measured by the decision function distance DFD. The decision function distance represents the distance between the sample point and the hyperplane. The support vector machine determines the category to which the sample point belongs based on its size. The larger the distance, the higher the probability of being classified into that category, and the more trust is given to that category. Specifically, in the decision function, the sign function classifies the input sample according to the size of the decision function distance. The confidence of the classification model is: Among them, C i is the current confidence of the i-th classification model; x is the input sample, x i is the support vector, α i is the Lagrange multiplier of the support vector, y i is the category label of the support vector, rho is the bias term, K(x i ,x) is the kernel function.

6. A visual detection system for mist gas leaks based on a visible light sensing device using the method according to any one of claims 1 to 5, comprising: Image acquisition module, which acquires the global image of the leakage scene in real time based on visible light sensing equipment; Suspected leakage area extraction module, extracts the suspected leakage area and segments the area image; The data preprocessing module takes the image of the suspected leak area as input and uses the structural characteristics of the leaking gas container to further locate the background target area under different channels; The mist gas leakage multi-feature extraction module is used to extract several mist gas leakage features in the target area and generate corresponding mist gas leakage feature vectors; The gas leakage status determination module is used to take multiple mist gas leakage feature vectors as input, combine the trained support vector machine classification model and adaptive weighted voting ensemble learning algorithm, determine the gas leakage status and output the results.

7. The visual detection system for mist gas leakage based on visible light sensing equipment according to claim 6 is characterized in that: The mist leakage characteristics include transmittance characteristics, conformity characteristics, uniformity characteristics and HOG characteristics.

8. The visual detection system for mist gas leakage based on visible light sensing equipment according to claim 6, characterized in that: The support vector machine introduces a real-time optimization weight of confidence, so that the weight can change dynamically with the degree of confidence of the classification model for different input samples.

Images