Special gas leakage detection method and system

Abstract

The invention relates to a special gas leakage detection method and system. The method comprises the following steps: acquiring an infrared image of a to-be-detected area; and inputting the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result. The method has the advantages that by proposing an infrared image enhancement algorithm based on MSRCR, WLS filtering and CLAHE, the image contrast and texture details are effectively improved, and the foreground and background marginal definition is enhanced, so that the robustness and accuracy of a subsequent deep learning network on leakage region identification are improved; a data set is expanded in various modes of network downloading, field shooting, software synthesis and the like, so that the model can learn richer features from a large number of diversified samples, the over-fitting risk is reduced, the generalization ability of the model is enhanced, and the model can adapt to different environments and leakage conditions; the combination of the lightweight model and the image enhancement algorithm enables the algorithm to achieve a faster detection speed while guaranteeing the recognition accuracy, and meets the real-time requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of gas leakage detection, and in particular to a special gas leakage detection method, a special gas leakage detection system, a computer device and a computer-readable storage medium. Background Art

[0002] Since dangerous gases are prone to leaks during production, storage and transportation, and fires and explosions caused by leaks often cause significant casualties, economic losses and environmental pollution, gas leak detection has become a hot topic in the oil and gas industry and a top priority in ensuring safe gas transportation.

[0003] The prior art method for detecting special gas leakage is as follows: obtaining infrared video data with gas leakage and preprocessing the data, using Opencv video framing technology to obtain infrared image sequences; using the image annotation tool Labelme to perform pixel-level labeling on the target area of ​​the natural gas infrared image sequence; using U 2 -Net image segmentation network replaces the background modeling method to extract the leaking gas area; by migrating the VGG16 network model structure and convolution layer parameters, adding a BN layer between the convolution layer and the excitation layer, and replacing the last pooling layer with a dynamic adaptive pooling method based on the maximum pooling algorithm, leakage detection is performed.

[0004] However, the prior art special gas leakage detection method has the following defects:

[0005] 1. U 2 -Net network models usually require a lot of computational cost to make accurate predictions for each pixel, perform poorly on low-performance devices, and are almost unsuitable for low-power devices;

[0006] 2. The patent's preprocessing of infrared images tends to split and annotate infrared images, but does not perform real-time enhancement processing on the collected infrared images. Therefore, when identifying low-quality infrared images (high noise, unclear foreground and background edges, etc.), the effect is poor and the robustness is poor;

[0007] 3. The existing data set only has 750 images, and only 600 of them are used for training. The insufficient sample size will directly affect the generalization of the final model.

[0008] At present, no effective solution has been proposed for the problems of special gas leakage detection methods in related technologies, such as being unsuitable for low-power devices, poor results, poor robustness, and insufficient sample size. Summary of the invention

[0009] The purpose of this application is to address the deficiencies in the prior art and to provide a special gas leakage detection method, a special gas leakage detection system, a computer device and a computer-readable storage medium, so as to at least solve the problems existing in the related art that the special gas leakage detection method is not suitable for low-power devices, has poor effects, poor robustness and insufficient sample size.

[0010] To achieve the above purpose, the technical solution adopted by this application is:

[0011] In a first aspect, the present invention provides a method for detecting a special gas leak, comprising:

[0012] Acquire an infrared image of the area to be inspected;

[0013] The infrared image is input into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

[0014] In some embodiments, obtaining an infrared image of the area to be inspected includes:

[0015] Acquire an infrared image sequence of the area to be detected;

[0016] The infrared image sequence is subjected to frame processing to obtain a plurality of infrared images.

[0017] In some embodiments, obtaining an infrared image of the area to be inspected includes:

[0018] Acquire the original infrared image of the area to be detected;

[0019] Performing image enhancement processing on the original infrared image to obtain an enhanced infrared image

[0020] In some embodiments, obtaining an infrared image of the area to be inspected includes:

[0021] Acquire the original infrared image sequence of the area to be detected;

[0022] Performing frame processing on the original infrared image sequence to obtain a plurality of original infrared images;

[0023] Image enhancement processing is performed on the plurality of original infrared images to obtain a plurality of enhanced infrared images.

[0024] In some embodiments, obtaining an infrared image of the area to be inspected includes:

[0025] Acquire an original infrared image sequence of the area to be detected, wherein the original infrared image sequence includes a plurality of original infrared images;

[0026] Performing image enhancement processing on the original infrared image sequence to obtain an enhanced infrared image sequence;

[0027] The enhanced infrared image sequence is subjected to frame processing to obtain a plurality of enhanced infrared images.

[0028] In some of the embodiments, the image enhancement process includes:

[0029] Performing MSRCR adaptive gain processing on the original infrared image to obtain a first processed image;

[0030] Performing WLS filtering on the first processed image to obtain a second processed image;

[0031] The second processed image is subjected to CLAHE image enhancement processing to obtain an enhanced infrared image.

[0032] In some of the embodiments, the image enhancement process includes:

[0033] Performing MSRCR adaptive gain processing on the original infrared image sequence to obtain a first processed image sequence;

[0034] Performing WLS filtering on the first processed image sequence to obtain a second processed image sequence;

[0035] The second processed image sequence is subjected to CLAHE image enhancement processing to obtain an enhanced infrared image sequence.

[0036] In some embodiments, performing MSRCR adaptive gain processing on the original infrared image to obtain a first processed image includes:

[0037] The infrared image is processed using a Gaussian filter to obtain a first infrared sub-image;

[0038] Mapping the first infrared sub-image structure to a logarithmic domain and performing linear stretching processing to obtain a second infrared sub-image;

[0039] Perform multi-scale Retinex processing on the second infrared sub-image to obtain a third infrared sub-image;

[0040] Performing color restoration processing on the second infrared sub-image to obtain a fourth infrared sub-image;

[0041] The third infrared sub-image and the fourth infrared sub-image are combined to obtain a first processed image.

[0042] In some embodiments, performing WLS filtering on the first processed image to obtain a second processed image includes:

[0043] Calculating the gradients of the first processed image in the horizontal direction and the vertical direction to obtain gradient information;

[0044] Calculate the weight of each pixel in the first processed image in the horizontal direction and the vertical direction according to the gradient information to obtain weight information;

[0045] An objective function is constructed and an iterative method or a matrix decomposition method is used to solve and optimize the problem according to the gradient information and the weight information to obtain a second processed image.

[0046] In some embodiments, performing CLAHE image enhancement processing on the second processed image to obtain an enhanced infrared image includes:

[0047] dividing the second processed image to obtain a plurality of image regions;

[0048] Performing equalization processing on the plurality of image regions to obtain a plurality of enhanced image regions;

[0049] A plurality of the enhanced image regions are interpolated and synthesized to obtain an enhanced infrared image.

[0050] In some embodiments, inputting the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result includes:

[0051] Performing feature extraction on the infrared image to obtain image features;

[0052] Determining whether the image feature is a preset image feature;

[0053] In the case where the image feature is the preset image feature, the area containing the image feature is marked.

[0054] In some embodiments, inputting the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result includes:

[0055] Performing convolution processing on the infrared image to obtain a feature layer;

[0056] Extracting the feature layer to obtain shallow feature information and deep feature information;

[0057] The shallow feature information and the deep feature information are fused to obtain a feature map;

[0058] Determine whether there is a leakage area in the characteristic map;

[0059] In the case where there is a leakage area in the characteristic map, the characteristic map is marked.

[0060] In a second aspect, the present invention provides a special gas leakage detection system, comprising:

[0061] An image acquisition unit, used to acquire an infrared image of the area to be detected;

[0062] The detection unit is used to input the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

[0063] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the special gas leakage detection method as described in the first aspect when executing the computer program.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the special gas leakage detection method as described in the first aspect.

[0065] Compared with the related art, the embodiments of the present application provide a special gas leakage detection method, a special gas leakage detection system, a computer device and a computer-readable storage medium. By proposing an infrared image enhancement algorithm based on MSRCR, WLS filtering and CLAHE, the image contrast and texture details are effectively improved, and the edge clarity of the foreground and background is enhanced, thereby improving the robustness and accuracy of the subsequent deep learning network in identifying the leakage area; by improving the YOLOv5 network model, a lightweight model is constructed, and the hardware resource consumption is reduced, making it suitable for low-power devices such as portable laptops or mobile phones, which is convenient for on-site detection; the data set is expanded through various methods such as network download, field shooting and software synthesis, so that the model can learn richer features from a large number of diverse samples, reduce the risk of overfitting, and enhance the generalization ability of the model, so that it can adapt to different environments and leakage situations; the combination of lightweight models and image enhancement algorithms enables the algorithm to achieve faster detection speed while ensuring recognition accuracy, meeting real-time requirements; the special gas leakage detection method provided by the present invention can detect leaks earlier, so as to take measures in time to reduce safety risks, protect the personal safety of workers, and ensure production safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0067] Figure 1 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (I);

[0068] Figure 2 is a flow chart (II) of a method for detecting a special gas leak according to an embodiment of the present application;

[0069] Figure 3is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (III);

[0070] Figure 4 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (IV);

[0071] Figure 5 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (V);

[0072] Figure 6 is a flow chart (six) of a method for detecting a special gas leak according to an embodiment of the present application;

[0073] Figure 7 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (VII);

[0074] Figure 8 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (eight);

[0075] Fig. 9 is a flow chart (IX) of a method for detecting a special gas leak according to an embodiment of the present application;

[0076] Fig.10 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (X);

[0077] Fig.11 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (XI);

[0078] Fig.12 is the original infrared image;

[0079] Fig.13 It is an enhanced infrared image;

[0080] Fig.14 It is the network framework of the special gas leak detection model;

[0081] Fig.15 It is a framework diagram of a special gas leakage detection system according to an embodiment of the present application.

[0082] The reference numerals are: 1410, image acquisition unit; 1420, image enhancement unit; 1430, detection unit. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0084] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0085] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0086] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or units (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0087] Example 1

[0088] This embodiment relates to a special gas leakage detection method of the present invention.

[0089] Figure 1 FIG. 1 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application (I). Figure 1 As shown, the present invention provides a special gas leakage detection method based on deep learning, comprising:

[0090] Step S110, obtaining an infrared image of the area to be detected;

[0091] Step S120: input the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

[0092] Figure 2 FIG. 2 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 2 As shown, step S110 includes:

[0093] Step S210, obtaining an original infrared image sequence of the area to be detected;

[0094] Step S220: performing frame processing on the original infrared image sequence to obtain a plurality of original infrared images.

[0095] It should be noted that the infrared image sequence of the area to be detected is usually acquired by using an infrared camera, and the infrared camera can acquire an infrared image sequence of 30fps per second.

[0096] It should be noted that the infrared image sequence is framed using Opencv video framing technology to obtain 30 infrared images.

[0097] Furthermore, step S110 further includes:

[0098] Step S230: performing image enhancement processing on a plurality of original infrared images to obtain a plurality of enhanced infrared images.

[0099] Figure 3 FIG. 3 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 3 As shown, step S230 includes:

[0100] Step S310, performing MSRCR adaptive gain processing on the original infrared image to obtain a first processed image;

[0101] Step S320, performing WLS filtering on the first processed image to obtain a second processed image;

[0102] Step S330: Perform CLAHE image enhancement processing on the second processed image to obtain an enhanced infrared image.

[0103] It should be noted that the MSRCR adaptive gain processing combines two parts: multi-scale Retinex (MSR) and color restoration (CR).

[0104] It should be noted that the MSRCR algorithm usually includes the following steps: multi-scale decomposition: decomposing the image into representations of multiple scales, usually using a Gaussian pyramid or other multi-scale transformation methods; adaptive gain control: at each scale, adjusting the contrast or gain of the image to enhance details and reduce noise; Retinex processing: processing the image according to the relationship between illumination and reflectivity to simulate the adaptability of the human eye to changes in illumination; reconstruction: recombining the processed multi-scale representations to generate the final enhanced image.

[0105] It should be noted that the WLS filter minimizes the difference between the smoothed image (enhanced or denoised image) and the original image by optimizing an objective function, while reducing the smoothing effect in areas with large gradients (edges) to achieve the purpose of edge preservation.

[0106] It should be noted that the key features of WLS filtering include: data term: the data term is usually a square function, which is used to measure the difference between the current pixel value and the surrounding pixel values; smoothing term: the smoothing term is used to control the smoothness of the filter, which is usually related to the gradient or structure of the image; weighting scheme: the weighting scheme allows the algorithm to dynamically adjust the filtering strength according to the image content (such as edges, textures, etc.); optimization problem: WLS filtering is usually implemented by solving a linear system, which can be solved by various numerical methods.

[0107] Figure 4 FIG. 4 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 4 As shown, step S310 includes:

[0108] Step S410, processing the infrared image using a Gaussian filter to obtain a first infrared sub-image;

[0109] Step S420, mapping the first infrared sub-image structure to a logarithmic domain and performing linear stretching processing to obtain a second infrared sub-image;

[0110] Step S430, performing multi-scale Retinex processing on the second infrared sub-image to obtain a third infrared sub-image;

[0111] Step S440, performing color restoration processing on the second infrared sub-image to obtain a fourth infrared sub-image;

[0112] Step S450: Combine the third infrared sub-image and the fourth infrared sub-image to obtain a first processed image.

[0113] It should be noted that multi-scale Retinex processing (MSR) calculates the weighted sum of Retinex under fuzzy filtering at different scales, and retains the details and overall structure of the image through multi-scale enhancement.

[0114] Among them, the calculation formula of multi-scale Retinex processing (MSR) is as follows:

[0115]

[0116] Among them, R MSR (x, y) is the output of multi-scale Retinex; R S (x, y) is the Retinex output of the sth scale; w s are weights of different scales, ∑w s =1; S is the scale number.

[0117] It should be noted that the color restoration process aims to overcome the problem of color distortion after Retinex processing. By introducing a color restoration function, each channel is adjusted to make the enhanced image closer to the natural visual perception of the human eye.

[0118] Among them, the color restoration function is as follows:

[0119]

[0120] Among them, α is the adjustment parameter, which is generally a constant; I c (x, y) is the value of each color channel (such as R, G, B);

[0121] It should be noted that the first processed image is obtained by combining the outputs of the multi-scale Retinex process (MSR) and the color restoration process.

[0122] The calculation formula for combining the third infrared sub-image and the fourth infrared sub-image is as follows:

[0123] I MSRCR (x, y) = G·R MSR (x, y) · CR (x, y) + b;

[0124] Where G is the gain factor and b is the offset.

[0125] Figure 5 FIG. 5 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 5 As shown, step S320 includes:

[0126] Step S510, calculating the gradients of the first processed image in the horizontal direction and the vertical direction to obtain gradient information;

[0127] Step S520, calculating the weights of each pixel in the first processed image in the horizontal direction and the vertical direction according to the gradient information to obtain weight information;

[0128] Step S530: construct an objective function and use an iterative method or a matrix decomposition method to solve and optimize according to the gradient information and the weight information to obtain a second processed image.

[0129] It should be noted that the objective function is as follows:

[0130]

[0131] Among them, u i，j is the smoothed image to be solved, s i，j is the pixel value of the first processed image; λ is the smoothness control parameter; w x (i, j) and w y(i, j) are the weights of each pixel in the horizontal and vertical directions respectively.

[0132] It should be noted that the calculation formula for solving the optimization using the iteration method or matrix decomposition method based on the gradient information and weight information is as follows:

[0133]

[0134] in, and are the gradients of the first processed image in the horizontal and vertical directions respectively; α is the intensity parameter controlling the edge strength; β is the power exponent controlling the gradient amplitude.

[0135] It should be noted that since the smoothing effect is reduced at the edge, the WLS filter can well preserve the edge details of the image. This effectively enhances the local contrast and details of the image. The WLS filter is relatively robust to noise and can enhance the image without significantly increasing the noise. Finally, the contrast-limited histogram equalization (CLAHE) process is applied to the processed result to obtain the final enhanced image.

[0136] Figure 6 FIG. 6 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 6 As shown, step S330 includes:

[0137] Step S610, dividing the second processed image to obtain a plurality of image regions;

[0138] Step S620, performing equalization processing on a plurality of image regions to obtain a plurality of enhanced image regions;

[0139] Step S630: interpolate and synthesize a plurality of enhanced image regions to obtain an enhanced infrared image.

[0140] It should be noted that the second processed image is divided into blocks, and the second processed image is divided into a number of non-overlapping small blocks (i.e., image regions); wherein each sub-block is of size b×b, and is divided into M×N blocks in total, then H is the height of the image, w is the width of the image, and b is the side length of the image area.

[0141] Figure 7 FIG. 7 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 7 As shown, step S620 includes:

[0142] Step S710, calculating histograms of several image regions to obtain several histogram information;

[0143] Step S720: Compare the histogram information of an image region with a cropping threshold. If the histogram information of the image region is greater than the cropping threshold, perform contrast cropping on the image region to obtain a first grayscale histogram and a second grayscale histogram;

[0144] Step S730: Evenly distribute the second grayscale histogram to all the remaining image regions;

[0145] Step S740: Repeat Step S720 to Step S730 until the histogram information of each graphic region is not greater than the cropping threshold, and obtain a number of enhanced image regions.

[0146] It should be noted that the calculation formula for the histogram of each image region is as follows:

[0147]

[0148] where H ij (k) is the histogram information of the image region in the i-th row and j-th column, and k is the gray level (ranging from 0 to L - 1, usually L = 256); δ(x) is the Dirac delta function,

[0149] It should be noted that to avoid excessive contrast enhancement, a cropping threshold T is set. When H ij (k) > T, the excess part (i.e., the second grayscale histogram) is evenly distributed to all image regions.

[0150] It should be noted that the calculation method for the second grayscale histogram is as follows:

[0151]

[0152] where E is the second grayscale histogram and T is the cropping threshold.

[0153] It should be noted that when H ij (k) < T, the processing of Step S720 to Step S730 is not performed on this image region.

[0154] It should be noted that the number of first grayscale histograms is the same as the number of image regions; the number of second grayscale histograms is less than the number of image regions.

[0155]

[0156] where H′ ij (k) is the histogram information of the first grayscale histogram after cropping for the image region in the i-th row and j-th column; H″ ij (k) is the histogram information of the image region after even distribution.

[0157] It should be noted that after multiple processes of step S730, the image area where originally H ij (k)<T may have the situation where H ij (k)>T, then the image area needs to be processed from step S720 to step S730.

[0158] It should be noted that the equalization process is to calculate the cumulative distribution function (CDF) F ij (k) of the image area at the i-th row and j-th column according to the restricted H″ ij (k).

[0159]

[0160] T ij (k)=(L - 1)×F ij (k);

[0161] Among them, F ij (k) is the cumulative distribution function of the image area at the i-th row and j-th column; T ij (k) is the transformation function of the image area at the i-th row and j-th column.

[0162] It should be noted that for any pixel position in the second processed image, calculate its relative position in the context area and calculate the weights of adjacent sub-blocks. Then the new gray value I′(x, y) after interpolation is obtained by interpolating the equalization change functions of four adjacent sub-blocks:

[0163] I′(x, y)=(1 - α)(1 - β)T ij (I(x, y))+α(1 - β)T i+1，j (I(x, y))+(1 - α)βT i，j+1 (I(x, y))+αβT i+1，j+1 (I(x, y));

[0164]

[0165] Among them, (x, y) is the position of any pixel in the infrared image; α, β are the weights of adjacent image areas; I′(x, y) is the gray value of the enhanced infrared image.

[0166] It should be noted that CLAHE divides the second processed image into image areas (i.e., sub-blocks), equalizes each image area separately, and avoids over-enhancement through contrast clipping. Finally, the equalization results of each image area are interpolated and synthesized to generate a globally equalized image. This method can improve the detail contrast of the image while reducing noise amplification.

[0167] In addition, Fig.12 、 Fig.13The image comparison can clearly see the difference after image enhancement ( Fig.12 is the original infrared image, Fig.13 is the infrared image enhanced by the enhancement algorithm).

[0168] Processing infrared images through the above method can improve detection accuracy, enhance image contrast and details, make leakage areas more obvious and easier to identify; reduce misjudgment, by reducing image noise and enhancing edges, reduce the risk of misjudgment caused by poor image quality; adapt to complex environments, the enhanced images can better adapt to different environmental conditions and improve the robustness of the system in various environments; optimize the deep learning model, provide clearer image input for the deep learning network, and enhance the recognition and prediction capabilities of the model; improve real-time performance, quickly process images, meet the needs of real-time monitoring, and ensure timely detection of leaks.

[0169] Figure 8 FIG. 8 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Figure 8 As shown, step S120 includes:

[0170] Step S810, extracting features from the infrared image to obtain image features;

[0171] Step S820, determining whether the image feature is a preset image feature;

[0172] Step S830: When the image feature is the preset image feature, annotate the area containing the image feature.

[0173] Fig. 9 FIG. 9 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Fig. 9 As shown, step S120 includes:

[0174] Step S910, performing convolution processing on the infrared image to obtain a feature layer;

[0175] Step S920: extracting the feature layer to obtain shallow feature information and deep feature information;

[0176] Step S930: fusing the shallow feature information and the deep feature information to obtain a feature map;

[0177] Step S940, determining whether there is a leakage area in the characteristic graph;

[0178] Step S950: if there is a leakage area in the feature map, mark the feature map.

[0179] It should be noted that the special gas leakage detection model is an improved YOLOv5 model. The improved YOLOv5 model uses the lightweight CNN model RepViT network module as the backbone, uses the Transformer encoder block (TRE) to replace the CSP structure in the Neck part of the original YOLOv5, adds a CBAM module to the neck of the original YOLOv5 model, and introduces the SPPELAN module in the spatial pyramid pooling part to more effectively utilize shallow and deep features, add a small target detection layer, and improve the detection accuracy.

[0180] It should be noted that if Fig.14 As shown in the figure, the network framework of the special gas leakage detection model can be divided into four parts: input, backbone network, neck and detection head. The backbone network first performs convolution processing on the input image, and then inputs the obtained feature layer into the RepViT network module for training. After the training of the backbone network, a series of feature maps of different scales will be obtained. The neck is mainly a PAN structure, which adds bottom-to-top information fusion on the basis of FPN (region candidate network) from top to bottom information fusion. This structure aggregates the features of different detection layers in different backbone layers, further improving the feature extraction ability of the network, and the SPPELAN module can increase the receptive field, separate the most significant contextual features, and solve the problem of multi-scale targets to a certain extent. Head is the prediction part of the model, which adopts a multi-scale prediction method to predict, generate bounding boxes and determine categories for four types of targets: small, small, medium and large.

[0181] Among them, the Backbone part is the backbone of the algorithm, which is responsible for extracting the features of the target image, and the four Stages output the extracted feature images respectively; the Neck part is the neck of the algorithm, which is responsible for feature fusion, that is, the shallow feature information (usually contains more detailed information, such as low-level feature information such as color, texture, edge and corner points of the image) such as the feature map output by Stage1 and 2 is fused with the deep feature information (usually including higher-level semantic information in the image, such as the shape of the object, etc. It has a larger receptive field and can capture a wider range of contextual information) such as the feature map output by Stage3 and 4. In this way, in the Neck part, there are bottom-up, top-down and lateral connections. For feature information at different levels, tensor concatenation (Concate) is used for multi-scale feature fusion, such as concatenating two tensors of 26*26*256 and 26*26*512, and the result is 26*26*768; the Head part is responsible for analyzing and identifying the fused feature map and outputting the result.

[0182] The input of the RepViT network module is in the form of B*3*H*W, where B represents the batch size (BatchSize), that is, the number of samples processed simultaneously; 3 represents the number of channels of the image, usually a color image has three channels (such as RGB); H represents the height of the image; and W represents the width of the image. 3*3, 1*1, etc. in the structure diagram represent the size of the convolution kernel, Stride represents the step size of the convolution, ⊕ represents element-by-element addition, DW represents depthwise convolution, and SE represents the SE channel attention mechanism, which can enhance the network's ability to focus on information of interest. RepViT is used as the backbone network of the model, and it is only needed to extract feature information. The final pooling layer (Pooling) and fully connected layer (FC) are not helpful for extracting feature information, so they are discarded.

[0183] The Transformer encoder module mainly includes a multi-head attention block and a feedforward neural network structure (MLP). LayerNorm and Dropout enable the network to converge better and prevent overfitting, while multi-head attention can help the network not only focus on the information at the current position, but also obtain related semantic information.

[0184] Among them, CBAM (Convolutional Block Attention Module) is a module that combines channel attention and spatial attention. The input feature map is first processed by channel attention: ① Global maximum pooling and global average pooling: The input feature map is subjected to global maximum pooling and global average pooling respectively, the spatial dimension is compressed to 1, the channel information is retained, and two pooled feature vectors are obtained, which represent the maximum eigenvalue and average eigenvalue on each channel respectively. ② Multi-layer perceptron (MLP) processing: The above two pooled features are sent to the shared multi-layer perceptron (MLP) to extract features. . ③ Addition and activation: The pooled features after MLP are added, and then the final channel attention weight is obtained through the Sigmoid activation function. Then the channel attention weight is multiplied with the original feature map to obtain the intermediate feature layer. Then the spatial attention is processed: ① Maximum pooling and average pooling: The intermediate feature map is subjected to maximum pooling and average pooling in the channel dimension respectively, the channel dimension is compressed to 1, and the spatial information is retained. ② Concatenation and convolution: concatenate the pooled features along the channel dimension, then extract features through a convolution layer, and reduce the channel dimension to 1. ③ Activation: After Sigmoid activation, the final spatial attention weight (including channel attention) is obtained. After completing these two steps in series, the spatial attention feature is multiplied with the original feature map to obtain the feature map processed by the channel and spatial attention mechanisms.

[0185] Among them, the SPPELAN (Spatial Pyramid Pooling Enhanced with ELAN) module is a combination of SPP and ELAN (Efficient Local Aggregation Network). This module performs multiple operations on the input feature map: ① Perform spatial pyramid pooling, that is, perform maximum pooling operations on windows of multiple different scales to obtain multi-scale feature maps. ② Perform convolution processing on the input feature map simultaneously. ③ Add the multiple feature images obtained along the channel number, and finally output the processed and fused feature map.

[0186] This module aims to improve the effect of target detection by integrating the advantages of both. It can make full use of the spatial pyramid pooling capability of SPP and the efficient feature aggregation capability of ELAN, and can capture more detailed target features at different scales, thereby improving the accuracy of target detection.

[0187] It should be noted that the input size of the special gas leakage detection model is 512*512, the number of iterations (Epoch) is set to 300, the backbone network is frozen in the first 60 trainings, the batch_size is 64, and only the pre-trained weights are loaded. The backbone network is unfrozen in the last 240 trainings, and the batch_size is 32. The network model training hyperparameters are set as follows: the optimizer uses SGD, the momentum factor is 0.937, and the decay coefficient is 0.0005. The maximum learning rate of the training process is set to 0.04, the minimum learning rate is limited to 0.0016, and the Mosaic data enhancement and cosine annealing method are used to control the learning rate. The so-called cosine annealing method makes the learning rate during model training dynamically adjusted according to the cosine function to avoid the model falling into a local optimal solution.

[0188] In addition, the special gas leakage detection method of the embodiment of the present application can be implemented by a computer device. The components of the computer device may include but are not limited to a processor and a memory storing computer program instructions.

[0189] In some embodiments, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0190] In some embodiments, the memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a specific embodiment, the memory is a non-volatile memory. In a specific embodiment, the memory includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0191] The memory may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor.

[0192] The processor implements any one of the deep learning-based special gas leak detection methods in the above-mentioned embodiments by reading and executing computer program instructions stored in the memory.

[0193] In some embodiments, the computer device may further include a communication interface and a bus, wherein the processor, the memory, and the communication interface are connected via the bus and communicate with each other.

[0194] The communication interface is used to realize the communication between each unit, device, unit and / or device in the embodiment of the present application. The communication interface can also realize data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0195] The bus includes hardware, software or both, and couples the components of the computer device to each other. The bus includes but is not limited to at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0196] The computer device can execute the special gas leakage detection method in the embodiment of the present application.

[0197] In addition, in combination with the special gas leakage detection method in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the special gas leakage detection methods based on deep learning in the above embodiment is implemented.

[0198] The present invention proposes an infrared image enhancement algorithm based on MSRCR, WLS filtering and CLAHE, which effectively improves image contrast and texture details, and enhances the edge clarity of foreground and background, thereby improving the robustness and accuracy of subsequent deep learning network in identifying leakage areas; by improving the YOLOv5 network model, a lightweight model is constructed, which reduces hardware resource consumption and makes it suitable for low-power devices, such as portable laptops or mobile phones, to facilitate on-site detection; the data set is expanded through multiple methods such as network download, on-site shooting and software synthesis, so that the model can learn richer features from a large number of diverse samples, reduce the risk of overfitting, enhance the generalization ability of the model, and enable it to adapt to different environments and leakage situations; the combination of the lightweight model and the image enhancement algorithm enables the algorithm to achieve faster detection speed while ensuring recognition accuracy, meeting real-time requirements; the special gas leakage detection method provided by the present invention can detect leakage earlier, so as to take measures in time to reduce safety risks, protect the personal safety of workers, and ensure production safety.

[0199] Example 2

[0200] This embodiment is a variation of the embodiment 1. The main difference between this embodiment and the embodiment 1 is that the step S110 is different.

[0201] Fig.10 FIG. 10 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Fig.10 As shown, step S1010 includes:

[0202] Step S1010, obtaining an original infrared image of the area to be detected;

[0203] Step S1020: Perform image enhancement processing on the original infrared image to obtain an enhanced infrared image.

[0204] It should be noted that step S1010 can directly obtain the original infrared image of the area to be detected, and perform image enhancement processing on the original infrared image.

[0205] It should be noted that step S1020 is the same as step S230 in embodiment 1, and will not be described in detail here.

[0206] Example 3

[0207] This embodiment is a variation of Embodiment 1 to Embodiment 2. The main difference between this embodiment and Embodiment 1 to Embodiment 2 is that: Step S110 is different.

[0208] Fig.11 FIG. 11 is a flow chart of a method for detecting a special gas leak according to an embodiment of the present application. Fig.11As shown, step S110 includes:

[0209] Step S1110, obtaining an original infrared image sequence of the area to be detected, wherein the original infrared image sequence includes a plurality of original infrared images;

[0210] Step S1120, performing image enhancement processing on the original infrared image sequence to obtain an enhanced infrared image sequence;

[0211] Step S1130: performing frame processing on the enhanced infrared image sequence to obtain a plurality of enhanced infrared images.

[0212] It should be noted that step S1120 is the same as step S230 in embodiment 1, and will not be described in detail here.

[0213] Example 3

[0214] This embodiment provides a special gas leak detection system, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the terms "module", "unit", "subunit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0215] Fig.15 is a framework diagram of a special gas leakage detection system according to an embodiment of the present application. Fig.15 As shown, the special gas leakage detection system includes an image acquisition unit 1510 and a detection unit 1530. The image acquisition unit 1510 is used to acquire an infrared image of the area to be detected; the detection unit 1530 is used to input the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

[0216] Furthermore, the system also includes an image enhancement unit 1520. The image enhancement unit 1520 is used to perform image enhancement processing on the original infrared image to obtain an enhanced infrared image.

[0217] Specifically, the image acquisition unit 1510 includes a first image acquisition module, wherein the first image acquisition module is used to acquire an infrared image of the area to be detected.

[0218] Specifically, the image acquisition unit 1510 includes a second image acquisition module and a first image segmentation module. The second image acquisition module is used to acquire an infrared image sequence of the area to be detected; and the first image segmentation module is used to perform frame processing on the infrared image sequence to obtain a plurality of infrared images.

[0219] Specifically, the image acquisition unit 1510 includes a third image acquisition module, wherein the third image acquisition module is used to acquire an infrared image sequence of the area to be detected.

[0220] Specifically, the image enhancement unit 1520 includes a first gain module, a first filtering module and a first enhancement module. The first gain module is used to perform MSRCR adaptive gain processing on the original infrared image to obtain a first processed image; the first filtering module is used to perform WLS filtering processing on the first processed image to obtain a second processed image; and the first enhancement module is used to perform CLAHE image enhancement processing on the second processed image to obtain an enhanced infrared image.

[0221] Specifically, the image enhancement unit 1520 includes a second gain module, a second filtering module, a second enhancement module, and a second image segmentation module. The second gain module is used to perform MSRCR adaptive gain processing on the original infrared image sequence to obtain a first processed image sequence; the second filtering module is used to perform WLS filtering processing on the first processed image sequence to obtain a second processed image sequence; the second enhancement module is used to perform CLAHE image enhancement processing on the second processed image sequence to obtain an enhanced infrared image sequence; and the second image segmentation module is used to perform frame processing on the enhanced infrared image sequence to obtain a plurality of enhanced infrared images.

[0222] The processing methods of the first gain module and the second gain module are substantially the same as step S410 to step S450 of embodiment 1, and are not described in detail here.

[0223] The processing methods of the first filtering module and the second filtering module are substantially the same as steps S510 to S530 of the first embodiment, and are not described in detail herein.

[0224] The processing methods of the first enhancement module and the second enhancement module are substantially the same as steps S610 to S630 and steps S710 to S740 of Embodiment 1, and are not described in detail herein.

[0225] Specifically, the detection unit 1530 includes a first extraction module, a first recognition module and a first marking module. The first extraction module is used to extract features from the infrared image to obtain image features; the first recognition module is used to determine whether the image features are preset image features; and the first marking module is used to mark the area containing the image features when the image features are preset image features.

[0226] Specifically, the detection unit 1530 includes a convolution module, a second extraction module, a fusion module, a second recognition module, and a second marking module. The convolution module is used to perform convolution processing on the infrared image to obtain a feature layer; the second extraction module is used to perform extraction processing on the feature layer to obtain shallow feature information and deep feature information; the fusion module is used to fuse the shallow feature information and the deep feature information to obtain a feature map; the second recognition module is used to determine whether there is a leakage area in the feature map; and the second marking module is used to mark the feature map if there is a leakage area in the feature map.

[0227] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0228] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting leakage of special gas, characterized in that: include: Acquire an infrared image of the area to be inspected; The infrared image is input into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

2. The method for detecting leakage of special gas according to claim 1, characterized in that: Acquiring an infrared image of the area to be inspected includes: Acquire an infrared image sequence of the area to be detected; The infrared image sequence is subjected to frame processing to obtain a plurality of infrared images.

3. The method for detecting special gas leakage according to claim 1 or 2, characterized in that: Acquiring an infrared image of the area to be inspected includes: Acquire the original infrared image of the area to be detected; Performing image enhancement processing on the original infrared image to obtain an enhanced infrared image; and / or Acquire the original infrared image sequence of the area to be detected; Performing frame processing on the original infrared image sequence to obtain a plurality of original infrared images; Performing image enhancement processing on the plurality of original infrared images to obtain a plurality of enhanced infrared images; and / or Acquire an original infrared image sequence of the area to be detected, wherein the original infrared image sequence includes a plurality of original infrared images; Performing image enhancement processing on the original infrared image sequence to obtain an enhanced infrared image sequence; The enhanced infrared image sequence is subjected to frame processing to obtain a plurality of enhanced infrared images.

4. The method for detecting leakage of special gas according to claim 3, characterized in that: Image enhancement processing includes: Performing MSRCR adaptive gain processing on the original infrared image and / or the original infrared image sequence to obtain a first processed image and / or a first processed image sequence; Performing WLS filtering processing on the first processed image and / or the first processed image sequence to obtain a second processed image and / or a second processed image sequence; The second processed image and / or the second processed image sequence is subjected to CLAHE image enhancement processing to obtain an enhanced infrared image and / or an enhanced infrared image sequence.

5. The method for detecting leakage of special gas according to claim 4, characterized in that: Performing MSRCR adaptive gain processing on the original infrared image and / or the original infrared image sequence to obtain a first processed image and / or a first processed image sequence includes: Processing the original infrared image using a Gaussian filter to obtain a first infrared sub-image; Mapping the first infrared sub-image structure to a logarithmic domain and performing linear stretching processing to obtain a second infrared sub-image; Performing multi-scale Retinex processing on the second infrared sub-image to obtain a third infrared sub-image; Performing color restoration processing on the second infrared sub-image and / or the second infrared sub-image sequence to obtain a fourth infrared sub-image; combining the third infrared sub-image and the fourth infrared sub-image to obtain a first processed image; and / or Performing WLS filtering on the first processed image to obtain a second processed image includes: Calculating the gradients of the first processed image in the horizontal direction and the vertical direction to obtain gradient information; Calculate the weights of each pixel in the horizontal direction and the vertical direction of the first processed image according to the gradient information to obtain weight information; Constructing an objective function and optimizing the solution using an iteration method or a matrix decomposition method according to the gradient information and the weight information to obtain a second processed image; and / or Performing CLAHE image enhancement processing on the second processed image to obtain an enhanced infrared image includes: dividing the second processed image to obtain a plurality of image regions; Performing equalization processing on the plurality of image regions to obtain a plurality of enhanced image regions; A plurality of the enhanced image regions are interpolated and synthesized to obtain an enhanced infrared image.

6. The method for detecting leakage of special gas according to any one of claims 1 to 5, characterized in that: Inputting the infrared image into a pre-trained special gas leak detection model to obtain a special gas leak detection result includes: Performing feature extraction on the infrared image to obtain image features; Determining whether the image feature is a preset image feature; In the case where the image feature is the preset image feature, the area containing the image feature is marked.

7. The method for detecting leakage of special gas according to any one of claims 1 to 5, characterized in that: Inputting the infrared image into a pre-trained special gas leak detection model to obtain a special gas leak detection result includes: Performing convolution processing on the infrared image to obtain a feature layer; Extracting the feature layer to obtain shallow feature information and deep feature information; The shallow feature information and the deep feature information are fused to obtain a feature map; Determine whether there is a leakage area in the characteristic map; In the case where there is a leakage area in the characteristic map, the characteristic map is marked.

8. A special gas leak detection system, characterized in that: include: An image acquisition unit, used to acquire an infrared image of the area to be detected; The detection unit is used to input the infrared image into a pre-trained special gas leakage detection model to obtain a special gas leakage detection result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for detecting a special gas leak according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for detecting leakage of special gas as described in any one of claims 1 to 7 is implemented.

Images