Gas leakage detection method and system based on infrared image

Abstract

The invention relates to a gas leakage detection method and system based on an infrared image, and belongs to the technical field of gas leakage detection.The gas leakage detection method based on the infrared image comprises the steps that a to-be-detected infrared video is acquired; inputting the infrared video into a gas detection model frame by frame so as to output a gas target detection result of each frame of image in the infrared video; in response to the gas target detection result that the gas target exists, intercepting continuous N frames of images from the infrared video, and performing three-channel expansion on the N frames of images to obtain N frames of three-channel graphs; performing moving target detection on the N frames of three-channel graphs to output a moving target detection result; and inputting the three-channel graph into a gas detection model in response to the motion target detection result that the motion target exists, so as to output a gas leakage detection result. The method has the advantages that the data processing amount can be reduced, real-time detection is achieved, and meanwhile the detection accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of gas leakage detection, and in particular, to a gas leakage detection method and system based on infrared images. Background Art

[0002] In the production process of petrochemical plants, finished and semi-finished oils need to be transported inside pipelines, and these pipelines are often in a state of high temperature and high pressure. This environment poses extremely high requirements on the tightness and safety of the pipelines. However, at key positions such as when loading and opening valves and pipeline connections, due to various factors, such as pipeline aging, loose connections, improper operations, etc., leakage problems are likely to occur. Once a leakage problem occurs, it will not only cause waste of oil products and environmental pollution, but may also trigger serious accidents such as fires and explosions. Therefore, quickly discovering and detecting the leakage problem of refined oil is crucial for the petrochemical industry.

[0003] Currently, infrared image technology is usually used to detect gas leakage in pipelines. Infrared image technology utilizes the principle of infrared thermal imaging. Through an infrared radiation detector, it receives the infrared rays radiated by the object to be detected. After a series of conversions and processes, the infrared radiation energy is converted into a visual image, and then visual detection means are used to detect the visual image to determine whether there is a gas leakage problem in the pipeline. However, in order to improve the accuracy of infrared image visual detection, relatively complex image processing algorithms are usually adopted, which require a large amount of calculations. For complex pipeline systems or large-scale detection areas, the required amount of calculation may increase sharply. How to reduce the amount of data calculation while improving the accuracy of gas leakage detection is an urgent problem to be solved currently. Summary of the Invention

[0004] In order to reduce the amount of data calculation while improving the accuracy of gas leakage detection, the present application provides a gas leakage detection method and system based on infrared images.

[0005] In a first aspect, a gas leakage detection method based on infrared images provided by the present application adopts the following technical solutions:

[0006] A gas leakage detection method based on infrared images includes:

[0007] Obtain an infrared video to be detected;

[0008] Input the infrared video frame by frame into a gas detection model to output the gas target detection result of each frame image in the infrared video;

[0009] In response to the gas target detection result indicating the existence of a gas target, intercept N consecutive frame images from the infrared video and perform three-channel expansion on the N frame images to obtain N three-channel images;

[0010] Perform moving target detection on the N-frame three-channel images to output the moving target detection result;

[0011] In response to the moving target detection result indicating the existence of a moving target, input the three-channel images into a gas detection model to output the gas leakage detection result.

[0012] Optionally, the obtaining of the infrared video to be detected specifically includes:

[0013] Based on the preset preset positions within the range to be detected, set the rotation speed of the infrared camera;

[0014] Output a control command according to the rotation speed and the residence duration of each preset position; wherein, the control command is used to control the infrared camera to sequentially sample the preset preset positions;

[0015] Obtain the infrared video returned by the infrared camera.

[0016] Optionally, it further includes a training step of training a preset neural network to obtain a gas detection model; the preset neural network includes a feature extraction module, an inverse attribute change module, a feature processing module, a first prediction module, and a second prediction module;

[0017] The feature extraction module includes an attention sub-module and a downsampling layer, and is used to receive a sample image and perform pyramid feature extraction on the sample image to obtain a downsampled feature map;

[0018] The inverse attribute transformation module, connected to the feature extraction module, includes a convolutional layer with adjustable convolution kernel size, and is used to receive the downsampled feature map and perform a convolution operation with size preservation on the downsampled feature map to obtain an inverse attribute feature map;

[0019] The feature processing module, connected to the inverse attribute transformation module, includes an upsampling layer, and is used to receive the inverse attribute feature map and perform bottom-up path enhancement processing on the inverse attribute feature map to obtain an enhanced feature map;

[0020] The first prediction module, connected to the inverse attribute transformation module, is used to output a prediction result according to the inverse attribute feature map;

[0021] The second prediction module, connected to the feature processing module, is used to output a prediction result according to the enhanced feature map;

[0022] Wherein, the downsampled feature map and the inverse attribute feature map have the same number of channels and size, and the resolution of the first prediction module is greater than the resolution of the second prediction module.

[0023] Optionally, the training step includes:

[0024] Obtain a training dataset and label corresponding prediction labels for each sample image in the training dataset;

[0025] Input the sample image into the feature extraction module, and set the initial value of the convolution kernel size of the convolutional layer in the inverse attribute transformation module according to the size and number of channels of the downsampled feature map output by the feature extraction module;

[0026] Input the downsampled image into the inverse attribute transformation module to output an inverse attribute feature map, and input the inverse attribute feature map into the first prediction module for prediction to obtain a prediction result;

[0027] Based on the prediction label and the prediction result of the sample image, for each pixel in each channel of the downsampled feature map, iteratively update the initial value of the convolution kernel size of the convolutional layer in the inverse attribute transformation module to obtain a gas detection model.

[0028] Optionally, the method of intercepting N consecutive frames of images from the infrared video and performing three-channel expansion on the N frames of images to obtain N frames of three-channel images specifically includes:

[0029] Normalize the gray values of the N frames of images respectively to obtain normalized gray values;

[0030] Calculate the brightness component in the HSV color space based on the normalized gray value, and preset the hue component and saturation component in the HSV color space;

[0031] Based on the brightness component, hue component, and saturation component, perform color space conversion on the N frames of images to obtain the intensity values of three channels in the RGB color space, so as to generate N frames of three-channel images respectively.

[0032] Optionally, the method of performing moving target detection on the N frames of three-channel images to output a moving target detection result specifically includes:

[0033] Perform differential summation processing on the N frames of three-channel images to generate a difference contour map corresponding to each frame of three-channel image;

[0034] Accumulate all the difference contour maps to extract the overall difference contour of the N frames of three-channel images;

[0035] Judge whether the area of the overall difference contour is greater than a preset area threshold. If so, determine that the moving target detection result is that there is a moving target in the corresponding three-channel image; if not, determine that the moving target detection result is that there is no moving target in the corresponding three-channel image.

[0036] Optionally, the method of performing differential summation processing on the N frames of three-channel images to generate a difference contour map corresponding to each frame of three-channel image specifically includes:

[0037] Calculate the difference values between the last three-channel image frame and each of the remaining three-channel image frames in N frames of three-channel images;

[0038] Sum up the difference values of the same pixel points in all N frames of three-channel images to obtain a differential summation result;

[0039] Perform a binarization operation on the differential summation result of each pixel point to obtain a binarized image;

[0040] Perform morphological processing on the binarized image to obtain a difference contour map.

[0041] In a second aspect, the present application provides a gas leakage detection system based on infrared images, adopting the following technical solution:

[0042] A gas leakage detection system based on infrared images, comprising:

[0043] A data acquisition module, configured to acquire an infrared video to be detected;

[0044] A gas target detection module, configured to input the infrared video frame by frame into a gas detection model to output the gas target detection result of each frame image in the infrared video;

[0045] A channel expansion module, configured to, in response to the gas target detection result indicating the presence of a gas target, intercept N consecutive frames of images from the infrared video and perform three-channel expansion on the N frames of images to obtain N frames of three-channel images;

[0046] A moving target detection module, configured to perform moving target detection on the N frames of three-channel images to output a moving target detection result;

[0047] A gas leakage detection module, configured to, in response to the moving target detection result indicating the presence of a moving target, input the three-channel image into the gas detection model to output a gas leakage detection result.

[0048] In a third aspect, the present application provides an image processing device, adopting the following technical solution:

[0049] An image processing device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program of any of the above methods.

[0050] In a fourth aspect, the present application provides a computer-readable storage medium, adopting the following technical solution:

[0051] A computer-readable storage medium, comprising a computer program stored therein that can be loaded and executed by a processor for any of the above methods.

[0052] In summary, the present application includes the following beneficial technical effects:

[0053] 1. First, input the infrared video frame by frame into the gas detection model to quickly and preliminarily detect gas targets in each frame of the image, which can rapidly screen out the frames that may have gas targets, greatly reducing the amount of data for subsequent processing. When a gas target is detected, this technology intercepts N consecutive frames of images from the infrared video and performs three-channel expansion on these images. This not only retains the key frames directly related to gas leakage but also increases the amount of information in the images by adding channel information, which helps for more accurate subsequent analysis. Then, perform moving target detection on the N three-channel images. Since gas leakage is usually accompanied by the flow or diffusion of substances, the detection of moving targets can further confirm the existence of leakage and exclude static interferences. When the moving target detection result indicates the presence of a moving target, input the three-channel images into the gas detection model again for fine analysis to output the final gas leakage detection result. Thus, it can not only reduce the amount of data processing to achieve real-time detection but also improve the accuracy of detection.

[0054] 2. By performing differential summation processing on the N three-channel images, generate the difference contour map corresponding to each frame of the three-channel image. The differential summation operation can highlight the changes between adjacent frames, thus effectively extracting the contours of moving objects, which can eliminate the interference of the static background to a certain extent and eliminate the differences of pixel points at the same position between different frames caused by image jitter. Then, accumulate all the difference contour maps to extract the overall difference contour of the N three-channel images. By judging whether the area of the overall difference contour is greater than the preset area threshold, determine the moving target detection result, which can avoid misjudgment caused by the changes of individual pixel points to a certain extent and ensure that only when the difference contour reaches a certain scale is it considered that there is a moving target, achieving effective detection of moving targets and helping to better identify the leaked gas. Description of the Drawings

[0055] Figure 1 is the overall flowchart of the gas leakage detection method in one embodiment of the present application.

[0056] Figure 2 is the structural schematic diagram of the gas detection model in one embodiment of the present application.

[0057] Figure 3 is the schematic diagram of the inverse attribute transformation module in one embodiment of the present application.

[0058] Figure 4 is the method flowchart of the gas detection model training in one embodiment of the present application.

[0059] Figure 5 is the flowchart of the channel expansion method in one embodiment of the present application.

[0060] Figure 6It is a flowchart of a method for detecting moving targets in one embodiment of the present application.

[0061] Figure 7 It is a block diagram of a gas leakage detection system in one embodiment of the present application. Detailed implementation manners

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] An embodiment of the present application discloses a gas leakage detection method based on infrared images. Refer to Figure 1 , a gas leakage detection method based on infrared images, includes:

[0064] Step S101: Obtain an infrared video to be detected;

[0065] Step S102: Input the infrared video frame by frame into a gas detection model to output the gas target detection results of each frame image in the infrared video;

[0066] In this embodiment, the gas detection model uses the YOLOv5 model. In other embodiments, other versions of the YOLO model can also be selected for the gas detection model. YOLO (You Only Look Once) is an object detection algorithm, the principle of which is to treat the object detection task as a regression problem, directly perform regression processing on the picture at one time, and obtain the coordinates of the bounding box, the localization confidence, and the class probability.

[0067] Step S103: In response to the gas target detection result indicating the existence of a gas target, intercept N consecutive frames of images from the infrared video, and perform three-channel expansion on the N frames of images to obtain N frames of three-channel images;

[0068] Among them, intercepting N consecutive frames of images from the infrared video means intercepting N consecutive frames of images starting from the current frame image where a gas target is detected.

[0069] It should be noted that the infrared video captures single-channel images. In step 102, the single-channel images are directly input into the gas detection model. The gas detection model will copy each frame image in the single-channel infrared video three times to make it have the same spatial shape as the visible light image, that is, convert it into a pseudo three-channel image, so as to receive the input in the way expected by the gas detection model. In this way, directly inputting single-channel frame images into the gas detection model in step S102 can reduce the data processing volume, improve the detection efficiency, and achieve real-time detection. However, it may cause problems such as the mismatch between weights and convolution kernels, the decline of feature extraction ability, and the difference in data distribution during the data processing of the gas detection model, which is likely to lead to the decline of the performance of the gas detection model.

[0070] To solve the above problems, step S103 is specially set. That is, in step S102, the single-channel images are first used for preliminary detection. Since the processing speed is fast, the target area can be quickly screened. When there is a possible leakage area, then step S103 is executed to perform more detailed processing by expanding the channels, effectively ensuring the accuracy of gas leakage identification. It should be understood that although single-channel images can be quickly detected, they may be limited by insufficient information and it is difficult to capture all details. While three-channel images provide richer information, can more comprehensively reflect the target features, and improve the accuracy of model recognition. Thus, the requirements of both real-time performance and recognition accuracy are met.

[0071] Step S104: Perform moving target detection on the N three-channel images to output the moving target detection result;

[0072] It should be understood that gas leakage usually appears in the form of gas diffusion, and this diffusive motion state is a direct sign of the occurrence of leakage. By detecting the moving targets in the three-channel images, it is convenient to quickly lock the leakage area.

[0073] Step S105: In response to the moving target detection result indicating the existence of a moving target, input the three-channel image into the gas detection model to output the gas leakage detection result.

[0074] In the above embodiments, the infrared video is input frame by frame into the gas detection model to quickly and preliminarily detect the gas targets in each frame of the image, which can quickly screen out the frames that may have gas targets and greatly reduce the amount of data to be processed subsequently. When a gas target is detected, the technology intercepts N consecutive frames of images from the infrared video and performs three-channel expansion on these images, which not only retains the key frames directly related to gas leakage but also increases the amount of information in the images by adding channel information, contributing to more accurate subsequent analysis. Then, motion target detection is performed on the N-frame three-channel images. Since gas leakage is usually accompanied by the flow or diffusion of substances, the detection of motion targets can further confirm the existence of the leakage and exclude static interference. When the motion target detection result indicates the existence of motion targets, the three-channel images are input into the gas detection model again for fine analysis to output the final gas leakage detection result. Thus, it is possible to both reduce the amount of data processing to achieve real-time detection and improve the accuracy of detection.

[0075] As an embodiment of step S101, step S101 specifically includes:

[0076] Step S1011: Set the rotation speed of the infrared camera based on the preset preset points within the range to be detected;

[0077] Step S1012: Output a control command according to the rotation speed and the residence duration of each preset point; wherein, the control command is used to control the infrared camera to sequentially sample the preset preset points;

[0078] Step S1013: Obtain the infrared video returned by the infrared camera.

[0079] In the above embodiments, setting the rotation speed of the infrared camera based on the preset preset points facilitates the effective coverage of multiple key points within the range to be detected. Outputting a control command according to the rotation speed and the residence duration of each preset point can ensure that the infrared camera precisely and orderly samples each preset point. By obtaining the infrared video returned by the infrared camera, the status of the range to be detected can be monitored in real time. Since the multi-point detection method is adopted, it is possible to reduce the number of cameras while ensuring the requirements for the detection range and accuracy.

[0080] Refer to Figure 2 、 3 As a further embodiment of the gas leakage detection method based on infrared images, the gas leakage detection method based on infrared images further includes a training step of training a preset neural network to obtain a gas detection model; the preset neural network includes a feature extraction module, an inverse attribute change module, a feature processing module, a first prediction module, and a second prediction module;

[0081] The feature extraction module includes an attention sub-module and a downsampling layer, which are used to receive a sample image and perform pyramid feature extraction on the sample image to obtain a downsampled feature map;

[0082] Referring to Figure 2 , Figure 2 in which, the input is the input sample image, and C1, C2, C3, C4, and C5 are feature maps with pyramidically decreasing number of channels.

[0083] The inverse attribute transformation module, connected to the feature extraction module, includes a convolutional layer with adjustable convolutional kernel size, which is used to receive the downsampled feature map and perform a convolutional operation with size preservation on the downsampled feature map to obtain an inverse attribute feature map;

[0084] The feature processing module, connected to the inverse attribute transformation module, includes an upsampling layer, which is used to receive the inverse attribute feature map and perform a bottom-up path enhancement process on the inverse attribute feature map to obtain an enhanced feature map;

[0085] Referring to Figure 2 , the feature processing module partially adopts the bottom-up path enhancement structure of PANet (Path Aggregation Network).

[0086] The first prediction module, connected to the inverse attribute transformation module, is used to output a prediction result according to the inverse attribute feature map;

[0087] The second prediction module, connected to the feature processing module, is used to output a prediction result according to the enhanced feature map;

[0088] Among them, the downsampled feature map and the inverse attribute feature map have the same number of channels and size, and the resolution of the first prediction module is greater than that of the second prediction module.

[0089] Among them, the second prediction module is three prediction heads originally included in a preset neural network. These three prediction heads have different resolutions (80×80, 40×40, and 20×20 respectively). It should be understood that the preset neural network usually uses multi-scale prediction heads (also known as multi-scale fusion or multi-level prediction) to capture context information at different scales, thereby improving the performance of the model. Each prediction head usually corresponds to an output with a specific resolution, and different prediction heads need to obtain inputs from feature maps at different levels.

[0090] In this embodiment, the resolution of the first prediction module is 160×160. It should be noted that the number and size of the prediction heads in the first prediction module and the second prediction module can be set according to actual needs and are not specifically limited here.

[0091] It should be understood that in this embodiment, a first prediction module with a higher resolution is added. Therefore, it is necessary to ensure that there is a corresponding feature map with a higher resolution available for its use to achieve the resolution required by the first prediction module. In this embodiment, by inputting the downsampled feature map output by the feature extraction module into the inverse attribute transformation module, a convolution operation with size preservation is performed on the downsampled feature map, thereby obtaining an inverse attribute feature map with a higher resolution to adapt to the first prediction module, thus realizing the detection of a feature map with a higher resolution.

[0092] It should also be noted that the prediction results output by the first prediction module and the second prediction module are presented in the form of regression bounding boxes, that is, the prediction results include the coordinates and sizes of the prediction boxes. Specifically, before generating the prediction results, it is necessary to first generate Anchor boxes (anchor boxes) through k-means clustering to form candidate bounding boxes. And the first prediction module and the second prediction module with different resolutions are defined at multiple different scales to adapt to small, medium, and large targets.

[0093] Refer to Figure 3 、 4 , as an implementation manner of the training step, the training step specifically includes:

[0094] Step S201: Obtain a training data set and label the corresponding prediction labels for each sample image in the training data set;

[0095] Step S202: Input the sample image into the feature extraction module, and set the initial value of the convolution kernel size of the convolution layer in the inverse attribute transformation module according to the size and number of channels of the downsampled feature map output by the feature extraction module;

[0096] Among them, the initial values of the convolution kernel sizes of the convolution layers in the inverse attribute transformation module set for all channels are the same.

[0097] Step S203: Input the downsampled image into the inverse attribute transformation module to output an inverse attribute feature map, and input the inverse attribute feature map into the first prediction module for prediction to obtain a prediction result;

[0098] Step S204: Based on the prediction label and the prediction result of the sample image, for each pixel in each channel of the downsampled feature map, iteratively update the initial value of the convolution kernel size of the convolution layer in the inverse attribute transformation module to obtain a gas detection model.

[0099] Combined with Figure 3 , the initial value of the convolution kernel of the convolution layer in the inverse attribute transformation module is represented as H∈R H*W*K*K*G , where H and W represent the height and width of the feature map, K is the convolution kernel size, and G represents the number of channels. During training, each pixel x in each channeli , i ∈ R c The sizes of the convolution kernels of the corresponding convolutional layers are iteratively updated based on the initial values, that is, for each pixel x i,i ∈ R c The convolution kernel of the corresponding convolutional layer is denoted as H i,j,,,g ∈ R k *K, g = 1, 2, …, G. Each pixel x obtained by iterative training i , i ∈ R c The convolution kernels of the corresponding convolutional layers are shared (kept consistent) across channels. Therefore, the special figure output by the inverse attribute transformation module is

[0100] Refer to Figure 5 , as an implementation manner of step S103, step S103 specifically includes:

[0101] Step S1031: Normalize the grayscale values of N frames of images respectively to obtain normalized grayscale values;

[0102] Step S1032: Calculate the luminance component in the HSV color space based on the normalized grayscale values, and preset the hue component and saturation component in the HSV color space;

[0103] Step S1033: Based on the luminance component, hue component, and saturation component, perform color space conversion on N frames of images to obtain the intensity values of three channels in the RGB color space, so as to generate N frames of three-channel images respectively.

[0104] Refer to Figure 6 , as an implementation manner of step S104, step S104 specifically includes:

[0105] Step S1041: Perform differential summation processing on N frames of three-channel images to generate a difference contour map corresponding to each frame of three-channel image;

[0106] Step S1042: Accumulate all the difference contour maps to extract the overall difference contour of N frames of three-channel images;

[0107] Specifically, all the difference contour maps are denoted as φ3(absdiff1, absdiff2, absdiff3,..., diff n-1 ), where diff n-1 is the differential result absdiff of the (n - 1)th time, and the overall difference contour obtained after accumulating the differential results is

[0108] Step S1043: Determine whether the area of the overall difference contour is greater than a preset area threshold. If so, execute Step S1044; if not, execute Step S1045;

[0109] Step S1044: Determine that there is a moving target in the corresponding three-channel image for the moving target detection result;

[0110] Step S1045: Determine that there is no moving target in the corresponding three-channel image for the moving target detection result.

[0111] In the above implementation, by performing differential summation processing on N frames of three-channel images to generate a difference contour map corresponding to each frame of the three-channel image, the differential summation operation can highlight the changes between adjacent frames, thereby effectively extracting the contour of the moving object, being able to eliminate the interference of the static background to a certain extent, and eliminating the difference of pixel points at the same position between different frames caused by the shaking of the picture. Then, all the difference contour maps are accumulated to extract the overall difference contour of the N frames of three-channel images. By judging whether the area of the overall difference contour is greater than the preset area threshold, the moving target detection result is determined, avoiding misjudgment caused by the change of individual pixel points to a certain extent, ensuring that only when the difference contour reaches a certain scale is it considered that there is a moving target, realizing the effective detection of the moving target, and helping to better identify the leaked gas.

[0112] As an implementation of Step S1041, Step S1041 specifically includes:

[0113] Step S10411: Calculate the difference value between the last frame of the N frames of three-channel images and each of the remaining frames of the three-channel images;

[0114] Among them, the N frames of three-channel images are represented as where Xn is the nth frame, that is, the last frame. Then the difference value of each pixel value of every two adjacent frames of the N frames of three-channel images is represented as φ2(X i -X n ) i ∈ (1, n - 1), and X i -X n refers to taking the absolute value of the difference between the pixel values of the X i frame and the X n frame (the last frame).

[0115] Step S10412: Sum the difference values of the same pixel points in all N frames of three-channel images to obtain the differential summation result;

[0116] Specifically, sum the difference values obtained by summing the frame differences n - 1 times for the difference value between the first frame and the last frame, that is, the differential summation result

[0117] Step S10413: Perform a binarization operation on the differential summation result of each pixel to obtain a binarized image;

[0118] Specifically, the binarized image is where th is a preset binarization threshold, and th can be set according to the actual situation without specific limitation here.

[0119] Step S10414: Perform morphological processing on the binarized image to obtain a differential contour map.

[0120] Among them, the morphological processing includes erosion operation and dilation operation. For example, in this embodiment, the binarized image can be subjected to an erosion operation with a kernel of [t, t / 2] for image processing, and then a dilation operation with [2t, t / 2] for image processing. It should be noted that the number and order of the dilation operation and the erosion operation can be adjusted according to the actual situation without specific limitation here.

[0121] In the above embodiment, the difference values between the last frame and each of the remaining frames in the N-frame three-channel images are calculated, and then the difference values of the same pixel points in all N frames are summed to obtain a differential summation result, so as to accumulate the changes of the pixel points at the same position in all frames, thereby highlighting the continuously changing areas. Then, a binarization operation is performed on the differential summation result of each pixel point to further simplify the image information, separate the differential contour from the background, and greatly reduce the data volume of subsequent processing. Finally, a differential contour map is extracted through morphological processing. The morphological processing can eliminate the noise and fine structures in the binarized image, making the differential contour clearer and more accurate.

[0122] In addition, an embodiment of the present application discloses a gas leakage detection system based on infrared images. The gas leakage detection system based on infrared images can be applied to an image processing device, which is a schematic diagram of the architecture of the image processing device provided by the embodiment of the present invention for implementing the above method. In this embodiment, the image processing device may include a gas leakage detection system based on infrared images, a machine-readable storage medium, and a processor.

[0123] In this embodiment, the machine-readable storage medium and the processor may be located in the image processing device and are separately arranged. The machine-readable storage medium may also be independent of the image processing device and be accessed by the processor. The gas leakage detection system based on infrared images may include multiple functional modules stored in the machine-readable storage medium, such as the various software functional modules included in the gas leakage detection system based on infrared images. When the processor executes the computer program corresponding to the software functional module in the gas leakage detection system based on infrared images, the gas leakage detection system based on infrared images provided by the foregoing method embodiment is implemented.

[0124] In this embodiment, the image processing device may include one or more processors. The processor may process information and / or data related to a service request to perform one or more functions described in the present invention. In some embodiments, the processor may include one or more processing engines (e.g., a single-core processor or a multi-core processor). By way of example only, the processor may include one or more hardware processors such as a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, or the like, or any combination thereof.

[0125] The machine-readable storage medium may store data and / or instructions. In some embodiments, the machine-readable storage medium may store acquired data or materials. In some embodiments, the machine-readable storage medium may store data and / or instructions for the image processing device to execute or use, and the image processing device may implement the exemplary methods described in this application by executing or using the data and / or instructions. In some embodiments, the machine-readable storage medium may include a mass storage device, a removable storage device, a volatile read / write memory, a read-only memory (ROM), or the like, or any combination of the foregoing. Exemplary mass storage devices may include magnetic disks, optical disks, solid state disks, etc. Exemplary removable storage devices may include flash drives, floppy disks, optical disks, memory cards, compact disks, magnetic tapes, etc. Exemplary volatile read / write memories may include random access memory (RAM). Exemplary random access memories may include dynamic RAM, double data rate synchronous dynamic RAM, static RAM, thyristor RAM, and zero capacitor RAM, etc. Exemplary ROMs may include masked ROM, programmable ROM, erasable programmable ROM, electrically erasable programmable ROM, compact disk ROM, and digital versatile disk ROM, etc.

[0126] Among them, the gas leakage detection system based on infrared images included in the image processing device may include one or more software function modules. The software function modules may be programs and instructions stored in the machine-readable storage medium, and when executed by the corresponding processor, are used to implement the above methods. For example, when executed by the processor of a drone, they are used to implement the method steps executed by the above drone, or when executed by the image processing device, they are used to implement the method steps executed by the above image processing device.

[0127] Refer in detail to Figure 7 , an embodiment of the present application discloses a gas leakage detection system based on infrared images. A gas leakage detection system based on infrared images includes:

[0128] A data acquisition module for acquiring an infrared video to be detected;

[0129] A gas target detection module for inputting the infrared video frame by frame into a gas detection model to output the gas target detection result of each frame image in the infrared video;

[0130] A channel expansion module for, in response to the gas target detection result indicating the existence of a gas target, intercepting N consecutive frame images from the infrared video and performing three-channel expansion on the N frame images to obtain N three-channel images;

[0131] A moving target detection module for performing moving target detection on the N three-channel images to output a moving target detection result;

[0132] A gas leakage detection module for, in response to the moving target detection result indicating the existence of a moving target, inputting the three-channel image into the gas detection model to output a gas leakage detection result.

[0133] A gas leakage detection system based on infrared images provided by the present application can implement the above-mentioned gas leakage detection method based on infrared images, and the specific working process of a gas leakage detection system based on infrared images can refer to the corresponding process in the above method embodiments.

[0134] It should be noted that in the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] The present invention also discloses a computer-readable storage medium, which includes a computer program stored thereon that can be loaded and executed by a processor as in any of the above methods.

[0136] In the embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. Also, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0137] In addition, in each embodiment of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0138] The above are all preferred embodiments of the present application, and do not limit the protection scope of the present application accordingly. Any feature disclosed in this specification (including the abstract and drawings), unless specifically described, can be replaced by other equivalent or alternative features with similar purposes. That is, unless specifically described, each feature is only an example of a series of equivalent or similar features.

Claims

1. A gas leakage detection method based on infrared images, characterized in that, Including: Obtain an infrared video to be detected; Input the infrared video frame by frame into a gas detection model to output the gas target detection result of each frame image in the infrared video; In response to the gas target detection result indicating the existence of a gas target, intercept N consecutive frame images from the infrared video, and perform three-channel expansion on the N frame images to obtain N three-channel images; Perform moving target detection on the N three-channel images to output a moving target detection result; In response to the moving target detection result indicating the existence of a moving target, input the three-channel image into the gas detection model to output a gas leakage detection result.

2. The method according to claim 1, wherein The obtaining of the infrared video to be detected specifically includes: Based on preset preset points within the range to be detected, set the rotation speed of the infrared camera; Output a control command according to the rotation speed and the residence duration of each preset point; wherein, the control command is used to control the infrared camera to sequentially sample the preset preset points; Obtain the infrared video returned by the infrared camera.

3. The method according to claim 1, wherein It also includes a training step of training a preset neural network to obtain a gas detection model; the preset neural network includes a feature extraction module, an inverse attribute change module, a feature processing module, a first prediction module, and a second prediction module; The feature extraction module includes an attention sub-module and a downsampling layer, and is used to receive a sample image and perform pyramid feature extraction on the sample image to obtain a downsampled feature map; The inverse attribute transformation module, connected to the feature extraction module, includes a convolutional layer with adjustable convolutional kernel size, and is used to receive the downsampled feature map and perform a convolution operation with size preservation on the downsampled feature map to obtain an inverse attribute feature map; The feature processing module, connected to the inverse attribute transformation module, includes an upsampling layer, and is used to receive the inverse attribute feature map and perform a bottom-up path enhancement process on the inverse attribute feature map to obtain an enhanced feature map; The first prediction module, connected to the inverse attribute transformation module, is used to output a prediction result according to the inverse attribute feature map; The second prediction module, connected to the feature processing module, is used to output a prediction result according to the enhanced feature map; Wherein, the downsampled feature map and the inverse attribute feature map have the same number of channels and size, and the resolution of the first prediction module is greater than the resolution of the second prediction module.

4. The method according to claim 3, wherein The training step includes: Obtain a training data set, and label corresponding prediction labels for each sample image in the training data set; Input the sample image into the feature extraction module, and set the initial value of the convolutional kernel size of the convolutional layer in the inverse attribute transformation module according to the size and number of channels of the downsampled feature map output by the feature extraction module; Input the downsampled image into the inverse attribute transformation module to output an inverse attribute feature map, and input the inverse attribute feature map into the first prediction module for prediction to obtain a prediction result; Based on the prediction label and the prediction result of the sample image, iteratively update the initial value of the convolutional kernel size of the convolutional layer in the inverse attribute transformation module for each pixel in each channel of the downsampled feature map to obtain a gas detection model.

5. The method according to claim 1, wherein Intercept N consecutive frames of images from the infrared video, and perform three-channel expansion on the N frames of images to obtain N frames of three-channel images, specifically including: Perform normalization processing on the gray values of the N frames of images respectively to obtain normalized gray values; Calculate the brightness component in the HSV color space based on the normalized gray values, and preset the hue component and saturation component in the HSV color space; Based on the brightness component, hue component, and saturation component, perform color space conversion on the N frames of images to obtain the intensity values of the three channels in the RGB color space, so as to generate N frames of three-channel images respectively.

6. The method according to claim 1, wherein Perform moving target detection on the N frames of three-channel images to output the moving target detection result, specifically including: Perform differential summation processing on the N frames of three-channel images to generate a difference contour map corresponding to each frame of three-channel image; Accumulate all the difference contour maps to extract the overall difference contour of the N frames of three-channel images; Judge whether the area of the overall difference contour is greater than a preset area threshold. If so, determine that the moving target detection result is that there is a moving target in the corresponding three-channel image; if not, determine that the moving target detection result is that there is no moving target in the corresponding three-channel image.

7. The method according to claim 6, wherein Perform differential summation processing on the N frames of three-channel images to generate a difference contour map corresponding to each frame of three-channel image, specifically including: Calculate the difference values between the last frame of the N frames of three-channel images and each of the other frames of three-channel images; Sum up the difference values of the same pixel points in all N frames of three-channel images to obtain the differential summation result; Perform a binarization operation on the differential summation result of each pixel point to obtain a binarized image; Perform morphological processing on the binarized image to obtain a difference contour map.

8. A gas leakage detection system based on infrared images, characterized in that, Including: A data acquisition module for acquiring the infrared video to be detected; A gas target detection module for inputting the infrared video frame by frame into the gas detection model to output the gas target detection result of each frame of image in the infrared video; A channel expansion module for, in response to the gas target detection result indicating the existence of a gas target, intercepting N consecutive frames of images from the infrared video and performing three-channel expansion on the N frames of images to obtain N frames of three-channel images; A moving target detection module for performing moving target detection on the N frames of three-channel images to output the moving target detection result; A gas leakage detection module for, in response to the moving target detection result indicating the existence of a moving target, inputting the three-channel image into the gas detection model to output the gas leakage detection result.

9. An image processing apparatus, characterized in that: Including a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program of any one of the methods in claims 1-7.

10. A computer-readable storage medium, characterized in that, Including a computer program stored that can be loaded and executed by a processor for any one of the methods in claims 1-7.

Images