Gas leakage detection method, program product and equipment

Abstract

The invention discloses a gas leakage detection method, a program product and equipment, and the method comprises the steps: obtaining a collected continuous frame infrared image, and obtaining a multi-frame visible light image corresponding to the continuous frame infrared image; forming infrared input data of a pre-trained gas leakage detection model based on the continuous frames of infrared images, and forming visible light input data of the pre-trained gas leakage detection model based on multiple frames of visible light images; based on the infrared input data and the visible light input data, a gas fusion feature map is obtained through a gas feature extraction network in the gas leakage detection model; and based on the gas fusion feature map, identifying gas leakage position data through a gas position identification network in the gas leakage detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of gas detection, and in particular to a gas leakage detection method, a computer program product and a gas leakage detection device. Background Art

[0002] Gas leak detection is an important part of ensuring production safety, preventing environmental pollution and optimizing resource utilization. With the deepening of oil and gas extraction activities and the continuous advancement of technology, gas leak detection methods have also undergone a significant evolution from traditional to modern, forming a diversified technical system. Gas leak detection in existing technologies mainly relies on manual inspections, chemical reagent detection, and physical methods such as acoustics and vibration. Although these methods can detect leaks to a certain extent, they have many limitations. Manual inspections are inefficient and difficult to cover all areas; although chemical reagent detection is accurate, the operation is complicated and the cost is high; acoustic and vibration detection are easily affected by environmental noise, affecting detection accuracy. In addition, some high-end detection technologies such as laser scanning and fiber optic sensing have high precision and high sensitivity, but the equipment is complex and the cost is high, making it difficult to promote and apply. Summary of the invention

[0003] In order to solve the existing technical problems, the present invention provides a gas leakage detection method, a computer program product and a gas leakage detection device, which improve the accuracy of leakage detection of various gases.

[0004] In a first aspect, a gas leakage detection method is provided, comprising: acquiring collected continuous frame infrared images, and acquiring multiple frames of visible light images corresponding to the continuous frame infrared images; forming infrared input data of a pre-trained gas leakage detection model based on the continuous frame infrared images, and forming visible light input data of the pre-trained gas leakage detection model based on multiple frames of the visible light images; obtaining a gas fusion feature map through a gas feature extraction network in the gas leakage detection model based on the infrared input data and the visible light input data; and identifying gas leakage position data through a gas position identification network in the gas leakage detection model based on the gas fusion feature map.

[0005] In a second aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the gas leakage detection method as described in any embodiment of the present application is implemented.

[0006] In a third aspect, a gas leakage detection device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas leakage detection method as described in any embodiment of the present application.

[0007] The embodiment of the present application forms infrared input data of a pre-trained gas leakage detection model based on continuous frame infrared images, and more gas temperature information can be provided through the infrared input data, and forms visible light input data of a pre-trained gas leakage detection model based on multiple frames of visible light images, and more background features, color features, and gas spatial change morphological features can be provided through the visible light input data; the gas leakage detection model first outputs an infrared feature map and a visible light feature map respectively, and then fuses the infrared feature map and the visible light feature map to obtain a gas fusion feature map, which includes infrared features and visible light features. More gas temperature information can be highlighted through the infrared features, and more background information and color information can be highlighted through the visible light features, so that the leakage location points of various gases can be more accurately detected through the gas position recognition network, thereby improving the accuracy of leakage detection of various gases. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is an application environment diagram of a gas leakage detection method in an embodiment;

[0009] Figure 2 is a flow chart of a gas leakage detection method in one embodiment;

[0010] Figure 3 A flow chart of generating a gas fusion characteristic map in a gas leakage detection method in one embodiment;

[0011] Figure 4 A schematic diagram of a network structure of a gas leakage detection model in an embodiment;

[0012] Figure 5 is a flow chart of training a gas leak detection model in one embodiment;

[0013] Figure 6 is a schematic diagram of a gas leakage detection device in one embodiment;

[0014] Figure 7 Schematic diagram of a gas leakage detection device in one embodiment. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0017] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0018] See also Figure 1 , is an application environment diagram of a gas leakage detection method in an embodiment. The gas leakage detection method is applied to a gas leakage detection device 10, and the gas leakage detection device 10 includes an image acquisition device 12, a processor 13 and a memory 14. The image acquisition device 12 is used to acquire continuous frame infrared images and continuous frame visible light images in front of the gas leakage detection device 10, and the processor 13 processes the continuous frame infrared images and continuous frame visible light images acquired by the image acquisition device 12 to identify the gas area in the image. The memory 14 is used to store programs and data corresponding to the implementation of the gas leakage detection method.

[0019] The gas leakage detection device 10 includes but is not limited to a handheld detection device, a non-handheld autonomously movable detection device, and a non-handheld non-autonomously movable detection device. Among them, the handheld detection device includes but is not limited to a handheld imaging device with an infrared thermal imaging function and a handheld imaging device with a visible light imaging function. Among them, the non-handheld autonomously movable detection device includes but is not limited to an autonomously movable detection device with an infrared thermal imaging function and an autonomously movable detection device with a visible light imaging function. The non-handheld non-autonomously movable detection device includes but is not limited to a non-handheld and non-autonomously movable device with an infrared thermal imaging function and a non-handheld and non-autonomously movable device with a visible light imaging function. Among them, through the handheld detection device and the non-handheld autonomously movable detection device, the user can perform gas leakage detection on the current scene in motion during the movement, and can also perform gas leakage detection on a fixed scene. The non-handheld non-autonomously movable detection device can be used to perform gas leakage detection in a fixed scene. Therefore, the gas leakage detection method provided in the present application can be applied to various complex motion change scenes and fixed scenes.

[0020] The image acquisition device 12 may be a combination of one or more sensors. The image acquisition device 12 may be a monocular vision sensor or a multi-vision sensor. For example, it may be a combination of one or more sensors selected from the group consisting of a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a laser radar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0021] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The gas leakage detection device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship or a robot, etc. The gas leakage detection device 10 may also be fixedly installed on a fixed device or at a fixed position in a fixed scene.

[0022] The gas leakage detection device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion posture sensors. Environmental perception sensors include but are not limited to one or more combinations of the following sensors: brightness sensors, temperature sensors, haze sensors and other environmental sensors. Motion posture sensors include but are not limited to one or more combinations of the following: inertial sensors (I nert ia lMeasurement Unit, I MU), speed sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors and gravity sensors, etc.

[0023] The gas leakage detection device 10 may further include a display terminal for displaying images.

[0024] In some embodiments, the gas leakage detection device 10 can be applied to a variety of scenarios, including but not limited to oil and gas field scenarios, office area scenarios in buildings, factory indoor scenes, factory outdoor scenes, etc.

[0025] See also Figure 2 , is a flow chart of a gas leakage detection method provided in an embodiment of the present application. The gas leakage detection method is applied to a gas leakage detection device, and the gas leakage detection method includes the following steps:

[0026] S11, acquiring collected continuous frame infrared images, and acquiring multiple frames of visible light images corresponding to the continuous frame infrared images.

[0027] In this embodiment, the continuous frame infrared image and the multiple frames of visible light image are images collected for the same current scene. Infrared imaging technology uses the thermal radiation characteristics of objects in the infrared band to generate thermal images by capturing and analyzing infrared radiation, thereby intuitively displaying the temperature distribution on the surface of the object. In recent years, with the rapid development of sensor technology, signal processing technology and computer technology, oil and gas field gas leakage detection has ushered in new technological innovations. Among them, the detection method based on infrared imaging technology has attracted much attention due to its unique advantages. In gas leakage detection, the gas around the leakage point will form an abnormal temperature area due to diffusion and cooling. Infrared imaging technology can quickly capture these anomalies and realize fast and non-contact leakage detection. Therefore, whether it is colored gas or colorless gas, by collecting infrared images, the temperature changes around the gas leakage point can be captured, thereby achieving accurate positioning. Since gas is a moving target, by collecting continuous frame infrared images and analyzing the feature changes between frames, it is convenient to locate the gas area more accurately in the future.

[0028] Visible light images are acquired based on visible light sensors. Most gases are invisible under visible light, so compared to infrared images, visible light images can be seen as the background of infrared images. Moreover, for colored gases, it is easier to detect gases based on visible light images than infrared images. Visible light images can capture the color characteristics of colored gases, so that the gas area can be located more accurately through color characteristics.

[0029] S12. Based on the continuous frame infrared images, infrared input data of a pre-trained gas leak detection model is formed, and based on the multiple frames of visible light images, visible light input data of a pre-trained gas leak detection model is formed.

[0030] In this embodiment, in order to facilitate the subsequent gas leakage detection model to extract more and more accurate gas features, it is necessary to preprocess the continuous frame infrared image to obtain infrared input data, and preprocess the multiple frames of visible light image to obtain visible light input data. The preprocessing includes but is not limited to denoising, contrast enhancement, brightness adjustment, histogram equalization, edge sharpening, and image size adjustment. The preprocessing operation can improve the image quality and enhance the details and edge information of the image. By adjusting the image size, the size of the infrared image and the visible light image is adjusted to the same. In some implementations, due to the different chip architectures of the image sensor, the obtained picture resolution is also different, so the image resolution needs to be readjusted. In one embodiment, the resolution of the infrared image is used as a reference.

[0031] The gas leakage detection model is pre-trained based on the training data set. Each sample image in the training data set includes continuous frames of infrared sample images and multiple frames of visible light sample images, and gas label data corresponding to the sample images. During the training process, the gas leakage detection model learns the gas features in the infrared sample images and the background features, gas color features, etc. in the visible light sample images. After the training is completed, the parameters in the gas leakage detection model have been solidified and applied to various scenes, and can accurately detect the gas location area. In an optional implementation, the gas leakage detection model is a model formed using a deep learning network architecture.

[0032] S13. Based on the infrared input data and the visible light input data, a gas fusion feature map is obtained through a gas feature extraction network in a gas leakage detection model.

[0033] In this embodiment, the gas leakage detection model includes a gas feature extraction network and a gas position identification network. The gas feature extraction network is used to obtain the gas infrared features corresponding to the infrared input data and the gas visible light features corresponding to the visible light input data based on the infrared input data and the visible light input data, respectively, and then fuse the gas infrared features and the gas visible light features to obtain a gas fusion feature map. Therefore, the gas fusion feature map includes infrared features and visible light features. More gas temperature information can be highlighted through infrared features, and more background information, color features, etc. can be highlighted through visible light features.

[0034] S14. Based on the gas fusion feature map, the gas leakage location data is identified through the gas location identification network in the gas leakage detection model.

[0035] In this embodiment, the gas position recognition network is based on the gas fusion feature map. The gas fusion feature map is first classified to determine whether there is a gas leakage in the gas fusion feature map. If there is a leakage in the gas fusion feature map, the convolutional neural network in the gas position recognition network is further used to obtain the precise position of the gas in the image, and finally the gas leakage position data is output, wherein the gas leakage position data includes but is not limited to: the position coordinates of the leakage position, the position data of the gas area around the leakage position, the position box of the leakage position, etc.

[0036] In the above embodiment, infrared input data of a pre-trained gas leakage detection model is formed based on continuous frame infrared images, and more gas temperature information can be provided through the infrared input data, and visible light input data of a pre-trained gas leakage detection model is formed based on multiple frames of visible light images, and more background features, color features, and gas space change morphological features can be provided through the visible light input data; the gas leakage detection model first outputs an infrared feature map and a visible light feature map respectively, and then the infrared feature map and the visible light feature map are fused to obtain a gas fusion feature map, which includes infrared features and visible light features. More gas temperature information can be highlighted through the infrared features, and more background information and color information can be highlighted through the visible light features, so that the leakage location points of various gases can be more accurately detected through the gas position recognition network, thereby improving the accuracy of gas leakage detection.

[0037] In some embodiments, the step of acquiring the collected continuous frames of infrared images and acquiring multiple frames of visible light images corresponding to the continuous frames of infrared images includes:

[0038] Control the infrared image sensor and the visible light image sensor to synchronously collect the current scene image to obtain continuous frame infrared images and continuous frame visible light images;

[0039] From the continuous frames of visible light images, multiple frames of images are extracted in a time sequence to obtain multiple frames of visible light images.

[0040] In this embodiment, continuous infrared image frames and continuous visible light image frames are images representing the same scene, so it is necessary to control the infrared image sensor and the visible light image sensor to synchronously capture images of the same scene in time, so that continuous infrared image frames and continuous visible light image frames can be aligned in the time dimension.

[0041] Multi-frame visible light images are extracted from continuous frame visible light images in a time sequence. Since multi-frame visible light images are extracted images, for dynamic scenes, the position changes between frames in multi-frame visible light images are relatively large, and the spatial position changes are discontinuous. Therefore, in addition to being used as background, visible light images can also be used to extract spatial feature information of video streams, such as background displacement and gas displacement changes. Continuous frame infrared images are continuous, so they can capture changes in gas in the time dimension, such as gas movement trends. For example, if visible light images from the 1st frame to the 10th frame are continuously collected, the 2nd frame, the 6th frame, the 8th frame, and the 10th frame can be extracted from these ten frames as multi-frame visible light images. The extraction intervals can be uniform intervals or non-uniform intervals.

[0042] In the above embodiment, by extracting frames from continuous frames of visible light images to obtain multiple frames of visible light images, background spatial feature change information and morphological characteristics of the gas in space can be extracted from the multiple frames of visible light images. The morphological characteristics of the subsequent body in space are then fused with the temperature information represented by the infrared characteristics in time series, so that the gas area can be detected more accurately and the accuracy of gas leakage detection can be improved.

[0043] In some embodiments, the method further comprises:

[0044] The gas position frame corresponding to the gas leakage position data is superimposed and displayed on at least one of the following images, wherein the at least one image includes one or more of the following: a current frame infrared image, a current frame visible light image, and a fused image obtained based on the fusion of the current frame infrared image and the current frame visible light image.

[0045] In this embodiment, the gas position frame includes the leakage position and the surrounding gas area. The current frame infrared image is the current frame in the continuous frame infrared image, and the current frame visible light image is the current frame in the continuous frame visible light image. The current frame infrared image and the current frame visible light image are fused by the image fusion algorithm to obtain a fused image. The shape of the gas position frame includes but is not limited to other geometric shapes such as rectangle and circle. By highlighting the leakage position on the image, it is convenient for users to intuitively observe the leaking gas area.

[0046] In the above embodiment, by highlighting the leakage position on the image, the user can intuitively observe the leaking gas area.

[0047] In some embodiments, Figure 3 As shown, Figure 3 This is a flow chart of generating a gas fusion characteristic map in a gas leakage detection method in an embodiment; S13 may also include:

[0048] S131. Based on the infrared input data, the infrared feature extraction network is used to extract the infrared feature data of the gas, and based on the visible light input data, the visible light feature extraction network is used to extract the visible light feature data of the gas.

[0049] Optionally, the infrared feature extraction network includes a plurality of infrared feature sub-networks, the visible light feature extraction network includes a plurality of visible light feature sub-networks, and the infrared feature extraction network is used to extract the infrared feature data of the gas based on the infrared input data, and the visible light feature extraction network is used to extract the visible light feature data of the gas based on the visible light input data, including:

[0050] Obtaining the output of the infrared feature sub-network, and obtaining the output of the visible light feature sub-network corresponding to the infrared feature sub-network,

[0051] The output of the infrared feature subnetwork and the output of the visible light feature subnetwork are fused to obtain a fused feature map, and the fused feature map is used as the input of the next infrared feature subnetwork of the infrared feature subnetwork and the input of the visible light feature subnetwork corresponding to the next infrared feature subnetwork, until all the infrared feature subnetworks are executed sequentially to obtain the infrared feature data of the gas, and until all the visible light feature subnetworks are executed sequentially to obtain the visible light feature data of the gas.

[0052] In this embodiment, the infrared feature extraction network is used to extract infrared features, and the visible light feature extraction network is used to extract visible light features. The infrared feature extraction network includes multiple infrared feature sub-networks, and the visible light feature extraction network includes multiple visible light feature sub-networks. Among the multiple infrared feature sub-networks, the first infrared feature sub-network obtains the output of the first infrared feature sub-network based on the infrared input data. Among the multiple visible light feature sub-networks, the first visible light feature sub-network obtains the output of the first visible light feature sub-network based on the visible light input data. The infrared input data undergoes a first feature extraction through the infrared feature extraction network, and respectively obtains contour information based on infrared light, and the visible light input data undergoes a first feature extraction through the visible light feature extraction network, and respectively obtains contour information based on visible light.

[0053] Then, the output of the first visible light feature subnetwork and the output of the first infrared feature subnetwork are fused to obtain the first fused feature map, which is used as the input of the second visible light feature subnetwork and the second infrared feature subnetwork, and the output of the second visible light feature subnetwork and the output of the second infrared feature subnetwork are obtained respectively. The output of the second visible light feature subnetwork and the output of the second infrared feature subnetwork are fused to obtain the second fused feature map, which is used as the input of the third visible light feature subnetwork and the third infrared feature subnetwork. The above steps are executed in sequence to obtain the output of the last visible light feature subnetwork, that is, the visible light feature data of the gas, and the output of the last infrared feature subnetwork is obtained to obtain the infrared feature data of the gas. By utilizing the feature extraction capability of deep learning, in each feature fusion process, the spatial feature information and background information of the visible light feature are transferred to the infrared light feature; at the same time, the timing information of the infrared light will also be transferred to the visible light feature, so as to facilitate the final extraction of more and more accurate gas features and obtain the final gas features.

[0054] like Figure 4 As shown, Figure 4 FIG. 1 is a schematic diagram of a network structure of a gas leakage detection model in an embodiment. Figure 4In the invention, the infrared feature extraction network includes three infrared feature sub-networks, and the visible light feature extraction network includes three visible light feature sub-networks. The first infrared feature sub-network extracts the first infrared feature based on the infrared input data to obtain the output of the first infrared feature sub-network. Similarly, the first visible light feature sub-network extracts the first visible light feature based on the visible light input data to obtain the output of the first visible light feature sub-network. The output of the first infrared feature sub-network is fused with the output of the first visible light feature sub-network to obtain a fused feature map. The fused feature map is used as the input of the second infrared feature sub-network and the input of the second visible light feature sub-network respectively. Similar operations are performed on the third infrared feature sub-network and the third visible light feature sub-network. The output of the third infrared feature sub-network is the infrared feature data of the gas; the output of the third visible light feature sub-network is the visible light feature data of the gas.

[0055] Optionally, the number of network channels of the infrared feature extraction network is smaller than the number of network channels of the visible light feature extraction network.

[0056] In this embodiment, the number of network channels is used to represent the number of convolution parameters. Since the infrared light branch has more input images, in order to reduce the overall calculation amount of the model, the number of network channels of the infrared light branch is smaller than that of the visible light branch. By reducing the number of network channels of the infrared feature extraction network, the speed of gas detection can be improved and the real-time performance of gas detection can be ensured.

[0057] S132. Obtain a gas fusion characteristic map based on the gas infrared characteristic data and the gas visible light characteristic data.

[0058] In this embodiment, a fusion algorithm is used to fuse the gas infrared characteristic data and the gas visible light characteristic data to obtain a gas fusion characteristic map. Figure 4 As shown, the output of the third visible light feature sub-network and the output of the third infrared feature sub-network are feature fused to obtain a gas fusion feature map as the input of the gas position recognition network.

[0059] In the above embodiment, infrared features are extracted through an infrared feature extraction network, visible light features are extracted through a visible light feature extraction network, and the infrared features are fused with the visible light features. The fused data is used as the input of the next network. In each feature fusion process, the spatial feature information and background information of the visible light features are transferred to the infrared light features. At the same time, the timing information of the infrared light will also be transferred to the visible light features, thereby facilitating the final extraction of more and more accurate gas features and improving the accuracy of gas leakage detection.

[0060] In some embodiments, Figure 5Flow chart of training a gas leak detection model in one embodiment, the method further includes:

[0061] S151, obtaining a training data set, wherein each sample image in the training data set includes a continuous frame infrared sample image and a plurality of frames of visible light sample images, and gas label data corresponding to the sample image.

[0062] In this embodiment, the multiple frames of visible light sample images are images obtained by extracting frames from continuous frames of visible light sample images, and this step is similar to the step of obtaining multiple frames of visible light images. The gas label data is the label data in the training process. In an optional manner, when gas leak detection is applied to a fixed scene, a large number of sample images can be collected for the fixed scene to improve the gas detection accuracy of the fixed scene.

[0063] In the training data set, continuous frame infrared sample images are used, so that colorless gas and colored gas can be captured. Through multiple frames of visible light sample images, the color characteristics of colored gas can also be captured. Therefore, based on the training data set, the gas leak detection model is trained. After the training is completed, the gas leak detection model can detect various colored gases and colorless gases.

[0064] S152. Construct an initial gas leakage detection model, wherein the initial gas leakage detection model includes an initial infrared feature extraction network, an initial visible light feature extraction network, and an initial gas position recognition network.

[0065] In this embodiment, the infrared feature extraction network includes multiple infrared feature sub-networks, and the visible light feature extraction network includes multiple visible light feature sub-networks. The initial infrared feature extraction network and the initial visible light feature extraction network are similar to those described in the above embodiments, except that the parameters in the network are not trained well and need to be trained with a training data set. The network layers in the infrared feature extraction network, the visible light feature extraction network, and the gas position recognition network include but are not limited to one or more of the following combinations: upsampling layer, convolution layer, downsampling layer, pooling layer, laser layer, etc.

[0066] S153, iteratively training the initial gas leakage detection model using the training data set until the loss function converges, thereby obtaining the pre-trained gas leakage detection model.

[0067] Optionally, the iterative training of the initial gas leakage detection model by using the training data set until the loss function converges to obtain the pre-trained gas leakage detection model includes:

[0068] Acquire sample images from the training data set, form infrared sample input data of the gas leakage detection model in the current iteration based on the continuous frame infrared sample images, and form visible light sample input data of the gas leakage detection model in the current iteration based on multiple frames of visible light sample images;

[0069] Based on the infrared sample input data, output infrared sample feature data through the infrared feature extraction network in the current iteration, and based on the visible light sample input data, output visible light sample feature data through the visible light feature extraction network in the current iteration;

[0070] fusing the infrared sample feature data and the visible light sample feature data to obtain fused sample feature data;

[0071] Based on the fused sample feature data, outputting the gas position data in the current iteration through the gas position recognition network in the current iteration;

[0072] Based on the loss function, the loss value between the gas position data in the current iteration and the gas label data corresponding to the sample image is calculated to obtain the loss value of the current iteration;

[0073] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain sample images from the training data set and iteratively train the gas leakage detection model. If the current iteration meets the iteration termination condition, the gas leakage detection model after stopping the iteration is used as the pre-trained gas leakage detection model.

[0074] In this embodiment, in each iteration process, the step of obtaining infrared sample feature data is similar to the step of obtaining infrared feature data, and the step of obtaining visible light sample feature data is similar to the step of obtaining visible light feature data. The output of the infrared feature subnetwork in the current iteration is obtained, and the output of the visible light feature subnetwork corresponding to the infrared feature subnetwork in the current iteration is obtained. The output of the infrared feature subnetwork in the current iteration and the output of the corresponding visible light feature subnetwork are fused to obtain a fused sample feature graph, and the fused sample feature graph is used as the input of the next infrared feature subnetwork and the input of the visible light feature subnetwork corresponding to the next infrared feature subnetwork, until all the infrared feature subnetworks are executed sequentially to obtain infrared sample feature data, and until all visible light feature subnetworks are executed sequentially to obtain visible light sample feature data. The iteration termination conditions include but are not limited to the number of iterations and the loss value of the current iteration is less than the preset loss value. The loss function includes but is not limited to the cross entropy loss function and the mean square error loss function.

[0075] During the training process, in each feature fusion process, the spatial feature information and background information of the visible light feature are transferred to the infrared light feature; at the same time, the temporal information of the infrared light will also be transferred to the visible light feature, so that the background features in the multi-frame visible light sample images, the morphological change features of the gas in space, and the infrared features are learned during the training process of the gas leak detection model. After the training is completed, the gas leak detection model accurately extracts the infrared features in the continuous frame infrared images and the visible light features in the multi-frame visible light images, so that it can be used to detect colored gas and colorless gas in the scene.

[0076] In the above embodiment, the initial gas leak detection model is trained by using a training data set, so that the gas leak detection model learns the infrared features in continuous frame infrared sample images, the background features in multiple frames of visible light sample images, and the morphological features of the gas in space. After the training is completed, the parameters in the model are solidified, and the infrared features and visible light features in the image can be accurately extracted from the input image data, and the two features are fused. Based on the fused feature map, the accurate gas position area is output, thereby improving the accuracy of gas detection.

[0077] In some embodiments, the continuous frame infrared images and the multiple frames of visible light images are images collected from oil and gas field scenes; and / or the sample images in the training data set are images collected from oil and gas field scenes.

[0078] In this embodiment, the gas leakage detection method can be used in oil and gas field scenes. In order to enable the gas leakage detection model to better detect the leakage locations of various gases in the oil and gas field, the training data set can be collected from the oil and gas field scene. At the same time, the background data in the sample image is as rich as possible, wherein the background data includes but is not limited to: walls, equipment, sky, white clouds, sea, etc. Add enough background data so that the gas leakage detection model can learn these background features during the training process, so that after the training is completed, the background data can be accurately identified to reduce the misjudgment of gas detection. In contrast, the detection method based on infrared imaging technology shows good application prospects in gas leakage detection in oil and gas fields. First, infrared imaging technology has the characteristics of non-contact and rapid response, and can perform real-time detection without interrupting production. Secondly, infrared imaging technology is sensitive to temperature anomalies, and can accurately capture temperature changes around the leakage point to achieve precise positioning; finally, infrared imaging equipment is relatively simple and portable, and is easy to deploy and use on site in oil and gas fields. The detection method based on infrared imaging technology has good application value and development prospects in gas leakage detection in oil and gas fields. With the continuous advancement of technology and the gradual reduction of costs, this technology is expected to become one of the mainstream methods for gas leakage detection in oil and gas fields in the future.

[0079] In the above embodiments, the gas leakage detection method is used in the oil and gas field scenario, which can improve the accuracy of gas detection in the oil and gas field scenario.

[0080] In some embodiments, the method further comprises:

[0081] After the gas leakage location data is identified, a gas alarm signal is issued, wherein the gas alarm signal includes one or more of the following: a sound alarm signal, a light alarm signal.

[0082] In this embodiment, the current image can be displayed in real time on the user interface, wherein the gas position frame corresponding to the gas leakage position data is superimposed on the current image. The alarm data can also be sent to the terminal device communicating with the gas leakage detection device so that the user can remotely receive the alarm data, wherein the alarm data includes but is not limited to the current image, the alarm position point, the leakage time, the temperature information, the position frame of the leaked gas, etc. For example, the gas leakage detection device may include a positioning module, and the alarm position point where the gas leakage detection device is located can be obtained through the positioning module. The leakage time can be determined according to the time of collecting the current frame image. The temperature information can be determined according to the received infrared intensity. When the gas leakage detection device is installed at a fixed position, the pre-stored position can be obtained as the alarm position point. When the alarm is triggered, the sound alarm signal can be the sound of a buzzer, and the light alarm signal can be the flashing of a light, etc. In an optional implementation, the gas leakage detection device 10 can also display and exchange modules, and display at least one of the following data through the display and exchange modules: continuous frame infrared images, continuous frame visible light images, gas leakage position data, and device status information. The gas leakage detection device 10 may also integrate physical buttons or touch screen virtual buttons to realize interactive operations between the user and the device, such as starting detection, switching displayed views, adjusting parameters, saving detection results, etc. In an optional implementation, the gas leakage detection device 10 may also communicate with a remote monitoring center, where the monitoring center includes one or more servers, and alarm data may be sent to the remote monitoring center through a wireless communication module so that timely response measures can be taken.

[0083] In the above embodiment, after the gas leakage location data is identified, an alarm prompt can be given in time, thereby reducing the harm of the gas leakage.

[0084] On the other hand, the present application provides a computer program product, including a computer program, which implements the gas leakage detection method described in any embodiment of the present application when executed by a processor.

[0085] Wherein, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the gas leakage detection method may be a gas leakage detection device.

[0086] See also Figure 6 An embodiment of the present application provides a gas leakage detection device, including: an acquisition module 61, used to acquire collected continuous frame infrared images, and acquire multiple frames of visible light images corresponding to the continuous frame infrared images; an input module 62, used to form infrared input data of a pre-trained gas leakage detection model based on the continuous frame infrared images, and to form visible light input data of a pre-trained gas leakage detection model based on multiple frames of the visible light images; a detection module 63, used to obtain a gas fusion feature map through a gas feature extraction network in the gas leakage detection model based on the infrared input data and the visible light input data; the detection module 63 is also used to identify gas leakage position data through a gas position identification network in the gas leakage detection model based on the gas fusion feature map.

[0087] Optionally, the acquisition module 61 is further used for:

[0088] Control the infrared image sensor and the visible light image sensor to synchronously collect the current scene image to obtain continuous frame infrared images and continuous frame visible light images;

[0089] From the continuous frames of visible light images, multiple frames of images are extracted in a time sequence to obtain multiple frames of visible light images.

[0090] Optionally, the detection module 63 is further used for:

[0091] The gas position frame corresponding to the gas leakage position data is superimposed and displayed on at least one of the following images, wherein the at least one image includes one or more of the following: a current frame infrared image, a current frame visible light image, and a fused image obtained based on the fusion of the current frame infrared image and the current frame visible light image.

[0092] Optionally, the detection module 63 is further used for:

[0093] Based on the infrared input data, the infrared feature extraction network is used to extract the infrared feature data of the gas, and based on the visible light input data, the visible light feature extraction network is used to extract the visible light feature data of the gas;

[0094] The gas fusion characteristic map is obtained based on the gas infrared characteristic data and the gas visible light characteristic data.

[0095] Optionally, the infrared feature extraction network includes a plurality of infrared feature sub-networks, the visible light feature extraction network includes a plurality of visible light feature sub-networks, and the detection module 63 is further used for:

[0096] Obtaining the output of the infrared feature sub-network, and obtaining the output of the visible light feature sub-network corresponding to the infrared feature sub-network,

[0097] The output of the infrared feature subnetwork and the output of the visible light feature subnetwork are fused to obtain a fused feature map, and the fused feature map is used as the input of the next infrared feature subnetwork of the infrared feature subnetwork and the input of the visible light feature subnetwork corresponding to the next infrared feature subnetwork, until all the infrared feature subnetworks are executed sequentially to obtain the infrared feature data of the gas, and until all the visible light feature subnetworks are executed sequentially to obtain the visible light feature data of the gas.

[0098] Optionally, the number of network channels of the infrared feature extraction network is smaller than the number of network channels of the visible light feature extraction network.

[0099] Optionally, the detection module 63 is further used for:

[0100] Acquire a training data set, wherein each sample image in the training data set includes a continuous frame infrared sample image and a plurality of frames of visible light sample images, and gas label data corresponding to the sample image;

[0101] Constructing an initial gas leakage detection model, wherein the initial gas leakage detection model includes an initial infrared feature extraction network, an initial visible light feature extraction network, and an initial gas position recognition network;

[0102] The initial gas leakage detection model is iteratively trained using the training data set until the loss function converges, thereby obtaining the pre-trained gas leakage detection model.

[0103] Optionally, the detection module 63 is further used for:

[0104] Acquire sample images from the training data set, form infrared sample input data of the gas leakage detection model in the current iteration based on the continuous frame infrared sample images, and form visible light sample input data of the gas leakage detection model in the current iteration based on multiple frames of visible light sample images;

[0105] Based on the infrared sample input data, output infrared sample feature data through the infrared feature extraction network in the current iteration, and based on the visible light sample input data, output visible light sample feature data through the visible light feature extraction network in the current iteration;

[0106] fusing the infrared sample feature data and the visible light sample feature data to obtain fused sample feature data;

[0107] Based on the fused sample feature data, outputting the gas position data in the current iteration through the gas position recognition network in the current iteration;

[0108] Based on the loss function, the loss value between the gas position data in the current iteration and the gas label data corresponding to the sample image is calculated to obtain the loss value of the current iteration;

[0109] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain sample images from the training data set and iteratively train the gas leakage detection model. If the current iteration meets the iteration termination condition, the gas leakage detection model after stopping the iteration is used as the pre-trained gas leakage detection model.

[0110] Optionally, the continuous frame infrared images and the multiple frames of visible light images are images collected from oil and gas field scenes; and / or the sample images in the training data set are images collected from oil and gas field scenes.

[0111] Optionally, the detection module 63 is further used for:

[0112] After the gas leakage location data is identified, a gas alarm signal is issued, wherein the gas alarm signal includes one or more of the following: a sound alarm signal, a light alarm signal.

[0113] It can be understood by those skilled in the art that Figure 6 The structure of the gas leakage detection device does not constitute a limitation on the gas leakage detection device, and the various modules can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the gas leakage detection device in the form of hardware, or can be stored in the memory in the gas leakage detection device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the gas leakage detection device may include more or fewer modules than those shown in the figure.

[0114] See also Figure 7 On the other hand, a gas leakage detection device 10 is provided, including a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 executes the steps of the gas leakage detection method provided in any of the above embodiments of the present application.

[0115] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire gas leakage detection device, and executes various functions and processes data of the gas leakage detection device by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0116] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the gas leakage detection device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0117] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the gas leakage detection method provided in any of the above embodiments of the present application.

[0118] In another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the gas leakage detection method as described in any embodiment of the present application is implemented.

[0119] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0120] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A gas leakage detection method, characterized in that: include: Acquire the collected continuous frame infrared images, and acquire multiple frames of visible light images corresponding to the continuous frame infrared images; Based on the continuous frames of infrared images, forming infrared input data of a pre-trained gas leak detection model, and based on the multiple frames of visible light images, forming visible light input data of a pre-trained gas leak detection model; Based on the infrared input data and the visible light input data, a gas fusion feature map is obtained through a gas feature extraction network in the gas leakage detection model; Based on the gas fusion characteristic graph, gas leakage position data is identified through a gas position identification network in the gas leakage detection model.

2. The gas leakage detection method according to claim 1, characterized in that: The step of acquiring the collected continuous infrared image frames and acquiring multiple visible light image frames corresponding to the continuous infrared image frames includes: Control the infrared image sensor and the visible light image sensor to synchronously collect the current scene image to obtain continuous frame infrared images and continuous frame visible light images; From the continuous frames of visible light images, multiple frames of images are extracted in a time sequence to obtain multiple frames of visible light images.

3. The gas leakage detection method according to claim 1, characterized in that: The method further comprises: The gas position frame corresponding to the gas leakage position data is superimposed and displayed on at least one of the following images, wherein the at least one image includes one or more of the following: a current frame infrared image, a current frame visible light image, and a fused image obtained based on the fusion of the current frame infrared image and the current frame visible light image.

4. The gas leakage detection method according to claim 1, characterized in that: The method of obtaining a gas fusion feature map based on the infrared input data and the visible light input data through a gas feature extraction network in the gas leakage detection model includes: Based on the infrared input data, the infrared feature extraction network is used to extract the infrared feature data of the gas, and based on the visible light input data, the visible light feature extraction network is used to extract the visible light feature data of the gas; The gas fusion characteristic map is obtained based on the gas infrared characteristic data and the gas visible light characteristic data.

5. The gas leakage detection method according to claim 4, characterized in that: The infrared feature extraction network includes a plurality of infrared feature sub-networks, the visible light feature extraction network includes a plurality of visible light feature sub-networks, the infrared feature extraction network is used to extract the infrared feature data of the gas based on the infrared input data, and the visible light feature extraction network is used to extract the visible light feature data of the gas based on the visible light input data, including: Obtaining the output of the infrared feature sub-network, and obtaining the output of the visible light feature sub-network corresponding to the infrared feature sub-network, The output of the infrared feature subnetwork and the output of the visible light feature subnetwork are fused to obtain a fused feature map, and the fused feature map is used as the input of the next infrared feature subnetwork of the infrared feature subnetwork and the input of the visible light feature subnetwork corresponding to the next infrared feature subnetwork, until all the infrared feature subnetworks are executed sequentially to obtain the infrared feature data of the gas, and until all the visible light feature subnetworks are executed sequentially to obtain the visible light feature data of the gas.

6. The gas leakage detection method according to claim 4, characterized in that: The number of network channels of the infrared feature extraction network is smaller than the number of network channels of the visible light feature extraction network.

7. The gas leakage detection method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a training data set, wherein each sample image in the training data set includes a continuous frame infrared sample image and a plurality of frames of visible light sample images, and gas label data corresponding to the sample image; Constructing an initial gas leakage detection model, wherein the initial gas leakage detection model includes an initial infrared feature extraction network, an initial visible light feature extraction network, and an initial gas position recognition network; The initial gas leakage detection model is iteratively trained using the training data set until the loss function converges, thereby obtaining the pre-trained gas leakage detection model.

8. The gas leakage detection method according to claim 7, characterized in that: The iterative training of the initial gas leakage detection model by the training data set until the loss function converges to obtain the pre-trained gas leakage detection model includes: Acquire sample images from the training data set, form infrared sample input data of the gas leakage detection model in the current iteration based on the continuous frame infrared sample images, and form visible light sample input data of the gas leakage detection model in the current iteration based on multiple frames of visible light sample images; Based on the infrared sample input data, output infrared sample feature data through the infrared feature extraction network in the current iteration, and based on the visible light sample input data, output visible light sample feature data through the visible light feature extraction network in the current iteration; fusing the infrared sample feature data and the visible light sample feature data to obtain fused sample feature data; Based on the fused sample feature data, outputting the gas position data in the current iteration through the gas position recognition network in the current iteration; Based on the loss function, the loss value between the gas position data in the current iteration and the gas label data corresponding to the sample image is calculated to obtain the loss value of the current iteration; Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain sample images from the training data set and iteratively train the gas leakage detection model. If the current iteration meets the iteration termination condition, the gas leakage detection model after stopping the iteration is used as the pre-trained gas leakage detection model.

9. The gas leakage detection method according to claim 1 or 7, characterized in that: The continuous frame infrared images and the multiple frames of visible light images are images acquired from oil and gas field scenes; and / or the sample images in the training data set are images acquired from oil and gas field scenes.

10. The gas leakage detection method according to claim 1, characterized in that: The method further comprises: After the gas leakage location data is identified, a gas alarm signal is issued, wherein the gas alarm signal includes one or more of the following: a sound alarm signal, a light alarm signal.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the gas leakage detection method according to any one of claims 1 to 10 is implemented.

12. A gas leakage detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas leakage detection method according to any one of claims 1 to 10.

Images