Alkane gas leakage real-time identification and positioning method and system based on deep learning, and medium

Abstract

The invention provides an alkane gas leakage real-time identification and positioning method and system based on deep learning and a medium. The method comprises the following steps: acquiring an infrared image and a visible light image; performing rapid registration based on the imaging area and view field offset correction to obtain a registered visible light image; performing gas target detection according to the infrared image; performing pseudo-color conversion according to the detected gas target and gas concentration; and fusing the pseudo color to the registered visible light image. Gas detected in an infrared view field is subjected to pseudo-color fusion in a visible light view field, and finally plug flow output is performed. According to the invention, the deep learning detection algorithm and the dual-light rapid registration algorithm are combined to solve the alkane gas leakage detection task, and compared with the existing artificial observation alkane gas leakage detection method, the detection error caused by limited artificial observation is reduced, the gas leakage detection speed is greatly accelerated, and the safety and reliability are better.

Description

Technical Field

[0001] The present invention relates to the field of infrared image processing technology, and in particular to the application of deep learning in the field of infrared image processing, and specifically to a real-time identification method, system and medium for alkane gas leakage based on deep learning. Background Art

[0002] Gas leak detection based on infrared imaging technology has the advantages of high efficiency, wide application scenarios, and short response time, and has become an important means of gas leak detection. However, traditional gas leak detection is performed on-site by people operating equipment, which not only has a high probability of false detection and missed detection, but also has a high risk factor. Summary of the invention

[0003] The purpose of the present invention is to propose a real-time identification and positioning system for alkane gas leaks based on deep learning, which can quickly detect gas leaks through artificial intelligence algorithms, and quickly match infrared and visible light fields to mark gas leaks on the visible light field, thereby solving the problem of insufficient manual detection accuracy, improving the detection accuracy of gas leaks, and achieving better monitoring effects.

[0004] According to a first aspect of the present invention, a method for real-time identification and positioning of alkane gas leakage based on deep learning is proposed, comprising the following steps:

[0005] Imaging the target object based on the infrared camera to obtain an infrared image;

[0006] Imaging the target object based on a visible light camera to obtain a visible light image;

[0007] According to the acquired infrared image and visible light image, rapid registration is performed based on imaging area and field of view offset correction to obtain a registered visible light image;

[0008] Performing gas target detection according to the acquired infrared image;

[0009] Perform pseudo-color conversion according to the detected gas target and gas concentration, and obtain pseudo-color through piecewise linear transformation to represent the concentration of leaked gas in real time; and

[0010] Fusion of pseudo-color to registered visible light images.

[0011] As an optional implementation, the fast registration based on the imaging area and the field of view offset correction to obtain the registered visible light image includes:

[0012] Step 1.1: For a target ×N target The distance from the observation target to the center of the infrared camera of the infrared oil and gas leakage imaging device is obtained as Dtarget , unit is m; the aforementioned M target ×N target Respectively represent the equivalent length and width of the observed target;

[0013] Step 1.2: Get the infrared camera resolution in M ir ×N ir , the unit is pixel, the pixel spacing of the infrared camera I ir , unit is um, focal length F ir , the unit is mm, and the area of ​​the observed target in the imaging field of the infrared camera is MD ir ×ND ir , then there is MD ir and ND ir The calculation formula is as follows:

[0014]

[0015] Step 1.3: Get the visible light camera resolution to M vis ×N vis , the unit is pixel, the pixel spacing of the visible light camera I vis , unit is um, focal length F vis , the unit is mm, and the area of ​​the observed target in the imaging field of the visible light camera is MD vis ×ND vis , then there is MD vis and ND vis The calculation formula is as follows:

[0016]

[0017] Step 1.4: Get the distance baseline between the infrared camera and the visible light camera as D line , the unit is mm, then the offset T of the visible light camera imaging field of view with the center of the infrared camera imaging field of view as the origin is:

[0018]

[0019] Step 1.5: Get the coordinates of the target object in the infrared camera imaging field of view as [X ir Y ir L ir H ir ],in:

[0020]

[0021] L ir =MD ir ;

[0022] H ir =NDir ;

[0023] Among them, X ir Y is the horizontal coordinate of the left vertex of the target object in the infrared camera imaging field of view, ir is the left vertex ordinate of the target object in the infrared camera imaging field of view, L ir is the length of the target in the infrared camera imaging field of view, H ir is the width of the target in the imaging field of the infrared camera;

[0024] Step 1.6: Get the coordinates of the target object in the visible light camera imaging field of view as [X vis Y vis L vis H vis ],in:

[0025]

[0026] L vis =MD vis ;

[0027] H vis =ND vis ;

[0028] Among them, X vis Y is the horizontal coordinate of the left vertex of the target object in the imaging field of view of the visible light camera, vis is the ordinate of the left vertex of the target object in the imaging field of view of the visible light camera, L vis is the length of the target in the imaging field of the visible light camera, H vis is the width of the target in the imaging field of view of the visible light camera.

[0029] As an optional implementation, the performing gas target detection according to the acquired infrared image includes:

[0030] The acquired infrared image is input into the trained target detection model, and an infrared image with a detection frame is output.

[0031] As an optional implementation, the pseudo-color conversion is performed according to the detected gas target and gas concentration, and the pseudo-color color is obtained by piecewise linear transformation for real-time characterization of the concentration of the leaked gas, including:

[0032] Step 5.1: Let the size of the detection frame area of ​​the infrared image output by the gas target detection be R×C, and count the gray level L of the pixels in the detection frame area and the number L corresponding to each gray level i ; Set the gray level threshold L T Divide all pixels in the detection frame area into two categories: background and target, where the number of background pixels is N0 and the number of target pixels is N1;

[0033] Step 5.2: Let the ratio of background pixels to the entire detection frame area be W0, and the grayscale value mean be u0; let the ratio of target pixels to the entire detection frame area be W1, and the grayscale value mean be u1; let the total grayscale mean of the detection frame area be u, then:

[0034]

[0035] N0+N1=R×C;

[0036] W0+W1=1;

[0037] Then we further get:

[0038]

[0039] Step 5.3: Let the inter-class variance of the detection box area be Var, then:

[0040] Var=W0×(u0-u) 2 +W1×(u1-u) 2 =W0×W1×(u0-u1) 2 ;

[0041] The traversal method is used to traverse L gray levels to obtain the gray level threshold L that maximizes the inter-class variance Var of the detection frame area. T , the infrared image detection frame area is binarized into two categories: background and target;

[0042] Step 5.4: According to step 5.3, the grayscale value I(i, j) of the target category in the infrared image detection frame area is read pixel by pixel, and grayscale pseudo-color conversion is performed according to the concentration characteristics of alkane gas. Based on the piecewise linear transformation, each grayscale value in the original grayscale image is sent to the R, G, and B converters with different transformation rules, and they are encoded respectively to generate three components of the RGB color space. Then, the three primary colors R, G, and B are synthesized into corresponding colors according to the corresponding proportions of different components, where I(i, j) represents the grayscale value of the pixel at the coordinate (i, j) in the image, R(i, j), G(i, j), and B(i, j) respectively represent the component values ​​of red, green, and blue at the coordinate (i, j) in the image, and the pseudo-color conversion encoding formula is as follows:

[0043]

[0044] As an optional implementation, the step of fusing the pseudo-color into the registered visible light image includes:

[0045] Step 6.1: Based on the three-channel component values ​​obtained by grayscale pseudo-color conversion, the pixel values ​​are recorded as I R(i,j) ,IG(i,j) ,I B(i,j) ;

[0046] Step 6.2: According to the pixel position of the target category, read the three-channel component values ​​of the corresponding area of ​​the registered visible light image pixel by pixel, and record them as V R(i,j) 、V G(i,j) 、V B(i,j) , and then reassign the pixel positions corresponding to the visible light image as follows:

[0047]

[0048] Thus, the pseudo-color is fused into the registered visible light image and output.

[0049] As an optional implementation, the method further includes the following steps:

[0050] The infrared images and visible light images fused with gas pseudo-color are stitched and transcoded, and then output through server streaming.

[0051] As an optional implementation, the step of stitching and transcoding the infrared image and the visible light image fused with the gas pseudo-color, and outputting the image through server streaming, includes:

[0052] The original infrared image and the visible light image fused with gas pseudo-color are stitched and encoded into RTMP video stream, and then pushed through the server.

[0053] According to the second aspect of the present invention, a real-time identification and positioning device for alkane gas leakage based on deep learning is also proposed, comprising:

[0054] An infrared image acquisition module is used to image a target object based on an infrared camera to acquire an infrared image;

[0055] A visible light image acquisition module is used to image the target object based on a visible light camera to acquire a visible light image;

[0056] A fast registration module is configured to perform fast registration based on the acquired infrared image and visible light image based on imaging area and field of view offset correction to obtain a registered visible light image;

[0057] A gas target detection module, used for performing gas target detection according to the acquired infrared image;

[0058] A grayscale pseudo-color conversion module is used to perform pseudo-color conversion according to the detected gas target and gas concentration, and obtain pseudo-color through piecewise linear transformation to represent the concentration of leaked gas in real time; and

[0059] Image fusion module, used to fuse pseudo-color into the registered visible light image.

[0060] According to a third aspect of the present invention, a computer system is also provided, comprising:

[0061] one or more processors;

[0062] A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of executing the aforementioned deep learning-based real-time identification and positioning method for alkane gas leaks.

[0063] According to a fourth aspect of the present invention, a computer-readable medium storing software is also proposed, wherein the software includes instructions that can be executed by one or more computers, and when the instructions are executed by the one or more computers, the process of performing the aforementioned deep learning-based real-time identification and positioning method for alkane gas leaks is performed.

[0064] According to a fifth aspect of the present invention, a computer program product is also proposed, comprising a computer program, which, when executed by a processor, implements the process of the aforementioned deep learning-based real-time identification and positioning method for alkane gas leaks.

[0065] According to the technical scheme of the above implementation mode, in response to the detection demand for real-time identification of gas leaks, according to the grayscale characteristics of alkane gases under infrared imaging, the real-time identification and positioning method of alkane gas leaks based on deep learning proposed in the present invention quickly detects gas leaks through artificial intelligence algorithms and performs grayscale pseudo-color conversion through image processing. On the basis of rapid matching of infrared and visible light fields of view, the grayscale pseudo-color of the gas leak is fused and labeled in the visible light field of view, thereby solving the problem of insufficient manual detection accuracy, improving the detection accuracy of gas leaks, and having a better monitoring effect.

[0066] Compared with the prior art, the present invention has the following significant beneficial effects:

[0067] First, compared with traditional feature point matching, when the target is known, dual-light rapid registration can achieve real-time field of view matching of infrared and visible light according to the distance change of the target, providing consistent field of view and fusion position alignment for gas leakage visible light pseudo-color fusion;

[0068] Second, in the method of the present invention, on the basis of extracting the detection frame area on the infrared image according to the deep learning algorithm, through the classification statistics of the target and the background, combined with the grayscale value of the target category in the detection frame area of ​​the infrared image according to the concentration characteristics of the alkane gas, the grayscale pseudo-color conversion is performed, and each grayscale value in the original grayscale image is sent to the RGB converter with different transformation rules, and it is encoded to generate three components of the RGB color space, and then the three primary colors of R, G, and B are synthesized into corresponding colors according to the corresponding proportions of different components to achieve output based on piecewise linear transformation;

[0069] Third, the present invention combines deep learning algorithms with traditional image processing algorithms to reduce the computational complexity brought by deep learning algorithms. While ensuring detection accuracy, it uses traditional image processing algorithms to improve detection rate, achieve real-time monitoring, field of view matching, real-time fusion and real-time visual output, and has high detection real-time and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are not intended to be drawn to scale. In the accompanying drawings, each identical or nearly identical component shown in each figure may be represented by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings.

[0071] Figure 1 It is a flow chart of a method for real-time identification and positioning of alkane gas leakage based on deep learning according to an exemplary embodiment of the present invention.

[0072] Figure 2 It is a principle block diagram of a device for real-time identification and positioning of alkane gas leakage based on deep learning according to an exemplary embodiment of the present invention.

[0073] Figure 3 is a schematic diagram of a registered visible light image and an infrared image according to an example of the present invention.

[0074] Figure 4 Schematic diagram of real-time dual-light registration streaming according to an example of the present invention.

[0075] Figure 5 It is a schematic diagram of real-time visible light large field of view fusion streaming according to an example of the present invention. DETAILED DESCRIPTION

[0076] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0077] Various aspects of the present invention are described in this disclosure with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the present disclosure are not necessarily intended to include all aspects of the present invention. It should be understood that the various concepts and embodiments introduced above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed by the present invention are not limited to any implementation. In addition, some aspects disclosed by the present invention can be used alone or in any appropriate combination with other aspects disclosed by the present invention.

[0078] {Example 1}

[0079] Combination Figure 1 As shown, the real-time identification and positioning method of alkane gas leakage based on deep learning according to an embodiment of the present invention includes the following steps:

[0080] Imaging the target object based on the infrared camera to obtain an infrared image;

[0081] Imaging the target object based on a visible light camera to obtain a visible light image;

[0082] According to the acquired infrared image and visible light image, rapid registration is performed based on imaging area and field of view offset correction to obtain a registered visible light image;

[0083] Performing gas target detection according to the acquired infrared image;

[0084] Perform pseudo-color conversion according to the detected gas target and gas concentration, and obtain pseudo-color through piecewise linear transformation to represent the concentration of leaked gas in real time; and

[0085] Fusion of pseudo-color to registered visible light images.

[0086] In combination with the implementation of the above methods, the real-time identification and positioning method for alkane gas leaks based on deep learning proposed in the present invention aims to combine the deep learning detection algorithm and the dual-light fast registration algorithm to solve the alkane gas leak detection task and realize rapid detection of gas leaks. Compared with the existing manual operation and observation alkane gas leak detection method, it reduces the detection error caused by manual observation, improves the gas leak detection speed, and has better safety and reliability.

[0087] At the same time, combined with the rapid matching method of infrared and visible light fields of view, the gas leakage situation can be marked and displayed in real time on the visible light field of view, solving the problem of low accuracy of manual detection, improving the detection accuracy of gas leakage, and using the large visible light field of view to determine the location of gas leakage, which has a better monitoring effect.

[0088] In the embodiment of the present invention, for obtaining infrared images, infrared data stream packets can be captured according to the UDP video stream protocol decoding to form a complete infrared image. For obtaining visible light images, visible light images can be obtained by pulling streams and decoding according to the RTSP video stream protocol.

[0089] As an optional implementation, the rapid registration based on the imaging area and the field of view offset correction to obtain the registered visible light image includes:

[0090] Step 1.1: For a target ×N target The distance from the observation target to the center of the infrared camera of the infrared oil and gas leakage imaging device is obtained as D target , unit is m; the aforementioned M target ×N target Respectively represent the equivalent length and width of the observed target;

[0091] Step 1.2: Get the infrared camera resolution in M ir ×N ir , the unit is pixel, the pixel spacing of the infrared camera is L ir , unit is um, focal length F ir , the unit is mm, and the area of ​​the observed target in the imaging field of the infrared camera is MD ir ×ND ir , then there is MD ir and ND ir The calculation formula is as follows:

[0092]

[0093] Step 1.3: Get the visible light camera resolution to M vis ×N vis , the unit is pixel, the pixel spacing of the visible light camera I vis , unit is um, focal length F vis , the unit is mm, and the area of ​​the observed target in the imaging field of the visible light camera is MD vis ×ND vis , then there is MD vis and ND vis The calculation formula is as follows:

[0094]

[0095] Step 1.4: Get the distance baseline between the infrared camera and the visible light camera as D line , the unit is mm, then the offset T of the visible light camera imaging field of view with the center of the infrared camera imaging field of view as the origin is:

[0096]

[0097] Step 1.5: Get the coordinates of the target object in the infrared camera imaging field of view as [X ir Y ir L ir H ir ],in:

[0098]

[0099] L ir =MD ir ;

[0100] H ir =ND ir ;

[0101] Among them, X ir Y is the horizontal coordinate of the left vertex of the target object in the infrared camera imaging field of view, ir is the left vertex ordinate of the target object in the infrared camera imaging field of view, L ir is the length of the target in the infrared camera imaging field of view, H ir is the width of the target in the imaging field of the infrared camera;

[0102] Step 1.6: Get the coordinates of the target object in the visible light camera imaging field of view as [X vis Y vis L vis H vis ],in:

[0103]

[0104] L vis =MD vis ;

[0105] H vis =ND vis ;

[0106] Among them, X vis Y is the horizontal coordinate of the left vertex of the target object in the imaging field of view of the visible light camera, vis is the ordinate of the left vertex of the target object in the imaging field of view of the visible light camera, L vis is the length of the target in the imaging field of the visible light camera, H vis is the width of the target in the imaging field of view of the visible light camera.

[0107] In combination with the above embodiments, based on the area of ​​the target object in the infrared camera and visible light camera imaging field of view, and the offset of the target object in the visible light camera imaging field of view, the coordinates of the target object in the infrared camera and visible light camera imaging field of view are obtained respectively. Therefore, compared with the traditional feature point matching, in the case of a known target, the dual-light fast registration can achieve real-time infrared and visible light field of view matching according to the distance change of the target, providing real-time, accurate and complete large-field visible light images for subsequent gas leakage visible light pseudo-color fusion. Figure 3 The example shown shows an infrared image and a registered visible light image.

[0108] As an optional implementation, the performing gas target detection according to the acquired infrared image includes:

[0109] The acquired infrared image is input into the trained target detection model, and an infrared image with a detection frame is output.

[0110] Among them, the aforementioned target detection model is particularly preferably a target detection model based on a deep learning neural network, including but not limited to models based on network structures such as YOLO v5, RetinaNet, CNN, RNN, and R-CNN. In the example of the present invention, taking the model based on YOLO v5 as an example, the model trained based on yolo v5 is deployed in a computer system after TensorRT reasoning, and the infrared image is input and then the detection output is performed.

[0111] As an example, the acquired infrared image is input into the trained target detection model, and the infrared image with the detection box is output, which includes the following process:

[0112] Step 4.1: Convert the YOLO v5-based target detection model trained by the PyTorch framework to the onnx format through torch.onnx.export;

[0113] Step 4.2: Use the ONNX-TensorRT tool to convert the onnx format model generated in step 4.1 into the TensorRT inference engine and deploy it in the computer system;

[0114] Step 4.3: Input the infrared image into the TensorRT inference engine for detection output, and output the infrared image with the detection box.

[0115] It should be understood that the detection frame here particularly refers to a rectangular detection frame.

[0116] As an optional implementation, the pseudo-color conversion is performed according to the detected gas target and gas concentration, and the pseudo-color color is obtained by piecewise linear transformation for real-time characterization of the concentration of the leaked gas, including:

[0117] Step 5.1: Let the size of the detection frame area of ​​the infrared image output by the gas target detection be R×C, and count the gray level L of the pixels in the detection frame area and the number L corresponding to each gray level i ; Set the gray level threshold L T Divide all pixels in the detection frame area into two categories: background and target, where the number of background pixels is N0 and the number of target pixels is N1;

[0118] Step 5.2: Let the ratio of background pixels to the entire detection frame area be W0, and the grayscale value mean be u0; let the ratio of target pixels to the entire detection frame area be W1, and the grayscale value mean be u1; let the total grayscale mean of the detection frame area be u, then:

[0119]

[0120] N0+N1=R×C;

[0121] W0+W1=1;

[0122] Then we further get:

[0123]

[0124] Step 5.3: Let the inter-class variance of the detection box area be Var, then:

[0125] Var=W0×(u0-u) 2 +W1×(u1-u) 2 =W0×W1×(u0-u1) 2 ;

[0126] The traversal method is used to traverse L gray levels to obtain the gray level threshold L that maximizes the inter-class variance Var of the detection frame area. T , the infrared image detection frame area is binarized into two categories: background and target;

[0127] Step 5.4: According to step 5.3, the grayscale value I(i, j) of the target category in the infrared image detection frame area is read pixel by pixel, and grayscale pseudo-color conversion is performed according to the concentration characteristics of alkane gas. Based on the piecewise linear transformation, each grayscale value in the original grayscale image is sent to the R, G, and B converters with different transformation rules, and they are encoded respectively to generate three components of the RGB color space. Then, the three primary colors R, G, and B are synthesized into corresponding colors according to the corresponding proportions of different components, where I(i, j) represents the grayscale value of the pixel at the coordinate (i, j) in the image, R(i, j), G(i, j), and B(i, j) respectively represent the component values ​​of red, green, and blue at the coordinate (i, j) in the image, and the pseudo-color conversion encoding formula is as follows:

[0128]

[0129] As an optional implementation, the step of fusing the pseudo-color into the registered visible light image includes:

[0130] Step 6.1: Based on the three-channel component values ​​obtained by grayscale pseudo-color conversion, the pixel values ​​are recorded as I R(i,j) ,I G(i,j) ,I B(i,j) ;

[0131] Step 6.2: According to the pixel position of the target category, read the three-channel component values ​​of the corresponding area of ​​the registered visible light image pixel by pixel, and record them as V R(i,j) 、V G(i,j) 、V B(i,j) , and then reassign the pixel positions corresponding to the visible light image as follows:

[0132]

[0133] Thus, the pseudo-color is fused into the registered visible light image and output.

[0134] According to the pseudo-color conversion and fusion operation of the above example, the detection frame area is first extracted from the infrared image according to the deep learning algorithm. Since the detection target of the embodiment of the present disclosure is only one type of alkane gas (taking methane as an example), the detection frame area can be divided into two categories, target and background, by using the traditional algorithm, and the grayscale values ​​of the pixels of the corresponding classification are counted. The grayscale threshold L that maximizes the inter-class variance of the grayscale proportion of the target and background areas is obtained by calculation. T The infrared image detection frame area is binarized into two categories: background and target. The target category in the detection frame area is further transformed by piecewise linear transformation using the grayscale-color transformation method. The pseudo-color after linear transformation can be used to determine the concentration of leaked gas in real time. Finally, the target area with pseudo-color is fused on the visible light image to achieve real-time visual representation and annotation of leakage.

[0135] As an optional implementation, the method further includes the following steps:

[0136] The infrared images and visible light images fused with gas pseudo-color are stitched and transcoded, and then output through server streaming.

[0137] As an optional implementation, the step of stitching and transcoding the infrared image and the visible light image fused with the gas pseudo-color, and outputting the image through server streaming, includes:

[0138] The original infrared image and the visible light image fused with gas pseudo-color are stitched and encoded into RTMP video stream, and then pushed through the server.

[0139] As an optional embodiment, the aforementioned computer system is implemented using an embedded system, such as an ARM-based Ubuntu embedded operating system, and a server is built in the embedded system according to the RTMP video stream protocol to perform image stitching and RTMP video stream encoding, and the stream is pushed through the server.

[0140] Combination Figure 4 The figure shows the pseudo-color fusion flow of leaking gas after dual-light registration. Figure 5 It shows real-time visible light large field of view fusion streaming. In an embodiment of the present invention, in view of the defects of the irregular edge of the gas target that are difficult to detect under a large field of view in actual application, it is necessary to use an infrared small field of view for targeted leakage point detection. Since the scene is not rich under the infrared small field of view, it is difficult to quickly determine the specific location of the leakage point. Therefore, in the method of the present invention, the aligned visible light image is displayed on the large field of view of the visible light camera after pseudo-color fusion. The output visible light large field of view fusion streaming can help detection personnel to more quickly determine the location of gas leakage.

[0141] {Example 2}

[0142] Combination Figure 2 As shown, according to Figure 1 As well as the process of the real-time identification and positioning method of alkane gas leakage based on deep learning in the above embodiment, according to the embodiment disclosed in the present invention, a real-time identification and positioning device of alkane gas leakage based on deep learning is also proposed, including:

[0143] An infrared image acquisition module is used to image a target object based on an infrared camera to acquire an infrared image;

[0144] A visible light image acquisition module is used to image the target object based on a visible light camera to acquire a visible light image;

[0145] A fast registration module is configured to perform fast registration based on the acquired infrared image and visible light image based on imaging area and field of view offset correction to obtain a registered visible light image;

[0146] A gas target detection module, used for performing gas target detection according to the acquired infrared image;

[0147] A grayscale pseudo-color conversion module is used to perform pseudo-color conversion according to the detected gas target and gas concentration, and obtain pseudo-color through piecewise linear transformation to represent the concentration of leaked gas in real time; and

[0148] Image fusion module, used to fuse pseudo-color into the registered visible light image.

[0149] In an optional embodiment, the aforementioned real-time identification and positioning device for alkane gas leakage based on deep learning

[0150] The image output module is used to stitch and transcode infrared images and visible light images fused with gas pseudo-color, and output them through server streaming.

[0151] As an optional implementation, the image output module is configured to perform streaming output in the following manner:

[0152] The original infrared image and the visible light image fused with gas pseudo-color are stitched and encoded into RTMP video stream, and then pushed through the server.

[0153] It should be understood that the functions, outputs and specific implementation processes of the above-mentioned infrared image acquisition module, visible light image acquisition module, fast registration module, gas target detection module, grayscale pseudo-color conversion module and image fusion module can be implemented in the same manner as the corresponding process of the real-time identification and positioning method for alkane gas leaks based on deep learning in the aforementioned embodiment.

[0154] {Example 3}

[0155] In combination with the above embodiments, according to the present disclosure, a computer system is further proposed, including:

[0156] one or more processors;

[0157] Memory stores instructions that can be operated.

[0158] Wherein, when the instruction is executed by the one or more processors, the one or more processors perform operations, and the operations include the process of executing the real-time identification and positioning method of alkane gas leakage based on deep learning of the aforementioned embodiment.

[0159] {Example 4}

[0160] In combination with the above embodiments, according to the present disclosure, a computer-readable medium storing software is also proposed, wherein the software includes instructions that can be executed by one or more computers, and when the instructions are executed by the one or more computers, the process of the real-time identification and positioning method of alkane gas leaks based on deep learning of the aforementioned embodiment is performed.

[0161] {Example 5}

[0162] In combination with the above embodiments, according to the present disclosure, a computer program product is also proposed, including a computer program, which, when executed by a processor, implements the process of the real-time identification and positioning method of alkane gas leakage based on deep learning in the aforementioned embodiment.

[0163] According to the technical scheme of the above implementation mode, in response to the detection demand for real-time identification of gas leaks, according to the grayscale characteristics of alkane gases under infrared imaging, the real-time identification and positioning method of alkane gas leaks based on deep learning proposed in the present invention quickly detects gas leaks through artificial intelligence algorithms and performs grayscale pseudo-color conversion through image processing. On the basis of rapid matching of infrared and visible light fields of view, the grayscale pseudo-color of the gas leak is fused and labeled in the visible light field of view, thereby solving the problem of insufficient manual detection accuracy, improving the detection accuracy of gas leaks, and having a better monitoring effect.

[0164] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A real-time identification and positioning method for alkane gas leakage based on deep learning, characterized in that: include: Imaging the target object based on the infrared camera to obtain an infrared image; Imaging the target object based on a visible light camera to obtain a visible light image; According to the acquired infrared image and visible light image, rapid registration is performed based on imaging area and field of view offset correction to obtain a registered visible light image; Performing gas target detection according to the acquired infrared image; Perform pseudo-color conversion according to the detected gas target and gas concentration, and obtain pseudo-color through piecewise linear transformation to represent the concentration of leaked gas in real time; and Fusion of pseudo-color to registered visible light images.

2. The real-time identification and positioning method for alkane gas leakage based on deep learning according to claim 1 is characterized in that: The method of performing rapid registration based on imaging area and field of view offset correction to obtain a registered visible light image includes: Step 1.1: For a target ×N target The distance from the observation target to the center of the infrared camera of the infrared oil and gas leakage imaging device is obtained as D target , unit is m; the aforementioned M target ×N target Respectively represent the equivalent length and width of the observed target; Step 1.2: Get the infrared camera resolution in M ir ×N ir , the unit is pixel, the pixel spacing of the infrared camera I ir , unit is um, focal length F ir , the unit is mm, and the area of ​​the observed target in the imaging field of the infrared camera is MD ir ×ND ir , then there is MD ir and ND ir The calculation formula is as follows: Step 1.3: Get the visible light camera resolution to M vis ×N vis , the unit is pixel, the pixel spacing of the visible light camera I vis , unit is um, focal length F vis , the unit is mm, and the area of ​​the observed target in the imaging field of the visible light camera is MD vis ×ND vis , then there is MD vis and ND vis The calculation formula is as follows: Step 1.4: Get the distance baseline between the infrared camera and the visible light camera as D line , the unit is mm, then the offset T of the visible light camera imaging field of view with the center of the infrared camera imaging field of view as the origin is: Step 1.5: Get the coordinates of the target object in the infrared camera imaging field of view as [X ir Y ir L ir H ir ],in: L ir =MD ir ; H ir =ND ir ; Among them, X ir Y is the horizontal coordinate of the left vertex of the target object in the infrared camera imaging field of view, ir is the left vertex ordinate of the target object in the infrared camera imaging field of view, L ir is the length of the target in the infrared camera imaging field of view, H ir is the width of the target in the imaging field of the infrared camera; Step 1.6: Get the coordinates of the target object in the visible light camera imaging field of view as [X vis Y vis L vis H vis ],in: L vis =MD vis ; H vis =ND vis ; Among them, X vis Y is the horizontal coordinate of the left vertex of the target object in the imaging field of view of the visible light camera, vis is the ordinate of the left vertex of the target object in the imaging field of view of the visible light camera, L vis is the length of the target in the imaging field of the visible light camera, H vis is the width of the target in the imaging field of view of the visible light camera.

3. The real-time identification and positioning method for alkane gas leakage based on deep learning according to claim 1 is characterized in that: The gas target detection according to the acquired infrared image includes: The acquired infrared image is input into the trained target detection model, and an infrared image with a detection frame is output.

4. The real-time identification and positioning method for alkane gas leakage based on deep learning according to claim 3 is characterized in that: The pseudo-color conversion is performed according to the detected gas target and gas concentration, and the pseudo-color color is obtained by piecewise linear transformation to represent the concentration of the leaked gas in real time, including: Step 5.1: Let the size of the detection frame area of ​​the infrared image output by the gas target detection be R×C, and count the gray level L of the pixels in the detection frame area and the number L corresponding to each gray level i ; Set the gray level threshold L T Divide all pixels in the detection frame area into two categories: background and target, where the number of background pixels is N0 and the number of target pixels is N1; Step 5.2: Let the ratio of background pixels to the entire detection frame area be W0, and the grayscale value mean be u0; let the ratio of target pixels to the entire detection frame area be W1, and the grayscale value mean be u1; let the total grayscale mean of the detection frame area be u, then: N0+N1=R×C; W0+W1=1; Then we further get: Step 5.3: Let the inter-class variance of the detection box area be Var, then: Var=W0×(u0-u) 2 +W1×(u1-u) 2 =W0×W1×(u0-u1) 2 ; The traversal method is used to traverse L gray levels to obtain the gray level threshold L that maximizes the inter-class variance Var of the detection frame area. T , the infrared image detection frame area is binarized into two categories: background and target; Step 5.4: According to step 5.3, the grayscale value I(i, j) of the target category in the infrared image detection frame area is read pixel by pixel, and grayscale pseudo-color conversion is performed according to the concentration characteristics of alkane gas. Each grayscale value in the original grayscale image is sent to the R, G, and B converters with different transformation rules, and they are encoded respectively to generate three components of the RGB color space. Then, the three primary colors R, G, and B are synthesized into corresponding colors according to the corresponding proportions of different components to achieve output based on piecewise linear transformation; wherein, I(i, j) represents the grayscale value of the pixel at the coordinate (i, j) in the image, R(i, j), G(i, j), and B(i, j) respectively represent the component values ​​of red, green, and blue at the coordinate (i, j) in the image, and the pseudo-color conversion encoding formula is as follows:

5. The real-time identification and positioning method for alkane gas leakage based on deep learning according to claim 4 is characterized in that: The step of fusing the pseudo-color into the registered visible light image comprises: Step 6.1: Based on the three-channel component values ​​obtained by grayscale pseudo-color conversion, the pixel values ​​are recorded as I R(i,j) ,I G(i,j) ,I B(i,j) ; Step 6.2: According to the pixel position of the target category, read the three-channel component values ​​of the corresponding area of ​​the registered visible light image pixel by pixel, and record them as V R(i,j) 、V G(i,j) 、V B(i,j) , and then reassign the pixel positions corresponding to the visible light image as follows: Thus, the pseudo-color is fused into the registered visible light image and output.

6. The real-time identification and positioning method for alkane gas leakage based on deep learning according to claim 1 is characterized in that: The method further comprises the following steps: The infrared images and visible light images fused with gas pseudo-color are stitched and transcoded, and then output through server streaming.

7. The method for real-time identification and positioning of alkane gas leakage based on deep learning according to claim 6 is characterized in that: The step of stitching and transcoding the infrared image and the visible light image fused with the gas pseudo-color, and outputting the image through server streaming, includes: The original infrared image and the visible light image fused with gas pseudo-color are stitched and encoded into RTMP video stream, and then pushed through the server.

8. A computer system, characterized in that: include: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, wherein the operations include the process of executing the method described in any one of claims 1 to 7.

9. A computer-readable medium storing software, characterized in that: The software includes instructions that can be executed by one or more computers, and when the instructions are executed by the one or more computers, the process of the method according to any one of claims 1 to 7 is performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the process of the method according to any one of claims 1 to 7 is implemented.

Images