Three-dimensional imaging method, device, electronic device and storage medium for gas cloud

Abstract

The present application relates to the technical field of gas detection, and discloses a method, device, electronic device and storage medium for three-dimensional imaging of a gas cloud; the method includes: collecting binocular infrared images and a single visible light image of a target area; extracting the gas cloud contour of the leaked gas based on the binocular infrared images, and determining the gas type of the leaked gas; reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness; constructing a gas concentration inversion model, and inverting the gas concentration of the leaked gas in combination with the gas cloud thickness and the gas type; fusing the single visible light image, the gas concentration and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud. The present application can improve the accuracy of gas concentration inversion and obtain an intuitive three-dimensional visualization image.

Description

Technical Field

[0001] The present disclosure relates to the field of gas detection technology, and in particular to a gas cloud three-dimensional imaging method, device, electronic device and storage medium. Background Art

[0002] Gas leakage is a common safety problem in many industrial productions, and early warning is a major need for emergency response. Commonly used sensors in the field of gas detection include electrochemical sensors, optical sensors, infrared sensors, etc., and there are active and passive detection based on the working principle. For example, laser gas detectors use the wavelength tuning characteristics of lasers and the selective absorption of gases to monitor concentrations. They have high stability, but require their own light source and are active detection. Infrared imaging can visually display gas clouds, which helps to quickly locate the source of the leak.

[0003] Based on the Beer-Lambert law and the characteristic absorption theory of gas molecules in the infrared spectrum, the Infrared Multi-aperture Snapshot Spectral Imaging System (IMASSI) can quickly realize the simultaneous perception of multiple gases, with the advantages of non-contact, fast response, and large-range detection. However, IMASSI is a planar imaging system based on a two-dimensional camera and cannot perceive the depth information of air masses. Therefore, traditional infrared gas cloud cameras cannot distinguish between the two scenes of "sparse and wide" and "dense and small". Summary of the invention

[0004] In view of the above situation, the embodiments of the present application provide a gas-cloud three-dimensional imaging method, device, electronic device and storage medium, which aim to solve the above problems or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present application provides a gas cloud three-dimensional imaging method, the method comprising:

[0006] Collect binocular infrared images and a single visible light image of the target area;

[0007] Extracting the gas cloud contour of the leaking gas based on the binocular infrared image, and determining the gas type of the leaking gas;

[0008] reconstructing three-dimensional point cloud information of the gas cloud based on the binocular infrared image and the gas cloud contour, wherein the three-dimensional point cloud information of the gas cloud includes the thickness of the gas cloud;

[0009] Construct a gas concentration inversion model, and invert the gas concentration of the leaked gas based on the gas cloud thickness and gas type;

[0010] A single visible light image, gas concentration and three-dimensional point cloud information of the gas cloud are fused to generate a three-dimensional image of the gas cloud.

[0011] In a second aspect, an embodiment of the present application further provides a gas cloud three-dimensional imaging device, which includes:

[0012] A binocular infrared camera for collecting binocular infrared images of a target area;

[0013] A visible light camera for collecting visible light images of the target area;

[0014] A gas detection module for extracting the gas cloud contour of the leaked gas based on the binocular infrared images;

[0015] A gas qualitative module for determining the gas type of the leaked gas based on the binocular infrared images;

[0016] A three-dimensional reconstruction module for reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness;

[0017] A concentration inversion module for constructing a gas concentration inversion model and inversing the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type;

[0018] An image fusion module for fusing a single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud.

[0019] In a third aspect, an embodiment of the present application further provides a gas cloud three-dimensional imaging device, which includes:

[0020] An acquisition module for acquiring binocular infrared images and a single visible light image of a target area;

[0021] A processing module for extracting the gas cloud contour of the leaked gas and determining the gas type of the leaked gas based on the binocular infrared images; reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness; constructing a gas concentration inversion model and inversing the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type; fusing a single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud.

[0022] In a fourth aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the steps of the first aspect above.

[0023] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute the steps of the first aspect above.

[0024] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects: By accurately obtaining the three-dimensional point cloud information (including thickness) of the gas cloud through binocular infrared images, combining visible light images to enhance the spatial scene details, and constructing a concentration inversion model based on the gas cloud thickness and gas type, it effectively solves the problem that traditional two-dimensional imaging cannot distinguish between "thin and wide" and "thick and small" gas clouds, significantly improves the gas concentration inversion accuracy. At the same time, intuitive three-dimensional visualization images are generated through multi-modal data fusion, realizing the rapid positioning of the leakage source and the accurate assessment of the diffusion trend, with high precision, strong anti-interference ability and real-time performance. Brief Description of the Drawings

[0025] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0026] Figure 1 A schematic flow chart of the gas cloud three-dimensional imaging method provided by the embodiments of the present application is shown;

[0027] Figure 2 A structural diagram of the gas cloud three-dimensional imaging device provided by the embodiments of the present application is shown;

[0028] Figure 3 A structural diagram of an electronic device provided by the embodiments of the present application is shown. Detailed Embodiments

[0029] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such use can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants should be interpreted as an open term meaning "including but not limited to".

[0031] As described in the background art, IMASSI is a planar imaging system based on a two-dimensional camera and cannot sense the depth information of the air mass. Therefore, traditional infrared gas cloud cameras cannot distinguish between two scenarios: "thin and wide" and "thick and small". Accurately detecting the thickness of the air mass is a prerequisite for correctly calculating the gas concentration. In view of this situation, there is an urgent need to develop a technology that can achieve three-dimensional imaging of leaked gas.

[0032] The principle of binocular stereo vision is to use two or more cameras to observe the same scene from two or more perspectives, collect two or more images at different perspectives at the same time, and obtain the corresponding spatial position information and size information according to the corresponding viewpoint differences of the same target object in different images. The binocular detection algorithm in the visible light band is relatively mature, while the research on three-dimensional imaging of infrared gas based on stereo vision is less.

[0033] Based on this, the present application proposes a gas cloud three-dimensional imaging method, which accurately obtains the three-dimensional point cloud information (including thickness) of the gas cloud through binocular infrared images, combines visible light images to enhance the spatial scene details, constructs a concentration inversion model based on the gas cloud thickness and gas type, effectively solves the problem that traditional two-dimensional imaging cannot distinguish between "thin and wide" and "thick and small" gas clouds, significantly improves the gas concentration inversion accuracy, and at the same time generates an intuitive three-dimensional visualization image through multi-modal data fusion, realizing the rapid positioning of the leakage source and the accurate assessment of the diffusion trend, with high precision, strong anti-interference ability and real-time performance.

[0034] The following will describe in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

[0035] Figure 1 The flowchart of the gas cloud three-dimensional imaging method provided by the embodiment of the present application is shown. From Figure 1 it can be seen that the present application at least includes steps S101-step S105:

[0036] Step S101: Collect binocular infrared images and a single visible light image of the target area.

[0037] In some embodiments, two infrared 9-channel multi-aperture spectral cameras are used to simultaneously collect infrared images to obtain infrared images from two perspectives. Among them, the 9 spectral channels include a full-pass channel and 8 spectral channels with central wavelengths corresponding to the spectral absorption peaks of various gases. The infrared 9-channel multi-aperture spectral camera splits light through a microlens array, uses the full-pass channel to detect dynamic dispersed air masses, and the filter channels are designed according to the central wavelength and absorption bandwidth corresponding to the detected gas, and can be replaced or adjusted according to actual application requirements.

[0038] In some embodiments, preprocessing is performed on the collected binocular infrared images, including reference correction and non-uniformity correction. Among them, reference correction is to eliminate baseline offsets (such as dark current, environmental noise, etc.) and restore the data to the true physical signal. Non-uniformity correction is to eliminate the image non-uniformity (such as stripes, brightness differences) caused by inconsistent responses of each pixel of the detector.

[0039] Step S102: Extract the gas cloud contour of the leaked gas based on the binocular infrared images, and determine the gas type of the leaked gas.

[0040] Step S103: Reconstruct the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour. The three-dimensional point cloud information of the gas cloud includes the gas cloud thickness.

[0041] Step S104: Construct a gas concentration inversion model, and invert the gas concentration of the leaked gas in combination with the gas cloud thickness and the gas type.

[0042] Step S105: Fuse a single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud.

[0043] From Figure 1 As can be seen from the method shown above, the present application accurately obtains the three-dimensional point cloud information (including thickness) of the gas cloud through binocular infrared images, enhances the spatial scene details in combination with visible light images, constructs a concentration inversion model based on the gas cloud thickness and the gas type, effectively solves the problem that traditional two-dimensional imaging cannot distinguish between "thin and wide" and "thick and small" gas clouds, significantly improves the gas concentration inversion accuracy, and at the same time generates an intuitive three-dimensional visualization image through multi-modal data fusion, realizing the rapid positioning of the leakage source and the accurate assessment of the diffusion trend, with high precision, strong anti-interference ability and real-time performance.

[0044] In the gas cloud three-dimensional imaging method provided by the present application, after collecting and preprocessing the infrared images in the above step S101, channel segmentation is performed on the infrared images to obtain full-channel data and single-channel data. Specifically, during channel segmentation, the image is split by determining the coordinates of the starting point and the ending point of each channel.

[0045] Further, the processed data of each segmented channel is processed. Specifically, first, equalization processing is performed on the images of each channel, and a two-point correction model for channel 1 and channel 2 is established as follows:

[0046]

[0047]

[0048] Among them, and 、 The original data output by the detectors of infrared camera 1 and infrared camera 2 respectively, and the data corrected by two-point correction; and 、 and are the gain and bias matrices of channel 1 and channel 2 respectively.

[0049] Furthermore, perform linear transformation and quadratic equalization correction between channels. Taking channel 1 as the reference benchmark, make the correction model of channel 2 consistent with that of channel 1. The output data of channel 2 after quadratic equalization is as follows:

[0050]

[0051] Combining the two-point correction models of channel 1 and channel 2, we can get:

[0052]

[0053] Finally, the output result of band-pass filtering is obtained by subtracting the outputs of the two channels as follows, that is, the corrected single-channel data:

[0054]

[0055] In the gas cloud three-dimensional imaging method provided by this application, the gas cloud contour of the leaking gas is extracted based on the full-channel data in the above step S102. In some embodiments, the frame difference method is used to perform difference operation between the current frame and the gas-free reference frame, extract the gas cloud contour, and obtain the gas cloud region; enhance the contrast of the gas cloud region based on guided filtering; extract gas traces based on a convolutional neural network, and perform pseudo-color transformation on the gas cloud region.

[0056] Specifically, by comparing the current frame with gas and the gas-free reference frame, the radiance change caused by gas absorption or emission in the scene is found. When there is no gas in the path, the radiance received by the detector is the radiance from the background. When there is gas in the path, the radiance received by the detector includes the background radiance absorbed by the gas and the radiance from the gas. For the imaging system, assuming that the transmission path is a uniform atmosphere and the path transmission efficiency is set to 1, subtracting the current frame with gas from the gas-free reference frame can obtain the change in pupil radiance caused by gas absorption / emission, so as to directly separate the radiance change region caused by the gas cloud. Further, enhance the contrast of the gas cloud region through guided filtering and reduce noise. Combine a pre-trained convolutional neural network to extract gas traces in the gas cloud region, and perform pseudo-color transformation on the gas cloud region to suppress the interference of complex moving backgrounds such as people and vehicles existing in the actual scene.

[0057] In the gas cloud three-dimensional imaging method provided by this application, in the above step S102, the gas type of the leaked gas is specifically determined based on single-channel data. In some embodiments, the change amount of the DN value of each channel is determined according to the single-channel data; the change amount of the DN value is subjected to similarity matching with the standard gas response library, and the gas with the highest similarity is selected as the candidate gas; based on the matching degree between the peak value of the change amount of the DN value and the standard peak value of the candidate gas, it is determined whether the leaked gas is the candidate gas.

[0058] Specifically, the change amount of the DN value is subjected to similarity matching with the standard gas response library. For example, the similarity is calculated through the Euclidean distance, cosine similarity, Pearson correlation coefficient, etc., and the gas with the highest similarity is selected as the candidate gas. Further, peak detection is performed on the actual DN value change curve, the standard peak position and the actual peak position of the candidate gas are extracted, and the position offset error is calculated. If the offset error is less than the preset error, the candidate gas is used as the gas type of the leaked gas. In addition, the intensity matching degree between the actual peak value and the standard peak value can also be compared. If the intensity matching degree is greater than the threshold, it is determined that the candidate gas is the effective gas type of the leaked gas.

[0059] In the gas cloud three-dimensional imaging method provided by this application, the above step S103 can be specifically implemented as: performing distortion correction and time synchronization on the binocular infrared images; within the gas cloud contour region, searching for matching points of the left and right images based on the epipolar constraint and the adaptive window matching algorithm; calculating the gas cloud depth according to the disparity of the matching points; calculating the gas cloud thickness based on the gas cloud depth, and constructing the gas cloud three-dimensional point cloud information.

[0060] Specifically, the internal parameter distortion of the binocular infrared camera is eliminated through distortion correction to ensure the geometric consistency of the left and right images. Through time synchronization, the left and right images are strictly aligned in time to avoid motion blur or frame misalignment. Through the epipolar constraint, the search for matching points of the right image is restricted to the epipolar line corresponding to the left image, reducing the computational complexity. Taking the pixel to be matched as the center, a regular-shaped rectangular matching window is established. During the process of searching for matching points, while the window moves along the epipolar line, it is judged whether the pixels within the window and the pixel to be matched come from the same region after image segmentation, and the pixel points within the window that do not come from the same region as the pixel to be matched are excluded. The matching point with the highest similarity is selected, the disparity between the two is calculated, the depth is determined through the disparity, the thickness of the gas cloud in the three-dimensional space is calculated by integrating the depth change amount along the line of sight direction, and a three-dimensional point cloud map is generated based on the depth of each pixel point.

[0061] In some embodiments of this application, the gas concentration inversion model is determined based on the following formula:

[0062]

[0063] where, Determined based on the gas type, is the gas absorption coefficient, is the gas cloud thickness, is the gas concentration, is the radiation power received by the detector, and are the background radiation and gas radiation respectively, is the wavelength, is the characteristic spectral band, is the background temperature, is the gas temperature, is the time, is the calibration coefficient.

[0064] Specifically, assume that at the characteristic spectral band (i.e., within the bandwidth of a certain channel), the background radiation at time e is . Assume that gas starts to leak at time (e + 1). At this time, the background radiation is still considered equal to that at time e and is considered to remain unchanged for a relatively long period. Then there is

[0065]

[0066] wherein, is the radiance absorbed / emitted by the gas, , wherein, A is the area of the sensor optical aperture, Ω is the solid angle of the sensor instantaneous field of view, and τ(λ) is the transmission efficiency, is the Planck blackbody formula, , wherein, is the wavelength, is the absolute temperature of the blackbody, , .

[0067] Furthermore, if the concentration at time e is known and assuming that the temperature of the leaked gas remains unchanged, the information at time (e + 1) can be directly inverted using the information at time e.

[0068]

[0069] That is, let , and the background radiation and gas thermal radiation are constants that do not change with time within a certain period. Then:

[0070]

[0071] Combined with the Lambert-Beer law: , the gas concentration inversion model can be obtained:

[0072]

[0073] In the embodiments of the present application, when constructing the concentration inversion model, by combining the gas cloud thickness, the accurate inversion of the concentration of the leaked gas is realized, filling the gap that the traditional gas cloud camera cannot distinguish the gas cloud density difference.

[0074] In the gas cloud three-dimensional imaging method provided by the present application, step S105 can be specifically implemented as projecting the gas cloud three-dimensional point cloud information onto the two-dimensional plane of a single visible light image; performing pseudo-color mapping on the gas cloud area of the visible light image according to the gas concentration, and marking the concentration contour line and the gas cloud thickness to generate the concentration contour line.

[0075] Based on the above method, the present application also provides a gas cloud three-dimensional imaging device, which includes a binocular infrared camera, a visible light camera, a gas detection module, a gas qualitative module, a three-dimensional reconstruction module, a concentration inversion module, and an image fusion module.

[0076] Among them, the binocular infrared camera is used to collect the binocular infrared images of the target area;

[0077] The visible light camera is used to collect the visible light images of the target area;

[0078] The gas detection module is used to extract the gas cloud contour of the leaked gas based on the binocular infrared images;

[0079] The gas qualitative module is used to determine the gas type of the leaked gas based on the binocular infrared images;

[0080] The three-dimensional reconstruction module is used to reconstruct the gas cloud three-dimensional point cloud information based on the binocular infrared images and the gas cloud contour, and the gas cloud three-dimensional point cloud information includes the gas cloud thickness;

[0081] The concentration inversion module is used to construct a gas concentration inversion model, and inversely calculate the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type;

[0082] The image fusion module is used to fuse a single visible light image, the gas concentration, and the gas cloud three-dimensional point cloud information to generate a three-dimensional image of the gas cloud.

[0083] In some embodiments, the gas detection module is specifically used to perform differential operation on the current frame and the gas-free reference frame based on the frame difference method to extract the gas cloud contour and obtain the gas cloud area; enhance the contrast of the gas cloud area based on guided filtering; extract the gas trace based on the convolutional neural network, and perform pseudo-color transformation on the gas cloud area.

[0084] In some embodiments, the gas qualitative module is specifically used to determine the change amount of the DN value of each channel according to the single-channel data; perform similarity matching on the change amount of the DN value with the standard gas response library, and select the gas with the highest similarity as the candidate gas; determine whether the leaked gas is the candidate gas based on the matching degree between the peak value of the change amount of the DN value and the standard peak value of the candidate gas.

[0085] In some embodiments, the 3D reconstruction module is specifically configured to perform distortion correction and time synchronization on the binocular infrared images; search for matching points of the left and right images within the gas cloud contour region based on the epipolar constraint and the adaptive window matching algorithm; calculate the gas cloud depth according to the disparity of the matching points; calculate the gas cloud thickness based on the gas cloud depth, and construct the 3D point cloud information of the gas cloud.

[0086] In some embodiments, the image fusion module is specifically configured to project the 3D point cloud information of the gas cloud onto the two-dimensional plane of a single visible light image; perform pseudo-color mapping on the gas cloud region of the visible light image according to the gas concentration, and mark the concentration isocontours and the gas cloud thickness to generate the concentration isocontours.

[0087] This application combines a multi-aperture snapshot infrared spectroscopic gas cloud imaging system with binocular vision technology for the first time. The design of the dual system can not only bring three-dimensional information, but also expand the number of spectral channels, overcome the limitations of a single imaging device in gas detection, and enhance the ability to locate and detect gas leaks; utilize visible light and infrared image fusion technology to comprehensively display the dual-light three-dimensional shape and distribution information of the gas cloud, improving the comprehensiveness and accuracy of gas cloud monitoring; introduce visible-infrared binocular vision information inversion technology to accurately measure the gas cloud thickness information, and then accurately invert the concentration of the leaked gas, filling the gap that traditional gas cloud cameras cannot distinguish the gas cloud density difference; establish a gas concentration inversion model in an open space, solve the problem that traditional models cannot consider complex diffusion characteristics, and provide a complete method for accurately predicting the gas concentration in an open space.

[0088] In some embodiments of the present application, a gas cloud 3D imaging device is provided, and the gas cloud 3D imaging device corresponds one-to-one to the gas cloud 3D imaging method in the above embodiments. As Figure 2 shown, the gas cloud 3D imaging device includes an acquisition module 101 and a processing module 102.

[0089] The acquisition module 101 is configured to acquire binocular infrared images and a single visible light image of the target area;

[0090] The processing module 102 is configured to extract the gas cloud contour of the leaked gas based on the binocular infrared images and determine the gas type of the leaked gas; reconstruct the 3D point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour, where the 3D point cloud information of the gas cloud includes the gas cloud thickness; construct a gas concentration inversion model, and invert the gas concentration of the leaked gas in combination with the gas cloud thickness and the gas type; fuse the single visible light image, the gas concentration, and the 3D point cloud information of the gas cloud to generate a 3D image of the gas cloud.

[0091] In some embodiments of the present application, in the above-mentioned device, the processing module 102 is specifically configured to perform channel segmentation on the binocular infrared image to obtain full-channel data and single-channel data; extract the gas cloud contour of the leaked gas based on the full-channel data; and determine the gas type of the leaked gas based on the single-channel data.

[0092] In some embodiments of the present application, in the above-mentioned device, the processing module 102 is specifically configured to perform two-point correction on each channel of data respectively; perform secondary equalization correction on each channel of data with the target channel as the reference to make the filter channel response characteristics consistent with the all-pass channel; calculate the difference between each channel of data and the channel data after secondary equalization correction, and output the target single-channel data and the target full-channel data.

[0093] In some embodiments of the present application, in the above-mentioned device, the processing module 102 is specifically configured to use the inter-frame difference method to perform difference operation on the current frame and the gas-free reference frame, extract the gas cloud contour to obtain the gas cloud region; enhance the contrast of the gas cloud region based on guided filtering; extract gas traces based on a convolutional neural network, and perform pseudo-color transformation on the gas cloud region.

[0094] In some embodiments of the present application, in the above-mentioned device, the processing module 102 is specifically configured to determine the change amount of the DN value of each channel according to the single-channel data; perform similarity matching between the change amount of the DN value and the standard gas response library, and select the gas with the highest similarity as the candidate gas; determine whether the leaked gas is the candidate gas based on the matching degree between the peak value of the change amount of the DN value and the standard peak value of the candidate gas.

[0095] In some embodiments of the present application, in the above-mentioned device, the processing module 102 is specifically configured to perform distortion correction and time synchronization on the binocular infrared image; search for matching points of the left and right images within the gas cloud contour region based on the epipolar constraint and the adaptive window matching algorithm; calculate the gas cloud depth according to the disparity of the matching points; calculate the gas cloud thickness based on the gas cloud depth, and construct the three-dimensional point cloud information of the gas cloud.

[0096] In some embodiments of the present application, in the above-mentioned device, the gas concentration inversion model is determined based on the following formula:

[0097]

[0098] Where, Determined based on the gas type, Is the gas absorption coefficient, Is the gas cloud thickness, Is the gas concentration, Is the radiation power received by the detector, And Are the background radiation and gas radiation respectively, is the wavelength, is the characteristic spectral band, is the background temperature, is the gas temperature, is the time, is the calibration coefficient.

[0099] In some embodiments of the present application, in the above device, the processing module 102 is specifically configured to project the three-dimensional point cloud information of the gas cloud onto the two-dimensional plane of a single visible light image; perform pseudo-color mapping on the gas cloud region of the visible light image according to the gas concentration, and mark the concentration contour lines and the thickness of the gas cloud to generate the concentration contour lines.

[0100] It should be noted that any of the above gas cloud three-dimensional imaging devices can correspondingly implement the foregoing gas cloud three-dimensional imaging method, which will not be elaborated here.

[0101] Figure 3 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 3 shown, at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0102] The processor, network interface, and memory can be interconnected through an internal bus, and the internal bus can be an ISA (Industry Standard Architecture, industrial standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture, extended industrial standard structure) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 3 only a bidirectional arrow is used in

[0103] The memory is used to store a program. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0104] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a three-dimensional imaging device for gas clouds at the logical level. The processor executes the program stored in the memory and is specifically used to execute the aforementioned method.

[0105] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0106] This electronic device can execute the gas cloud three-dimensional imaging method provided by multiple embodiments of the present application and implement the functions of the gas cloud three-dimensional imaging device in Figure 3 the embodiments shown. The embodiments of the present application will not be elaborated here.

[0107] The embodiments of the present application also propose a computer-readable storage medium. The computer-readable storage medium stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute the gas cloud three-dimensional imaging method provided by multiple embodiments of the present application.

[0108] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0110] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0112] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0113] The memory may include non-permanent memory in the computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0117] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A three-dimensional imaging method for gas clouds, characterized in that, The method includes: Collecting binocular infrared images and a single visible light image of the target area; Extracting the gas cloud contour of the leaked gas based on the binocular infrared images and determining the gas type of the leaked gas; Reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness; Constructing a gas concentration inversion model and inversely calculating the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type; Fusing the single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud; The gas concentration inversion model is determined based on the following formula: Among them, Determined based on the gas type, Is the gas absorption coefficient, Is the thickness of the gas cloud, Is the gas concentration, Is the radiation power received by the detector, And Are the background radiation and gas radiation respectively, Is the wavelength, Is the characteristic spectral band, Is the background temperature, Is the gas temperature, Is the time, Is the calibration coefficient.

2. The method according to claim 1, wherein The extracting the gas cloud contour of the leaked gas based on the binocular infrared images and determining the gas type of the leaked gas includes: Performing channel segmentation on the binocular infrared images to obtain full-channel data and single-channel data; Extracting the gas cloud contour of the leaked gas based on the full-channel data; Determining the gas type of the leaked gas based on the single-channel data.

3. The method according to claim 2, wherein After obtaining the full-channel data and the single-channel data, the method further includes: Performing two-point correction on each channel data respectively; Taking the target channel as a reference, performing secondary equalization correction on each channel data to make the filter channel response characteristics consistent with the full-pass channel; Calculating the difference between each channel data and the channel data after secondary equalization correction, and outputting the target single-channel data and the target full-channel data.

4. The method according to claim 2 or 3, characterized in that, The extracting the gas cloud contour of the leaked gas based on the full-channel data includes: Using the inter-frame difference method to perform difference operation between the current frame and the gas-free reference frame, extracting the gas cloud contour, and obtaining the gas cloud area; Enhancing the contrast of the gas cloud area based on guided filtering; Extracting gas traces based on a convolutional neural network and performing pseudo-color transformation on the gas cloud area.

5. The method according to claim 2 or 3, characterized in that, The determining the gas type of the leaked gas based on the single-channel data includes: Determining the change amount of the DN value of each channel according to the single-channel data; Performing similarity matching between the change amount of the DN value and the standard gas response library, and selecting the gas with the highest similarity as the candidate gas; Determining whether the leaked gas is the candidate gas based on the matching degree between the peak value of the change amount of the DN value and the standard peak value of the candidate gas.

6. The method according to claim 1, characterized in that The reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared images and the gas cloud contour includes: Performing distortion correction and time synchronization on the binocular infrared images; Searching for matching points of the left and right images within the gas cloud contour area based on the epipolar constraint and the adaptive window matching algorithm; Calculating the gas cloud depth according to the disparity of the matching points; Calculating the gas cloud thickness based on the gas cloud depth and constructing the three-dimensional point cloud information of the gas cloud.

7. The method according to claim 1, characterized in that The fusing the single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud includes: Projecting the three-dimensional point cloud information of the gas cloud onto the two-dimensional plane of the single visible light image; Performing pseudo-color mapping on the gas cloud area of the visible light image according to the gas concentration, and annotating the concentration isopleth and the gas cloud thickness to generate the concentration isopleth.

8. A three-dimensional imaging device for gas clouds, characterized in that, The device includes: A binocular infrared camera for collecting binocular infrared images of the target area; A visible light camera for collecting visible light images of the target area; A gas detection module for extracting the gas cloud contour of the leaked gas based on the binocular infrared images; A gas qualitative module for determining the gas type of the leaked gas based on the binocular infrared image; A three-dimensional reconstruction module for reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared image and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness; A concentration inversion module for constructing a gas concentration inversion model and inversely calculating the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type; An image fusion module for fusing a single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud; The gas concentration inversion model is determined based on the following formula: Among them, Determined based on the gas type, Is the gas absorption coefficient, Is the thickness of the gas cloud, Is the gas concentration, Is the radiation power received by the detector, And Are the background radiation and gas radiation respectively, Is the wavelength, Is the characteristic spectral band, Is the background temperature, Is the gas temperature, Is the time, Is the calibration coefficient.

9. A three-dimensional imaging device for gas clouds, characterized in that, The device includes: An acquisition module for acquiring a binocular infrared image and a single visible light image of the target area; A processing module for extracting the gas cloud contour of the leaked gas based on the binocular infrared image and determining the gas type of the leaked gas; reconstructing the three-dimensional point cloud information of the gas cloud based on the binocular infrared image and the gas cloud contour, where the three-dimensional point cloud information of the gas cloud includes the gas cloud thickness; constructing a gas concentration inversion model and inversely calculating the gas concentration of the leaked gas by combining the gas cloud thickness and the gas type; fusing a single visible light image, the gas concentration, and the three-dimensional point cloud information of the gas cloud to generate a three-dimensional image of the gas cloud; The gas concentration inversion model is determined based on the following formula: Among them, is determined based on the gas type, is the gas absorption coefficient, is the gas cloud thickness, is the gas concentration, is the radiation power received by the detector, and are the background radiation and gas radiation respectively, is the wavelength, is the characteristic spectral band, is the background temperature, is the gas temperature, is the time, is the calibration coefficient.

Images