Image processing method, device and equipment based on gas imaging

Abstract

The present application provides an image processing method, device and equipment based on gas imaging, which is applied to the field of image processing technology. The method includes: collecting gas imaging images and visible light images for the target scene; for each frame of gas imaging image, detecting the air mass, gas position mapping table and gas confidence table in the gas imaging image according to the preset gas detection model to obtain the gas detection result; for the gas detection result, obtaining the foreground frame with air mass and the initial background frame without air mass, and determining the real background frame based on the initial background frame and the corresponding weight coefficient; extracting the gas morphology concentration map from the gas imaging image according to the gas position mapping table, gas confidence table and real background frame in the foreground frame; after the gas morphology concentration map is processed by pseudo-color, it is superimposed on the visible light image to obtain the gas pseudo-color image. Through the above method, the extraction accuracy of gas can be effectively improved, and it is convenient for users to observe.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and equipment based on gas imaging. Background Art

[0002] In industrial and commercial environments, gas leaks can cause serious safety hazards, especially flammable gas leaks that can easily lead to fires, explosions, and poisoning. Therefore, accurate detection and location of gas leaks are critical to ensuring personnel safety and facility integrity.

[0003] At present, the detection of gases such as volatile organic compounds (VOCs) is mainly achieved through infrared detection and non-visible light layer observation, using infrared bands.

[0004] However, infrared imaging is not visible light, and it is difficult to observe the location of gas leakage with the human eye. Summary of the invention

[0005] The present application provides an image processing method, device and equipment based on gas imaging to solve at least one of the above problems.

[0006] In a first aspect, the present application provides an image processing method based on gas imaging, comprising:

[0007] Collect gas imaging images and visible light images of the target scene;

[0008] For each frame of the gas imaging image, the gas mass, the gas position mapping table and the gas confidence table in the gas imaging image are detected according to a preset gas detection model to obtain a gas detection result;

[0009] According to the gas detection result, a foreground frame with an air mass and an initial background frame without an air mass are obtained, and a real background frame is determined based on the initial background frame and a corresponding weight coefficient;

[0010] Extracting a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame;

[0011] The gas form concentration map is processed by pseudo-color and then superimposed on the visible light image to obtain a gas pseudo-color image.

[0012] In one embodiment, the method further comprises:

[0013] Obtaining the preset buffer quantity of the initial background frame;

[0014] Determine the weight coefficient of each initial background frame according to the position index of each initial background frame and the cache quantity; wherein the position index is determined based on the acquisition time of the corresponding initial background frame;

[0015] The determining of the real background frame according to the initial background frame comprises:

[0016] The corresponding initial background frames are weighted according to the weight coefficient of each initial background frame, and each weighted initial background frame is summed to obtain the real background frame.

[0017] In one embodiment, extracting a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame includes:

[0018] Subtracting the real background frame from the gas imaging image to obtain a differential image, filtering the differential image according to a gas position mapping table in the foreground frame, and extracting an initial gas morphology concentration map from the gas imaging image;

[0019] According to the gas confidence table, each pixel in the gas imaging image is weighted by a first gas confidence, and the confidence-weighted image is processed according to the gas position mapping table to obtain a weighted gas form concentration map;

[0020] The gas form concentration map is generated according to the initial gas form concentration map and the weighted gas form concentration map.

[0021] In one embodiment, generating the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map includes:

[0022] Obtaining a first pixel value of each pixel point in the initial gas form concentration map and a second pixel value of each pixel point in the weighted gas form concentration map;

[0023] For each pixel point, selecting the largest pixel value between the first pixel value and the second pixel value as the pixel value of the pixel point;

[0024] The second gas confidence weighting is performed on the pixel points selected by pixel value according to the gas confidence table to obtain the gas form concentration map.

[0025] In one embodiment, the method further comprises:

[0026] For each frame of the gas imaging image, obtaining an undetected frame in the gas imaging image that has not been detected;

[0027] For the undetected frame, obtaining a gas position mapping table of a previous frame of the undetected frame, and determining an initial gas morphology concentration map of the undetected frame according to the gas position mapping table of the previous frame and the real background frame; and,

[0028] The gas form concentration map of the undetected frame is determined according to the initial gas form concentration map of the undetected frame and the gas confidence table of the previous frame.

[0029] In one embodiment, the method further comprises:

[0030] If the previous frame of the undetected frame is an image frame detected by the gas detection model, the gas confidence table of the previous frame is determined as the gas confidence table of the undetected frame;

[0031] If the previous frame of the undetected frame is an image frame that has not been detected by the gas detection model, the gas confidence table of the undetected frame is determined according to the gas concentration morphology diagram of the previous frame and the determined gas confidence table of the previous frame.

[0032] In one embodiment, the method further comprises:

[0033] Obtaining a predetermined density and color mapping table for pixel points, wherein the density and color mapping table includes mapping relationships between different densities and colors of pixel points;

[0034] The step of subjecting the gas form concentration map to pseudo-color processing comprises:

[0035] For each pixel point in the gas form concentration map, query the color value in the concentration and color mapping table according to the concentration value of the pixel point;

[0036] According to the color value of each pixel, pseudo-color processing is performed on the gas form concentration map.

[0037] In one implementation, the gas position mapping table is used to characterize the position of each gas pixel, and the gas confidence table is used to characterize the gas confidence of each pixel.

[0038] According to a second aspect of the present application, there is provided an image processing device based on gas imaging, comprising:

[0039] An image acquisition module, which is configured to acquire a gas imaging image and a visible light image of a target scene;

[0040] A detection module, which is configured to detect the gas mass, gas position mapping table and gas confidence table in each frame of the gas imaging image according to a preset gas detection model to obtain a gas detection result for each frame; wherein the gas position mapping table is used to characterize the position of each gas pixel point, and the gas confidence table is used to characterize the gas confidence of each pixel point;

[0041] A type frame acquisition module, which is configured to obtain a foreground frame with an air mass and an initial background frame without an air mass for each frame of gas detection results through a background frame subtraction algorithm, and determine a real background frame based on the initial background frame and a corresponding weight coefficient;

[0042] a concentration map extraction module, configured to extract a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame;

[0043] The superposition module is configured to superimpose the gas form concentration image on the visible light image after pseudo-color processing to obtain a gas pseudo-color image.

[0044] In one embodiment, the device further comprises:

[0045] A buffer quantity acquisition module, which is configured to acquire the preset buffer quantity of the initial background frame;

[0046] A weight coefficient determination module, which is configured to determine the weight coefficient of each initial background frame according to the position index of each initial background frame and the cache quantity; wherein the position index is determined based on the acquisition time of the corresponding initial background frame;

[0047] The type frame acquisition module includes:

[0048] The background frame determination unit is configured to weight the corresponding initial background frame according to the weight coefficient of each initial background frame, and sum each weighted initial background frame to obtain the real background frame.

[0049] In one embodiment, the concentration map extraction module includes:

[0050] an initial concentration map extraction unit, configured to subtract the real background frame from the gas imaging image to obtain a differential image, and filter the differential image according to a gas position mapping table in the foreground frame to extract an initial gas morphology concentration map from the gas imaging image;

[0051] a weighted concentration map extraction unit, configured to perform first gas confidence weighting on each pixel in the gas imaging image according to the gas confidence table, and process the confidence-weighted image according to the gas position mapping table to obtain a weighted gas morphology concentration map;

[0052] A concentration map generating unit is configured to generate the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map.

[0053] In one embodiment, the concentration map generating unit is specifically configured to: obtain a first pixel value for each pixel point in the initial gas morphology concentration map, and a second pixel value for each pixel point in the weighted gas morphology concentration map; for each pixel point, select the largest pixel value between the first pixel value and the second pixel value as the pixel value of the pixel point; and perform second gas confidence weighting on the pixel points selected by pixel value according to the gas confidence table to obtain the gas morphology concentration map.

[0054] In one embodiment, the device further comprises:

[0055] An undetected frame extraction module, which is configured to obtain, for each frame of the gas imaging image, an undetected frame that has not been detected in the gas imaging image;

[0056] An undetected frame complementation module is configured to obtain, for the undetected frame, a gas position mapping table of a previous frame of the undetected frame, and determine an initial gas morphology concentration map of the undetected frame based on the gas position mapping table of the previous frame and the real background frame; and determine a gas morphology concentration map of the undetected frame based on the initial gas morphology concentration map of the undetected frame and a gas confidence table of the previous frame.

[0057] In one embodiment, the device further comprises:

[0058] A first determination module, configured to determine the gas confidence table of the previous frame as the gas confidence table of the undetected frame if the previous frame of the undetected frame is an image frame detected by the gas detection model;

[0059] The second determination module is configured to determine the gas confidence table of the undetected frame based on the gas concentration morphology diagram of the previous frame and the gas confidence table determined for the previous frame if the previous frame of the undetected frame is an image frame that has not been detected by the gas detection model.

[0060] In one embodiment, the device further comprises:

[0061] A mapping table acquisition module, which is configured to acquire a predetermined density and color mapping table for a pixel point, wherein the density and color mapping table includes a mapping relationship between different densities and colors of the pixel point;

[0062] The superposition module comprises:

[0063] A query unit, configured to query the color value in the concentration and color mapping table according to the concentration value of each pixel in the gas form concentration map;

[0064] The pseudo-color processing unit is configured to perform pseudo-color processing on the gas form concentration map according to the color value of each pixel.

[0065] In one implementation, the gas position mapping table is used to characterize the position of each gas pixel, and the gas confidence table is used to characterize the gas confidence of each pixel.

[0066] According to a third aspect of the present application, there is provided an electronic device, comprising: a processor, a memory communicatively connected to the processor, and a display;

[0067] The memory stores computer-executable instructions;

[0068] The processor executes the computer-executable instructions stored in the memory to implement the image processing method based on gas imaging as provided in any one of the first aspects above.

[0069] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the image processing method based on gas imaging provided in any one of the first aspects above.

[0070] According to a fifth aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, and the computer program implements the method described in any of the above aspects when executed by a processor.

[0071] The image processing method, device and equipment based on gas imaging provided by the present application use a gas detection model to identify the detection results corresponding to the air mass, gas position mapping table and gas confidence table in the gas imaging image, and extract the foreground frame and the real background frame calculated by the weight coefficient based on the detection result, and use the gas position mapping table, the gas confidence table and the real background frame to accurately extract the gas morphology and concentration in the non-background frame of the gas imaging image, and superimpose the extracted gas morphology and concentration on the visible light image in a pseudo-color manner, so that the user can clearly observe the gas distribution position and concentration; in addition, by supplementing the calculation of the gas morphology concentration map for the undetected frames that have not passed the detection model, the refresh frame rate of the gas morphology concentration map can be increased to the same refresh frame rate as that of the visible light camera, thereby achieving the effect of frame supplementation and greatly improving the user's gas observation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0073] Figure 1 A possible scenario diagram provided for an embodiment of the present application;

[0074] Figure 2 A schematic flow chart of an image processing method based on gas imaging provided in an embodiment of the present application;

[0075] Figure 3 for Figure 2 One of the flowchart diagrams of step S204;

[0076] Figure 4 for Figure 2 2. Schematic diagram of the process of step S204;

[0077] Figure 5 One of the flow charts of another image processing method based on gas imaging provided in an embodiment of the present application;

[0078] Figure 6 This is a flow chart of the frame supplement algorithm of the embodiment of the present application;

[0079] Figure 7 A schematic flow chart of another image processing method based on gas imaging provided in an embodiment of the present application;

[0080] Figure 8 A schematic flow chart of an image processing method based on gas imaging provided by an exemplary embodiment of the present application;

[0081] Fig. 9A schematic diagram of the structure of an image processing device based on gas imaging provided in an embodiment of the present application;

[0082] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0083] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0085] To facilitate the understanding of the embodiments of the present application, the terms involved in the embodiments of the present application are first explained:

[0086] Gas cloud imaging: Detect flammable gases in specific bands using thermal imaging technology based on the spectral absorption characteristics of gases.

[0087] Pseudo color: Through computer image enhancement technology, different false colors are given to the grayscale of the target image to improve the recognizability of the image content and improve the visual effect.

[0088] Binocular positioning: Binocular positioning is the process of determining the geometric relationship between two cameras, including the intrinsic matrix, extrinsic matrix and fundamental matrix.

[0089] Frame interpolation: Frame interpolation refers to inserting extra frames into a video or animation through technical means to achieve smooth motion and improve picture fluency.

[0090] Frame rate: Video is a seemingly continuous image composed of independent pictures. Each picture is called a frame. In order to ensure continuity and smoothness, the number of frames per second of the video is fixed, that is, the number of images per second is called the frame rate. For example, 25 frames / second means that one frame of image is output every 40ms, and 25 images are output per second.

[0091] Figure 1 The application scenario diagram of the image processing method, device and equipment based on gas imaging provided in the embodiments of the present application is as follows: Figure 1As shown, the scene includes at least one electronic device 101, a gas imaging device 102, and a visible light imaging device 103, wherein the gas imaging device 102 can be an infrared thermal imaging camera, an optical gas imaging (OGI) camera or other gas imaging device, and the visible light imaging device 103 can be any visible light camera. The gas imaging device 102 and the visible light imaging device 103 can be installed in the same scene (such as a location where the user believes that a gas leak may occur) to collect images in the same scene, namely, gas imaging images and visible light images. Among them, the electronic device 101 is communicated with the gas imaging device 102 and the visible light imaging device 103 respectively, and the electronic device 101 is used to collect the gas imaging image of the gas imaging device 102 and the visible light image of the visible light imaging device 103, and detect the gas detection results corresponding to the air mass, gas position mapping table and gas confidence table in the gas imaging image according to the preset gas detection model, and then obtain the foreground frame and the real background frame for the gas detection result, so as to extract the gas morphology concentration map from the gas imaging image, and superimpose the gas morphology concentration map on the visible light image after pseudo-color processing to obtain a gas pseudo-color image. The electronic device can include but is not limited to computers, smart phones, tablet computers, e-book readers, portable computers, car computers, wearable devices, desktop computers, set-top boxes, smart TVs, or can also be servers or cloud servers, etc.

[0092] Exemplarily, the same scene is photographed using the gas imaging device 102 and the visible light device 103, and the original gas imaging image is obtained and recorded as R_frm and the original visible light image is recorded as V_frm respectively. The original thermal imaging image and the original visible light image are registered using an image registration algorithm to obtain the transformation relationship between the original thermal imaging image and the original visible light image, and a mathematical model of image coordinate transformation is established based on the transformation relationship. The original thermal imaging image and the original visible light image are transformed into the same coordinate system by solving the parameters of the mathematical model, and the coordinate mapping relationship is recorded as COOR (x, y), thereby realizing the combination processing of the two images to obtain a gas pseudo-color image.

[0093] It should be noted that the communication connection between the electronic device 101 and the gas imaging device 102 and the visible light imaging device 103 can be a wired connection, connecting the devices through physical cables such as data cables and network cables, such as Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Ethernet and other interfaces to connect devices for data transmission. It can also be a wireless connection, communicating through radio wave signals, such as wireless fidelity (WiFi), Bluetooth, infrared and other wireless technologies to achieve communication between devices. Or it can be a remote connection, connecting devices through network technologies such as the Internet to achieve remote control and data transmission, such as cloud services, remote desktops, etc. It can also be a near field communication (NFC) connection, which realizes the connection between devices through near field communication technology, usually used for applications such as file transfer; low-power Bluetooth connection, used for short-distance communication connection between devices, such as the connection of smart bracelets, smart watches and other devices; wireless communication technology connection, used for low-speed, short-distance communication connection between devices, suitable for low-power Internet of Things devices. This application does not specifically limit the specific method of communication connection between physical devices.

[0094] This application does not specifically limit the specific form and type of any of the above-mentioned physical devices.

[0095] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0096] Figure 2 The flowchart of the image processing method based on gas imaging provided in the embodiment of the present application is shown in FIG. 1 , and the execution subject may be the electronic device 101 in the above application scenario, such as Figure 2 As shown, the method may include steps S201-S205:

[0097] Step S201: Acquire a gas imaging image and a visible light image of a target scene.

[0098] In this embodiment, the electronic device can collect gas imaging images and visible light images of the target scene through a gas imaging device and a visible light imaging device installed in the same target scene. The target scene can be a scene with potential gas leakage risks, such as a natural gas facility, a chemical plant, etc.

[0099] Step S202: for each frame of the gas imaging image, detect the air mass, the gas position mapping table and the gas confidence table in the gas imaging image according to a preset gas detection model to obtain a gas detection result.

[0100] Among them, the gas position mapping table is used to characterize the position of each gas pixel, and the gas confidence table is used to characterize the gas confidence of each pixel. It can be understood that the gas position mapping table, that is, the distribution position of the pixels belonging to the gas in the image detected by the model, each pixel can have a value range of 0~1, 1 means it is identified as a gas; the gas confidence table, that is, the confidence of each pixel in the image detected by the model being identified as a gas, each pixel can have a value range of 0~100, the higher the value, the higher the confidence of the gas.

[0101] Different from the gas detection scheme in the related art, this embodiment introduces the concept of thermal imaging gas confidence. By using the gas detection model, in addition to outputting the gas mass and gas position mapping table, a thermal imaging gas confidence table con_map can also be output. Its specific format can be a grayscale image with the same size as the original thermal imaging image, in which the value range of each pixel is 0-100. The larger the value, the higher the probability that the point in the original thermal imaging image corresponding to the pixel belongs to the gas. When extracting the gas form concentration, the related art only uses the original thermal imaging image and the background frame to subtract to extract the gas form concentration. In actual scenes, due to the differences in the physical spectral absorption characteristics of different objects, the location where the gas leak occurs in the thermal imaging image may be imaged in black, resulting in the form concentration map extracted by the above method. It cannot reflect the real gas form and concentration. In extreme scenes, even no gas information can be extracted, resulting in the risk of missed or false alarms, and ultimately causing serious consequences that the gas, especially flammable gas or toxic gas leakage, cannot be handled in time. This embodiment is based on a method combining gas confidence, which can effectively improve the accuracy and success rate of gas extraction in different scenarios, thereby avoiding the risk of missed or false alarms in gas detection.

[0102] Optionally, the gas detection model can be a deep learning model, using a model trained by a neural network model, and its training process can be that the gas detection model can be collected by collecting a large amount of gas imaging image data, which should include scenes with and without gas under various conditions to ensure the generalization ability of the model, and the data can be collected in a laboratory setting, a simulated environment, or an actual scene. By annotating the collected image data, such as experts manually marking the gas area in the image, generating labels for training, for example, creating multiple annotations for each image: such as gas mass annotation (i.e., area annotation where gas exists), gas position mapping (a value of 0-1, indicating whether a pixel is gas), and another is gas confidence (which can be a continuous value between 0 and 100, indicating the possibility of gas existence). In addition, these labels can also include the location, shape, and possible concentration information of the gas. After data collection and labeling, you can select models such as convolutional neural network (CNN), regional convolutional neural network (R-CNN), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) as the initial model, and use the labeled data set to train the initial model to obtain a gas detection model that can be used to detect air masses, gas position mapping tables, and gas confidence tables.

[0103] It should be noted that the gas detection model in this embodiment can be pre-stored in the electronic device for use, or a model obtained through internal communication or external communication, for example, it can be a model trained by the electronic device, or it can be a model obtained from other devices or servers.

[0104] Step S203: acquiring a foreground frame with an air mass and an initial background frame without an air mass according to the gas detection result, and determining a real background frame based on the initial background frame and a corresponding weight coefficient.

[0105] Different from the related art, the background detection algorithm (such as Gaussian background modeling) is directly used to extract the background frame in the image, that is, the gas detection model is used to detect the gas, and the background detection algorithm is used to detect the background. Two algorithms are needed to detect the foreground frame and the background frame respectively, which will bring more computing performance occupation. This embodiment uses the results output by a detection model to accurately identify the foreground frame and the background frame, without the need to distinguish the foreground and background targets through background modeling or other background detection algorithms, saving system computing power and improving the extraction efficiency of the foreground frame and the background frame.

[0106] For example, the gas imaging image can be output to the detection model algorithm for detection. When the number of air mass targets outputted by the algorithm is greater than 0, it indicates that there is gas in the current frame and it belongs to the foreground frame. The original image of the current frame can be recorded as the original foreground frame N (that is, foreground frame N); when the number of air mass targets outputted by the algorithm is 0, it indicates that there is no gas in the current frame and it belongs to the background frame. The original gas imaging image of the current frame is recorded as the original background frame M (that is, the initial background frame M). Instead of the Gaussian modeling method, the original background frame is obtained by reusing the output results of the gas detection model without increasing the computing performance.

[0107] In an optional implementation, in order to effectively improve the accuracy of the real background frame, thereby further improving the extraction accuracy of the gas concentration map in the subsequent steps, the method provided in this embodiment may also include the following steps: obtaining the preset cache quantity of the initial background frame; determining the weight coefficient of each initial background frame according to the position index of each initial background frame and the cache quantity; wherein the position index is determined based on the acquisition time of the corresponding initial background frame; the step of determining the real background frame according to the initial background frame in the above step S203 includes: weighting the corresponding initial background frame according to the weight coefficient of each initial background frame, and summing each weighted initial background frame to obtain the real background frame.

[0108] Considering that gas leakage often lasts for a period of time, there will be a time difference between the background frame and the current frame. During this time difference, the background area in the foreground frame, except for the gas target area, may change from the corresponding area in the background frame. When extracting the gas concentration by subtracting the background frame, it may cause false detection. Therefore, this embodiment calculates the background frame weight by caching the number and position index, thereby improving the calculation accuracy of the real background frame.

[0109] Exemplarily, by determining the weight coefficient of each initial background frame, the weight coefficient can be a weight calculation method based on time sequence to calculate the weight of each background frame. For example, the preset cache quantity can be 10 frames (in some embodiments, it can also be other quantities, which is not particularly limited in this embodiment). After caching 10 frames of original background frames (i.e., initial background frames), the real background frames are generated based on the weight of each frame.

[0110] Background frame M is stored in the background frame buffer pool, with a maximum buffer of 10 frames. When the buffer exceeds 10 frames, the oldest frame will be discarded and a new background frame will be put in. According to the order in which the background frames are put into the background frame buffer pool, that is, the time sequence of background frame acquisition, the weight coefficient X of each background frame is calculated. Assuming that n is the position of the background frame in the background frame buffer pool, the weight coefficient calculation method is Xn=1 / 55*(11-n), where n is the position index, and the value range is [1,10]. If the total size of the background buffer is 10 frames, the latest frame is put into M1, and the oldest frame is put into M10. The sum of the position indexes is used as the weight sum, that is, 1+2+...+9+10 = 55. The weight of the latest frame of M1 calculated by this formula is the largest, which is 10 / 55, that is, the reliability is greater. This weight calculation method ensures that the weight of the latest background frame is greater when calculating the real background frame, making the calculation result of the real background frame more accurate.

[0111] Specifically, each time a new background frame is added, the real background frame is calculated according to the weight coefficient of each frame and recorded as BG_img. The calculation method is as follows:

[0112] BG_img(x,y)= M1(x,y) *X1+ M2(x,y)*X2+…+ M9(x,y)*X9+ M10(x,y)*X10

[0113] In the formula, BG_img (x, y) is the real background frame, M1 (x, y) is the first original background frame acquired according to time, X1 is the weight coefficient of the first original background frame, and so on. It can be understood that (x, y) are the horizontal and vertical coordinates of each pixel in the background frame.

[0114] It can be understood that when the number of air masses is greater than 0, it indicates that there is a gas target in the frame image, and the gas detection model can obtain the gas location map table (denoted as site_map) and the gas confidence table (denoted as con_map).

[0115] It should be noted that, in some other examples, the weight coefficient corresponding to the initial frame may also be adaptively adjusted and determined by the user according to actual applications, and this embodiment is not limited to the specific determination method of the above-mentioned weight coefficient.

[0116] Step S204: extracting a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame.

[0117] In this embodiment, after extracting the foreground frame and the background frame, the gas position mapping table, the gas confidence table and the real background frame in the foreground frame can be used to extract the gas morphology concentration map through the background frame subtraction algorithm. Specifically, the real background frame determined above can be subtracted from the current gas imaging image to obtain a more accurate regional position where the gas is detected, and the image minus the real background frame can be screened using the position mapping table to extract the potential gas area. At the same time, the original gas intensity is weighted in combination with the confidence value of each pixel in the gas confidence table. The intensity of the high confidence area is maintained or amplified, while the intensity of the low confidence area is suppressed. This weighting method makes it possible to pay more attention to those high confidence areas when generating the final gas morphology concentration map, thereby reducing the impact of misidentification or missed identification and greatly improving the detection accuracy.

[0118] In addition, compared to the background subtraction algorithm in the related art that uses the original image to subtract the background frame to extract the foreground frame, in order to solve the problem that the background area other than the gas target area in the foreground frame may change from the corresponding area in the background frame, resulting in false detection, this embodiment combines the gas position mapping table when performing background frame subtraction, and only extracts the gas form and concentration in the foreground target, thereby effectively improving the accuracy.

[0119] In an alternative embodiment, in combination Figure 3 and Figure 4 As shown, the above step S204 extracts the gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame, and may include the following steps S2041-S2043:

[0120] S2041. Subtract the real background frame from the gas imaging image to obtain a differential image, filter the differential image according to the gas position mapping table in the foreground frame, and extract an initial gas morphology concentration map from the gas imaging image.

[0121] Exemplarily, this embodiment extracts an initial gas morphology concentration map according to the gas position map table and the real background frame BG_img, which is recorded as GAS_img. The specific extraction method can be obtained according to the following formula:

[0122] GAS-img1(x,y)= site_map(x,y)*(R_frm(x,y)-BG_img(x,y)), recorded as the first gas form concentration map

[0123] Where GAS-img1(x,y) represents the initial gas morphology concentration map, site_map(x,y) is the gas position mapping table, R_frm(x,y)-BG_img(x,y) represents the differential image, R_frm(x,y) is the gas imaging image (original frame), and BG_img(x,y) is the real background frame.

[0124] S2042. Perform first gas confidence weighting on each pixel in the gas imaging image according to the gas confidence table, and process the confidence-weighted image according to the gas position mapping table to obtain a weighted gas morphology concentration map.

[0125] Exemplarily, the weighted gas morphology concentration map is calculated by combining the gas confidence table con_map output by the gas detection module. Specifically, the confidence factor in the confidence map (i.e., the confidence corresponding to each pixel) is weighted with the original thermal imaging image R_frm, and the formula is as follows:

[0126] GAS-img2(x,y)= site_map(x,y)*(R_frm(x,y)* con_map (x,y)), recorded as the second gas form concentration map. In the formula, GAS-img2(x,y) represents the weighted gas form concentration map, con_map (x,y) represents the gas confidence table, and R_frm(x,y)* con_map (x,y) represents the image after the gas imaging image is confidence-weighted using the gas confidence table. It can be understood that the confidence-weighted image is processed according to the gas position mapping table, that is, the process of site_map(x,y)*(R_frm(x,y)* con_map (x,y)).

[0127] S2043. Generate the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map.

[0128] Exemplarily, in order to further improve the accuracy of the gas morphology concentration map, the above-mentioned step S2043 generates the gas morphology concentration map according to the initial gas morphology concentration map and the weighted gas morphology concentration map, and can be used in the following manner: obtain the first pixel value of each pixel point in the initial gas morphology concentration map, and the second pixel value of each pixel point in the weighted gas morphology concentration map; for each pixel point, select the largest pixel value between the first pixel value and the second pixel value as the pixel value of the pixel point; and weight the pixel points selected by the pixel value according to the gas confidence table by the second gas confidence to obtain the gas morphology concentration map.

[0129] After the initial gas form concentration map and the weighted gas form concentration map are calculated respectively through the above steps S2041 and S2042, the final gas form concentration map is extracted by calculating the size of each gas pixel in the two concentration maps and combining the weight table again. The specific formula is as follows:

[0130] GAS-img3(x,y)= MAX(GAS-img1(x,y), GAS-img2(x,y))* con_map (x,y), recorded as the final gas form concentration map GAS_img.

[0131] Where GAS-img3(x,y) represents the final gas form concentration map, GAS-img1(x,y) is the initial gas form concentration map, MAX(GAS-img1(x,y), GAS-img2(x,y)) is the weighted gas form concentration map, which compares the sizes of pixels in the initial gas form concentration map with those in the weighted gas form concentration map. After comparing the sizes, con_map(x,y) is used to weight them again to extract the final gas form concentration map.

[0132] In this process, the reliability is higher through the re-weighting method. By combining the confidence factor, the high-confidence area will be given a higher weight, which can effectively filter out possible false alarms or noise. By selecting the larger pixel value between the initial gas morphology concentration map and the weighted gas morphology concentration map, it can be ensured that no significant changes in gas concentration are missed. In addition, through the re-weighting method, it can be more adaptable to gas detection in different scenarios and conditions, especially in complex backgrounds or low contrast conditions.

[0133] In some examples, in addition to generating a gas morphology concentration map in the above manner, a gas morphology concentration map can also be generated in other ways, for example, by adaptive fusion, such as adaptively adjusting the fusion strategy according to local features or statistical information of the image, such as adjusting the fusion weight of each pixel according to local contrast or edge information, and then generating the final gas morphology concentration map. This embodiment does not specifically limit this.

[0134] Through the above technical solution, in the process of extracting the gas form concentration map, by performing multi-level analysis and weighting on the image, the final gas form concentration map effectively reduces the false alarms and missed alarms of gas, has higher accuracy, and can better reflect the distribution and concentration of the gas in the image.

[0135] Continue to refer to Figure 2 Step S205: Process the gas morphology concentration map by pseudo-color processing and superimpose it on the visible light image to obtain a gas pseudo-color image.

[0136] Compared with the related art, which uses only infrared bands for gas detection, it is difficult to observe the gas leakage position with human eyes, resulting in poor user experience. After extracting the gas morphology concentration map, this embodiment superimposes it on the visible light image after pseudo-color processing, so that users can clearly see the gas leakage position in the gas pseudo-color image.

[0137] Through the above technical scheme, the gas detection model is used to identify the detection results corresponding to the air mass, gas position mapping table and gas confidence table in the gas imaging image, and the foreground frame and the real background frame calculated by the weight coefficient are extracted based on the detection results. The gas position mapping table, the gas confidence table and the real background frame can be used to accurately extract the gas morphology and concentration in the non-background frame of the gas imaging image, and the extracted gas morphology and concentration are superimposed on the visible light image in a pseudo-color manner, so that the user can clearly observe the gas distribution position and concentration.

[0138] Figure 5 The present application provides another flow chart of an image processing method based on gas imaging. Based on the above embodiment, this embodiment takes into account the computing power and other hardware limitations of the device. Most devices do not have the ability to detect continuous frames in real time, which may result in a low display frame rate. To further optimize this problem, this embodiment extracts the gas concentration map of the undetected frame to achieve the effect of frame supplementation, thereby further facilitating the device user to observe the gas distribution. Specifically, in addition to the above steps S201-S205, the method provided in this embodiment may also include the following steps S501-S503.

[0139] Step S501: for each frame of the gas imaging image, obtaining an undetected frame in the gas imaging image that has not been detected.

[0140] The above method is limited by hardware performance limitations and adopts a weighted frame supplementation method. The specific method is to use the site_map output last time to extract the first gas form concentration map for the frame without gas model detection. Then, the first gas form density map and the gas confidence table con_map output last time are used to obtain the final gas form concentration map of the frame without gas model detection.

[0141] In practical applications, considering factors such as hardware performance limitations, the gas detection model can only complete detection of 6.25 frames of images per second, which will result in the extracted gas morphology concentration map being refreshed at a frequency of 6.25 frames. In other words, the device is usually unable to achieve real-time detection and processing of each frame of the image, but instead detects and generates the final gas morphology concentration map in a frame extraction manner. Although it solves the problem of hardware performance limitations, its display frame rate is low, which may cause gas morphology jamming and poor viewing experience. Based on this, this embodiment obtains undetected frames and performs supplementary calculations on the gas morphology concentration map for the undetected frames to achieve frame supplementation.

[0142] Among them, the undetected frames that have not been detected are frames that have not been input into the model for detection. For example, the device extracts frames every 4 frames for detection, R-FRM0, R-FRM1, R-FRM2, R-FRM3, the model performs gas detection on R-FRM3, and R-FRM0, R-FRM1, and R-FRM2 are undetected frames.

[0143] Step S502: for the undetected frame, obtaining a gas position mapping table of a previous frame of the undetected frame, and determining an initial gas morphology concentration map of the undetected frame according to the gas position mapping table of the previous frame and the real background frame; and

[0144] Step S503: Determine the gas form concentration map of the undetected frame according to the initial gas form concentration map of the undetected frame and the gas confidence table of the previous frame.

[0145] For example, Figure 6 As shown, for frames without gas model detection (undetected frames), since the interval between two frames is usually small, this embodiment uses the site_map output by the previous frame to extract the initial gas morphology concentration map, and can use the initial gas morphology density map and the gas confidence table con_map output by the previous frame to obtain the final gas morphology concentration map of the frame without gas model detection, and then use the size of the pixel point in the final gas morphology concentration map and the con_map output by the last time to perform weighting, and obtain a new con_map2 recorded as the second gas confidence table. The specific formula is as follows:

[0146] GAS-img1(x,y)= site_map(x,y)*(R_frm(x,y)-BG_img(x,y)), which is the initial gas form concentration map of the frame without gas model detection.

[0147] GAS-img2(x,y)= site_map(x,y)*(R_frm(x,y)* con_map (x,y)), which is the final gas speciation concentration map of the frame without gas model detection.

[0148] And the confidence map of the next frame without gas model detection can be updated: con_map2(x,y) = GAS-img2 (x,y) *0.5 + con_map(x,y)*0.5, that is, the next frame is the second gas confidence table of the detected frame.

[0149] Through the above technical solution, supplementary calculation of the gas morphology concentration map is performed for the undetected frames, and the refresh frame rate of the gas morphology concentration map can be increased to the same refresh frame rate as the visible light camera, that is, 25fps, thereby achieving the effect of frame supplementation and greatly improving the user's gas observation experience.

[0150] Furthermore, in response to the problem of the lack of a gas confidence table in the undetected image, the present embodiment may also include the following steps: if the previous frame of the undetected frame is an image frame detected by the gas detection model, then the gas confidence table of the previous frame is determined as the gas confidence table of the undetected frame; if the previous frame of the undetected frame is an image frame that has not been detected by the gas detection model, then the gas confidence table of the undetected frame is determined based on the gas concentration morphology diagram of the previous frame and the gas confidence table determined for the previous frame.

[0151] Exemplarily, the updating steps of the gas confidence table within a complete detection cycle may be as follows: Gas imaging image R_frm0 (first frame): sent to the gas detection model for detection, and the current frame confidence table con_map (R_frm0) is output;

[0152] Gas imaging image R_frm1 (second frame): No deep learning detection model detection is performed, and the current frame confidence table reuses the previous frame confidence: con_map(R_frm1) = con_map(R_frm0);

[0153] Gas imaging image R_frm2 (third frame): No deep learning detection model detection is performed. The confidence table of the current frame is obtained by weighting the final gas morphology concentration map of the previous frame and the confidence table of the previous frame: con_map(R_frm2)= GAS-img2(R_frm1) *0.5 + con_map(R_frm1)*0.5

[0154] Gas imaging image R_frm3 (the fourth frame): No deep learning detection model detection is performed. The confidence table of the current frame is obtained by weighting the final gas morphology concentration map of the previous frame and the confidence table of the previous frame: con_map(R_frm3)= GAS-img2(R_frm2) *0.5 + con_map(R_frm2)*0.5

[0155] Gas imaging image R_frm4 (fifth frame): sent to the deep learning detection model for detection, and the current frame confidence table con_map (R_frm4) is output

[0156] …

[0157] By analogy, through the above technical solution, even if it is impossible to detect and process each frame due to factors such as equipment hardware performance limitations, the gas confidence table of each frame can be obtained in real time through the above calculation method, and then combined with the gas confidence table, a more accurate gas morphology concentration map can be extracted.

[0158] Figure 7 This is a flow chart of another flow chart of an image processing method based on gas imaging provided by an embodiment of the present application. On the basis of the above embodiment, this embodiment performs pseudo-color processing on the gas morphology concentration map by pixel color mapping to facilitate user observation. Specifically, in addition to the above steps S201-S205, the method provided by this embodiment may also include the following step S701, and step S205 is further divided into steps S2051 and S2052.

[0159] Step S701: Obtain a predetermined density and color mapping table for pixels, wherein the density and color mapping table includes mapping relationships between different densities and colors of pixels.

[0160] Exemplarily, the concentration and color mapping table can be determined in the following manner, such as first determining the concentration range of the pixel points, and selecting a color model (such as a red, green, and blue RGB model, and in some embodiments, other models such as a hue, saturation, and value HSV model) to represent different concentration values, assigning a specific color to each concentration value according to the selected color model, and forming a mapping table between concentration and color by pairing the concentration value with its corresponding color value.

[0161] Step S2051: for each pixel point in the gas form concentration map, query the color value in the concentration and color mapping table according to the concentration value of the pixel point.

[0162] Specifically, by obtaining pixel data of the gas morphology concentration map, these data can be stored in the form of a two-dimensional array, where each element represents the concentration value of a pixel point. By looking up a table, the color value of each pixel point, such as the RGB color value, can be quickly determined.

[0163] Step S2052: performing pseudo-color processing on the gas form concentration map according to the color value of each pixel, and superimposing the pseudo-color processed gas form concentration map on the visible light image to obtain a gas pseudo-color image.

[0164] By mapping the pixel depth in the final gas morphology map to pseudo color, and then drawing the pseudo color into the visible light image V_frm according to the image registration mapping relationship obtained above, the leaked gas position, morphology and concentration information can be intuitively observed in real time in the visible light image.

[0165] For example, for 0-255, i.e. 256 concentration values, a color is pre-specified according to the concentration value range, and each color is composed of different RGB values, which are recorded as color_map. By looking up a table, a color in color_map is selected for each gas pixel. The formula is as follows:

[0166] R(x,y) = color_map_R(GAS_img(x,y));

[0167] G(x,y) = color_map_G(GAS_img(x,y));

[0168] B(x,y) = color_map_B(GAS_img(x,y));

[0169] Where GAS_color_img is the image converted to pseudo color, R(x, y), G(x, y), B(x, y) respectively represent the red, green and blue channel values ​​of the pixel at coordinate (x, y) in the pseudo color processed image GAS_color_img, color_map_R, color_map_G, color_map_B are mapping tables used to map gas concentration values ​​to corresponding RGB color values, where color_map_R, color_map_G and color_map_B correspond to the mapping of red, green and blue channels respectively.

[0170] The formula for mapping gas pseudo-color to visible light image is as follows:

[0171] V_color_frm(x,y)=GAS_color_img(coor(x,y).x, coor(x,y).y)*0.7+ V_color_frm(x,y)*0.3.

[0172] Wherein, V_color_frm(x, y) is the gas pseudo-color image; coor(x, y) is the coordinate mapping relationship obtained by binocular positioning, which is used to convert the coordinates (x, y) in the gas image into the corresponding coordinates in the visible light image, so as to realize image registration in different imaging devices, so as to ensure the spatial alignment of the two images; 0.7 and 0.3 are coefficients, respectively, which are used to control the superposition ratio of the pseudo-color image and the visible light image, which can be adaptively determined by those skilled in the art.

[0173] To facilitate understanding of the embodiments of the present application, the present application also provides an exemplary embodiment, such as Figure 8 As shown, the process includes the following: the device collects visible light images (each frame of visible light image is represented by V-FRM0~ V-FRMn) and gas (thermal) imaging images (each frame of gas imaging image is represented by R-FRM0~ R-FRMn), based on the hardware performance of the device, it can be set to extract frames every four frames and send them to the gas detection model (6.25fps) for gas detection. According to the detection results, it is determined whether the number of air masses is greater than 0. If it is equal to 0, it means that no gas is detected and it is identified as the initial background frame. However, due to factors such as gas diffusion, in order to extract a background frame with higher accuracy, the background frame cache mechanism and the position index calculation mechanism are used to determine the real background frame BFrm (for relevant instructions, please refer to the above embodiment); if it is greater than 0, it means that gas is detected, and the corresponding gas position MAP and gas confidence MAP are obtained. For each frame of gas imaging image, the gas form concentration is extracted through background modeling according to the real background frame BFrm and the gas position MAP to obtain an initial gas form concentration map, and the confidence is weighted in combination with the gas confidence MAP to obtain a weighted gas form concentration map, so as to obtain the final gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map; and then the gas form concentration map (after frame supplementation) is pseudo-colored and fused into the visible light image to obtain a gas pseudo-color visible frame (25fps), that is, a gas pseudo-color image.

[0174] Through the above technical solution, the gas morphology and concentration in the black background area of ​​thermal imaging are accurately extracted through deep learning model detection processing and traditional background modeling algorithm. When the equipment is actually used, there is no need to worry about whether the spectral absorption characteristics of the monitored area are the same as those of the gas, which effectively improves the adaptability and accuracy of the product; and the calculation of gas confidence is combined in the process of gas morphology concentration map, which effectively improves the accuracy and success rate of gas extraction in different scenarios; a frame supplement algorithm is proposed for undetected frames, which can increase the gas morphology refresh frame rate from 6.25fps to 25 frames, thereby increasing the gas morphology refresh frame rate to the acquisition frame rate of 25fps without increasing hardware costs, which is convenient for equipment users to observe; in addition, through infrared cameras and visible light cameras, the flammable gas morphology that is invisible to the naked eye is integrated into the visible light in real time through pseudo-color, so that users can observe the gas distribution that was originally invisible in the visible light imaging device in the visible light image, which improves the user experience.

[0175] Fig. 9 is a schematic diagram of the structure of an image processing device based on gas imaging provided in an embodiment of the present application, such as Fig. 9 As shown, the device 900 includes an image acquisition module 901, a detection module 902, a type frame acquisition module 903, a concentration map extraction module 904 and an overlay module 905, wherein:

[0176] An image acquisition module 901 is configured to acquire a gas imaging image and a visible light image of a target scene;

[0177] A detection module 902 is configured to detect the gas mass, the gas position mapping table and the gas confidence table in each frame of the gas imaging image according to a preset gas detection model to obtain a gas detection result for each frame; wherein the gas position mapping table is used to characterize the position of each gas pixel point, and the gas confidence table is used to characterize the gas confidence of each pixel point;

[0178] The type frame acquisition module 903 is configured to acquire a foreground frame with an air mass and an initial background frame without an air mass for each frame of gas detection result, and determine a real background frame based on the initial background frame and a corresponding weight coefficient;

[0179] a concentration map extraction module 904, which is configured to extract a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame;

[0180] The superposition module 905 is configured to superimpose the gas form concentration map on the visible light image after pseudo-color processing to obtain a gas pseudo-color image.

[0181] In one embodiment, the device further comprises:

[0182] A buffer quantity acquisition module, which is configured to acquire the preset buffer quantity of the initial background frame;

[0183] A weight coefficient determination module, which is configured to determine the weight coefficient of each initial background frame according to the position index of each initial background frame and the cache quantity; wherein the position index is determined based on the acquisition time of the corresponding initial background frame;

[0184] The type frame acquisition module 903 includes:

[0185] The background frame determination unit is configured to weight the corresponding initial background frame according to the weight coefficient of each initial background frame, and sum each weighted initial background frame to obtain the real background frame.

[0186] In one embodiment, the concentration map extraction module 904 includes:

[0187] an initial concentration map extraction unit, configured to subtract the real background frame from the gas imaging image to obtain a differential image, and filter the differential image according to a gas position mapping table in the foreground frame to extract an initial gas morphology concentration map from the gas imaging image;

[0188] a weighted concentration map extraction unit, configured to perform first gas confidence weighting on each pixel in the gas imaging image according to the gas confidence table, and process the confidence-weighted image according to the gas position mapping table to obtain a weighted gas morphology concentration map;

[0189] A concentration map generating unit is configured to generate the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map.

[0190] In one embodiment, the concentration map generating unit is specifically configured to: obtain a first pixel value for each pixel point in the initial gas morphology concentration map, and a second pixel value for each pixel point in the weighted gas morphology concentration map; for each pixel point, select the largest pixel value between the first pixel value and the second pixel value as the pixel value of the pixel point; and perform second gas confidence weighting on the pixel points selected by pixel value according to the gas confidence table to obtain the gas morphology concentration map.

[0191] In one embodiment, the device further comprises:

[0192] An undetected frame extraction module, which is configured to obtain, for each frame of the gas imaging image, an undetected frame that has not been detected in the gas imaging image;

[0193] An undetected frame complementation module is configured to obtain, for the undetected frame, a gas position mapping table of a previous frame of the undetected frame, and determine an initial gas morphology concentration map of the undetected frame based on the gas position mapping table of the previous frame and the real background frame; and determine a gas morphology concentration map of the undetected frame based on the initial gas morphology concentration map of the undetected frame and a gas confidence table of the previous frame.

[0194] In one embodiment, the device further comprises:

[0195] A first determination module, configured to determine the gas confidence table of the previous frame as the gas confidence table of the undetected frame if the previous frame of the undetected frame is an image frame detected by the gas detection model;

[0196] The second determination module is configured to determine the gas confidence table of the undetected frame based on the gas concentration morphology diagram of the previous frame and the gas confidence table determined for the previous frame if the previous frame of the undetected frame is an image frame that has not been detected by the gas detection model.

[0197] In one embodiment, the device further comprises:

[0198] A mapping table acquisition module, which is configured to acquire a predetermined density and color mapping table for a pixel point, wherein the density and color mapping table includes a mapping relationship between different densities and colors of the pixel point;

[0199] The superposition module 905 includes:

[0200] A query unit, configured to query the color value in the concentration and color mapping table according to the concentration value of each pixel in the gas form concentration map;

[0201] The pseudo-color processing unit is configured to perform pseudo-color processing on the gas form concentration map according to the color value of each pixel.

[0202] In one implementation, the gas position mapping table is used to characterize the position of each gas pixel, and the gas confidence table is used to characterize the gas confidence of each pixel.

[0203] The devices provided in the above-mentioned embodiments are used to execute the image processing method based on gas imaging provided in the above-mentioned method embodiment. The implementation principles and technical effects thereof are similar and will not be described in detail here.

[0204] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig.10As shown, the electronic device includes: a processor 1002, and a memory 1001 and a display 1003 which are communicatively connected to the processor 1002;

[0205] The memory 1001 stores computer-executable instructions;

[0206] The processor 1002 executes the computer-executable instructions stored in the memory 1001 to implement the image processing method based on gas imaging in any method embodiment; the display 1003 can be used to display the above-mentioned gas pseudo-color image.

[0207] The electronic devices provided in the above-mentioned embodiments are used to execute the image processing method based on gas imaging provided in the above-mentioned method embodiment. The implementation principles and technical effects thereof are similar and will not be described in detail here.

[0208] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the image processing method based on gas imaging provided in the above method embodiment.

[0209] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0210] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0211] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution provided by any of the above method embodiments can be implemented.

[0212] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0213] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0214] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0215] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method based on gas imaging, characterized in that: include: Collect gas imaging images and visible light images of the target scene; For each frame of the gas imaging image, the gas mass, the gas position mapping table and the gas confidence table in the gas imaging image are detected according to a preset gas detection model to obtain a gas detection result; According to the gas detection result, a foreground frame with an air mass and an initial background frame without an air mass are obtained, and a real background frame is determined based on the initial background frame and a corresponding weight coefficient; Subtracting the real background frame from the gas imaging image to obtain a differential image, and filtering the differential image according to a gas position mapping table in the foreground frame to extract an initial gas morphology concentration map from the gas imaging image; According to the gas confidence table, each pixel in the gas imaging image is weighted by a first gas confidence, and the confidence-weighted image is processed according to the gas position mapping table to obtain a weighted gas form concentration map; generating the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map; The gas form concentration map is processed by pseudo-color and then superimposed on the visible light image to obtain a gas pseudo-color image.

2. The method according to claim 1, characterized in that: Also includes: Obtaining the preset buffer quantity of the initial background frame; Determine the weight coefficient of each initial background frame according to the position index of each initial background frame and the cache quantity; wherein the position index is determined based on the acquisition time of the corresponding initial background frame; The determining of the real background frame based on the initial background frame and the corresponding weight coefficient comprises: The corresponding initial background frames are weighted according to the weight coefficient of each initial background frame, and each weighted initial background frame is summed to obtain the real background frame.

3. The method according to claim 1, characterized in that The step of generating the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map comprises: Obtaining a first pixel value of each pixel point in the initial gas form concentration map and a second pixel value of each pixel point in the weighted gas form concentration map; For each pixel point, selecting the largest pixel value between the first pixel value and the second pixel value as the pixel value of the pixel point; The second gas confidence weighting is performed on the pixel points selected by pixel value according to the gas confidence table to obtain the gas form concentration map.

4. The method according to claim 1, characterized in that Also includes: For each frame of the gas imaging image, obtaining an undetected frame in the gas imaging image that has not been detected; For the undetected frame, obtaining a gas position mapping table of a previous frame of the undetected frame, and determining an initial gas morphology concentration map of the undetected frame according to the gas position mapping table of the previous frame and the real background frame; as well as, The gas form concentration map of the undetected frame is determined according to the initial gas form concentration map of the undetected frame and the gas confidence table of the previous frame.

5. The method according to claim 4, characterized in that Also includes: If the previous frame of the undetected frame is an image frame detected by the gas detection model, the gas confidence table of the previous frame is determined as the gas confidence table of the undetected frame; If the previous frame of the undetected frame is an image frame that has not been detected by the gas detection model, the gas confidence table of the undetected frame is determined according to the gas concentration morphology diagram of the previous frame and the determined gas confidence table of the previous frame.

6. The method according to claim 1, characterized in that Also includes: Obtaining a predetermined density and color mapping table for pixel points, wherein the density and color mapping table includes mapping relationships between different densities and colors of pixel points; The step of subjecting the gas form concentration map to pseudo-color processing comprises: For each pixel point in the gas form concentration map, query the color value in the concentration and color mapping table according to the concentration value of the pixel point; According to the color value of each pixel, pseudo-color processing is performed on the gas form concentration map.

7. The method according to any one of claims 1 or 2 or 3-6, characterized in that: The gas position mapping table is used to characterize the position of each gas pixel point, and the gas confidence table is used to characterize the gas confidence of each pixel point.

8. An image processing device based on gas imaging, characterized in that: include: An image acquisition module configured to acquire a gas imaging image and a visible light image of a target scene; A detection module, which is configured to detect the gas mass, gas position mapping table and gas confidence table in each frame of the gas imaging image according to a preset gas detection model to obtain a gas detection result for each frame; wherein the gas position mapping table is used to characterize the position of each gas pixel point, and the gas confidence table is used to characterize the gas confidence of each pixel point; A type frame acquisition module, which is configured to acquire a foreground frame with an air mass and an initial background frame without an air mass for each frame of gas detection results, and determine a real background frame based on the initial background frame and a corresponding weight coefficient; a concentration map extraction module, configured to extract a gas morphology concentration map from the gas imaging image according to the gas position mapping table in the foreground frame, the gas confidence table and the real background frame; A superposition module, which is configured to superimpose the gas form concentration map on the visible light image after pseudo-color processing to obtain a gas pseudo-color image; The concentration map extraction module comprises: an initial concentration map extraction unit, configured to subtract the real background frame from the gas imaging image to obtain a differential image, and filter the differential image according to a gas position mapping table in the foreground frame to extract an initial gas morphology concentration map from the gas imaging image; a weighted concentration map extraction unit, configured to perform first gas confidence weighting on each pixel in the gas imaging image according to the gas confidence table, and process the confidence-weighted image according to the gas position mapping table to obtain a weighted gas morphology concentration map; A concentration map generating unit is configured to generate the gas form concentration map according to the initial gas form concentration map and the weighted gas form concentration map.

9. An electronic device, characterized in that: include: A processor, a memory and a display connected in communication with the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the image processing method based on gas imaging according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the image processing method based on gas imaging according to any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the image processing method based on gas imaging according to any one of claims 1 to 7.

Images