Gas leakage detection method and device based on infrared image processing and terminal

Abstract

The invention discloses a gas leakage detection method and device based on infrared image processing and a terminal, relates to the technical field of image processing, and mainly aims to solve the problem of low detection precision of existing gas leakage. Comprising the steps that when it is determined that a suspicious gas object exists based on foreground image pixels of a to-be-detected target area, a foreground difference matrix is constructed based on foreground image data, background image data and a self-adaptive threshold value, and the self-adaptive threshold value is determined based on a maximum value in a fitting curve of a difference frequency division mapping relation; constructing an average gray level signal matrix of the foreground difference matrix, and performing filtering processing on the average gray level signal matrix to obtain image data to be detected; and when the background image data passes the delay verification, performing weight fusion on the to-be-detected image data and the visible light image data to generate gas leakage detection result image data.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to a gas leakage detection method, device, and terminal based on infrared image processing. Background Art

[0002] Gas leakage may lead to accidents such as environmental pollution, explosion accidents, and poisoning of personnel, and even trigger fires, posing a serious threat to human health, the ecological environment, and industrial production safety. At this time, gas detection imaging technology came into being. Among them, passive infrared detection imaging is more suitable for complex environments because of its characteristics such as long detection distance and wide detection coverage.

[0003] Currently, the existing gas leakage detection usually uses passive infrared detection imaging technology for detection. However, since the gas signal in infrared gas imaging is usually weak and the image contrast is low, the distinguishability between the gas and the background is not high, making it difficult to clearly identify the presence and spatial distribution of the gas. Moreover, the weak gas signal is prone to losing detail information during the imaging process. Especially in a complex background environment, the texture and shape around the gas leakage point may be blurred or covered, affecting the accurate judgment of the leakage position and degree, resulting in low detection accuracy of gas leakage. Summary of the Invention

[0004] In view of this, this application provides a gas leakage detection method, device, and terminal based on infrared image processing, mainly aiming to solve the problem of low detection accuracy of existing gas leakage.

[0005] According to one aspect of this application, a gas leakage detection method based on infrared image processing is provided, including: When it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, background image data, and an adaptive threshold, where the adaptive threshold is determined by the maximum value in the fitting curve of the difference frequency mapping relationship; Construct an average gray signal matrix of the foreground difference matrix, and perform filtering processing on the average gray signal matrix to obtain the image data to be detected; When the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data.

[0006] Further, before constructing the foreground difference matrix based on the foreground image data, background image data, and an adaptive threshold, the method further includes: Determine the foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on the foreground field radius and the background update factor. The foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data; When the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of the region of the foreground image pixels is greater than a second preset threshold, it is determined that there is a suspicious gas object.

[0007] Further, the constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold includes: Obtain the adaptive threshold; Determine the pixel differences between multiple frames of the foreground image data and the background image data, and compare the pixel differences with the adaptive threshold; Configure the pixel differences less than or equal to the adaptive threshold to zero values, and construct the foreground difference matrix based on the pixel differences greater than the adaptive threshold and the zero values.

[0008] Further, the obtaining the adaptive threshold includes: Obtain an initial threshold, and construct an initial foreground difference matrix based on the initial threshold, the foreground image data, and the background image data; Perform Gaussian smoothing processing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; Perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches the preset condition in the fitting curve as the adaptive threshold.

[0009] Further, the constructing an average gray signal matrix of the foreground difference matrix and performing filtering processing on the average gray signal matrix to obtain the image data to be detected includes: Determine the average gray value of multiple frames of the foreground image data based on the differences in the foreground difference matrix, and generate the average gray signal matrix based on the average gray value; Statistically analyze the two-dimensional frequency in the average gray signal matrix to generate a discrete distribution mapping relationship, where the discrete distribution mapping relationship includes the relationship between the gray value and the frequency; Perform Gaussian filtering processing on the discrete distribution mapping relationship to obtain the image data to be detected composed of gray values higher than the preset high-frequency signal range.

[0010] Further, before the method of performing weight fusion on the image data to be detected and the visible light image data to generate fused image data when the background image data passes the delay check, the method further includes: When the target pixel in the background image data is within the gas region and the gray value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data has completed the delay verification; When the target pixel in the background image data is not within the gas region and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated; Among them, the gas region is determined based on the foreground difference matrix.

[0011] Further, before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Obtaining the gain correction coefficient and the bias correction coefficient of the infrared image, and constructing a correction mapping relationship based on the gain correction coefficient, the bias correction coefficient, and the mapping relationship between gray scale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0012] According to another aspect of the present application, a gas leakage detection device based on infrared image processing is provided, including: A construction module, configured to construct a foreground difference matrix based on the foreground image data, the background image data, and an adaptive threshold when it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, and the adaptive threshold is determined by the maximum value in the fitting curve of the difference frequency mapping relationship; A processing module, configured to construct an average gray signal matrix of the foreground difference matrix and perform filtering processing on the average gray signal matrix to obtain the image data to be detected; A generation module, configured to perform weight fusion on the image data to be detected and the visible light image data when the background image data passes the delay verification, and generate gas leakage detection result image data.

[0013] Further, the device further includes: A determination module, configured to determine the foreground image pixels and the background image pixels obtained by infrared image acquisition of the target area based on the foreground field radius and the background update factor, where the foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data; when the number of pixels of the foreground image pixels is greater than the first preset threshold and the area of the foreground image pixels is greater than the second preset threshold, it is determined that there is a suspicious gas object.

[0014] Further, the building module is specifically configured to obtain the adaptive threshold; determine the pixel difference between multiple frames of the foreground image data and the background image data, and compare the pixel difference with the adaptive threshold; configure the pixel difference less than or equal to the adaptive threshold to zero, and build the foreground difference matrix based on the pixel difference greater than the adaptive threshold and the zero value.

[0015] Further, the building module is specifically configured to obtain an initial threshold, and build an initial foreground difference matrix based on the initial threshold, the foreground image data, and the background image data; perform Gaussian smoothing processing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches the preset condition in the fitting curve as the adaptive threshold.

[0016] Further, the processing module is specifically configured to determine the average gray value of multiple frames of the foreground image data based on the difference value in the foreground difference matrix, and generate the average gray signal matrix based on the average gray value; count the two-dimensional frequency in the average gray signal matrix to generate a discrete distribution mapping relationship, where the discrete distribution mapping relationship includes the relationship between the gray value and the frequency; perform Gaussian filtering processing on the discrete distribution mapping relationship to obtain the image data to be detected composed of gray values higher than the preset high-frequency signal range.

[0017] Further, the device further includes: An update module, configured to determine that the background image data has completed delay verification when the target pixel in the background image data is in the gas region and the gray value of the target pixel is greater than the preset high-frequency signal range; when the target pixel in the background image data is not in the gas region, and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, update the background image data; where the gas region is determined based on the foreground difference matrix.

[0018] Further, the device further includes: A correction module, configured to obtain the gain correction coefficient and the offset correction coefficient of the infrared image, and build a correction mapping relationship based on the gain correction coefficient, the offset correction coefficient, and the mapping relationship between the gray value and the temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0019] According to another aspect of the present application, there is provided a storage medium, in which at least one executable instruction is stored, and the executable instruction causes the processor to perform the operations corresponding to the above-mentioned gas leakage detection method based on infrared image processing.

[0020] According to another aspect of the present application, a terminal is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned gas leakage detection method based on infrared image processing.

[0021] By means of the above technical solution, the technical solution provided by the embodiments of the present application has at least the following advantages: The present application provides a gas leakage detection method, device and terminal based on infrared image processing. Compared with the prior art, in the embodiments of the present application, when it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold, and the adaptive threshold is determined based on the maximum value in the fitting curve of the differential frequency mapping relationship; an average gray signal matrix of the foreground difference matrix is constructed, and the average gray signal matrix is filtered to obtain the image data to be detected; when the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data, which increases the distinction accuracy between the foreground and the background, highlights the detection object of foreground gas leakage, improves the recognition ability of gas signals, and thus meets the requirement of accurate gas leakage detection in a complex background environment.

[0022] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings: Figure 1 A flowchart of a gas leakage detection method based on infrared image processing provided by an embodiment of the present application is shown; Figure 2 A schematic diagram of infrared detection of ethylene gas provided by an embodiment of the present application is shown; Figure 3 A schematic diagram of the overall process of infrared gas detection provided by an embodiment of the present application is shown; Figure 4 The block diagram of a gas leakage detection device based on infrared image processing provided by an embodiment of the present application is shown; Figure 5 The structural schematic diagram of a terminal provided by an embodiment of the present application is shown. Detailed implementation manners

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0025] An embodiment of the present application provides a gas leakage detection method based on infrared image processing, as Figure 1 shown, the method includes: 101. When it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold.

[0026] In an embodiment of the present application, as the execution entity for gas leakage detection, the current execution end can be a terminal device or a server device, so as to collect infrared images of the target area to be detected and perform leakage detection. Among them, the target area to be detected is an area where gas leakage may occur. At this time, the gas can include, but is not limited to, combustible gas, toxic gas, etc., such as methane and ethylene, which are not specifically limited in the embodiments of the present application. Among them, the foreground image pixels are the pixels in the foreground image data, and the foreground image data and the background image data are combined into the collected infrared image data. The foreground image data represents the area where gas leakage may exist relative to the background image data. Therefore, after determining the foreground image data, the statistical contour or area of the foreground image pixels is performed to determine whether there is a suspicious gas object, so as to initiate the steps of gas detection in this embodiment. When it is determined that there is a suspicious gas object, the current execution end constructs a difference matrix based on the foreground image data, the background image data, and an adaptive threshold. At this time, the difference matrix is used to represent the difference matrix of frequency domain statistics, so as to convert the determination of the threshold for distinguishing noise and gas into a numerical solution, avoiding the limitations of the traditional fixed threshold, and is particularly suitable for gas leakage detection scenarios with low contrast and dynamic changes.

[0027] It should be noted that the adaptive threshold is determined by the maximum value in the fitting curve based on the differential frequency mapping relationship. At this time, the differential frequency mapping relationship includes the relationship between different differential values and the occurrence frequencies. A differential matrix for updating the adaptive threshold can be constructed based on the first several frame image data in the real-time foreground image data, and the differential frequency mapping relationship can be statistically obtained for fitting, and the maximum value in the fitting curve can be determined, so as to determine the adaptive threshold based on this maximum value and use it to construct the foreground differential matrix.

[0028] In another embodiment of the present application, for further limitation and explanation, the step of constructing the foreground differential matrix based on the foreground image data, the background image data, and the adaptive threshold includes: Obtain the adaptive threshold; Determine the pixel differences between multiple frames of the foreground image data and the background image data, and compare the pixel differences with the adaptive threshold; Configure the pixel differences less than or equal to the adaptive threshold to zero values, and construct the foreground differential matrix based on the pixel differences greater than the adaptive threshold and the zero values.

[0029] In order to analyze the gas gray distribution characteristics by means of a dynamic threshold, combining frequency domain filtering and a delayed update mechanism, not only retains the spatial concentration information of the gas plume, but also effectively suppresses the problem of background misjudgment, so as to improve the accuracy and stability of gas imaging. Currently, the execution end first obtains the adaptive threshold. Among them, multiple frames of foreground image data may include, but are not limited to, the pixels of the foreground image data of the first three frames at the current moment. By calculating the difference between the pixels of each frame and the pixels in the background image data one by one, and then comparing each pixel difference with the adaptive threshold. At this time, retain the pixel differences greater than the adaptive threshold, configure the pixel differences less than or equal to the adaptive threshold to zero, and finally combine the qualified pixel differences and zeros into the foreground differential matrix.

[0030] In another embodiment of the present application, for further limitation and explanation, the step of obtaining the adaptive threshold includes: Obtain an initial threshold, and construct an initial foreground differential matrix based on the initial threshold, the foreground image data, and the background image data; Perform Gaussian smoothing processing on the initial foreground differential matrix to obtain a difference frequency mapping relationship; Perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the differential value corresponding to the maximum value that matches the preset condition in the fitting curve as the adaptive threshold.

[0031] To avoid the limitations of gas recognition brought by traditional fixed thresholds, the current execution end first configures an initial threshold, preferably 3. That is, an initial foreground difference matrix is constructed based on the initial threshold, foreground image data, and background image data. Specifically, the current execution end continuously obtains three frames of foreground image data from the infrared video stream captured in real time, denoted as M_a, M_b, M_c, and calls the current background image data, denoted as M. At this time, these three frames of foreground image data and background image data are used to capture the dynamically changing gas signal. When constructing the foreground difference matrix, specifically, first, the pixels in the three frames of foreground image data are averaged to obtain the average foreground image pixel M_avg. Then, the average foreground image display is compared with the background model pixel by pixel to calculate the pixel difference at each pixel point. At this time, if the pixel difference exceeds an initial threshold, such as the initial threshold is 3, the pixel difference is retained; otherwise, it is configured to zero. Finally, a difference matrix D(x, y) is generated to represent the significant change area between the current scene and the background.

[0032] In addition, after obtaining the foreground difference matrix, Gaussian smoothing processing is performed. The Gaussian-smoothed difference matrix is traversed, and the occurrence frequencies of all non-zero difference values are counted to construct a difference value-frequency mapping relationship. For example, if there are 100 pixels with a difference value of 5, it is recorded as P(5)=100, so as to map the discrete difference values to their corresponding frequency distributions and form a "difference value-frequency" relationship table, which is the difference value-frequency mapping relationship. Among them, the "difference value" obtained by statistics in the difference value-frequency mapping relationship is used as the independent variable (x), and the corresponding "frequency" is used as the dependent variable (y). These data points can be curve-fitted by a fourth-order polynomial. At this time, the fourth-order polynomial can flexibly describe the rising, falling, and inflection point characteristics of the frequency distribution, while avoiding overfitting that may be caused by high-order polynomials. At the same time, the coefficients of the fourth-order polynomial are solved by the least squares method, and finally a fitting curve is obtained. Furthermore, the first derivative of the fitted polynomial curve is calculated to find the points where the derivative is zero, which are the extreme points of the fitting curve. At this time, within a preset difference value range, such as 3 to 30, it is verified whether these extreme points are maximum points. For example, it is confirmed through the second derivative. The first maximum point that appears is selected as the candidate value of the optimal threshold, that is, the optimal adaptive threshold R. If no maximum point is found within the preset difference value range, the previous adaptive or initial threshold can be used, and the embodiments of the present application do not make specific limitations.

[0033] In a specific implementation scenario, when constructing an initial foreground difference matrix with an initial threshold, in order to capture as many gas targets as possible, a relatively small transient initial threshold R' is preset, such as R' = 3, and the average difference matrix D(x, y) between the background mask and the foreground image data of the nearest three frames is calculated. At this time, the signal points where the difference between the foreground image data and the background image data is greater than or equal to the initial soil quality R' will be extracted. In addition, when processing based on Gaussian filtering, Gaussian filtering is used to reduce noise and smooth the image. After performing two-dimensional frequency mapping on the difference matrix, Gaussian filtering is applied to obtain the discrete distribution relationship between the difference and the frequency, which is P(k). Among them, ; ; Specifically, is the foreground image data of the nearest three frames, is the background mask, that is, the background image data. X and Y respectively represent the number of rows and columns of the matrix D(x, y), and k is the number of differences.

[0034] 102. Construct the average gray signal matrix of the foreground difference matrix, and perform filtering processing on the average gray signal matrix to obtain the image data to be detected.

[0035] In the embodiment of the present application, after obtaining the foreground difference matrix, the average gray signal matrix of this foreground difference matrix is constructed. The average gray signal matrix includes the average gray signal of the gas area to reflect the gray intensity distribution of the gas area, so as to judge whether there is a leaked gas based on this gray intensity distribution. In addition, in order to obtain a part of the image data to be detected for gas leakage detection and improve the effectiveness of the signal intensity, filtering processing is performed on the average gray signal matrix, including but not limited to Gaussian filtering and high-pass filtering, that is, the gray signals belonging to the high-frequency signal part are retained, so as to distinguish the gas signal from the background noise.

[0036] In another embodiment of the present application, for further limitation and explanation, the step of constructing the average gray signal matrix of the foreground difference matrix and performing filtering processing on the average gray signal matrix to obtain the image data to be detected includes: Determine the average gray value of multiple frames of the foreground image data based on the difference in the foreground difference matrix, and generate the average gray signal matrix based on the average gray value; Statistical two-dimensional frequency in the average gray signal matrix to generate a discrete distribution mapping relationship; Perform Gaussian filtering processing on the discrete distribution mapping relationship to obtain the image data to be detected composed of gray values higher than the preset high-frequency signal range.

[0037] To highlight the distinction between the foreground and the background, thereby improving the detection accuracy of gas leakage, when the current execution end constructs the average gray signal matrix, specifically, based on the differences in the foreground difference matrix, the average gray value of the selected multiple frames of foreground image data is determined, and this average gray value is used to form the average gray signal matrix. That is, for the pixels corresponding to the gas region marked in the foreground difference matrix, such as the part where D(x,y)>0, calculate the average gray value of these pixel differences in the selected nearest three frames of image data to form the average gray signal matrix S(x,y). At this time, the average gray signal matrix is used to characterize the gray intensity distribution of the gas region. In addition, the discrete distribution mapping relationship includes the relationship between the gray value and the frequency, that is, two-dimensional frequency analysis is performed on the average gray signal matrix S(x,y), including counting the occurrence frequencies of different gray values in the matrix to generate the discrete distribution mapping relationship g(k), so as to achieve the purpose of revealing the main concentration range of the gas signal by quantifying the signal intensity distribution.

[0038] It should be noted that, in order to provide clearer data features and enhance the continuity of the signal distribution, the current execution end performs Gaussian filtering on the discrete distribution mapping relationship g(k) to eliminate noise interference and smooth data fluctuations. Furthermore, during the filtering process, a high-pass filter is used to filter out low-frequency gray signals. For example, gray values below 81 are filtered out, and the high-frequency part is retained, such as gray values in the range of 82-100. In the embodiments of the present application, the setting of the preset high-frequency signal range high is not specifically limited. In addition, the high-frequency signal corresponds to the region with a relatively high concentration and significant changes in the gas plume. Therefore, high-frequency filtering can effectively distinguish the gas signal from the background noise, and then obtain the image data to be detected composed of gray values higher than the preset high-frequency signal range.

[0039] In the implementation scenario of background image data update, after calculating the concentration signal distribution of the gas in the foreground image data, during the background image data update process, wield is used to avoid misclassifying the gas signal as the background, thereby ensuring the integrity of the gas plume. First, use the latest adaptive threshold R calculated by the difference model to recalculate the foreground difference matrix D(x,y) and the average gray signal matrix of the foreground of the nearest three frames of images , expressed as: ; Furthermore, perform two-dimensional frequency mapping and Gaussian filtering on the matrix to obtain the two-dimensional discrete correspondence relationship between the gas signal gray value (signal) and the frequency (frequency) , expressed as: ; where X and Y respectively represent the number of rows and columns of the matrix, and k is the gray value.

[0040] 103. When the background image data passes the latency verification, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data.

[0041] In the embodiments of the present application, in order to ensure that the background image data is the latest background mask to avoid misjudgment of the background, the current execution end performs latency verification on the background image data, that is, determines whether to delay the update of the background image data. After completing the latency verification of the background image data, for example, when the pixels in the background image data meet the latency conditions (such as including spatial conditions or grayscale conditions), the update is delayed or postponed. At this time, the obtained image data to be detected is directly pixel-level weighted and fused with the visible light image data. At this time, a higher weight is configured for the image data to be detected as an infrared image, and the visible light image data with a lower weight is superimposed pixel by pixel to obtain the final fused image, which is the gas leakage detection result image data. As shown in the infrared detection schematic diagram of ethylene gas, this is to avoid the "ghost" phenomenon that may be caused by the movement of non-gas objects (such as equipment, obstacles, etc.) when there is no gas leakage. In addition, the weight configuration can be based on different detection requirements. The visible light image data is obtained by a non-infrared imaging device, and the embodiments of the present application do not make specific limitations. Figure 2 Shown in the infrared detection schematic diagram of ethylene gas, this is to avoid the "ghost" phenomenon that may be caused by the movement of non-gas objects (such as equipment, obstacles, etc.) when there is no gas leakage. In addition, the weight configuration can be based on different detection requirements. The visible light image data is obtained by a non-infrared imaging device, and the embodiments of the present application do not make specific limitations.

[0042] In some embodiments, the image data to be detected and the visible light image data are linearly weighted and superimposed pixel by pixel to generate a fused image denoted as F(x, y), and the formula is expressed as: ; Wherein, is the pixel value at position in the infrared image; is the pixel value at position in the visible light image; is the pre-configured weight coefficient, such as a static weight or a dynamic weight.

[0043] It should be noted that, in order to meet different visualization requirements, the current execution end can also render the infrared thermal signal through a pseudo-color mapping method, such as rendering red as high-temperature gas to improve intuitiveness. In addition, the detected pixels and their neighborhood sample sets can be updated through a preset update rate and probability. For example, if a pixel is located in a gas region and within the target grayscale range, the update is delayed to maintain the integrity of the gas plume. The embodiments of the present application do not make specific limitations.

[0044] In another embodiment of the present application, for further limitation and explanation, before the step of constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Determine the foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on the foreground field radius and the background update factor; When the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of the region of the foreground image pixels is greater than a second preset threshold, it is determined that there is a suspicious gas object.

[0045] In order to improve the effectiveness of gas detection for the foreground and background and highlight the difference between the foreground and background, before the current execution end triggers the construction of the foreground difference matrix, based on the pre-configured foreground field radius R (used to characterize the range covered by foreground pixels) and the background update factor Ф (used to characterize the parameter for background update), determine the foreground image pixels and background image pixels obtained by infrared image acquisition of the target area. Among them, in the neighborhood with a radius R of the foreground field radius, each pixel point is compared with the mask, and each sample pixel in the target area is compared. At this time, since an array is established for each pixel, therefore, the difference can be made between the gray value of the current pixel point and all the values of the pixel point array at the corresponding position, and the absolute value is taken for comparison. When the absolute value is less than the threshold, it means that the foreground and background are similar. Further judge the number of pixel points. When the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of the region of the foreground image pixels is greater than a second preset threshold, it means that the current foreground pixels and the background are the same, otherwise, it is determined as the foreground image pixels, that is, it is determined that there is a suspicious gas object in the foreground image. Among them, the foreground image pixels are used to form the foreground image data where there may be leaked gas, and the background image pixels are used to form the background image data where there is no leaked gas.

[0046] It should be noted that when statistically counting the number of foreground pixels in each frame in real time, morphological operations can be performed for statistical counting, including statistical counting of pixels such as dilation, erosion, and median filtering, and whether there are interfering objects (such as pedestrians and vehicles) is judged according to the foreground area and contour. If the number of foreground pixels exceeds the first preset threshold, the construction of the foreground difference matrix is triggered. In addition, for the configuration of the first preset threshold and the second preset threshold, it can be configured based on the accuracy requirements of gas detection, and can also be configured based on requirements such as image resolution. The embodiments of the present application do not make specific limitations.

[0047] In another embodiment of the present application, for further limitation and illustration, before the step of when the background image data passes the delay verification and the weight fusion of the image data to be detected and the visible light image data is performed to generate the fusion image data, the method further includes: When the target pixel in the background image data is in the gas region and the gray value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data has completed the delay verification; When the target pixel in the background image data is not within the gas region, and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated.

[0048] To improve the accuracy of gas detection and distinguish the gas update effect between the foreground and the background, the current execution end performs a review and verification on the background image data. Specifically, each pixel point in the background image data is judged one by one whether it is within the gas region. At this time, the gas region is determined based on the foreground difference matrix, that is, each pixel point in the background image data is matched with the pixel points with numerical elements in the foreground difference matrix as the basis for each pixel point to be within the gas region. When the target pixel is within the gas region and the gray value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data has completed the delay verification, that is, the delay update of the background image data is avoided, and the gas plume is prevented from being misjudged as the background, thereby maintaining the integrity of gas imaging. When the target pixel in the background image data is not within the gas region, and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated, thereby improving the detection accuracy of the gas plume, as Figure 3 shown in the schematic diagram of the overall infrared gas detection process.

[0049] In addition, when the background image data passes the delay verification, or after the fused image data is generated, the pixel points in the image do not show the gas region. Therefore, it indicates that no gas leakage is detected. Furthermore, the three-frame difference method and median filtering can be introduced to review the background image data. If the difference signal between the image frames is lower than the preset difference threshold, the background subtraction method can be used to directly update the background to eliminate the "ghost" interference caused by non-gas moving objects, such as equipment shaking.

[0050] In another embodiment of the present application, for further limitation and illustration, before the step of constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Obtaining the gain correction coefficient and bias correction coefficient of the infrared image, and constructing a correction mapping relationship based on the gain correction coefficient, the bias correction coefficient, and the mapping relationship between gray scale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0051] To increase the detection effectiveness of gas leakage based on infrared images, the current execution end needs to correct the infrared imaging device before detection to increase the distinction effect between the foreground and the background. First, obtain the gain correction coefficient and bias correction coefficient of the infrared image. At this time, the calculation expression of the gain correction coefficient: ; the calculation expression of the bias correction coefficient: ; among them, and are the response values of the pixels of the infrared detector at high and low temperatures respectively, and are the average response values of the pixels at high and low temperatures, is the gain correction coefficient, is the bias correction coefficient. i and j respectively represent the row and column coordinates of the pixels in the infrared image, and the response value is represented as the gray signal in the infrared image. The corrected pixel output response formula is expressed as: ; among them, is the pixel output response after correction, is the original pixel output response.

[0052] It should be noted that in the embodiments of the present application, a refrigerated infrared gas cloud camera system can be used to collect infrared images and visible light images, including a visible light camera component and a refrigerated infrared detector component. In some embodiments, the infrared filter is located behind the Dewar window and in front of the detector chip, and the detection band is 3.7μm ± 0.2μm to 4.8μm ± 0.2μm. Preferably, the resolution of the mercury cadmium telluride infrared detector is 640×512, and the pixel size is 15μm. Preferably, the industrial area array camera component in the visible light camera component provides the background environment layer, with a resolution parameter of 1624 × 1240 and a frame rate of 60 fps, and the exposure mode is fully automatic mode. At this time, the change in the detector temperature may cause fluctuations in the system sensitivity, an increase in thermal noise, and non-uniformity changes, which may lead to distortion of the gas signal detection. Therefore, in the infrared gas cloud camera system, a two-point correction method is used for non-uniformity correction, that is, a correlation model (i.e., the corrected pixel output response formula) is constructed between the response value of the detector and the temperature to effectively compensate for the influence of temperature changes on the detector performance, thereby improving the detection accuracy and stability of the system and ensuring reliable imaging results can still be obtained under complex and variable environmental conditions.

[0053] In a specific implementation scenario, the registration of the infrared and visible light cameras is performed, specifically including: feature point extraction and feature point matching. Among them, for feature point extraction, a checkerboard calibration board can be used to take multi-angle images under the visible light and infrared cameras respectively, and significant feature points are manually selected or detected in the two images. For example, by observing stationary objects in the images or manually marking key points. For feature point matching, the position of the feature points in the two images can be manually compared to establish a corresponding relationship. For example, the corresponding heat source position (such as the human body contour) in the visible light image is found in the infrared image, or matching is performed by observing similar texture regions to complete the device registration.

[0054] In a specific implementation scenario, for the two-point blackbody correction in non-uniformity correction, it is used to calibrate the temperature measurement accuracy of an infrared camera and eliminate the detector non-uniformity error. Specifically, first, select two high-precision blackbody sources, and set the temperatures to a low temperature point T1 (such as 25 °C) and a high temperature point T2 (such as 50 °C) respectively to cover the target temperature measurement range. Second, through environmental control, place the infrared camera and the blackbody source in a stable environment (to avoid interference from airflows and strong light), and let it stand for 30 minutes to balance the temperature. Further, for the low-temperature point correction, adjust the blackbody source to T1. After the temperature stabilizes, aim the infrared camera at the blackbody surface and collect multiple frames of images (such as 10 frames) to obtain the average response value. Then, for the high-temperature point correction, repeat the above operation, adjust the blackbody source to T2, and obtain the average response value at high temperature. Finally, calculate the correction parameters. According to the response values and theoretical temperatures of the two points, generate a gain matrix (Gain) and an offset matrix (Offset) through linear fitting (or polynomial model). The formula is expressed as: .

[0055] The embodiment of the present application provides a gas leakage detection method based on infrared image processing. Compared with the prior art, in the embodiment of the present application, when it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold, and the adaptive threshold is determined by the maximum value in the fitting curve based on the difference frequency mapping relationship; an average gray signal matrix of the foreground difference matrix is constructed, and the average gray signal matrix is filtered to obtain the image data to be detected; when the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data, which increases the distinction accuracy between the foreground and the background, highlights the detection object of the foreground gas leakage, and improves the recognition ability of gas signals, so as to meet the requirement of accurate gas leakage detection in a complex background environment.

[0056] Further, as an implementation of the method shown above, the embodiment of the present application provides a gas leakage detection device based on infrared image processing, as Figure 1 shown. The device includes: Figure 4 shown, the device includes: A construction module 21, configured to construct a foreground difference matrix based on the foreground image data, the background image data, and an adaptive threshold when it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, and the adaptive threshold is determined by the maximum value in the fitting curve based on the difference frequency mapping relationship; A processing module 22, configured to construct an average gray signal matrix of the foreground difference matrix, and filter the average gray signal matrix to obtain the image data to be detected; A generating module 23, configured to perform weighted fusion on the image data to be detected and visible light image data when the background image data passes the delay check, and generate gas leakage detection result image data.

[0057] Further, the device further includes: A determining module, configured to determine foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on a foreground field radius and a background update factor, where the foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data; when the number of pixels of the foreground image pixels is greater than a first preset threshold, and the area of the region of the foreground image pixels is greater than a second preset threshold, it is determined that there is a suspicious gas object.

[0058] Further, the constructing module is specifically configured to obtain the adaptive threshold; determine the pixel differences between multiple frames of the foreground image data and the background image data, and compare the pixel differences with the adaptive threshold; configure the pixel differences less than or equal to the adaptive threshold to zero values, and construct a foreground difference matrix based on the pixel differences greater than the adaptive threshold and the zero values.

[0059] Further, the constructing module is specifically configured to obtain an initial threshold, and construct an initial foreground difference matrix based on the initial threshold, the foreground image data, and the background image data; perform Gaussian smoothing processing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches a preset condition in the fitting curve as the adaptive threshold.

[0060] Further, the processing module is specifically configured to determine the average gray value of multiple frames of the foreground image data based on the differences in the foreground difference matrix, and generate an average gray signal matrix based on the average gray value; count the two-dimensional frequency in the average gray signal matrix to generate a discrete distribution mapping relationship, where the discrete distribution mapping relationship includes the relationship between the gray value and the frequency; perform Gaussian filtering processing on the discrete distribution mapping relationship to obtain the image data to be detected composed of gray values higher than a preset high-frequency signal range.

[0061] Further, the device further includes: An update module, configured to determine that the background image data has completed delay verification when a target pixel in the background image data is within a gas region and the gray value of the target pixel is greater than a preset high-frequency signal range; when the target pixel in the background image data is not within the gas region and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, update the background image data; wherein, the gas region is determined based on the foreground difference matrix.

[0062] Further, the device further includes: A calibration module, configured to obtain a gain calibration coefficient and an offset calibration coefficient of an infrared image, and construct a calibration mapping relationship based on the gain calibration coefficient, the offset calibration coefficient, and the mapping relationship between gray scale and temperature, so as to calibrate the infrared image acquisition through the calibration mapping relationship.

[0063] An embodiment of the present application provides a gas leakage detection device based on infrared image processing. Compared with the prior art, in the embodiment of the present application, when it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold, and the adaptive threshold is determined by the maximum value in the fitting curve of the difference frequency mapping relationship; an average gray signal matrix of the foreground difference matrix is constructed, and the average gray signal matrix is filtered to obtain image data to be detected; when the background image data passes the delay verification, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data, which increases the distinction accuracy between the foreground and the background, highlights the detection object of foreground gas leakage, improves the recognition ability of gas signals, and thus meets the requirement of accurate gas leakage detection in a complex background environment.

[0064] According to an embodiment of the present application, a storage medium is provided, and the storage medium stores at least one executable instruction, and the computer executable instruction can execute the gas leakage detection method based on infrared image processing in any of the above method embodiments.

[0065] Figure 5 The structural schematic diagram of a terminal provided according to an embodiment of the present application is shown. The specific implementation of the terminal is not limited in the specific embodiment of the present application.

[0066] As Figure 5 shown, the terminal may include: a processor 302, a communication interface 304, a memory 306, and a communication bus 308.

[0067] Among them: The processor 302, the communication interface 304, and the memory 306 communicate with each other through the communication bus 308.

[0068] The communication interface 304 is used to communicate with network elements of other devices such as clients or other servers.

[0069] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above embodiments of the gas leakage detection method based on infrared image processing.

[0070] Specifically, the program 310 may include program code, and the program code includes computer operation instructions.

[0071] The processor 302 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the terminal may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0072] The memory 306 is used to store the program 310. The memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0073] The program 310 is specifically used to cause the processor 302 to perform the following operations: When it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold, and the adaptive threshold is determined by the maximum value in the fitting curve of the difference frequency mapping relationship; Construct an average gray signal matrix of the foreground difference matrix, and perform filtering processing on the average gray signal matrix to obtain the image data to be detected; When the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data.

[0074] Obviously, those skilled in the art should understand that the various modules or steps of the present application described above can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0075] The foregoing is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A gas leakage detection method based on infrared image processing, characterized in that, Including: When it is determined that there is a suspicious gas object based on the foreground image pixels of the target area to be detected, a foreground difference matrix is constructed based on the foreground image data, the background image data, and an adaptive threshold, and the adaptive threshold is determined based on the maximum value in the fitting curve of the difference frequency mapping relationship; Construct an average gray signal matrix of the foreground difference matrix, and perform filtering processing on the average gray signal matrix to obtain the image data to be detected; When the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate gas leakage detection result image data.

2. The method according to claim 1, wherein Before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Determine the foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on the foreground domain radius and the background update factor. The foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data; When the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of the foreground image pixels is greater than a second preset threshold, it is determined that there is a suspicious gas object.

3. The method according to claim 2, wherein Constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold includes: Obtain the adaptive threshold; Determine the pixel differences between multiple frames of the foreground image data and the background image data, and compare the pixel differences with the adaptive threshold; Configure the pixel differences less than or equal to the adaptive threshold to zero values, and construct the foreground difference matrix based on the pixel differences greater than the adaptive threshold and the zero values.

4. The method according to claim 3, characterized in that, Obtaining the adaptive threshold includes: Obtain an initial threshold, and construct an initial foreground difference matrix based on the initial threshold, the foreground image data, and the background image data; Perform Gaussian smoothing processing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; Perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches the preset condition in the fitting curve as the adaptive threshold.

5. The method according to claim 4, wherein Constructing the average gray signal matrix of the foreground difference matrix, and performing filtering processing on the average gray signal matrix to obtain the image data to be detected includes: Determine the average gray value of multiple frames of the foreground image data based on the differences in the foreground difference matrix, and generate the average gray signal matrix based on the average gray value; Statistically calculate the two-dimensional frequency in the average gray signal matrix to generate a discrete distribution mapping relationship, and the discrete distribution mapping relationship includes the relationship between the gray value and the frequency; Perform Gaussian filtering processing on the discrete distribution mapping relationship to obtain the image data to be detected composed of gray values higher than the preset high-frequency signal range.

6. The method according to claim 1, characterized in that, Before, when the background image data passes the delay check, the image data to be detected is weighted and fused with the visible light image data to generate fused image data, the method further includes: When the target pixel in the background image data is within the gas region and the gray value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data has completed the delay verification; When the target pixel in the background image data is not within the gas region, and / or the gray value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated; wherein, the gas region is determined based on the foreground difference matrix.

7. The method according to any one of claims 1-6, characterized in that, Before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Obtaining the gain correction coefficient and the offset correction coefficient of the infrared image, and constructing a correction mapping relationship based on the gain correction coefficient, the offset correction coefficient, and the mapping relationship between gray scale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

8. A gas leakage detection device based on infrared image processing, characterized in that, including: A construction module, configured to construct a foreground difference matrix based on the foreground image data, the background image data, and an adaptive threshold when it is determined that there is a suspicious gas object based on the foreground image pixels of the target region to be detected, and the adaptive threshold is determined by the maximum value in the fitting curve of the difference frequency mapping relationship; A processing module, configured to construct an average gray signal matrix of the foreground difference matrix, and perform filtering processing on the average gray signal matrix to obtain the image data to be detected; A generation module, configured to, when the background image data passes the delay verification, perform weight fusion on the image data to be detected and the visible light image data to generate gas leakage detection result image data.

9. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method described in claim 1 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method described in claim 1.

Images