Gas leakage detection method, device and terminal based on infrared image processing

Abstract

The present application discloses a gas leak detection method, device, and terminal based on infrared image processing, which relates to the field of image processing technology and aims to solve the problem of low accuracy in existing gas leak detection. The method comprises: when a suspicious gas object is determined to exist based on foreground image pixels in a target area to be detected, constructing a foreground difference matrix based on foreground image data, background image data, and an adaptive threshold, wherein the adaptive threshold is determined by the maximum value in a fitting curve based on a differential frequency mapping relationship; constructing an average grayscale signal matrix of the foreground difference matrix, and filtering the average grayscale signal matrix to obtain image data to be detected; and when the background image data passes the delay check, weighted fusion of the image data to be detected and the visible light image data is performed to generate image data of gas leak detection results.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a gas leakage detection method, device and terminal based on infrared image processing. Background Art

[0002] Gas leaks can cause environmental pollution, explosions, poisoning, and even fires, posing serious threats to human health, the ecological environment, and industrial safety. This is why gas detection imaging technology has emerged. Passive infrared detection imaging, with its long detection range and wide coverage, is particularly suitable for complex environments.

[0003] Currently, gas leak detection is usually carried out using passive infrared detection imaging technology. However, since the gas signal in infrared gas imaging is usually weak and the image contrast is low, the distinction between gas and background is not high, making it difficult to clearly identify the presence and spatial distribution of gas. In addition, weak gas signals are prone to losing detailed information during the imaging process. Especially in complex background environments, the texture and shape around the gas leak point may be blurred or obscured, affecting the accurate judgment of the leak location and extent, resulting in low gas leak detection accuracy. Summary of the Invention

[0004] In view of this, the present application provides a gas leakage detection method, device and terminal based on infrared image processing, the main purpose of which is to solve the problem of low accuracy in existing gas leakage detection.

[0005] According to one aspect of the present application, a gas leak detection method based on infrared image processing is provided, comprising:

[0006] When a suspicious gas object is determined to exist based on foreground image pixels of the target area to be detected, a foreground difference matrix is ​​constructed based on the foreground image data, the background image data, and an adaptive threshold value, wherein the adaptive threshold value is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship;

[0007] Constructing an average grayscale signal matrix of the foreground difference matrix, and performing filtering processing on the average grayscale signal matrix to obtain image data to be detected;

[0008] When the background image data passes the delay check, the image data to be detected and the visible light image data are weightedly fused to generate gas leakage detection result image data.

[0009] Furthermore, before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes:

[0010] Determine foreground image pixels and background image pixels obtained by performing infrared image acquisition on the target area based on the foreground area radius and the background update factor, wherein the foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data;

[0011] When the number of pixels of the foreground image is greater than a first preset threshold, and the area of ​​the foreground image pixels is greater than a second preset threshold, it is determined that a suspicious gas object exists.

[0012] Furthermore, constructing a foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold includes:

[0013] Obtaining the adaptive threshold;

[0014] Determining pixel differences between a plurality of frames of the foreground image data and the background image data, and comparing the pixel differences with the adaptive threshold;

[0015] Pixel difference values ​​less than or equal to the adaptive threshold are configured as zero values, and the foreground difference matrix is ​​constructed based on pixel difference values ​​greater than the adaptive threshold and the zero value.

[0016] Furthermore, obtaining the adaptive threshold includes:

[0017] Obtaining an initial threshold, and constructing an initial foreground difference matrix based on the initial threshold and the foreground image data and the background image data;

[0018] Performing Gaussian smoothing on the initial foreground difference matrix to obtain a difference frequency mapping relationship;

[0019] Curve fitting is performed on the difference-frequency mapping relationship to obtain a fitting curve, and a difference value corresponding to a maximum value matching a preset condition in the fitting curve is determined as an adaptive threshold.

[0020] Furthermore, the constructing of the average grayscale signal matrix of the foreground difference matrix and filtering the average grayscale signal matrix to obtain the image data to be detected includes:

[0021] determining an average grayscale value of a plurality of frames of foreground image data based on the difference values ​​in the foreground difference matrix, and generating the average grayscale signal matrix based on the average grayscale value;

[0022] Counting the two-dimensional frequencies in the average grayscale signal matrix to generate a discrete distribution mapping relationship, wherein the discrete distribution mapping relationship includes a relationship between grayscale values ​​and frequencies;

[0023] Gaussian filtering is performed on the discrete distribution mapping relationship to obtain the image data to be detected composed of grayscale values ​​higher than a preset high-frequency signal range.

[0024] Furthermore, when the background image data passes the delay check, the image data to be detected is weighted-fused with the visible light image data to generate the fused image data, and before the method further comprises:

[0025] When the target pixel in the background image data is within the gas region and the grayscale value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data completes the delay check;

[0026] When the target pixel in the background image data is not in the gas area, and / or the grayscale value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated;

[0027] The gas region is determined based on the foreground difference matrix.

[0028] Furthermore, before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes:

[0029] A gain correction coefficient and a bias correction coefficient of the infrared image are obtained, and a correction mapping relationship is constructed based on the gain correction coefficient, the bias correction coefficient and the mapping relationship between grayscale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0030] According to another aspect of the present application, a gas leak detection device based on infrared image processing is provided, comprising:

[0031] a construction module for constructing a foreground difference matrix based on foreground image data, background image data, and an adaptive threshold when a suspicious gas object is determined to exist based on foreground image pixels in a target area to be detected, wherein the adaptive threshold is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship;

[0032] a processing module, configured to construct an average grayscale signal matrix of the foreground difference matrix, and perform filtering processing on the average grayscale signal matrix to obtain image data to be detected;

[0033] The generating module is used for performing weighted fusion on the image data to be detected and the visible light image data when the background image data passes the delay check, so as to generate gas leakage detection result image data.

[0034] Furthermore, the device further comprises:

[0035] A determination module is used to determine foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on the foreground area radius and the background update factor, the foreground image pixels are used to constitute the foreground image data, and the background image pixels are used to constitute the background image data; when the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of ​​the foreground image pixels is greater than a second preset threshold, it is determined that a suspicious gas object exists.

[0036] Furthermore, the construction module is specifically used to obtain the adaptive threshold; determine the pixel difference between multiple frames of the foreground image data and the background image data, and compare the pixel difference with the adaptive threshold; configure the pixel difference less than or equal to the adaptive threshold as zero value, and construct the foreground difference matrix based on the pixel difference greater than the adaptive threshold and the zero value.

[0037] Furthermore, the construction module is specifically used to obtain an initial threshold and construct an initial foreground difference matrix based on the initial threshold and the foreground image data and background image data; perform Gaussian smoothing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches the preset conditions in the fitting curve as the adaptive threshold.

[0038] Furthermore, the processing module is specifically used to determine the average grayscale value of multiple frames of foreground image data based on the difference in the foreground difference matrix, and generate the average grayscale signal matrix based on the average grayscale value; count the two-dimensional frequencies in the average grayscale signal matrix to generate a discrete distribution mapping relationship, and the discrete distribution mapping relationship includes the relationship between grayscale value and frequency; perform Gaussian filtering on the discrete distribution mapping relationship to obtain the image data to be detected composed of grayscale values ​​higher than a preset high-frequency signal range.

[0039] Furthermore, the device further comprises:

[0040] An updating module is configured to determine that the background image data completes delayed verification when a target pixel in the background image data is within a gas area and a grayscale value of the target pixel is greater than a preset high-frequency signal range; and to update the background image data when a target pixel in the background image data is not within a gas area and / or a grayscale value of the target pixel is less than or equal to a preset high-frequency signal range; wherein the gas area is determined based on the foreground difference matrix.

[0041] Furthermore, the device further comprises:

[0042] The correction module is used to obtain the gain correction coefficient and the bias correction coefficient of the infrared image, and construct a correction mapping relationship based on the gain correction coefficient, the bias correction coefficient and the mapping relationship between grayscale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0043] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored. The executable instruction enables a processor to perform operations corresponding to the above-mentioned gas leakage detection method based on infrared image processing.

[0044] According to another aspect of the present application, there is provided a terminal, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned gas leakage detection method based on infrared image processing.

[0046] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages:

[0047] The present application provides a gas leak detection method, device and terminal based on infrared image processing. Compared with the prior art, the embodiments of the present application construct a foreground difference matrix based on foreground image data, background image data and an adaptive threshold when a suspicious gas object is determined to exist based on foreground image pixels of a target area to be detected, wherein the adaptive threshold is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship; construct an average grayscale signal matrix of the foreground difference matrix, and filter the average grayscale signal matrix to obtain image data to be detected; when the background image data passes the delay check, the image data to be detected and the visible light image data are weightedly fused to generate gas leak detection result image data, thereby increasing the accuracy of distinguishing between the foreground and the background, highlighting the detection object of the foreground gas leakage, and improving the recognition ability of the gas signal, thereby meeting the needs of accurate gas leak detection in a complex background environment.

[0048] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0050] Figure 1 A flow chart of a gas leak detection method based on infrared image processing provided by an embodiment of the present application is shown;

[0051] Figure 2 A schematic diagram of infrared detection of ethylene gas provided in an embodiment of the present application is shown;

[0052] Figure 3 A schematic diagram of the overall process of infrared gas detection provided by an embodiment of the present application is shown;

[0053] Figure 4 The following is a block diagram showing the composition of a gas leakage detection device based on infrared image processing provided by an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] The present invention provides a method for detecting gas leaks based on infrared image processing. Figure 1 As shown, the method includes:

[0057] 101. When it is determined that a suspicious gas object exists based on foreground image pixels of the target area to be detected, a foreground difference matrix is ​​constructed based on foreground image data, background image data, and an adaptive threshold.

[0058] In an embodiment of the present application, as the execution subject of gas leak detection, the current execution end can be a terminal device or a server device, so as to collect infrared images of the target area to be detected and perform leak detection. The target area to be detected is an area where gas leakage may occur. In this case, the gas may include but is not limited to combustible gases, toxic gases, such as methane and ethylene, which are not specifically limited in this embodiment of the present application. The foreground image pixels are pixels in the foreground image data. The foreground image data is combined with the background image data to form the collected infrared image data. The foreground image data represents the area where gas leakage may occur relative to the background image data. Therefore, after determining the foreground image data, the foreground image pixels are statistically analyzed for contour or area to determine whether a suspicious gas object exists, which serves as a step to start the gas detection in this embodiment. When the presence of a suspicious gas object is determined, the current execution end constructs a difference matrix based on the foreground image data and the background image data and an adaptive threshold. In this case, the difference matrix is ​​used to represent the difference matrix of frequency domain statistics, so as to convert the threshold determination for distinguishing noise from gas into a numerical solution, avoiding the limitations of traditional fixed thresholds. It is particularly suitable for low-contrast, dynamically changing gas leak detection scenarios.

[0059] It should be noted that the adaptive threshold is determined by the maximum value in the fitting curve based on the differential frequency mapping relationship. At this time, the differential frequency mapping relationship includes the relationship between different differential values ​​and the frequency of occurrence. The differential matrix for updating the adaptive threshold can be constructed based on the previous multiple frame image data in the real-time foreground image data, and the differential frequency mapping relationship is statistically obtained for fitting, and the maximum value in the fitting curve is determined to determine the adaptive threshold based on this maximum value for constructing the foreground differential matrix.

[0060] In another embodiment of the present application, for further definition and explanation, the step of constructing a foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold includes:

[0061] Obtaining the adaptive threshold;

[0062] Determining pixel differences between a plurality of frames of the foreground image data and the background image data, and comparing the pixel differences with the adaptive threshold;

[0063] Pixel difference values ​​less than or equal to the adaptive threshold are configured as zero values, and the foreground difference matrix is ​​constructed based on pixel difference values ​​greater than the adaptive threshold and the zero value.

[0064] In order to analyze the grayscale distribution characteristics of gases through a dynamic threshold method, combined with frequency domain filtering and a delayed update mechanism, the spatial concentration information of the gas plume is retained while effectively suppressing the background misjudgment problem, thereby improving the accuracy and stability of gas imaging. The current execution end first obtains an adaptive threshold. Among them, the multi-frame foreground image data may include but is not limited to the pixels of the foreground image data of the first three frames at the current moment. By calculating the difference between the pixels of each frame and the pixels in the background image data one by one, each pixel difference is compared with the adaptive threshold. At this time, the pixel difference greater than the adaptive threshold is retained, and the pixel difference less than or equal to the adaptive threshold is configured to zero. Finally, the pixel difference that meets the conditions is combined with zero to form a foreground difference matrix.

[0065] In another embodiment of the present application, for further definition and explanation, the step of obtaining the adaptive threshold includes:

[0066] Obtaining an initial threshold, and constructing an initial foreground difference matrix based on the initial threshold and the foreground image data and the background image data;

[0067] Performing Gaussian smoothing on the initial foreground difference matrix to obtain a difference frequency mapping relationship;

[0068] Curve fitting is performed on the difference-frequency mapping relationship to obtain a fitting curve, and a difference value corresponding to a maximum value matching a preset condition in the fitting curve is determined as an adaptive threshold.

[0069] To avoid the limitations of gas identification caused by traditional fixed thresholds, the current execution end first configures an initial threshold, preferably 3, and constructs an initial foreground difference matrix based on the initial threshold and the foreground and background image data. Specifically, the current execution end continuously acquires three frames of foreground image data from the real-time infrared video stream, denoted as M_a, M_b, and M_c, and calls the current background image data, denoted as M. These three frames of foreground and background image data are then used to capture dynamically changing gas signals. To construct the foreground difference matrix, the pixels in the three foreground image frames are first averaged to obtain the average foreground pixel value M_avg. The average foreground image is then compared pixel by pixel with the background model, calculating the pixel difference for each pixel. If the pixel difference exceeds an initial threshold, such as 3, the pixel difference is retained; otherwise, it is set to zero. Finally, a difference matrix D(x,y) is generated, representing the areas of significant change between the current scene and the background.

[0070] In addition, after obtaining the foreground difference matrix, Gaussian smoothing is performed, and the difference matrix after Gaussian smoothing is traversed to count the frequency of occurrence of all non-zero difference values ​​to construct a difference frequency mapping relationship. For example, if there are 100 pixels with a difference value of 5, then it is recorded as P(5)=100 to map the discrete difference value to its corresponding frequency distribution, forming a "difference value-frequency" relationship table, which is the difference frequency mapping relationship. Among them, the difference frequency mapping relationship includes the statistically obtained "difference value" as the independent variable (x) and the corresponding "frequency" as the dependent variable (y). These data points can be curve fitted by a fourth-order polynomial. At this time, the fourth-order polynomial can flexibly describe the rise, fall and inflection point characteristics of the frequency distribution, while avoiding the overfitting that may be caused by high-order polynomials. At the same time, the coefficients of the fourth-order polynomial are solved by the least squares method, and finally a fitting curve is obtained. Then, the first-order derivative of the fitted polynomial curve is calculated, and the point where the derivative is zero is found, which is the extreme point of the fitting curve. At this time, it is possible to verify whether these extreme points are maximum points within a preset differential value range, such as 3 to 30. For example, confirmation can be made through the second-order derivative, so that the first maximum point that appears is selected as the candidate value of the optimal threshold, that is, the optimal adaptive threshold R. If no maximum point is found within a preset differential value range, the previous adaptive or initial threshold can be used. This embodiment of the present application does not make specific limitations.

[0071] In a specific implementation scenario, when constructing the initial foreground difference matrix with the initial threshold, in order to capture as many gas targets as possible, a smaller transient initial threshold R' is pre-set, such as R' = 3, and the average difference matrix D(x,y) between the background mask and the last three frames of foreground image data is calculated. At this time, signal points whose difference between the foreground image data and the background image data is greater than or equal to the initial soil quality R' will be extracted. In addition, when processing based on Gaussian filtering, Gaussian filtering is used to reduce noise and smooth the image. After two-dimensional frequency mapping of the difference matrix, Gaussian filtering is applied to obtain a discrete distribution relationship between the difference (difference) and the frequency (frequency), which is P(k). Among them,

[0072] ; ;

[0073] Specifically, is the most recent three frames of foreground image data, is the background mask, that is, the background image data, X and Y represent the number of rows and columns of the matrix D(x,y), and k is the difference number.

[0074] 102. Construct an average grayscale signal matrix of the foreground difference matrix, and perform filtering processing on the average grayscale signal matrix to obtain image data to be detected.

[0075] In the present embodiment, after obtaining a foreground difference matrix, an average grayscale signal matrix is ​​constructed from this foreground difference matrix. The average grayscale signal matrix includes the average grayscale signal of the gas region, reflecting the grayscale intensity distribution of the gas region, and the presence of a gas leak is determined based on this grayscale intensity distribution. Furthermore, to obtain a portion of the image data to be tested that can be used for gas leak detection and to improve the effectiveness of signal intensity, the average grayscale signal matrix is ​​filtered, including but not limited to Gaussian filtering and high-pass filtering. This process retains the grayscale signal of the high-frequency signal portion, thereby distinguishing the gas signal from background noise.

[0076] In another embodiment of the present application, for further definition and explanation, the step of constructing an average grayscale signal matrix of the foreground difference matrix and filtering the average grayscale signal matrix to obtain the image data to be detected includes:

[0077] determining an average grayscale value of a plurality of frames of foreground image data based on the difference values ​​in the foreground difference matrix, and generating the average grayscale signal matrix based on the average grayscale value;

[0078] Counting the two-dimensional frequencies in the average grayscale signal matrix to generate a discrete distribution mapping relationship;

[0079] Gaussian filtering is performed on the discrete distribution mapping relationship to obtain the image data to be detected composed of grayscale values ​​higher than a preset high-frequency signal range.

[0080] To distinguish foreground from background, thereby improving gas leak detection accuracy, the current execution end constructs the average grayscale signal matrix. Specifically, the average grayscale value of multiple frames of selected foreground image data is determined based on the differences in the foreground difference matrix. This average grayscale value forms the average grayscale signal matrix. Specifically, for pixels in the foreground difference matrix marked as gas regions (e.g., where D(x,y)>0), the average grayscale value of these pixel differences in the three most recent selected image frames is calculated to form the average grayscale signal matrix S(x,y). This average grayscale signal matrix is ​​used to characterize the grayscale intensity distribution of the gas regions. Furthermore, the discrete distribution mapping relationship includes the relationship between grayscale value and frequency. Specifically, a two-dimensional frequency analysis is performed on the average grayscale signal matrix S(x,y), including counting the frequency of occurrence of different grayscale values ​​in the matrix, to generate the discrete distribution mapping relationship g(k). This quantifies the signal intensity distribution and reveals the primary concentration range of the gas signal.

[0081] It should be noted that in order to provide clearer data features and enhance the continuity of signal distribution, the current execution end performs Gaussian filtering on the discrete distribution mapping relationship g(k) to eliminate noise interference and smooth data fluctuations. Furthermore, in the filtering process, a high-pass filter is used to filter out low-frequency grayscale signals, for example, grayscale values ​​below 81 are filtered out, and high-frequency parts, such as grayscale values ​​in the range of 82 to 100, are retained. In the embodiment of the present application, the setting of the preset high-frequency signal range high is not specifically limited. In addition, high-frequency signals correspond to areas in the gas plume where the concentration is high and the changes are significant. Therefore, high-frequency filtering can effectively distinguish gas signals from background noise, thereby obtaining image data to be detected based on grayscale values ​​higher than the preset high-frequency signal range.

[0082] In the implementation scenario of background image data update, after calculating the concentration signal distribution of the gas in the foreground image data, wield avoids mistakenly classifying the gas signal as background during the background image data update process, thereby ensuring the integrity of the gas plume. First, the latest adaptive threshold R calculated by the difference model is used to recalculate the foreground difference matrix D(x,y) of the last three frames of images and the foreground average grayscale signal matrix , expressed as:

[0083] ;

[0084] Furthermore, for the matrix Perform two-dimensional frequency mapping and Gaussian filtering to obtain a two-dimensional discrete correspondence between the grayscale value of the gas signal (signal) and the frequency (frequency) , expressed as:

[0085] ; Where X and Y represent the number of rows and columns of the matrix respectively, and k is the grayscale value.

[0086] 103. When the background image data passes the delay check, weighted fusion is performed on the image data to be detected and the visible light image data to generate gas leakage detection result image data.

[0087] In an embodiment of the present application, in order to ensure that the background image data is the latest background mask to avoid background misjudgment, the current execution end performs a delayed check on the background image data, that is, determines whether the background image data needs to be delayed updated. After completing the delayed check on the background image data, for example, when the pixels in the background image data meet the delay conditions (such as spatial conditions or grayscale conditions), the update is delayed or suspended. At this time, the obtained image data to be detected is directly fused with the visible light image data at the pixel level. At this time, a higher weight is configured for the image data to be detected as an infrared image, and a lower weight is configured for the visible light image data to be superimposed pixel by pixel to obtain the final fused image, which is the gas leak detection result image data, such as Figure 2 The following figure shows a schematic diagram of infrared detection of ethylene gas. This avoids the "ghosting" phenomenon that may be caused by the movement of non-gas objects (such as equipment and obstacles) when no gas leak occurs. Furthermore, the weight configuration can be customized based on different detection requirements. The visible light image data is captured by non-infrared imaging equipment, and this embodiment of the application does not specifically limit this.

[0088] In some embodiments, the image data to be detected and the visible light image data are linearly weighted superimposed pixel by pixel to generate a fusion image represented by F(x, y), which is expressed as follows:

[0089] ;

[0090] in, is the position in the infrared image Pixel value of is the position in the visible light image Pixel value of A pre-configured weight coefficient, such as a static weight or a dynamic weight.

[0091] It should be noted that, to meet different visualization requirements, the current execution end can also render infrared thermal signals using pseudo-color mapping, such as rendering red to represent high-temperature gas, to improve intuitiveness. Furthermore, detected pixels and their neighborhood sample sets can be updated using a preset update rate and probability. For example, if a pixel is located in a gas region and within a target grayscale range, the update is delayed to maintain the integrity of the gas plume. This is not specifically limited in this embodiment of the present application.

[0092] In another embodiment of the present application, for further definition and explanation, before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes:

[0093] Determine the foreground image pixels and background image pixels obtained by performing infrared image acquisition on the target area based on the foreground area radius and the background update factor;

[0094] When the number of pixels of the foreground image is greater than a first preset threshold, and the area of ​​the foreground image pixels is greater than a second preset threshold, it is determined that a suspicious gas object exists.

[0095] To improve the effectiveness of foreground and background gas detection and highlight the distinction between the two, the current execution end determines foreground and background image pixels obtained from infrared image acquisition of the target area based on a preconfigured foreground radius R (which represents the range covered by the foreground pixels) and a background update factor Φ (a parameter that indicates whether background updates are performed) before triggering the construction of the foreground difference matrix. Each pixel within the foreground radius R is compared with the mask and the sample pixels in the target area. Since an array is created for each pixel, the grayscale value of the current pixel is subtracted from all values ​​in the corresponding pixel array, and the absolute value is compared. If the absolute value is less than a threshold, the foreground and background are similar. The number of pixels is further determined. If the number of foreground pixels exceeds a first preset threshold and the area of ​​the foreground pixel region exceeds a second preset threshold, the current foreground pixel is considered identical to the background. Otherwise, it is determined to be a foreground pixel, indicating the presence of a suspicious gas object in the foreground image. The foreground image pixels are used to form foreground image data indicating possible gas leakage, and the background image pixels are used to form background image data indicating no gas leakage.

[0096] It should be noted that when counting the number of foreground pixels per frame in real time, morphological operations can be performed, including pixel count statistics such as dilation, erosion, and median filtering. The presence of interference objects (such as pedestrians or vehicles) can be determined based on the area and outline of the foreground region. If the number of foreground pixels exceeds a first preset threshold, the construction of the foreground difference matrix is ​​triggered. In addition, the configuration of the first and second preset thresholds can be based on the accuracy requirements of gas detection and can also be based on requirements such as image resolution, and this is not specifically limited in the embodiments of this application.

[0097] In another embodiment of the present application, for further definition and explanation, when the background image data passes the delay check, the image data to be detected is weighted-fused with the visible light image data to generate the fused image data, the method further includes:

[0098] When the target pixel in the background image data is within the gas region and the grayscale value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data completes the delay check;

[0099] When the target pixel in the background image data is not in the gas area, and / or the grayscale value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated.

[0100] In order to improve the accuracy of gas detection and distinguish the gas update effects in the foreground and background, the current execution end reviews and verifies the background image data. Specifically, each pixel in the background image data is judged one by one to see if it is in the gas area. At this time, the gas area is determined based on the foreground differential matrix, that is, each pixel in the background image data is matched with the pixel with a numerical element in the foreground differential matrix, as a basis for each pixel to be in the gas area. When the target pixel is in the gas area, and the grayscale value of the target pixel is greater than the preset high-frequency signal range, it is determined that the background image data completes the delayed check, that is, the background image data is delayed to update, to avoid the gas plume being misjudged as the background, thereby maintaining the integrity of the gas imaging. When the target pixel in the background image data is not in the gas area, and / or the grayscale value of the target pixel is less than or equal to the preset high-frequency signal range, the background image data is updated, thereby improving the detection accuracy of the gas plume, such as Figure 3 The overall process diagram of infrared gas detection is shown in FIG.

[0101] In addition, when the background image data passes the delayed verification, or after the fused image data is generated, the pixels in the image do not show a gas area, therefore, it means that the gas leakage has not been detected. Then, the three-frame difference method and median filtering can be introduced to review the background image data. If the difference signal between each image frame is lower than the preset difference threshold, the background subtraction method can be used to directly update the background to eliminate the "ghost" interference caused by non-gas moving objects, such as equipment shaking.

[0102] In another embodiment of the present application, for further definition and explanation, before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes:

[0103] A gain correction coefficient and a bias correction coefficient of the infrared image are obtained, and a correction mapping relationship is constructed based on the gain correction coefficient, the bias correction coefficient and the mapping relationship between grayscale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0104] In order to increase the effectiveness of gas leak detection based on infrared images, the current execution end needs to calibrate the infrared imaging device before detection to improve the distinction between foreground and background. First, obtain the gain correction coefficient and offset correction coefficient of the infrared image. At this time, the calculation expression of the gain correction coefficient is: ; Calculation expression of bias correction coefficient: ;in, and are the response values ​​of the infrared detector pixels at high and low temperatures, and is the average response value of the pixel at high and low temperatures, is the gain correction factor, is the bias correction coefficient, i and j represent the row and column coordinates of the pixel in the infrared image, and the response value is expressed as the grayscale signal in the infrared image. The corrected pixel output response formula is expressed as: ;in, is the pixel output response after correction, Output the response for the original pixel.

[0105] It should be noted that in the embodiments of the present application, a cooled infrared gas cloud camera system can be used to capture infrared and visible light images, including a visible light camera assembly and a cooled infrared detector assembly. In some embodiments, the infrared filter is located after the Dewar window and before the detector chip, with a detection band of 3.7μm±0.2μm to 4.8μm±0.2μm. Preferably, the HgCdTe infrared detector has a resolution of 640×512 and a pixel size of 15μm. Preferably, the industrial area array camera assembly in the visible light camera assembly provides a background environment layer, with a resolution of 1624×1240, a frame rate of 60 fps, and a fully automatic exposure mode. At this time, changes in detector temperature may cause fluctuations in system sensitivity, increase in thermal noise, and changes in non-uniformity, which in turn lead to distortion in gas signal detection. Therefore, in the infrared gas cloud camera system, a two-point correction method is used for non-uniformity correction, that is, by constructing a correlation model between the detector response value and temperature (that is, the corrected pixel output response formula) to effectively compensate for the impact of temperature changes on detector performance, thereby improving the system's detection accuracy and stability, and ensuring that reliable imaging results can still be obtained under complex and changing environmental conditions.

[0106] In a specific implementation scenario, the infrared and visible light cameras are aligned, specifically including feature point extraction and feature point matching. Feature point extraction can use a checkerboard calibration plate, capture multi-angle images with the visible light and infrared cameras, and manually select or detect significant feature points in both images, for example, by observing stationary objects in the images or manually marking key points. Feature point matching can establish a correspondence by manually comparing the positions of feature points in the two images. For example, the corresponding heat source position (such as a human body outline) in the visible light image is found in the infrared image, or by observing similar texture areas for matching, to complete device alignment.

[0107] In a specific implementation scenario, two-point blackbody calibration is used to calibrate the temperature measurement accuracy of an infrared camera and eliminate detector non-uniformity errors. Specifically, two high-precision blackbody sources are selected and set to a low-temperature point, T1 (e.g., 25°C), and a high-temperature point, T2 (e.g., 50°C), respectively, to cover the target temperature measurement range. Next, using environmental control, the infrared camera and blackbody source are placed in a stable environment (avoiding airflow and strong light interference) and left to equilibrate for 30 minutes. Next, for low-temperature calibration, the blackbody source is adjusted to T1. After the temperature stabilizes, the infrared camera is focused on the blackbody surface, and multiple frames (e.g., 10 frames) are captured to calculate the average response value. Next, for high-temperature calibration, the above steps are repeated, adjusting the blackbody source to T2 to obtain the average response value at high temperature. Finally, the calibration parameters are calculated. Based on the response values ​​at the two points and the theoretical temperature, a linear fit (or polynomial model) is used to generate the gain matrix (Gain) and the offset matrix (Offset). The formula is expressed as:

[0108] .

[0109] An embodiment of the present application provides a gas leak detection method based on infrared image processing. Compared with the prior art, the embodiment of the present application constructs a foreground difference matrix based on foreground image data, background image data and an adaptive threshold when a suspicious gas object is determined to exist based on foreground image pixels of a target area to be detected, wherein the adaptive threshold is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship; constructs an average grayscale signal matrix of the foreground difference matrix, and filters the average grayscale signal matrix to obtain image data to be detected; when the background image data passes a delay check, weighted fusion is performed on the image data to be detected and the visible light image data to generate gas leak detection result image data, thereby increasing the accuracy of distinguishing between the foreground and the background, highlighting the detection object of the foreground gas leakage, and improving the recognition ability of the gas signal, thereby meeting the demand for accurate gas leak detection in a complex background environment.

[0110] Furthermore, as a response to the above Figure 1 The present invention provides a gas leakage detection device based on infrared image processing. Figure 4 As shown, the device includes:

[0111] A construction module 21 is configured to construct a foreground difference matrix based on the foreground image data, the background image data, and an adaptive threshold value when a suspicious gas object is determined to exist based on foreground image pixels in the target area to be detected, wherein the adaptive threshold value is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship;

[0112] A processing module 22 is used to construct an average grayscale signal matrix of the foreground difference matrix and perform filtering processing on the average grayscale signal matrix to obtain image data to be detected;

[0113] The generating module 23 is configured to perform weighted fusion on the image data to be detected and the visible light image data when the background image data passes the delay check, so as to generate gas leakage detection result image data.

[0114] Furthermore, the device further comprises:

[0115] A determination module is used to determine foreground image pixels and background image pixels obtained by infrared image acquisition of the target area based on the foreground area radius and the background update factor, the foreground image pixels are used to constitute the foreground image data, and the background image pixels are used to constitute the background image data; when the number of pixels of the foreground image pixels is greater than a first preset threshold and the area of ​​the foreground image pixels is greater than a second preset threshold, it is determined that a suspicious gas object exists.

[0116] Furthermore, the construction module is specifically used to obtain the adaptive threshold; determine the pixel difference between multiple frames of the foreground image data and the background image data, and compare the pixel difference with the adaptive threshold; configure the pixel difference less than or equal to the adaptive threshold as zero value, and construct the foreground difference matrix based on the pixel difference greater than the adaptive threshold and the zero value.

[0117] Furthermore, the construction module is specifically used to obtain an initial threshold and construct an initial foreground difference matrix based on the initial threshold and the foreground image data and background image data; perform Gaussian smoothing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; perform curve fitting on the difference frequency mapping relationship to obtain a fitting curve, and determine the difference value corresponding to the maximum value that matches the preset conditions in the fitting curve as the adaptive threshold.

[0118] Furthermore, the processing module is specifically used to determine the average grayscale value of multiple frames of foreground image data based on the difference in the foreground difference matrix, and generate the average grayscale signal matrix based on the average grayscale value; count the two-dimensional frequencies in the average grayscale signal matrix to generate a discrete distribution mapping relationship, and the discrete distribution mapping relationship includes the relationship between grayscale value and frequency; perform Gaussian filtering on the discrete distribution mapping relationship to obtain the image data to be detected composed of grayscale values ​​higher than a preset high-frequency signal range.

[0119] Furthermore, the device further comprises:

[0120] An updating module is configured to determine that the background image data completes delayed verification when a target pixel in the background image data is within a gas area and a grayscale value of the target pixel is greater than a preset high-frequency signal range; and to update the background image data when a target pixel in the background image data is not within a gas area and / or a grayscale value of the target pixel is less than or equal to a preset high-frequency signal range; wherein the gas area is determined based on the foreground difference matrix.

[0121] Furthermore, the device further comprises:

[0122] The correction module is used to obtain the gain correction coefficient and the bias correction coefficient of the infrared image, and construct a correction mapping relationship based on the gain correction coefficient, the bias correction coefficient and the mapping relationship between grayscale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

[0123] An embodiment of the present application provides a gas leak detection device based on infrared image processing. Compared with the prior art, the embodiment of the present application constructs a foreground difference matrix based on foreground image data, background image data and an adaptive threshold when a suspicious gas object is determined to exist based on foreground image pixels of a target area to be detected, wherein the adaptive threshold is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship; constructs an average grayscale signal matrix of the foreground difference matrix, and filters the average grayscale signal matrix to obtain image data to be detected; when the background image data passes a delay check, weighted fusion is performed on the image data to be detected and the visible light image data to generate gas leak detection result image data, thereby increasing the accuracy of distinguishing between the foreground and the background, highlighting the detection object of the foreground gas leakage, and improving the recognition ability of the gas signal, thereby meeting the demand for accurate gas leak detection in a complex background environment.

[0124] According to one embodiment of the present application, a storage medium is provided, wherein the storage medium stores at least one executable instruction. The computer-executable instruction can execute the gas leakage detection method based on infrared image processing in any of the above method embodiments.

[0125] Figure 5 A schematic diagram of the structure of a terminal provided according to an embodiment of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the terminal.

[0126] like Figure 5 As shown, the terminal may include: a processor 302 , a communications interface 304 , a memory 306 , and a communication bus 308 .

[0127] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via a communication bus 308 .

[0128] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0129] The processor 302 is configured to execute the program 310 , and specifically to execute the relevant steps in the above-mentioned embodiment of the gas leakage detection method based on infrared image processing.

[0130] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0131] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0132] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, or may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0133] The program 310 may be specifically configured to cause the processor 302 to perform the following operations:

[0134] When a suspicious gas object is determined to exist based on foreground image pixels of the target area to be detected, a foreground difference matrix is ​​constructed based on the foreground image data, the background image data, and an adaptive threshold value, wherein the adaptive threshold value is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship;

[0135] Constructing an average grayscale signal matrix of the foreground difference matrix, and performing filtering processing on the average grayscale signal matrix to obtain image data to be detected;

[0136] When the background image data passes the delay check, the image data to be detected and the visible light image data are weightedly fused to generate gas leakage detection result image data.

[0137] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0138] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A gas leak detection method based on infrared image processing, characterized in that: include: When a suspicious gas object is determined to exist based on foreground image pixels of the target area to be detected, a foreground difference matrix is ​​constructed based on the foreground image data, the background image data, and an adaptive threshold value, wherein the adaptive threshold value is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship; Constructing an average grayscale signal matrix of the foreground difference matrix, and performing filtering processing on the average grayscale signal matrix to obtain image data to be detected; When the background image data passes the delay check, the image data to be detected and the visible light image data are weightedly fused to generate gas leakage detection result image data; The foreground difference matrix constructed based on the foreground image data, the background image data and the adaptive threshold comprises: Obtaining the adaptive threshold; Determining pixel differences between multiple frames of foreground image data and background image data, and comparing the pixel differences with the adaptive threshold; configuring pixel difference values ​​less than or equal to the adaptive threshold as zero values, and constructing the foreground difference matrix based on pixel difference values ​​greater than the adaptive threshold and the zero value; The obtaining of the adaptive threshold comprises: Obtaining an initial threshold, and constructing an initial foreground difference matrix based on the initial threshold and the foreground image data and the background image data; Performing Gaussian smoothing on the initial foreground difference matrix to obtain a difference frequency mapping relationship; Curve fitting is performed on the differential frequency mapping relationship to obtain a fitting curve, and a differential value corresponding to a maximum value matching a preset condition in the fitting curve is determined as an adaptive threshold.

2. The method according to claim 1, characterized in that Before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: Determine foreground image pixels and background image pixels obtained by performing infrared image acquisition on the target area based on the foreground area radius and the background update factor, wherein the foreground image pixels are used to form the foreground image data, and the background image pixels are used to form the background image data; When the number of pixels of the foreground image is greater than a first preset threshold, and the area of ​​the foreground image pixels is greater than a second preset threshold, it is determined that a suspicious gas object exists.

3. The method according to claim 1, characterized in that The step of constructing an average grayscale signal matrix of the foreground difference matrix and performing filtering on the average grayscale signal matrix to obtain image data to be detected includes: determining an average grayscale value of a plurality of frames of foreground image data based on the difference values ​​in the foreground difference matrix, and generating the average grayscale signal matrix based on the average grayscale value; Counting the two-dimensional frequencies in the average grayscale signal matrix to generate a discrete distribution mapping relationship, wherein the discrete distribution mapping relationship includes a relationship between grayscale values ​​and frequencies; Gaussian filtering is performed on the discrete distribution mapping relationship to obtain the image data to be detected composed of grayscale values ​​higher than a preset high-frequency signal range.

4. The method according to claim 1, wherein When the background image data passes the delay check, the image data to be detected is weighted-fused with the visible light image data to generate fused image data, and the method further includes: When the target pixel in the background image data is within the gas region and the grayscale value of the target pixel is within the preset high-frequency signal range, it is determined that the background image data completes the delay check; When the target pixel in the background image data is not within the gas region, and / or the grayscale value of the target pixel is not within the preset high-frequency signal range, updating the background image data; The gas region is determined based on the foreground difference matrix.

5. The method according to any one of claims 1 to 4, characterized in that Before constructing the foreground difference matrix based on the foreground image data, the background image data, and the adaptive threshold, the method further includes: A gain correction coefficient and a bias correction coefficient of the infrared image are obtained, and a correction mapping relationship is constructed based on the gain correction coefficient, the bias correction coefficient and the mapping relationship between grayscale and temperature, so as to correct the infrared image acquisition through the correction mapping relationship.

6. A gas leak detection device based on infrared image processing, characterized in that: include: a construction module for constructing a foreground difference matrix based on foreground image data, background image data, and an adaptive threshold when a suspicious gas object is determined to exist based on foreground image pixels in a target area to be detected, wherein the adaptive threshold is determined by a maximum value in a fitting curve based on a differential frequency mapping relationship; a processing module, configured to construct an average grayscale signal matrix of the foreground difference matrix, and perform filtering processing on the average grayscale signal matrix to obtain image data to be detected; A generating module, configured to perform weighted fusion of the image data to be detected and the visible light image data to generate gas leak detection result image data when the background image data passes the delay check; The construction module is specifically configured to obtain the adaptive threshold; determine a pixel difference between multiple frames of foreground image data and background image data, and compare the pixel difference with the adaptive threshold; configuring pixel difference values ​​less than or equal to the adaptive threshold as zero values, and constructing the foreground difference matrix based on pixel difference values ​​greater than the adaptive threshold and the zero value; The construction module is specifically used to obtain an initial threshold value, and construct an initial foreground difference matrix based on the initial threshold value and the foreground image data and the background image data; perform Gaussian smoothing on the initial foreground difference matrix to obtain a differential frequency mapping relationship; Curve fitting is performed on the differential frequency mapping relationship to obtain a fitting curve, and a differential value corresponding to a maximum value matching a preset condition in the fitting curve is determined as an adaptive threshold.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to claim 1.

Images