Solid waste abnormal temperature intelligent positioning method based on infrared image

Abstract

The invention relates to the field of image processing, in particular to a solid waste abnormal temperature intelligent positioning method based on an infrared image, and the method comprises the steps: firstly screening candidate hot spots through a temperature threshold value, and obtaining a synchronous visible light image and a historical temperature sequence of the candidate hot spots; then, for each pixel in the hot spot, based on static optical-thermodynamic coupling characteristics and dynamic time domain thermal inertia, calculating artifact confidence; and then, integrating the confidence coefficients of all pixels in the region to obtain an overall real risk score of the candidate hot spot. And finally, comparing the score with a preset threshold value so as to accurately judge and locate a real abnormal heat source. According to the method, multi-modal information is fused, real heat sources and artifacts can be effectively distinguished, and the false alarm rate is remarkably reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing. More particularly, the present application relates to an intelligent positioning method for abnormal temperature of solid waste based on infrared images. BACKGROUND

[0002] Solid waste yards, such as landfills, industrial waste accumulation areas, etc., are important infrastructures for the operation of modern society, but at the same time, they also contain huge safety hazards. Due to the complex composition of waste, containing a large amount of combustible materials, and the internal processes such as anaerobic fermentation and chemical reactions that generate heat, the internal temperature of the waste is elevated, which easily leads to spontaneous combustion or fire accidents. Such fires not only cause serious economic losses and environmental pollution, but also may threaten the safety of people around. Therefore, it is a core link in safety management to monitor the temperature of solid waste yards all-weather and high-precision, and to find potential fire hotspots in time.

[0003] Infrared thermal imaging technology has become the mainstream technical means for monitoring the temperature distribution of such scenes due to its advantages of non-contact, long-distance, and large-scale temperature measurement. Traditional monitoring systems usually set a fixed or simple dynamic temperature threshold on the infrared thermal image to identify high-temperature points. When the temperature of a certain area in the image exceeds the threshold, the system will trigger an alarm. However, this recognition method relying only on a single temperature dimension has many drawbacks in practical application, and the core problem is the high false alarm rate.

[0004] The main reason for false alarms is the widespread existence of reflection artifacts. In outdoor environments, sunlight is the biggest source of interference. When sunlight is incident on the surface of some objects in the yard, such as metal fragments, glass bottles, plastic films, or wet surfaces, specular reflection occurs. These reflected solar radiation is received by the infrared thermal imager, forming a bright spot with extremely high temperature on the image, whose reading is much higher than the ambient temperature, thus being mistakenly judged as an abnormal hot spot by the system. The temperature characteristics of this artifact are very similar to those of real early fire sources, and it is difficult to distinguish them only by temperature data, leading to frequent responses to false alarms by security personnel.

[0005] In summary, how to exclude the interference of reflection artifacts to achieve accurate heat source detection has become the research focus of the present application. SUMMARY

[0006] To solve the problem of how to exclude the interference of reflection artifacts to achieve accurate heat source detection, the present application proposes an intelligent positioning method for abnormal temperature of solid waste based on infrared images, which includes the following steps: a. Preliminarily screening out candidate abnormal hot spots on the original infrared image through a temperature threshold; b. For each of the candidate abnormal hot spots, obtain its data in the visible light image and the infrared image of the synchronous acquisition, and extract its historical temperature sequence; c. For each pixel point in the candidate abnormal hot spot, calculate a pixel-level artifact confidence based on its static optical-thermal coupling characteristics and dynamic time-domain thermal inertia characteristics; d. Based on the artifact confidence of all pixels in the candidate abnormal hot spot, calculate the overall real risk score of the candidate abnormal hot spot; e. Compare the real risk score with a preset alarm threshold to determine whether the candidate abnormal hot spot is a real abnormal heat source, and locate the hot spot determined to be a real abnormal heat source.

[0007] The application preliminarily screens out candidate abnormal hot spots by temperature, and only analyzes the candidate abnormal hot spots subsequently, thereby effectively reducing the calculation amount. Further, the artifact confidence is introduced to accurately distinguish real heat sources from false reflection hot spots, thereby greatly reducing the false alarm rate. Further, the difference characteristics of the reflection artifact and the real hot spot in the static optical and thermal characteristics are described to reflect the case that the pixel is a reflection artifact. Further, considering that some high reflection areas may also be real hot spots, the real hot spot corresponding to the high reflection area cannot be identified only by the static optical and thermal characteristics, and therefore the dynamic time-domain thermal inertia characteristics are introduced to identify the real hot spot corresponding to the reflection area.

[0008] Preferably, the candidate abnormal hot spots are preliminarily screened out by a temperature threshold, including: The original infrared image is subjected to binaryzation processing and connected domain analysis, and each independent connected domain is identified as a candidate abnormal hot spot.

[0009] The application can screen out candidate abnormal hot spots by simple binaryzation processing, and the implementation method is relatively simple and has higher implementation efficiency.

[0010] Preferably, the pixel-level artifact confidence is calculated, including: A static artifact confidence representing static characteristics is calculated; A dynamic time-domain thermal inertia index representing dynamic characteristics is calculated; The static artifact confidence and the dynamic time-domain thermal inertia index are used to calculate the pixel-level artifact confidence.

[0011] This method analyzes artifact confidence using two components: a static artifact confidence metric and a dynamic time-domain thermal inertia index. The static artifact confidence metric assesses the degree to which the spatial and physical properties of a hotspot at the current moment are consistent with an artifact; the dynamic time-domain thermal inertia index assesses the degree to which its temperature changes over time are consistent with an artifact. The combination of these two components forms a complete chain of evidence across space and time, greatly enhancing the reliability of the final judgment.

[0012] Preferably, the static artifact confidence is obtained by coupling a specular reflection potential index calculated based on visible light images and a local thermodynamic inconsistency calculated based on infrared images.

[0013] The present invention combines two typical characteristics of artifacts: the reflection characteristic in viewing angle and the sudden change of artifacts in temperature to jointly determine the possible situations of artifacts, thereby achieving more comprehensive artifact determination.

[0014] Preferably, the specular reflection potential index is calculated based on the brightness component and the saturation component after the visible light image is converted into the HSV color space.

[0015] The present invention takes into account the characteristics of high brightness and low color saturation of the reflective area, and uses this characteristic to accurately determine the situation as a reflective feature.

[0016] Preferably, the local thermodynamic inconsistency is obtained by calculating the difference between the temperature of a pixel point and the average temperature of a neighborhood excluding the pixel point itself.

[0017] Preferably, the dynamic time-domain thermal inertia index is calculated based on the statistical characteristics of historical temperature sequences.

[0018] Preferably, the dynamic time-domain thermal inertia index is obtained by calculating the ratio of the standard deviation of the historical temperature series to the absolute mean and mapping it through an exponential function.

[0019] Preferably, the overall true risk score of the candidate abnormal hotspot is the weighted average temperature of all pixels within the candidate abnormal hotspot, with the artifact suppression degree thereof being weighted, wherein the artifact suppression degree is derived from the artifact confidence.

[0020] The present invention has the following beneficial effects: The present invention initially screens candidate abnormal hotspots by temperature, and subsequently performs analysis based only on the candidate abnormal hotspots, effectively reducing the amount of calculation; Furthermore, the artifact confidence level is introduced to accurately distinguish between real heat sources and false reflection hotspots, significantly reducing the false alarm rate; Furthermore, by describing the difference between reflection artifacts and real hot spots in static optics and thermodynamics, we can preliminarily reflect the situation where the pixel is a reflection artifact. Furthermore, considering that some high-reflection areas may also be real hotspots, the real hotspots corresponding to the high-reflection areas cannot be identified only by static optics and thermodynamics. Therefore, the dynamic time-domain thermal inertia characteristics are introduced to identify the real hotspots corresponding to the reflective areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of the steps of the method for intelligently locating abnormal temperature of solid waste based on infrared images provided by an embodiment of the present invention; Figure 2 is a visible light image provided by an embodiment of the present invention; Figure 3 is the original infrared image provided by the embodiment of the present invention; Figure 4 The candidate abnormal hotspot area image provided by the embodiment of the present invention; Figure 5 This is a real abnormal heat source image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] See also Figure 1 , which shows a flowchart of the steps of the method for intelligently locating abnormal temperature of solid waste based on infrared images provided by an embodiment, the method includes the following steps: S1: Synchronously acquire the original infrared image and the spatially registered visible light image.

[0023] Specifically, a coaxial or near-axial imaging system including an infrared thermal imager and a visible light RGB camera is deployed to synchronously capture the original infrared image at the current moment and the visible light image spatially aligned with it. Figure 2 and Figure 3 The visible light image and the original infrared image are shown respectively. The visible light image shows that the image contains some highly reflective areas.

[0024] S2: Preliminarily screen out candidate abnormal hotspots based on the temperature threshold in the original infrared image.

[0025] Preferably, as an example, preliminary screening of candidate abnormal hot spots based on temperature thresholds on the original infrared image includes: First, the original infrared image at the current moment is binarized based on a preset temperature threshold, wherein the pixel values ​​above the preset temperature threshold are set to 1, and the pixel values ​​below the temperature threshold are set to 0.

[0026] Subsequently, the pixels with pixel value of 1 in the binary image are subjected to connected domain analysis, and each independent connected domain is identified as a candidate abnormal hotspot.

[0027] Figure 4 The candidate abnormal hotspot region image is shown, wherein the pixel region with white pixel value in the image represents the candidate abnormal hotspot region. It can be seen from the image that the segmented candidate abnormal hotspot region contains some reflection artifacts, and the interference of the reflection artifacts needs to be excluded.

[0028] S3: For each of the candidate abnormal hotspots, data thereof in the synchronously collected visible light image and infrared image are obtained, and a historical temperature sequence thereof is extracted.

[0029] It should be noted that, in order to obtain the corresponding pixel of the candidate abnormal hotspot in the visible light image, pixel-level spatial alignment needs to be performed on the original infrared image and the visible light image. Then the corresponding pixel of the candidate abnormal hotspot in the visible light image is obtained.

[0030] Preferably, as an example, for each of the candidate abnormal hotspots, data thereof in the synchronously collected visible light image and infrared image are obtained, and a historical temperature sequence thereof is extracted, including: Step 1, alignment processing.

[0031] Based on the Zhang Zhengyou calibration method, the internal and external parameters and distortion coefficients of the infrared thermal imager and the visible light camera based on the pinhole camera model are obtained respectively, and the rotation and translation matrixes between the two are calculated, so as to establish an accurate mapping relationship between the pixels. Based on the mapping relationship, the original infrared image and the visible light image at the current time are subjected to alignment processing.

[0032] Step 2, for each of the candidate abnormal hotspots, data thereof in the synchronously collected visible light image and infrared image are obtained.

[0033] Based on the alignment result, the region corresponding to the position of the candidate abnormal hotspot in the visible light image and the infrared image is obtained respectively.

[0034] Step 3, historical temperature data is obtained.

[0035] The infrared images at the previous preset first number of time points of the current time are obtained, a matching algorithm based on the intersection over union (IOU) of the bounding box or a more advanced target tracking algorithm is used to perform cross-frame tracking of the candidate abnormal hotspots on the infrared images at the previous preset first number of time points, based on the tracking result, the historical temperature value corresponding to each pixel in the candidate abnormal hotspot is extracted from the infrared images at the previous preset first number of time points, to form a historical temperature sequence. Exemplarily, the preset first number can be 20.

[0036] S4: For each pixel in the candidate abnormal hot spot, a pixel-level artifact confidence is calculated based on its static optical-thermal coupling characteristic and dynamic time-domain thermal inertia characteristic.

[0037] It should be noted that since there may be some reflection artifact interference in the extracted candidate abnormal hot spot, the following needs to be distinguished based on the difference between the reflection artifact and the real heat source in the instantaneous image and the difference in time sequence.

[0038] S40: Obtain a static optical-thermal coupling characteristic.

[0039] It should be noted that the real heat source and the reflection artifact have certain distinguishability on the instantaneous image, and the following will extract some distinguishability characteristics based on the distinguishability on the instantaneous image.

[0040] S400: Calculate a pixel-level mirror reflection potential index.

[0041] It should be noted that the mirror reflection region usually exhibits the characteristics of high brightness and low saturation, so the distinguishability characteristics can be extracted based on this feature.

[0042] Preferably, as an example, the pixel-level mirror reflection potential index is calculated, including: Convert the visible light image from the RGB color space to the HSV color space to obtain the brightness value and the saturation value of each pixel.

[0043]

[0044] Wherein, represents the pixel-level mirror reflection potential index of the jth pixel in the ith candidate abnormal hot spot, represents the saturation value of the jth pixel in the ith candidate abnormal hot spot, represents the brightness value of the jth pixel in the ith candidate abnormal hot spot, represents the maximum value of the brightness value of all pixels, used for normalization.

[0045] It can be understood that the mirror reflection region has the characteristics of high brightness and low saturation, so the larger the brightness of a pixel position and the lower the saturation, the greater the possibility of mirror reflection at the pixel position, and the greater the pixel-level mirror reflection potential index at the pixel position.

[0046] S401: Calculate local thermal inconsistency.

[0047] It should be noted that a real, internally generated heat source will diffuse heat to the surrounding medium, forming a thermal halo with a temperature gradient on the infrared image; while the sunspot artifact is a surface reflection of external energy, so its temperature boundary with the surrounding area will show great discontinuity.

[0048] Preferably, as an example, the local thermodynamic inconsistency is calculated, including:

[0049] wherein, Tij represents the temperature value of the jthpixel in the ithcandidate abnormal hotspot, Tij represents the mean value of the temperature values of all pixels in the preset neighborhood centered on the jthpixel in the ithcandidate abnormal hotspot, Tambient represents the ambient temperature at the current moment, LTIij represents the local thermodynamic inconsistency of the jthpixel in the ithcandidate abnormal hotspot, Zij represents a preset zero-prevention parameter for preventing the denominator from being 0, and an example is, 0.001.

[0050] It can be understood that when a high-temperature point is a sunspot artifact, its temperature will be much higher than that of its adjacent area, and thus the calculated local thermodynamic inconsistency is larger.

[0051] S402: Calculate the static artifact confidence based on the pixel-level specular reflection potential index and the local thermodynamic inconsistency.

[0052] Preferably, as an example, the static artifact confidence is calculated based on the pixel-level specular reflection potential index and the local thermodynamic inconsistency, including:

[0053] wherein, SACij represents the static artifact confidence of the jthpixel in the ithcandidate abnormal hotspot, LTIij represents the local thermodynamic inconsistency of the jthpixel in the ithcandidate abnormal hotspot, LN represents a linear normalization process.

[0054] It can be understood that the probability that the pixel in the candidate abnormal hotspot is a reflection artifact is jointly determined by the two combinations. The larger the value is, the greater the probability that the candidate abnormal hotspot is a reflection artifact is.

[0055] S41: Obtain a dynamic time-domain thermal inertia feature.

[0056] It should be noted that the static artifact confidence obtained in the above steps can effectively identify most of the instantaneous sunspot artifacts formed by specular reflection. However, if a highly reflective object is heated by the sun for a long time, it will also appear as a high reflectivity in the visible light image, and in the infrared image, there will be a significant temperature difference between the object and its surroundings due to its high temperature. Therefore, the sun-heated highly reflective object cannot be identified by the above features.

[0057] It should be further noted that, due to the specific heat capacity of the object, the temperature of the sun-heated object changes slowly in time sequence. The sun-heated highly reflective object can be identified based on this feature.

[0058] Preferably, as an example, the dynamic time-domain thermal inertia index is obtained, including:

[0059] wherein, represents the variance of all data in the historical temperature sequence of the jth pixel in the ith candidate abnormal hotspot, represents the mean of all data in the historical temperature sequence of the jth pixel in the ith candidate abnormal hotspot, represents an exponential function with a natural constant as the base, represents the dynamic time-domain thermal inertia index of the jth pixel in the ith candidate abnormal hotspot.

[0060] It can be understood that, reflects the relative fluctuation degree of heat at a pixel position. The larger the value, the greater the heat instability at the pixel position, and since the true heat does not fluctuate greatly in time sequence, the probability of the pixel position being a true heat is smaller, and therefore the dynamic time-domain thermal inertia index is smaller.

[0061] S42: Calculate the artifact confidence.

[0062] Preferably, as an example, the artifact confidence is calculated, including:

[0063] wherein, represents the artifact confidence of the jth pixel in the ith candidate abnormal hotspot.

[0064] It can be understood that, reflects the possibility of a pixel being an artifact in the instantaneous image. The larger the value, the greater the possibility of the pixel being an artifact, reflects the thermal inertia condition. The larger the value, the smaller the possibility of the pixel being an artifact.

[0065] S5: calculating an overall real risk score of the hot spot based on the artifact confidence of all pixels in the candidate abnormal hot spot.

[0066] It should be noted that the above process is to analyze each pixel in the candidate abnormal hot spot to determine whether each pixel is an artifact. The following needs to comprehensively judge each candidate abnormal hot spot as a real hot spot based on the situation of all pixels in the candidate abnormal hot spot.

[0067] Preferably, as an example, the overall real risk score of the hot spot is calculated based on the artifact confidence of all pixels in the candidate abnormal hot spot, including:

[0068] Wherein, represents the number of pixel points in the i-th candidate abnormal hot spot, represents the real risk score of the i-th candidate abnormal hot spot.

[0069] It can be understood that, reflects the possibility of a pixel point being a real heat, and the greater the value, the greater the possibility of the pixel point being a real heat. Using as a weight, the temperature of each pixel point is weighted and summed to obtain the real temperature excluding the interference of artifacts, that is, the overall real risk score.

[0070] S6: comparing the real risk score with a preset alarm threshold to determine whether the candidate abnormal hot spot is a real abnormal heat source, and positioning the hot spot determined to be a real abnormal heat source.

[0071] Preferably, as an example, the real risk score is compared with a preset alarm threshold to determine whether the candidate abnormal hot spot is a real abnormal heat source, and the hot spot determined to be a real abnormal heat source is positioned, including: The real risk score is compared with the preset alarm threshold, and the candidate abnormal hot spot with a real risk score greater than the preset alarm threshold is determined to be a real abnormal heat source. The location of the real abnormal heat source is marked.

[0072] Figure 5 The real abnormal heat source image is shown in FIG. 5. The area identified in the black rectangular frame in the figure is the real abnormal heat source. It can be seen from the image that the reflection artifact has been excluded, and the real abnormal heat source is obtained.

[0073] The above only describes the preferred embodiments of the present application and should not be used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. A solid waste abnormal temperature intelligent positioning method based on an infrared image, characterized in that, The method comprises: a. preliminary screening candidate abnormal hot spots on original infrared images by temperature threshold; b. for each of the candidate abnormal hot spots, obtaining its data in the synchronously collected visible light image and infrared image, and extracting its historical temperature sequence; c. for each pixel point in the candidate abnormal hot spot, calculating a pixel-level artifact confidence based on its static optical-thermal coupling characteristics and dynamic time-domain thermal inertia characteristics; d. calculating the overall real risk score of the candidate abnormal hot spot based on the artifact confidence of all pixels in the candidate abnormal hot spot; e. comparing the real risk score with a preset alarm threshold to determine whether the candidate abnormal hot spot is a real abnormal heat source, and positioning the hot spot determined as a real abnormal heat source.

2. The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 1, characterized in that, The preliminary screening of candidate abnormal hot spots by temperature threshold comprises: performing binaryzation processing on the original infrared image and performing connected domain analysis, and identifying each independent connected region as a candidate abnormal hot spot. 3.The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 1, characterized in that, The calculation of a pixel-level artifact confidence comprises: calculating a static artifact confidence representing static characteristics; calculating a dynamic time-domain thermal inertia index representing dynamic characteristics; calculating the pixel-level artifact confidence from the static artifact confidence and the dynamic time-domain thermal inertia index.

4. The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 3, characterized in that, The static artifact confidence is obtained by coupling a specular reflection potential index calculated based on a visible light image and a local thermodynamic inconsistency calculated based on an infrared image.

5. The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 4, characterized in that, The specular reflection potential index is calculated based on the brightness component and the saturation component after converting the visible light image to HSV color space. 6.The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 4, characterized in that, The local thermodynamic inconsistency is obtained by calculating the difference between the temperature of a pixel point and the average temperature in the neighborhood excluding itself.

7. The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 3, characterized in that, The dynamic time-domain thermal inertia index is calculated based on the statistical characteristics of the historical temperature sequence.

8. The method of claim 7, wherein, The dynamic time-domain thermal inertia index is obtained by calculating the ratio of the standard deviation to the absolute value mean of the historical temperature sequence, and mapping through an exponential function. 9.The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 1, wherein, The overall real risk score of the candidate abnormal hot spot is the weighted average temperature of all pixel points in the candidate abnormal hot spot, with the artifact suppression degree as the weight, wherein the artifact suppression degree is derived from the artifact confidence. 10.The infrared image-based solid waste abnormal temperature intelligent positioning method according to claim 1, characterized in that, Marking the position of the real abnormal heat source.

Images