Infrared temperature field reconstruction method, imaging temperature measurement device and system, product and medium

Abstract

The application provides an infrared temperature field reconstruction method, an imaging temperature measuring device and system, a product and a medium. The infrared temperature field reconstruction method comprises the following steps: acquiring temperature data of an infrared image corresponding to a target field of view; acquiring depth data; correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the infrared image. The depth data contains the shooting distance of all imaging targets in the infrared image. The temperature of target objects at different distances in the infrared image is corrected by using the depth data to obtain the reconstructed temperature field data of the infrared image, so that the target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an infrared temperature field reconstruction method, an imaging temperature measurement device, an imaging temperature measurement system, a computer program product, and a computer-readable storage medium. Background Technology

[0002] Infrared thermal imaging uses photoelectric technology to detect infrared signals in a specific band that emit thermal radiation from an object, converting these signals into images and graphics that can be discerned by the human eye, and further calculating the temperature value. Currently, infrared thermal imaging is widely used in infrastructure construction, urban management, industrial production, traffic control, resource exploration, inspection and quarantine, and fire safety due to its advantages such as long operating range, strong concealment, strong penetration, all-weather operation, resistance to strong light interference, and ability to identify concealed targets.

[0003] However, infrared thermal imaging images can only calculate the temperature field of all images in the same image based on a certain target distance. It cannot effectively measure the distance between all images and the thermal imager, nor can it reflect the distance differences of the measured targets in the image due to their three-dimensional characteristics. As a result, the temperature field data is not accurate enough, and the distance of the measured targets needs to be corrected during the later analysis. Summary of the Invention

[0004] To address the existing technical problems, this application provides an infrared temperature field reconstruction method for obtaining an accurate temperature field, an imaging temperature measurement device, an imaging temperature measurement system, a computer program product, and a computer-readable storage medium.

[0005] Firstly, an infrared temperature field reconstruction method is provided, including:

[0006] Acquire temperature data from the infrared image corresponding to the target's field of view;

[0007] Obtain depth data;

[0008] The temperature data is corrected using the depth data to obtain the reconstructed temperature field data of the infrared image.

[0009] In a second aspect, an imaging temperature measurement device is provided, including a memory, a processor, and an infrared thermal imaging data acquisition module and a depth data acquisition module connected to the processor;

[0010] The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target's field of view.

[0011] The depth data acquisition module is used to acquire depth data;

[0012] The memory stores computer programs;

[0013] When the computer program is executed by the processor, it implements the infrared temperature field reconstruction method according to any embodiment of this application.

[0014] Thirdly, an imaging temperature measurement system is provided, including an imaging temperature measurement device and a terminal device that is communicatively connected to the imaging temperature measurement device;

[0015] The imaging temperature measurement device includes an infrared thermal imaging data acquisition module and a depth data acquisition module. The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target field of view and send it to the terminal device. The depth data acquisition module is used to acquire depth data and send it to the terminal device.

[0016] The terminal device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the infrared temperature field reconstruction method described in any embodiment of this application.

[0017] Fourthly, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the infrared temperature field reconstruction method described in any embodiment of this application.

[0018] Fifthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the infrared temperature field reconstruction method according to any embodiment of this application.

[0019] The infrared temperature field reconstruction method provided in the above embodiments obtains depth data from the temperature data of infrared images, and uses the depth data to correct the temperature data. The depth data includes the shooting distance of all imaging targets in the infrared image. The temperature difference of target objects at different distances in the infrared image is corrected using the depth data to obtain the reconstructed temperature field data of the infrared image, so that the target objects at different distances within the same field of view or different parts of the same target object can achieve accurate temperature measurement.

[0020] The imaging temperature measurement device, imaging temperature measurement system, computer program product, and computer-readable storage medium provided in the above embodiments belong to the same concept as the corresponding infrared temperature field reconstruction method embodiments, and thus have the same technical effects as the corresponding infrared temperature field reconstruction method embodiments, which will not be repeated here. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of one possible application scenario for the infrared temperature field reconstruction method.

[0022] Figure 2 This is a schematic diagram illustrating another possible application scenario for the infrared temperature field reconstruction method.

[0023] Figure 3 A schematic flowchart of an infrared temperature field reconstruction method in one embodiment.

[0024] Figure 4 This is a schematic diagram of the image taken at the first object distance for a sample measurement point.

[0025] Figure 5 The temperature calculation results were obtained using a non-actual object distance of 6m when the measured point was at an actual distance of 2.2 meters. It can be seen that... Figure 4 The color and temperature values ​​are different.

[0026] Figure 6 The temperature calculation results were obtained when the measured point was at an actual distance of 6 meters, using a distance of 6 meters. It can be seen that... Figure 4 The color and temperature values ​​are kept consistent.

[0027] Figure 7 This is a flowchart illustrating the infrared temperature field reconstruction method in another embodiment.

[0028] Figure 8 This is a schematic diagram of a dual-light camera in an example.

[0029] Figure 9 This is a flowchart illustrating an example of an infrared temperature field reconstruction method.

[0030] Figure 10 This is a timing logic diagram of an example infrared temperature field reconstruction method.

[0031] Figure 11 This is a timing logic diagram for an infrared temperature field reconstruction method in another example.

[0032] Figure 12 This is a timing logic diagram for an infrared temperature field reconstruction method in yet another example.

[0033] Figure 13 This is a timing logic diagram of the infrared temperature field reconstruction method in another example.

[0034] Figure 14 This is a schematic diagram of the structure of an imaging temperature measurement device in one embodiment.

[0035] Figure 15 This is a schematic diagram of the imaging temperature measurement system in one embodiment. Detailed Implementation

[0036] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] In the following description, the phrase "some embodiments" refers to a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0039] It should also be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "inner," "outer," "left," "right," and similar expressions used herein are for illustrative purposes only in conjunction with the embodiments in the accompanying drawings and do not represent the only possible implementations.

[0040] Please see Figure 1 This diagram illustrates optional application scenarios of the infrared temperature field reconstruction method provided in this application embodiment. The infrared temperature field reconstruction method is applied to the imaging temperature measurement device 20. For example, the imaging temperature measurement device 20 is designed as a handheld infrared thermal imager used in applications such as power inspection, intelligent computer room monitoring, and power distribution room monitoring to capture infrared images of target devices in real time for temperature measurement and identification of high-temperature risks. The imaging temperature measurement device 20 includes a data acquisition module 21, a memory 22, and a processor 23 for correcting temperature data to reconstruct temperature field data. The imaging temperature measurement device 20 can be loaded with a computer program that executes the infrared temperature field reconstruction method provided in this application embodiment. During the real-time image acquisition process, the original temperature data of the infrared image is corrected by acquiring depth data to obtain reconstructed temperature field data that can eliminate the temperature differences of the imaging target caused by the difference in object distance, and obtain the accurate temperature values ​​of each imaging target in the infrared image.

[0041] Optional, please refer to Figure 2The infrared temperature field reconstruction method can be applied to the terminal device 30 that is connected to the imaging temperature measurement device 20'. For example, the imaging temperature measurement device 20' is designed as a handheld infrared thermal imager for applications such as power inspection, intelligent computer room monitoring, and power distribution room monitoring, which captures infrared images of target devices in real time to measure temperature and identify high temperature risks. The terminal device 30 can be a computer device, mobile terminal, cloud terminal, or other intelligent device with storage and computing functions that is connected to the imaging temperature measurement device 20'. Terminal device 30 can be loaded with a computer program that executes the infrared temperature field reconstruction method provided in the embodiments of this application. During the real-time image acquisition process of imaging temperature measuring device 20', the temperature data and depth data of the infrared image are acquired and sent to terminal device 30. Terminal device 30 obtains the depth data to correct the original temperature data of the infrared image, thereby obtaining reconstructed temperature field data that can eliminate the temperature difference of the imaging target caused by the difference in object distance and obtain the accurate temperature value of each imaging target in the infrared image. The reconstructed temperature field data can be returned to imaging temperature measuring device 20' for the temperature display of the imaging target in the real-time acquired image, or it can be used by terminal device 30 on its local display interface for the temperature display of the imaging target in the real-time acquired image of imaging temperature measuring device 20'.

[0042] It should be noted that in the following embodiments provided in this application, the infrared temperature field reconstruction method is mainly used as an example of its application in an imaging temperature measurement device (i.e., the imaging temperature measurement device is the main execution subject), but this is not intended to limit the present invention.

[0043] Please see Figure 3 An infrared temperature field reconstruction method provided in one embodiment includes the following steps:

[0044] S101, acquire the temperature data of the infrared image corresponding to the target's field of view.

[0045] S103, acquire depth data.

[0046] S105, the temperature data is corrected using the depth data to obtain the reconstructed temperature field data of the infrared image.

[0047] The target field of view refers to various scenes that require infrared image capture. Obtaining the temperature data of the infrared image corresponding to the target field of view can refer to the infrared image acquisition module simultaneously saving the temperature data required for infrared image imaging; or it can refer to analyzing the infrared image and extracting the temperature data used for infrared image imaging. For ease of understanding and description, the temperature data of the infrared image is represented by a temperature dataset T. mn express;

[0048] T p =(x p ,w p ,tp ),T p ∈T mn ;

[0049] Where, x p ,w p Let p be the coordinates of point p, and t be the coordinates of point p. p Temperature data at point p; temperature dataset T mn It can be, but is not limited to, data forms such as sequences, vectors, and arrays.

[0050] Depth data refers to data contained in infrared images, such as the relative position and distance between the imaging target and the capturing device. Acquiring depth data can involve simultaneously acquiring depth data using a depth data acquisition device while simultaneously capturing infrared images of the target's field of view; alternatively, it can be done by simultaneously acquiring binocular image data of the target's field of view and then calculating the depth information of objects in the scene using the visual differences between the images. For ease of understanding and description, depth data will be represented using a depth dataset D. ij express:

[0051] d k =(u k ,v k ,l k ),d k ∈D ij ;

[0052] Among them, u k ,v k Let k be the coordinates, and l be the coordinates of point k. k Let k be the depth value, i.e., the object distance.

[0053] Deep dataset D ij The data can be represented in various forms, including but not limited to sequences, vectors, and arrays. Depth data acquisition devices can include, but are not limited to, depth sensors, radar devices, and laser scanning devices. Types of depth sensors include structured light / coded light, stereo vision, Time-of-Flight (TOF), and lidar.

[0054] Infrared images typically calculate the temperature of all images of targets within an infrared image based on a fixed target distance. The relative distances between different targets and the imaging device within the same target field of view do not affect the image display of the target in the infrared image. In other words, the same target will appear consistently in infrared images captured at different distances relative to the imaging device. (See also...) Figure 4 and Figure 5This refers to the temperature and color differences exhibited by the same imaging target when photographed at the same object distance using different distance parameters. Therefore, temperature calculations using distance parameters other than the actual object distance will yield different results compared to calculations using the actual object distance; the results obtained using the actual object distance parameters are more accurate. Depth data, including the shooting distances of all imaging targets in the infrared image, is used to correct the temperature data. By using the depth data of each imaging target to correct its individual temperature, the reconstructed temperature field data of the infrared image is obtained. This ensures that the temperature of the same target object can be calculated using the actual object distance parameters under any circumstances, guaranteeing consistent temperature data across different shooting distances. Please refer to [link to relevant documentation]. Figure 4 and Figure 6 When the same imaging target is photographed at different object distances, the temperature of the imaging target in the infrared image remains consistent (within the error range).

[0055] The infrared temperature field reconstruction method provided in the above embodiments obtains depth data from the temperature data of infrared images, and uses the depth data to correct the temperature data. The depth data includes the shooting distance of all imaging targets in the infrared image. The temperature of target objects at different distances in the infrared image is corrected using the depth data to obtain the reconstructed temperature field data of the infrared image, so that the target objects at different distances within the same field of view or different parts of the same target object can achieve accurate temperature measurement.

[0056] In some embodiments, please refer to Figure 7 Before step S105, the following steps are included:

[0057] S104, the depth data is registered to obtain registered depth data.

[0058] Temperature data and depth data from infrared images are acquired at different steps within the target's field of view. Inconsistencies between the devices used to acquire temperature and depth data lead to inconsistencies in the target content. By registering the depth and temperature data, the descriptions of the target in terms of position and size can be made consistent between the two data. There is a one-to-one data correlation between the depth and temperature data, which helps to reduce the amount of computation required for correction.

[0059] In some embodiments, step S104 includes:

[0060] Based on the homography matrix obtained by calibrating the dual-light scene shooting module, a projective transformation is performed on the depth data;

[0061] The depth data after projective transformation is resized to obtain the registration depth data.

[0062] A dual-light scene capture module refers to a capture module capable of acquiring a homography matrix, based on calibration, that can be converted between dual-light scene data. The combination of dual-light scene data can be a combination of at least one of visible light image data and depth data, and at least one of infrared image data and temperature data. In an optional specific example, a dual-light scene capture module refers to an image capture module that simultaneously acquires infrared and visible light images for a target field of view. For example, it can be a dual-light camera formed by integrating an infrared image sensor and a visible light image sensor, or it can be a combined image capture module formed by assembling an infrared camera that independently captures infrared images and a visible light camera that captures visible light images. The dual-light scene capture module can capture infrared and visible light images based on the same image capture command. For example, the user can operate a button to activate a certain shooting mode on the imaging temperature measurement device, or touch a button to start shooting to form an image capture command, thereby completing the synchronous acquisition of infrared and visible light images for the same target field of view.

[0063] In this embodiment, the dual-light scene shooting module is determined based on the product form of the imaging temperature measurement device using the infrared temperature field reconstruction method described in this application embodiment. For example, if the imaging temperature measurement device is a handheld infrared thermal imager, and the handheld infrared thermal imager includes a dual-light camera formed by integrating an infrared image sensor and a visible light image sensor, then the homography matrix refers to the calibration obtained for the dual-light camera of the handheld infrared thermal imager. The calibration can be completed before the handheld infrared thermal imager leaves the factory, or it can be completed by the user in the camera calibration mode after the handheld infrared thermal imager leaves the factory by setting the camera calibration mode on the handheld infrared thermal imager. It should be noted that calibration completed before leaving the factory and calibration completed by the user in the camera calibration mode after leaving the factory can be included simultaneously, thereby providing the user with the option to choose whether to use the camera calibration mode to update the homography matrix during use according to actual needs.

[0064] Projective transformation is a mathematical transformation that defines a one-to-one correspondence between two straight lines or two planes through the product of a finite number of central projections. This transformation preserves the homography ratio of three collinear points and the cross-ratio of four collinear points. Therefore, various transformations, including orthogonal transformations, similarity transformations, and affine transformations, can be considered special cases of projective transformation. Based on the homography matrix obtained from calibrating a dual-light scene imaging module, the coordinates of the depth data are projectively transformed using this matrix. This results in a depth dataset registered with the infrared image. The dimensions of the depth data are then adjusted to match the dimensions of the temperature data, resulting in depth data registered with and consistent with the temperature data.

[0065] In some embodiments, obtaining the homography matrix includes:

[0066] The dual-light scene shooting module captures dual-light scene data in the calibrated area.

[0067] Based on the feature point imaging information of the preset feature points in the calibration region contained in the dual-light scene data, the homography matrix is ​​calculated.

[0068] The calibration operation for obtaining the homography matrix by calibrating the dual-light scene imaging module includes capturing dual-light scene data in the calibration area using the dual-light scene imaging module, and calculating the homography matrix based on the feature point imaging information of preset feature points in the calibration area contained in the dual-light scene data. The dual-light scene data can refer to infrared images and visible light images; it can also refer to related image data that can express equivalent image information to infrared images, such as temperature data required for infrared image imaging, and related image data that can express equivalent image information to visible light images, such as depth data acquired simultaneously with visible light images.

[0069] In an optional example, a set of calibration images is captured on the calibration drawing using a dual-light scene capture module, and the homography matrix H is calculated based on eight feature points on the calibration images. 33 As shown in Formula 1 below:

[0070]

[0071] In the above embodiments, for each dual-light scene shooting module, the homography matrix obtained through calibration has a one-to-one correspondence. The dual-light scene shooting module collects dual-light scene data of the calibration area containing feature point imaging information of preset feature points. The homography matrix is ​​calculated through the feature point imaging information, which simplifies the calculation method of the homography matrix and improves the calibration accuracy.

[0072] In some embodiments, the dual-light scene shooting module includes an RGB-D data acquisition module and an infrared thermal imaging data acquisition module. The dual-light scene data includes an infrared image of the target field of view and the temperature data corresponding to the infrared image, a visible light image and the depth data corresponding to the visible light image.

[0073] The homography matrix is ​​calculated based on the feature point imaging information of preset feature points in the calibration region contained in the dual-light scene data, including at least one of the following:

[0074] Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and the visible light image in the dual-light scene data, the homography matrix is ​​calculated.

[0075] Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and depth data in the dual-light scene data, the homography matrix is ​​calculated.

[0076] Based on the temperature data in the dual-light scene data and the feature point imaging information of the preset feature points in the calibration area contained in the visible light image, the homography matrix is ​​calculated.

[0077] Based on the feature point imaging information of preset feature points in the calibration region contained in the temperature data and depth data in the dual-light scene data, the homography matrix is ​​calculated.

[0078] An RGB-D data acquisition module is an image acquisition module that integrates a traditional RGB image sensor and a depth sensor, such as a TOF camera (Time-Of-Light, depth imaging camera). An RGB-D data acquisition module can directly obtain depth information of the imaging scene while acquiring RGB images. Therefore, depth data can express the same image information as visible light images. The dual-light scene data used to calculate the homography matrix can be a visible light image and an infrared image or temperature data, or depth data and an infrared image or temperature data. For ease of understanding and description, the infrared image is represented using the image dataset Y. mn express:

[0079] Y p =(x p ,w p ,y p ),Y p ∈Y mn ;

[0080] Where, x p ,w p Let p be the coordinates, y p Let p be the gray value of point p. Similarly, the image dataset Y... mn It can be, but is not limited to, data formats such as sequences, vectors, and arrays. Image dataset Y mn Data and temperature dataset T mn The data in the data have a one-to-one correspondence.

[0081] The visible light image is processed using the visible light color dataset C. ij express:

[0082] c k =(u k ,v k ,r k ,g k b k ),c k ∈C ij;

[0083] Among them, u k ,v k Let k be the coordinates, and r be the coordinates of point k. k ,g k b k Let K be the color value of the visible light color dataset. ij It can be, but is not limited to, data formats such as sequences, vectors, and arrays. Visible light color dataset C ij Data and Deep Dataset D ij The data in the data have a one-to-one correspondence.

[0084] In the above embodiments, the visible light image and depth data in the dual-light scene shooting module are synchronously acquired by the RGB-D data acquisition module. In the process of calculating the homography matrix using dual-light scene data, the depth data and the visible light image are interchangeable. In the embodiment that uses a combination of depth data and temperature data to calculate the homography matrix, the depth data is registered using the homography matrix, and then the temperature data is corrected by registering the depth data. Thus, the process of correcting the temperature data only requires the use of depth data and temperature data in the calculation.

[0085] In some embodiments, the dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module, wherein the dual-light scene data includes an infrared image of the target field of view and corresponding temperature and depth data.

[0086] The homography matrix is ​​calculated based on the feature point imaging information of preset feature points in the calibration region contained in the dual-light scene data, including at least one of the following:

[0087] Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and depth data in the dual-light scene data, the homography matrix is ​​calculated.

[0088] Based on the feature point imaging information of preset feature points in the calibration region contained in the temperature data and depth data in the dual-light scene data, the homography matrix is ​​calculated.

[0089] In this embodiment, depth data is acquired using an independent depth sensor. The dual-light scene shooting module may also include an independent visible light image acquisition module to acquire visible light images, or it may not acquire visible light images itself. That is, the dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module. The infrared thermal imaging data acquisition module acquires infrared images of the target field of view and the corresponding temperature data, while the depth sensor acquires the corresponding depth data. In this case, the dual-light scene data used to calculate the homography matrix can be depth data and infrared images, or depth data and temperature data. Visible light images may not be included in the process of correcting the temperature data to obtain reconstructed temperature field data, or in the process of applying the reconstructed temperature field data for display.

[0090] Optionally, the dual-light scene shooting module may not capture visible light images itself, but instead communicate with other terminal devices that have visible light image acquisition capabilities to acquire visible light images in real time. This is used to correct temperature data to obtain reconstructed temperature field data, and then to display the reconstructed temperature field data. In this case, it will further include registering the visible light image using a homography matrix. The registration process can be the same as the registration process for depth data, and will not be described in detail here.

[0091] In some embodiments, adjusting the size of the projective transformed depth data to obtain registered depth data includes:

[0092] Based on the infrared image in the dual-light scene data, the projective transformed depth data is cropped to obtain size-matched registration depth data.

[0093] A projective transformation is performed on the depth data to obtain depth data registered with the infrared image. The projective transformed depth data is then cropped with the infrared image as a reference to obtain registered depth data that is the same size as the infrared image and is registered.

[0094] In this embodiment, the registration of depth data mainly includes performing projective transformation on the depth data using a homography matrix and cropping the projective transformed depth data according to the infrared image. The homography matrix only needs to be obtained once by the imaging temperature measurement device. During the use of the imaging temperature measurement device, in the process of reconstructing the temperature field based on the temperature data of the real-time infrared image, the homography matrix can be directly used to calculate the projective transformation, which helps to reduce the amount of calculation for data registration and facilitates the improvement of the real-time performance of the infrared temperature field reconstruction process.

[0095] Optionally, in other embodiments, the step of adjusting the size of the projective transformed depth data to obtain registered depth data includes:

[0096] The depth data is cropped based on the temperature data in the dual-light scene data to obtain size-matched registration depth data.

[0097] A projective transformation is performed on the depth data to obtain depth data registered with the temperature data. The projectively transformed depth data is then cropped using the temperature data as a reference to obtain registered depth data that is the same size as the temperature data. The projectively transformed depth data can be cropped using either the infrared image or the temperature data as a reference to obtain size-matched registered depth data. Thus, the temperature field reconstruction process only requires the depth and temperature data for calculation.

[0098] In some embodiments, step S105 includes:

[0099] For the temperature data value of the target point, the temperature correction function is used to correct it according to the depth data value of the target point in the registration depth data to obtain the corrected temperature data value of the target point;

[0100] Based on the corrected temperature data values ​​of the target points constituting each target object in the infrared image, the reconstructed temperature field data of the infrared image is obtained.

[0101] The registration depth data and temperature data are in one-to-one correspondence, and the target point can be any pixel in the infrared image. In this embodiment, the temperature correction function can be obtained using known data statistical methods under the guidance of the technical concept of using registration depth data for correction proposed in this application. As long as the temperature data of imaging targets at different distances can be corrected to a certain extent using their corresponding distance values, the accuracy of the reconstructed temperature field data can be improved to a certain extent. This application does not impose any restrictions here.

[0102] In the above embodiments, for the temperature data value of each target point in the image, the corrected temperature data value is calculated using the depth data value corresponding to the target point through a set temperature correction function, which simplifies the correction calculation.

[0103] Optionally, in a specific example, the temperature correction function is as follows: Formula 2:

[0104]

[0105] Among them, t p This represents the temperature data value at target point p, l” p This represents the depth data value of the target point p, where A1, B1, E1, A0, B0, and E0 represent correction coefficients, and f_t p This represents the corrected depth data value for the target point p.

[0106] Temperature data values ​​can be the raw temperature data required for infrared image acquisition by the infrared sensor, or temperature data corrected for emissivity, atmospheric temperature, reflectance, and atmospheric transmittance. The depth data value l” for target point p. p It is the shooting distance of the target in the infrared image.

[0107] In the above embodiment, the depth data value l” of the target point p is used. p A temperature correction function is generated to compensate for the temperature data values. The correction coefficients A1, B1, E1, A0, B0, and E0 can be calibrated using test data, which can improve the accuracy of the reconstructed temperature field data.

[0108] In some embodiments, the infrared temperature field reconstruction method further includes:

[0109] Acquire the infrared image corresponding to the target's field of view;

[0110] The reconstructed temperature field data is applied to the infrared image for display.

[0111] In this embodiment, while acquiring the temperature data of the infrared image of the target field of view, the infrared image of the target field of view is also acquired. After the temperature data is corrected using the depth data to obtain the reconstructed temperature field data, the reconstructed temperature field data can be applied to the infrared image for display. The reconstructed temperature field data can be used to display a more accurate temperature value of each imaging target in the infrared image.

[0112] In some embodiments, the infrared temperature field reconstruction method further includes:

[0113] Acquire the visible light image corresponding to the target field of view;

[0114] The visible light image is registered to obtain a registered visible light image;

[0115] The reconstructed temperature field data is applied to the registered visible light image for display; and / or the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display.

[0116] In this embodiment, while acquiring the temperature data of the infrared image of the target field of view, the embodiment also includes acquiring the visible light image of the target field of view. After correcting the temperature data using depth data to obtain reconstructed temperature field data, the reconstructed temperature field data can be applied to the visible light image for display. The reconstructed temperature field data can be used to display a more accurate temperature value of each imaging target in the visible light image. Alternatively, the reconstructed temperature field data can be applied to a dual-light fusion image of the infrared image and the registered visible light image for display. The reconstructed temperature field data can be used to display a more accurate temperature value of each imaging target in the dual-light fusion image.

[0117] To gain a more comprehensive understanding of the infrared temperature field reconstruction method provided in the embodiments of this application, please refer to [the relevant documentation / reference]. Figure 8 and Figure 9 Taking the application of the infrared temperature field reconstruction method to a dual-light camera integrating a TOF camera 85 and an infrared thermal imager 80 as an example, the dual-light camera includes a TOF camera 85 as an RGB-D data acquisition module, an infrared thermal imaging data acquisition module 81, a storage module 82, a data processing module 83, and an image temperature display module 84. The TOF camera 85 is used to acquire visible light images and depth data; the infrared thermal imaging data acquisition module 81 is used to acquire infrared images and temperature data; the storage module 82 is used to store and cache data that needs to be temporarily stored during data processing; the data processing module 83 is used to execute the steps of the infrared temperature field reconstruction method described in the embodiments of this application; and the image temperature display module 84 is used to display the image and temperature data.

[0118] The infrared temperature field reconstruction method includes the following steps:

[0119] S11, the infrared thermal imaging data acquisition module acquires the temperature dataset and infrared image dataset of infrared images.

[0120] T p =(x p ,w p ,t p ), T p ∈T mn ;where x p w p Let p be the coordinates of point p, and t be the coordinates of point p. p T represents the temperature data at point P after corrections for emissivity, atmospheric temperature, reflectance, and atmospheric transmittance. mn This represents a temperature dataset.

[0121] Y p =(x p ,w p ,y p ), Y p ∈Y mn ;where x p w p Let y be the coordinates of the point. p Y represents the grayscale value of each point. mn This represents an infrared image dataset.

[0122] S12, the RGB-D data acquisition module acquires RGB-D datasets and establishes visible light image datasets and depth datasets.

[0123] c k =(u k ,vk ,r k ,g k b k ),c k ∈C ij ;where u k ,v k Let k be the coordinates, and r be the coordinates of point k. k ,g k b k C represents the color value of that point. ij This represents a visible light image dataset.

[0124] d k =(u k ,v k ,l k ),d k ∈D ij ; where u k ,v k Let k be the coordinates, and l be the coordinates of point k. k D represents the depth value at that point, i.e., the object distance. ij This represents a deep dataset.

[0125] S13, obtain the homography matrix obtained using calibration.

[0126] Calibration can be completed before the dual-light camera leaves the factory, or it can be performed by the user in the camera's calibration mode after the camera leaves the factory. The dual-light camera captures infrared and visible light images of the calibration drawings. Based on eight feature points on the calibration drawings, the homography matrix H is calculated. 33 .

[0127] S14: Perform projective transformation on the coordinates of the visible light image dataset and the depth dataset respectively to obtain the visible light image dataset and the corresponding depth image that are registered and aligned with the infrared image.

[0128] (1) Perform a projective transformation on the visible light image to obtain a visible light dataset C that is registered with the infrared image. IJ 'As shown below:

[0129] c′ K1 =(u′ K1 ,v′ K1 ,r′ K1 ,g′ K1 ,b′ K1 ), c′ K1 ∈C IJ ';

[0130] Where, u′ K1 , v′ K1 Let K1 be the coordinates of point K1 after the projective transformation of point k, as shown in Formula 3 below;

[0131]

[0132] r′ K ,g′ K ,b′ K Let r' be the color value of the mapped point k. If K1 has no mapped point, then r' K ,g′ K ,b′ K Assign 0.

[0133] (2) Perform the same projective transformation on the depth dataset to obtain the depth dataset D registered with the infrared image. IJ 'As shown below:

[0134] d′ K1 =(u′ K1 ,v′ K1 ,l′ K1 ), d′ K1 ∈D IJ ';

[0135] Where, u′ K1 ,v′ K1 Let K1 be the coordinates of point K1 after the projective transformation of point k, as shown in Formula 4 below:

[0136]

[0137] l′ K1 Let l' be the depth value of the mapped point k. If K1 has no mapped point, then l'... K1 Assign 0.

[0138] S15, based on the infrared image, crop the visible light image dataset and depth dataset to obtain a visible light dataset and depth dataset that are the same size as the infrared image and registered.

[0139] (1) For the visible light image dataset C registered with infrared images IJ 'A new visible light dataset C is obtained by cropping and recombining the infrared images according to their dimensions.' mn '.

[0140] (2) For the depth dataset D registered with infrared images IJ 'The depth dataset D is obtained by cropping and recombining the infrared images according to their dimensions.' mn '.

[0141] S16. Through the aforementioned steps S11 to S15, each dataset with a one-to-one correspondence is obtained.

[0142] Visible light dataset C' mn ', Deep dataset D' mn Temperature dataset Tmn and infrared image dataset Y mn .

[0143] S17, distance correction is applied to the temperature dataset to obtain accurate reconstructed temperature field data.

[0144] The temperature correction at point p is the same as in Formula 2 in the aforementioned embodiment, f_t p ∈F_T mn F_T mn This indicates the reconstruction of temperature field data.

[0145] S18, perform image rendering processing on the registered visible light dataset and infrared image dataset, and apply the reconstructed temperature field data to the visible light dataset, infrared image dataset, or the fused image data obtained by fusing the visible light dataset and infrared image dataset to the display interface.

[0146] This involves acquiring visible light images and depth data using a TOF camera. The depth data and visible light images are interchangeable. (See [link to relevant documentation]). Figures 10 to 13 These are different embodiments of the timing logic diagram for the infrared temperature field reconstruction method.

[0147] like Figure 10 As shown, the dual-light camera includes an RGB-D data acquisition module, an infrared thermal imaging data acquisition module, a storage module, a processing module, and a display module. The RGB-D data acquisition module acquires visible light image data and depth data, while the infrared thermal imaging data acquisition module acquires infrared image data and temperature data. The storage module stores the acquired visible light image data, depth data, infrared image data, and temperature data. The processing flow of the infrared temperature field reconstruction method includes:

[0148] S21. Homography matrix is ​​obtained by calibrating dual-light scene data, which combines visible light image data and infrared image data.

[0149] S22, the visible light image data is projectively transformed using the homography matrix to obtain visible light image data aligned with the infrared image.

[0150] S23, crop the visible light image data to obtain registered visible light image data with the same size as the infrared image.

[0151] S24. The homography matrix is ​​used to perform a projective transformation on the depth data to obtain depth data aligned with the infrared image.

[0152] S25, cropping depth data to obtain registration depth data consistent with the size of the infrared image.

[0153] S26. The temperature data is corrected using the registration depth data to obtain the reconstructed temperature field.

[0154] S27 renders the registered visible light image data and infrared image data, and displays them together with the reconstructed temperature field data on the device interface.

[0155] like Figure 11 As shown, with Figure 10 The main difference in the illustrated embodiment is that step S21 is replaced by step S31.

[0156] S31 uses dual-light scene data combining depth data and infrared image data to obtain a homography matrix through calibration.

[0157] like Figure 12 As shown, with Figure 10 The main difference in the illustrated embodiment is that step S21 is replaced by step S41.

[0158] S41, using dual-light scene data combining visible light image data and temperature data, the homography matrix is ​​obtained through calibration.

[0159] like Figure 13 As shown, with Figure 10 The main difference in the illustrated embodiment is that step S21 is replaced by step S51.

[0160] S51 uses dual-light scene data combining depth and temperature data to obtain a homography matrix through calibration.

[0161] for Figure 13 In the embodiment shown, which uses a combination of depth and temperature scene data to calculate the homography matrix, the depth data is registered using the homography matrix, and then the temperature data is corrected using the registered depth data. In this case, the registered depth data can be the result of calibration directly. Thus, the process of correcting the temperature data only requires the use of depth and temperature data in the calculation.

[0162] The infrared temperature field reconstruction method provided in this application registrations visible light and infrared images, then maps temperature and depth data together, uses depth data to correct temperature data, and reconstructs the temperature data. This eliminates temperature differences caused by using the same object distance calculation for objects at different depths of field, enabling accurate temperature measurement of target objects at different distances within the same field of view or different parts of the same target object. The reconstructed temperature field data can be applied to the temperature display of various imaging targets in two-dimensional or three-dimensional images to achieve more accurate temperature measurement.

[0163] In another aspect of the embodiments of this application, please refer to Figure 14The application also provides an imaging temperature measurement device 20, including a memory 22, a processor 23, and an infrared thermal imaging data acquisition module 212 and a depth data acquisition module 211 connected to the processor 23; the infrared thermal imaging data acquisition module 212 is used to acquire temperature data of the infrared image corresponding to the target field of view; the depth data acquisition module 211 is used to acquire depth data; the memory 22 stores a computer program; when the computer program is executed by the processor 23, it implements the infrared temperature field reconstruction method described in any embodiment of this application.

[0164] Optionally, the infrared thermal imaging data acquisition module 212 is further configured to acquire infrared images corresponding to the target's field of view. The infrared thermal imaging data acquisition module 212 may include an infrared sensor, which acquires infrared images while simultaneously storing the temperature data corresponding to the infrared image formation.

[0165] Optionally, the depth data acquisition module 211 includes one of the following: 1. The depth data acquisition module 211 includes a depth sensor, which is used to acquire depth data, and the reconstructed temperature field data is applied to the infrared image for display. 2. The depth data acquisition module 211 includes a depth imaging camera, which is used to acquire a visible light image of the target field of view and the corresponding depth data, and the reconstructed temperature field data is displayed in one of the following ways: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display. 3. The depth data acquisition module 211 includes a visible light sensor and a depth sensor. The depth sensor is used to acquire depth data, and the visible light sensor is used to acquire a visible light image of the target field of view. The display method of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to the dual-light fusion image of the infrared image and the registered visible light image for display.

[0166] In one example, the imaging temperature measurement device 20 is a handheld infrared thermal imager.

[0167] In another aspect of the embodiments of this application, please refer to Figure 15Furthermore, an imaging temperature measurement system is provided, including an imaging temperature measurement device 20' and a terminal device 30 communicatively connected to the imaging temperature measurement device 20'. The imaging temperature measurement device 20' includes an infrared thermal imaging data acquisition module 212 and a depth data acquisition module 211. The infrared thermal imaging data acquisition module 212 is used to acquire temperature data of an infrared image corresponding to the target's field of view and send it to the terminal device 30. The depth data acquisition module 211 is used to acquire depth data and send it to the terminal device 30. The terminal device 30 includes a memory 31 and a processor 32. The memory 31 stores a computer program, which, when executed by the processor 32, implements the infrared temperature field reconstruction method described in any embodiment of this application.

[0168] Optionally, the imaging temperature measurement device 20' further includes a display module 213; the imaging temperature measurement device 20' is also used to receive the reconstructed temperature field data returned by the terminal device 30, and display the reconstructed temperature field data on the display module 213 in one of the following display methods: applying the reconstructed temperature field data to the infrared image for display; applying the reconstructed temperature field data to the registered visible light image for display; applying the reconstructed temperature field data to a dual-light fusion image of the infrared image and the registered visible light image for display.

[0169] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the infrared temperature field reconstruction method described in any embodiment of this application.

[0170] In another aspect, this application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the processes of the aforementioned infrared temperature field reconstruction method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0171] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, infrared thermal imager, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0173] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An infrared temperature field reconstruction method, characterized by, include: Acquire temperature data from the infrared image corresponding to the target's field of view; Obtain depth data; The depth data is registered to obtain registered depth data; For the temperature data value of the target point, the depth data value of the target point in the registration depth data is corrected by a temperature correction function to obtain the corrected temperature data value of the target point; based on the corrected temperature data value of the target points constituting each target object in the infrared image, the reconstructed temperature field data of the infrared image is obtained. The temperature correction function is as follows: ； in, This represents the temperature data value at target point p. This represents the depth data value of the target point p. , , , , , This represents the correction factor. This represents the corrected temperature data value for the target point p.

2. The infrared temperature field reconstruction method as described in claim 1, characterized in that, The process of registering the depth data to obtain registered depth data includes: Based on the homography matrix obtained by calibrating the dual-light scene shooting module, a projective transformation is performed on the depth data; The depth data after projective transformation is resized to obtain the registration depth data.

3. The infrared temperature field reconstruction method as described in claim 2, characterized in that, To obtain the homography matrix, the following steps are required: The dual-light scene shooting module captures dual-light scene data in the calibrated area. Based on the feature point imaging information of the preset feature points in the calibration region contained in the dual-light scene data, the homography matrix is ​​calculated.

4. The infrared temperature field reconstruction method as described in claim 3, characterized in that, The dual-light scene shooting module includes an RGB-D data acquisition module and an infrared thermal imaging data acquisition module. The dual-light scene data includes an infrared image of the target field of view and the temperature data corresponding to the infrared image, a visible light image and the depth data corresponding to the visible light image. The homography matrix is ​​calculated based on the feature point imaging information of preset feature points in the calibration region contained in the dual-light scene data, including at least one of the following: Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and the visible light image in the dual-light scene data, the homography matrix is ​​calculated. Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and depth data in the dual-light scene data, the homography matrix is ​​calculated. Based on the temperature data in the dual-light scene data and the feature point imaging information of the preset feature points in the calibration area contained in the visible light image, the homography matrix is ​​calculated. Based on the feature point imaging information of preset feature points in the calibration region contained in the temperature data and depth data in the dual-light scene data, the homography matrix is ​​calculated.

5. The infrared temperature field reconstruction method as described in claim 3, characterized in that, The dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module. The dual-light scene data includes infrared images and temperature data of the target field of view, as well as depth data. The homography matrix is ​​calculated based on the feature point imaging information of preset feature points in the calibration region contained in the dual-light scene data, including at least one of the following: Based on the feature point imaging information of preset feature points in the calibration area contained in the infrared image and depth data in the dual-light scene data, the homography matrix is ​​calculated. Based on the feature point imaging information of preset feature points in the calibration region contained in the temperature data and depth data in the dual-light scene data, the homography matrix is ​​calculated.

6. The infrared temperature field reconstruction method as described in claim 2, characterized in that, The step of adjusting the size of the depth data after projective transformation to obtain registered depth data includes: Based on the infrared image in the dual-light scene data, the projective transformed depth data is cropped to obtain size-matched registration depth data.

7. The infrared temperature field reconstruction method as described in claim 2, characterized in that, The step of adjusting the size of the depth data after projective transformation to obtain registered depth data includes: The depth data is cropped based on the temperature data in the dual-light scene data to obtain size-matched registration depth data.

8. The infrared temperature field reconstruction method as described in claim 1, characterized in that, Also includes: Acquire the infrared image corresponding to the target's field of view; The reconstructed temperature field data is applied to the infrared image for display.

9. The infrared temperature field reconstruction method as described in claim 8, characterized in that, Also includes: Acquire the visible light image corresponding to the target field of view; The visible light image is registered to obtain a registered visible light image; The reconstructed temperature field data is applied to the registered visible light image for display; and / or the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display.

10. An imaging temperature measurement device, characterized in that, Includes a memory, a processor, and an infrared thermal imaging data acquisition module and a depth data acquisition module connected to the processor; The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target's field of view. The depth data acquisition module is used to acquire depth data; The memory stores computer programs; When the computer program is executed by the processor, it implements the infrared temperature field reconstruction method as described in any one of claims 1 to 9.

11. The imaging temperature measurement device as described in claim 10, characterized in that, The infrared thermal imaging data acquisition module is also used to acquire infrared images corresponding to the target's field of view; and / or, The depth data acquisition module includes one of the following: The depth data acquisition module includes a depth sensor, which is used to acquire depth data. The reconstructed temperature field data is applied to the infrared image for display. The depth data acquisition module includes a depth imaging camera, which is used to acquire visible light images of the target field of view and depth data corresponding to the visible light images. The reconstructed temperature field data can be displayed in one of the following ways: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; or the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display. The depth data acquisition module includes a visible light sensor and a depth sensor. The depth sensor is used to acquire depth data, and the visible light sensor is used to acquire a visible light image of the target field of view. The display method of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display.

12. The imaging temperature measurement device as described in claim 10, characterized in that, The imaging temperature measurement device is a handheld infrared thermal imager.

13. An imaging temperature measurement system, characterized in that, Includes an imaging temperature measurement device and a terminal device that is communicatively connected to the imaging temperature measurement device; The imaging temperature measurement device includes an infrared thermal imaging data acquisition module and a depth data acquisition module. The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target field of view and send it to the terminal device. The depth data acquisition module is used to acquire depth data and send it to the terminal device. The terminal device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the infrared temperature field reconstruction method as described in any one of claims 1 to 9.

14. The imaging temperature measurement system as described in claim 13, characterized in that, The imaging temperature measurement device also includes a display module; The imaging temperature measurement device is also used to receive the reconstructed temperature field data returned by the terminal device, and to display the reconstructed temperature field data on the display module in one of the following ways: applying the reconstructed temperature field data to the infrared image for display; applying the reconstructed temperature field data to the registered visible light image for display; or applying the reconstructed temperature field data to a dual-light fusion image of the infrared image and the registered visible light image for display.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the infrared temperature field reconstruction method as described in any one of claims 1 to 9.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the infrared temperature field reconstruction method as described in any one of claims 1 to 9.

Images