Infrared temperature field reconstruction method, imaging temperature measurement device and system, product and medium
By acquiring the temperature data and depth data of the infrared image, the infrared image is corrected, which solves the problem of insufficient temperature field data in infrared thermal imaging, and realizes accurate temperature measurement at different distances and locations.
Patent Information
- Application Number
- CN202411294766.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-14
AI Technical Summary
Infrared thermal imaging images cannot effectively measure the distance between the imaging object and the thermal imager, resulting in inaccurate temperature field data and inability to reflect the distance differences caused by the three-dimensional three-dimensional characteristics in the image.
By acquiring the temperature data and depth data of the infrared image, using the depth data to correct the temperature data to obtain the reconstruction temperature field data, thereby eliminating the temperature difference caused by the difference in object distance and achieving accurate temperature measurement.
Accurate temperature measurement of target objects at different distances in the same field of view or different parts of the same target object is achieved, and the accuracy of infrared temperature field data is improved.
Smart Images

Figure CN119085859A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an infrared temperature field reconstruction method, an imaging temperature measurement device, an imaging temperature measurement system, a computer program product, and a computer-readable storage medium. Background Art
[0002] Infrared thermal imaging uses photoelectric technology to detect infrared signals of specific wavelengths of thermal radiation from objects, converts the signals into images and graphics that can be distinguished by human vision, and further calculates the temperature value. At present, infrared thermal imaging is widely used in infrastructure construction, urban management, industrial production, traffic control, resource exploration, inspection and quarantine, fire protection and security due to its advantages such as long working distance, strong concealment, strong penetration, all-weather working ability, resistance to strong light interference, and ability to identify hidden targets.
[0003] However, infrared thermal imaging images can only calculate the temperature field of all imaging objects in the same image based on a certain target distance. It cannot effectively measure the distance between all imaging objects and the thermal imager, nor can it reflect the distance difference of the measured targets in the image due to the three-dimensional characteristics. As a result, the temperature field data is not accurate enough, and the distance of the measured target needs to be corrected in the later analysis. Summary of the invention
[0004] In order to solve the existing technical problems, the present application provides an infrared temperature field reconstruction method for obtaining an accurate temperature field, an imaging temperature measurement device, an imaging temperature measurement system, a computer program product and a computer-readable storage medium.
[0005] In a first aspect, a method for reconstructing an infrared temperature field is provided, comprising:
[0006] Acquire temperature data of the infrared image corresponding to the target field of view;
[0007] Get depth data;
[0008] The temperature data is corrected by using the depth data to obtain reconstructed temperature field data of the infrared image.
[0009] In a second aspect, an imaging temperature measurement device is provided, including a memory, a processor, and an infrared thermal imaging data acquisition module and a depth data acquisition module connected to the processor;
[0010] The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target field of view;
[0011] The depth data acquisition module is used to collect depth data;
[0012] The memory stores a computer program;
[0013] When the computer program is executed by the processor, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0014] In a third aspect, an imaging temperature measurement system is provided, including an imaging temperature measurement device and a terminal device communicatively connected to the imaging temperature measurement device;
[0015] The imaging temperature measurement device includes an infrared thermal imaging data acquisition module and a depth data acquisition module. The infrared thermal imaging data acquisition module is used to acquire temperature data of an infrared image corresponding to the target field of view and send it to the terminal device. The depth data acquisition module is used to acquire depth data and send it to the terminal device.
[0016] The terminal device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0017] In a fourth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0018] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0019] The infrared temperature field reconstruction method provided in the above embodiment obtains depth data for the temperature data of the infrared image, and uses the depth data to correct the temperature data. The depth data includes the shooting distances of all imaging targets in the infrared image. The depth data is used to correct the temperature differences of target objects at different distances in the infrared image to obtain reconstructed temperature field data of the infrared image, so that target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement.
[0020] The imaging temperature measurement equipment, imaging temperature measurement system, computer program product and computer-readable storage medium provided in the above embodiments belong to the same concept as the corresponding infrared temperature field reconstruction method embodiments, and thus have the same technical effects as the corresponding infrared temperature field reconstruction method embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of an optional application scenario of the infrared temperature field reconstruction method.
[0022] Figure 2 A schematic diagram of another optional application scenario of the infrared temperature field reconstruction method.
[0023] Figure 3 A schematic flow chart of an infrared temperature field reconstruction method in one embodiment.
[0024] Figure 4 It is a schematic diagram of imaging of a measured point at a first object distance in an example.
[0025] Figure 5 The actual distance of the measured point is 2.2 meters, and the temperature is calculated using the non-actual object distance of 6 meters. Figure 4 The color and temperature values are different.
[0026] Figure 6 When the actual distance between the measured point and the measured point is 6 meters, the temperature is calculated at a distance of 6 meters. Figure 4 The color and temperature values are consistent.
[0027] Figure 7 FIG. 4 is a flow chart of an infrared temperature field reconstruction method in another embodiment.
[0028] Figure 8 Schematic diagram of the structure of a dual-light camera in an example.
[0029] Fig. 9 Schematic diagram of the process of infrared temperature field reconstruction method in an example.
[0030] Fig.10 This is a timing logic diagram of an infrared temperature field reconstruction method in an example.
[0031] Fig.11 This is a timing logic diagram of the infrared temperature field reconstruction method in another example.
[0032] Fig.12 This is a timing logic diagram of the infrared temperature field reconstruction method in another example.
[0033] Fig.13 This is a timing logic diagram of the infrared temperature field reconstruction method in yet another example.
[0034] Fig.14 Schematic diagram of the structure of an imaging temperature measurement device in one embodiment.
[0035] Fig.15 Schematic diagram of the structure of an imaging temperature measurement system in one embodiment. DETAILED DESCRIPTION
[0036] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0038] In the following description, the expression "some embodiments" is involved, which describes a subset of all possible embodiments. It should be noted that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
[0039] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may also be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are only for the purpose of explanation in conjunction with the embodiments of the accompanying drawings and do not represent the only implementation method.
[0040] See also Figure 1 , is a schematic diagram of an optional application scenario of the infrared temperature field reconstruction method provided in the embodiment of the present application, the infrared temperature field reconstruction method is applied to an imaging temperature measuring device 20, such as an imaging temperature measuring device 20 designed to be used in power inspection, intelligent machine room monitoring, distribution room monitoring and other application scenarios to take infrared images of target equipment in real time to measure temperature and identify high temperature risks. The imaging temperature measuring device 20 includes a data acquisition module 21, a memory 22 and a processor 23 that corrects temperature data to reconstruct temperature field data. The imaging temperature measuring device 20 can be loaded with a computer program that executes the infrared temperature field reconstruction method provided in the embodiment of the present application. In the process of real-time image acquisition, the original temperature data of the infrared image imaging is corrected by acquiring depth data, and the temperature difference of the imaging target caused by the difference in object distance is obtained, so as to obtain reconstructed temperature field data of the accurate temperature value of each imaging target in the infrared image.
[0041] Optional, see Figure 2The infrared temperature field reconstruction method can be applied to a terminal device 30 that is communicatively connected to an imaging temperature measuring device 20'. For example, the imaging temperature measuring device 20' is designed to be a handheld infrared thermal imager used in power inspection, intelligent machine room monitoring, power distribution room monitoring and other application scenarios to take infrared images of target equipment in real time to measure temperature and identify high temperature risks. The terminal device 30 can be a computer device, a mobile phone terminal, a cloud terminal or other intelligent device with storage and computing functions that is communicatively connected to the imaging temperature measuring device 20'. The terminal device 30 can be loaded with a computer program that executes the infrared temperature field reconstruction method provided in the embodiment of the present application. During the use of the imaging temperature measuring device 20' to collect images in real time, the temperature data and depth data of the collected infrared image are sent to the terminal device 30. The terminal device 30 obtains the depth data to correct the original temperature data of the infrared image imaging, and obtains reconstructed temperature field data that can eliminate the temperature difference of the imaging target caused by the object distance difference, and obtains the accurate temperature value of each imaging target in the infrared image. The reconstructed temperature field data can be returned to the imaging temperature measuring device 20' for displaying the temperature of the imaging target in the image collected in real time, and can also be used by the terminal device 30 on the local display interface to display the temperature of the imaging target in the image collected in real time by the imaging temperature measuring device 20'.
[0042] It should be noted that in the following embodiments provided in the present application, the infrared temperature field reconstruction method is mainly described by applying it to an imaging temperature measurement device (i.e., the imaging temperature measurement device is the execution subject), but this is not intended to limit the present invention.
[0043] See also Figure 3 , is an infrared temperature field reconstruction method provided by an embodiment, comprising the following steps:
[0044] S101, acquiring temperature data of an infrared image corresponding to a target field of view.
[0045] S103, acquiring depth data.
[0046] S105, correcting the temperature data using the depth data to obtain reconstructed temperature field data of the infrared image.
[0047] The target field of view refers to various scenes that need to capture infrared images. Acquiring the temperature data of the infrared image corresponding to the target field of view can refer to the temperature data required for infrared image imaging saved by the infrared image acquisition module when acquiring the infrared image; it can also refer to analyzing the infrared image and extracting the temperature data based on the infrared image imaging. For ease of understanding and description, the temperature data of the infrared image is represented by the temperature data set T mn express;
[0048] T p =(x p ,w p ,tp ),T p ∈T mn ;
[0049] Among them, x p ,w p is the coordinate of point p, t p is the temperature data of point p; temperature data set T mn The data may be in the form of, but not limited to, series, vectors, arrays, etc.
[0050] Depth data refers to the relative position and distance between the imaging target and the shooting device contained in the infrared image. Acquiring depth data can mean collecting infrared images for the target field of view and simultaneously collecting depth data through a depth data acquisition device; or it can mean collecting binocular image data for the target field of view and then calculating the depth information of the object in the scene using the visual difference between the images. For ease of understanding and description, the depth data is referred to as the depth data set D ij express:
[0051] d k =(u k ,v k ,l k ),d k ∈D ij ;
[0052] Among them, u k ,v k is the coordinate of point k, l k is the depth value of point k, i.e., the object distance.
[0053] Depth Dataset D ij The data may be in the form of, but not limited to, a series, vector, array, etc. Among them, the depth data acquisition equipment may include, but is not limited to, a depth sensor, a radar device, a laser scanning device, etc. The types of depth sensors include structured light / coded light, stereo vision, TOF, laser radar, etc.
[0054] Infrared images are usually calculated based on a fixed target distance. The relative distance between different imaging targets and the camera in the same target field of view will not affect the imaging display of the imaging target in the infrared image. That is, the same imaging target will remain consistent in the infrared image when it is captured at different distances from the camera. Please refer to Figure 4 and Figure 5, when the same imaging target is photographed at the same object distance using different distance parameters, the imaging target presents temperature differences and color differences in the infrared image. In this way, the temperature data of the infrared image, when the temperature is calculated using the distance parameter of the non-actual object distance, is different from the temperature calculation using the distance parameter of the actual object distance. The result obtained according to the actual distance parameter is more accurate. Depth data, including the shooting distance of all imaging targets in the infrared image, is corrected by depth data, and the temperature of each imaging target is corrected using the depth data to obtain the reconstructed temperature field data of the infrared image, so that the same target object can use the distance parameter of the actual object distance to calculate the temperature under any circumstances, so as to ensure that the temperature data of the imaging can be consistent when shooting at different distances. Please refer to Figure 4 and Figure 6 ,When the same imaging target is photographed at different object distances, the temperature of the imaging target in the infrared image remains consistent (within the error range).
[0055] The infrared temperature field reconstruction method provided in the above embodiment obtains depth data for the temperature data of the infrared image, and uses the depth data to correct the temperature data. The depth data includes the shooting distance of all imaging targets in the infrared image. The depth data is used to correct the temperature of target objects at different distances in the infrared image to obtain reconstructed temperature field data of the infrared image, so that target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement.
[0056] In some embodiments, see Figure 7 , before step S105, including:
[0057] S104: register the depth data to obtain registered depth data.
[0058] The temperature data and depth data of infrared images are obtained in different steps for the target field of view. The inconsistency of the equipment for collecting and obtaining temperature data and depth data leads to inconsistency in the target content of the two. By aligning the depth data and temperature data, the description of the target by the depth data and the temperature data is kept consistent in position and size. There is a one-to-one data correlation between the depth data and the temperature data, which is beneficial to reduce the amount of correction calculations.
[0059] In some embodiments, step S104 includes:
[0060] Performing a projective transformation on the depth data based on a homography matrix obtained by calibrating a dual-light scene shooting module;
[0061] The depth data after the projective transformation is resized to obtain the registered depth data.
[0062] The dual-light scene shooting module refers to a shooting module that can collect the homography matrix that can be converted between dual-light scene data based on calibration. The combination of dual-light scene data can be a combination of at least one of visible light image data and depth data, and at least one of infrared image data and temperature data. In an optional specific example, the dual-light scene shooting module refers to an image shooting module that simultaneously collects infrared images and visible light images for the target field of view, such as a dual-light camera formed by integrating an infrared image sensor and a visible light image sensor, or a combined image shooting module formed by assembling an infrared camera that independently shoots infrared images and a visible light camera that shoots visible light images. The dual-light scene shooting module can shoot infrared images and visible light images based on the same image shooting instruction. For example, the user can operate a shooting mode button on the imaging temperature measurement device, or touch the button to start shooting to form an image shooting instruction to complete the synchronous acquisition of infrared images and visible light images for the same target field of view.
[0063] In this embodiment, the dual-light scene shooting module is determined according to the product form of the imaging temperature measuring device to which the infrared temperature field reconstruction method described in the embodiment of the present application is applied. For example, if the imaging temperature measuring device is a certain handheld infrared thermal imager, and the handheld infrared thermal imager includes a dual-light camera formed by integrating an infrared image sensor and a visible light image sensor, then the homography matrix refers to the calibration of the dual-light camera of the handheld infrared thermal imager. The calibration can be completed before the handheld infrared thermal imager leaves the factory, or it can be completed by the user in the camera calibration mode after the handheld infrared thermal imager leaves the factory by setting the camera calibration mode on the handheld infrared thermal imager. It should be noted that the calibration before leaving the factory and the calibration after leaving the factory to support the user to complete the calibration in the camera calibration mode can be included at the same time, thereby providing the user with the option of whether to use the camera calibration mode to update the homography matrix during use according to actual needs.
[0064] A projective transformation is a mathematical transformation that defines a one-to-one transformation between two lines or a one-to-one transformation between two planes through the product of a finite number of central projections. This transformation keeps the single ratio of three collinear points unchanged, and the cross ratio of four collinear points unchanged, so a variety of transformations including orthogonal transformations, similarity transformations, and affine transformations can be considered special cases of projective transformations. Based on the homography matrix obtained by calibrating the dual-light scene shooting module, the coordinates of the depth data are projectively transformed using the homography matrix, and the depth data is transformed through projective transformation to obtain a depth data set that is aligned with the infrared image. The size of the depth data is then adjusted to have the same size as the temperature data, and depth data that is aligned with the temperature data and has the same size is obtained.
[0065] In some embodiments, obtaining a homography matrix includes:
[0066] The dual-light scene shooting module is used to shoot dual-light scene data for the calibration area;
[0067] A homography matrix is calculated according to feature point imaging information of feature points preset in the calibration area contained in the dual-light scene data.
[0068] The calibration operation of calibrating the dual-light scene shooting module to obtain the homography matrix includes shooting dual-light scene data for the calibration area through the dual-light scene shooting module, and calculating the homography matrix according to the feature point imaging information of the feature points preset in the calibration area contained in the dual-light scene data. The dual-light scene data may refer to infrared images and visible light images; or may refer to related image data that can express the same image information as the infrared image, such as the temperature data required for infrared image imaging, and related image data that can express the same image information as the visible light image, such as the depth data collected synchronously with the visible light image.
[0069] In an optional example, a set of calibration images is captured by a dual-light scene capture module for the calibration drawing, and the homography matrix H is calculated based on the eight feature points on the calibration image. 33 , as shown in the following formula 1:
[0070]
[0071] In the above embodiment, for each dual-light scene shooting module, the homography matrix obtained through calibration has a one-to-one correspondence. The dual-light scene shooting module is used to collect dual-light scene data of a calibration area containing feature point imaging information of preset feature points, and the homography matrix is calculated through the feature point imaging information. This can simplify the calculation method of the homography matrix and improve the calibration accuracy.
[0072] In some embodiments, the dual-light scene shooting module includes an RGB-D data acquisition module and an infrared thermal imaging data acquisition module, and the dual-light scene data includes an infrared image of the target field of view and temperature data corresponding to the infrared image, a visible light image and depth data corresponding to the visible light image.
[0073] The step of calculating the homography matrix according to the feature point imaging information of the preset feature points in the calibration area contained in the dual light scene data includes at least one of the following:
[0074] Based on the feature point imaging information of the feature points preset in the calibration area respectively contained in the infrared image and the visible light image in the dual-light scene data, a homography matrix is calculated;
[0075] Calculating a homography matrix based on feature point imaging information of feature points preset in the calibration area respectively contained in the infrared image in the dual-light scene data and the depth data;
[0076] Based on the temperature data in the dual-light scene data and the feature point imaging information of the feature points preset in the calibration area respectively contained in the visible light image, a homography matrix is calculated;
[0077] A homography matrix is calculated based on feature point imaging information of feature points preset in the calibration area respectively contained in the temperature data and the depth data in the dual-light scene data.
[0078] The RGB-D data acquisition module is an image acquisition module that integrates traditional RGB image sensors and depth sensors, such as TOF cameras (Time-Of-Light, depth imaging cameras). The RGB-D data acquisition module can directly obtain the depth information of the imaging scene while acquiring the RGB image, so that the depth data can express the same image information as the visible light image. The dual-light scene data used to calculate the homography matrix can be a visible light image and an infrared image or temperature data, or a depth data and an infrared image or temperature data. For ease of understanding and description, the infrared image is taken as the image dataset Y mn express:
[0079] Y p =(x p ,w p ,y p ),Y p ∈Y mn ;
[0080] Among them, x p ,w p is the coordinate of point p, y p is the gray value of point p. Similarly, the image dataset Y mn The data may be in the form of, but not limited to, a sequence, vector, array, etc. Image dataset Y mn Medium data and temperature data set T mn The data in has a one-to-one correspondence.
[0081] The visible light image is taken from the visible light color dataset C ij express:
[0082] c k =(u k ,v k ,r k ,g k b k ),c k ∈C ij;
[0083] Among them, u k ,v k is the coordinate of point k, r k ,g k b k is the color value of point K. Visible light color dataset C ij The data may be in the form of, but not limited to, a series, vector, array, etc. Visible light color data set C ij The data in the deep dataset D ij The data in has a one-to-one correspondence.
[0084] In the above embodiment, the visible light image and depth data in the dual-light scene shooting module are synchronously acquired by the RGB-D data acquisition module. In the process of calculating the homography matrix using the dual-light scene data, the depth data and the visible light image are interchangeable. In the embodiment of calculating the homography matrix using the dual-light scene data of a combination of depth data and temperature data, the depth data is aligned using the homography matrix, and then the temperature data is corrected by aligning the depth data. In this way, the process of correcting the temperature data only requires the use of the depth data and temperature data to participate in the calculation to complete.
[0085] In some embodiments, the dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module, and the dual-light scene data includes an infrared image of the target field of view and temperature data and depth data corresponding to the infrared image.
[0086] The step of calculating the homography matrix according to the feature point imaging information of the preset feature points in the calibration area contained in the dual light scene data includes at least one of the following:
[0087] Calculating a homography matrix based on feature point imaging information of feature points preset in the calibration area respectively contained in the infrared image in the dual-light scene data and the depth data;
[0088] A homography matrix is calculated based on feature point imaging information of feature points preset in the calibration area respectively contained in the temperature data and the depth data in the dual-light scene data.
[0089] In this embodiment, the depth data is acquired by using an independent depth sensor. The dual-light scene shooting module may further include an independent visible light image acquisition module to acquire visible light images, or it may not acquire visible light images by itself. That is, the dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module. The infrared image of the target field of view and the temperature data corresponding to the infrared image are acquired by the infrared thermal imaging data acquisition module, and the corresponding depth data is acquired by the depth sensor. At this time, the dual-light scene data used to calculate the homography matrix may be depth data and infrared image, or depth data and temperature data. The visible light image may not be included in the process of correcting the temperature data to obtain the reconstructed temperature field data and using the reconstructed temperature field data for display.
[0090] Optionally, the dual-light scene shooting module may not capture visible light images by itself, but may communicate with other terminal devices with visible light image capture functions to obtain visible light images captured by the terminal devices in real time for correcting the temperature data to obtain reconstructed temperature field data, and applying the reconstructed temperature field data for display. At this time, it will further include using the homography matrix to align the visible light images. The alignment process can be the same as the alignment process for aligning the depth data, which will not be repeated here.
[0091] In some embodiments, resizing the depth data after the projective transformation to obtain the registered depth data includes:
[0092] The depth data after the projective transformation is cropped based on the infrared image in the dual-light scene data to obtain size-matched registered depth data.
[0093] The depth data is projectively transformed to obtain depth data registered with the infrared image, and the depth data after the projective transformation is cropped with reference to the infrared image to obtain registered depth data that is consistent in size with the infrared image and registered.
[0094] In this embodiment, the registration of depth data mainly includes using the homography matrix to perform a projective transformation on the depth data, and cropping the depth data after the projective transformation according to the infrared image. The homography matrix only needs to be obtained by calibrating the imaging temperature measurement equipment once. During the use of the imaging temperature measurement equipment, the homography matrix can be directly used to calculate the projective transformation in the processing flow of reconstructing the temperature field of the temperature data of the real-time infrared image, which is beneficial to reduce the amount of calculation for data registration and facilitate improving the real-time performance of the processing flow of infrared temperature field reconstruction.
[0095] Optionally, in some other embodiments, resizing the depth data after the projective transformation to obtain the registered depth data includes:
[0096] The depth data is cropped based on the temperature data in the dual-light scene data to obtain size-matched registered depth data.
[0097] The depth data is projectively transformed to obtain depth data that is registered with the temperature data. The depth data after the projective transformation is cropped with reference to the temperature data to obtain registered depth data that is consistent in size and registered with the temperature data. The depth data after the projective transformation is not limited to being cropped with reference to the infrared image, but can also be cropped with reference to the temperature data to obtain registered depth data of matching size. In this way, the process of reconstructing the temperature field of the temperature data only needs to use the depth data and temperature data to participate in the calculation to complete.
[0098] In some embodiments, step S105 includes:
[0099] For the temperature data value of the target point, correct it by using a temperature correction function according to the depth data value of the target point in the registered depth data to obtain the corrected temperature data value of the target point;
[0100] The reconstructed temperature field data of the infrared image is obtained according to the corrected temperature data values of the target points constituting each target object in the infrared image.
[0101] The registration depth data corresponds to the temperature data one by one, and the target point can be any pixel point in the infrared image. In this embodiment, the temperature correction function can be obtained by using known data statistical means under the guidance of the technical concept of using the registration depth data to correct it proposed in this application. As long as the temperature data of imaging targets at different distances can be corrected to a certain extent using their corresponding distance values, the accuracy of the reconstructed temperature field data can be improved to a certain extent, and this application does not make any restrictions here.
[0102] In the above embodiment, for the temperature data value of each target point in the image, the corrected temperature data value is calculated by using the depth data value corresponding to the target point through a set temperature correction function, which can simplify the correction calculation.
[0103] Optionally, in an optional specific example, the temperature correction function is as follows:
[0104]
[0105] Among them, t p Indicates the temperature data value of the target point p, l" p represents the depth data value of the target point p, A1, B1, E1, A0, B0, E0 represent the correction coefficients, f_t p Represents the corrected depth data value of the target point p.
[0106] The temperature data value can be the original temperature data required by the infrared core to collect infrared images, or it can be the temperature data corrected by emissivity, atmospheric temperature, reflection temperature and atmospheric transmittance. The depth data value l of the target point p" p is the shooting distance of the imaging target in the infrared image.
[0107] In the above embodiment, the depth data value l" of the target point p is used. p A temperature correction function is formed to compensate the temperature data value. The correction coefficients A1, B1, E1, A0, B0, and E0 can be calibrated through test data to improve the accuracy of reconstructed temperature field data.
[0108] In some embodiments, the infrared temperature field reconstruction method further includes:
[0109] Acquire an infrared image corresponding to the target field of view;
[0110] The reconstructed temperature field data is applied to the infrared image for display.
[0111] In this embodiment, while obtaining the temperature data of the infrared image of the target field of view, the infrared image of the target field of view is also obtained. After the temperature data is corrected using the depth data to obtain reconstructed temperature field data, the reconstructed temperature field data can be applied to the infrared image for display. The reconstructed temperature field data can be used to display more accurate temperature values of each imaging target in the infrared image.
[0112] In some embodiments, the infrared temperature field reconstruction method further includes:
[0113] Acquire a visible light image corresponding to the target field of view;
[0114] Registering the visible light image to obtain a registered visible light image;
[0115] The reconstructed temperature field data is applied to the registered visible light image for display; and / or the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display.
[0116] In this embodiment, while acquiring the temperature data of the infrared image of the target field of view, it also includes acquiring the visible light image of the target field of view. After the temperature data is corrected using the depth data to obtain reconstructed temperature field data, the reconstructed temperature field data can be applied to the visible light image for display, and the reconstructed temperature field data can be used to display a more accurate temperature value of each imaging target in the visible light image; or the reconstructed temperature field data can be applied to a dual-light fusion image of the infrared image and the aligned visible light image for display, and the reconstructed temperature field data can be used to display a more accurate temperature value of each imaging target in the dual-light fusion image.
[0117] In order to have a more comprehensive understanding of the infrared temperature field reconstruction method provided in the embodiment of the present application, please refer to Figure 8 and Fig. 9 , the infrared temperature field reconstruction method is applied to a dual-light camera integrating a TOF camera 85 and an infrared thermal imager 80 as an example for explanation, and the dual-light camera includes a TOF camera 85 as an RGB-D data acquisition module, an infrared thermal imaging data acquisition module 81, a storage module 82, a data processing module 83 and an image temperature display module 84. The TOF camera 85 is used to collect visible light images and depth data; the infrared thermal imaging data acquisition module 81 is used to collect infrared images and temperature data; the storage module 82 is used to store and cache data required for temporary storage during data processing; the data processing module 83 is used to execute the steps of the infrared temperature field reconstruction method described in the embodiment of the present application; and the image temperature display module 84 is used to display images and temperature data.
[0118] The infrared temperature field reconstruction method comprises the following steps:
[0119] S11, the infrared thermal imaging data acquisition module acquires a temperature data set and an infrared image data set of infrared images.
[0120] T p =(x p ,w p ,t p ), T p ∈T mn ; where x p , w p is the coordinate of point p, t p is the temperature data value of point P after correction of emissivity, atmospheric temperature, reflection temperature and atmospheric transmittance, T mn Represents a temperature dataset.
[0121] Y p =(x p ,w p ,y p ), Y p ∈Y mn ; where x p , w p is the point coordinate, y p is the gray value of each point, Y mn Represents the infrared image dataset.
[0122] S12, the RGB-D data acquisition module acquires the RGB-D data set and establishes a visible light image data set and a depth data set.
[0123] c k =(u k ,vk ,r k ,g k b k ),c k ∈C ij ; where u k ,v k is the coordinate of point k, r k ,g k b k is the color value of the point, C ij Represents the visible light image dataset.
[0124] d k =(u k ,v k ,l k ),d k ∈D ij ; Among them u k ,v k is the coordinate of point k, l k is the depth value of the point, i.e. the object distance, D ij Represents a deep dataset.
[0125] S13, obtaining a homography matrix obtained by calibration.
[0126] Calibration can be done before the dual-light camera leaves the factory, or it can be done by the user in the camera calibration mode after leaving the factory. The dual-light camera takes the calibration drawing to obtain infrared images and visible light images, and calculates the homography matrix H based on the 8 feature points on the calibration drawing. 33 .
[0127] S14, performing projective transformation on the coordinates of the visible light image dataset and the depth dataset respectively, to obtain the visible light image dataset aligned with the infrared image and the corresponding depth image.
[0128] (1) Projectively transform the visible light image to obtain the visible light dataset C that is registered with the infrared image. IJ ' is as follows:
[0129] c′ K1 =(u′ K1 ,v′ K1 ,r′ K1 ,g′ K1 ,b′ K1 ), c′ K1 ∈C IJ ';
[0130] Among them, u′ K1 , v′ K1 is the coordinate of point K1 after the projective transformation of point k, as shown in the following formula 3;
[0131]
[0132] r′ K ,g′ K ,b′ K is the color value of the mapped k point. If K1 has no mapping point, then r′ K ,g′ K ,b′ K Assign 0.
[0133] (2) Perform the same projective transformation on the depth dataset to obtain the depth dataset D that is aligned with the infrared image. IJ ' is as follows:
[0134] d′ K1 =(u′ K1 ,v′ K1 ,l′ K1 ), d′ K1 ∈D IJ ';
[0135] Among them, u′ K1 ,v′ K1 is the coordinate of point K1 after the projective transformation of point k, as shown in the following formula 4:
[0136]
[0137] l′ K1 is the depth value of the mapped k point. If K1 has no mapping point, then l′ K1 Assign 0.
[0138] S15, cropping the visible light image dataset and the depth dataset according to the infrared image to obtain a visible light dataset and a depth dataset that are consistent in size with the infrared image and are aligned.
[0139] (1) For the visible light image dataset C registered with the infrared image IJ 'Crop and reorganize the infrared image according to its size to obtain a new visible light dataset C' mn '.
[0140] (2) Depth dataset D registered with infrared image IJ 'Cut and reassemble according to the size of the infrared image to obtain the depth dataset D' mn '.
[0141] S16, through the above steps S11 to S15, various data sets with one-to-one correspondence are obtained.
[0142] Visible light dataset C' mn ', depth dataset D' mn ', temperature data set Tmn and infrared image dataset Y mn .
[0143] S17, performing distance correction on the temperature data set to obtain accurate reconstructed temperature field data.
[0144] The temperature correction at point p is the same as that in the above embodiment, formula 2, f_t p ∈F_T mn , F_T mn Represents the reconstructed temperature field data.
[0145] S18, performing image rendering processing on the registered visible light dataset and infrared image dataset, applying the reconstructed temperature field data to the visible light dataset, infrared image dataset, or displaying the fused image data obtained by fusion of the visible light dataset and the infrared image dataset on a display interface.
[0146] The TOF camera is used to collect visible light images and depth data. The depth data and visible light images are interchangeable. Please refer to Figures 10 to 13 , which are different embodiments of the timing logic diagram of the infrared temperature field reconstruction method.
[0147] like Fig.10 As shown, the dual-light camera includes an RGB-D data acquisition module, an infrared thermal imaging data acquisition module, a storage module, a processing module and a display module. The RGB-D data acquisition module acquires visible light image data and depth data, and the infrared thermal imaging data acquisition module acquires infrared image data and temperature data; the storage module stores the acquired visible light image data, depth data, infrared image data and temperature data. The processing flow of the infrared temperature field reconstruction method includes:
[0148] S21, obtaining a homography matrix by calibrating the bi-light scene data composed of the visible light image data and the infrared image data.
[0149] S22, performing a projective transformation on the visible light image data using a homography matrix to obtain visible light image data aligned with the infrared image.
[0150] S23, cropping the visible light image data to obtain registered visible light image data having a size consistent with the infrared image.
[0151] S24, performing a projective transformation on the depth data using a homography matrix to obtain depth data aligned with the infrared image.
[0152] S25, cropping the depth data to obtain registration depth data consistent with the size of the infrared image.
[0153] S26, using the registered depth data to correct the temperature data to obtain a reconstructed temperature field.
[0154] S27, rendering the registered visible light image data and infrared image data, and displaying them together with the reconstructed temperature field data on the device interface.
[0155] like Fig.11 As shown, Fig.10 The difference between the illustrated embodiment mainly lies in that step S21 is replaced by step S31.
[0156] S31, obtaining a homography matrix by calibrating bi-light scene data composed of depth data and infrared image data.
[0157] like Fig.12 As shown, Fig.10 The difference between the illustrated embodiment mainly lies in that step S21 is replaced by step S41.
[0158] S41, obtaining a homography matrix by calibrating the bi-light scene data composed of the visible light image data and the temperature data.
[0159] like Fig.13 As shown, Fig.10 The difference between the illustrated embodiment mainly lies in that step S21 is replaced by step S51.
[0160] S51, obtaining a homography matrix by calibrating the bi-optical scene data composed of the depth data and the temperature data.
[0161] for Fig.13 In the embodiment shown in which the dual-light scene data of a combination of depth data and temperature data is used to calculate the homography matrix, the depth data is aligned using the homography matrix, and then the temperature data is corrected by aligning the depth data. At this time, the aligned depth data can be directly used as the result of calibration. In this way, the process of correcting the temperature data only requires the use of depth data and temperature data to participate in the calculation.
[0162] The infrared temperature field reconstruction method provided in the embodiment of the present application aligns the visible light image and the infrared image, and then associates the temperature data and the depth data for mapping, uses the depth data to correct the temperature data, and reconstructs the temperature data, thereby eliminating the temperature difference caused by using the same object distance calculation for the objects under test with different depths of field, so that target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement, and the reconstructed temperature field data can be applied to the temperature display of each imaging target in a two-dimensional or three-dimensional image to achieve more accurate temperature measurement.
[0163] On the other hand, please refer to Fig.14, an imaging temperature measurement device 20 is also provided, including a memory 22, a processor 23, and an infrared thermal imaging data acquisition module 212 and a depth data acquisition module 211 connected to the processor 23; the infrared thermal imaging data acquisition module 212 is used to collect temperature data of the infrared image corresponding to the target field of view; the depth data acquisition module 211 is used to collect depth data; a computer program is stored on the memory 22; when the computer program is executed by the processor 23, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0164] Optionally, the infrared thermal imaging data acquisition module 212 is also used to acquire an infrared image corresponding to the target field of view. The infrared thermal imaging data acquisition module 212 may include an infrared core, which acquires infrared images and stores temperature data corresponding to the infrared images.
[0165] Optionally, the depth data acquisition module 211 includes one of the following: 1. The depth data acquisition module 211 includes a depth sensor, which is used to collect depth data, and the reconstructed temperature field data is applied to the infrared image for display. 2. The depth data acquisition module 211 includes a depth imaging camera, which is used to collect the visible light image of the target field of view and the depth data corresponding to the visible light image, and the display method of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to the dual-light fusion image of the infrared image and the registered visible light image for display. 3. The depth data acquisition module 211 includes a visible light sensor and a depth sensor, the depth sensor is used to collect depth data, and the visible light sensor is used to collect visible light images of the target field of view; the display method of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to the dual-light fusion image of the infrared image and the registered visible light image for display.
[0166] In one example, the imaging temperature measurement device 20 is a handheld infrared thermal imager.
[0167] On the other hand, please refer to Fig.15, an imaging temperature measurement system is also provided, including an imaging temperature measurement device 20' and a terminal device 30 connected to the imaging temperature measurement device 20' in communication. The imaging temperature measurement device 20' includes an infrared thermal imaging data acquisition module 212 and a depth data acquisition module 211, the infrared thermal imaging data acquisition module 212 is used to collect temperature data of the infrared image corresponding to the target field of view and send it to the terminal device 30, and the depth data acquisition module 211 is used to collect depth data and send it to the terminal device 30. The terminal device 30 includes a memory 31 and a processor 32, the memory 31 stores a computer program, and when the computer program is executed by the processor 32, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0168] Optionally, the imaging temperature measuring device 20' also includes a display module 213; the imaging temperature measuring device 20' is also used to receive the reconstructed temperature field data returned by the terminal device 30, and display the reconstructed temperature field data on the display module 213 using one of the following display methods: applying the reconstructed temperature field data to the infrared image for display; applying the reconstructed temperature field data to the registered visible light image for display; applying the reconstructed temperature field data to the dual-light fusion image of the infrared image and the registered visible light image for display.
[0169] In another aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the infrared temperature field reconstruction method described in any embodiment of the present application is implemented.
[0170] On the other hand, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processes of the above-mentioned infrared temperature field reconstruction method embodiments are implemented, and the same technical effects can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0171] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, infrared thermal imager or network equipment, etc.) to execute the methods described in each embodiment of the present invention.
[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for reconstructing an infrared temperature field, characterized in that: include: Acquire temperature data of the infrared image corresponding to the target field of view; Get depth data; The temperature data is corrected by using the depth data to obtain reconstructed temperature field data of the infrared image.
2. The infrared temperature field reconstruction method according to claim 1, characterized in that: Before the temperature data is corrected by the depth data to obtain the reconstructed temperature field data of the infrared image, the method includes: The depth data is registered to obtain registered depth data.
3. The infrared temperature field reconstruction method according to claim 2, characterized in that: The registering the depth data to obtain the registered depth data includes: Performing a projective transformation on the depth data based on a homography matrix obtained by calibrating a dual-light scene shooting module; The depth data after the projective transformation is resized to obtain the registered depth data.
4. The infrared temperature field reconstruction method according to claim 3, characterized in that: Get the homography matrix, including: The dual-light scene shooting module is used to shoot dual-light scene data for the calibration area; A homography matrix is calculated according to feature point imaging information of feature points preset in the calibration area contained in the dual-light scene data.
5. The infrared temperature field reconstruction method according to claim 4, characterized in that: The dual-light scene shooting module includes an RGB-D data acquisition module and an infrared thermal imaging data acquisition module, and the dual-light scene data includes an infrared image of the target field of view and temperature data corresponding to the infrared image, a visible light image and depth data corresponding to the visible light image; The step of calculating the homography matrix according to the feature point imaging information of the preset feature points in the calibration area contained in the dual light scene data includes at least one of the following: Based on the feature point imaging information of the feature points preset in the calibration area respectively contained in the infrared image and the visible light image in the dual-light scene data, a homography matrix is calculated; Calculating a homography matrix based on feature point imaging information of feature points preset in the calibration area respectively contained in the infrared image in the dual-light scene data and the depth data; Based on the temperature data in the dual-light scene data and the feature point imaging information of the feature points preset in the calibration area respectively contained in the visible light image, a homography matrix is calculated; A homography matrix is calculated based on feature point imaging information of feature points preset in the calibration area respectively contained in the temperature data and the depth data in the dual-light scene data.
6. The infrared temperature field reconstruction method according to claim 4, characterized in that: The dual-light scene shooting module includes a depth sensor and an infrared thermal imaging data acquisition module, and the dual-light scene data includes an infrared image, temperature data, and depth data of the target field of view; The step of calculating the homography matrix according to the feature point imaging information of the preset feature points in the calibration area contained in the dual light scene data includes at least one of the following: Calculating a homography matrix based on feature point imaging information of feature points preset in the calibration area respectively contained in the infrared image in the dual-light scene data and the depth data; A homography matrix is calculated based on feature point imaging information of feature points preset in the calibration area respectively contained in the temperature data and the depth data in the dual-light scene data.
7. The infrared temperature field reconstruction method according to claim 3, characterized in that: The step of resizing the depth data after the projective transformation to obtain the registered depth data includes: The depth data after the projective transformation is cropped based on the infrared image in the dual-light scene data to obtain size-matched registered depth data.
8. The infrared temperature field reconstruction method according to claim 3, characterized in that: The step of resizing the depth data after the projective transformation to obtain the registered depth data includes: The depth data is cropped based on the temperature data in the dual-light scene data to obtain size-matched registered depth data.
9. The infrared temperature field reconstruction method according to claim 2, characterized in that: The step of correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the infrared image includes: For the temperature data value of the target point, correct it by using a temperature correction function according to the depth data value of the target point in the registered depth data to obtain the corrected temperature data value of the target point; The reconstructed temperature field data of the infrared image is obtained according to the corrected temperature data values of the target points constituting each target object in the infrared image.
10. The infrared temperature field reconstruction method according to claim 9, characterized in that: The temperature correction function is as follows: Among them, t p Indicates the temperature data value of the target point p, l' p ' represents the depth data value of the target point p, A1, B1, E1, A0, B0, E0 represent the correction coefficients, f_t p Represents the corrected depth data value of the target point p.
11. The infrared temperature field reconstruction method according to claim 1, characterized in that: Also includes: Acquire an infrared image corresponding to the target field of view; The reconstructed temperature field data is applied to the infrared image for display.
12. The infrared temperature field reconstruction method according to claim 11, characterized in that: Also includes: Acquire a visible light image corresponding to the target field of view; Registering the visible light image to obtain a registered visible light image; The reconstructed temperature field data is applied to the registered visible light image for display; and / or the reconstructed temperature field data is applied to a dual-light fusion image of the infrared image and the registered visible light image for display.
13. An imaging temperature measurement device, characterized in that: It includes a memory, a processor, and an infrared thermal imaging data acquisition module and a depth data acquisition module connected to the processor; The infrared thermal imaging data acquisition module is used to acquire temperature data of the infrared image corresponding to the target field of view; The depth data acquisition module is used to collect depth data; The memory stores a computer program; When the computer program is executed by the processor, the infrared temperature field reconstruction method according to any one of claims 1 to 12 is implemented.
14. The imaging temperature measurement device according to claim 13, characterized in that: The infrared thermal imaging data module is also used to collect the infrared image corresponding to the target field of view; and / or, The depth data acquisition module includes one of the following: The depth data acquisition module includes a depth sensor, which is used to acquire depth data, and the reconstructed temperature field data is applied to the infrared image for display; The depth data acquisition module includes a depth imaging camera, which is used to acquire a visible light image of the target field of view and depth data corresponding to the visible light image. The display mode of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to the dual-light fusion image of the infrared image and the registered visible light image for display; The depth data acquisition module includes a visible light sensor and a depth sensor, the depth sensor is used to collect depth data, and the visible light sensor is used to collect a visible light image of the target field of view; the display method of the reconstructed temperature field data includes one of the following: the reconstructed temperature field data is applied to the infrared image for display; the reconstructed temperature field data is applied to the registered visible light image for display; the reconstructed temperature field data is applied to the dual-light fusion image of the infrared image and the registered visible light image for display.
15. The imaging temperature measurement device according to claim 13, characterized in that: The imaging temperature measuring device is a handheld infrared thermal imager.
16. An imaging temperature measurement system, characterized in that: It includes an imaging temperature measuring device and a terminal device connected to the imaging temperature measuring device for communication; The imaging temperature measurement device includes an infrared thermal imaging data acquisition module and a depth data acquisition module. The infrared thermal imaging data acquisition module is used to acquire temperature data of an infrared image corresponding to the target field of view and send it to the terminal device. The depth data acquisition module is used to acquire depth data and send it to the terminal device. The terminal device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the infrared temperature field reconstruction method according to any one of claims 1 to 12 is implemented.
17. The imaging temperature measurement system according to claim 16, characterized in that: The imaging temperature measurement device also includes a display module; The imaging temperature measurement device is also used to receive the reconstructed temperature field data returned by the terminal device, and display the reconstructed temperature field data on the display module using one of the following display methods: applying the reconstructed temperature field data to the infrared image for display; applying the reconstructed temperature field data to the registered visible light image for display; applying the reconstructed temperature field data to the dual-light fusion image of the infrared image and the registered visible light image for display.
18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the infrared temperature field reconstruction method according to any one of claims 1 to 12 is implemented.
19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the infrared temperature field reconstruction method according to any one of claims 1 to 12 is implemented.
Citation Information
Patent Citations
Temperature measurement method and system
CN111537074A
Image registration method and device, terminal equipment and computer readable storage medium
CN112085771A
Image processing method and device
CN112634337A
Thermal infrared imager with TOF sensor
CN114235157A
Multi-modal three-dimensional sensing method and system based on space-time domain registration
CN118209991A
