Three-dimensional imaging temperature measurement method, device and system, product and medium
By acquiring and reconstructing the depth information in the image data, the problem that traditional infrared thermal imaging technology cannot measure the distance between the target and the thermal imager is solved, and accurate temperature measurement in three-dimensional images is achieved, meeting the needs of the industry and power industries for scene analysis. .
Patent Information
- Application Number
- CN202411294791.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional infrared thermal imaging technology can only provide two-dimensional images, and cannot accurately measure the distance between the target and the thermal imager, resulting in inaccurate temperature field data, and cannot fully utilize the three-dimensional three-dimensional characteristics in the image, making it impossible to achieve industrial and power industry scenarios. In-depth analysis and application.
By obtaining the image data, temperature data and depth data of the target field of view, reconstructing the image data based on the depth data, correcting the temperature data, and obtaining the reconstruction temperature field data in the three-dimensional image, realizing accurate temperature measurement of targets from different distances.
It realizes the accurate temperature value of the imaging target in three-dimensional images, improves the accuracy of temperature measurement, and meets the needs of the industry and power industries for scenario analysis and application.
Smart Images

Figure CN119085860A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a three-dimensional imaging temperature measurement method, a three-dimensional imaging temperature measurement device, a three-dimensional imaging temperature measurement system, a computer program product, and a computer-readable storage medium. Background Art
[0002] The industrial power industry has begun to integrate digitalization and entities, relying on information technology such as data intelligence and the establishment of a virtual information space in the objective world to empower the operation and maintenance of large-scale industries, power stations, power grids, etc. At present, traditional monitoring technologies and portable devices use the video stream connected to infrared thermal imaging to present images by superimposing temperature and color. The planar imaging characteristics of optical lenses result in the infrared thermal image information being obtained in the form of two-dimensional thermal imaging.
[0003] However, infrared thermal imaging images can only calculate the temperature field of all imaging objects in the same image based on a certain target distance. It cannot effectively measure the distance between all imaging objects and the thermal imager, nor can it reflect the distance difference of the measured target in the image due to the three-dimensional characteristics. As a result, the temperature field data is not accurate enough, and the distance of the measured target needs to be corrected in the later analysis. On the other hand, although some equipment technologies use binocular visible light cameras to build a three-dimensional model, this is only a three-dimensional image fusion, and has not yet fully utilized the characteristics of various feature data to meet the needs of major applications such as industry, power and other industries for scene analysis, mining, and retrieval. Summary of the invention
[0004] In order to solve the existing technical problems, the present application provides a three-dimensional imaging temperature measurement method, a three-dimensional imaging temperature measurement device, a three-dimensional imaging temperature measurement system, a computer program product and a computer-readable storage medium for obtaining an accurate temperature field for accurate temperature display of the measured points of the imaging target in the three-dimensional image data.
[0005] In a first aspect, a three-dimensional imaging temperature measurement method is provided, comprising:
[0006] Acquire image data, temperature data, and depth data of the target field of view;
[0007] Reconstructing the image data based on the depth data to obtain three-dimensional image data;
[0008] Correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the image data;
[0009] The reconstructed temperature field data is applied to the three-dimensional image data for display.
[0010] In a second aspect, a three-dimensional imaging temperature measurement device is provided, comprising a memory, a processor, and a data acquisition module connected to the processor;
[0011] The data acquisition module is used to collect image data, temperature data and depth data of the target field of view;
[0012] The memory stores a computer program;
[0013] When the computer program is executed by the processor, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0014] In a third aspect, a three-dimensional imaging temperature measurement system is provided, including an imaging temperature measurement device and a terminal device communicatively connected to the imaging temperature measurement device;
[0015] The imaging temperature measurement device includes a data acquisition module, which is used to collect image data, temperature data and depth data of the target field of view and send them to the terminal device;
[0016] The terminal device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0017] In a fourth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0018] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0019] The three-dimensional imaging temperature measurement method provided in the above embodiment obtains image data, temperature data and depth data of the target field of view, the depth data includes the shooting distance information of all imaging targets in the image data, reconstructs the image data based on the depth data to obtain three-dimensional image data, uses the depth data to correct the temperature data, and corrects the temperature difference of imaging targets at different relative distances in the image data according to the depth data to obtain reconstructed temperature field data of the infrared image, so that target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement, and applies the reconstructed temperature field data to the three-dimensional image data for display, so that the accurate temperature value of the imaging target can be displayed in the three-dimensional image data to achieve the purpose of accurate temperature measurement.
[0020] The three-dimensional imaging temperature measurement equipment, three-dimensional imaging temperature measurement system, computer program product and computer-readable storage medium provided in the above embodiments belong to the same concept as the corresponding three-dimensional imaging temperature measurement method embodiments, and thus have the same technical effects as the corresponding three-dimensional imaging temperature measurement method embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of an optional application scenario of the three-dimensional imaging temperature measurement method.
[0022] Figure 2 This is a schematic diagram of another optional application scenario of the three-dimensional imaging temperature measurement method.
[0023] Figure 3 A schematic flow chart of a three-dimensional imaging temperature measurement method in one embodiment.
[0024] Figure 4 This is a schematic diagram of the structure of a three-dimensional imaging temperature measurement device in an example.
[0025] Figure 5 Schematic diagram of the process of a three-dimensional imaging temperature measurement method in an example.
[0026] Figure 6 Schematic diagram of the structure of an imaging temperature measurement device in one embodiment.
[0027] Figure 7 Schematic diagram of the structure of an imaging temperature measurement system in one embodiment. DETAILED DESCRIPTION
[0028] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0030] In the following description, the expression "some embodiments" is involved, which describes a subset of all possible embodiments. It should be noted that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
[0031] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may also be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are only for the purpose of explanation in conjunction with the embodiments of the accompanying drawings and do not represent the only implementation method.
[0032] See also Figure 1 , is a schematic diagram of an optional application scenario of the three-dimensional imaging temperature measurement method provided in the embodiment of the present application, the three-dimensional imaging temperature measurement method is applied to an imaging temperature measurement device 20, such as an imaging temperature measurement device 20 designed to be used in power inspection, intelligent machine room monitoring, distribution room monitoring and other application places to take infrared images of target devices in real time for temperature measurement to identify high temperature risks. The imaging temperature measurement device 20 includes a data acquisition module 21, a memory 22 and a processor 23. The imaging temperature measurement device 20 can be loaded with a computer program for executing the three-dimensional imaging temperature measurement method provided in the embodiment of the present application. The data acquisition module 21 is used to collect image data, temperature data and depth data of the target field of view. In the process of real-time acquisition of image data, the original temperature data of the imaging target in the image data is corrected by acquiring the depth data corresponding to the image data, and the temperature difference of the imaging target caused by the difference in object distance is eliminated, and the reconstructed temperature field data of the accurate temperature value of each imaging target in the image data is obtained, and the image data is reconstructed in three dimensions using the depth data to obtain the three-dimensional image data, and the reconstructed temperature field data is applied to the three-dimensional image data for display, so that the accurate temperature value of the imaging target can be displayed in the three-dimensional image data.
[0033] Optional, see Figure 2The three-dimensional imaging temperature measurement method can be applied to a terminal device 30 that is communicatively connected to an imaging temperature measurement device 20'. For example, the imaging temperature measurement device 20' is designed to be a handheld infrared thermal imager used in power inspection, intelligent machine room monitoring, power distribution room monitoring and other application scenarios to take infrared images of target equipment in real time to measure temperature and identify high temperature risks. The terminal device 30 can be a computer device, a mobile phone terminal, a cloud terminal or other intelligent device with storage and computing functions that is communicatively connected to the imaging temperature measurement device 20'. The terminal device 30 can be loaded with a computer program that executes the three-dimensional imaging temperature measurement method provided in the embodiment of the present application. During the use of the imaging temperature measurement device 20' to collect images in real time, the collected image data, temperature data and depth data are sent to the terminal device 30. The terminal device 30 performs three-dimensional reconstruction of the image based on the depth data, and corrects the original temperature data of the imaging target in the image data according to the depth data, so as to obtain reconstructed temperature field data that can eliminate the temperature difference of the imaging target caused by the difference in object distance, and obtain the accurate temperature value of each imaging target in the image data. The reconstructed temperature field data can be returned to the imaging temperature measurement device 20' for displaying the temperature of the imaging target in the image data collected in real time, and can also be used by the terminal device 30 on the local display interface to display the temperature of the imaging target in the three-dimensional image data obtained after three-dimensional reconstruction of the data image collected in real time by the imaging temperature measurement device 20'.
[0034] It should be noted that in the following embodiments provided in the present application, the three-dimensional imaging temperature measurement method is mainly described by applying it to an imaging temperature measurement device (i.e., the imaging temperature measurement device is the execution body), but this is not intended to limit the present invention.
[0035] See also Figure 3 , a three-dimensional imaging temperature measurement method provided by an embodiment includes the following steps:
[0036] S101, acquiring image data, temperature data, and depth data of a target field of view.
[0037] Image data may refer to a visible light image, an infrared image, or both a visible light image and an infrared image.
[0038] S103: reconstruct the image data based on the depth data to obtain three-dimensional image data.
[0039] Depth data can reflect the shooting distance of the imaging target in the image data, and can provide geometric depth information of the object surface of the imaging target in the image data. The three-dimensional model of the imaging target can be calculated and constructed through this geometric depth information.
[0040] In an optional example, reconstructing the image data using the depth data to obtain the three-dimensional image data mainly includes:
[0041] From the camera pose matrix of the depth data, we can find the mapping point v of point p in the world coordinate system in the camera coordinate system, and from the camera intrinsic parameter matrix, we can back-project point v to find the corresponding pixel x in the depth image. The depth value of pixel x is value(x), and the distance from point v to the origin of the camera coordinate system is distance(v).
[0042] The signed distance function value sdf of p is sdf(p)=value(x)-distance(v).
[0043] Calculate tsdf(p), within the cutoff distance u, tsdf(p) = sdf(p) / |u|, otherwise, if sdf(p)>0, tsdf(p) = 1, sdf(p)<0, tsdf(p) = -1.
[0044] The calculation formula of the weight w(p) is: w(p)=cos(θ) / distance(v); where θ is the angle between the projection light and the surface normal vector. The weight w(p) of each frame is used to fuse multiple frame data to construct a three-dimensional image.
[0045] Specifically, in this embodiment, the method for reconstructing three-dimensional image data may include the following steps:
[0046] S1, calculate the position of voxel x in physical coordinates. Let the coordinates of voxel x on the TSDF (Truncated Signed Distance Function) map (Vx, Vy, Vz), and the position of the corresponding pixel point x in the physical world coordinate system be shown in the following formula 1.
[0047] Px,wrd=(X0+Vx*voxel.x,y0+Vy*voxel.y,z0+Vz*voxel.z); (Formula 1)
[0048] S2, calculate the position of voxel x in the camera coordinate system.
[0049] Assume that the position of the camera relative to the physical coordinate system is R and T, and the position of voxel x in the camera coordinate system is shown in the following formula 2.
[0050] Px,cam=RPx,wrd+T; (Formula 2)
[0051] S3, calculate the depth camz(x) of voxel x relative to the camera. According to the camera imaging model, it is shown in the following formula 3:
[0052] camz(x)*Ix=KPx,cam; (Formula 3)
[0053] Where K represents the intrinsic parameter matrix of the camera, and Ix represents the pixel coordinates of the voxel x projected on the camera imaging plane.
[0054] Draw a straight line along the optical center of the camera and voxel x. This line will intersect the cross section of the object at a point P. The depth of point P is dp. The current depth map is D. In actual calculation, dp = D(Ix). Then the sdf value of voxel x can be calculated using the following formula 4:
[0055] sdf(x)=dp-dx=D(Ix)-camz(x); (Formula 4)
[0056] The depth value of pixel point x is D(Ix), and the distance from point x to the origin of the camera coordinate system is camz(x).
[0057] S4, calculate the tsdf value.
[0058] sdf(x)>0 means that voxel x is between the camera and the object surface; sdf(x)<0 means that voxel x is behind the object surface. Based on the sdf value of voxel x, its tsdf value can be calculated as shown in the following formula 5:
[0059] tsdf(x)=max[-1,min(1,sdf(x) / t)]; (Formula 5)
[0060] The meaning of sdf(x) / t, i.e. truncated, is:
[0061] When -t≤sdf(x)≤t, tsdf(x)=sdf(x) / t∈[-1,1];
[0062] When sdf(x)>t or sdf(x)<-t, tsdf(x)=1 or -1.
[0063] Here t can be regarded as the threshold of the depth difference between the voxel x and the corresponding point P in the cross section. When the voxel is very close to the surface, the tsdf value is close to zero; when the voxel is very far from the surface, the tsdf value tends to 1 or -1. When initializing, the tsdf value of all voxels is set to 1, which is equivalent to no surface in this TSDF map.
[0064] S5, calculate the TSDF weight of the current frame.
[0065] Traverse all voxels (1~p), take a voxel at the three-dimensional position x in the world coordinate system as an example, the calculation formula of the weight w(p) is: w(x)=cos(θ) / camz(x), where θ is the angle between the projection light and the surface normal vector, that is, calculate the tsdf value and weight value of all voxels in this frame
[0066] S6, the current frame is fused with the global fusion result.
[0067] If the current frame is the first frame, the first frame is the result, otherwise, the current frame needs to be fused with the previous fusion result.
[0068] TSDF(p) represents the fused TSDF value of voxel p, W(p) is the fused weight value, tsdf(p) is the current TSDF value of voxel p, and w(p) is the weight value of the current frame. TSDF(p) is updated by tsdf(p), as shown in the following formulas 6 and 7:
[0069]
[0070] W(p)=W(p)+w(p) ; (Formula 7)
[0071] S7, rgb value calculation.
[0072] Let the RGB image corresponding to the depth map be RGB, then rgb(x)=RGB(Ix).
[0073] Reconstructing image data based on depth data to obtain three-dimensional image data includes but is not limited to key frame selection, depth map estimation, real-time three-dimensional reconstruction and other processing steps. Key frame selection aims to ensure that the fields of view corresponding to adjacent key frames are as close as possible. The depth map trajectory is based on the acquired depth data, and a depth map is formed by calculating and estimating the depth value of each pixel. Real-time three-dimensional reconstruction is to construct a three-dimensional model of an object based on the depth map and other relevant information. The implementation process of using depth data to perform three-dimensional reconstruction to obtain three-dimensional image data can be implemented by selecting a known algorithm under the technical teaching of this application, and this application does not impose any restrictions on this.
[0074] S105, correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the image data.
[0075] The temperature data of the image data of the target field of view can be collected and recorded by the infrared detector when the infrared image is collected, or obtained by processing and analyzing the infrared image that has been collected, and is the original temperature data required for imaging the infrared image corresponding to the target field of view, so that the temperature data has a one-to-one correspondence with the image data of the infrared image. The imaging of the infrared image is usually based on a fixed target distance to calculate the temperature of all imaging targets in the infrared image. In this way, the difference in relative distance between different imaging targets and the shooting device in the same target field of view will affect the imaging display of the imaging target in the infrared image, that is, in the infrared images obtained by shooting the same imaging target at different distances relative to the shooting device, the imaging target will also show temperature differences and color differences in the infrared image. In this way, the temperature data of the infrared image cannot reflect the distance difference of the imaging target in the infrared image due to the three-dimensional characteristics, which will affect the accuracy of the temperature data value of the imaging target in the infrared image, resulting in affecting the temperature measurement accuracy in various application scenarios where the infrared image is used to measure the temperature of the target object.
[0076] The depth data corresponding to the image data includes the shooting distances of all imaging targets in the image data. The temperature data is corrected by the depth data. The depth data of each imaging target in the image data is used to correct the imaging temperature difference caused by the distance difference of the target objects at different shooting distances, so as to obtain the reconstructed temperature field data of the image data. The target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement. Among them, the method of correcting the temperature data using the depth data can be to determine the linear proportion according to the shooting distance size of different imaging targets by measuring the temperature difference caused by the known distance difference; or the technical personnel in this field can correct the temperature data set using the known data correction or compensation method under the guidance of the technical concept of this application, and this application does not limit it here.
[0077] S107, applying the reconstructed temperature field data to the three-dimensional image data for display.
[0078] The three-dimensional image data is obtained by reconstructing the image data based on the depth data. It is understandable that the image data here may refer only to the visible light image, which is reconstructed based on the depth data to obtain the three-dimensional visible light image; only to the infrared image, which is reconstructed based on the depth data to obtain the three-dimensional infrared image; or may refer to the dual-light fusion image of the visible light image and the infrared image, which is reconstructed based on the depth data to obtain the three-dimensional dual-light fusion image. The reconstructed temperature field data is applied to the three-dimensional image data for display, and the data temperature value of the measured point of the imaging target of the three-dimensional shape and structure in the three-dimensional image data is determined based on the reconstructed temperature field data.
[0079] The three-dimensional imaging temperature measurement method provided in the above embodiment obtains image data, temperature data and depth data of the target field of view, the depth data includes the shooting distance information of all imaging targets in the image data, reconstructs the image data based on the depth data to obtain three-dimensional image data, uses the depth data to correct the temperature data, and corrects the temperature difference of imaging targets at different relative distances in the image data according to the depth data to obtain reconstructed temperature field data of the infrared image, so that target objects at different distances in the same field of view or different parts of the same target object can achieve accurate temperature measurement, and applies the reconstructed temperature field data to the three-dimensional image data for display, so that the accurate temperature value of the imaging target can be displayed in the three-dimensional image data to achieve the purpose of accurate temperature measurement.
[0080] In some embodiments, the image data includes a visible light image and an infrared image; the step S103 includes:
[0081] Reconstructing the visible light image based on the depth data to obtain a three-dimensional visible light image;
[0082] The infrared image is registered and fused with the three-dimensional visible light image to obtain a three-dimensional dual-light fused image.
[0083] In this embodiment, the image data includes an infrared image and a visible light image. The image data is reconstructed based on the depth data to obtain three-dimensional image data, which first includes reconstructing the visible light image based on the depth data to obtain a three-dimensional visible light image, and then registering and fusing the infrared image with the three-dimensional visible light image to obtain a three-dimensional dual-light fusion image. The data in the depth data and the data in the visible light image have a one-to-one correspondence. Using the depth data and the visible light image for three-dimensional reconstruction can avoid the registration processing step, and is also more conducive to the quality of the reconstructed three-dimensional image. The registration and fusion of infrared images and three-dimensional visible light images involve more spatial information and dimensions than the registration between two-dimensional images. It can be achieved by using known registration technologies, and one of the following can be used: first, a registration method based on feature point matching, which extracts key points from infrared images and three-dimensional visible light images, calculates the feature descriptors of these key points, and then uses the feature descriptors for matching to achieve registration. The feature points can be image edges, corners or markers; second, a registration method based on phase correlation, which uses Fourier transform to transform the infrared image and three-dimensional visible light image into the frequency domain, calculates the cross-correlation function between the two, and achieves image registration by maximizing the cross-correlation function; third, a registration method based on projection transformation, which is based on the geometric shape features of the image and achieves image registration by translating, rotating, scaling and other transformations on the image.
[0084] In the above embodiment, an implementation method is provided for first performing three-dimensional reconstruction of a visible light image and the acquired depth data to obtain a three-dimensional image, and then aligning and fusing the infrared image with the three-dimensional image to obtain a three-dimensional dual-light fusion image. This can simplify the reconstruction process of three-dimensional images of multiple types of image data including infrared images and improve the quality of the reconstructed three-dimensional dual-light fusion image.
[0085] In some embodiments, step S107 includes:
[0086] The three-dimensional dual-light fusion image is rendered and displayed based on the reconstructed temperature field data.
[0087] Among them, the three-dimensional image data is a three-dimensional dual-light fusion image obtained by three-dimensionally reconstructing the visible light image and the infrared image based on the depth data. The three-dimensional dual-light fusion image and the reconstructed temperature field data are rendered on the display interface, and the user can intuitively obtain the three-dimensional dual-light fusion image and temperature display. It should be noted that the three-dimensional image data may also refer to a three-dimensional visible light image obtained by three-dimensionally reconstructing a visible light image based on depth data, or may refer to a three-dimensional infrared image obtained by reconstructing an infrared image based on depth data. The three-dimensional imaging temperature measurement device using the three-dimensional imaging temperature measurement method provided in the embodiment of the present application may include multiple temperature measurement display modes: first, a visible light display mode, in which the reconstructed temperature field data is applied to a three-dimensional visible light image for display; second, an infrared display mode, in which the reconstructed temperature field data is applied to a three-dimensional infrared image for display; third, a fused image display mode, in which the reconstructed temperature field data is applied to a three-dimensional dual-light fusion image for display. The three-dimensional imaging temperature measurement device switches to the selected target temperature measurement display mode according to the user's selected instruction for the temperature measurement display mode, and displays the three-dimensional visible light image and the temperature data values of the measured points, displays the three-dimensional infrared image and the temperature data values of the measured points, or displays the three-dimensional dual-light fusion image and the temperature data values of the measured points in the display interface.
[0088] In some embodiments, step S105 includes:
[0089] For the temperature data value of the target point, correct it by using a temperature correction function according to the depth data value of the target point in the depth data to obtain the corrected temperature data value of the target point;
[0090] The reconstructed temperature field data of the infrared image is obtained according to the corrected temperature data values of the target points constituting each target object in the infrared image.
[0091] Among them, the depth data can be registered with the infrared image, or with the temperature data of the infrared image, to ensure a one-to-one correspondence between the data in the depth data and the temperature, and the target point can be any pixel point in the infrared image. The temperature correction function can be a correction function that compensates the temperature data value by a certain proportion based on the relative relationship between the shooting distance of the imaging target in the infrared image and the actual object distance. In this embodiment, the temperature correction function can be obtained by using known data statistical means under the guidance of the technical concept of using the registered depth data to correct it proposed in this application. As long as the temperature data of imaging targets at different distances can be compensated to a certain extent using their corresponding distance values, the accuracy of the reconstructed temperature field data can be improved to a certain extent, and this application does not limit it here.
[0092] In the above embodiment, for the temperature data value of each target point in the image, the corrected temperature data value is calculated by using the depth data value corresponding to the target point through a set temperature correction function, which can simplify the correction calculation.
[0093] In order to have a more comprehensive understanding of the three-dimensional imaging temperature measurement method provided in the embodiment of the present application, please refer to Figure 4 and Figure 5 , the three-dimensional imaging temperature measurement method is applied to a three-dimensional imaging temperature measurement device with a high-performance processor and multiple sensors as an example for explanation, and the three-dimensional imaging temperature measurement device includes a main processing unit 83, a depth sensor 85, a visible light module 82, an infrared module 81 and a display screen 86. The main processing unit 83 is a high-performance mobile processor platform, and the main processing unit 83 can access and process the above-mentioned various sensor data, including CPU, GPU and NPU for realizing data processing and software operation, and for executing the three-dimensional imaging temperature measurement method described in the embodiment of the present application; the depth sensor 85 can be a three-dimensional point cloud information acquisition module, which is used to collect three-dimensional spatial information on the surface of an object and to collect ranging data; the visible light module 82 can obtain a visible light image of a target object; the infrared module 81 is used to obtain an infrared image and a temperature data set of a target object; the display screen 86 is used to visualize the data processed by the main processing unit, or the three-dimensional imaging temperature measurement device can be visualized by a PC through a communication connection with a PC, or the visualization is directly processed by the PC.
[0094] The three-dimensional imaging temperature measurement method comprises the following steps:
[0095] S11, obtaining depth data of the target object from the depth sensor. The depth data mainly refers to three-dimensional point cloud space information.
[0096] S12, acquiring infrared image data through the infrared module.
[0097] S13, obtaining infrared temperature data through the infrared module.
[0098] S14, acquiring a visible light image through a visible light module.
[0099] S15, the main processing unit fuses the visible light image with the three-dimensional point cloud spatial information using reconstruction and image fusion algorithms to obtain a three-dimensional visible light image; wherein the image fusion mainly includes data preprocessing, registration and fusion algorithms.
[0100] S16, the main processing unit registers and fuses the infrared image data with the three-dimensional visible light image using image registration and image fusion algorithms to obtain a three-dimensional dual-light fusion image.
[0101] S17, the main processing unit corrects the infrared temperature data through a temperature correction algorithm based on the depth data to obtain reconstructed temperature field data.
[0102] S18, the main processing unit renders the three-dimensional dual-light fusion image and the reconstructed temperature field data on the display interface. The user can intuitively obtain the three-dimensional dual-light fusion image and the accurate temperature display of the measured point.
[0103] The three-dimensional imaging temperature measurement method provided in the embodiment of the present application makes full use of the feature data and characteristics of various types of image data, and uses the depth data obtained during image data acquisition to reconstruct high-quality three-dimensional images and high-precision temperature field data for the intuitive display of high-precision temperature data of the measured points of the imaging target in the three-dimensional image, so as to meet the needs of major applications such as equipment operation and maintenance and overhaul in industries such as industry and electric power.
[0104] On the other hand, please refer to Figure 6 , an imaging temperature measurement device 20 is also provided, comprising a memory 22, a processor 23 and a data acquisition module 21 connected to the processor 23. The data acquisition module 21 is used to collect image data, temperature data and depth data of the target field of view; the memory 22 stores a computer program; when the computer program is executed by the processor 23, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0105] Optionally, the data acquisition module 21 includes an infrared module 212 and a depth data acquisition module 211. The infrared module 212 is used to acquire an infrared image of the target field of view and corresponding temperature data. The depth data acquisition module 211 includes one of the following:
[0106] The depth data acquisition module 211 includes a depth sensor, which is used to acquire depth data corresponding to the target field of view and apply the reconstructed temperature field data to a three-dimensional infrared image for display;
[0107] The depth data acquisition module 211 includes a visible light and depth data acquisition device, which is used to acquire a visible light image of the target field of view and depth data corresponding to the visible light image. The display method of the reconstructed temperature field data includes one of the following: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display;
[0108] The depth data acquisition module 211 includes a visible light module and a depth sensor, the depth sensor is used to acquire the depth data corresponding to the infrared image, and the visible light sensor is used to acquire the visible light image of the target field of view; the display method of the reconstructed temperature field data includes one of the following: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display.
[0109] Optionally, the depth sensor mainly refers to a point cloud sensor, which can be selected from one of the following: TOF sensor, laser ranging sensor, radar ranging sensor. The visible light and depth data acquisition device can be selected from one of the following: TOF camera, structured light / coded light camera, stereo vision camera. TOF sensor, laser ranging sensor, radar ranging sensor can be divided into dot array sensor and surface array sensor.
[0110] Optionally, the infrared module 212 is selected from one of the following: a short-wave infrared module, a medium-wave infrared module, and a long-wave infrared module.
[0111] Optionally, the infrared module 212 is selected from one of the following: a cooling infrared module and a non-cooling infrared module.
[0112] Optionally, the three-dimensional imaging temperature measurement device is a handheld infrared device.
[0113] Optionally, the three-dimensional imaging temperature measuring device also includes a communication module, and the three-dimensional imaging temperature measuring device is communicatively connected to the terminal device through the communication module, and can send the reconstructed temperature field data and the three-dimensional image data to the terminal device for display by the terminal device.
[0114] On the other hand, please refer to Figure 7, an imaging temperature measurement system is also provided, comprising an imaging temperature measurement device 20' and a terminal device 30 connected to the imaging temperature measurement device 20' in communication. The imaging temperature measurement device 20' comprises a data acquisition module 21, which is used to collect image data, temperature data and depth data of the target field of view and send them to the terminal device 30; the terminal device 30 comprises a memory 31 and a processor 32, the memory 31 stores a computer program, and when the computer program is executed by the processor 32, the three-dimensional imaging temperature measurement method described in any embodiment of the present application is implemented.
[0115] Optionally, the imaging temperature measuring device 20' also includes a display module 213; the imaging temperature measuring device 20' is also used to receive the reconstructed temperature field data and the three-dimensional image data returned by the terminal device 30, and display the reconstructed temperature field data on the display module 213 using one of the following display methods: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display.
[0116] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the three-dimensional imaging temperature measurement method described in any embodiment of the present application.
[0117] On the other hand, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the process of each of the above-mentioned three-dimensional imaging temperature measurement method embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0118] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, infrared thermal imager or network equipment, etc.) to execute the methods described in each embodiment of the present invention.
[0120] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A three-dimensional imaging temperature measurement method, characterized in that: include: Acquire image data, temperature data, and depth data of the target field of view; Reconstructing the image data based on the depth data to obtain three-dimensional image data; Correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the image data; The reconstructed temperature field data is applied to the three-dimensional image data for display.
2. The three-dimensional imaging temperature measurement method according to claim 1, characterized in that: The image data includes a visible light image and an infrared image; and reconstructing the image data based on the depth data to obtain three-dimensional image data includes: Reconstructing the visible light image based on the depth data to obtain a three-dimensional visible light image; The infrared image is registered and fused with the three-dimensional visible light image to obtain a three-dimensional dual-light fused image.
3. The three-dimensional imaging temperature measurement method according to claim 2, characterized in that: The applying the reconstructed temperature field data to the three-dimensional image data for display comprises: The three-dimensional dual-light fusion image is rendered and displayed based on the reconstructed temperature field data.
4. The three-dimensional imaging temperature measurement method according to claim 1, characterized in that: The step of correcting the temperature data by using the depth data to obtain reconstructed temperature field data of the image data includes: For the temperature data value of the target point, correct it by using a temperature correction function according to the depth data value of the target point in the depth data to obtain the corrected temperature data value of the target point; The reconstructed temperature field data of the infrared image is obtained according to the corrected temperature data values of the target points constituting each target object in the infrared image.
5. A three-dimensional imaging temperature measurement device, characterized in that: It includes a memory, a processor and a data acquisition module connected to the processor; The data acquisition module is used to collect image data, temperature data and depth data of the target field of view; The memory stores a computer program; When the computer program is executed by the processor, the three-dimensional imaging temperature measurement method according to any one of claims 1 to 4 is implemented.
6. The three-dimensional imaging temperature measurement device according to claim 5, characterized in that: The data acquisition module includes an infrared module and a depth data acquisition module; The infrared module is used to collect the infrared image of the target field of view and the corresponding temperature data; The depth data acquisition module includes one of the following: The depth data acquisition module includes a depth sensor, which is used to acquire depth data corresponding to the target field of view, and apply the reconstructed temperature field data to a three-dimensional infrared image for display; The depth data acquisition module includes a visible light and depth data acquisition device, which is used to acquire a visible light image of the target field of view and depth data corresponding to the visible light image. The display method of the reconstructed temperature field data includes one of the following: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display; The depth data acquisition module includes a visible light module and a depth sensor, the depth sensor is used to acquire the depth data corresponding to the infrared image, and the visible light module is used to acquire the visible light image of the target field of view; the display method of the reconstructed temperature field data includes one of the following: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display.
7. The three-dimensional imaging temperature measurement device according to claim 6, characterized in that: The depth sensor is selected from one of the following: a TOF sensor, a laser ranging sensor, and a radar ranging sensor; The visible light and depth data acquisition device is selected from one of the following: a TOF camera, a structured light / coded light camera, and a stereo vision camera.
8. The three-dimensional imaging temperature measurement device according to claim 6, characterized in that: The infrared module is selected from one of the following: Short-wave infrared module, medium-wave infrared module, long-wave infrared module.
9. The three-dimensional imaging temperature measurement device according to claim 6, characterized in that: The infrared module is selected from one of the following: a refrigerated infrared module and a non-refrigerated infrared module.
10. The three-dimensional imaging temperature measurement device according to claim 5, characterized in that: It also includes a communication module, through which the three-dimensional imaging temperature measurement device is connected to a terminal device for communication, and the reconstructed temperature field data and the three-dimensional image data are sent to the terminal device for display.
11. The three-dimensional imaging temperature measurement device according to claim 5, characterized in that: The three-dimensional imaging temperature measurement device is a handheld infrared device.
12. A three-dimensional imaging temperature measurement system, characterized in that: It includes an imaging temperature measuring device and a terminal device connected to the imaging temperature measuring device for communication; The imaging temperature measurement device includes a data acquisition module, which is used to collect image data, temperature data and depth data of the target field of view and send them to the terminal device; The terminal device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the three-dimensional imaging temperature measurement method according to any one of claims 1 to 4 is implemented.
13. The three-dimensional imaging temperature measurement system according to claim 12, characterized in that: The imaging temperature measurement device also includes a display module; The imaging temperature measurement device is also used to receive the reconstructed temperature field data and the three-dimensional image data returned by the terminal device, and display the reconstructed temperature field data on the display module using one of the following display methods: applying the reconstructed temperature field data to a three-dimensional infrared image for display; applying the reconstructed temperature field data to a registered three-dimensional visible light image for display; applying the reconstructed temperature field data to a three-dimensional dual-light fusion image for display.
14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the three-dimensional imaging temperature measurement method according to any one of claims 1 to 4 is implemented.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the three-dimensional imaging temperature measurement method according to any one of claims 1 to 4 is implemented.
Citation Information
Cited By
Thermal image processing methods, computer-readable storage media, and electronic devices
TWI913125B
