Laser radar and infrared thermal imager dual light fusion three-dimensional temperature measuring device and method

Abstract

This application provides a three-dimensional temperature measurement device and method that integrates lidar and infrared thermal imager. It includes: a dual-light sensor module for acquiring three-dimensional point cloud data and infrared thermal image data of a target scene; a synchronization control unit for outputting a synchronization trigger signal to the dual-light sensor module based on a unified clock, and injecting a corresponding unified timestamp when the three-dimensional point cloud data and infrared thermal image data are formed; a data processing unit for establishing a mapping relationship between three-dimensional points and infrared pixels, and generating distance mapping data corresponding to each infrared pixel based on the mapping relationship; the data processing unit is also used to perform pixel-by-pixel compensation on the original temperature measurement data in the infrared thermal image data according to the distance mapping data to generate corrected temperature distribution data; and a three-dimensional temperature field output unit for generating three-dimensional temperature field data carrying temperature attributes. This application can improve the real-time performance of distance compensation, enhance the accuracy of dual-light synchronization, and strengthen the three-dimensional temperature field output capability.

Description

Technical Field

[0001] This application relates to the field of non-contact infrared temperature measurement and multi-sensor fusion technology, and in particular to a three-dimensional temperature measurement device and method that fuses the light from a lidar and an infrared thermal imager. Background Technology

[0002] Infrared thermal imaging temperature measurement technology features non-contact operation, fast response speed, and area array temperature measurement, and has been widely used in scenarios such as industrial equipment inspection, power facility monitoring, building energy efficiency diagnosis, and specific target reconnaissance. Traditional infrared thermal imagers can acquire the two-dimensional temperature distribution of a target surface, but it is difficult to simultaneously acquire the target's three-dimensional spatial position and geometry. This results in temperature measurement results lacking distance and spatial reference, and cannot directly meet the needs of three-dimensional temperature analysis, spatial positioning, and visualization.

[0003] With the development of lidar technology, integrating lidar with infrared thermal imagers to achieve 3D infrared temperature measurement has become an important direction. Existing 3D temperature measurement devices typically acquire temperature images using infrared thermal imagers, obtain point cloud data using lidar, and then perform temperature correction and data fusion using blackbody calibration or backend software processing. While some solutions can achieve multi-sensor hardware integration and utilize external blackbody for periodic temperature drift compensation, they primarily focus on the temperature measurement accuracy of the infrared thermal imager itself, failing to fully utilize the real-time distance information acquired by the lidar for pixel-by-pixel distance temperature compensation of the infrared thermal image.

[0004] Therefore, the existing technology has at least the following problems: First, it lacks a real-time distance temperature compensation processing structure embedded in the device, making it difficult to continuously correct infrared temperature measurement errors at the hardware level; second, the lidar point cloud data and infrared thermal image data lack a unified synchronous triggering and unified timestamp injection mechanism, which easily leads to spatiotemporal misalignment; third, the compensated temperature data and the three-dimensional point cloud lack real-time fusion and rendering output capabilities, making it difficult to form three-dimensional temperature field data that can be directly used for on-site analysis. Summary of the Invention

[0005] In view of this, the present application provides a three-dimensional temperature measurement device and method that fuses LiDAR and infrared thermal imager to solve the problems of poor real-time performance of distance temperature compensation, low accuracy of dual-light data synchronization, and insufficient three-dimensional temperature field output capability in the prior art.

[0006] This application provides a three-dimensional temperature measurement device that fuses LiDAR and infrared thermal imager, comprising: a dual-light sensor module for acquiring three-dimensional point cloud data and infrared thermal image data of a target scene under the same rigid mounting reference; a synchronization control unit for outputting a synchronization trigger signal to the dual-light sensor module based on a unified clock, and injecting a corresponding unified timestamp when the three-dimensional point cloud data and infrared thermal image data are formed; a data processing unit for projecting the three-dimensional point cloud data onto the pixel coordinate system of the infrared thermal image data according to preset spatiotemporal calibration parameters, establishing a mapping relationship between three-dimensional points and infrared pixels, and generating distance mapping data corresponding to each infrared pixel based on the mapping relationship; the data processing unit is also used to call distance temperature compensation parameters fixed on the device end, and perform pixel-by-pixel compensation on the original temperature measurement data in the infrared thermal image data according to the distance mapping data to generate corrected temperature distribution data; a three-dimensional temperature field output unit for mapping the corrected temperature distribution data to the corresponding three-dimensional points in the three-dimensional point cloud data to generate three-dimensional temperature field data carrying temperature attributes; and a communication and storage unit for associating, encapsulating, transmitting, and storing the three-dimensional temperature field data, the corrected temperature distribution data, and the unified timestamp.

[0007] The second aspect of this application provides a three-dimensional temperature measurement method based on the system of the first aspect, comprising: synchronously acquiring three-dimensional point cloud data and infrared thermal image data of a target scene based on a unified clock driven dual-light sensor module, and injecting a unified timestamp into the three-dimensional point cloud data and infrared thermal image data; projecting the three-dimensional point cloud data onto the pixel coordinate system of the infrared thermal image data according to preset spatiotemporal calibration parameters, establishing a mapping relationship between three-dimensional points and infrared pixels, and generating distance mapping data corresponding to each infrared pixel; calling distance temperature compensation parameters fixed on the device end, and performing pixel-by-pixel compensation on the original temperature measurement data of the infrared pixels according to the distance mapping data to generate corrected temperature distribution data; mapping the corrected temperature distribution data to the corresponding three-dimensional points in the three-dimensional point cloud data, generating and outputting three-dimensional temperature field data carrying temperature attributes.

[0008] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: A dual-light sensor module is used to acquire 3D point cloud data and infrared thermal image data of a target scene under the same rigid mounting reference. A synchronization control unit is used to output a synchronization trigger signal to the dual-light sensor module based on a unified clock and inject a corresponding unified timestamp when the 3D point cloud data and infrared thermal image data are formed. A data processing unit is used to project the 3D point cloud data onto the pixel coordinate system of the infrared thermal image data according to preset spatiotemporal calibration parameters, establish a mapping relationship between 3D points and infrared pixels, and generate distance mapping data corresponding to each infrared pixel based on the mapping relationship. The data processing unit is also used to call the distance temperature compensation parameters fixed at the device end, and perform pixel-by-pixel compensation on the original temperature measurement data in the infrared thermal image data according to the distance mapping data to generate corrected temperature distribution data. A 3D temperature field output unit is used to map the corrected temperature distribution data to the corresponding 3D points in the 3D point cloud data to generate 3D temperature field data carrying temperature attributes. A communication and storage unit is used to associate, encapsulate, transmit, and store the 3D temperature field data, corrected temperature distribution data, and unified timestamp. This application can improve the real-time performance of distance compensation, enhance the accuracy of dual-light synchronization, and strengthen the 3D temperature field output capability. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a block diagram of the overall hardware architecture of the three-dimensional temperature measurement device provided in the embodiments of this application; Figure 2 This is a schematic diagram of the overall structure of the three-dimensional temperature measurement device that fuses lidar and infrared thermal imager according to an embodiment of this application. Figure 3 This is a schematic diagram of the installation structure of the dual-light sensor module provided in the embodiments of this application; Figure 4 This is a schematic diagram of the FPGA parallel architecture of the synchronization control unit provided in the embodiments of this application; Figure 5 This is a flowchart illustrating the three-dimensional temperature measurement method based on the dual-light fusion of lidar and infrared thermal imager provided in this application embodiment. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] Infrared thermal imaging temperature measurement technology has been widely used in fields such as industrial equipment inspection, power facility monitoring, building energy efficiency diagnosis, and military reconnaissance due to its advantages such as non-contact operation, fast response speed, and ability to perform area array temperature measurement. However, traditional infrared thermal imagers can only acquire the two-dimensional temperature distribution of the target surface and cannot simultaneously acquire the target's three-dimensional geometric information. This results in a lack of spatial reference in the temperature measurement results, making it difficult to meet the needs of three-dimensional temperature analysis and visualization.

[0013] With the development of lidar technology, it has become possible to achieve three-dimensional infrared temperature measurement by combining infrared thermal imagers with lidar. In the prior art, Chinese patent application CN202111608741 discloses a multi-sensor fusion three-dimensional temperature measurement device, including a data acquisition unit and a blackbody. The data acquisition unit housing is equipped with a visible light lens module, an infrared thermal imager, and a three-dimensional lidar. Through periodic real-time calibration using the thermal imager and a programmable blackbody, high-precision temperature measurement with an uncooled thermal imager is achieved. This solution primarily focuses on hardware integration and temperature drift compensation using a programmable blackbody, but it has the following shortcomings: First, it does not address how to utilize distance information acquired by lidar for real-time distance-temperature compensation of infrared temperature measurement results. When the distance between the target and the temperature measuring device changes, the accuracy of infrared temperature measurement will significantly decrease due to factors such as atmospheric attenuation. Second, it does not provide a high-precision synchronous triggering mechanism between multiple sensors, resulting in spatiotemporal misalignment between lidar point cloud data and infrared thermal image data, affecting fusion accuracy. Third, the temperature compensation in this solution relies entirely on periodic calibration using an external blackbody, making it impossible to achieve pixel-by-pixel real-time compensation based on distance information. This results in a slow response speed and makes it difficult to meet the temperature measurement needs of dynamic scenarios.

[0014] Furthermore, FPGA-based multi-sensor synchronous triggering technology has been applied in fields such as autonomous driving. Lattice Semiconductor's 3D sensor fusion reference design utilizes FPGA to achieve synchronous triggering and fusion processing of sensors such as LiDAR, radar, and cameras. However, this solution is mainly aimed at environmental perception scenarios in autonomous driving and does not specifically address the temperature compensation requirements of infrared thermometry, nor does it involve hardware-level solidification of the distance temperature compensation module.

[0015] In summary, existing 3D temperature measurement devices still have shortcomings in the following aspects: First, they lack an embedded real-time distance-temperature compensation processing module, making it impossible to implement the compensation algorithm at the hardware level; second, the lack of a high-precision unified timestamp injection mechanism for multi-sensor data acquisition makes it difficult to guarantee the spatiotemporal consistency between lidar point cloud and infrared thermal image data; third, they lack a dedicated module for real-time rendering and output of the 3D temperature field. Therefore, there is an urgent need for a dual-light fusion 3D temperature measurement device that can integrate real-time distance-temperature compensation functionality at the hardware level, possess high-precision synchronous control capabilities, and support real-time rendering and output of the 3D temperature field.

[0016] In view of the problems existing in the prior art, this application provides a three-dimensional temperature measurement device that fuses LiDAR and infrared thermal imager. The device includes a dual-light sensor module (940nm solid-state LiDAR and uncooled vanadium oxide infrared thermal imager), a synchronization control unit based on an FPGA parallel architecture (driven by optically isolated circuits to trigger synchronization and inject a unified timestamp), a data processing unit (embedded with a spatiotemporal calibration module, a point cloud-temperature mapping module, a hardware-level solidified distance compensation module, and a three-dimensional temperature field rendering module), and a communication and storage unit. This application achieves pixel-by-pixel real-time temperature compensation without relying on an external blackbody by solidifying the distance-temperature error compensation algorithm at the hardware level. Simultaneously, the unified timestamp injection mechanism ensures the spatiotemporal consistency of point cloud and thermal image data, enabling direct output of a high-precision three-dimensional temperature field model at the device end. This device is suitable for various scenarios such as industrial equipment inspection, power facility monitoring, and building energy efficiency diagnosis.

[0017] This application aims to address the problems of insufficient temperature compensation accuracy and poor spatiotemporal consistency between point cloud and thermal image data in existing three-dimensional temperature measurement devices due to the lack of an embedded real-time distance temperature compensation module and a high-precision multi-sensor synchronous control mechanism. It provides a three-dimensional temperature measurement device that integrates real-time distance temperature compensation and high-precision synchronous control functions at the hardware level, combining a lidar and an infrared thermal imager.

[0018] The following description, in conjunction with the accompanying drawings and embodiments, illustrates the product structure of the three-dimensional temperature measurement device of this application in a real-world scenario. Figure 1 This is a block diagram of the overall hardware architecture of the three-dimensional temperature measurement device provided in the embodiments of this application, such as... Figure 1 As shown, the device may specifically include the following components: (a) Dual-light sensor module The dual-light sensor module includes a lidar module and an infrared thermal imager module, both fixedly mounted on the same rigid bracket. The lidar module is used to acquire 3D point cloud data of the target scene, while the infrared thermal imager module is used to simultaneously acquire infrared thermal images of the same target scene.

[0019] Furthermore, the lidar module is a 940nm wavelength solid-state lidar. Compared to a 905nm wavelength lidar, the 940nm wavelength laser source has a higher human eye safety threshold and a longer effective ranging capability. The all-solid-state solution has the advantages of small size, low cost, and high precision, and can provide centimeter-level or even millimeter-level ranging accuracy within a working distance range of 1 to 30 meters.

[0020] Furthermore, the infrared thermal imager module employs an uncooled vanadium oxide focal plane array detector, with a temperature measurement range of -20℃ to 1500℃ and a resolution of 384×256 or 640×512 pixels or higher.

[0021] Furthermore, the dual-light sensor module may optionally include a visible light camera module for acquiring visible light images of the target scene, providing auxiliary information for subsequent data analysis and visualization.

[0022] Furthermore, the lidar module, infrared thermal imager module, and optional visible light camera module are fixedly connected by a rigid bracket, with an optical axis alignment deviation of no more than 0.1° and a field-of-view overlap of no less than 80%. The high-precision alignment of the optical axis can reduce the initial error of multi-sensor data registration, and the high field-of-view overlap ensures the spatial coverage integrity of the fused data.

[0023] (ii) Synchronization control unit The synchronous control unit is built on an FPGA parallel architecture and drives the lidar module and infrared thermal imager module to synchronously trigger acquisition through an optically isolated circuit.

[0024] Furthermore, the synchronization control unit includes a unified timestamp injection module. This module injects FPGA counter values ​​as unified timestamps when the lidar point cloud data acquisition is completed and the infrared thermal imager A / D conversion is completed, respectively. Through a cross-correlation algorithm, the lidar scanning start signal and the infrared thermal imager sampling time are aligned to ensure strict temporal alignment between the lidar 3D point cloud data and the infrared thermal image data.

[0025] Furthermore, the synchronization control unit adopts a dual redundant clock source design, with the main clock source and the backup clock source serving as backups for each other. When the main clock source fails, it automatically switches to the backup clock source to ensure high reliability of the synchronization control.

[0026] Furthermore, the optocoupler isolation circuit provides electrical isolation between the FPGA and each sensor, effectively suppressing electromagnetic interference, preventing sensor noise from affecting the accuracy of synchronous control, and ensuring the consistency of synchronous triggering of multiple sensors.

[0027] (III) Data Processing Unit The data processing unit is a hardware processing module embedded in the device, including a spatiotemporal calibration module, a point cloud-temperature mapping module, a distance compensation module, and a three-dimensional temperature field rendering module.

[0028] Furthermore, the spatiotemporal calibration module is used to solve for the rotation and translation transformation matrix between the coordinate system of the lidar module and the coordinate system of the infrared thermal imager module, as well as the intrinsic parameter matrix of the infrared thermal imager module. The calibration method adopts a two-stage strategy of coarse registration and fine registration: the coarse registration stage uses a temperature-controlled calibration target to obtain preliminary spatial transformation parameters, and the fine registration stage utilizes natural scene features for nonlinear iterative optimization.

[0029] Furthermore, the point cloud-temperature mapping module is used to project the 3D point cloud acquired by the lidar module onto the pixel coordinate system of the infrared thermal image based on the spatial transformation relationship obtained by the spatiotemporal calibration module, establishing a mapping correspondence between the 3D points and the pixels of the infrared thermal image, and obtaining the distance information corresponding to each pixel in the infrared thermal image. When multiple 3D points are mapped to the same infrared thermal image pixel position, the 3D point with the smallest distance value is selected as the valid point corresponding to that pixel.

[0030] Furthermore, the distance compensation module is a hardware-level compensation module embedded in the device. It performs real-time, pixel-by-pixel temperature compensation on the infrared thermal image based on the distance information corresponding to each pixel obtained from the point cloud-temperature mapping module and a pre-defined distance-temperature error compensation model, outputting a corrected temperature distribution image. The distance-temperature error compensation model is obtained through calibration experiments using a standard blackbody radiation source under multiple known real temperature and distance conditions, employing one or more combinations of exponential function models, polynomial function models, and piecewise function models. The distance compensation module embeds the compensation algorithm into hardware logic circuits or embedded processor firmware, enabling real-time operation at the hardware level without relying on an external host computer for software calculations.

[0031] Furthermore, the 3D temperature field rendering module is used to reverse map the temperature data in the corrected temperature distribution image output by the distance compensation module to the corresponding 3D points in the 3D point cloud of the lidar module, generate a 3D point cloud model with temperature information (i.e., a 3D temperature field model), and perform pseudo-color rendering to achieve real-time visualization output of the 3D temperature field.

[0032] Furthermore, the data processing unit also includes an emissivity dynamic correction module, which combines the multispectral material classification results with the surface normal direction information of the lidar to adaptively optimize the emissivity of the target surface and incorporate it into the temperature compensation calculation process to further improve the temperature measurement accuracy.

[0033] Furthermore, the LiDAR module supports dynamic scanning mode switching: a high-speed scanning mode with a frame rate of no less than 100Hz is activated for moving targets; a high-precision mode with a frame rate of no less than 30Hz is activated for static targets. During mode switching, the synchronization control unit automatically adjusts the trigger frequency. The data processing unit adaptively adjusts the update rate of the distance compensation module according to the current scanning mode.

[0034] (iv) Communication and storage unit The communication and storage unit supports the transmission and local / cloud storage of multimodal data. Multimodal data includes lidar 3D point cloud data, infrared thermal image data, compensated and corrected temperature distribution data, 3D temperature field model data, and timestamp information.

[0035] Furthermore, the communication and storage unit supports data transmission via multiple communication protocols such as Ethernet, Wi-Fi, and 5G, enabling real-time uploading of collected data and compensation results to a host computer or cloud platform; it also integrates a local large-capacity storage module, supporting local caching and offline export of data.

[0036] Compared with the prior art, this application has the following beneficial effects: (1) Embedded real-time distance-temperature compensation. This application integrates a distance compensation module in the data processing unit, and solidifies the distance-temperature error compensation algorithm into hardware logic circuits or embedded firmware, realizing pixel-by-pixel real-time temperature compensation at the hardware level. Unlike the existing technology that relies on an external blackbody for periodic offline calibration compensation, the compensation module of this application can use the distance information acquired by the lidar in real time to perform dynamic, pixel-by-pixel continuous compensation on the infrared temperature measurement results, which significantly improves the temperature measurement accuracy and response speed.

[0037] (2) High-precision multi-sensor synchronization. This application adopts a synchronization control unit based on FPGA parallel architecture, which drives the synchronous triggering of multiple sensors through optocoupler isolation circuit and injects a unified timestamp to ensure the spatiotemporal consistency of lidar point cloud and infrared thermal image data, effectively solving the problem of spatiotemporal misalignment of multi-sensor data.

[0038] (3) Real-time rendering of three-dimensional temperature field. This application integrates a three-dimensional temperature field rendering module, which can complete the reverse mapping and pseudo-color rendering of the compensated temperature data to the three-dimensional point cloud in real time on the device, and output a three-dimensional temperature field model with high-precision temperature information, providing direct support for three-dimensional visualization and fault diagnosis, without relying on an external host computer for post-processing.

[0039] (4) High system integration and convenient deployment. This application integrates the dual-light sensor module, synchronous control unit, data processing unit (including distance compensation module) and communication storage unit into one unit to form a complete independent temperature measurement device, which can be quickly deployed and used in various scenarios such as industrial sites, power facilities, and building environments.

[0040] (5) High hardware reliability. The synchronous control unit adopts a dual redundant clock source design to ensure the high reliability of the system; the design of the optocoupler isolation circuit effectively suppresses electromagnetic interference and improves the stability and consistency of multi-sensor synchronous triggering.

[0041] The following description, in conjunction with the accompanying drawings, illustrates the specific structure of the three-dimensional temperature measurement device based on the dual-light fusion of lidar and infrared thermal imager in a real-world scenario. Figure 2 This is a schematic diagram of the overall structure of the three-dimensional temperature measurement device that fuses lidar and infrared thermal imager according to an embodiment of this application. Figure 1 As shown, the device may specifically include the following: The dual-light sensor module 201 is used to acquire three-dimensional point cloud data and infrared thermal image data of the target scene under the same rigid mounting reference. The synchronization control unit 202 is used to output a synchronization trigger signal to the dual-light sensor module based on a unified clock, and to inject a corresponding unified timestamp when the three-dimensional point cloud data and infrared thermal image data are formed. The data processing unit 203 is used to project the three-dimensional point cloud data onto the pixel coordinate system of the infrared thermal image data according to the preset spatiotemporal calibration parameters, establish the mapping relationship between the three-dimensional points and the infrared pixels, and generate the distance mapping data corresponding to each infrared pixel based on the mapping relationship. The data processing unit 203 is also used to call the distance temperature compensation parameters solidified at the device end, and to perform pixel-by-pixel compensation on the original temperature measurement data in the infrared thermal image data according to the distance mapping data to generate corrected temperature distribution data. The three-dimensional temperature field output unit 204 is used to map the corrected temperature distribution data to the corresponding three-dimensional points in the three-dimensional point cloud data to generate three-dimensional temperature field data carrying temperature attributes. The communication and storage unit 205 is used to associate, encapsulate, transmit, and store three-dimensional temperature field data, corrected temperature distribution data, and unified timestamps.

[0042] In some embodiments, the synchronization control unit is specifically used for: The synchronization control unit generates multiple synchronization trigger signals based on the unified counting clock of the programmable logic device, and drives the lidar module and infrared thermal imager module in the dual-light sensor module to perform synchronous acquisition through optocoupler isolation circuits respectively. When the 3D point cloud data acquisition is completed and the infrared thermal image data conversion is completed, the synchronous control unit captures the count value corresponding to the unified counting clock and writes the corresponding count value into the data frame of the 3D point cloud data and the infrared thermal image data respectively. The synchronization control unit aligns the start time of lidar scanning and the sampling time of infrared thermal images based on a unified timestamp.

[0043] Specifically, in this embodiment, the synchronization control unit is located on the internal control board of the device and is connected to the lidar module and the infrared thermal imager module, respectively. The synchronization control unit uses a programmable logic device (PLD) as the timing control core. After power-on initialization, it loads unified counting clock parameters, trigger period parameters, sensor response delay parameters, and data frame writing rules. The unified counting clock is provided by a high-precision clock source and forms a continuously increasing count value within the PLD, serving as a unified time reference for the dual-light acquisition data.

[0044] During synchronous acquisition, the programmable logic device (PLD) generates two or more synchronous trigger signals according to a preset trigger cycle. One trigger signal is output to the LiDAR module, causing it to begin scanning the target scene according to the set scanning cycle. The other trigger signal is output to the infrared thermal imager module, causing it to begin infrared exposure and temperature sampling at the same time reference. Before entering the LiDAR and infrared thermal imager modules, the trigger signals undergo electrical isolation via an optocoupler isolation circuit. This circuit separates the electrical circuits of the control side from the sensor side, reducing the impact of electromagnetic interference from equipment such as motors, transformers, and switching power supplies in industrial environments on the trigger pulse edges. For devices equipped with a visible light camera module, the synchronous control unit can also output a shared trigger signal to the visible light camera module, ensuring that the visible light image, 3D point cloud data, and infrared thermal image data form a data set under the same time reference.

[0045] When the LiDAR module completes the acquisition of a frame of 3D point cloud data, it sends a point cloud frame end signal to the synchronization control unit. The synchronization control unit captures the count value corresponding to the current unified counting clock and writes this count value into the data frame header of the 3D point cloud data. When the infrared thermal imager module completes the analog-to-digital conversion of a frame of infrared thermal image data, the synchronization control unit captures the current count value under the same unified counting clock and writes this count value into the data frame header of the infrared thermal image data. Thus, each frame of 3D point cloud data and each frame of infrared thermal image data carries a comparable unified timestamp, allowing subsequent data processing units to group the dual-light acquisition data into corresponding frame groups according to the unified timestamp.

[0046] For example, in an industrial motor inspection scenario, the device operates at an infrared sampling frequency of 30 frames per second, and the LiDAR module synchronously outputs 3D point cloud data of the same target motor surface. The synchronization control unit writes a count value when the 120th frame of LiDAR point cloud acquisition is completed, and writes the corresponding count value when the infrared thermal image data conversion is completed within the same acquisition cycle. The data processing unit determines the time offset between the LiDAR scan start time and the infrared thermal image sampling time based on the difference between the two count values, and, combined with the cross-correlation results of the scan start signal and the infrared sampling signal, aligns the point cloud frames and thermal image frames to form a group of synchronized data frames for subsequent distance mapping and temperature compensation.

[0047] When the master clock source output is abnormal or the continuity of the count value does not meet the preset conditions, the synchronization control unit can switch to the backup clock source and record the switching status in the data frame append field, enabling subsequent data processing units to identify the clock source within the acquisition cycle. Through the above-mentioned synchronization triggering, isolation driving, unified timestamp injection, and sampling time alignment processing, this embodiment can improve the temporal consistency between 3D point cloud data and infrared thermal image data, reduce the misalignment error of dual-light data in dynamic temperature measurement scenarios, and provide a stable data foundation for pixel-by-pixel distance temperature compensation.

[0048] In some embodiments, the data processing unit is specifically used for: The data processing unit acquires 3D point cloud data and infrared thermal image data with a unified timestamp correspondence, and calls the preset spatiotemporal calibration parameters composed of spatial extrinsic parameters and infrared imaging intrinsic parameters. The data processing unit converts the 3D point cloud data to the infrared thermal imager coordinate system based on the spatial extrinsic parameters, and projects the converted 3D points to the pixel coordinate system of the infrared thermal image data based on the infrared imaging intrinsic parameters. The data processing unit performs field-of-view validity screening on the projected 3D points, establishes a mapping relationship between valid 3D points and corresponding infrared pixels, and generates the distance value corresponding to the infrared pixels based on the spatial distance of the valid 3D points. When the same infrared pixel corresponds to multiple valid 3D points, the data processing unit selects the valid 3D point with the smallest distance value to generate the corresponding distance value, and forms distance mapping data based on the distance value corresponding to each infrared pixel.

[0049] Specifically, in this embodiment, the data processing unit is located on a heterogeneous computing board inside the device and is communicatively connected to the synchronization control unit, the lidar module, and the infrared thermal imager module. After the synchronization control unit completes the unified timestamp injection, the data processing unit reads the 3D point cloud data and infrared thermal image data within the same acquisition period from the cache queue according to the unified timestamp, and groups data frames with timestamp differences within a preset tolerance range into dual-light synchronous data groups. For data frames with timestamp differences exceeding the preset tolerance range, the data processing unit marks the corresponding data frame as a frame to be corrected or discards it to avoid asynchronous data from participating in distance mapping.

[0050] After the device completes factory calibration or on-site recalibration, the data processing unit calls the preset spatiotemporal calibration parameters. These preset spatiotemporal calibration parameters include spatial extrinsic parameters from the lidar module coordinate system to the infrared thermal imager module coordinate system, as well as the infrared imaging intrinsic parameters of the infrared thermal imager module. The spatial extrinsic parameters can be constructed using rotational and translational transformations, while the infrared imaging intrinsic parameters may include focal length, principal point position, and distortion correction parameters. During calibration, an initial correspondence between point cloud reflection characteristics and infrared heating characteristics can be obtained first using a temperature-controlled calibration target. Then, iterative optimization is performed by combining structural features such as contour edges, equipment edges, and pipe boundaries in the natural scene to obtain spatiotemporal calibration parameters suitable for the actual installation posture.

[0051] During the mapping process, the data processing unit first performs basic screening on the 3D point cloud data, removing invalid distance points, low-reflection noise points, and obvious outliers. Then, based on spatial extrinsic parameters, it transforms the filtered 3D points from the lidar module coordinate system to the infrared thermal imager module coordinate system. After the transformation, the data processing unit projects the 3D points onto the pixel coordinate system of the infrared thermal image data based on the infrared imaging intrinsic parameters, obtaining the pixel coordinates, depth information, and spatial distance corresponding to each projected point. Projected points whose pixel coordinates fall outside the imaging range of the infrared thermal image, whose depth values ​​do not meet the effective ranging range, or whose locations are in the non-overlapping area of ​​the dual-beam field of view are determined to be invalid projected points and removed from the mapping relationship.

[0052] For example, in an industrial motor inspection scenario, the LiDAR module acquires 3D point cloud data of the motor casing, junction box, and heat sink area, while the infrared thermal imager module simultaneously acquires infrared thermal image data of the same area. The data processing unit projects the motor casing point cloud onto the corresponding pixel positions in the infrared thermal image according to calibration parameters and binds each valid 3D point to an infrared pixel. When multiple 3D points on the edge of the motor casing are projected onto the same infrared pixel, the data processing unit compares the spatial distances corresponding to these valid 3D points and selects the valid 3D point with the smallest distance value as the effective distance source for that infrared pixel. This reduces interference from distant background points or points behind obstructions on the target surface temperature compensation. For infrared pixels that do not have valid 3D points but are located within adjacent valid areas, distance filling can be performed based on the neighboring valid distance values, and a filling identifier is recorded to facilitate the subsequent distance compensation module in distinguishing between direct distance measurements and interpolated distance values.

[0053] After completing the above processing, the data processing unit reconstructs the distance values ​​corresponding to each infrared pixel into distance mapping data according to the pixel arrangement of the infrared thermal image data. This distance mapping data is then associated with the original temperature measurement data, a unified timestamp, and a mapping confidence identifier, and output to the distance compensation module. Through this embodiment, 3D point cloud data can be accurately converted into infrared pixel-level distance information, reducing dual-field-of-view mismatch, occlusion point mismatch, and background point interference. This improves the spatial consistency and usability of the distance-temperature compensation input data, providing a reliable foundation for subsequent pixel-by-pixel real-time temperature compensation.

[0054] In some embodiments, the preset spatiotemporal calibration parameters are generated by the spatiotemporal calibration module in the data processing unit. The spatiotemporal calibration module acquires the point cloud calibration features and infrared calibration features corresponding to the temperature control calibration target, and generates initial spatial transformation parameters between the coordinate system of the lidar module and the coordinate system of the infrared thermal imager module based on the point cloud calibration features and infrared calibration features. The spatiotemporal calibration module iteratively optimizes the initial spatial transformation parameters based on the point cloud structure features and infrared edge features in the natural scene, and generates preset spatiotemporal calibration parameters composed of spatial extrinsic parameters and infrared imaging intrinsic parameters.

[0055] Specifically, in this embodiment, the spatiotemporal calibration module is located within the data processing unit and performs calibration processing after the device is assembled at the factory, verified during on-site installation, or after maintenance and adjustment of the rigid support. The lidar module and infrared thermal imager module in the dual-light sensor module are fixed to the same rigid support, with the optical axis alignment deviation controlled within 0.1° and the field of view overlap not less than 80%. Based on this mechanical installation, the spatiotemporal calibration module further generates spatiotemporal calibration parameters for point cloud projection and infrared pixel positioning.

[0056] The calibration process begins with a coarse registration stage. The operator places the temperature-controlled calibration target within a preset distance in front of the device. The target is marked with reflective feature areas identifiable by the lidar and heating feature areas identifiable by the infrared thermal imager. The lidar module acquires 3D point cloud data of the temperature-controlled calibration target, and the spatiotemporal calibration module extracts point cloud calibration features such as the target's border, corners, and reflective block center from the 3D point cloud data. Simultaneously, the infrared thermal imager module acquires infrared thermal image data, and the spatiotemporal calibration module extracts infrared calibration features such as the heating block boundary, hot spot center, and temperature gradient contour from the infrared thermal image data.

[0057] Since the reflective and heating feature regions on the temperature-controlled calibration target have a preset geometric correspondence, the spatiotemporal calibration module can calculate the initial rotation and translation relationships from the lidar module coordinate system to the infrared thermal imager module coordinate system based on the matching relationship between the point cloud calibration features and the infrared calibration features, and simultaneously estimate the focal length, principal point position, and distortion correction parameters in the infrared imaging intrinsic parameters.

[0058] After the coarse registration result is generated, the spatiotemporal calibration module enters the fine registration stage. The device performs dual-light synchronous acquisition of natural scenes such as industrial equipment, pipes, junction boxes, and heat sinks. The spatiotemporal calibration module extracts point cloud structural features such as equipment edges, planar boundaries, pipe contours, and shell turning lines from the 3D point cloud data, and extracts infrared edge features such as temperature change edges, hot zone contours, and component boundaries from the infrared thermal image data. Using the initial spatial transformation parameters obtained in the coarse registration stage as the starting point for iteration, the spatiotemporal calibration module projects the point cloud structural features onto the pixel coordinate system of the infrared thermal image data and calculates the positional deviation, orientation deviation, and scale deviation between the projected contour and the infrared edge features. Subsequently, the spatiotemporal calibration module performs joint iterative optimization of rotation parameters, translation parameters, and infrared imaging intrinsic parameters, gradually aligning the point cloud projected boundary with the infrared edge features.

[0059] For example, in an industrial motor inspection scenario, the temperature control calibration target is first placed at distances of 3 meters, 8 meters, and 15 meters from the device. The spatiotemporal calibration module acquires point cloud calibration features and infrared calibration features at different distances to obtain initial calibration parameters covering the actual working distance. Subsequently, the device performs natural scene verification on the motor housing, heat sink, and junction box. The spatiotemporal calibration module uses the structural contour of the dense edges of the heat sink in the point cloud and the temperature gradient boundary formed by the edge of the heat sink in the infrared thermal image to iteratively optimize the initial spatial transformation parameters. The optimized spatial extrinsic parameters, infrared imaging intrinsic parameters, calibration time, and applicable distance range are written into the device's storage area for subsequent use by the point cloud temperature mapping module.

[0060] Through the above embodiments, the spatiotemporal calibration module can combine the controllable characteristics of the temperature control calibration target with the actual structural characteristics of the natural scene to generate preset spatiotemporal calibration parameters that match the device installation posture and application scenario. This improves the accuracy of projecting three-dimensional point cloud data onto the pixel coordinate system of infrared thermal image data, reduces the spatial registration error between the point cloud and infrared pixels, and improves the reliability of subsequent distance mapping, pixel-by-pixel temperature compensation, and three-dimensional temperature field output.

[0061] In some embodiments, the data processing unit performs field-of-view validity screening on the projected 3D points, establishes a mapping relationship between valid 3D points and corresponding infrared pixels, and generates distance values ​​corresponding to the infrared pixels based on the spatial distance of the valid 3D points, including: The data processing unit acquires the pixel coordinates and depth information of the projected 3D points in the infrared thermal image data, and filters the projected 3D points according to the imaging range, field of view overlap area and effective depth range of the infrared thermal image data to generate effective 3D points. The data processing unit binds the pixel coordinates of the effective 3D points to the corresponding infrared pixels, generating a mapping relationship between the effective 3D points and the corresponding infrared pixels; The data processing unit calculates the spatial distance of the effective 3D point relative to the dual-light sensor module based on the spatial coordinates of the effective 3D point in the lidar coordinate system or the infrared thermal imager coordinate system, and uses the spatial distance as the distance value of the corresponding infrared pixel.

[0062] Specifically, in this embodiment, after the data processing unit completes the projection of the 3D point cloud data onto the pixel coordinate system of the infrared thermal image data, it establishes an association record between the pixel coordinates, depth information, spatial coordinates, reflection intensity, and unified timestamp of the projected 3D points. Since the lidar module and the infrared thermal imager module are fixed on the same rigid bracket, and their field of view overlap is not less than 80%, the data processing unit first removes the projected 3D points whose pixel coordinates are outside the infrared imaging range based on the width, height, and effective imaging boundary of the infrared thermal image data, and then removes the projected 3D points located in the non-overlapping area based on the overlapping area of ​​the dual-beam field of view.

[0063] After the field of view is filtered, the data processing unit continues to filter the projected 3D points based on the effective depth range. Taking an industrial equipment inspection scenario as an example, the working distance range of the LiDAR module can be set from 1 meter to 30 meters. The data processing unit marks projected 3D points with a depth less than 1 meter, a depth greater than 30 meters, a null depth value, or a reflection intensity lower than the noise threshold as invalid points. For target areas such as motor casings, transformer bushings, and pipe surfaces, the data processing unit retains projected 3D points that fall within the effective imaging range, the overlapping area of ​​the field of view, and the effective depth range, and uses these retained points as valid 3D points.

[0064] Subsequently, the data processing unit binds the effective 3D points to corresponding infrared pixels based on their pixel coordinates in the infrared thermal image data. During binding, the data processing unit establishes an index table using the row and column positions of the infrared pixels as indices, and writes the spatial coordinates, depth information, and point cloud frame identifier of the effective 3D points into the corresponding index entries. If the same infrared pixel corresponds to only one effective 3D point, the data processing unit directly reads the spatial distance of the effective 3D point relative to the dual-light sensor module as the distance value for that infrared pixel. The spatial distance can be calculated based on the 3D coordinates of the effective 3D point in the lidar coordinate system or based on the depth information in the infrared thermal imager coordinate system.

[0065] When a single infrared pixel corresponds to multiple valid 3D points, the data processing unit sorts these points according to a distance-first rule and selects the valid 3D point with the smallest spatial distance as the distance source for that infrared pixel. This processing is suitable for scenarios such as motor edges, pipe obstructions, and overlapping equipment supports, and can reduce interference from distant background points, points behind obstructions, or reflected noise points on the distance value of the infrared pixel. For infrared pixels that are not bound to valid 3D points but are located on continuous target surfaces, the data processing unit can perform neighborhood filling based on the valid distance values ​​of adjacent infrared pixels.

[0066] For example, when the device inspects an industrial motor in operation, the motor's heat sink, bearing end cover, and junction box form a continuous temperature measurement area in the infrared thermal image, and the lidar module simultaneously acquires the three-dimensional point cloud of the area. When two layers of point clouds at the gap of the heat sink are simultaneously projected onto the same infrared pixel, the data processing unit selects the effective three-dimensional point with the smaller distance in the front layer as the distance value and writes this distance value into the distance mapping data.

[0067] Through the above embodiments, the data processing unit can filter out effective three-dimensional points from the projected three-dimensional points that have a valid spatial correspondence with the infrared thermal image data, establish infrared pixel-level distance mapping data, reduce the distance error introduced by points outside the field of view, invalid depth points, and occluded background points, and improve the accuracy of distance temperature compensation input data and the spatial consistency of the three-dimensional temperature field construction.

[0068] In some embodiments, the data processing unit is further configured to acquire the material classification result of the target surface and the surface normal direction information extracted from the three-dimensional point cloud data, determine the initial emissivity parameter based on the material classification result, and perform angle correction on the initial emissivity parameter based on the surface normal direction information to generate an emissivity correction parameter; the data processing unit inputs the emissivity correction parameter and the distance mapping data into the distance temperature compensation process to generate the temperature compensation amount corresponding to the infrared pixel.

[0069] Specifically, in this embodiment, after generating infrared pixel-level distance mapping data, the data processing unit further performs emissivity dynamic correction processing. The dual-light sensor module is fixed to the same rigid bracket, the lidar module synchronously outputs three-dimensional point cloud data of the target scene, and the infrared thermal imager module synchronously outputs infrared thermal image data. When the device is equipped with a visible light camera module, the data processing unit can also acquire texture, color, and edge information in the visible light image, and combine it with the temperature gradient, grayscale distribution, and point cloud reflection intensity in the infrared thermal image to form material recognition input data.

[0070] The data processing unit first aligns the material recognition input data to the same infrared pixel position according to the mapping relationship between 3D points and infrared pixels, and then determines the material classification result of the target surface based on a preset material classification model or material parameter library. The material classification result can include categories such as metal shell, painted surface, rubber insulation layer, ceramic sleeve, and plastic protective cover. Different material categories correspond to different initial emissivity parameters. The data processing unit retrieves the corresponding parameters from the device's storage area as the initial emissivity parameters of the infrared pixels.

[0071] Simultaneously, the data processing unit calculates the surface normal direction information of the target based on the 3D point cloud data. Specifically, the data processing unit selects neighboring points centered on the effective 3D points to construct a local point cloud plane, and generates surface normal direction information based on the direction of the local point cloud plane. Subsequently, the data processing unit performs angle correction on the initial emissivity parameters based on the angle between the surface normal direction information and the observation direction of the infrared thermal imager, generating emissivity correction parameters. For regions where the normal direction is close to the observation direction, the emissivity correction magnitude is small; for regions with a large oblique angle, the data processing unit increases the correction magnitude according to the angle correction relationship, and stores the corrected parameters in association with the infrared pixel positions.

[0072] For example, in an industrial motor inspection scenario, the motor housing has a painted metal surface, the junction box housing has a plastic surface, and the bearing end cover has exposed metal areas. The data processing unit identifies different surface materials based on visible light texture, infrared temperature distribution, and point cloud reflection intensity, and then calls the corresponding initial emissivity parameters for each. When the heat sink sidewall is at a large tilt angle relative to the infrared thermal imager, the data processing unit calculates the surface normal direction using the heat sink point cloud neighborhood and corrects the initial emissivity parameters of the infrared pixels in that area by angle. For transformer inspection scenarios, different emissivity correction parameters can also be generated for ceramic bushings, metal connection terminals, and rubber seals.

[0073] Furthermore, after completing the dynamic emissivity correction, the data processing unit inputs the emissivity correction parameters, distance mapping data, and original temperature measurement data into the distance-temperature compensation process. The distance compensation module calls the distance-temperature compensation parameters based on the distance value corresponding to the infrared pixel, and adjusts the temperature compensation amount of the corresponding infrared pixel in conjunction with the emissivity correction parameters, thereby generating corrected temperature distribution data. The emissivity correction parameters, distance values, temperature compensation amounts, and unified timestamps can be jointly written into the compensation result data frame for use by the three-dimensional temperature field output unit.

[0074] Through the above embodiments, the data processing unit can incorporate target material and observation angle information during the distance temperature compensation process, reduce radiation difference errors caused by different material surfaces and oblique viewing temperature measurement conditions, improve the adaptability of infrared pixel temperature compensation, and enhance the temperature measurement accuracy and spatial consistency of three-dimensional temperature field data in complex industrial scenarios.

[0075] In some embodiments, the data processing unit is further configured to: The data processing unit retrieves pre-calibrated and fixed distance temperature compensation parameters from the device's storage area. These parameters are generated based on calibration data collected from a standard temperature source at different real temperatures and measurement distances. The data processing unit aligns the pixel distance values ​​in the distance mapping data with the original temperature measurement data in the infrared thermal image data according to the infrared pixel positions, and generates pixel-by-pixel compensation input data. The data processing unit matches the corresponding distance-temperature compensation parameters based on the pixel distance value and calculates the temperature compensation amount for the corresponding infrared pixel. The data processing unit uses temperature compensation to correct the original temperature measurement data of the corresponding infrared pixels, and reconstructs the corrected temperature distribution data according to the position of the infrared pixels.

[0076] Specifically, in this embodiment, the distance compensation module is located within the data processing unit and operates via programmable logic device hardware logic circuitry, embedded processor firmware, or a combination of both. During factory calibration, a standard blackbody radiation source is positioned within a working distance range of 1 to 30 meters, and multiple actual temperature settings are set, such as 50°C, 100°C, 300°C, and 600°C. The infrared thermal imager module acquires calibration thermal images at each distance and temperature setting, while the lidar module simultaneously acquires distance data from the surface of the blackbody radiation source. The data processing unit, based on the deviation relationship between the actual temperature, the original temperature measurement data, and the measured distance, fits and generates distance-temperature compensation parameters, and writes these compensation parameters into the non-volatile storage area of ​​the device.

[0077] During actual temperature measurement, the data processing unit first retrieves distance-temperature compensation parameters from the device's storage area that match the current temperature measurement range, working distance range, and sensor configuration. These parameters can be stored using lookup tables, piecewise function parameters, polynomial parameters, or exponential function parameters. When the industrial environment has a wide temperature range, the data processing unit can retrieve the corresponding parameter set according to the temperature and distance ranges. When the target is within a short distance range, the data processing unit can retrieve a simplified parameter set for the corresponding distance range to reduce the computational load on the device.

[0078] After receiving the distance mapping data output by the point cloud temperature mapping module, the data processing unit aligns the pixel distance values ​​with the original temperature measurement data of the corresponding pixels according to the pixel row and column positions of the infrared thermal image data, forming pixel-by-pixel compensation input data. For pixel distance values ​​directly generated from valid 3D points, the data processing unit marks the corresponding distance value as direct ranging input; for pixel distance values ​​obtained by neighborhood distance filling, the data processing unit marks the corresponding distance value as interpolated ranging input, and uses adjacent pixel smoothing constraints in subsequent compensation to reduce the impact of local distance abrupt changes on the compensation result.

[0079] For example, in an industrial motor inspection scenario, the infrared thermal imager module acquires raw temperature data of the motor housing, bearing end cover, and junction box area, while the lidar module simultaneously acquires 3D point cloud data of these areas. A certain infrared pixel on the motor housing corresponds to a distance of 8 meters and a raw temperature of 76°C. The data processing unit matches the corresponding distance-temperature compensation parameters based on the 8-meter distance segment and calculates the corresponding temperature compensation amount for that pixel. Another infrared pixel in the junction box area corresponds to a distance of 12 meters, and the data processing unit calls the corresponding parameters for the 12-meter distance segment and generates the corresponding temperature compensation amount. For a transformer inspection scenario, the data processing unit can match compensation parameters to the bushing, terminal block, and heat sink areas according to the pixel distance values, ensuring that infrared pixels at different spatial locations receive corresponding compensation amounts.

[0080] After generating the temperature compensation value, the data processing unit superimposes and corrects the temperature compensation value with the original temperature measurement data of the corresponding infrared pixels, and reconstructs the corrected temperature distribution data according to the pixel arrangement of the infrared thermal image data. The corrected temperature distribution data, along with the distance mapping data, unified timestamp, and compensation parameter identifier, are written into the compensation result data frame and output to the three-dimensional temperature field output unit for subsequent reverse mapping of temperature data to the three-dimensional point cloud and pseudo-color rendering.

[0081] Through the above embodiments, the data processing unit can use the distance-temperature compensation parameters fixed at the device end to perform pixel-level real-time compensation on infrared thermal image data, reduce infrared temperature measurement deviation caused by changes in target distance, improve the correction speed and consistency of temperature distribution data in dynamic scenes, and provide more accurate temperature input for the generation of three-dimensional temperature field data.

[0082] In some embodiments, the three-dimensional temperature field output unit is used for: The three-dimensional temperature field output unit calls the mapping relationship between three-dimensional points and infrared pixels to determine the position of the infrared pixel corresponding to each three-dimensional point in the three-dimensional point cloud data. The three-dimensional temperature field output unit extracts the corrected temperature value corresponding to the infrared pixel position from the corrected temperature distribution data and uses the corrected temperature value as the temperature attribute of the corresponding three-dimensional point. When there is a neighborhood temperature correlation at the corresponding infrared pixel position, the three-dimensional temperature field output unit interpolates and assigns the temperature attribute of the corresponding three-dimensional point based on the neighborhood temperature data. The three-dimensional temperature field output unit generates three-dimensional temperature field data according to the three-dimensional point spatial coordinates and corresponding temperature attributes.

[0083] Specifically, in this embodiment, the three-dimensional temperature field output unit is located after the data processing unit and is connected to the distance compensation module and the communication and storage unit. After the distance compensation module generates the corrected temperature distribution data, the three-dimensional temperature field output unit reads the three-dimensional point cloud data corresponding to the same unified timestamp, the mapping relationship between the three-dimensional points and infrared pixels, the corrected temperature distribution data, and the distance mapping data, and uses the unified timestamp as the data assembly identifier to prevent the mixing of point cloud data and temperature data from different acquisition cycles.

[0084] The 3D temperature field output unit first calls the mapping index formed by the point cloud temperature mapping module in the preprocessing stage to determine the infrared pixel position of each valid 3D point in the 3D point cloud data after projection onto the infrared thermal image data. For 3D points that already have a clear pixel binding relationship, the 3D temperature field output unit directly extracts the corrected temperature value of the corresponding infrared pixel position from the corrected temperature distribution data and writes the corrected temperature value into the data attribute field of the 3D point, so that the 3D point simultaneously carries spatial coordinates, distance value, temperature attribute, timestamp, and mapping confidence identifier.

[0085] For some 3D points that cannot be directly mapped to effective infrared pixels due to occlusion, edge projection errors, or differences in point cloud density, the 3D temperature field output unit performs interpolation based on the temperature correlation of the neighborhood. Specifically, the 3D temperature field output unit uses the spatial coordinates or projected pixel position of the 3D point as the center, searches for neighboring 3D points or neighboring infrared pixels that already have temperature attributes, and determines the neighborhood weight based on spatial distance, pixel distance, normal direction consistency, and temperature continuity. The temperature attribute of the 3D point is then generated using the neighborhood-corrected temperature value. For 3D points located at device edges, hole boundaries, or occlusion junctions, the 3D temperature field output unit can reduce the weight of cross-boundary neighborhood temperature data to reduce temperature misassignment between different object surfaces.

[0086] For example, in an industrial motor inspection scenario, the LiDAR module acquires 3D point cloud data of the motor housing, heat sink, bearing end cover, and junction box. Simultaneously, the infrared thermal imager module acquires infrared thermal image data of the corresponding areas. The distance compensation module generates corrected temperature distribution data based on the distance values ​​of each infrared pixel. The 3D temperature field output unit binds the point cloud of the heat sink's front surface to the corresponding temperature pixels in the infrared thermal image according to the mapping index and writes the compensated temperature values ​​to the corresponding 3D points. For sparse point cloud areas within the heat sink grooves, the 3D temperature field output unit interpolates based on the temperature attributes and spatial continuity of point clouds on adjacent heat sink surfaces, ensuring the 3D temperature field remains continuous in local areas. For areas at the junction box edge where both the foreground housing and the background exist, the 3D temperature field output unit combines distance mapping data and depth differences, assigning values ​​only using the neighboring temperature data of the same foreground surface.

[0087] Furthermore, after the 3D temperature field output unit completes the writing of temperature attributes, it generates 3D temperature field data according to the 3D point spatial coordinates and corresponding temperature attributes, and converts the temperature attributes into pseudo-color rendering parameters according to the preset temperature range. For devices equipped with an embedded graphics rendering engine, the 3D temperature field output unit can directly generate a 3D temperature field model locally. For scenarios involving inspection robots or drones, the 3D temperature field data can also be sent to a host computer or cloud platform via the communication and storage unit, simultaneously saving point cloud coordinates, temperature attributes, unified timestamps, and rendering parameters.

[0088] Through the above embodiments, the three-dimensional temperature field output unit can accurately fuse the compensated two-dimensional temperature data into the three-dimensional point cloud space, reduce the loss of temperature attributes caused by sparse point clouds, edge occlusion and unbound pixels, and improve the spatial integrity, temperature continuity and visualization output efficiency of the three-dimensional temperature field data.

[0089] In some embodiments, the lidar module has a dynamic scanning mode corresponding to the target's motion state. The data processing unit determines the target's motion state based on the spatial displacement changes of continuous frame three-dimensional point cloud data and generates scanning mode control information based on the target's motion state. The synchronization control unit adjusts the synchronization trigger frequency output to the dual-light sensor module based on the scanning mode control information. The data processing unit updates the generation frequency of distance mapping data and the calling frequency of distance-temperature compensation parameters based on the adjusted synchronization trigger frequency.

[0090] Specifically, in this embodiment, the lidar module is pre-configured with dynamic scanning modes, including a high-speed scanning mode and a high-precision scanning mode. The high-speed scanning mode is used for inspection scenarios involving moving targets or platforms with high movement speeds; the frame rate can be set to at least 100Hz, resulting in a shorter point cloud acquisition cycle. The high-precision scanning mode is used for temperature measurement scenarios involving static targets or slowly moving targets; the frame rate can be set to at least 30Hz, resulting in a higher point cloud density. After receiving continuous frames of 3D point cloud data, the data processing unit first determines the time interval between adjacent point cloud frames based on a unified timestamp, then extracts the centroid, contour boundary, local plane center, and feature point positions within the target area, and calculates the spatial displacement change, contour change, and velocity characterization between continuous frames.

[0091] In fixed-point inspection scenarios of industrial equipment, the device is mounted on a bracket to perform fixed-point temperature measurement on motors, transformers, and pipelines. If the spatial displacement changes of the motor housing, heat sink, and junction box are below the static threshold in multiple consecutive frames of 3D point cloud data, the data processing unit determines that the target is in a static state and generates high-precision scanning mode control information. After receiving the high-precision scanning mode control information, the synchronization control unit adjusts the synchronization trigger frequency output to the lidar module and the infrared thermal imager module to a frequency that matches the high-precision mode. This allows the lidar module to acquire target surface details with a higher point cloud density, and the infrared thermal imager module to acquire infrared thermal image data according to the corresponding sampling period. The data processing unit then reduces the update frequency of the distance mapping data and reuses the distance mapping results of stable regions between adjacent frames to reduce redundant calculations.

[0092] In scenarios where inspection robots or drones are deployed, the device moves along the equipment channel with the platform. Pipe supports, equipment casings, and wiring terminals in the target scene show significant spatial displacement in continuous frames of 3D point cloud data. The data processing unit determines the target or platform is in motion based on the centroid displacement and feature point velocity of the continuous frame point cloud data and generates high-speed scanning mode control information. The synchronization control unit increases the synchronization trigger frequency based on the high-speed scanning mode control information, enabling the LiDAR module to output 3D point cloud data at a frame rate of no less than 100Hz, while simultaneously enabling the infrared thermal imager module to complete infrared sampling and data conversion within the corresponding trigger cycle. The data processing unit increases the distance mapping data generation frequency based on the adjusted synchronization trigger frequency and calls the distance-temperature compensation parameters according to each acquisition cycle to avoid lag in compensation parameters caused by rapid changes in target distance during movement.

[0093] During mode switching, the data processing unit can also set the state holding period and switching threshold. Scanning mode control information is only generated when the displacement changes of the point cloud across multiple consecutive frames meet the motion or static state determination conditions. When adjusting the trigger frequency, the synchronization control unit writes the switching timestamp, mode identifier, and trigger frequency into the corresponding data frame. The distance compensation module selects the compensation parameter call rhythm based on the mode identifier. For switching from high-speed scanning mode to high-precision scanning mode, the data processing unit can first maintain high-speed compensation updates for several frames, and then gradually reduce the distance mapping update frequency to maintain the continuity of the three-dimensional temperature field data.

[0094] Through the above embodiments, the device can adaptively adjust the lidar scanning mode, dual-light synchronous triggering frequency, distance mapping data generation frequency, and distance temperature compensation parameter calling frequency according to the target's motion state, thereby improving the timing matching capability of dual-light acquisition and the distance compensation response speed in motion scenarios. At the same time, it reduces the amount of invalid calculations and improves the spatial detail representation capability of point clouds in static scenarios, thereby enhancing the real-time performance, completeness, and temperature measurement stability of three-dimensional temperature field data.

[0095] The following description, in conjunction with the accompanying drawings and specific embodiments, will illustrate the composition, structure, and function of the three-dimensional temperature measurement device based on the dual-light fusion of lidar and infrared thermal imager of this application. Specifically, it may include the following: Example 1 This embodiment provides a three-dimensional temperature measurement device that fuses LiDAR and infrared thermal imager, and its overall hardware architecture is as follows: Figure 1 As shown.

[0096] I. Dual-light sensor module See Figure 3 , Figure 3This is a schematic diagram of the installation structure of the dual-light sensor module provided in this application embodiment. The dual-light sensor module includes a lidar module 1 and an infrared thermal imager module 2, both fixedly mounted on the same rigid bracket. The lidar module uses a 940nm wavelength solid-state lidar with a working distance range of 1 to 30 meters and a ranging accuracy of centimeters. The 940nm wavelength laser source has a high eye safety threshold and good atmospheric transmission characteristics, making it suitable for outdoor and industrial environments. The all-solid-state solution achieves miniaturization and cost reduction of the lidar, which is beneficial for the integrated deployment of the device.

[0097] The infrared thermal imager module uses an uncooled vanadium oxide focal plane array detector with a temperature measurement range of -20℃ to 1500℃, a resolution of 640×512 pixels, and a noise equivalent temperature difference (NETD) better than 50mK. Uncooled detectors do not require cryogenic cooling equipment, have low power consumption, and are small in size, making them suitable for integration into portable or fixed temperature measurement devices.

[0098] In this embodiment, the dual-light sensor module also includes a visible light camera module 4, used to acquire visible light images of the target scene with a resolution of 1920×1080 pixels or higher. The visible light images can serve as auxiliary references for infrared thermal images and point cloud data, facilitating intuitive positioning of the temperature measurement target by the operator.

[0099] The lidar module, infrared thermal imager module, and visible light camera module are fixedly connected by a rigid bracket. Through precision machining and optical adjustment, it is ensured that the optical axis alignment deviation of the three is no greater than 0.1° and the field of view overlap is no less than 80%. The rigid bracket is made of aluminum alloy or carbon fiber composite material, which has high rigidity and good thermal stability, and can effectively suppress structural deformation caused by temperature changes.

[0100] II. Synchronization Control Unit See Figure 4 , Figure 4 This is a schematic diagram of the FPGA parallel architecture of the synchronization control unit provided in this application embodiment. The synchronization control unit is built based on the FPGA parallel architecture. The synchronization control unit includes the following sub-modules: (1) Synchronous trigger signal generation module: Based on the high-precision clock inside the FPGA, multiple synchronous trigger pulse signals are generated to drive the lidar module, infrared thermal imager module and visible light camera module to perform synchronous data acquisition. The timing accuracy of the trigger signal reaches the microsecond level, ensuring strict alignment of the three sensors in time.

[0101] (2) Optical isolation circuit: An optical isolation circuit is set between the FPGA output port and the input ports of each sensor to provide electrical isolation, effectively suppress electromagnetic interference and ground loop noise, and prevent sensor noise from interfering with the synchronous control accuracy of the FPGA.

[0102] (3) Timestamp Injection Module: Upon completion of lidar point cloud data acquisition and infrared thermal imager A / D conversion, the current value of the high-precision counter inside the FPGA is captured and injected as a unified timestamp into the frame header of the corresponding data frame. The timestamp resolution can reach the nanosecond level. During data fusion, the lidar scanning start signal and the infrared thermal imager sampling time are aligned using a cross-correlation algorithm to ensure strict temporal consistency between the point cloud and thermal image data.

[0103] (4) Dual Redundant Clock Source Module: Includes a master clock source (high-precision temperature-compensated crystal oscillator TCXO) and a backup clock source (a backup crystal oscillator of equal precision), which serve as backups for each other. When the FPGA detects an abnormality in the output of the master clock source, it automatically switches to the backup clock source to ensure the continuity and reliability of synchronous control.

[0104] III. Data Processing Unit The data processing unit is a hardware processing module embedded in the device, employing an FPGA+ARM heterogeneous computing architecture or an embedded GPU module, and includes the following functional modules: (1) Spatiotemporal calibration module: used to solve the rotation and translation transformation matrices R and T between the coordinate system of the lidar module and the coordinate system of the infrared thermal imager module, as well as the intrinsic parameter matrix K of the infrared thermal imager module. The calibration method adopts a two-stage strategy of coarse registration and fine registration: in the coarse registration stage, a temperature-controlled calibration target is used to obtain preliminary spatial transformation parameters. The temperature-controlled calibration target is equipped with reflection features detectable by the lidar and heating features identifiable by the infrared thermal imager. In the fine registration stage, nonlinear iterative optimization (such as the ICP algorithm) is performed using natural scene features to finally obtain high-precision calibration parameters.

[0105] (2) Point Cloud-Temperature Mapping Module: Based on the spatial transformation relationship obtained by the spatiotemporal calibration module, this module projects the 3D point cloud acquired by the lidar module onto the pixel coordinate system of the infrared thermal image, establishing a mapping correspondence between the 3D points and the pixels of the infrared thermal image. For each pixel in the infrared thermal image, the corresponding 3D point is extracted from the 3D point cloud, and its distance value is obtained. When multiple 3D points are mapped to the same pixel location, the 3D point with the smallest distance value is selected as the valid point corresponding to that pixel; when there is no corresponding 3D point at a pixel location, the neighborhood interpolation method is used to fill the distance of the pixel.

[0106] (3) Distance Compensation Module: This module is one of the core innovative modules of this device and is implemented using a hardware-level solidification method. Specifically, the distance compensation module solidifies the pre-calibrated distance-temperature error compensation model into the hardware logic circuit in the FPGA or the firmware program in the embedded ARM / DSP processor. The parameters of the compensation model are obtained in advance through standard blackbody radiation source calibration experiments and burned into non-volatile memory.

[0107] The distance compensation module takes as input the pixel-wise distance value d(i, j) output by the point cloud-temperature mapping module and the original measured temperature value T_measured(i, j) output by the infrared thermal imager module, and outputs the compensated and corrected temperature value T_corrected(i, j). The calculation formula is as follows: T_corrected(i, j) = T_measured(i, j) + ΔT(d(i, j)) Wherein, ΔT(d(i, j)) is the temperature compensation amount calculated based on the distance d(i, j) using the compensation model. The compensation model adopts an exponential function model: ΔT(d) = a × exp(b × d) + c × exp(f × d), where the parameters a, b, c, and f are determined by fitting the calibration data using the least squares method. This model can accurately capture the characteristic that the infrared temperature measurement error changes sharply with increasing distance and then tends to level off.

[0108] The distance compensation module implements the fixed operation of the compensation algorithm at the hardware level, with a processing latency of no more than one frame of data acquisition cycle, which can meet the requirements of real-time temperature measurement. Compared with the existing technology CN202111608741, which relies on an external blackbody for periodic offline calibration, this module uses real-time distance information from the lidar for dynamic compensation, which has significant advantages such as fast response, no need for an external blackbody, and continuous online operation.

[0109] (4) 3D Temperature Field Rendering Module: This module maps the temperature data from the corrected temperature distribution image output by the distance compensation module back to the corresponding 3D points in the 3D point cloud of the lidar module, so that each 3D point carries temperature attribute information, generating a 3D point cloud model with high-precision temperature information, i.e., a 3D temperature field model. This module performs pseudo-color rendering on the 3D temperature field model, mapping different temperature ranges to different colors (e.g., blue represents low temperature and red represents high temperature), generating a visualized 3D temperature field image. The 3D temperature field rendering module integrates embedded graphics rendering engines such as OpenGL ES or Vulkan, and completes the rendering output directly on the device without relying on an external host computer.

[0110] (5) Dynamic Emissivity Correction Module: This module combines the target surface normal direction information obtained by the lidar module with the material feature information in the infrared thermal image to adaptively optimize the emissivity of the target surface. The emissivity correction value is incorporated into the temperature compensation calculation process of the distance compensation module, further improving the temperature measurement accuracy in complex scenarios.

[0111] Furthermore, the LiDAR module supports dynamic scanning mode switching. When a moving target is detected, a high-speed scanning mode is automatically activated, with a frame rate of no less than 100Hz and a point cloud density of no less than 50 points / m². When the target is stationary, a high-precision mode is activated, with a frame rate of no less than 30Hz and a point cloud density of no less than 200 points / m². During mode switching, the synchronization control unit automatically adjusts the trigger frequency, and the data processing unit 6 adaptively adjusts the update rate of the distance compensation module according to the current scanning mode.

[0112] IV. Communication and Storage Unit The communication and storage unit includes a communication module and a storage module. The communication module supports multiple communication protocols such as Gigabit Ethernet, Wi-Fi 6, and 5G, enabling real-time uploading of acquired data and compensation results to a host computer or cloud platform. The storage module integrates 512GB or larger solid-state storage, supporting local data caching and offline export. Multimodal data is packaged and stored in a unified data format; each data frame contains lidar point cloud data, raw infrared thermal image data, compensated and corrected temperature data, three-dimensional temperature field model data, and timestamp information.

[0113] Example 2 This embodiment, based on Embodiment 1, has been adapted and optimized for industrial equipment inspection scenarios. The entire device is encapsulated in a metal housing with an IP65 protection rating, enabling it to withstand harsh environments such as high temperature, high humidity, and dust in industrial settings. The compensation model parameters of the distance compensation module have been optimized and calibrated for the surface emissivity characteristics of common industrial equipment (such as motors, transformers, and pipelines). The 3D temperature field rendering module has added an automatic abnormal temperature labeling function; when the temperature of a certain area in the 3D temperature field model exceeds a preset threshold, it is automatically highlighted with a bright color and an alarm signal is output.

[0114] Example 3 This embodiment, based on Embodiment 1, integrates the device onto a drone or inspection robot platform. The dual-light sensor module features a lightweight design, with a total weight not exceeding 2kg. The communication and storage unit incorporates low-latency image transmission capabilities, transmitting the three-dimensional temperature field to the ground control station in real time via a 5G network. The synchronization control unit utilizes the PPS (pulse per second) signal provided by GNSS (Global Navigation Satellite System) as an external clock reference to achieve global time synchronization among multiple devices, suitable for large-scale temperature measurement tasks involving multiple devices working together.

[0115] The above embodiments have described in detail the specific components and functions of the three-dimensional temperature measurement device of this application. The implementation process of the three-dimensional temperature measurement method of this application will be described in detail below with reference to specific embodiments. Figure 5 This is a flowchart illustrating the three-dimensional temperature measurement method based on the dual-light fusion of lidar and infrared thermal imager provided in this application embodiment. Figure 5As shown, the method may specifically include the following steps: S501, based on a unified clock, drives the dual-light sensor module to synchronously acquire 3D point cloud data and infrared thermal image data of the target scene, and injects a unified timestamp into the 3D point cloud data and infrared thermal image data; S502: Based on preset spatiotemporal calibration parameters, the three-dimensional point cloud data is projected onto the pixel coordinate system of the infrared thermal image data to establish the mapping relationship between the three-dimensional points and the infrared pixels, and the distance mapping data corresponding to each infrared pixel is generated. S503 calls the distance temperature compensation parameters embedded in the device and performs pixel-by-pixel compensation on the original temperature measurement data of the infrared pixels based on the distance mapping data to generate corrected temperature distribution data. S504 maps the corrected temperature distribution data to the corresponding 3D points in the 3D point cloud data, generating and outputting 3D temperature field data carrying temperature attributes.

[0116] Specifically, in the embodiments of this application, the three-dimensional temperature measurement method that fuses LiDAR and infrared thermal imager can be executed by the aforementioned three-dimensional temperature measurement device in scenarios such as industrial equipment inspection, power facility monitoring, building surface diagnosis, and temperature measurement on inspection robots. This method is not simply a post-processing overlay of LiDAR point clouds and infrared thermal images, but rather achieves real-time fusion temperature measurement of dual-light data through a continuous processing chain of unified clock control, spatiotemporal calibration, pixel-level distance mapping, device-side solidification compensation, and three-dimensional temperature field output.

[0117] In S501, after the device starts up, the synchronization control unit generates multiple synchronization trigger signals based on the unified counting clock inside the programmable logic device, and drives the LiDAR module and the infrared thermal imager module to perform synchronous acquisition through optocoupler isolation circuits. The LiDAR module acquires 3D point cloud data of the target scene, and the infrared thermal imager module simultaneously acquires infrared thermal image data of the same target scene. When the LiDAR module completes a frame of point cloud scanning, the synchronization control unit captures the count value corresponding to the unified counting clock and writes the count value into the point cloud data frame; when the infrared thermal imager module completes the infrared thermal image data conversion, the synchronization control unit also captures the corresponding count value and writes it into the infrared thermal image data frame.

[0118] Therefore, the 3D point cloud data and infrared thermal image data have a unified timestamp, and the data processing unit can group the dual-light data within the same acquisition cycle into a synchronous data group based on the unified timestamp. For example, in an industrial motor inspection scenario, the device operates at an infrared sampling frequency of 30 frames per second, and the lidar module synchronously outputs 3D point cloud data of the motor casing, heat sink, and junction box area. Subsequent processing matches the point cloud frames and infrared frames with a unified timestamp to avoid misalignment between point cloud and thermal images in dynamic scenes.

[0119] In S502, the data processing unit acquires 3D point cloud data and infrared thermal image data with a unified timestamp correspondence, and calls preset spatiotemporal calibration parameters. The preset spatiotemporal calibration parameters may include spatial extrinsic parameters between the coordinate system of the lidar module and the coordinate system of the infrared thermal imager module, as well as the infrared imaging intrinsic parameters of the infrared thermal imager module. The spatial extrinsic parameters can be obtained through coarse registration using the point cloud calibration features and infrared calibration features corresponding to the temperature-controlled calibration target, and then iteratively optimized by combining the point cloud structure features and infrared edge features in the natural scene.

[0120] The data processing unit converts the 3D point cloud data to the infrared thermal imager coordinate system based on spatial extrinsic parameters, and then projects the converted 3D points onto the pixel coordinate system of the infrared thermal image data based on infrared imaging intrinsic parameters. After projection, the data processing unit performs validity screening on the projected 3D points based on the imaging range of the infrared thermal image data, the overlapping area of ​​the dual-beam field of view, and the effective depth range, removing points outside the field of view, invalid depth points, low-reflection noise points, and points in non-overlapping areas, and retaining valid 3D points.

[0121] Subsequently, the data processing unit binds the effective 3D points to their corresponding infrared pixels and generates distance values ​​for the corresponding infrared pixels based on the spatial distance of the effective 3D points relative to the dual-light sensor module. When the same infrared pixel corresponds to multiple effective 3D points, the effective 3D point with the smallest distance value is selected first to reduce interference from background points or occluded points on the target surface distance value. For infrared pixels that do not have effective 3D points but are located on a continuous target surface, they can be filled based on neighborhood distance values, ultimately forming infrared pixel-level distance mapping data.

[0122] In S503, the data processing unit calls the distance-temperature compensation parameters stored on the device and aligns the distance mapping data with the original temperature measurement data in the infrared thermal image data according to the infrared pixel positions, forming pixel-by-pixel compensation input data. The distance-temperature compensation parameters can be pre-calibrated using a standard blackbody radiation source at multiple real temperatures and multiple measurement distances, and stored in the non-volatile storage area on the device in the form of a lookup table, piecewise function parameters, polynomial parameters, or exponential function parameters. During actual temperature measurement, the data processing unit matches the corresponding distance-temperature compensation parameters according to the pixel distance value corresponding to each infrared pixel, calculates the temperature compensation amount for that infrared pixel, and uses the temperature compensation amount to correct the original temperature measurement data of that infrared pixel.

[0123] For implementations with dynamic emissivity correction, the data processing unit can also combine the target surface material classification results and point cloud surface normal direction information to generate emissivity correction parameters, and input these parameters along with distance mapping data into the distance-temperature compensation process. For example, motor housings, plastic junction boxes, and metal bearing end caps have different emissivities, and the heat sink sidewalls may also have large observation angles. The data processing unit can adjust the compensation amount according to the material type and surface normal direction. After completing pixel-by-pixel compensation, the data processing unit reconstructs and generates corrected temperature distribution data according to the infrared pixel arrangement.

[0124] In S504, the 3D temperature field output unit calls the mapping relationship between 3D points and infrared pixels to determine the infrared pixel position corresponding to each 3D point in the 3D point cloud data, and extracts the corrected temperature value of the corresponding infrared pixel position from the corrected temperature distribution data, and writes the corrected temperature value into the temperature attribute field of the corresponding 3D point. For 3D points that are not directly bound to temperature values ​​due to occlusion, edge projection errors, or insufficient point cloud density, the 3D temperature field output unit can perform interpolation based on neighborhood temperature data, and determine the neighborhood weight by combining spatial distance, pixel distance, normal direction consistency, and temperature continuity, so as to reduce the misassignment of temperature values ​​across object boundaries.

[0125] Furthermore, after the temperature attributes are written, the 3D temperature field output unit generates 3D temperature field data according to the 3D point spatial coordinates and corresponding temperature attributes. This data can then be further processed with pseudo-color rendering to convert different temperature ranges into corresponding visual identifiers. The generated 3D temperature field data can be uploaded to a host computer or cloud platform via the communication and storage unit, or it can be directly displayed in real-time and stored locally on the device.

[0126] Using the above method, this application can synchronously acquire and match lidar point cloud data and infrared thermal image data under a unified time reference. It generates infrared pixel-level distance mapping data using the 3D point cloud and performs real-time, pixel-by-pixel correction of the infrared temperature measurement results based on distance-temperature compensation parameters fixed to the device, ultimately generating 3D temperature field data carrying temperature attributes. This method improves the real-time performance of distance-temperature compensation and the spatiotemporal consistency of dual-light data fusion, enhancing the accuracy, completeness, and visualization capabilities of the 3D temperature field output in complex industrial scenarios.

[0127] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A three-dimensional temperature measuring device of laser radar and infrared thermal imager dual light fusion, characterized in that, include: The dual-light sensor module is used to acquire 3D point cloud data and infrared thermal image data of the target scene under the same rigid mounting reference. The synchronization control unit is used to output a synchronization trigger signal to the dual-light sensor module based on a unified clock, and to inject a corresponding unified timestamp when the three-dimensional point cloud data and infrared thermal image data are formed. The data processing unit is used to project the three-dimensional point cloud data onto the pixel coordinate system of the infrared thermal image data according to preset spatiotemporal calibration parameters, establish a mapping relationship between three-dimensional points and infrared pixels, and generate distance mapping data corresponding to each infrared pixel based on the mapping relationship. The data processing unit is also used to call the distance temperature compensation parameters solidified at the device end, and perform pixel-by-pixel compensation on the original temperature measurement data in the infrared thermal image data according to the distance mapping data to generate corrected temperature distribution data. A three-dimensional temperature field output unit is used to map the corrected temperature distribution data to the corresponding three-dimensional points in the three-dimensional point cloud data to generate three-dimensional temperature field data carrying temperature attributes. The communication and storage unit is used to associate, encapsulate, transmit, and store the three-dimensional temperature field data, the corrected temperature distribution data, and the unified timestamp.

2. The apparatus of claim 1, wherein, The synchronization control unit is specifically used for: The synchronization control unit generates multiple synchronization trigger signals based on the unified counting clock of the programmable logic device, and drives the lidar module and infrared thermal imager module in the dual-light sensor module to perform synchronous acquisition through an optocoupler isolation circuit. When the acquisition of the three-dimensional point cloud data is completed and the conversion of the infrared thermal image data is completed, the synchronization control unit captures the count value corresponding to the unified counting clock and writes the corresponding count value into the data frame of the three-dimensional point cloud data and the infrared thermal image data, respectively. The synchronization control unit aligns the start time of the lidar scan and the sampling time of the infrared thermal image based on the unified timestamp.

3. The apparatus of claim 1, wherein, The data processing unit is specifically used for: The data processing unit acquires three-dimensional point cloud data and infrared thermal image data with a unified timestamp correspondence, and calls the preset spatiotemporal calibration parameters composed of spatial extrinsic parameters and infrared imaging intrinsic parameters. The data processing unit converts the three-dimensional point cloud data to the infrared thermal imager coordinate system according to the spatial extrinsic parameters, and projects the converted three-dimensional points to the pixel coordinate system of the infrared thermal image data according to the infrared imaging intrinsic parameters. The data processing unit performs field-of-view effectiveness screening on the projected 3D points, establishes a mapping relationship between effective 3D points and corresponding infrared pixels, and generates the distance value corresponding to the infrared pixels based on the spatial distance of the effective 3D points. When the same infrared pixel corresponds to multiple valid 3D points, the data processing unit selects the valid 3D point with the smallest distance value to generate the corresponding distance value, and forms the distance mapping data based on the distance value corresponding to each infrared pixel.

4. The apparatus of claim 3, wherein, The preset spatiotemporal calibration parameters are generated by the spatiotemporal calibration module in the data processing unit. The spatiotemporal calibration module acquires the point cloud calibration features and infrared calibration features corresponding to the temperature control calibration target, and generates the initial spatial transformation parameters between the coordinate system of the lidar module and the coordinate system of the infrared thermal imager module based on the point cloud calibration features and infrared calibration features. The spatiotemporal calibration module iteratively optimizes the initial spatial transformation parameters based on the point cloud structure features and infrared edge features in the natural scene to generate the preset spatiotemporal calibration parameters composed of spatial extrinsic parameters and infrared imaging intrinsic parameters.

5. The apparatus of claim 3, wherein, The data processing unit performs field-of-view validity screening on the projected 3D points, establishes a mapping relationship between valid 3D points and corresponding infrared pixels, and generates distance values ​​corresponding to the infrared pixels based on the spatial distances of the valid 3D points, including: The data processing unit acquires the pixel coordinates and depth information of the projected three-dimensional points in the infrared thermal image data, and filters the projected three-dimensional points according to the imaging range, field of view overlap area and effective depth range of the infrared thermal image data to generate effective three-dimensional points. The data processing unit binds the pixel coordinates of the effective three-dimensional points with the corresponding infrared pixels to generate a mapping relationship between the effective three-dimensional points and the corresponding infrared pixels; The data processing unit calculates the spatial distance between the effective three-dimensional point and the dual-light sensor module based on the spatial coordinates of the effective three-dimensional point in the lidar coordinate system or the infrared thermal imager coordinate system, and uses the spatial distance as the distance value of the corresponding infrared pixel.

6. The apparatus of claim 5, wherein, The data processing unit is also used to obtain the material classification result of the target surface and the surface normal direction information extracted from the three-dimensional point cloud data, determine the initial emissivity parameter according to the material classification result, and perform angle correction on the initial emissivity parameter according to the surface normal direction information to generate emissivity correction parameter; The data processing unit inputs the emissivity correction parameter and the distance mapping data into the distance-temperature compensation process to generate the temperature compensation amount corresponding to the infrared pixel.

7. The apparatus of claim 1, wherein, The data processing unit is also used for: The data processing unit retrieves pre-calibrated and fixed distance temperature compensation parameters from the device's storage area. These distance temperature compensation parameters are generated based on calibration data collected from a standard temperature source at different real temperatures and different measurement distances. The data processing unit aligns the pixel distance values ​​in the distance mapping data with the original temperature measurement data in the infrared thermal image data according to the infrared pixel positions, and generates pixel-by-pixel compensation input data. The data processing unit matches the corresponding distance temperature compensation parameters according to the pixel distance value and calculates the temperature compensation amount for the corresponding infrared pixel. The data processing unit uses the temperature compensation amount to correct the original temperature measurement data of the corresponding infrared pixel, and reconstructs the corrected temperature distribution data according to the position of the infrared pixel.

8. The apparatus of claim 1, wherein, The three-dimensional temperature field output unit is used for: The three-dimensional temperature field output unit calls the mapping relationship between the three-dimensional points and infrared pixels to determine the position of the infrared pixel corresponding to each three-dimensional point in the three-dimensional point cloud data. The three-dimensional temperature field output unit extracts the corrected temperature value corresponding to the infrared pixel position from the corrected temperature distribution data, and uses the corrected temperature value as the temperature attribute of the corresponding three-dimensional point. When there is a neighborhood temperature correlation at the corresponding infrared pixel position, the three-dimensional temperature field output unit interpolates and assigns values ​​to the temperature attributes of the corresponding three-dimensional point based on the neighborhood temperature data. The three-dimensional temperature field output unit generates the three-dimensional temperature field data according to the three-dimensional point spatial coordinates and corresponding temperature attributes.

9. The apparatus according to claim 1, characterized in that, The lidar module has a dynamic scanning mode corresponding to the target's motion state. The data processing unit determines the target's motion state based on the spatial displacement changes of continuous frame three-dimensional point cloud data, and generates scanning mode control information based on the target's motion state. The synchronization control unit adjusts the synchronization trigger frequency output to the dual-light sensor module based on the scanning mode control information, and the data processing unit updates the generation frequency of the distance mapping data and the calling frequency of the distance temperature compensation parameters according to the adjusted synchronization trigger frequency.

10. A three-dimensional temperature measurement method based on the device according to any one of claims 1 to 9, characterized in that, include: Based on a unified clock, the dual-light sensor module synchronously acquires 3D point cloud data and infrared thermal image data of the target scene, and injects a unified timestamp into the 3D point cloud data and infrared thermal image data. The three-dimensional point cloud data is projected onto the pixel coordinate system of the infrared thermal image data according to the preset spatiotemporal calibration parameters, a mapping relationship between three-dimensional points and infrared pixels is established, and distance mapping data corresponding to each infrared pixel is generated. The distance temperature compensation parameters, which are fixed on the device, are invoked, and the original temperature measurement data of the infrared pixels are compensated pixel by pixel based on the distance mapping data to generate corrected temperature distribution data. The corrected temperature distribution data is mapped to the corresponding three-dimensional points in the three-dimensional point cloud data to generate and output three-dimensional temperature field data carrying temperature attributes.

Images