A dot matrix infrared temperature measurement and 3D camera fusion three-dimensional temperature measurement method

Abstract

The application relates to the technical field of infrared temperature measurement and three-dimensional measurement, in particular to a dot matrix infrared temperature measurement and 3D camera fusion three-dimensional temperature measurement method. The method comprises the following steps: cooperative calibration, a pixel-level mapping relationship between a temperature measurement module and a three-dimensional acquisition module is established through multi-pose calibration to obtain calibration parameters; synchronous data acquisition, temperature data and three-dimensional depth data of a measured target are synchronously exposed and collected by controlling the two modules; coordinate mapping and data fusion, two-dimensional pixel coordinates of each temperature measurement point are matched to corresponding three-dimensional coordinates by calling the calibration parameters, and three-dimensional point cloud data with temperature labels are generated; emissivity adaptive compensation and three-dimensional temperature field reconstruction, after a normal vector and a curvature are calculated according to the three-dimensional coordinates, the emissivity is automatically corrected and the temperature value is compensated, and a three-dimensional temperature field model is generated again; and data output.

Description

Technical Field

[0001] This application relates to the field of infrared thermometry and three-dimensional measurement technology, and more specifically, to a three-dimensional temperature measurement method that integrates dot matrix infrared thermometry with a 3D camera. Background Technology

[0002] The three-dimensional temperature measurement method, which integrates dot matrix infrared thermometry with three-dimensional sensing technology, is an important means to realize the spatialization and digitization of the target temperature field. It has broad application prospects in fields such as industrial precision inspection, power equipment condition assessment, medical thermal imaging diagnosis, and building energy consumption analysis. By fusing infrared thermometry data with three-dimensional spatial coordinates, it can generate three-dimensional point clouds or three-dimensional temperature field models with temperature labels. This provides a data foundation for spatial tracing of temperature anomalies, three-dimensional localization of thermal hazards, and three-dimensional visualization assessment of equipment thermal status. As the world's largest energy consumer and industrial manufacturer, my country has a huge demand for three-dimensional temperature field measurement methods in equipment inspection and condition monitoring in industries such as coal, power, petrochemicals, and steel.

[0003] However, existing three-dimensional temperature measurement methods have long suffered from three major bottlenecks: inaccurate measurement point correspondence, low temperature accuracy, and poor method compatibility. Firstly, there is a lack of effective collaborative calibration methods between matrix infrared thermography and three-dimensional data acquisition. Existing solutions mostly involve software post-registration between discrete devices, achieving a coarse alignment between temperature data and the three-dimensional point cloud through feature point matching or iterative nearest-point algorithms. This only achieves centimeter-level or even coarser mapping accuracy, failing to achieve pixel-level precise correspondence between each infrared thermography point and the three-dimensional coordinates. When the infrared thermography matrix is ​​small, the sparse feature points cause the registration algorithm to have difficulty converging or produce large deviations, with the problem of ambiguous spatial positioning of temperature measurement points being particularly prominent. Secondly, traditional infrared thermography methods require manually setting the emissivity of the target object based on experience, but the emissivity of different materials varies significantly. Even with the same material, different surface morphologies can lead to significant changes in emissivity. In actual measurements, the surface morphologies of different areas of the target object are often different. Using a single emissivity setting value cannot reflect the actual emissivity differences in each area. The emissivity setting error is directly transmitted to the temperature inversion result, resulting in a significant decrease in temperature measurement accuracy. There is no existing technical solution based on three-dimensional spatial information to achieve point-by-point adaptive compensation of emissivity at each temperature measurement point. In addition, existing three-dimensional temperature measurement methods are often strongly tied to specific types of depth acquisition equipment. The depth calculation logic of different technical routes such as 3D structured light, TOF, passive binoculars, and active binoculars is different. When switching scenes, it is necessary to redevelop the entire calibration process and fusion algorithm, which lacks universality and is not conducive to the migration and promotion of the method in different application scenarios.

[0004] Therefore, there is an urgent need for a three-dimensional temperature measurement method that can achieve pixel-level accurate mapping between dot matrix infrared thermometry and three-dimensional perception, adaptive emissivity compensation, and is compatible with multiple three-dimensional acquisition technology routes. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned problems in the prior art, this application provides a three-dimensional temperature measurement method that integrates dot matrix infrared thermometry with a 3D camera.

[0006] The embodiments of this application are implemented as follows: This application provides a three-dimensional temperature measurement method that fuses dot-matrix infrared thermography with a 3D camera, including: Collaborative calibration establishes a pixel-level mapping relationship between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to obtain calibration parameters; Synchronous data acquisition involves controlling the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to expose synchronously and acquire temperature data and three-dimensional depth data of the target being measured. Coordinate mapping and data fusion are performed by calling the calibration parameters and matching the corresponding three-dimensional coordinates based on the two-dimensional pixel coordinates of each infrared temperature measurement point in the temperature data to generate three-dimensional point cloud data with temperature labels. Emissivity adaptive compensation and three-dimensional temperature field reconstruction: The normal vector and curvature of each temperature measurement point are calculated based on the three-dimensional coordinates in the temperature-tagged three-dimensional point cloud data. The emissivity is corrected based on the normal vector and curvature, and the temperature value is compensated and calibrated. Then, a three-dimensional reconstruction algorithm is used to generate a three-dimensional temperature field model of the target under test. Data output: The processed 3D temperature field data, temperature data, and 3D point cloud data are output to external devices through the communication interface.

[0007] In one possible implementation, the collaborative calibration step specifically includes: Prepare a standard blackbody and a 3D calibration plate, and place the standard blackbody at a preset position on the 3D calibration plate; Fix the measuring device and adjust the distance between the device and the 3D calibration plate so that the 3D calibration plate is completely within the field of view of the device; The system controls the dot matrix infrared temperature measurement module to collect the temperature data of the standard blackbody and the infrared dot matrix data of the 3D calibration plate, and outputs the two-dimensional pixel coordinates of each infrared temperature measurement point. The system also controls the three-dimensional acquisition module to collect the depth data of the 3D calibration plate and outputs the three-dimensional coordinates of each feature point. By changing the angle between the device and the 3D calibration plate, infrared two-dimensional coordinate data and three-dimensional coordinate data under multiple postures can be obtained; The dataset under the multi-pose conditions is fitted and calculated to solve the extrinsic and intrinsic parameters between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module, establish a one-to-one correspondence between each infrared temperature measurement point and the three-dimensional feature point, complete the collaborative calibration, and store the calibration parameters in the data processing module.

[0008] In one possible implementation, the distance between the device and the 3D calibration plate is 0.2m to 10m, and the angle between the device and the 3D calibration plate varies from 0° to 90°.

[0009] In one possible implementation, the synchronous data acquisition step specifically includes: The measuring device is aligned with the target to be measured, and the synchronous control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to simultaneously expose and acquire data. The dot matrix infrared temperature measurement module collects the dot matrix temperature information of the target under test and outputs the two-dimensional pixel coordinates and corresponding temperature values ​​of multiple temperature measurement points. The three-dimensional acquisition module acquires the three-dimensional coordinates of the feature points on the surface of the target being measured.

[0010] In one possible implementation, the synchronization control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to simultaneously expose and acquire data with a synchronization error of less than 1 μs.

[0011] In one possible implementation, based on the two-dimensional pixel coordinates of the infrared temperature measurement point, coordinate transformation is performed through the extrinsic matrix and the intrinsic parameters to obtain the three-dimensional coordinates of the three-dimensional acquisition module, and the three-dimensional coordinates corresponding to each temperature measurement point are matched with the three-dimensional coordinates acquired by the three-dimensional acquisition module. The matched 3D coordinates and the temperature values ​​are aligned to generate 3D point cloud data with temperature labels.

[0012] In one possible implementation, the coordinate matching employs a nearest neighbor algorithm.

[0013] In one possible implementation, the emissivity adaptive compensation and three-dimensional temperature field reconstruction steps specifically include: Based on the three-dimensional coordinates in the three-dimensional point cloud data with temperature labels, calculate the normal vector and curvature of each temperature measurement point on the surface of the target being measured; Based on the preset correspondence between normal vector, curvature and emissivity, the emissivity of each temperature measurement point is automatically corrected; The temperature value is compensated and calibrated based on the corrected emissivity; A three-dimensional reconstruction algorithm is used to process the compensated and calibrated three-dimensional point cloud data with temperature labels to generate a three-dimensional temperature field model of the target under test.

[0014] In one possible implementation, the preset correspondence between the normal vector, curvature, and emissivity is obtained by fitting experimental data, covering the emissivity characteristics of different materials.

[0015] In one possible implementation, the 3D acquisition module is selected from any one of a 3D structured light module, a TOF 3D depth module, a passive binocular stereo camera, or an active binocular module. When the three-dimensional acquisition module is a 3D structured light module, the coded speckle or regular dot matrix is ​​projected onto the calibration plate or the surface of the target under test by the VCSEL infrared dot matrix emitter, the infrared CMOS camera captures the image of the light spot deformation, and the depth calculation chip calculates the three-dimensional coordinates by the triangulation method. When the three-dimensional acquisition module is a planar TOF three-dimensional depth module, near-infrared floodlight is projected onto the calibration plate or the surface of the target being measured by a near-infrared floodlight emitter. The planar TOF depth sensor detects the time-of-flight difference of light, and the depth calculation chip calculates the three-dimensional coordinates through the time-of-flight ranging algorithm. When the three-dimensional acquisition module is a passive binocular stereo camera, the left and right views of the calibration plate or the target under test are acquired simultaneously by the left infrared camera and the right infrared camera. The binocular stereo matching chip calculates the left and right parallax and solves the three-dimensional coordinates by combining the triangulation method. When the 3D acquisition module is an active binocular module, coded speckle is projected onto the calibration plate or the surface of the target being measured through the VCSEL infrared speckle projector. The left and right infrared cameras simultaneously acquire left and right views, and the depth calculation chip combines parallax calculation and pattern distortion analysis to solve the 3D coordinates.

[0016] The technical solution provided in this application can achieve at least the following beneficial effects: This application provides a three-dimensional temperature measurement method that integrates a dot-matrix infrared thermometer with a 3D camera. By rigidly connecting the dot-matrix infrared thermometer module and the 3D acquisition module within the same housing and ensuring overlapping fields of view and a fixed baseline, the coordinate transformation relationship between the two remains stable throughout the device's lifecycle. Only one factory calibration is required to meet the full lifecycle usage needs, completely solving the problems of unstable coordinate transformation and repeated on-site calibration required in discrete combination schemes. This achieves long-term stable and accurate correspondence between each temperature measurement point and the 3D coordinates, eliminating the usage threshold and efficiency loss caused by on-site calibration. Furthermore, a synchronization control module uses hardware triggering to control the synchronous exposure and data acquisition of the dot-matrix infrared thermometer module and the 3D acquisition module, controlling the synchronization error within 1μs. This ensures precise alignment of temperature data and 3D depth data in the time dimension, avoiding spatial misalignment caused by inconsistent data acquisition times. This allows the device to accurately capture the temperature and spatial information of dynamic or quasi-dynamic targets. This technology fills the gap in existing technologies that cannot perform precise three-dimensional temperature measurement of moving targets. Then, through a data processing module, preset calibration parameters are called to map the two-dimensional pixel coordinates of each temperature measurement point to three-dimensional space, generating three-dimensional point cloud data with temperature labels. This ensures that each temperature value has precise three-dimensional spatial coordinates, enabling accurate three-dimensional spatial tracing of temperature anomalies and fundamentally solving the technical problem of dot-matrix infrared thermometry, which "measures temperature but cannot pinpoint location." Simultaneously, by integrating a visible light RGB camera, temperature information and visible light images are superimposed and displayed, forming a fusion visualization of temperature distribution and high-resolution texture. This facilitates scene understanding and rapid target identification, further improving the device's usability and data readability. Furthermore, through the combined design of a heat dissipation structure, protective glass, and filters, the device ensures long-term stable operation and measurement accuracy under complex conditions such as high and low temperatures, high dust levels, and strong ambient light in industrial environments, demonstrating the environmental adaptability and reliability of an industrial-grade product.

[0017] Furthermore, this application is compatible with four types of 3D acquisition modules: 3D structured light, area array TOF, passive binoculars, and active binoculars. A single hardware architecture can cover various application scenarios, including short-range high precision, medium-to-long-range, outdoor strong light, and indoor weak texture. The passive binocular solution has no active light source emission, low power consumption, and low cost. The TOF solution has strong environmental anti-interference capabilities and is suitable for medium-to-long-range applications. The active binocular solution has both strong light resistance and weak texture adaptability, forming a product matrix that covers different working conditions. This significantly reduces the R&D costs and cycle of product development and scenario adaptation, enabling the device to quickly respond to diverse market demands. It has significant industrial practical value and industrialization prospects, and can be widely used in fields such as industrial precision inspection, power equipment inspection, medical precision temperature measurement, outdoor complex scenarios, and robot environmental perception. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart illustrating a three-dimensional temperature measurement method that integrates dot matrix infrared thermometry with a 3D camera, as shown in an exemplary embodiment of this application. Figure 2 This is a flowchart illustrating the collaborative calibration steps of an exemplary embodiment of this application; Figure 3 This is a flowchart illustrating the synchronous data acquisition steps of an exemplary embodiment of this application; Figure 4 This is a schematic flowchart illustrating the emissivity adaptive compensation and three-dimensional temperature field reconstruction steps in an exemplary embodiment of this application; Figure 5 This is a schematic diagram of signal processing for a three-dimensional temperature measurement method that integrates dot matrix infrared thermometry with a 3D camera, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0020] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0021] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0022] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0023] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0024] Next, the technical solutions of this application and how they solve the aforementioned technical problems will be described in detail through embodiments and in conjunction with the accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0025] In one exemplary embodiment, such as Figure 1 and Figure 5 As shown, a three-dimensional temperature measurement method integrating dot matrix infrared thermometry and a 3D camera is provided. In this embodiment, the method may include the following steps: Step 100: Collaborative calibration, establishing a pixel-level mapping relationship between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module, and obtaining calibration parameters; Step 200: Synchronous data acquisition, control the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to expose synchronously, and acquire the temperature data and three-dimensional depth data of the target under test; Step 300: Coordinate mapping and data fusion. Call the calibration parameters and match the corresponding three-dimensional coordinates according to the two-dimensional pixel coordinates of each infrared temperature measurement point in the temperature data to generate three-dimensional point cloud data with temperature labels. Step 400: Emissivity adaptive compensation and three-dimensional temperature field reconstruction. Calculate the normal vector and curvature of each temperature measurement point based on the three-dimensional coordinates in the temperature-tagged three-dimensional point cloud data. Correct the emissivity and compensate and calibrate the temperature value based on the normal vector and curvature. Then, use a three-dimensional reconstruction algorithm to generate a three-dimensional temperature field model of the target under test. Step 500: Data output. The processed 3D temperature field data, temperature data, and 3D point cloud data are output to external devices through the communication interface.

[0026] In one embodiment, the specific implementation process of the three-dimensional temperature measurement method that integrates dot matrix infrared thermometry with a 3D camera includes five steps: collaborative calibration, synchronous data acquisition, coordinate mapping and data fusion, emissivity adaptive compensation and three-dimensional temperature field reconstruction, and data output. The technical features of the claim will be elaborated in detail below with reference to specific embodiments.

[0027] This invention provides a three-dimensional temperature measurement method that integrates dot-matrix infrared thermometry with a 3D camera. The method is implemented using a three-dimensional temperature measurement device that integrates dot-matrix infrared thermometry with a 3D camera. This device includes a dot-matrix infrared thermometry module, a three-dimensional acquisition module, a synchronization control module, a data processing module, and a communication interface module. The dot-matrix infrared thermometry module acquires the dot-matrix temperature information of the target and outputs the two-dimensional pixel coordinates and corresponding temperature value of each measurement point. The three-dimensional acquisition module acquires the three-dimensional depth information of the target and outputs the three-dimensional coordinates of feature points. The dot-matrix infrared thermometry module and the three-dimensional acquisition module are rigidly connected, with overlapping fields of view and a fixed baseline distance.

[0028] I. Collaborative calibration: Establishing a pixel-level mapping relationship between the dot matrix infrared temperature measurement module and the 3D acquisition module to obtain calibration parameters: Collaborative calibration is a fundamental step in the method of this invention. Its purpose is to accurately establish the geometric mapping relationship between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module. Since the dot matrix infrared temperature measurement module outputs two-dimensional pixel coordinates (u_ir, v_ir), while the three-dimensional acquisition module outputs three-dimensional spatial coordinates (X, Y, Z), the two are in different coordinate spaces. Therefore, calibration is needed to determine the coordinate transformation relationship between the two. The calibration parameters obtained after calibration include extrinsic parameters (rotation matrix R, translation vector T) and intrinsic parameters (focal length, principal point, distortion coefficient). These parameters are stored in the data processing module for subsequent actual measurement.

[0029] II. Synchronous data acquisition: The dot matrix infrared temperature measurement module and the 3D acquisition module are exposed synchronously to acquire temperature data and 3D depth data of the target being measured. After completing the collaborative calibration, align the measuring device with the target and start the formal measurement. The core of synchronous data acquisition is to ensure that the dot matrix infrared temperature measurement module and the three-dimensional acquisition module acquire the information of the target at the same time. Since the dot matrix infrared temperature measurement module acquires temperature data and the three-dimensional acquisition module acquires three-dimensional depth data, if there is a time deviation between the two acquisition times, it will have little impact on static targets, but for dynamic or quasi-dynamic targets, the time deviation will directly cause the temperature data and the three-dimensional spatial data to be misaligned in space, affecting the measurement accuracy.

[0030] To achieve high-precision synchronization, the synchronization control module adopts a hardware triggering method, which simultaneously starts the exposure and data acquisition processes of the dot matrix infrared temperature measurement module and the three-dimensional acquisition module through a single synchronization trigger signal.

[0031] During the synchronous data acquisition process, the dot matrix infrared temperature measurement module acquires the dot matrix temperature information of the target under test and outputs the two-dimensional pixel coordinates (u_ir_i, v_ir_i) and corresponding temperature values ​​T_i (i=1,2,…,N) of N temperature measurement points; the three-dimensional acquisition module acquires the three-dimensional coordinates (X_j,Y_j,Z_j) (j=1,2,…,M, where M≥N) of the feature points on the surface of the target under test.

[0032] III. Coordinate Mapping and Data Fusion: By calling calibration parameters, the system matches the corresponding 3D coordinates to the 2D pixel coordinates of each infrared thermometer point in the temperature data, generating 3D point cloud data with temperature labels. After completing synchronous data acquisition, the data processing module obtains two types of data: two-dimensional pixel coordinates (u_ir_i, v_ir_i) and temperature values ​​T_i from N temperature measurement points of the dot matrix infrared temperature measurement module, and M three-dimensional coordinate points (X_j, Y_j, Z_j) from the three-dimensional acquisition module. It can fuse these two types of data together so that each temperature measurement point not only has a temperature value, but also accurate three-dimensional spatial coordinates.

[0033] Specifically, the data processing module calls the stored calibration parameters (external parameter matrices R, T and internal parameter parameters), and maps the two-dimensional pixel coordinates (u_ir_i, v_ir_i) of each infrared temperature measurement point to three-dimensional space through coordinate transformation formula. Then, it matches the three-dimensional coordinates (X_i, Y_i, Z_i) corresponding to the temperature measurement point from the M three-dimensional coordinate points output by the three-dimensional acquisition module. The coordinate matching can use the nearest neighbor algorithm to improve matching efficiency and accuracy.

[0034] After matching is completed, the matched 3D coordinates and temperature values ​​are aligned to generate 3D point cloud data (T_i,X_i,Y_i,Z_i) (i=1,2,…,N) with temperature labels. Each point in these point cloud data contains both temperature information and 3D spatial location information, which is the data foundation for realizing 3D temperature field reconstruction and spatial tracing of temperature anomalies.

[0035] IV. Emissivity Adaptive Compensation and 3D Temperature Field Reconstruction: Based on the 3D coordinates of the temperature-tagged 3D point cloud data, the normal vector and curvature of each temperature measurement point are calculated. The emissivity is corrected based on the normal vector and curvature, and the temperature value is compensated and calibrated. Then, a 3D reconstruction algorithm is used to generate a 3D temperature field model of the target being measured. It can solve the problem of temperature measurement error caused by inaccurate emissivity setting in traditional infrared temperature measurement methods.

[0036] Traditional infrared thermometry requires users to manually set the emissivity of the target based on experience. However, the emissivity of different materials varies greatly. Even for the same material, different surface morphologies will result in significant changes in emissivity. For targets with complex surface morphologies, a single emissivity setting cannot reflect the actual emissivity differences in different areas, leading to a significant decrease in temperature measurement accuracy.

[0037] Using the 3D coordinates (X_i, Y_i, Z_i) from the obtained temperature-tagged 3D point cloud data, the normal vector and curvature of each temperature measurement point on the surface of the target are calculated. The normal vector reflects the orientation of the surface at that point, and the curvature reflects the degree of curvature of the surface at that point. There is a definite physical relationship between these two geometric parameters and emissivity: for metallic materials, the larger the angle between the surface normal vector and the observation direction, the greater the emissivity is usually; the greater the surface curvature (the more curved the surface), the greater the emissivity is. For non-metallic materials, the emissivity is more significantly affected by the surface curvature.

[0038] Based on the preset correspondence between normal vector, curvature and emissivity, the emissivity of each temperature measurement point is automatically corrected. Then, based on the corrected emissivity, the temperature value T_i is compensated and calibrated to eliminate the temperature measurement deviation caused by the emissivity setting error.

[0039] Finally, a 3D reconstruction algorithm is used to process the compensated and calibrated 3D point cloud data with temperature labels to generate a 3D temperature field model of the target. The 3D reconstruction algorithm can be any of the methods such as Poisson reconstruction, Delaunay triangulation, or moving least squares surface reconstruction. The generated 3D temperature field model can realize interactive operations such as rotation, scaling, and sectioning, and intuitively display the spatial temperature distribution of the target, which is convenient for spatial tracing and visualization analysis of hot spots / cold spots.

[0040] V. Data Output: The processed 3D temperature field data, temperature data, and 3D point cloud data are output to external devices via the communication interface. After completing the 3D temperature field reconstruction, the data processing module outputs the processed data to external devices through the communication interface module. The output data formats include, but are not limited to: 3D point cloud data, temperature data, 3D temperature field visualization images, and fused images of temperature and visible light superimposed on each other. External devices can be personal computers, tablets, smartphones, industrial control systems, or cloud servers, etc. The communication interface module includes one or more of USB-C interface, Wi-Fi module, and Bluetooth module. Users can choose the appropriate data format and transmission method according to their actual needs.

[0041] In one embodiment, such as Figure 2 As shown, the collaborative calibration steps specifically include the following sub-steps: Step 110: Prepare a standard blackbody and a 3D calibration board, and place the standard blackbody in the preset position on the 3D calibration board.

[0042] A standard blackbody is an ideal radiation source with a known precise temperature and an emissivity close to 1. It is used to provide a precise temperature reference for the matrix infrared temperature measurement module. The temperature of the standard blackbody is known and adjustable, preferably from 0℃ to 100℃, with each adjustment interval of 10℃ to ensure the comprehensiveness of the calibration data. The 3D calibration board has multiple feature points with known three-dimensional coordinates to provide a spatial reference for the three-dimensional acquisition module. The standard blackbody is placed at a preset position on the 3D calibration board to ensure that the standard blackbody is simultaneously within the field of view of both the matrix infrared temperature measurement module and the three-dimensional acquisition module.

[0043] Step 120: Fix the measuring device and adjust the distance between the device and the 3D calibration plate so that the 3D calibration plate is completely within the field of view of the device.

[0044] Fix the measuring device on a tripod or special fixture, and adjust the distance between the device and the 3D calibration board so that the 3D calibration board is completely within the field of view of the device. The distance between the device and the 3D calibration board should be selected according to the type of the 3D acquisition module. When adjusting the distance, it is necessary to ensure that the entire 3D calibration board is located within the overlapping area of ​​the field of view of the dot matrix infrared temperature measurement module and the 3D acquisition module.

[0045] Step 130: Control the dot matrix infrared temperature measurement module to collect the temperature data of the standard blackbody and the infrared dot matrix data of the 3D calibration board, and output the two-dimensional pixel coordinates of each infrared temperature measurement point; control the three-dimensional acquisition module to collect the depth data of the 3D calibration board, and output the three-dimensional coordinates of each feature point.

[0046] After the relative positions of the device and the 3D calibration board are fixed, the synchronous control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to collect data simultaneously. The dot matrix infrared temperature measurement module collects the temperature data of the standard blackbody (as a temperature reference value) and the infrared dot matrix data of the 3D calibration board, and outputs the two-dimensional pixel coordinates (u_ir, v_ir) of each infrared temperature measurement point. Since the standard blackbody is located at the preset position of the 3D calibration board, the temperature value of the temperature measurement point corresponding to the position of the standard blackbody in the infrared image output by the dot matrix infrared temperature measurement module should be consistent with the set temperature of the standard blackbody. This provides a benchmark for subsequent temperature calibration.

[0047] Meanwhile, the 3D acquisition module acquires the depth data of the 3D calibration board and outputs the 3D coordinates (X,Y,Z) of each feature point. The depth calculation method of the 3D acquisition module depends on its specific type: the 3D structured light module calculates the depth using triangulation; the TOF module calculates the depth using time-of-flight ranging algorithm; the passive binocular camera calculates the depth using left and right parallax triangulation; and the active binocular module calculates the depth using parallax calculation and pattern distortion fusion.

[0048] Step 140: Change the angle between the device and the 3D calibration plate to obtain infrared two-dimensional coordinate data and three-dimensional coordinate data under multiple postures.

[0049] Change the angle between the device and the 3D calibration plate (from 0° to 90°) and repeat the acquisition operation in step 1.3 under multiple different postures. The purpose of multi-posture calibration is to obtain sufficient sample data to improve the accuracy and robustness of the calibration parameters. By acquiring data at different angles, a wider range of observation geometry conditions can be covered, so that the calibration results can maintain high mapping accuracy under different measurement angles.

[0050] Step 150: Fit the dataset under multiple poses, solve the extrinsic and intrinsic parameters between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module, establish a one-to-one correspondence between each infrared temperature measurement point and the three-dimensional feature point, complete the collaborative calibration, and store the calibration parameters in the data processing module.

[0051] The data processing module performs fitting calculations on the datasets acquired under multiple postures. Specifically, for the data acquired under each posture, the three-dimensional coordinates (X,Y,Z) of each feature point on the 3D calibration board and the two-dimensional pixel coordinates (u_ir,v_ir) of the corresponding infrared temperature measurement point in the dot matrix infrared temperature measurement module are known. Through multiple sets of corresponding two-dimensional-three-dimensional coordinate pairs, the extrinsic parameter matrix and intrinsic parameters between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module can be solved.

[0052] After calibration, a one-to-one correspondence is established between each infrared temperature measurement point (u_ir, v_ir) and the feature points (X, Y, Z) of the three-dimensional acquisition module. The calibration parameters are stored in the flash memory of the data processing module for subsequent actual measurement. Since the dot matrix infrared temperature measurement module and the three-dimensional acquisition module are rigidly connected, the calibration parameters remain stable during use and do not need to be recalibrated before use.

[0053] In one embodiment, the distance between the device and the 3D calibration plate is selected according to the type of the 3D acquisition module. For a 3D structured light module, its depth measurement range is typically 0.3m to 2m, and the preferred calibration distance is 0.5m to 1.5m. For a TOF area array 3D depth module, its depth measurement range is typically 0.2m to over 10m, and the preferred calibration distance can be selected within the range of 0.2m to 10m depending on the actual application scenario. For passive binocular stereo cameras and active binocular modules, their depth measurement range is typically 0.3m to 5m, and the preferred calibration distance is 0.5m to 3m. The overall adaptation range is 0.2m to 10m, covering various application scenarios from close-range high precision to mid-to-long-range large field of view.

[0054] During the collaborative calibration process, calibration data under multiple orientations is obtained by changing the angle between the device and the 3D calibration board. The angle is defined as the angle between the optical axis of the device and the normal of the plane of the 3D calibration board. The angle range is from 0° to 90°. It is preferred to collect data at seven angles: 0°, 15°, 30°, 45°, 60°, 75°, and 90°. A 0° angle indicates that the device is facing the 3D calibration board directly (the optical axis is parallel to the normal of the plane of the calibration board), and a 90° angle indicates that the device is facing the 3D calibration board from the side (the optical axis is perpendicular to the normal of the plane of the calibration board). Multi-angle calibration ensures that the calibration parameters have good applicability under different observation angles.

[0055] The specific implementation of multi-pose calibration is as follows: the 3D calibration plate is fixed on a rotatable turntable, and the measuring device is fixed on the bracket and kept stationary. The angle between the 3D calibration plate and the device is changed by rotating the turntable. Alternatively, the measuring device is fixed on an adjustable gimbal, and the angle between the device and the fixed 3D calibration plate is changed by rotating the gimbal. Both methods can achieve multi-pose calibration. The former is more convenient to operate, while the latter is more suitable for scenarios where the device is already fixedly installed.

[0056] In one embodiment, such as Figure 3 As shown, the synchronous data acquisition steps specifically include the following sub-steps: Step 210: Align the measuring device with the target to be measured, and the synchronous control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to simultaneously expose and acquire data.

[0057] After completing the collaborative calibration and moving the measuring device to the measurement site, the operator aligns the measuring device with the target to be measured, ensuring that the target is completely within the field of view of the device. The measurement program is then started through an external control terminal (such as a smartphone APP or PC software). The synchronous control module sends a hardware synchronous trigger signal, which simultaneously triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to start exposure and data acquisition.

[0058] The synchronization control module uses hardware triggering instead of software triggering. Hardware triggering generates a precise synchronization pulse signal through FPGA or CPLD. The rising or falling edge of this signal arrives at the trigger input of both modules simultaneously, with a transmission delay difference of less than 100ns. Since the acquisition actions of all modules are started by the same hardware trigger signal, the data acquisition times of each module are highly aligned in time.

[0059] For implementations that use passive binocular stereo cameras or active binocular modules as 3D acquisition modules, the synchronization control module also needs to simultaneously trigger the left and right infrared cameras to work synchronously with the dot matrix infrared temperature measurement module to ensure the timing consistency of multiple sensors.

[0060] Step 220: The dot matrix infrared temperature measurement module collects the dot matrix temperature information of the target under test and outputs the two-dimensional pixel coordinates and corresponding temperature values ​​of multiple temperature measurement points.

[0061] After being synchronously triggered, the dot matrix infrared temperature measurement module begins to collect the dot matrix temperature information of the target being measured. The core component of the dot matrix infrared temperature measurement module is a thermopile sensor array or a microbolometer array, and its dot matrix size can be selected as 8×8, 16×16, 32×32 or other specifications according to actual application requirements.

[0062] The dot matrix infrared temperature measurement module outputs two-dimensional pixel coordinates (u_ir_i, v_ir_i) and corresponding temperature values ​​T_i (i=1,2,…,N) for N temperature measurement points. The two-dimensional pixel coordinates (u_ir, v_ir) represent the row and column coordinates of the temperature measurement point in the infrared image. The temperature value T_i is in ℃ or K. The dot matrix infrared temperature measurement module also integrates signal conditioning circuits and analog-to-digital conversion circuits to amplify, filter, and digitize the weak analog signals output by the sensor array.

[0063] Step 230: The 3D acquisition module acquires the 3D coordinates of the feature points on the surface of the target being measured.

[0064] After being synchronously triggered, the 3D acquisition module begins to acquire the 3D depth information of the target under test. The 3D acquisition module outputs M 3D coordinate points (X_j, Y_j, Z_j) (j=1,2,…,M), where M≥N. That is, the number of points output by the 3D acquisition module is not less than the number of temperature measurement points output by the dot matrix infrared temperature measurement module, so as to ensure that each infrared temperature measurement point can find a corresponding matching point in the 3D point cloud.

[0065] The specific acquisition method of the 3D acquisition module depends on its type (3D structured light, TOF, passive binocular or active binocular).

[0066] In one embodiment, the synchronization error between the synchronous control module triggering the dot matrix infrared temperature measurement module and the three-dimensional acquisition module for synchronous exposure and synchronous data acquisition is less than 1μs.

[0067] To achieve high-precision synchronization at the microsecond level, the synchronization control module adopts a hardware triggering method instead of a software triggering method. The software triggering method is affected by factors such as operating system scheduling and interrupt response time, and the synchronization error is usually at the millisecond level or even larger, which cannot meet the requirements of high-precision spatiotemporal synchronization. This invention uses an FPGA or CPLD as the core of the synchronization control module, which integrates a high-precision clock source and a pulse generator.

[0068] The high-precision clock source uses a temperature-compensated crystal oscillator (TCXO) with a frequency stability better than ±1ppm, providing a precise time reference for the synchronization control module. The pulse generator generates a precise synchronization pulse signal based on this clock source, and the pulse width and period can be configured according to actual needs. The synchronization control module triggers the exposure start time of the dot matrix infrared temperature measurement module and the three-dimensional acquisition module simultaneously through a hardware synchronization signal (such as a TTL level pulse). The rising or falling edge of this hardware synchronization signal arrives at the trigger input of the two modules simultaneously, with a transmission delay difference of less than 100ns.

[0069] For the dot matrix infrared temperature measurement module, the synchronous trigger signal initiates the signal integration and readout process of the thermopile sensor array. For the 3D acquisition module, the synchronous trigger signal initiates the exposure and readout process of the infrared CMOS camera. Since the acquisition actions of all modules are initiated by the same hardware trigger signal, the data acquisition times of each module are highly aligned in time, with a synchronization error of less than 1μs.

[0070] The high-precision synchronization mechanism ensures accurate temporal alignment between temperature data and 3D depth data. For static targets, the impact of time alignment error is small; however, for dynamic or quasi-dynamic targets, time deviation will directly cause spatial misalignment between temperature data and 3D spatial data. A synchronization error within 1μs can control this spatial misalignment within the sub-millimeter level, meeting the accuracy requirements of most industrial application scenarios.

[0071] In one embodiment, the coordinate mapping and data fusion step specifically includes the following: Based on the two-dimensional pixel coordinates of the infrared temperature measurement points, coordinate transformation is performed using extrinsic and intrinsic parameters to match the three-dimensional coordinates acquired by the three-dimensional acquisition module to obtain the three-dimensional coordinates corresponding to each temperature measurement point.

[0072] The data processing module calls the stored calibration parameters, including the extrinsic parameter matrix and intrinsic parameter parameters.

[0073] The basic process of coordinate transformation is as follows: For the two-dimensional pixel coordinates (u_ir_i, v_ir_i) of each infrared temperature measurement point, firstly, they are converted into coordinates in the normalized image coordinate system through intrinsic parameters, and then they are converted into the coordinate system of the three-dimensional acquisition module through extrinsic parameter matrix to obtain the projection ray direction of the temperature measurement point in three-dimensional space. Then, from the M three-dimensional coordinate points (X_j, Y_j, Z_j) output by the three-dimensional acquisition module, the closest point is searched along the projection ray direction and used as the three-dimensional coordinates (X_i, Y_i, Z_i) corresponding to the temperature measurement point.

[0074] The accuracy of coordinate matching directly affects the accuracy of the final three-dimensional temperature field model. Since the dot matrix infrared temperature measurement module usually has a small dot matrix size (e.g., 16×16, a total of 256 points), while the number of points output by the three-dimensional acquisition module is usually large (e.g., tens of thousands of points), it is necessary to perform accurate nearest neighbor search in the three-dimensional point cloud. The nearest neighbor algorithm is preferred for coordinate matching to improve matching efficiency and accuracy.

[0075] The matched 3D coordinates and temperature values ​​are aligned to generate 3D point cloud data with temperature labels.

[0076] After completing the three-dimensional coordinate matching of each temperature measurement point, the matched three-dimensional coordinates (X_i, Y_i, Z_i) and temperature value T_i need to be aligned. The alignment process includes unifying the data format, calibrating the coordinate system, and removing outliers.

[0077] After alignment, temperature-labeled 3D point cloud data (T_i, X_i, Y_i, Z_i) (i=1,2,…,N) are generated. Each point in these point cloud data contains information in four dimensions: temperature value T_i and 3D spatial coordinates (X_i, Y_i, Z_i). The temperature-labeled 3D point cloud data is the data basis for subsequent emissivity adaptive compensation and 3D temperature field reconstruction.

[0078] In one embodiment, coordinate matching employs the nearest neighbor algorithm.

[0079] The basic principle of the nearest neighbor algorithm is: for a target point to be matched, search for several points closest to the target point in the reference point cloud, and take the closest point as the matching result. In the application scenario of this invention, the target point to be matched is the three-dimensional projection point obtained by coordinate transformation of the two-dimensional pixel coordinates of N infrared temperature measurement points output by the dot matrix infrared temperature measurement module, and the reference point cloud is the M three-dimensional coordinate points output by the three-dimensional acquisition module.

[0080] Specifically, for each infrared temperature measurement point i, its two-dimensional pixel coordinates (u_ir_i, v_ir_i) are transformed to obtain a three-dimensional projection point P_i=(X'_i,Y'_i,Z'_i). Then, among the M three-dimensional coordinate points (X_j,Y_j,Z_j) output by the three-dimensional acquisition module, the point with the closest Euclidean distance to P_i is searched, and this point is taken as the three-dimensional coordinates (X_i,Y_i,Z_i) corresponding to the temperature measurement point i. The computational complexity of the nearest neighbor algorithm is O(N×M). For N=256 and M=10000, the computation is about 2.56 million distance calculations, which can be completed in milliseconds on an embedded processor, meeting the requirements of real-time processing.

[0081] The nearest neighbor algorithm has the advantages of being simple to implement, fast to calculate, and high to match. It is especially suitable for application scenarios where the number of points in the dot matrix infrared temperature measurement module is small and the number of points in the three-dimensional acquisition module is large. In other implementations, the K-Nearest Neighbors algorithm or the radius search algorithm can be used as alternatives. However, considering the real-time requirements, the nearest neighbor algorithm is the preferred solution.

[0082] In one embodiment, such as Figure 4 As shown, the emissivity adaptive compensation and three-dimensional temperature field reconstruction steps specifically include the following sub-steps: Step 410: Calculate the normal vector and curvature of each temperature measurement point on the surface of the target object based on the three-dimensional coordinates in the three-dimensional point cloud data with temperature labels.

[0083] The data processing module calculates the normal vector and curvature of each temperature measurement point on the surface of the target object based on the three-dimensional coordinates (X_i,Y_i,Z_i) in the generated three-dimensional point cloud data (T_i,X_i,Y_i,Z_i) with temperature labels.

[0084] The normal vector can be calculated using principal component analysis (PCA): For each temperature measurement point i, take its K neighboring points (K is between 10 and 20), calculate the covariance matrix of these points, and the eigenvector corresponding to the smallest eigenvalue of the covariance matrix is ​​the direction of the normal vector at that point. The normal vector reflects the orientation of the surface at that point, that is, the direction of the tangent plane of the surface at that point.

[0085] Curvature can be calculated using a local fitting method: for each temperature measurement point i, take its K neighboring points, and perform least squares fitting on the neighboring points using the quadratic surface equation z=f(x,y). Then, calculate the Gaussian curvature and average curvature of the point based on the second-order partial derivative of the fitted surface. Curvature reflects the degree of curvature of the surface at that point; the greater the curvature, the more curved the surface.

[0086] Step 420: Automatically correct the emissivity of each temperature measurement point according to the preset correspondence between normal vector, curvature and emissivity.

[0087] The data processing module automatically corrects the emissivity of each temperature measurement point based on the preset correspondence between normal vector, curvature and emissivity. This correspondence is obtained by fitting a large amount of experimental data and covers the emissivity characteristics of different materials.

[0088] For metallic materials, the emissivity increases with the angle between the surface normal vector and the observation direction, and also with the surface curvature. For non-metallic materials, the emissivity is more significantly affected by the surface curvature, while the influence of the normal vector direction is relatively small. By pre-establishing an emissivity database or fitting formula for different materials under different normal vector and curvature conditions, the corresponding emissivity value can be automatically matched according to the three-dimensional geometric characteristics of each temperature measurement point in actual measurement.

[0089] Specifically, for each temperature measurement point i, based on its normal vector and curvature value, an interpolation query is performed in the emissivity database or the result is calculated by substituting into the fitting formula to obtain the emissivity correction value ε_i for that point. This process achieves point-by-point adaptive correction of emissivity without the need for manual intervention.

[0090] Step 430: Perform a compensation calibration on the temperature value based on the corrected emissivity.

[0091] The basic principle of infrared thermometry is based on Planck's blackbody radiation law. For a real object, the ratio of its radiative exitance M to the radiative exitance M_bb of a blackbody at the same temperature is defined as the emissivity ε. Infrared thermometry devices infer the temperature of the target by detecting the infrared radiation intensity of the target, and the radiation intensity is closely related to the emissivity of the target surface.

[0092] Where T is the actual temperature, V is the detector response signal, V_offset is the dark field offset signal, K is the instrument constant, ε is the emissivity, and n is the band-related exponent. When the emissivity ε changes, the actual temperature T needs to be adjusted accordingly.

[0093] The data processing module substitutes the corrected emissivity ε_i into the above radiation thermometry formula to compensate and calibrate the temperature value T_i. The compensated and calibrated temperature value T'_i eliminates the temperature measurement deviation caused by the emissivity setting error, and the accuracy is greatly improved.

[0094] Step 440: Use a 3D reconstruction algorithm to process the compensated and calibrated 3D point cloud data with temperature labels to generate a 3D temperature field model of the target under test.

[0095] The data processing module uses a 3D reconstruction algorithm to process the compensated and calibrated 3D point cloud data (T'_i,X_i,Y_i,Z_i) with temperature labels to generate a 3D temperature field model of the target.

[0096] 3D reconstruction algorithms can be implemented using various methods, including but not limited to: (1) Poisson Reconstruction: Based on the normal vector information of the point cloud, the implicit surface is reconstructed by solving the Poisson equation. Then, the isosurface mesh is extracted by the Moving Cubes algorithm to generate a three-dimensional surface model of the target. The temperature label is then mapped onto the model surface to form a three-dimensional temperature field model.

[0097] (2) Delaunay triangulation: Delaunay triangulation is performed on the 3D point cloud to generate a triangular mesh model. The temperature value is then assigned as a vertex attribute to each vertex of the triangular mesh. The temperature distribution of the entire mesh surface is obtained by interpolation between vertices.

[0098] (3) Moving Least Squares (MLS) surface reconstruction: Smooth surfaces are generated by local weighted least squares fitting, which is suitable for targets with high point cloud density and relatively smooth surfaces.

[0099] The generated three-dimensional temperature field model can perform interactive operations such as rotation, scaling, and sectioning, and intuitively display the spatial temperature distribution of the target being measured. This allows operators to observe and analyze the temperature distribution characteristics from any angle, enabling precise spatial tracing of hot / cold spots.

[0100] In one embodiment, the preset correspondence between normal vector, curvature and emissivity is obtained by fitting experimental data, covering the emissivity characteristics of different materials.

[0101] The experimental method for establishing this correspondence is as follows: Experimental preparation: Prepare standard samples of various materials, including but not limited to: different types of metals (such as aluminum, copper, steel, cast iron, etc., with samples of different surface roughness for each metal), different types of plastics (such as ABS, PC, PE, PP, etc.), ceramic materials, human skin simulation materials, painted surfaces of different colors, etc. Each sample is processed into a curved surface shape with different radii of curvature (such as flat, convex, concave) to cover different curvature ranges.

[0102] Experimental equipment: A standard radiation source with known emissivity (such as a standard blackbody) was used to accurately calibrate the array infrared temperature measurement module; a high-precision 3D scanner was used to obtain the accurate 3D geometric model of the sample; and a contact temperature sensor (such as a thermocouple) was used to measure the actual temperature of the sample as a reference.

[0103] Experimental Procedure: Each standard sample was placed in a constant temperature environment. After the temperature stabilized, the device of this invention was used to collect temperature data and three-dimensional depth data of each sample from different observation angles. For each measurement point on each sample, its three-dimensional coordinates, normal vector, curvature, measured temperature value and infrared temperature measurement value were recorded. By comparing the infrared temperature measurement value with the actual temperature value, the actual emissivity of the point under the geometric conditions was inferred. Statistical analysis was performed on a large amount of experimental data to establish the mapping relationship between normal vector, curvature and emissivity.

[0104] Data fitting: Perform multivariate nonlinear regression analysis on the experimental data to fit the functional relationship between the normal vector, curvature and emissivity. For different material categories, establish different fitting formulas or lookup tables. The fitting formula can be obtained by multinomial regression, neural network or Gaussian process regression and other methods.

[0105] Experimental data show that for metallic materials, emissivity increases monotonically with the increase of the angle between the normal vector and the observation direction, and the rate of increase is related to the type of metal and the surface roughness. For non-metallic materials, emissivity is more significantly affected by curvature. Emissivity increases with increasing curvature, and the rate of increase is different for different non-metallic materials.

[0106] The fitted correspondence is pre-stored in the data processing module. In actual measurement, for each temperature measurement point, the emissivity correction value can be obtained by querying or calculating the correspondence based on its material type, normal vector and curvature value.

[0107] In one embodiment, the 3D acquisition module is selected from any one of a 3D structured light module, a TOF area array 3D depth module, a passive binocular stereo camera, or an active binocular module. The method flow of the present invention is independent of the type of 3D acquisition module, and only needs to be adapted to the depth calculation logic of the corresponding module.

[0108] When the 3D acquisition module is a 3D structured light module: The 3D structured light module includes a VCSEL infrared dot matrix emitter, an infrared CMOS camera, and a depth computing chip.

[0109] In steps 130 (cooperative calibration) and 230 (synchronous data acquisition), the VCSEL infrared dot matrix emitter (operating band 940nm) projects coded speckle or regular dot matrix onto the calibration plate or the surface of the target under test. The coded speckle pattern has a high degree of randomness and uniqueness, which facilitates subsequent pattern matching and 3D reconstruction. The infrared CMOS camera captures the image of the light spot deformation projected onto the target surface. The depth calculation chip compares and matches the captured speckle pattern with the speckle pattern on the reference plane, calculates the parallax displacement of each speckle, and solves the 3D coordinates (X,Y,Z) of each feature point by combining the baseline distance between the emitter and the camera and the principle of triangulation.

[0110] The 3D structured light module is suitable for close-range (0.3m to 2m) and high-precision measurement scenarios, with a depth accuracy of ±1mm@1m.

[0111] When the 3D acquisition module is a planar TOF 3D depth module: The area array TOF 3D depth module includes a near-infrared floodlight emitter, an area array TOF depth sensor, and a depth calculation chip.

[0112] In steps 130 and 230, a near-infrared floodlight emitter (operating at 850nm or 940nm) projects near-infrared floodlight onto a calibration plate or the surface of the target being measured. Unlike structured light modules, TOF modules do not require the projection of a specific coded pattern; instead, they project uniform floodlight. The area array TOF depth sensor detects the time-of-flight difference: the near-infrared floodlight emitter emits a short pulse (nanosecond level) of near-infrared light onto the target surface. After reflection from the target surface, the light returns to the area array TOF depth sensor. The sensor detects the time difference Δt between the emitted and received pulses and calculates the distance value corresponding to each pixel using the formula d=c×Δt / 2 (where c is the speed of light). The depth calculation chip calculates the three-dimensional coordinates using a time-of-flight ranging algorithm.

[0113] The depth measurement range of the area array TOF 3D depth module is typically 0.2m to over 10m, with a depth accuracy of approximately ±2cm@3m. It is suitable for medium- to long-distance measurement scenarios and has strong anti-interference capabilities against ambient light.

[0114] When the 3D acquisition module is a passive binocular stereo camera: The passive binocular stereo camera includes a left infrared camera, a right infrared camera, and a binocular stereo matching chip, and has no infrared projection source.

[0115] In steps 130 and 230, the left and right infrared cameras simultaneously acquire left and right views of the calibration board or the target under test. The resolution of the two infrared cameras is preferably VGA (640×480) or higher, with a frame rate of 30fps. The baseline distance between them is 50mm to 100mm. The binocular stereo matching chip performs stereo matching on the two images, finding the corresponding point of each pixel in the left view in the right view, and calculating the horizontal disparity d between them. According to the triangulation principle, the relationship between depth Z and disparity d is Z = f × B / d, where f is the camera focal length and B is the baseline distance. Based on this, the binocular stereo matching chip calculates the three-dimensional coordinates (X, Y, Z) of each feature point.

[0116] Passive binocular stereo cameras do not require active light source emission, have low power consumption, and are suitable for outdoor high-light environments and battery-powered portable applications.

[0117] When the 3D acquisition module is an active binocular module: The active binocular module includes a left infrared camera, a right infrared camera, a VCSEL infrared speckle projector, and a depth calculation chip.

[0118] In steps 130 and 230, the VCSEL infrared speckle projector (operating at 940nm) projects coded speckles onto the calibration plate or the surface of the target object to supplement the target texture. The left and right infrared cameras simultaneously acquire left and right views. The depth calculation chip combines disparity calculation and pattern distortion analysis to calculate the 3D coordinates: on one hand, binocular stereo matching is performed using the left and right views acquired by the left and right infrared cameras to calculate disparity and obtain the 3D coordinates based on triangulation (i.e., passive binocular method); on the other hand, the depth value is corrected by analyzing the deformation of the speckle pattern projected by the VCSEL infrared speckle projector on the target surface (i.e., structured light method). The two methods complement and verify each other.

[0119] The active binocular module combines the advantages of passive binoculars in resisting strong light and adapting to weak textures under structured light, making it suitable for complex indoor and outdoor scenes.

[0120] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially as indicated, these steps are not necessarily executed in the indicated order. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A three-dimensional temperature measurement method integrating dot-matrix infrared thermography and a 3D camera, characterized in that, include: Collaborative calibration establishes a pixel-level mapping relationship between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to obtain calibration parameters; Synchronous data acquisition involves controlling the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to expose synchronously, acquiring temperature data and three-dimensional depth data of the target being measured. Coordinate mapping and data fusion are performed by calling the calibration parameters and matching the corresponding three-dimensional coordinates based on the two-dimensional pixel coordinates of each infrared temperature measurement point in the temperature data to generate three-dimensional point cloud data with temperature labels. Emissivity adaptive compensation and three-dimensional temperature field reconstruction: The normal vector and curvature of each temperature measurement point are calculated based on the three-dimensional coordinates in the three-dimensional point cloud data with temperature labels. The emissivity is corrected based on the normal vector and curvature, and the temperature value is compensated and calibrated. Then, a three-dimensional reconstruction algorithm is used to generate a three-dimensional temperature field model of the target under test. Data output: The processed 3D temperature field data, temperature data, and 3D point cloud data are output to external devices through the communication interface.

2. The three-dimensional temperature measurement method fused with dot matrix infrared thermometry and a 3D camera as described in claim 1, characterized in that, The collaborative calibration steps specifically include: Prepare a standard blackbody and a 3D calibration plate, and place the standard blackbody at a preset position on the 3D calibration plate; Fix the measuring device and adjust the distance between the device and the 3D calibration plate so that the 3D calibration plate is completely within the field of view of the device; The system controls the dot matrix infrared temperature measurement module to collect the temperature data of the standard blackbody and the infrared dot matrix data of the 3D calibration plate, and outputs the two-dimensional pixel coordinates of each infrared temperature measurement point. The system also controls the three-dimensional acquisition module to collect the depth data of the 3D calibration plate and outputs the three-dimensional coordinates of each feature point. By changing the angle between the device and the 3D calibration plate, infrared two-dimensional coordinate data and three-dimensional coordinate data under multiple postures can be obtained; The dataset under the multi-pose conditions is fitted and calculated to solve the extrinsic and intrinsic parameters between the dot matrix infrared temperature measurement module and the three-dimensional acquisition module, establish a one-to-one correspondence between each infrared temperature measurement point and the three-dimensional feature point, complete the collaborative calibration, and store the calibration parameters in the data processing module.

3. The three-dimensional temperature measurement method fused with dot matrix infrared thermometry and a 3D camera as described in claim 1, characterized in that, The distance between the device and the 3D calibration plate is 0.2m to 10m, and the angle between the device and the 3D calibration plate varies from 0° to 90°.

4. The three-dimensional temperature measurement method fused with dot matrix infrared thermometry and a 3D camera as described in claim 3, characterized in that, The synchronous data acquisition step specifically includes: The measuring device is aligned with the target to be measured, and the synchronous control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to simultaneously expose and acquire data. The dot matrix infrared temperature measurement module collects the dot matrix temperature information of the target under test and outputs the two-dimensional pixel coordinates and corresponding temperature values ​​of multiple temperature measurement points. The three-dimensional acquisition module acquires the three-dimensional coordinates of the feature points on the surface of the target being measured.

5. The three-dimensional temperature measurement method fused with dot matrix infrared thermometry and a 3D camera as described in claim 1, characterized in that, The synchronization control module triggers the dot matrix infrared temperature measurement module and the three-dimensional acquisition module to synchronize exposure and data acquisition, with a synchronization error of less than 1μs.

6. The three-dimensional temperature measurement method fusion of dot matrix infrared thermometry and 3D camera as described in claim 1, characterized in that, Based on the two-dimensional pixel coordinates of the infrared temperature measurement point, coordinate transformation is performed through the external parameter matrix and the internal parameter to obtain the three-dimensional coordinates of each temperature measurement point from the three-dimensional coordinates acquired by the three-dimensional acquisition module. The matched 3D coordinates and the temperature values ​​are aligned to generate 3D point cloud data with temperature labels.

7. The three-dimensional temperature measurement method fusion of dot matrix infrared thermometry and 3D camera as described in claim 6, characterized in that, The coordinate matching uses the nearest neighbor algorithm.

8. The three-dimensional temperature measurement method fusion of dot matrix infrared thermometry and 3D camera as described in claim 1, characterized in that, The emissivity adaptive compensation and three-dimensional temperature field reconstruction steps specifically include: Based on the three-dimensional coordinates in the three-dimensional point cloud data with temperature labels, calculate the normal vector and curvature of each temperature measurement point on the surface of the target being measured; Based on the preset correspondence between normal vector, curvature and emissivity, the emissivity of each temperature measurement point is automatically corrected; The temperature value is compensated and calibrated based on the corrected emissivity; A three-dimensional reconstruction algorithm is used to process the compensated and calibrated three-dimensional point cloud data with temperature labels to generate a three-dimensional temperature field model of the target under test.

9. The three-dimensional temperature measurement method fusion of dot matrix infrared thermometry and 3D camera as described in claim 8, characterized in that, The preset correspondence between normal vector, curvature and emissivity is obtained by fitting experimental data, covering the emissivity characteristics of different materials.

10. The three-dimensional temperature measurement method fused with dot matrix infrared thermometry and a 3D camera as described in claim 1, characterized in that, The three-dimensional acquisition module is selected from any one of the following: 3D structured light module, area array TOF three-dimensional depth module, passive binocular stereo camera, or active binocular module. When the three-dimensional acquisition module is a 3D structured light module, the coded speckle or regular dot matrix is ​​projected onto the calibration plate or the surface of the target under test by the VCSEL infrared dot matrix emitter, the infrared CMOS camera captures the image of the light spot deformation, and the depth calculation chip calculates the three-dimensional coordinates by the triangulation method. When the three-dimensional acquisition module is a planar TOF three-dimensional depth module, near-infrared floodlight is projected onto the calibration plate or the surface of the target being measured by a near-infrared floodlight emitter. The planar TOF depth sensor detects the time-of-flight difference of light, and the depth calculation chip calculates the three-dimensional coordinates through the time-of-flight ranging algorithm. When the three-dimensional acquisition module is a passive binocular stereo camera, the left and right views of the calibration plate or the target under test are acquired simultaneously by the left infrared camera and the right infrared camera. The binocular stereo matching chip calculates the left and right parallax and solves the three-dimensional coordinates by combining the triangulation method. When the 3D acquisition module is an active binocular module, coded speckle is projected onto the calibration plate or the surface of the target being measured through the VCSEL infrared speckle projector. The left and right infrared cameras simultaneously acquire left and right views, and the depth calculation chip combines parallax calculation and pattern distortion analysis to solve the 3D coordinates.

Images