POS prior assisted heterogeneous unmanned aerial vehicle fine thermal infrared color point cloud generation method
The generation of thermal infrared color point clouds through POS prior assisted methods solves the problems of low thermal infrared image resolution and poor flexibility of traditional calibration methods in long-distance operations of drones, and achieves high-precision thermal infrared color point cloud generation and image registration, which is suitable for heterogeneous drone platforms.
Patent Information
- Application Number
- CN202510554966.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-08-15
AI Technical Summary
In the long-distance operation of drones, the thermal infrared image has low resolution and severe noise interference, which leads to difficulty in three-dimensional reconstruction. The traditional calibration method is poor in flexibility and high cost, making it difficult to adapt to equipment vibration and environmental changes, and the registration of heterologous images is difficult.
The POS prior auxiliary method is used to generate visible light dense point clouds, calculate external orientation elements of the thermal infrared camera, combine with the GNSS/IMU system for geometric calibration, and use phase consistency and dense feature matching deep learning network for image registration to generate fine thermal infrared color point clouds.
It realizes high-precision generation of thermal infrared color point clouds under non-calibrated conditions, improves system deployment flexibility and image matching robustness, is suitable for heterogeneous drone platforms, and improves the spatial positioning accuracy and detail retention capabilities of thermal infrared images.
Smart Images
Figure CN120495364A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of photogrammetry and three-dimensional imaging, and in particular to a method for generating fine thermal infrared color point clouds of heterogeneous unmanned aerial vehicles based on a priori assistance of a Position & Orientation System (POS). Background Art
[0002] Visible light images have high spatial resolution and clear texture information, but their performance is limited in low-light environments such as darkness and smoke, and they cannot provide temperature information of the target object. In contrast, thermal infrared images can reflect the temperature distribution on the surface of an object and are suitable for detecting night environments and hidden targets. However, due to the long wavelength of thermal infrared images, their spatial resolution is low and the texture information is weak, resulting in blurred features in the image, making it difficult to accurately detect and describe, resulting in obvious differences between the visible light image and the thermal infrared image for the same object. In addition, two-dimensional images cannot effectively distinguish occlusion problems caused by overlapping object positions, while three-dimensional point clouds have spatial position information and have obvious advantages in distinguishing objects. Therefore, visible light point clouds with thermal information, which combine the advantages of visible light images and thermal infrared images, have shown important application prospects in the fields of building energy consumption assessment, defect monitoring and safety monitoring.
[0003] Thermal infrared color point clouds are typically reconstructed from sequential visible light and thermal infrared images using structure-from-motion (SfM) or multi-view stereo (MVS) methods. Point cloud registration or extrinsic parameters are then used to map the thermal infrared information to the visible light point cloud. Photogrammetry techniques based on SfM or MVS methods can effectively reconstruct three-dimensional thermal information within a small area. However, for long-range UAV operations, reconstruction becomes more challenging due to the dramatic decrease in thermal infrared image resolution. On the one hand, low-resolution thermal infrared images fail to provide sufficient valid connection points. On the other hand, subtle temperature differences and significant noise interference can easily lead to mismatches between thermal infrared images, making direct 3D point cloud reconstruction from thermal infrared images difficult. Therefore, in practical applications, extrinsic parameters are often used to map low-resolution thermal infrared images to existing visible light point clouds to obtain more effective 3D reconstruction results.
[0004] External parameter calibration is a key technology for achieving data fusion between visible light and thermal infrared sensors. Currently, mainstream methods rely primarily on two-dimensional calibration plates or three-dimensional test fields with targets. While these methods can achieve a certain degree of calibration accuracy, they require specialized calibration scenarios and equipment, are less flexible, and are costly. Furthermore, the calibration field is complex to maintain and requires a high level of personnel in actual operation. Furthermore, this static calibration method cannot cope with changes in the relative position of sensors caused by equipment vibration or environmental changes during operation, making it difficult for the calibration results to remain valid over the long term. Therefore, it is particularly important to conduct dynamic calibration based on actual operational data.
[0005] In recent years, target-free multi-sensor dynamic calibration methods have received widespread attention. These methods are primarily implemented through the automatic registration of heterogeneous imagery, offering flexibility and freedom from calibration field restrictions. However, heterogeneous imagery exhibits significant data discrepancies, making direct registration challenging. Current methods rely on strict spatiotemporal synchronization of visible and thermal infrared imagery to minimize the impact of perspective differences on registration results. This stringent synchronization requirement places high demands on the application scenario and payload capacity of the UAV, making it particularly difficult to apply to small UAV systems with separately deployed sensors. Therefore, it is crucial to develop a thermal infrared color point cloud generation method that is less demanding on spatiotemporal synchronization and applicable to heterogeneous UAV platforms. Summary of the Invention
[0006] The purpose of the present invention is to provide a method for generating fine thermal infrared color point clouds of heterogeneous UAVs based on POS prior information under the restrictive conditions of missing pre-calibration information, low resolution of thermal infrared images, and payload separation of visible light cameras and thermal infrared cameras.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] A method for generating fine thermal infrared color point clouds of heterogeneous UAVs based on POS prior assistance includes the following steps:
[0009] (1) Based on conventional photogrammetry methods, feature processing is performed on the acquired visible light images and corresponding POS data to generate a visible light dense point cloud;
[0010] (2) For the thermal infrared image collected by the thermal infrared-UAV system, the target thermal infrared image is aligned with the dense point cloud by selecting the same-name points, and the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system are calculated;
[0011] (3) Based on the exterior orientation elements obtained in step (2), combined with the POS data recorded by the thermal infrared-UAV system, the geometric calibration of the thermal infrared camera sensor and the Global Navigation Satellite System / Inertial Measurement Unit (GNSS / IMU) system is completed;
[0012] (4) Based on the geometric calibration model established in step (3), the external parameters of the thermal infrared camera in the point cloud coordinate system are calculated through the POS data, the spatial mapping relationship between the thermal infrared image and the visible light point cloud is constructed, and the preliminary thermal infrared color point cloud is automatically generated;
[0013] (5) The initially generated thermal infrared color point cloud is projected onto a two-dimensional image plane based on external parameters to generate a projection image. Based on the phase consistency and dense feature matching deep learning network, heterogeneous feature matching between the projection image and the thermal infrared image is achieved to obtain the relative orientation parameters of the two.
[0014] (6) Based on the relative orientation parameters of step (5), the thermal infrared color point cloud generated in step (4) is refined to finally obtain a refined thermal infrared color point cloud.
[0015] The step (1) generates a dense point cloud, comprising the following steps: ① firstly performing feature extraction and matching of same-name points on a plurality of images, establishing a spatial constraint relationship between the images, thereby generating a sparse point cloud; ② on this basis, using bundle adjustment to jointly optimize the observation values, and solving the high-precision internal and external orientation elements of the camera; ③ finally, based on the camera posture parameters obtained by the above optimization, dense matching is performed on the images to obtain a high-precision three-dimensional dense point cloud.
[0016] The step (2) calculates the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system, including the following steps: ① selecting a group of corresponding points of the same name in the thermal infrared image and the dense point cloud respectively, and establishing a correspondence between the two-dimensional image points and the three-dimensional point cloud; ② based on the above correspondence, using the rear intersection algorithm to solve the posture and obtain the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system.
[0017] The step (3) of geometric calibration of the thermal infrared camera and the GNSS / IMU includes the following:
[0018] ① Assume that the phase center of the GNSS antenna coincides with the center of the IMU, and regard the GNSS and IMU as an integrated sensor;
[0019] ② According to the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system obtained in step (2), combined with the rigid connection relationship between the thermal infrared camera and the GNSS / IMU, and the POS information corresponding to the collected image, determine the relative exterior orientation elements between the thermal infrared camera and the GNSS / IMU.
[0020] In the step (4), the step of obtaining the preliminary thermal infrared color point cloud includes: ① converting the point cloud coordinate system to the navigation coordinate system, and the conversion matrix is calculated based on the longitude and latitude of the GNSS / IMU center at the imaging time; ② converting the navigation coordinate system to the GNSS / IMU coordinate system, and the conversion matrix is calculated based on the attitude angle (including heading angle, pitch angle and roll angle) recorded by the IMU; ③ converting the GNSS / IMU coordinate system to the thermal infrared camera coordinate system, and the conversion matrix is calculated based on the geometric calibration relationship between the thermal infrared camera and the GNSS / IMU; ④ constructing the spatial mapping relationship between the thermal infrared image and the visible light point cloud based on the coordinate system conversion parameters obtained in steps ① to ③; ⑤ automatically projecting the remaining thermal infrared images onto the visible light point cloud based on the spatial mapping relationship to generate a preliminary thermal infrared color point cloud.
[0021] In the step (5), the relative orientation parameters between the virtual camera and the thermal infrared camera are obtained, which includes the following steps: ① constructing a virtual camera model based on the geometric calibration parameters and the POS data at the current imaging moment, wherein the resolution of the virtual camera is consistent with that of the thermal infrared camera; projecting the dense point cloud onto the two-dimensional image plane of the virtual camera to generate a projection image consistent with the resolution of the thermal infrared image; ② extracting feature points in the projection image and the thermal infrared image based on the phase consistency method, inputting the corresponding feature point pairs in the two images into a dense feature matching network, performing feature description and matching, and obtaining the relative orientation parameters between the two.
[0022] In the step (6), the step of obtaining a fine thermal infrared color point cloud includes: further correcting the relative spatial relationship between the point cloud coordinate system and the thermal infrared camera coordinate system based on the relative orientation parameters between the thermal infrared image and the projected image obtained in step (5); optimizing the spatial accuracy of the preliminary thermal infrared color point cloud through the corrected coordinate transformation relationship, thereby improving the geometric consistency and thermal information accuracy of the point cloud, and finally generating a fine thermal infrared color point cloud.
[0023] The present invention adopts the above technical solution, which has the following advantages:
[0024] 1. This invention does not rely on traditional targets or pre-built calibration scenarios and can complete the geometric calibration of thermal infrared cameras and GNSS / IMU systems under non-calibration conditions, improving the flexibility of system deployment and adaptability to field operations. It is particularly suitable for heterogeneous UAV platforms with separated sensor deployment.
[0025] 2. The present invention introduces POS prior information, combines the phase consistency feature extraction method with the dense feature deep learning network, and realizes robust registration of heterogeneous images, effectively overcoming the image matching difficulties caused by radiation differences and perspective changes, and significantly improving the spatial positioning accuracy and detail retention ability of thermal infrared color point clouds, and has strong robustness and automatic processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 The dense point cloud obtained;
[0027] Figure 2 Select a sample graph for the points of the same name;
[0028] Figure 3 Schematic diagram of the asynchronous and independent operation of the visible light-UAV system and the thermal infrared-UAV system;
[0029] Figure 4 This is the fusion result of some visible light point clouds and thermal infrared images;
[0030] Figure 5 These are the feature point detection results; (a) projection image feature points; (b) thermal infrared image feature points;
[0031] Figure 6 This is the result of heterogeneous image registration;
[0032] Figure 7 Thermal infrared color point cloud before and after refinement processing; (a) preliminary thermal infrared color point cloud; (b) refined thermal infrared color point cloud. DETAILED DESCRIPTION
[0033] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0034] The present invention provides a method for generating a fine thermal infrared color point cloud of a heterogeneous UAV assisted by a POS prior, comprising the following steps:
[0035] 1. Generate a visible light dense point cloud using conventional photogrammetry methods, including the following steps:
[0036] ① First, perform feature extraction and point matching on multiple visible light images, establish spatial geometric constraints between images, and generate a preliminary sparse point cloud;
[0037] ② On this basis, the bundle adjustment is used to jointly optimize the observation values to obtain the high-precision interior and exterior orientation elements of the camera;
[0038] ③Finally, based on the optimized camera pose parameters, the images are densely matched to reconstruct a high-precision three-dimensional dense point cloud. Figure 1 is the dense point cloud obtained.
[0039] 2. Calculate the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system:
[0040] The transformation parameters between thermal infrared imagery and visible light dense point clouds are solved using a manually selected point intersection method. This method has high pose accuracy and is suitable for initial registration between images and 3D point clouds. Figure 2 This is an example of selecting points of the same name.
[0041] To ensure the accuracy of the resection algorithm, the selected points of the same name should meet the following principles:
[0042] ① The 3D points in the point cloud and the 2D points in the image should form a non-collinear and non-coplanar spatial geometric structure;
[0043] ② The same-name points should be evenly distributed within the image to improve spatial stability;
[0044] ③ The number of pairs of points with the same name should not be less than 6 to meet the basic constraints of the resection solution;
[0045] ④ Prioritize the corners, inflection points and obvious edge areas in the image as the same-name points to improve the accuracy and robustness of the registration.
[0046] 3. Geometric calibration of thermal infrared camera sensor and GNSS / IMU system:
[0047] For thermal infrared-drone systems, the calibration of the thermal infrared camera and the GNSS / IMU is to determine the relative pose relationship between the two sensors. Since the GNSS antenna phase center is very close to the center of the IMU, we tentatively assume that the GNSS antenna phase center coincides with the center of the IMU, and in this case, the GNSS / IMU can be considered the same sensor. Figure 3 This demonstration demonstrates the asynchronous, independent operation of both the visible light-to-drone system and the thermal infrared-to-drone system. Since the thermal infrared camera and GNSS / IMU are rigidly connected, the thermal infrared camera's extrinsic parameters in the point cloud coordinate system can be calculated from the numerical values of either GNSS / IMU in the point cloud coordinate system, simply by determining their relative geometric relationship. The following describes the calibration method for the thermal infrared camera and GNSS / IMU, along with the derived formulas.
[0048] Assume there are two coordinate systems, the rotation matrix R and translation vector t between the two coordinate systems can form a homogeneous matrix
[0049]
[0050] To solve the posture of the thermal infrared camera coordinate system in the point cloud coordinate system and location It has to go through a series of coordinate system transformations. This process can be decomposed into multiple steps of coordinate system transformations, namely:
[0051]
[0052] in Represents the pose matrix of the thermal infrared camera coordinate system in the point cloud coordinate system, Represents the pose matrix of the navigation coordinate system in the point cloud coordinate system, Represents the pose matrix of the GNSS / IMU coordinate system in the navigation coordinate system, Represents the pose matrix of the thermal infrared camera coordinate system in the GNSS / IMU coordinate system.
[0053] The rotation matrix from the point cloud coordinate system to the thermal infrared camera coordinate system can be obtained from formula (2):
[0054]
[0055] in Represents the posture of the navigation coordinate system in the point cloud coordinate system, Represents the attitude of the GNSS / IMU coordinate system in the navigation coordinate system, Represents the attitude of the thermal infrared camera coordinate system in the GNSS / IMU coordinate system.
[0056] The translation vector from the point cloud coordinate system to the thermal infrared camera coordinate system can be obtained from formula (2):
[0057]
[0058] in Represents the position of the thermal infrared camera coordinate system in the GNSS / IMU coordinate system, Represents the position of the navigation coordinate system in the point cloud coordinate system.
[0059] in It is obtained from the longitude and latitude of the GNSS / IMU center at the moment of imaging, and the heading, pitch, and roll angles provided by the GNSS / IMU system. Obtained by resection. Resection can calculate the camera's pose using the known coordinates of a point in three-dimensional space and its pixel coordinates in a two-dimensional image. Therefore, the pose of the thermal infrared camera coordinate system in the GNSS / IMU coordinate system can be calibrated using the above method.
[0060] 4. Automatically generate preliminary thermal infrared color point cloud.
[0061] The calibration model can realize the geometric relative calibration of GNSS / IMU and thermal infrared camera sensor. Since GNSS / IMU and thermal infrared camera are tightly connected, the pose matrix of thermal infrared camera coordinate system in GNSS / IMU coordinate system is It remains unchanged during the entire image acquisition process. The position and pose of any image in the point cloud coordinate system can be determined, thereby achieving the fusion of the remaining sequence of thermal infrared images with the visible light point cloud. Combining the conversion relationship between the pixel coordinate system and the thermal infrared camera coordinate system, it is possible to achieve heterogeneous data fusion of the two-dimensional temperature information of the thermal infrared image and the three-dimensional spatial information of the visible light point cloud. Figure 4 It is the result of partial fusion.
[0062] Assume that the coordinates of a ground point in the thermal infrared camera coordinate system are (x Img ,y Img , z Img ), the coordinates in the pixel coordinate system are (u, v), then:
[0063]
[0064] where f x and f y is the focal length of the camera in the x and y directions, and u0 and v0 are the pixel coordinates of the image principal point.
[0065] 5. Obtain relative orientation parameters.
[0066] ① Construct a virtual camera based on the calibration parameters and the current POS data. The resolution of this virtual camera is consistent with that of the thermal infrared camera. Project the point cloud onto the 2D plane of this virtual camera to obtain the projected image.
[0067] ② Based on the phase congruency method, feature points are extracted from the projected image and the thermal infrared image. Two-dimensional Log-Gabor wavelets are used in feature point detection. They can provide useful edge feature information descriptions for multiple directions and multiple scales from multimodal image pairs. Generally speaking, the 2D Log-Gabor filter is expressed as follows:
[0068]
[0069] Where o and s represent the direction and scale of the Log-Gabor filter, respectively. β determines the bandwidth of the filter. f and F s Define the frequency and center frequency of the filter respectively. θ is the angular bandwidth, θ o Indicates the direction of the filter.
[0070] Since the two-dimensional Log-Gabor is a frequency domain filter, its spatial domain expression can be obtained by inverse Fourier transform based on the corresponding frequency response of the polar coordinate Log-Gabor filter. Therefore, the 2D Log-Gabor function in the spatial domain can usually be decomposed into an even symmetric filter and an odd symmetric filter, which are defined as follows:
[0071] LG(x, y, s, o) = LG even (x, y, s, o)+i·LG odd (x, y, s, o) (7)
[0072] Among them, the real part LG even (x, y, s, o) and the imaginary part LG odd (x, y, s, o) represent the even-symmetric and odd-symmetric filters of the Log-Gabor wavelet at scale s and direction o, respectively.
[0073] Therefore, convolving the image I(x,y) with two even-symmetric and odd-symmetric filters can obtain the spatial response components E(x,y,s,o) and O(x,y,s,o) of the Log-Gabor filter.
[0074]
[0075] Then the amplitude component A of I(x, y) at scale s and orientation o is so (x, y) and phase component φ so (x, y) can be obtained by the following formula:
[0076]
[0077] φ so (x, y) = arctan( so (x, y) / E so (x,y)) (10)
[0078] Taking into account the analysis results in all directions and at all scales, and introducing noise compensation T, the final 2D phase consistency model is:
[0079]
[0080] Figure 5 is the result of feature point detection;
[0081] ③ The feature matching process uses a dense feature network and explicitly constrains the relevant volume using external high-confidence feature points, significantly reducing the probability of mismatching caused by radiation differences and texture loss. The steps include:
[0082] 1) The extracted feature points are input into the dense feature matching network, and the feature point coordinates are scaled by 1 / s according to the downsampling ratio s (=8) of the feature encoder;
[0083] 2) Generate a binary mask M on the feature map of size H / s×W / s. Assign 1 to the positions corresponding to the feature points in the mask and 0 to the remaining positions. Then perform a 3×3 morphological dilation on M to expand the effective matching neighborhood.
[0084] 3) Keep the dual-branch convolutional neural network feature extractor structure unchanged and output feature pyramid {F l}; Construct a 4-D correlation volume C and perform filtering by mask:
[0085]
[0086] 4) Performing heterogeneous image registration on the projection image and the thermal infrared image based on the retained correlation volume to obtain relative orientation parameters between the two. Figure 6 It is the result of heterogeneous image registration.
[0087] 6. Refined processing of thermal infrared color point cloud
[0088] Obtaining relative orientation results through heterogeneous image registration Further correct the relative spatial relationship between the point cloud coordinate system and the thermal infrared camera This improves the spatial accuracy of the preliminary thermal infrared color point cloud and enables refined processing. The formula for obtaining the refined point cloud is as follows:
[0089]
[0090] Figure 7 The thermal infrared color point cloud before and after fine-tuning. It can be seen that the spatial accuracy of the thermal infrared color point cloud has been greatly improved after fine-tuning.
Claims
1. A method for generating fine thermal infrared color point clouds of heterogeneous UAVs assisted by POS priors, comprising the following steps: (1) Using a visible light-UAV system to acquire visible light images and corresponding Position & Orientation System (POS) data, perform feature processing to generate a visible light dense point cloud; (2) For the thermal infrared image collected by the thermal infrared-UAV system, the target thermal infrared image is aligned with the dense point cloud by selecting the same-name points, and the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system are calculated; (3) Based on the exterior orientation elements obtained in step (2), combined with the POS data recorded by the thermal infrared-UAV system, complete the geometric calibration of the thermal infrared camera sensor and the Global Navigation Satellite System / Inertial Measurement Unit (GNSS / IMU) system; (4) Based on the geometric calibration model established in step (3), the external parameters of the thermal infrared camera in the point cloud coordinate system are calculated through the POS data, the spatial mapping relationship between the thermal infrared image and the visible light point cloud is constructed, and the preliminary thermal infrared color point cloud is automatically generated; (5) The initially generated thermal infrared point cloud is projected onto a two-dimensional image plane according to the exterior orientation parameters to generate a projection image; the parallax between the projection image and its corresponding thermal infrared image is eliminated by aligning the perspective of the projection image with the corresponding thermal infrared image; and the two images are subjected to heterogeneous feature matching based on a deep learning network based on phase consistency and dense feature matching to obtain the relative orientation parameters between the two images; (6) Based on the relative orientation parameters of step (5), the thermal infrared point cloud generated in step (4) is refined to finally obtain a refined thermal infrared color point cloud.
2. The method according to claim 1, wherein The steps for calculating the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system include: 1) manually selecting the same-name points between a thermal infrared image and the dense point cloud, and establishing a correspondence between the two-dimensional same-name points and the three-dimensional same-name points; 2) Based on the corresponding relationship, the exterior orientation elements of the thermal infrared camera in the point cloud coordinate system are obtained through the rear intersection algorithm.
3. The method according to claim 1, wherein Geometric calibration of thermal infrared camera and GNSS / IMU: Assuming that the phase center of the GNSS antenna coincides with the center of the IMU, and considering the GNSS and IMU as the same sensor, the relative exterior orientation elements between the thermal infrared camera and GNSS / IMU are determined based on the exterior orientation elements obtained in step (2), combined with the rigid connection relationship between the thermal infrared camera and GNSS / IMU and the POS information of the collected image.
4. The method according to claim 1, wherein Based on the geometric calibration model established in step (3), a preliminary thermal infrared color point cloud is automatically generated, which includes the following steps: 1) From the point cloud coordinate system to the navigation coordinate system, the conversion matrix is calculated based on the longitude and latitude of the GNSS / IMU center at the moment of imaging; 2) The navigation coordinate system is converted to the GNSS / IMU coordinate system, and the conversion matrix is calculated based on the attitude angles recorded by the IMU; 3) The conversion matrix from the GNSS / IMU coordinate system to the thermal infrared camera coordinate system is calculated based on the geometric calibration relationship between the two; 4) constructing a spatial mapping relationship between the thermal infrared image and the visible light point cloud based on the coordinate system conversion parameters obtained from steps 1) to 3); 5) According to the mapping relationship, the remaining thermal infrared images are automatically mapped to the visible light point cloud to generate a preliminary thermal infrared color point cloud.
5. The method according to claim 1, wherein Generate a projection image, extract feature points from the projection image and the thermal infrared image based on phase consistency, and further perform heterogeneous image registration on the two images based on a dense feature matching network, the steps of which include: 1) Based on the calibration parameters and the POS data at the current imaging moment, a virtual camera model is constructed, where the resolution of the virtual camera is consistent with that of the thermal infrared camera; the dense point cloud is projected onto the two-dimensional image plane of the virtual camera to generate a projected image; 2) extracting feature points from the projection image and the thermal infrared image based on a phase congruency method; 3) Inputting the extracted feature points into a dense feature matching network, performing heterogeneous image registration on the projection image and the thermal infrared image, and obtaining relative orientation parameters between the two.
6. The method according to claim 1, wherein The relative spatial relationship between the point cloud coordinate system and the thermal infrared camera coordinate system is further corrected to improve the spatial accuracy of the preliminary thermal infrared color point cloud and achieve refined processing.