A binocular vision point cloud temperature field fast reconstruction method based on homographic transformation

Abstract

The disclosure provides a binocular vision point cloud temperature field fast reconstruction method based on homography transformation. During calibration, the binocular camera and the infrared thermal imager synchronously shoot the mark points; according to the corresponding points in the visible light image and the thermal image of the mark points obtained by the left camera, the homography matrix between the left camera and the infrared thermal imager is solved; based on the homography matrix, the external parameter matrix between the left camera and the infrared thermal imager is solved. During point cloud temperature field reconstruction, the binocular camera obtains the surface speckle image of the measured object and solves the three-dimensional point cloud of the surface of the measured object; the homography matrix is updated by using the plane features in the three-dimensional point cloud; and according to the internal parameter matrix and the updated homography matrix, the temperature information in the thermal image is mapped to the point cloud corresponding to the left camera visible light coordinate. The method only needs single calibration measurement, is simple to operate, has high precision, and realizes the measurement of the three-dimensional appearance and the temperature of each point of the material with unobvious surface features.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer image, and particularly relates to a binocular vision point cloud temperature field fast reconstruction method based on homographic transformation. BACKGROUND

[0002] In the current industry and scientific research field, accurate measurement and analysis of the three-dimensional morphology and temperature field of high-temperature objects have become a key requirement. In industrial detection, material science and physical research under high-temperature environment, it is crucial to accurately measure and analyze the temperature distribution of the object surface. Traditional temperature measurement techniques, such as thermocouples, infrared thermometers, etc., although can meet the demand of temperature detection to a certain extent, have certain limitations in the face of complex surfaces, non-contact measurement and high-resolution temperature field acquisition, etc. Especially in high-temperature environment, due to the enhancement of thermal radiation and the influence of complex surface geometry, traditional temperature measurement methods often have difficulty in providing sufficient accuracy and spatial resolution.

[0003] In recent years, with the development of computer vision technology, the measurement method based on binocular vision has gradually become one of the important means of high-precision, non-contact temperature field measurement. Binocular vision system uses two cameras to capture images of the same scene from different angles, and reconstructs the three-dimensional morphology of the object through the principle of triangulation. Combined with image processing technology, this method can accurately obtain the three-dimensional point cloud data of the object surface under high-temperature environment, providing geometric basis for further temperature field analysis.

[0004] However, in practical application, only relying on the reconstruction and fusion of geometric morphology cannot fully reflect the temperature distribution of the object surface, especially in high-temperature occasions, small changes in the surface temperature of the object may cause large thermal effects. Therefore, how to effectively combine the temperature field with the three-dimensional point cloud data has become a key problem in current research. Traditional temperature field reconstruction methods, such as thermal field analysis based on infrared images, often rely too much on the geometric features of the object surface, resulting in low accuracy of temperature field reconstruction, and difficult to meet the needs of planar block and smooth surface measurement.

[0005] Three-dimensional digital image correlation method (3D-DIC) based on binocular vision can analyze the speckle pattern on the object surface and reconstruct the point cloud, but cannot obtain temperature information.

[0006] The patent application with the publication number CN110379002A "Three-dimensional reconstruction surface temperature display method based on infrared and visible light image fusion" first calculates the scale factor of the spatial object between the infrared and visible light cameras, then uses the natural color of YUV space color transfer for fusion to obtain the target picture, and finally replaces the original visible light picture in the process of reconstructing the dense point cloud to the reconstructed surface to realize the three-dimensional reconstruction of the surface with temperature information. The scale factor and relative offset extracted by the fusion method are only applicable to the calibration plane, and when directly applied to the measurement plane, the factors are not corrected for the plane pose. When fusing the infrared camera image and the visible light image, pixel errors will be introduced due to the difference in offset and resolution between the two. Using the fused image to reconstruct the dense point cloud will bring this error into the point cloud coordinates, affecting the coordinate accuracy.

[0007] The patent application with the publication number CN110379002A "Strain field and temperature field coupling measurement method and system fusing infrared information" uses the following means to fuse the strain field and the temperature field: The bilinear interpolation method is used to process the infrared image, and the resolution difference between the three-dimensional strain image and the infrared temperature field image of the speckle sample at different times is compensated, so that the sizes of the two images are the same. The temperature and strain at any position of the material can be one-to-one corresponding, so as to obtain the corresponding three-dimensional strain field and temperature field at each time, and realize the coupling of the strain field and the temperature field. The measurement system only indicates that a circular dot calibration plate containing three concentric circles is used to calibrate three cameras, but does not indicate the steps and algorithm of the calibration, which is the key to accurate coupling of the strain field and the temperature field. In addition, the fusion of the strain field and the temperature field occurs in the final stage, and the original measurement image is not fully utilized. At the same time, the system needs to be calibrated before each measurement, and the process is complicated.

[0008] In order to solve these challenges, it is of great significance to fuse point cloud and temperature field while simplifying the calibration and calculation process. SUMMARY

[0009] Therefore, the present application provides a speckle point cloud temperature field fast reconstruction method based on homography transformation. Wherein an infrared thermal imager and a visible light binocular vision camera form a system, measure the speckle image of the object surface at the same time, calculate the mapping relationship between the visible light and the thermal image through the pre-calibration, realize the fusion of the point cloud and the temperature. The method only needs single calibration and measurement, is simple to operate, has high precision, and realizes the measurement of the three-dimensional appearance and the temperature of each point of the material with unobvious surface features.

[0010] In order to solve the above technical problems, the present application is implemented as follows.

[0011] A homographic transformation-based binocular vision point cloud temperature field rapid reconstruction method, the measurement system based on which comprises a set of binocular cameras and an infrared thermal imager close to the left camera in the binocular cameras; the measurement system acquires a visible light image of a photographed object through the binocular cameras and solves a three-dimensional point cloud; the temperature distribution information of the object surface is captured through the infrared thermal imager; the method comprises:

[0012] Step 1: calibration of the left camera and the infrared thermal imager: the binocular cameras and the infrared thermal imager synchronously photograph a mark point; the homographic matrix H between the left camera and the infrared thermal imager is solved according to the corresponding points in the mark point visible light image obtained by the left camera and the thermal image; a plane feature is acquired according to the three-dimensional point cloud solved by the binocular camera image; the extrinsic matrix between the left camera and the infrared thermal imager is solved based on the homographic matrix H and the plane feature, in combination with the intrinsic matrix of the left camera and the infrared thermal imager;

[0013] Step 2: point cloud temperature field reconstruction: the binocular cameras acquire a surface speckle image of a measurement object and solve a three-dimensional point cloud of the measurement object surface; the homographic matrix H is updated using the plane feature in the three-dimensional point cloud; the temperature information in the thermal image is mapped onto the point cloud corresponding to the left camera visible light coordinates according to the intrinsic matrix and the updated homographic matrix H, and the fusion of the temperature field and the three-dimensional point cloud is completed.

[0014] Preferably, in step 1, the homographic matrix H between the left camera and the infrared thermal imager is solved according to the corresponding points in the mark point visible light image obtained by the left camera and the thermal image by: fitting a plane using the least square method to obtain the homographic matrix H.

[0015] Preferably, in step 1, the extrinsic matrix between the left camera and the infrared thermal imager is solved based on the homographic matrix H and the plane feature, in combination with the intrinsic matrix of the left camera and the infrared thermal imager:

[0016] The coordinates of a point in space P = [X, Y, Z] T , the pixel coordinates in the left camera image are [u1, v1], and the pixel coordinates in the thermal image are [u0, v0]; written as homogeneous coordinates p1 = [u1, v1, 1] T , p0 = [u0, v0, 1] T ;

[0017] According to binocular stereo vision, the pixel coordinates and the world coordinates have the following relationship:

[0018] p1 = K1P, p0 = K0(RP + t)

[0019] wherein K1 and K0 are the intrinsic matrices of the left camera and the infrared thermal imager respectively; R and t are the rotation matrix and the translation vector in the extrinsic matrix between the left camera and the infrared thermal imager respectively.

[0020] In the world coordinate system, the target plane equation is expressed as:

[0021] n T P-d=0

[0022] The homographic transformation from point p1 to p0 is described as p0=Hp1, combined with the expressions of p0 and p1 and the target plane equation, the expression of the homographic transformation matrix H is as follows:

[0023]

[0024] The normal vector n=[n x ,n y ,n z ] T and the distance d from the plane to the coordinate origin are obtained by using the three-dimensional point cloud obtained by the binocular camera, and are brought into the expression (1) of H to obtain the transformation matrix R and t from the infrared thermal imager to the left camera.

[0025] Preferably, in step 2, the temperature information in the thermal image is mapped to the point cloud corresponding to the visible light coordinates according to the rotation matrix R, the translation vector t and the updated homographic matrix H, which is:

[0026] The mapping relationship between the thermal image coordinates p0 and the left camera visible light coordinates p1 is:

[0027]

[0028] According to the above formula, the temperature information in the thermal image is first mapped to the visible light coordinates, and then based on the correspondence between the left camera coordinates and the three-dimensional point cloud coordinates, the temperature information is mapped to the three-dimensional point cloud.

[0029] Preferably, in step 2, when mapping, the temperature information in the thermal image is calculated by interpolation to accurately map the temperature information in the thermal image to the point cloud corresponding to the left camera visible light coordinates.

[0030] Preferably, the interpolation calculation adopts a bicubic difference algorithm.

[0031] Preferably, in step 2, before mapping the temperature information of the thermal image, the thermal image is further corrected for distortion, adjusted for resolution and / or filtered for noise.

[0032] Preferably, in step 2, the binocular camera obtains a surface speckle image of the measured object, and the three-dimensional point cloud of the surface of the measured object is calculated by: analyzing the displacement change of the speckle by a three-dimensional digital image correlation (3D-DIC) method to calculate the three-dimensional point cloud of the surface of the measured object.

[0033] Preferably, the optical axes of the left camera and the infrared thermal imager are parallel.

[0034] Advantages:

[0035] (1) The application effectively combines the temperature field information captured by the infrared thermal imager and the high-precision three-dimensional point cloud data obtained by the binocular vision system through homography transformation. Compared with the prior art, the temperature and the topography can be combined with high precision after single calibration, the measurement error caused by the resolution difference between different devices is reduced, and the precision of the temperature field reconstruction is improved.

[0036] (2) When calibrating the external parameter matrix, the application only needs to measure the camera after it is fixed, obtain the homography matrix by using a single image, and combine the internal parameter matrix to obtain the calibration result of the external parameter matrix, without using multiple images to calculate the external parameter matrix. The operation steps are simplified, the user burden is reduced, and the practicability of the system is improved under the premise of ensuring the precision.

[0037] (3) The application is suitable for object surface temperature field measurement in different temperature environments. It can adapt to the measurement of object surfaces without obvious features, without the need for feature matching of visible light images and thermal images. At the same time, the temperature information of the thermal image is accurately mapped to the point cloud data by using the bicubic interpolation algorithm, further reducing the error caused by the resolution mismatch, ensuring the fusion of the temperature field and the three-dimensional topography of any scale, and providing more accurate temperature field reconstruction results. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of the object surface three-dimensional point cloud and temperature field reconstruction method constructed by the application;

[0039] Figure 2 is a specific embodiment flowchart of the calibration of the object surface three-dimensional point cloud and temperature field reconstruction measurement system constructed by the application;

[0040] Figure 3 is a schematic diagram of the object surface three-dimensional point cloud and temperature field reconstruction measurement system constructed by the application;

[0041] Figure 4 is a specific example diagram of camera measurement and collection, figure (a) is an image taken by a visible light camera, figure (b) is a three-dimensional point cloud and temperature color scale diagram with temperature information of the reconstructed surface;

[0042] Wherein: 1- controller, 2- left camera in binocular camera, 3- right camera in binocular camera, 4- infrared thermal imager, 5- tripod, 6- test furnace. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. The present application provides a binocular vision speckle point cloud temperature field reconstruction method based on homography transformation. By combining the measurement data of a binocular camera and an infrared thermal imager, the efficient fusion of point cloud and temperature field is realized based on homography transformation, which is suitable for the synchronous measurement and mapping of object surface temperature and three-dimensional topography.

[0044] Figure 3 The measurement system on which the reconstruction method of the present application relies is shown. As shown in Figure 3 , the measurement system comprises a binocular camera, an infrared thermal imager 4 and a controller 1. The binocular camera comprises a left camera 2 and a right camera 3. The infrared thermal imager 4 is adjacent to the left camera 2. In this embodiment, in order to facilitate installation, the binocular camera and the infrared thermal imager 4 are both installed on the same tripod 5. The controller 1 is connected to the binocular camera and the infrared thermal imager 4 to acquire and process collected images. A measurement object is arranged in a test furnace 6.

[0045] When the measurement system collects images, the binocular camera and the infrared thermal imager are used to synchronously photograph the target object. The binocular camera is responsible for acquiring the three-dimensional topography information of the object, while the infrared thermal imager captures the temperature distribution on the surface of the object.

[0046] In order to ensure the accuracy and synchronicity of the data, attention should be paid to the configuration and installation of the camera group and the synchronous control of image collection. Specifically, the left camera, the right camera and the infrared thermal imager are fixed on the same measurement platform, the distance between the left camera and the infrared thermal imager is close, and the parallax between them is reduced. The positional relationship between the binocular camera and the infrared thermal imager is fixed, thereby ensuring the geometric correspondence relationship between them. However, the spatial positional relationship between them is not limited, and it is appropriate that the optical axes of the left camera and the thermal imager are parallel. In the case of installation limitation, they can be approximately parallel. The specific position can be realized by a precise support and an adjusting mechanism to improve the consistency of image collection. The schematic diagram of the object surface three-dimensional point cloud and temperature field reconstruction measurement system is shown in Figure 3 .

[0047] For the synchronous control of image collection, a hardware synchronization trigger or control software is used to trigger the binocular camera and the infrared thermal imager at the same time, so as to ensure that they collect images at the same time point.

[0048] In this embodiment, as shown in Figure 1 , the object surface three-dimensional point cloud and temperature field reconstruction method based on homography transformation of the present application comprises the following steps:

[0049] Step S1: system calibration

[0050] The calibration process of the object surface three-dimensional point cloud and temperature field reconstruction scheme based on homography transformation of the present application is shown inFigure 2 System calibration is a key step to achieve accurate mapping between 3D point cloud and temperature field. By jointly calibrating binocular camera and infrared thermal imager, homography transformation matrix between them can be obtained.

[0051] The calibration process specifically includes the following steps:

[0052] Step S1.0: arrangement of marker points: a standard plane is used as a calibration board, and high-reflectivity marker points such as reflective patches or infrared LED arrays are arranged on the surface of the standard plane. These marker points will be used for image registration and calibration in subsequent steps. The selected marker points should be clearly identifiable in both visible light and infrared images under natural light or additional infrared light source. The arrangement of reflective marker points should be uniformly distributed on the surface of the standard plane, and the number should be sufficient to meet the needs of subsequent homography matrix calculation. In this embodiment, the number of marker points should be no less than 4, and the spatial positions of the marker points should be disordered, asymmetric and non-collinear. The positions of the marker points in the image will be used to calculate the extrinsic matrix of the camera.

[0053] Step S1.1: calibration of binocular camera and thermal imager: here, the intrinsic matrix is calibrated. The intrinsic matrix K1 of the binocular camera is calibrated by the classic Zhang Zhengyou calibration method or other calibration techniques based on checkerboard images, while the intrinsic matrix K0 of the thermal imager is accurately determined by a special thermal imaging calibration method.

[0054] Step S1.2: binocular camera and infrared thermal imager respectively capture the image of the calibration board: trigger the binocular camera and infrared thermal imager synchronously, the binocular camera captures the visible light image, and the registered solution obtains the 3D point cloud; the infrared thermal imager captures the thermal image; in order to ensure the accuracy of the calibration, it should be ensured that each camera and thermal imager can see at least four or more marker points.

[0055] Step S1.3: homography matrix solution:

[0056] The coordinates of a point in space P = [X, Y, Z] T , the pixel coordinates in the left camera image are [u1, v1], and the pixel coordinates in the thermal image are [u0, v0]. Written as homogeneous coordinates p1 = [u1, v1, 1] T , p0 = [u0, v0, 1] T .

[0057] The homographic transformation of point p1 to p0 can be described as follows.

[0058] p0 = Hp1

[0059] Where H is the homography matrix between the left camera and the infrared thermal imager;

[0060] In the actual calibration process, the corresponding relationship between the mark points is good. Therefore, the application uses a simple least square method to construct an equation group and linearly estimates the homography matrix H. The least square method fitting plane can obtain the H matrix, reduce the calibration error, and improve the accuracy of the homography transformation.

[0061] Will

[0062]

[0063] Since the points p0 and p1 adopt homogeneous coordinates, the homography matrix has only 8 degrees of freedom, and 1 pair of matching points can construct 2 constraints, so at least 4 pairs of matching points are required to solve the homography matrix.

[0064] Substitute the homography transformation expression, and the equation of 1 pair of matching points can be obtained:

[0065]

[0066] n pairs of the above matching point equations are combined and denoted as:

[0067] Ah=0

[0068] Where the size of A is 2n*9, h is the column vector corresponding to H, and the size is 9*1. Singular value decomposition is applied to obtain

[0069] A=U∑V T

[0070] Where U, V, and ∑ are two orthogonal matrices and a diagonal matrix obtained by singular value decomposition. h corresponds to the last column of V, that is, the right singular vector corresponding to the minimum singular value. Thus, H can be obtained.

[0071] Step S1.4: Plane feature extraction: according to the common points of the left and right cameras of binocular stereo vision, the three-dimensional coordinates of the calibration mark points can be obtained, and the normal vector n of the calibration plane is determined. [n x ,n y ,n z ] T and the distance d from the origin, that is, the plane feature.

[0072] In three-dimensional space, the plane can be represented as:

[0073] n x x+n y y+n z z-d=0

[0074] In the above formula, x, y, and z are the three-dimensional coordinates of the point cloud. In this step, the point cloud data of three mark points is substituted into the above formula, and the least square method is used to solve, realize plane fitting, and finally obtain the plane normal vector n and the distance d from the origin of the world coordinate system.

[0075] Step S1.5: Solve the extrinsic matrix R and t of the left camera and the thermal imager.

[0076] According to binocular stereo vision, the pixel coordinates and the world coordinates have the following relationship:

[0077] p1 = K1P, p0 = K0(RP + t)

[0078] Wherein, R and t are the extrinsic matrix of the thermal imager relative to the left camera, R is the rotation matrix, and t is the translation vector. K1 and K0 are the intrinsic matrix of the left camera and the thermal imager which have been calibrated. In the world coordinate system, the target plane equation can be expressed as:

[0079] n T P-d = 0

[0080] The expressions of p0 and p1 are brought in, and the expression of the homographic transformation matrix is as follows:

[0081]

[0082] Using the normal vector n = [n x , n y , n z ] T of the calibration plane and the distance d from the plane to the coordinate origin solved in step 1.4, the transformation matrix R and t of the thermal imager to the left camera can be obtained by bringing H expression.

[0083] Step S2: Point cloud reconstruction and temperature field mapping.

[0084] Step S2.1: Take pictures. This step uses a binocular camera to obtain the surface speckle visible light image of the measured object, and solves the three-dimensional point cloud topography of the measured object surface. In practice, speckle texture can be prepared on the surface of the measured object, and the binocular camera can take speckle images from different angles to generate a three-dimensional point cloud; or use the ADIC3D library of Matlab to analyze the displacement change of the speckle by the three-dimensional digital image correlation 3D-DIC method, solve the speckle visible light image, and generate a three-dimensional point cloud.

[0085] Step S2.2: Update the homographic matrix H, and map the temperature information in the thermal image to the point cloud corresponding to the left camera visible light coordinate.

[0086] The planes of the calibration process and the measurement process are different, resulting in different homographic matrix H. Therefore, the plane features need to be recalculated, and the homographic matrix H is updated.

[0087] The calculation formula of the planar feature is the same as that in step 1.4, except that the three-dimensional coordinates of the plurality of points in the three-dimensional point cloud information obtained in step S2.1 are used for fitting. Due to the roughness of the workpiece surface, the influence of calculation noise, and the like, the RANSAC algorithm can be further used to remove noise and improve the planar fitting accuracy. Then, the new planar feature is used to update the homography matrix H.

[0088] Since the left camera and the infrared thermal imager actually constitute another binocular vision system, the transformation matrix R and t of the thermal imager to the left camera have been obtained in the calibration. The pixels of the thermal imager are unified into the binocular vision system after completing the homographic transformation with the left camera pixels. Through point cloud-left camera pixel mapping and left camera-thermal imager pixel matching, the process of matching the point cloud with the thermal imager pixels is completed. By substituting the expression of H into the homographic transformation expression, the following expression can be obtained:

[0089]

[0090] Therefore, the temperature information in the thermal image can be mapped to the visible light coordinates. Since the left camera coordinates correspond to the three-dimensional point cloud coordinates, the mapping of the temperature information to the three-dimensional point cloud can be realized.

[0091] Since the resolutions of the thermal image and the point cloud do not match, direct mapping can cause a large error. The measured block area in the three-dimensional reconstruction of the temperature field of the point cloud is usually small, the surface is smooth and has few features, and the temperature change is relatively gentle. In order to reduce the computational complexity and improve the solving efficiency, the temperature information in the thermal image can be calculated by interpolation, and then accurately mapped to each point cloud point to complete the fusion of the temperature field and the three-dimensional point cloud. The bicubic interpolation algorithm can be used.

[0092] Further, in order to reduce the computational complexity, the thermal image can be preprocessed before mapping, such as resolution adjustment and noise filtering, to improve the efficiency of the bicubic interpolation calculation.

[0093] The temperature at p0 is calculated as follows:

[0094]

[0095] where (i,j) is the nearest thermal image pixel coordinate to (u0,v0), W is the kernel function, and here W = 1 / 16.

[0096]

[0097] The bicubic interpolation achieves a good balance between accuracy, smoothness, and computational efficiency. The temperature data mapping process of the present application ensures good real-time performance.

[0098] The thermal image collected by the infrared thermal imager is subjected to spatial interpolation processing by using a bicubic interpolation method, the resolution difference and decimal coordinates of the infrared thermal imager and the two visible light cameras are overcome, the point cloud points can complete accurate mapping of the temperature, and a three-dimensional visualized screenshot of the point cloud temperature field after interpolation mapping is as shown in Figure 4 The temperature at an arbitrary position of the object and the temperature can be one-to-one corresponding, and fast reconstruction of the point cloud temperature field is realized.

[0099] The above specific embodiments only describe the design principles of the present application, and the shapes and names of the components in the description can be different and are not limited. Therefore, the person skilled in the art of the present application can modify or equivalently replace the technical solutions described in the foregoing embodiments; and these modifications and replacements do not deviate from the purpose and technical solutions of the present application, and should all belong to the protection scope of the present application.

Claims

1. A homography transformation-based binocular vision point cloud temperature field fast reconstruction method, characterized in that, The method is based on a measurement system composed of a set of binocular cameras and an infrared thermal imager next to the left camera of the binocular cameras; the measurement system acquires a visible light image of a photographed object through the binocular cameras and calculates a three-dimensional point cloud; The temperature distribution information of the surface of the object is captured through the infrared thermal imager; the method comprises: Step 1: calibration of the left camera and the infrared thermal imager: the binocular cameras and the infrared thermal imager synchronously photograph a mark point; a homography matrix H between the left camera and the infrared thermal imager is solved according to corresponding points in a visible light image of the mark point obtained by the left camera and a thermal image; a three-dimensional point cloud is calculated according to a plane feature obtained from the binocular camera image; an extrinsic matrix between the left camera and the infrared thermal imager is calculated based on the homography matrix H and the plane feature, combined with intrinsic matrices of the left camera and the infrared thermal imager; the plane feature obtained from the three-dimensional point cloud calculated from the binocular camera image is that a normal vector n of a calibration plane and a distance d from the plane to a coordinate origin are calculated from the three-dimensional point cloud acquired by the binocular cameras; Step 2: point cloud temperature field reconstruction: a surface speckle image of a measured object is acquired by the binocular cameras, a three-dimensional point cloud of the surface of the measured object is calculated, and the homography matrix H is updated using a plane feature in the three-dimensional point cloud; temperature information in the thermal image is mapped onto a point cloud corresponding to a visible light coordinate according to the updated homography matrix H; a mapping relationship between a thermal image coordinate p0 and a left camera visible light coordinate p1 is as follows: Wherein, P is a coordinate of a certain point in space, K1 and K0 are intrinsic matrices of the left camera and the infrared thermal imager respectively, and the temperature information in the thermal image is mapped to the visible light coordinate according to the above formula; then, based on the corresponding relationship between the left camera coordinate and the three-dimensional point cloud coordinate, the mapping of the temperature information to the three-dimensional point cloud is realized.

2. The method of claim 1, wherein, In step 1, the homography matrix H between the left camera and the infrared thermal imager is solved according to corresponding points in a visible light image of the mark point obtained by the left camera and a thermal image by using a least square method to fit a plane and obtain the homography matrix H.

3. The method of claim 1, wherein, In step 1, the extrinsic matrix between the left camera and the infrared thermal imager is calculated based on the homography matrix H and the plane feature, combined with intrinsic matrices of the left camera and the infrared thermal imager, as follows: Point P in space has coordinates P = [X, Y, Z] T , pixel coordinates in the left camera image are [u1, v1], and in the thermal image are [u0, v0]; written in homogeneous coordinates as p1 = [u1, v1, 1] T , p0 = [u0, v0, 1] T ; According to binocular stereo vision, the pixel coordinate and the world coordinate have the following relationship: p1=K1P,p0=K0(RP+t) Wherein, K1 and K0 are intrinsic matrices of the left camera and the infrared thermal imager respectively; R and t are a rotation matrix and a translation vector in the extrinsic matrix between the left camera and the infrared thermal imager respectively; In the world coordinate system, the target plane equation is expressed as: n T P-d=0 The homographic transformation of point p1 to p0 is described as p0=Hp1, and the expression of the homographic transformation matrix H is obtained by combining the expressions of p0 and p1 and the target plane equation as follows: The three-dimensional point cloud obtained by using the binocular camera is used to solve the normal vector n = [n x ,n y ,n z ] T and the distance d of the plane to the coordinate origin, which are brought into the H expression (1) to solve R and t.

4. The homography transformation based binocular vision point cloud temperature field fast reconstruction method of claim 1, wherein, In step 2, when mapping, the temperature information in the thermal image is calculated by interpolation to accurately map the temperature information in the thermal image to the point cloud corresponding to the left camera visible light coordinate.

5. The homography transformation based binocular vision point cloud temperature field fast reconstruction method of claim 4, wherein, The interpolation calculation adopts a bicubic difference algorithm.

6. The homography transformation based binocular vision point cloud temperature field fast reconstruction method of claim 1, wherein, In step 2, before mapping the temperature information of the thermal image, the thermal image is further corrected for distortion, adjusted in resolution and / or filtered for noise.

7. The homography transformation based binocular vision point cloud temperature field fast reconstruction method of claim 1, wherein, In step 2, the binocular camera acquires the surface speckle image of the measurement object, and calculates the three-dimensional point cloud of the surface of the measurement object as follows: through a three-dimensional digital image correlation (3D-DIC) method, the displacement change of the speckle is analyzed, and the three-dimensional point cloud of the surface of the measurement object is calculated.

8. The homography transformation based binocular vision point cloud temperature field fast reconstruction method of claim 1, wherein, The optical axes of the left camera and the infrared thermal imager are parallel.

Images