Small-view-field precise three-dimensional reconstruction method

Abstract

According to the small-view-field precise three-dimensional reconstruction method provided by the invention, in a global calibration process, a calibration method based on a virtual plane is provided, the virtual plane is used for calibrating a spatial position relationship between a camera and a projector, a spatial line-line intersection algorithm is used for obtaining a three-dimensional coordinate under global calibration, and global parameters of an X coordinate and a Y coordinate are obtained; in the local calibration process, a coordinate correction method based on a PCA plane fitting ideal plane is provided, the mapping relation between the phase and the coordinate of each pixel level is constructed by using cross ratio invariance, the relation between the absolute phase of an image collected by a camera and the Z coordinate is obtained, and system local calibration parameters for the Z coordinate are obtained; in the three-dimensional reconstruction process, an image positioning method based on the difference reflectivity is provided, the reconstruction range is narrowed, and only the image in the reconstruction area is calculated in subsequent calculation; and an invalid phase value identification and removal method is provided, interference of invalid points is eliminated, and the accuracy of a reconstruction result is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of structured light three-dimensional reconstruction, and in particular to a small-field-of-view precision three-dimensional reconstruction method. Background Art

[0002] The camera-projector structured light system is an important branch of the 3D reconstruction system. The calibration of the structured light system is a key factor affecting the surface shape measurement of 3D objects. The traditional structured light system is a calibration model established when the optical model is known, and the optimization model belongs to global optimization; but in fact, the optical model of the traditional structured light system is only an approximation of the actual lens model, and the system has local measurement errors.

[0003] In the prior art, triangulated stereo calibration methods are widely used to achieve global calibration of structured light systems. Structured light system calibration can be regarded as conventional binocular calibration, with the projector regarded as an inverse camera. However, telecentric structured light systems have some limitations when performing three-dimensional reconstruction. The calibration model of the telecentric lens is inconsistent with the pinhole model, and the telecentric lens has a constant magnification and is insensitive to depth. During the calibration process, due to the sign ambiguity in the rotation matrix and the lack of Z-axis translation parameters, it is difficult to establish a mathematical model for nonlinear optimization calibration, that is, the calibration degradation problem. Therefore, it is very difficult to obtain the precise spatial relationship between the camera and the projector in a telecentric structured light system.

[0004] In order to solve the local measurement error and the requirements of orthogonal fringe projection, Andres G. Marruggo et al. first proposed a hybrid calibration method. In this method, a plane is placed at different heights, and a pixel-level mapping function is established by considering the three-dimensional coordinates and phase information to achieve three-dimensional reconstruction. However, the polynomial fitting used in this method makes the number of reconstructed plane groups proportional to the accuracy, that is, the more the number of reconstructed plane groups, the higher the accuracy. This results in the need to construct more plane groups and perform a lot of calculations in order to improve the calibration accuracy, which affects the overall efficiency of three-dimensional reconstruction.

[0005] When performing small-field precision three-dimensional measurement in the field of high-precision manufacturing, the existing three-dimensional reconstruction technology is difficult to meet the needs of precision measurement due to the diverse sizes and complex morphologies of the objects being measured. Summary of the invention

[0006] In order to solve the problem that the three-dimensional reconstruction technology in the prior art is difficult to meet the demand for precision measurement, the present invention provides a small-field-of-view precision three-dimensional reconstruction method, which can improve the accuracy of small-field-of-view three-dimensional reconstruction while ensuring the reconstruction speed.

[0007] The technical solution of the present invention is as follows: a small field of view precision three-dimensional reconstruction method, which comprises the following steps:

[0008] S1: Collect the calibration plate and its coding pattern to perform global calibration on the telecentric lens camera and projector;

[0009] S2: Obtain local calibration parameters of the system;

[0010] S3: Complete precise 3D reconstruction of small fields of view;

[0011] Features:

[0012] In step S1, during global calibration, a virtual plane is used to calibrate the spatial relationship between the projector and the camera to obtain system global calibration parameters for the X coordinate and the Y coordinate;

[0013] In step S2, when performing local calibration, the following steps are specifically included:

[0014] Reconstruct planes at different heights to obtain the phases at different heights and their corresponding original coordinates Z; use PCA plane fitting to correct the Z coordinates of the plane point cloud to obtain the ideal coordinates; use cross-ratio invariance to construct the mapping relationship between the phase and Z coordinates at each pixel level to obtain the system local calibration parameters for the Z coordinates;

[0015] In step S3, the three-dimensional reconstruction includes the following steps in detail:

[0016] Collect the unidirectional stripe coded image of the reconstructed object, locate the reconstruction area by the image segmentation method of differential reflectivity; calculate the absolute phase value of the reconstructed area, identify and remove the invalid phase;

[0017] Based on the system global calibration parameters and the system local calibration parameters, the coordinates of each pixel in the reconstruction area in the world coordinate system are mapped out through a fitting relationship to complete the precise three-dimensional reconstruction of a small field of view.

[0018] It is further characterized by:

[0019] In step S1, a virtual plane is used to calibrate the spatial relationship between the projector and the camera during global calibration, which includes the following steps in detail:

[0020] a1: construct a virtual plane;

[0021] The virtual plane refers to the plane where the mirror-parallel plane of the telecentric lens and the center of the projector are located;

[0022] a2: Assuming that the camera imaging plane coincides with the virtual plane, the spatial relationship from the projector to the camera is the spatial relationship from the projector to the virtual plane; the spatial relationship from the projector to the camera and Respectively expressed as:

[0023]

[0024] Among them, b i is the pose of the ith calibration plate, are the rotation matrix and translation vector from the i-th calibration plate pose to the camera coordinate system. Similarly, is the rotation matrix and translation vector of the i-th calibration plate pose to the projector coordinate system;

[0025] a3: Select the pose with the smallest error among all poses to calculate the rotation matrix and translation vector from the projector to the camera It is expressed as:

[0026]

[0027] in, is the final matrix from projector to camera, Min(L1,L2,...,L N ) represents the global error set (L1,L2,...,L N )

[0028] Global optimization function L i It is expressed as:

[0029]

[0030] Among them, N and M represent the number of calibration plate poses and the number of feature points on the calibration plate respectively;

[0031] P ij and P[x j ,y j ,z j ] T represents the coordinates of the jth point under the i-th calibration plate posture; c and p refer to the camera and projector viewing angles respectively;

[0032] and They are the points P[x j ,y j ,z j ] T The coordinates projected onto the camera and projector images are calculated as follows:

[0033]

[0034] in, and To detect the point P[x j ,y j ,z j ] T Coordinates on the camera and projector images respectively;

[0035] a4: Establishing an X and Y lookup table, wherein the lookup table stores the x coordinate and the y coordinate corresponding to the pixel in the image captured by the camera in the world coordinate system;

[0036] The lookup table is based on the formula generate;

[0037] Among them, X w , Y w is the world coordinate; and is the effective magnification of the telecentric lens in the X and Y directions; u c and v c It is the feature point recognized in the camera field of view;

[0038] In step S2, planes at different heights are reconstructed to obtain phases at different heights and their corresponding original coordinates Z, which specifically includes the following contents:

[0039] b1: Establish a world coordinate system in the virtual plane with the optical center of the projector as the origin;

[0040] q1=[t x t y 0] T and q2 = [0 0 0] T are the coordinates of the optical center of the camera and projector respectively;

[0041] b2: Connect a ray from the optical center of the camera and projector to the target point respectively. The direction of the ray is determined by the unit vector and express;

[0042] The camera light is perpendicular to the virtual plane, so The direction of the vector is from the projector optical center O P Point to the target point, assuming that the pixel position of the target point in the camera is (u c ,v c ,0), then It is expressed as:

[0043]

[0044] in, and is the effective magnification of the telecentric lens in the X and Y directions, is the distortion center of the camera, R and (t x ,t y ,0) represents the rotation matrix and translation vector in the camera extrinsic parameters;

[0045] b3: The coordinates of any point on the light emitted by the camera are expressed as λ1 is an arbitrary number. Similarly,

[0046] The minimum distance between P1 and P2 is:

[0047]

[0048] The solution is,

[0049]

[0050] Where v1′ and v2′ are the transposes of v1 and v2;

[0051] because Therefore, λ2 is the Z coordinate of the corresponding pixel in the world coordinate system;

[0052] In step S2, PCA plane fitting is used to correct the Z coordinate of the plane point cloud to obtain the ideal coordinate; specifically, the following steps are included:

[0053] c1: Perform 3D reconstruction on the plane at each height and obtain the point cloud (X, Y, Z);

[0054] c2: Project each point cloud (X, Y, Z) onto the ideal plane and perform PCA plane fitting to obtain the ideal point cloud

[0055]

[0056] in, is an ideal three-dimensional point of a pixel (i, j), t is the vertical distance between the plane and each point, n represents the normal vector of the plane, ‖n‖ represents the length of the vector; (a, b, c, d) are the fitting plane coefficients;

[0057] In step S2, the mapping relationship between the absolute phase and the Z coordinate of each pixel level is constructed by using the cross ratio invariance to obtain the local calibration parameters of the system, which specifically includes the following steps:

[0058] d1: Obtain the actual height and absolute phase values ​​of at least three sets of planes at different heights;

[0059] d2: Using the cross-ratio invariance, solve the corresponding relationship between the absolute phase value of each pixel in the image and the actual height:

[0060]

[0061] In the formula, c 0-2 is the pixel-level calibration coefficient of the planes at different heights under the projector view, φ is the absolute phase obtained under the projector view, is the Z-direction value of the ideal point cloud at different heights, (i, j) is the pixel position on the camera image;

[0062] In step d2, the method for confirming the direction of the optimal absolute phase φ specifically includes the following steps:

[0063] e1: The projector projects orthogonal fringes, thereby obtaining the absolute phase in both horizontal and vertical directions;

[0064] e2: Ensure that there is no object to be measured in the field of view and obtain the horizontal and vertical absolute phase of the plane;

[0065] e3: Place a gauge block on the plane and obtain the horizontal and vertical absolute phases; subtract the absolute phases obtained twice to obtain the phase of the gauge block, and calculate the average phase value of the horizontal and vertical absolute phases of the gauge block respectively;

[0066] e4: Compare the average phase values ​​of the two directions, select the stripe direction with the higher average phase value as the best stripe direction; use the absolute phase obtained under the best stripe direction as the fitting phase, that is, the absolute phase φ under the projector view is obtained;

[0067] In step S3, a unidirectional fringe coded image of the reconstructed object is collected, and the reconstruction area is located by an image segmentation method of difference reflectivity; specifically, the following steps are included:

[0068] f1: Use a high-contrast white plane as the background to identify and analyze the features of the object to be tested;

[0069] f2: Set the invalid point judgment threshold;

[0070] The invalid point judgment threshold means that a pixel gray value greater than this value is judged to be oversaturated;

[0071] f3: collects the unidirectional fringe coded image of the reconstructed object;

[0072] f4: judge all the pixels of the grayscale image obtained by each phase shift;

[0073] If the gray value of a pixel is less than the invalid point judgment threshold, the pixel is marked as a valid pixel;

[0074] f5: Count the total number of all phase shift patterns and count the number of times each pixel is marked as a valid pixel, based on which a saturation map and saturation histogram are constructed;

[0075] f6: Perform threshold segmentation and morphological processing according to the saturation histogram to complete the target positioning and obtain the final reconstructed area;

[0076] In step S3, the absolute phase value of the reconstruction area is calculated, and the invalid phase therein is identified and removed, which specifically includes the following steps:

[0077] g1: Calculate the absolute phase value of the unidirectional fringe coded image in the reconstruction area;

[0078] The projected fringes are represented as:

[0079] I n (x,y)=A(x,y)+B(x,y)cos[φ(x,y)+2πn / N],n=1,2,...,N;

[0080] Where A represents the average intensity, B represents the modulation degree of the deformed fringes, N is the number of phase shift steps; φ is the wrapping phase, expressed based on the absolute phase value; (x, y) is the pixel coordinate;

[0081] g2: Calculate the modulation contrast γ corresponding to each point (x, y) in the corresponding view of the reconstructed area;

[0082] The modulation contrast γ is expressed as:

[0083] γ(x,y)=A f (x,y) / B f (x,y);

[0084] The closer γ is to 1, the better the signal-to-noise ratio of the point is, and the closer γ is to 0, the worse the signal-to-noise ratio of the point is.

[0085] B f (x,y) is the modulation degree of fringe images with different frequencies;

[0086]

[0087] Where N is the number of phase shift steps, I n represents the stripe grayscale image of the nth frame, M f represents the modulation image with frequency f;

[0088] A f (x,y) is the average intensity at different frequencies;

[0089]

[0090] g3: According to the preset modulation contrast threshold, all points whose modulation contrast γ is less than the modulation contrast threshold are marked as abnormal point clouds;

[0091] g4: Use the unwrapped phase as sample data, and set the confidence level to 99.99%; approximate the unwrapped phase φ to a plane; the plane is expressed as:

[0092] φ(x,y)≈ax+by+d;

[0093] Where a, b, and d are the plane coefficients, and (x, y) are the corresponding coordinates of the unwrapped phase;

[0094] g5: Solve the plane coefficients. After the solution is completed, the fitting residual v(x,y) of each pixel is obtained, that is, the difference between the actual absolute phase value and the phase value obtained by the fitting coefficient;

[0095] The difference in phase value is expressed as:

[0096] v(x,y)=φ(x,y)-(ax+by+d)|;

[0097] Where φ(x,y) is the actual absolute phase value, (ax+by+d) is the fitted phase value, and v is the residual;

[0098] g6: setting a residual identification threshold, and identifying phase unwrapping error points in the view corresponding to the reconstructed area based on the residual identification threshold;

[0099] If a residual error exceeds the residual identification threshold, it is identified as an erroneous unwrapping and marked to obtain a phase unwrapping error point;

[0100] g7: removing all the abnormal point clouds and the phase unwrapping error points in the view corresponding to the reconstructed area;

[0101] The residual identification threshold is set to 4σ, where σ is the standard deviation of the residual v;

[0102] Then, the residual recognition threshold is expressed as:

[0103]

[0104] Among them, mask is a two-dimensional template for identifying error points, and v(x,y) is the value of the coordinate (x,y) in the residual v;

[0105] In step S3, the coordinates of each pixel in the world coordinate system are mapped out through the fitting relationship to complete the precise three-dimensional reconstruction of the small field of view; specifically, the following steps are included:

[0106] h1: Based on the mapping relationship between the absolute phase and the Z coordinate of each pixel level, the Z coordinate of each pixel in the world coordinate system is calculated. The mapping formula is:

[0107]

[0108] Among them, c 0-2 is the pixel-level calibration coefficient under the projector view, φ is the absolute phase obtained under the projector view, and Z w is the calculated depth value, (i, j) is the pixel position on the camera image;

[0109] h2: Based on the X and Y lookup tables, query the X and Y coordinates of each pixel in the world coordinate system to complete the 3D reconstruction.

[0110] The present application provides a small field of view precision three-dimensional reconstruction method. In view of the degradation problem of telecentric structured light model, a calibration method based on virtual plane is proposed in the global calibration process. The spatial position relationship between the camera and the projector is calibrated by using virtual plane, and the three-dimensional coordinates under global calibration are obtained by using the spatial line intersection algorithm, and the global parameters of the X coordinate and Y coordinate are obtained. At the same time, in view of the local reconstruction error caused by global calibration, a coordinate correction method based on PCA plane fitting ideal plane is proposed in the local calibration process, and the mapping relationship between the phase and coordinate of each pixel level is constructed by using the cross ratio invariance, so as to achieve fast reconstruction and coordinate correction, obtain the relationship between the absolute phase and Z coordinate of the camera-collected image, and obtain the system local calibration parameters for the Z coordinate. In view of the problem that it is difficult to determine the measurement area of ​​small objects, an image positioning method based on differential reflectivity is proposed in the three-dimensional reconstruction process to narrow the reconstruction range, and the subsequent calculation is only calculated for the image within the reconstruction area. In view of the problem that noise points affect the measurement accuracy in the three-dimensional reconstruction of small objects, an invalid phase value identification and removal method is proposed to eliminate the interference of invalid points and ensure the accuracy of the reconstruction results. The present method can effectively realize high-speed and high-precision three-dimensional reconstruction under small field of view conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 This is a flow chart of the small field of view precision three-dimensional reconstruction method in this application;

[0112] Figure 2 is the calibration plate diagram in the specific example;

[0113] Figure 3 It is a system structure diagram in a specific instance;

[0114] Figure 4 is a schematic diagram of multi-frequency heterodyne in a specific example;

[0115] Figure 5 It is a schematic diagram of the cross-ratio invariance principle in a specific example;

[0116] Figure 6 It is a schematic diagram of the process of the difference reflectance image segmentation method in a specific example. DETAILED DESCRIPTION

[0117] like Figure 1 As shown, the present application includes a small field of view precision three-dimensional reconstruction method, which includes the following steps.

[0118] S1: Collect the calibration plate and its coding pattern to perform global calibration on the telecentric lens camera and projector.

[0119] During global calibration, a virtual plane is used to calibrate the spatial relationship between the projector and the camera to obtain the system global calibration parameters for the X and Y coordinates.

[0120] Telecentric lens camera calibration. Different from the perspective projection principle of the pinhole lens, the telecentric lens follows the orthogonal projection principle. The mathematical model of the imaging of the distortion-free telecentric system can be expressed as:

[0121] p = AKP;

[0122] Among them, p is the camera coordinate system, P is the world coordinate system;

[0123] p = [uv 1] T , P = [x W y W z W 1] T ,

[0124] A is the camera internal parameter, K is the camera external parameter, A and K can be expressed as,

[0125]

[0126] in, and is the effective magnification of the telecentric lens in the X and Y directions, is the center of distortion of the camera. (r 11 ,…,r 23 ) and (t x ,t y ,t z ) represent the rotation matrix and translation vector in the extrinsic parameters respectively.

[0127] In a telecentric system, due to the design and manufacturing process of the lens, a certain degree of distortion is inevitable. In order to describe and quantify lens distortion, the following distortion formula is introduced:

[0128]

[0129] Among them, u d 、v d is the actual observed pixel coordinate, u and v are the ideal pixel coordinates, and r 2 =u 2 +v 2 , k1, q1, q2, p1, p2 are the distortion coefficients of the lens.

[0130] In addition, due to the imaging principle of the telecentric lens, it causes degradation to a certain extent, resulting in ambiguity in the sign of the rotation parameter. In order to solve this problem, the extrinsic parameter invariance of the telecentric lens is used to restore the correct sign. Specifically, the telecentric model can be regarded as a pinhole model with a very large focal length to a certain extent. From the process of converting the world coordinate system to the camera coordinate system, the two have the same geometric meaning. However, the telecentric model is not a pinhole model after all. Here, only the pinhole camera model is used to assist in solving the sign of the sum in each posture, thereby achieving the purpose of unambiguous signs.

[0131] Projector calibration. A set of gratings are projected onto the dot calibration plate, and the grating images are captured by cameras. The gratings are 24 horizontal phase-shift codes and 24 vertical phase-shift codes. The 24 horizontal phase-shift codes contain 8-step phase-shift stripes at 3 frequencies, and the 24 vertical phase-shift codes contain 8-step phase-shift stripes at 3 frequencies.

[0132] The absolute phase of the grating image is solved by the multi-frequency heterodyne principle, and the absolute phase φ H ,φ V and the identified feature points (u C ,v C ) to obtain the feature points (u p ,v p );

[0133] Feature point (u p ,v p ) and feature points (u C ,v C ) is:

[0134]

[0135] Among them, P H and P V is the period of the orthogonal fringes.

[0136] The internal and external parameters of the projector are obtained according to the absolute phase. The obtained features such as the rotation matrix and translation vector points of each calibration plate relative to the projector are calibrated using Zhang Zhengyou's calibration method to obtain the calibration parameters of the projector, lens distortion parameters, internal parameter matrix, etc.

[0137] Since the telecentric model lacks the translation parameter in the Z direction, which causes the degradation problem of stereo calibration, this method uses a virtual plane to construct the spatial relationship between the camera and the projector. The virtual plane refers to the plane where the mirror parallel plane of the telecentric lens and the center of the projector are located. The telecentric lens is insensitive to the Z direction. Assuming that the camera imaging plane coincides with the virtual plane, the spatial relationship from the projector to the camera is the spatial relationship from the projector to the virtual plane. The detailed steps include:

[0138] a1: construct a virtual plane;

[0139] The virtual plane is the plane where the mirror-parallel surface of the telecentric lens and the center of the projector are located.

[0140] a2: Assuming that the camera imaging plane coincides with the virtual plane, then: the spatial relationship from the projector to the camera is the spatial relationship from the projector to the virtual plane; the spatial relationship from the projector to the camera and Respectively expressed as:

[0141]

[0142] Among them, b i is the pose of the ith calibration plate, are the rotation matrix and translation vector from the i-th calibration plate pose to the camera coordinate system. Similarly, is the rotation matrix and translation vector of the i-th calibration plate pose to the projector coordinate system.

[0143] a3: In the actual calibration process, each calibration pose can calculate a set of rotation matrices and translation vectors from the projector to the camera. Therefore, in this method, in order to minimize the global optimization, the pose with the smallest error among all poses is selected to calculate the rotation matrix and translation vector from the projector to the camera. It is expressed as:

[0144]

[0145] in, is the final matrix from projector to camera, Min(L1,L2,...,L N ) represents the global error set (L1,L2,...,L N )

[0146] Global optimization function L i It is expressed as:

[0147]

[0148] Among them, N and M represent the number of calibration plate poses and the number of feature points on the calibration plate respectively;

[0149] P ij and P[x j ,y j ,z j ] T represents the coordinates of the jth point under the i-th calibration plate posture; c and p refer to the camera and projector viewing angles respectively; and They are the points P[x j ,y j ,z j ] T The coordinates projected onto the camera and projector images are calculated as follows:

[0150]

[0151] in, and They are the points P[x j ,y j ,z j ] T Coordinates projected onto the camera and projector images; and To detect the point P[x j ,y j ,z j ] T Coordinates on the camera and projector images respectively.

[0152] Finally, the global loss is optimized by the PnP (Perspective-n-Points) algorithm based on Levenberg-Marquardt.

[0153] a4: Create an X and Y lookup table, which stores the x-coordinate and y-coordinate of the pixels in the image captured by the camera in the world coordinate system.

[0154] The XY of a telecentric lens will not have the problem of large near and small far. The lookup table is based on the formula generate;

[0155] Among them, X w , Y w is the world coordinate; and is the effective magnification of the telecentric lens in the X and Y directions; u c and v c It is the feature point recognized in the camera field of view.

[0156] In this embodiment, a single grating telecentric structured light system is used as the structured light system to be calibrated, including a telecentric lens, a camera and a projector, and the relative positions of the camera and the projector remain fixed. The calibration plate is transformed into 10 to 30 postures in sequence. For each calibration plate posture, a complete structured light coding pattern is projected by the projector, and a structured light coding picture modulated by the surface shape of the calibration plate is taken by the camera.

[0157] The calibration plate pattern used in this embodiment is shown in Figure 2 , the system structure is shown in Figure 3 .by Figure 4 The figure shows the process of multi-frequency heterodyne solution of absolute phase using horizontal stripes as an example.

[0158] Forward phase unwrapping: First, the wrapped phases of the three frequency (64, 63, 56) fringe patterns are solved by the phase shift method. The wrapped phases of two different frequencies are superimposed by the heterodyne principle to obtain an equivalent phase with a lower frequency. The frequency after superposition is The frequencies f obtained here are 8, 7, and 1. Reverse phase unwrapping: Selecting higher frequency fringes for phase unwrapping will increase the signal-to-noise ratio and improve the accuracy. Therefore, the unwrapped phase obtained by reverse phase unwrapping when f=64 is the absolute phase.

[0159] Among them, the frequency selection and decoding method is based on the encoding and decoding of structured light using any complete coding mode in the prior art; after decoding, the correspondence between the camera pixel coordinates and the projector pixel coordinates is established according to the decoding results, and the inverse camera calibration method in the prior art is used for calibration; the camera is calibrated according to the identified feature points of the calibration plate, and the calibration results are preliminarily calculated according to the telecentric lens imaging model. The pinhole principle is used to calibrate the telecentric lens to solve the symbol ambiguity problem and complete the telecentric lens calibration; then based on the virtual plane theory in the prior art, the spatial relationship between the telecentric lens camera and the projector is constructed, and the nonlinear calibration optimization is completed using Levenberg-Marquardt. Finally, the X and Y lookup tables are obtained. The first stage of the global calibration of the single grating telecentric structured light system is completed.

[0160] S2: Reconstruct planes at different heights to obtain the phases at different heights and their corresponding original coordinates Z; use PCA plane fitting to correct the Z coordinates of the plane point cloud to obtain the ideal coordinates; use cross-ratio invariance to construct the mapping relationship between the phase and Z coordinates at each pixel level to obtain the system local calibration parameters for the Z coordinate.

[0161] Under the assumption of a virtual plane, the parameters in the z direction are missing, and the normal inversion formula cannot be used to solve the unknown world coordinates (X w ,Y w ,Z w ). At the same time, due to the existence of errors, the two straight lines will not intersect perfectly. Therefore, this method introduces the algorithm of the minimum distance of spatial rays to indirectly obtain (X w ,Y w ,Z w ).

[0162] In step S2, planes at different heights are reconstructed to obtain phases at different heights and their corresponding original coordinates Z, which specifically includes the following contents:

[0163] b1: Establish a world coordinate system in the virtual plane with the optical center of the projector as the origin;

[0164] q1=[t x t y 0] T and q2 = [0 0 0] T are the coordinates of the optical center of the camera and projector respectively;

[0165] In a specific implementation, a projector projects orthogonal stripes onto white diffuse reflection planes at different heights, and a spatial ray minimum distance algorithm is used to obtain the original three-dimensional coordinates of the white planes at different heights.

[0166] b2: Connect a ray from the optical center of the camera and projector to the target point respectively. The direction of the ray is determined by the unit vector and express;

[0167] The camera light is perpendicular to the virtual plane, so The direction of the vector is from the projector optical center O P Point to the target point, assuming that the pixel position of the target point in the camera is (u c ,v c ,0), then It is expressed as:

[0168]

[0169] in, and is the effective magnification of the telecentric lens in the X and Y directions, is the distortion center of the camera, R and (t x ,t y ,0) represents the rotation matrix and translation vector in the camera extrinsic parameters;

[0170] b3: The coordinates of any point on the light emitted by the camera are expressed as λ1 is an arbitrary number. Similarly,

[0171] The minimum distance between P1 and P2 is expressed as:

[0172]

[0173] The solution is:

[0174]

[0175] Where v1′ and v2′ are the transposes of v1 and v2;

[0176] because Therefore, λ2 is the Z coordinate of the corresponding pixel in the world coordinate system.

[0177] In this method, solving the minimum distance problem of spatial rays is converted into solving the values ​​of λ1 and λ2.

[0178] In step S2, PCA (Principal Component Analysis) plane fitting is used to correct the error of the Z coordinate of the plane point cloud to obtain the ideal coordinate; specifically, the following steps are included.

[0179] c1: Perform 3D reconstruction on the plane at each height and obtain the point cloud (X, Y, Z);

[0180] c2: Project each point cloud (X, Y, Z) onto the ideal plane and perform PCA plane fitting to obtain the ideal point cloud

[0181] Usually the ideal point cloud can be obtained using the expression aX+bY+cZ+d=0, where (a, b, c, d) are the fitting plane coefficients. In order to reduce the error of the reconstructed point coordinates, this method projects the points onto the ideal plane. The specific formula is:

[0182]

[0183] in, is an ideal three-dimensional point of a pixel point (i, j), t is the vertical distance between the plane and each point, m represents the normal vector of the plane, ‖b‖ represents the length of the vector; (a, b, c, d) are the fitting plane coefficients.

[0184] In step S2, the mapping relationship between the absolute phase and the Z coordinate of each pixel level is constructed using the cross-ratio invariance to obtain the local calibration parameters of the system.

[0185] The schematic diagram of the cross-ratio invariance principle is shown in the attached figure. Figure 5 , we can prove that:

[0186]

[0187] The rays where A, B, C, and D are located represent the different heights corresponding to the same pixel in the world coordinate system, so the actual meaning of AC, BD, BC, and AD is the height difference. The planes where A', B', C', and D' are located represent the normalized planes where the projector projects the image, so the actual meaning of A'C', B'D', B'C', and A'D' is the difference in absolute phase values. According to the formula, by obtaining the actual heights and absolute phase values ​​of at least three sets of planes at different heights, the corresponding relationship between the absolute phase value and the actual height of each pixel in the image can be obtained.

[0188] Then, the specific process in this method includes the following steps.

[0189] d1: Obtain the actual height and absolute phase values ​​of at least three sets of planes at different heights;

[0190] d2: Using the cross-ratio invariance, solve the corresponding relationship between the absolute phase value of each pixel in the image and the actual height:

[0191]

[0192] In the formula, c 0-2 is the pixel-level calibration coefficient of the planes at different heights under the projector view, φ is the absolute phase obtained under the projector view, is the Z value of the ideal point cloud at different heights, and (i, j) is the pixel position on the camera image.

[0193] Orthogonal fringes are used during reconstruction, but not every direction of the fringes is optimal for measurement accuracy, so it is necessary to determine the phase sensitivity of the fringes. In step d2, the method for determining the direction of the optimal absolute phase φ specifically includes the following steps:

[0194] e1: The projector projects orthogonal fringes, thereby obtaining the absolute phase in the horizontal and vertical directions. Due to the positional relationship between the camera and the projector, the sensitivity of the fringes in the two directions is different, and the fringes with high sensitivity are more suitable for constructing the above relationship;

[0195] e2: Ensure that there is no object to be measured in the field of view and obtain the horizontal and vertical absolute phase of the plane;

[0196] e3: Place a gauge block on the plane and obtain the horizontal and vertical absolute phases; subtract the absolute phases obtained twice to obtain the phase of the gauge block, and calculate the average phase value of the horizontal and vertical absolute phases of the gauge block respectively;

[0197] e4: Compare the average phase values ​​of the two directions, and select the stripe direction with the higher average phase value as the best stripe direction; use the absolute phase obtained under the best stripe direction as the fitting phase, that is, the absolute phase φ under the projector view is obtained.

[0198] S3: Collect the unidirectional fringe coded image of the reconstructed object, locate the reconstruction area by the image segmentation method of differential reflectivity; calculate the absolute phase value of the reconstructed area, identify and remove the invalid phase;

[0199] Based on the system global calibration parameters and the system local calibration parameters, the coordinates of each pixel in the reconstruction area in the world coordinate system are mapped out through the fitting relationship to complete the precise three-dimensional reconstruction of the small field of view.

[0200] In step S3, a unidirectional stripe coded image of the reconstructed object is collected, and the reconstruction area is located by using an image segmentation method of difference reflectivity.

[0201] The reconstruction area is located by using the image segmentation method of differential reflectance. The principle of the image segmentation method of differential reflectance is to use the difference in reflectance between objects of different colors (such as black and white objects) to achieve segmentation. Black objects absorb more light and have lower reflectance than white objects. Specifically, the following steps are included.

[0202] f1: Use a high-contrast white plane as the background to identify and analyze the features of the object to be tested;

[0203] f2: Set the invalid point judgment threshold;

[0204] The invalid point judgment threshold means that the pixel gray value greater than this value is judged as oversaturated;

[0205] The specific value of the invalid point judgment threshold is related to the camera parameters. In this embodiment, the invalid point judgment threshold is set to 246;

[0206] f3: collects the unidirectional fringe coded image of the reconstructed object;

[0207] f4: judge all the pixels of the grayscale image obtained by each phase shift;

[0208] If the gray value of a pixel is less than the invalid point judgment threshold, the pixel is marked as a valid pixel;

[0209] f5: Count the total number of all phase shift patterns and count the number of times each pixel is marked as a valid pixel, based on which a saturation map and saturation histogram are constructed;

[0210] f6: Perform threshold segmentation and morphological processing based on the saturation histogram to complete target positioning and obtain the final reconstructed area.

[0211] In the specific implementation, a set of unidirectional gratings is projected onto the object to be measured, and the image is captured by a trigger camera. The grating is a 24-frame horizontal phase shift code, which contains 8-step phase shift stripes of 3 frequencies. The saturation map is constructed by the gray value change in the sinusoidal stripe pattern of each phase shift. The target can be located by threshold segmentation and morphological processing based on the saturation histogram. The absolute phase of the unidirectional grating image in the positioning area is solved by the multi-frequency heterodyne principle.

[0212] The process of the difference reflectance image segmentation method is shown in the attached Figure 6As shown, the calibration plate and array crystal ball are taken as the research objects here. In the figure, (a) the fringe image is a fringe image of the grating acquired by the camera, in which the black background part is the object to be reconstructed; (d) the original image is the original image of the reconstructed object in (a); (b) the saturation image is calculated from (a), and then the saturation histogram is calculated based on (b), and then the threshold segmentation image (e) is obtained based on the threshold segmentation method, and finally the result image (f) is obtained after morphological processing, that is, the yellow background part in (f) is the reconstructed area. In this method, the reconstruction area is located by the image segmentation method of differential reflectance, and subsequent calculations are only performed on the image within the reconstruction area, which reduces the overall calculation amount and improves the efficiency of reconstruction.

[0213] The specific steps of saturation map, saturation histogram, threshold segmentation and morphological processing are implemented based on the existing technology and will not be specifically expanded in this method.

[0214] In actual operation, factors such as noise, uneven light intensity or surface reflection characteristics often cause invalid phase points to appear in some pixels, which cannot correctly reflect the surface information of the object. In this method, the shadow area is removed according to the modulation contrast method. In step S3, the absolute phase value of the reconstructed area is calculated, and the invalid phase is identified and removed, which specifically includes the following steps.

[0215] g1: calculate the absolute phase value of the unidirectional fringe coded image in the reconstruction area;

[0216] The projected fringes are represented by:

[0217] I n (x,y)=A(x,y)+B(x,y)cos[φ(x,y)+2πn / N],n=1,2,...,N;

[0218] Among them, A represents the average intensity, B represents the modulation degree of the deformed fringes, N is the number of phase shift steps; φ is the wrapped phase, which is expressed based on the absolute phase value; (x, y) is the pixel coordinate.

[0219] g2: Calculate the modulation contrast γ corresponding to each point (x, y) in the corresponding view of the reconstructed area; in this method, the modulation contrast γ measures the quality of the fringe image, based on which the abnormal point cloud in the view can be identified and invalid points can be removed;

[0220] The modulation contrast γ is expressed as:

[0221] γ(x,y)=A f (x,y) / B f (x,y);

[0222] The closer γ is to 1, the better the signal-to-noise ratio of the point is, and the closer γ is to 0, the worse the signal-to-noise ratio of the point is.

[0223] B f (x,y) is the modulation degree of fringe images with different frequencies;

[0224]

[0225] Where N is the number of phase shift steps, I n represents the stripe grayscale image of the nth frame, M f represents the modulation image with frequency f;

[0226] A f (x,y) is the average intensity at different frequencies;

[0227]

[0228] g3: According to the preset modulation contrast threshold, all points whose modulation contrast γ is less than the modulation contrast threshold are marked as abnormal point clouds.

[0229] In this method, multiple linear regression analysis (MLRA) is used to determine a clear critical value for phase unwrapping error. Any value beyond this critical range is considered to be a phase unwrapping error point.

[0230] g4: Use the unwrapped phase as sample data, and set the confidence level to 99.99%; approximate the unwrapped phase φ to a plane; the plane is expressed as:

[0231] φ(x,y)≈ax+by+d;

[0232] Where a, b, and d are the plane coefficients and (x,y) are the corresponding coordinates of the unwrapped phase.

[0233] g5: Solve the plane coefficients. After the solution is completed, the fitting residual v(x, y) of each pixel is obtained, that is, the difference between the actual absolute phase value and the phase value obtained by the fitting coefficients. In this embodiment, the least squares method is used to solve the plane coefficients a, b and d;

[0234] The difference in phase value is expressed as:

[0235] v(x,y)=φ(x,y)-(ax+by+d)|;

[0236] Where φ(x,y) is the actual absolute phase value, (ax+by+d) is the fitted phase value, and v is the residual.

[0237] g6: Set the residual recognition threshold, based on which the phase unwrapping error points in the corresponding view of the reconstructed area are identified;

[0238] If a residual error exceeds the residual identification threshold, it is identified as an erroneous unwrapping and marked to obtain a phase unwrapping error point.

[0239] g7: Remove all abnormal point clouds and phase unwrapping error points in the corresponding view of the reconstructed area.

[0240] At the set 99.99% confidence level, the probability that the residual error is distributed within the range of ±4σ (σ is the standard deviation of the residual v) is 99.99%. If the residual error exceeds 4σ, it is identified as an incorrect unwrapping and marked. Therefore, in this embodiment, the residual identification threshold is set to 4σ, where σ is the standard deviation of the residual v.

[0241] Then, the residual recognition threshold is expressed as:

[0242]

[0243] Among them, mask is a two-dimensional template for identifying error points, and v(x,y) is the value of the coordinate (x,y) in the residual v.

[0244] In step S3, based on the system global calibration parameters and the system local calibration parameters, the coordinates of each pixel in the world coordinate system are mapped out through a fitting relationship to complete the precise three-dimensional reconstruction of the small field of view; specifically, the following steps are included.

[0245] h1: Based on the mapping relationship between the absolute phase and the Z coordinate of each pixel level, the Z coordinate of each pixel in the world coordinate system is calculated. The mapping formula is:

[0246]

[0247] Among them, c 0-2 is the pixel-level calibration coefficient under the projector view, φ is the absolute phase obtained under the projector view, and Z w is the calculated depth value and (i,j) is the pixel position on the camera image.

[0248] h2: Based on the X and Y lookup tables, query to obtain the x and y coordinates of each pixel in the world coordinate system to complete the 3D reconstruction. That is, the X and Y coordinates use the formula The generated lookup table is obtained.

[0249] After using the technical solution of the present invention, a telecentric structured light system is used to collect images, a virtual plane is used to calibrate the spatial position relationship between the camera and the projector, and a spatial line-line intersection algorithm is used to obtain the three-dimensional coordinates under global calibration, which solves the problem of telecentric structured light model degradation. The image positioning method based on differential reflectivity reduces the reconstruction range and solves the problem that it is difficult to determine the measurement area of ​​small objects. At the same time, through the identification and removal method of invalid phase values, the interference of invalid points is eliminated, and the problem of noise points affecting the measurement accuracy in the three-dimensional reconstruction of small objects is solved. Based on this method, a MER2-503-36U3M / C telecentric lens with a working range of 46.9×39.3mm is used, which can achieve precise reconstruction of small objects at the level of 10×10mm.

Claims

1. A small field of view precision three-dimensional reconstruction method, comprising the following steps: S1: Collect the calibration plate and its coding pattern to perform global calibration on the telecentric lens camera and projector; S2: Obtain local calibration parameters of the system; S3: Complete precise 3D reconstruction of small fields of view; Features: In step S1, during global calibration, a virtual plane is used to calibrate the spatial relationship between the projector and the camera to obtain system global calibration parameters for the X coordinate and the Y coordinate; In step S2, when performing local calibration, the following steps are specifically included: Reconstruct planes at different heights to obtain the phases at different heights and their corresponding original coordinates Z; use PCA plane fitting to correct the Z coordinates of the plane point cloud to obtain the ideal coordinates; use cross-ratio invariance to construct the mapping relationship between the phase and Z coordinates at each pixel level to obtain the system local calibration parameters for the Z coordinates; In step S3, the three-dimensional reconstruction includes the following steps in detail: Collect the unidirectional stripe coded image of the reconstructed object, locate the reconstruction area by the image segmentation method of differential reflectivity; calculate the absolute phase value of the reconstructed area, identify and remove the invalid phase; Based on the system global calibration parameters and the system local calibration parameters, the coordinates of each pixel in the reconstruction area in the world coordinate system are mapped out through a fitting relationship to complete the precise three-dimensional reconstruction of a small field of view.

2. According to claim 1, a small field of view precision three-dimensional reconstruction method is characterized by: In step S1, a virtual plane is used to calibrate the spatial relationship between the projector and the camera during global calibration, which includes the following steps in detail: a1: construct a virtual plane; The virtual plane refers to the plane where the mirror-parallel plane of the telecentric lens and the center of the projector are located; a2: Assuming that the camera imaging plane coincides with the virtual plane, the spatial relationship from the projector to the camera is the spatial relationship from the projector to the virtual plane; the spatial relationship from the projector to the camera and Respectively expressed as: Among them, b i is the pose of the ith calibration plate, are the rotation matrix and translation vector from the i-th calibration plate pose to the camera coordinate system. Similarly, is the rotation matrix and translation vector of the i-th calibration plate pose to the projector coordinate system; a3: Select the pose with the smallest error among all poses to calculate the rotation matrix and translation vector from the projector to the camera It is expressed as: in, is the final matrix from projector to camera, Min(L1,L2,...,L N ) represents the global error set (L1,L2,...,L N ) Global optimization function L i It is expressed as: Among them, N and M represent the number of calibration plate poses and the number of feature points on the calibration plate respectively; P ij and P[x j ,y j ,z j ] T represents the coordinates of the jth point under the i-th calibration plate posture; c and p refer to the camera and projector viewing angles respectively; and They are the points P[x j ,y j ,z j ] T The coordinates projected onto the camera and projector images are calculated as follows: in, and To detect the point P[x j ,y j ,z j ] T Coordinates on the camera and projector images respectively; a4: Establishing an X and Y lookup table, wherein the lookup table stores the x coordinate and the y coordinate corresponding to the pixel in the image collected by the camera in the world coordinate system; The lookup table is based on the formula generate; Among them, X w , Y w is the world coordinate; and is the effective magnification of the telecentric lens in the X and Y directions; u c and v c It is the feature point recognized in the camera field of view.

3. The small field of view precision three-dimensional reconstruction method according to claim 1, characterized in that: In step S2, planes at different heights are reconstructed to obtain phases at different heights and their corresponding original coordinates Z, which specifically includes the following contents: b1: Establish a world coordinate system in the virtual plane with the optical center of the projector as the origin; q1=[t x t y 0] T and q2 = [0 0 0] T are the coordinates of the optical center of the camera and projector respectively; b2: Connect a ray from the optical center of the camera and projector to the target point respectively. The direction of the ray is determined by the unit vector and express; The camera light is perpendicular to the virtual plane, so The direction of the vector is from the projector optical center O P Point to the target point, assuming that the pixel position of the target point in the camera is (u c ,v c ,0), then It is expressed as: in, and is the effective magnification of the telecentric lens in the X and Y directions, is the distortion center of the camera, R and (t x ,t y ,0) represents the rotation matrix and translation vector in the camera extrinsic parameters; b3: The coordinates of any point on the light emitted by the camera are expressed as λ1 is an arbitrary number. Similarly, The minimum distance between P1 and P2 is: The solution is, Where v1′ and v2′ are the transposes of v1 and v2; because Therefore, λ2 is the Z coordinate of the corresponding pixel in the world coordinate system.

4. The small field of view precision three-dimensional reconstruction method according to claim 3, characterized in that: In step S2, PCA plane fitting is used to correct the Z coordinate of the plane point cloud to obtain the ideal coordinate; specifically, the following steps are included: c1: Perform 3D reconstruction on the plane at each height and obtain the point cloud (X, Y, Z); c2: Project each point cloud (X, Y, Z) onto the ideal plane and perform PCA plane fitting to obtain the ideal point cloud in, is an ideal three-dimensional point of a pixel point (i, j), t is the vertical distance between the plane and each point, n represents the normal vector of the plane, ‖n‖ represents the length of the vector; (a, b, c, d) are the fitting plane coefficients.

5. The small field of view precision three-dimensional reconstruction method according to claim 3, characterized in that: In step S2, the mapping relationship between the absolute phase and the Z coordinate of each pixel level is constructed by using the cross ratio invariance to obtain the local calibration parameters of the system, which specifically includes the following steps: d1: Obtain the actual height and absolute phase values ​​of at least three sets of planes at different heights; d2: Using the cross-ratio invariance, solve the corresponding relationship between the absolute phase value of each pixel in the image and the actual height: In the formula, c 0-2 is the pixel-level calibration coefficient of the planes at different heights under the projector view, φ is the absolute phase obtained under the projector view, is the Z value of the ideal point cloud at different heights, and (i, j) is the pixel position on the camera image.

6. A small field of view precision three-dimensional reconstruction method according to claim 5, characterized in that: In step d2, the optimal absolute phase φ direction confirmation method is as follows: The following steps are involved: e1: The projector projects orthogonal fringes, thereby obtaining the absolute phase in both horizontal and vertical directions; e2: Ensure that there is no object to be measured in the field of view and obtain the horizontal and vertical absolute phase of the plane; e3: Place a gauge block on the plane and obtain the horizontal and vertical absolute phases; subtract the absolute phases obtained twice to obtain the phase of the gauge block, and calculate the average phase value of the horizontal and vertical absolute phases of the gauge block respectively; e4: Compare the average phase values ​​of the two directions and select the stripe direction with the higher average phase value as the best stripe direction; The absolute phase obtained under the optimal fringe direction is used as the fitting phase, that is, the absolute phase φ under the projector view is obtained.

7. The small field of view precision three-dimensional reconstruction method according to claim 1, characterized in that: In step S3, a unidirectional fringe coded image of the reconstructed object is collected, and the reconstruction area is located by an image segmentation method of difference reflectivity; The specific steps include: f1: Use a high-contrast white plane as the background to identify and analyze the features of the object to be tested; f2: Set the invalid point judgment threshold; The invalid point judgment threshold means that a pixel gray value greater than this value is judged to be oversaturated; f3: collects the unidirectional fringe coded image of the reconstructed object; f4: judge all the pixels of the grayscale image obtained by each phase shift; If the gray value of a pixel is less than the invalid point judgment threshold, the pixel is marked as a valid pixel; f5: Count the total number of all phase shift patterns and count the number of times each pixel is marked as a valid pixel, based on which a saturation map and saturation histogram are constructed; f6: Perform threshold segmentation and morphological processing according to the saturation histogram to complete the target positioning and obtain the final reconstructed area.

8. The small field of view precision three-dimensional reconstruction method according to claim 7, characterized in that: In step S3, the absolute phase value of the reconstruction area is calculated, and the invalid phase therein is identified and removed, which specifically includes the following steps: g1: Calculate the absolute phase value of the unidirectional fringe coded image in the reconstruction area; The projected fringes are represented as: Yo n (x,y)=A(x,y)+B(x,y)cos[φ(x,y)+2πn / N],n=1,2,...,N; Where A represents the average intensity, B represents the modulation degree of the deformed fringes, N is the number of phase shift steps; φ is the wrapping phase, expressed based on the absolute phase value; (x, y) is the pixel coordinate; g2: Calculate the modulation contrast γ corresponding to each point (x, y) in the corresponding view of the reconstructed area; The modulation contrast γ is expressed as: γ(x,y)=A f (x,y) / B f (x,y); The closer γ is to 1, the better the signal-to-noise ratio of the point is, and the closer γ is to 0, the worse the signal-to-noise ratio of the point is. B f (x,y) is the modulation degree of fringe images with different frequencies; Where N is the number of phase shift steps, I n represents the stripe grayscale image of the nth frame, M f represents the modulation image with frequency f; A f (x,y) is the average intensity at different frequencies; g3: According to the preset modulation contrast threshold, all points whose modulation contrast γ is less than the modulation contrast threshold are marked as abnormal point clouds; g4: Use the unwrapped phase as sample data, and set the confidence level to 99.99%; approximate the unwrapped phase φ to a plane; the plane is expressed as: φ(x,y)≈ax+by+d; Where a, b, and d are the plane coefficients, and (x, y) are the corresponding coordinates of the unwrapped phase; g5: Solve the plane coefficients. After the solution is completed, the fitting residual v(x,y) of each pixel is obtained, that is, the difference between the actual absolute phase value and the phase value obtained by the fitting coefficient; The difference in phase value is expressed as: v(x,y)=|φ(x,y)-(ax+by+d)|; Where φ(x,y) is the actual absolute phase value, (ax+by+d) is the fitted phase value, and v is the residual; g6: setting a residual identification threshold, and identifying phase unwrapping error points in the view corresponding to the reconstructed area based on the residual identification threshold; If a residual error exceeds the residual identification threshold, it is identified as an erroneous unwrapping and marked to obtain a phase unwrapping error point; g7: Remove all the abnormal point clouds and the phase unwrapping error points in the view corresponding to the reconstructed area.

9. The small field of view precision three-dimensional reconstruction method according to claim 7, characterized in that: The residual identification threshold is set to 4σ, where σ is the standard deviation of the residual v; Then, the residual recognition threshold is expressed as: Among them, mask is a two-dimensional template for identifying error points, and v(x,y) is the value of the coordinate (x,y) in the residual v.

10. The small field of view precision three-dimensional reconstruction method according to claim 7, characterized in that: In step S3, the coordinates of each pixel in the world coordinate system are mapped out through the fitting relationship to complete the precise three-dimensional reconstruction of the small field of view; The following steps are involved: h1: Based on the mapping relationship between the absolute phase and the Z coordinate of each pixel level, the Z coordinate of each pixel in the world coordinate system is calculated. The mapping formula is: Among them, c 0-2 is the pixel-level calibration coefficient under the projector view, φ is the absolute phase obtained under the projector view, and Z w is the calculated depth value, (i, j) is the pixel position on the camera image; h2: Based on the X and Y lookup tables, query the X and Y coordinates of each pixel in the world coordinate system to complete the 3D reconstruction.

Images