Three-dimensional reconstruction point cloud data optimization method for telecentric optical system

Abstract

The invention relates to a method for optimizing three-dimensional reconstruction point cloud data of a telecentric optical system. The method comprises the following steps: acquiring camera pixel points of a target object on a camera plane; determining projector pixel points matched with the camera pixel points through a phase coordinate conversion method; according to a telecentric optical system three-dimensional imaging model, determining an epipolar line on a projector image plane matched with the camera pixel points as a reference epipolar line; and optimization is carried out according to the distance from the projector pixel points to the reference polar line, so that the projector pixel points in the point cloud data meet polar line constraints. According to the method, the reconstruction point cloud result is optimized according to the geometrical relationship, and the accuracy and precision of three-dimensional reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a method for optimizing point cloud data for three-dimensional reconstruction of a telecentric optical system. Background Art

[0002] Microscopic Fringe Projection Profilometry (MFPP), as a high-precision non-contact 3D surface measurement technology, is widely used in the field of precision manufacturing due to its high efficiency and precise measurement capabilities. The MFPP system using a telecentric lens can provide extremely high imaging quality and measurement accuracy, because the telecentric lens can produce a near-perfect orthogonal projection, greatly reducing image distortion and ensuring the accuracy of the measurement data. The MFPP system projects a sinusoidal fringe pattern onto the surface of an object through a projector, and uses a camera equipped with a telecentric lens to capture the fringe deformation caused by the morphological changes on the surface of the object, and then calculates the 3D surface information of the object. However, despite the many advantages of the MFPP technology using a telecentric lens, there are still some limitations in practical applications, especially in the processing of noise points during the reconstruction process. In existing MFPP systems, even if a high-performance telecentric lens is used, the reconstruction results are often affected by various factors, such as ambient light interference, special properties of the object surface (such as high reflectivity or occlusion shadows), and the accuracy limitations of the equipment itself, which results in a large number of noise points in the generated point cloud data.

[0003] To solve the above problems, existing solutions mainly focus on improving hardware equipment or adjusting the measurement environment, but these methods are often costly and complicated to operate. Another idea is to use point cloud filtering algorithms to remove noise points, but the denoising effect is limited and it is easy to cause the loss of some key point cloud data. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for optimizing point cloud data for three-dimensional reconstruction of a telecentric optical system, thereby removing noise points in the point cloud data and optimizing the data, thereby obtaining a more accurate three-dimensional reconstruction result.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a method for optimizing point cloud data of three-dimensional reconstruction of a telecentric optical system, comprising the following steps:

[0006] Get the camera pixel points of the target object on the camera plane;

[0007] Determine the projector pixel point that matches the camera pixel point by a phase coordinate conversion method;

[0008] Determine, according to the three-dimensional imaging model of the telecentric optical system, an epipolar line on the projector image plane that matches the camera pixel point as a reference epipolar line;

[0009] Optimization is performed based on the distance from the projector pixel to the reference epipolar line, so that the projector pixel points in the point cloud data all satisfy the epipolar line constraint.

[0010] Further, the optimization is performed according to the distance from the projector pixel to the reference epipolar line, including:

[0011] Calculating the distance from the projector pixel to the reference epipolar line as an epipolar deviation;

[0012] The epipolar line deviation is compared with a set threshold value. If the epipolar line deviation is greater than the set threshold value, the projector pixel point is eliminated; otherwise, the position of the projector pixel point is updated.

[0013] Furthermore, the updating of the position of the projector pixel point is achieved by updating the projector pixel point to the foot of its perpendicular to the reference polar line.

[0014] Furthermore, the polar line deviation is calculated by the following formula:

[0015]

[0016] in, is the epipolar deviation, u p and v p is the coordinate of the projector pixel, A, B and C are the reference epipolar lines Au p +Bv p +C=0 coefficient.

[0017] Furthermore, the updated projector pixel point is calculated by the following formula:

[0018]

[0019] in, and is the updated coordinate of the projector pixel, u p and v p is the coordinate of the projector pixel before updating, A, B and C are the reference polar lines Au p +Bv p +C=0 coefficient.

[0020] Furthermore, the method also includes the step of bringing the updated projector pixel points into the three-dimensional imaging model to calculate and obtain a three-dimensional reconstruction result.

[0021] Furthermore, the step of determining, according to the telecentric optical system three-dimensional imaging model, an epipolar line on the projector image plane that matches the camera pixel point as a reference epipolar line includes:

[0022] The constraint relationship between the camera pixel points and the projector pixel points is derived using a telecentric optical system three-dimensional imaging model, thereby determining the reference epipolar line.

[0023] Furthermore, the determining of the projector pixel points matching the camera pixel points by a phase coordinate conversion method comprises:

[0024] A four-step phase shift method and a multi-frequency technique are used to obtain the horizontal absolute phase and the vertical absolute phase, and then the projector pixel coordinates corresponding to the camera pixel points are calculated.

[0025] Beneficial Effects

[0026] Due to the adoption of the above-mentioned technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention constructs epipolar deviations by analyzing the epipolar constraints under the telecentric stereo vision system, and divides the reconstructed point cloud results into invalid noise points and optimizable error points according to the size of the epipolar deviations, which can effectively remove noise points with large errors in the reconstructed point cloud, and at the same time optimize the error points through the geometric relationship between the epipolar lines and the observed projector pixels, thereby improving the accuracy and precision of the three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of epipolar constraints of a telecentric stereo vision system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0030] The embodiments of the present invention relate to an MFPP reconstruction point cloud optimization method based on epipolar geometry constraints, such as Figure 1 As shown, the following steps are included:

[0031] (1) obtaining the camera pixel points of the target object on the camera plane;

[0032] (ii) determining the projector pixel points that match the camera pixel points by a phase coordinate conversion method;

[0033] (3) determining, based on the three-dimensional imaging model of the telecentric optical system, an epipolar line on the projector image plane that correctly matches the camera pixel point as a reference epipolar line;

[0034] (iv) Optimizing the distance from the projector pixel to the reference epipolar line so that all projector pixel points in the point cloud data satisfy the epipolar line constraint.

[0035] The above steps are further explained below.

[0036] (I) Obtaining the camera pixel points of the target object on the camera plane

[0037] Due to the orthographic projection property of the telecentric lens, the projection process of the object to the camera plane can be expressed as:

[0038]

[0039] Among them, (u c ,v c ) is the camera pixel of the target object, (X w ,Y w ,Z w ) is the world coordinate of the target object, M c Represents the corresponding projection matrix, which can be determined by the calibration parameters of the system.

[0040] (ii) Determine the projector pixel points that match the camera pixel points through the phase coordinate conversion method

[0041] Since the projector cannot directly form an image, this embodiment adopts a phase-coordinate conversion scheme to determine the pixel point (u c ,v c ) matches the projector pixel (u p ,v p ), the horizontal absolute phase is obtained by four-step phase shift method and multi-frequency technology and vertical absolute phase The corresponding projector pixel coordinates can be obtained as follows:

[0042]

[0043] where n h , n v Respectively represent the number of horizontal and vertical stripes of the projection, W p ×H p is the resolution of the projector image.

[0044] (III) Determine the epipolar line on the projector image plane that correctly matches the camera pixel point based on the three-dimensional imaging model of the telecentric optical system as the reference epipolar line

[0045] When the camera pixel (u c ,v c ) and projector pixels (u p ,v p), the three-dimensional coordinates of the object can be determined by solving the following formula:

[0046]

[0047] This formula contains four equations and three unknowns. The three-dimensional coordinates of the object can be solved by the least squares method. The premise for this method to accurately restore the three-dimensional information of the object is that (u c ,v c ) and (u p ,v p ) have a correct matching relationship to ensure that they correspond to the same point in space. However, in the actual measurement process, the existence of phase error makes the matching accuracy unguaranteed, resulting in a large number of noise points in the reconstructed point cloud.

[0048] In order to solve this problem, the present invention analyzes the epipolar constraint relationship under the telecentric stereo vision system, such as Figure 1 As shown. Due to the properties of orthogonal projection of telecentric lenses, given a point (u c ,v c ), the possible three-dimensional coordinates (P0, P1, P2) of the object to be measured are located on the ray l perpendicular to the camera image plane, and the orthographic projection e of l to the projector image plane is the same as (u c ,v c ) is the possible position of the projector pixel that matches. According to formula (3), the correct matching camera pixel (u c ,v c ) and projector pixels (u p ,v p ) have the following constraints:

[0049]

[0050] After conversion of formula (4), the equation expression of the polar line e in the projector pixel coordinate system can be obtained, which is expressed as:

[0051]

[0052] The values ​​of A, B and C can be obtained by conversion according to formula (4).

[0053] Formula (5) shows that for each camera pixel, its matching projector pixel satisfies the epipolar constraint, that is, it is located on the epipolar line e, and the corresponding epipolar equation can be determined one by one by the calibration parameters of the system.

[0054] (iv) Optimize the distance from the projector pixel to the reference epipolar line so that all projector pixels in the point cloud data meet the epipolar line constraint

[0055] In order to evaluate the quality of the match, the present invention defines the epipolar deviation is the projector pixel (u p ,v p ) to the epipolar line e:

[0056]

[0057] In the formula, τ is a manually set threshold. For points with an epipolar deviation greater than τ, the present invention regards them as invalid points and removes them; for points with an epipolar deviation less than τ, the present invention regards them as error points that can be optimized. In order to optimize the error points, the present invention takes the observed projector pixel point (u p ,v p ) to the foot of the epipolar line e as the optimal matching point

[0058]

[0059] The optimized projector pixel Substituting it into formula (3) for calculation, a more accurate reconstruction result can be obtained.

Claims

1. A method for optimizing point cloud data for three-dimensional reconstruction of a telecentric optical system, characterized in that: The following steps are involved: Get the camera pixel points of the target object on the camera plane; Determine the projector pixel point that matches the camera pixel point by a phase coordinate conversion method; Determine, according to the three-dimensional imaging model of the telecentric optical system, an epipolar line on the projector image plane that matches the camera pixel point as a reference epipolar line; Optimization is performed based on the distance from the projector pixel to the reference epipolar line, so that the projector pixel points in the point cloud data all satisfy the epipolar line constraint.

2. The method according to claim 1, characterized in that The optimizing according to the distance from the projector pixel to the reference epipolar line comprises: Calculating the distance from the projector pixel to the reference epipolar line as an epipolar deviation; The epipolar line deviation is compared with a set threshold value. If the epipolar line deviation is greater than the set threshold value, the projector pixel point is eliminated; otherwise, the position of the projector pixel point is updated.

3. The method according to claim 2, characterized in that The updating of the position of the projector pixel point is achieved by updating the projector pixel point to the foot of the perpendicular to the reference polar line.

4. The method according to claim 3, characterized in that: The polar deviation is calculated by the following formula in, is the epipolar deviation, u p and v p is the coordinate of the projector pixel, A, B and C are the reference epipolar lines Au p +Bv p +C=0 coefficient.

5. The method according to claim 3, characterized in that: The updated projector pixel is calculated by the following formula: in, and is the updated coordinate of the projector pixel, u p and v p is the coordinate of the projector pixel before updating, A, B and C are the reference polar lines Au p +Bv p +C=0 coefficient.

6. The method according to claim 2, characterized in that The method also includes the step of bringing the updated projector pixel points into the three-dimensional imaging model to calculate and obtain a three-dimensional reconstruction result.

7. The method according to claim 1, characterized in that The step of determining the epipolar line on the projector image plane that matches the camera pixel point as the reference epipolar line according to the telecentric optical system three-dimensional imaging model includes: The constraint relationship between the camera pixel points and the projector pixel points is derived using a telecentric optical system three-dimensional imaging model, thereby determining the reference epipolar line.

8. The method according to claim 1, characterized in that: The method of determining the projector pixel point that matches the camera pixel point by using a phase coordinate conversion method includes: A four-step phase shift method and a multi-frequency technique are used to obtain the horizontal absolute phase and the vertical absolute phase, and then the projector pixel coordinates corresponding to the camera pixel points are calculated.

Images