Microstructured light three-dimensional measurement system calibration method based on standard spherical surface

Abstract

The invention discloses a standard spherical surface-based microscopic structured light three-dimensional measurement system calibration method, and relates to the technical field of structured light three-dimensional measurement, and the method comprises the following steps: S1, building a three-dimensional measurement model based on a triangulation structured light system, and building a new world coordinate system according to constraint requirements; s2, building a system by using a telecentric lens, representing a world coordinate system by using a camera coordinate system and a projector coordinate system, and solving a three-dimensional coordinate formula according to a geometric constraint relationship; s3, based on the special geometric structure of the standard spherical surface, deriving a solving method of specific calibration parameters in the system calibration process; s4, regarding the projector as an inverse camera, and establishing a relationship between camera pixels and projector pixels by using a homonymy point matching method generated by structured light image coding projection; and S5, using a camera to shoot a group of horizontal and vertical stripe projection drawings of the sphere, and calculating calibration parameters. The calibration of the whole system can be completed by only collecting a group of horizontal and vertical projection fringe patterns, and the operation is simple.

Description

Technical Field

[0001] The present invention relates to the technical field of structured light three-dimensional measurement, and in particular to a calibration method for a microscopic structured light three-dimensional measurement system based on a standard sphere. Background Art

[0002] With its significant advantages such as non-contact, high precision and high efficiency, structured light 3D measurement technology has been widely used in industrial inspection, medical imaging and defect detection. The principle of structured light 3D measurement is to project light with stripe coding to the object to be measured, collect the corresponding deformed stripe pattern by the camera, extract the phase information through decoding, and finally obtain the 3D coordinates of the object to be measured by calibrating the internal and external parameters of the system.

[0003] Structured light system calibration is an important part of achieving three-dimensional measurement. At present, calibration methods are mainly divided into two categories: phase height method and triangulation method. The phase height method directly establishes the functional relationship between phase difference and height, but this method is limited by geometric constraints, and the error of height reconstruction is large when the phase difference is 0. Subsequently, a virtual reference plane was proposed, that is, a virtual reference plane with a known height was created along the original reference plane. For example, patent CN 202010248074.2 discloses "a fast large-scale phase unwrapping method based on dual reference planes". The virtual reference plane method produces higher reconstruction accuracy within the entire measurement range, but complicates system calibration. The triangulation method is to establish a system model including a camera coordinate system, a world coordinate system, and a projection coordinate system, and realize triangulation through the conversion relationship between the coordinate systems. In the system calibration, the projector is regarded as an "inverse camera", and the projection process is regarded as the inverse process of camera acquisition. At this time, the system calibration can be regarded as the calibration of two cameras. Camera calibration often uses Zhang Zhengyou calibration method, shooting a standard calibration plate, detecting corner points, and calibrating the internal and external parameters of the camera. This method usually requires multiple calibration images in different poses during calibration. The process is cumbersome and calibration is prone to failure when the depth of field is limited or the calibration plate is tilted at a large angle, resulting in poor versatility and is not universal in some industrial scenarios. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a calibration method for a microscopic structured light three-dimensional measurement system based on a standard sphere, which can complete the calibration of the entire system by only collecting a set of projection fringe patterns, and the operation is simple.

[0005] The technical problem to be solved by the present invention is achieved by adopting the following technical solutions:

[0006] The calibration method of the microstructured light three-dimensional measurement system based on the standard sphere includes the following steps:

[0007] S1. Build a 3D measurement model based on a triangulation structured light system and establish a new world coordinate system according to the constraint requirements.

[0008] The optical axes of camera A and projector B are in the same plane, the optimal imaging centers coincide, and the ordinates are parallel, that is, v A ∥v B ; Take any plane H perpendicular to the angular bisector of the two lens optical axes, the intersection points of H and the two optical axes are O′ and O″, the straight line where O′O″ is located is the X axis, parallel to the ordinate v A The Y axis is outward, the straight line where the optical axis angle bisector is located is vertically upward as the Z axis, which intersects O′O″ at O. With O as the origin, the world coordinate system O-XYZ is established, and the angle between the two optical axes of camera A and projector B is 2θ.

[0009] S2. Use a telecentric lens to build a system, express the world coordinate system using the camera coordinate system and the projector coordinate system, and obtain the three-dimensional coordinate formula based on the geometric constraint relationship.

[0010] With O′ and O″ as the origins respectively, the camera coordinate system O′-x′y′z′ and the projector coordinate system O″-x″y″z″ are established, where the x′ axis, x″ axis and X axis are on the same straight line and have the same direction, while the z′ axis, z″ axis and Z axis are parallel to each other and have the same direction.

[0011] For any point P in space, let the distance between O′ and O″ be 2l, then the coordinate constraint of point P is Point P is projected onto points C and D on plane H along the optical axes of camera A and projector B. From a geometric perspective, the world coordinates of point P are: The coordinates of point P are expressed using the coordinates of the local coordinate systems O′ and O″ respectively.

[0012] Since the telecentric lens has orthogonal projection characteristics and low distortion, the physical coordinates of the camera sensor pixels can be directly described. Assume that the lens magnifications of camera A and projector B are β A , β B According to the geometric relationship, the local coordinate (x i ′,y i ′) and image plane coordinates (u i ,v i ) Where i = A, B, substituting into point P, we can see that the absolute formula of the three-dimensional coordinates is: in, d 0 The unit of length for system calibration; μ A and μ B are the sensor pixel sizes of camera A and projector B respectively; (u A,v A ) and (u B ,v B ) are the image plane coordinates of camera A and projector B respectively. The intersection of the optical axis and the sensor is taken as the origin of the reference coordinate system. For the reference coordinate system, the relative coordinates of 3D reconstruction are

[0013] It can be seen from the above formula that the measurement accuracy of the system can reach the equal pixel level and is not affected by the distance l between the two lenses, which reflects the microscopic characteristics of the system.

[0014] S3. Based on the special geometric structure of the standard sphere, the solution method for specific calibration parameters in the system calibration process is derived.

[0015] Taking into account the installation error, the optical axes of camera A and projector B are not in the same plane, and the longitudinal coordinates are not parallel. Error correction is introduced in combination with the 3D reconstruction formula.

[0016] Optical axis non-coplanarity correction: Y = v A =εv B +d. Where d is the distance that projector B is translated along the ordinate direction of camera A; ε is a constant related to the relative magnification of the two camera lenses and their relative pixel sizes.

[0017] Correction for non-parallel vertical coordinate: The intersection of the optical axis and the sensor is W(u 0 ,v 0 ), taking camera A as the reference, projector B rotates around the optical axis by an angle α, and the positive direction of rotation is along the propagation direction of the optical axis, and we get Among them, (u′ B ,v′ B ) is the pixel coordinate of the image; (u B ,v B ) is the image plane coordinate after the camera rotates around the optical axis, and we get Among them, δv is a constant that is fixed when the camera is installed. In summary, the parameters that need to be calibrated in the three-dimensional coordinate formula are κ, θ, ε, α and δv 0 .

[0018] ε, α and δv 0 By taking three non-collinear points P i (X i ,Y i ,Z i )Form the equation Calculate. Among them, (Δu i ′,Δv i ′), i=1,2, indicating P i Relative to P 0 The pixel coordinates of Substituting into the set equation we can solve for δv 0 and ε.

[0019] To solve for κ and θ, we use the sphere to construct the equation and adopt the standard sphere solution. Set the radius of the standard sphere to R, and use the tetrahedron circumcenter coordinate formula to select four points P on the sphere i (i=0,1,2,3), the coordinates of the center of the sphere are Among them, D and D i (i=1,2,3) are all determinants, According to the tetrahedron circumcenter coordinate formula, we know Tetrahedron Vertex P i (i=0,1,2,3) should be located on the unit sphere and satisfy (X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 =1; choose a different value from P on the sphere i The fifth coordinate point P of (i=0,1,2,3) 4 (X 4 ,Y 4 ,Z 4 ), which also satisfies the unit sphere, and we get (X 4 -X C ) 2 +(Y 4 -Y C ) 2 +(Z 4 -Z C ) 2 =(X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 ; κ and θ can be solved from the two equations.

[0020] S4. Treat the projector as an inverse camera, project a stripe pattern onto the object, and decode the absolute phase by phase shift method. The phase of the same object point is the same. Use the matching method of the same-name points generated by structured light image coding projection to establish the relationship between the camera pixels and the projector pixels. Where p is the projector pixel; c is the camera pixel; W is the width of the coded fringe field of view; Φ is the absolute phase, ranging from (0, 2π).

[0021] S5. Use the camera to shoot a set of horizontal and vertical stripe projection images of the sphere, decode to obtain the absolute phase, and solve its coordinates according to the relationship between the camera pixels and the projector pixels; obtain the object points on the spherical surface as required, substitute them into the calibration parameter solution formula, and solve the calibration parameters κ, θ, ε, α and δv 0 , substitute the obtained calibration parameters into the three-dimensional coordinate formula to complete the calibration of the entire system.

[0022] The beneficial effects of the present invention are as follows: compared with the prior art, the structured light three-dimensional measurement system calibration method provided by the present invention has the following advantages:

[0023] 1. The calibration of the entire system can be completed by only shooting a set of projection fringe patterns, which reduces the acquisition of calibration images and is very convenient;

[0024] 2. The calibration of the whole system is completed by shooting the ball, which reduces the parameters that need to be calibrated;

[0025] 3. It is easier to obtain effective information from the spherical surface of the ball than from the calibration plate, which increases the probability of successful calibration when the depth of field is limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the geometric principle coordinate system diagram of the structured light system;

[0027] Figure 2 It is the reference coordinate system diagram of the measurement system;

[0028] Figure 3 The calibration ball image and absolute phase image collected during the calibration process;

[0029] Figure 4 This is the 3D reconstruction effect of the standard sphere. DETAILED DESCRIPTION

[0030] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further described below in conjunction with specific embodiments and diagrams.

[0031] Equipment preparation: camera, projector, telecentric lens, fixed bracket, calibration ball (r=4.9991mm).

[0032] Example 1

[0033] S1. Build Figure 1 The three-dimensional measurement model based on the triangulation structured light system shown establishes a new world coordinate system according to the constraint requirements.

[0034] The optical axes of camera A and projector B are in the same plane, the optimal imaging centers coincide, and the ordinates are parallel, that is, v A ∥vB ; Take any plane H perpendicular to the angular bisector of the two lens optical axes, the intersection points of H and the two optical axes are O′ and O″, the straight line where O′O″ is located is the X axis, parallel to the ordinate v A The Y axis is outward, the straight line where the optical axis angle bisector is located is vertically upward as the Z axis, which intersects O′O″ at O. With O as the origin, the world coordinate system O-XYZ is established, and the angle between the two optical axes of camera A and projector B is 2θ.

[0035] S2. Use a telecentric lens to build a system, express the world coordinate system using the camera coordinate system and the projector coordinate system, and obtain the three-dimensional coordinate formula based on the geometric constraint relationship.

[0036] With O′ and O″ as the origins respectively, the camera coordinate system O′-x′y′z′ and the projector coordinate system O″-x″y″z″ are established, where the x′ axis, x″ axis and X axis are on the same straight line and have the same direction, while the z′ axis, z″ axis and Z axis are parallel to each other and have the same direction.

[0037] Figure 2 So Figure 1 The reference coordinate system diagram of the measurement system with O as the origin. For any point P in space, let the distance between O′ and O″ be 2l, then the coordinate constraint condition of point P is Point P is projected onto points C and D on plane H along the optical axes of camera A and projector B. From a geometric perspective, the world coordinates of point P are: The coordinates of point P are expressed using the coordinates of the local coordinate systems O′ and O″ respectively.

[0038] Since the telecentric lens has orthogonal projection characteristics and low distortion, the physical coordinates of the camera sensor pixels can be directly described. Assume that the lens magnifications of camera A and projector B are β A , β B According to the geometric relationship, the local coordinate (x i ′,y i ′) and image plane coordinates (u i ,v i ) Where i = A, B, substituting into point P, we can see that the absolute formula of the three-dimensional coordinates is: in, d 0 The unit of length for system calibration; μ A and μ B are the sensor pixel sizes of camera A and projector B respectively; (u A ,v A ) and (u B ,v B) are the image plane coordinates of camera A and projector B respectively. The intersection of the optical axis and the sensor is taken as the origin of the reference coordinate system. For the reference coordinate system, the relative coordinates of 3D reconstruction are

[0039] It can be seen from the above formula that the measurement accuracy of the system can reach the equal pixel level and is not affected by the distance l between the two lenses, which reflects the microscopic characteristics of the system.

[0040] S3. Based on the special geometric structure of the standard sphere, the solution method for specific calibration parameters in the system calibration process is derived.

[0041] Taking into account the installation error, the optical axes of camera A and projector B are not in the same plane, and the longitudinal coordinates are not parallel. Error correction is introduced in combination with the 3D reconstruction formula.

[0042] Optical axis non-coplanarity correction: Y = v A =εv B +d. Where d is the distance that projector B is translated along the ordinate direction of camera A; ε is a constant related to the relative magnification of the two camera lenses and their relative pixel sizes.

[0043] Correction for non-parallel vertical coordinate: The intersection of the optical axis and the sensor is W(u 0 ,v 0 ), taking camera A as the reference, projector B rotates around the optical axis by an angle α, and the positive direction of rotation is along the propagation direction of the optical axis, and we get Among them, (u′ B ,v′ B ) is the pixel coordinate of the image; (u B ,v B ) is the image plane coordinate after the camera rotates around the optical axis, and we get Among them, δv is a constant that is fixed when the camera is installed. In summary, the parameters that need to be calibrated in the three-dimensional coordinate formula are κ, θ, ε, α and δv 0 .

[0044] ε, α and δv 0 By taking three non-collinear points P i (X i ,Y i ,Z i )Form the equation Calculate. Among them, (Δu i ′,Δv i ′), i=1,2, indicating P i Relative to P 0 The pixel coordinates of Substituting into the set equation we can solve for δv 0 and ε.

[0045] To solve for κ and θ, we use the sphere to construct the equation and adopt the standard sphere solution. Set the radius of the standard sphere to R, and use the tetrahedron circumcenter coordinate formula to select four points P on the sphere i (i=0,1,2,3), the coordinates of the center of the sphere are Among them, D and D i (i=1,2,3) are all determinants, According to the tetrahedron circumcenter coordinate formula, we know Tetrahedron Vertex P i (i=0,1,2,3) should be located on the unit sphere and satisfy (X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 =1; choose a different value from P on the sphere i The fifth coordinate point P of (i=0,1,2,3) 4 (X 4 ,Y 4 ,Z 4 ), which also satisfies the unit ball, we can get (X 4 -X C ) 2 +(Y 4 -Y C ) 2 +(Z 4 -Z C ) 2 =(X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 ; It is homogeneous with respect to the spatial coordinates. Since the coordinates are proportional to κ, in the calibration calculation, κ = 1 can be set to obtain the value of θ, and finally by Get κ. So far, all calibration parameters have been solved.

[0046] S4. Treat the projector as an inverse camera, project a stripe pattern onto the object, and decode the absolute phase by phase shift method. The phase of the same object point is the same. Use the matching method of the same-name points generated by structured light image coding projection to establish the relationship between the camera pixels and the projector pixels. Where p is the projector pixel; c is the camera pixel; W is the width of the coded fringe field of view; Φ is the absolute phase, ranging from (0, 2π).

[0047] S5. Use a computer to control the camera to capture a set of horizontal and vertical projection fringe patterns of the sphere, and decode them to obtain the absolute phase. The calibration ball captured by the camera and the resolved horizontal and vertical absolute phase patterns are as follows: Figure 3 As shown. Solve the coordinates according to the relationship between the camera pixels and the projector pixels; obtain the object points on the spherical surface as required, substitute them into the calibration parameter solution formula, and solve the calibration parameters κ, θ, ε, α and δv 0 , substitute the obtained calibration parameters into the three-dimensional coordinate formula to complete the calibration of the entire system.

[0048] Traditional calibration methods are usually divided into camera calibration and projector calibration. Camera calibration often uses Zhang Zhengyou calibration method, which shoots a standard calibration plate (chessboard or dot pattern), detects corner points, and calibrates the internal and external parameters of the camera. Projector calibration mainly establishes the relationship between camera pixels and projector pixels. The phase of the same object point of the two is the same, and the corner point coordinates of the camera can be mapped to the projector coordinates. After calibrating the internal parameters of the camera and projector, a new world coordinate system is established to complete the calibration of the entire system through rotation and translation. The calibration method of the present invention and the traditional calibration method are used to perform radius fitting on the calibration ball (r = 4.9991mm) ten times, and the root mean square error (RMS) of the point cloud fitting is calculated. The results are shown in Table 1.

[0049] Table 1

[0050]

[0051] From the data in Table 1, it can be seen that the average value of the fitting radius of the reconstructed ball by the traditional calibration method is 4.454945mm, the reconstruction accuracy is about 50μm, and the RMS average value of the point cloud fitting is about 0.050; while the average value of the fitting radius of the reconstructed ball by the calibration method of the present invention is 4.998746mm, the reconstruction accuracy is about 4μm, and the RMS average value of the point cloud fitting is about 0.029. This shows that the system accuracy of the calibration method of the present invention is improved by an order of magnitude compared with the traditional calibration method, and the quality of the obtained point cloud is also higher. Figure 4 In order to reconstruct the three-dimensional point cloud of the calibration ball after the system is calibrated using the method of the present invention, the surface is smooth without major deformation and loss.

[0052] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A calibration method for a microscopic structured light three-dimensional measurement system based on a standard sphere, characterized in that: The following steps are involved: S1. Build a 3D measurement model based on a triangulation structured light system and establish a new world coordinate system according to the constraint requirements; S2. Use a telecentric lens to build a system, express the world coordinate system using the camera coordinate system and the projector coordinate system, and obtain the three-dimensional coordinate formula based on the geometric constraint relationship; S3. Based on the special geometric structure of the standard sphere, the method for solving the specific calibration parameters in the system calibration process is derived; S4, treating the projector as an inverse camera, and using the same-name point matching method generated by structured light image coding projection to establish the relationship between camera pixels and projector pixels; S5. Use a camera to capture a set of horizontal and vertical stripe projection images of the sphere, obtain calibration parameters according to steps S2 and S3, and complete the calibration of the entire system.

2. The calibration method of the microscopic structured light three-dimensional measurement system based on the standard sphere according to claim 1, characterized in that: The constraints for establishing the world coordinate system in step S1 specifically include: the optical axes of camera A and projector B are in the same plane, the optimal imaging centers coincide, and the ordinates are parallel, that is, v A ∥v B ; Take any plane H perpendicular to the angular bisector of the two lens optical axes, the intersection points of H and the two optical axes are O′ and O″, the straight line where O′O″ is located is the X axis, parallel to the ordinate v A The Y axis is outward, the straight line where the optical axis angle bisector is located is vertically upward as the Z axis, which intersects O′O″ at O. With O as the origin, the world coordinate system O-XYZ is established, and the angle between the two optical axes of camera A and projector B is 2θ.

3. The calibration method of the microstructured light three-dimensional measurement system based on the standard sphere according to claim 1, characterized in that: The specific steps of obtaining the three-dimensional coordinate formula according to the geometric constraint relationship in step S2 are as follows: With O′ and O″ as the origins, establish the camera coordinate system O′-x′y′z′ and the projector coordinate system O″-x″y″z″, where the x′ axis and the x″ axis are in the same straight line and have the same direction as the X axis, and the z′ axis and the z″ axis are parallel to the Z axis and have the same direction; For any point P in space, let the distance between O′ and O″ be 2l, then the coordinate constraint of point P is Point P is projected onto points C and D on plane H along the optical axes of camera A and projector B. From a geometric perspective, the world coordinates of point P are: The coordinates of point P are expressed using the coordinates of the local coordinate systems O′ and O″ respectively. Assume that the lens magnifications of camera A and projector B are β A , β B According to the geometric relationship, the local coordinate (x i ′,y i ′) and image plane coordinates (u i ,v i ) Where i = A, B, substituting into point P, we can see that the absolute formula of the three-dimensional coordinates is: in, d0 is the system calibration length unit; μ A and μ B are the sensor pixel sizes of camera A and projector B respectively; (u A ,v A ) and (u B ,v B ) are the image plane coordinates of camera A and projector B respectively; the intersection of the optical axis and the sensor is taken as the origin of the reference coordinate system. For the reference coordinate system, the relative coordinates of 3D reconstruction are 4. The calibration method of the microstructured light three-dimensional measurement system based on the standard sphere according to claim 1, characterized in that: The method for solving the calibration parameters in step S3 specifically includes the following steps: Optical axis non-coplanarity correction: Y = v A =εv B +d; where d is the distance that projector B is translated along the ordinate direction of camera A; ε is a constant related to the relative magnification of the two camera lenses and their relative pixel sizes; Correction for non-parallel vertical coordinates: The intersection of the optical axis and the sensor is W(u0,v0). Taking camera A as the reference, projector B rotates around the optical axis by an angle α, and the positive direction of rotation is along the propagation direction of the optical axis. Among them, (u′ B ,v′ B ) is the pixel coordinate of the image; (u B ,v B ) is the image plane coordinate after the camera rotates around the optical axis, and we get Among them, δv is a constant that is determined when the camera is installed in place; further, we can get By taking three non-collinear object points P i (X i ,Y i ,Z i )Form the equation Calculate ε, α and δv0; where (Δu i ′,Δv i ′), i=1,2, indicating P i Relative to the pixel coordinates of P0, Substituting into the set equation we can solve for δv0 and ε. Set the radius of the standard sphere to R, and use the tetrahedron circumcenter coordinate formula to select four points P on the sphere. i (i=0,1,2,3), the coordinates of the center of the sphere are Among them, D and D i (i=1,2,3) are all determinants, According to the tetrahedron circumcenter coordinate formula, we know Tetrahedron Vertex P i (i=0,1,2,3) should be located on the unit sphere and satisfy (X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 =1; choose a different value from P on the sphere i The fifth coordinate point P4 (X4, Y4, Z4) of (i = 0, 1, 2, 3) is obtained (X4-X C ) 2 +(Y4-Y C ) 2 +(Z4-Z C ) 2 =(X i -X C ) 2 +(Y i -Y C ) 2 +(Z i -Z C ) 2 , solve for κ and θ.

5. The calibration method of the microstructured light three-dimensional measurement system based on the standard sphere according to claim 1, characterized in that: The relationship between the camera pixels and the projector pixels in step S4 is Where p is the projector pixel; c is the camera pixel; W is the width of the coded fringe field of view; Φ is the absolute phase, ranging from (0, 2π).

6. The calibration method of the microscopic structured light three-dimensional measurement system based on the standard sphere according to claim 1, characterized in that: The calibration of the overall system in step S5 specifically includes the following steps: using a camera to capture a set of horizontal and vertical stripe projection images of the sphere, decoding to obtain the absolute phase, and solving its coordinates based on the relationship between the camera pixels and the projector pixels; obtaining object points on the spherical surface as required, substituting them into the calibration parameter solution formula, solving the calibration parameters κ, θ, ε, α and δv0, and substituting the obtained calibration parameters into the three-dimensional coordinate formula to complete the calibration of the overall system.

Images