High-precision structured light three-dimensional measurement and calibration method suitable for single-axis MEMS projector

Abstract

The invention discloses a high-precision structured light three-dimensional measurement and calibration method suitable for a single-axis MEMS projector. Firstly, an MEMS coordinate system is established for an MEMS mirror, a simple constraint relation of a three-dimensional point in the MEMS coordinate system (MCS) is obtained, and then a simple triangular stereo method expression is obtained to recover three-dimensional information. In addition, a simple and efficient MEMS galvanometer calibration method is provided on the basis of MCS, model calibration is completed only through a checkerboard image at one position, the calibration process is greatly simplified, a three-dimensional measurement method convenient to transplant, an innovative calibration method suitable for a single-axis MEMS galvanometer and an efficient MEMS galvanometer projector calibration method are achieved, and the calibration efficiency is greatly improved. Compared with the prior art, the method has the advantages that the measurement accuracy and stability are improved, the system cost and complexity are reduced, the overall complexity and implementation cost of the system are effectively reduced, the MEMS projector can be widely applied to various application scenes, and the method is particularly suitable for measurement tasks needing miniaturization, low cost and high frequency.

Description

TECHNICAL FIELD

[0001] The application relates to a high-precision structured light three-dimensional reconstruction and calibration method, in particular to a high-precision structured light three-dimensional measurement and calibration method suitable for a single-axis MEMS projector. BACKGROUND

[0002] Three-dimensional reconstruction has important applications in the fields of reverse engineering, virtual reality, cultural relic protection and the like. Striated projection profilometry has become one of the most commonly used methods due to high precision and non-contact advantages. It is usually composed of a camera and a fringe generator. Sinusoidal fringes are projected onto an object, and the deformed fringes are captured by the camera. Finally, the three-dimensional topography is calculated through a decoding algorithm.

[0003] The most common fringe generators include digital light processing (DLP) projectors, micro-electro-mechanical system (MEMS) mirror projectors and liquid crystal on silicon (LCos) projectors. The DLP projector is widely used in three-dimensional measurement due to high brightness and high precision. However, it needs to modulate millions of micro-mirrors and relies on a lens to complete image projection, resulting in high cost and large size of the DLP projector. The MEMS mirror is a small drivable mirror, and the mirror diameter is usually only a few millimeters. The projector based on a single-axis MEMS mirror is usually composed of a line laser, a single-axis MEMS mirror and a FPGA controller. The spot of the line laser irradiates on the mirror of the MEMS mirror, and the single-directional pattern projection is realized by modulating the light intensity of the line laser through the FPGA according to different rotating angles of the MEMS mirror. Compared with the DLP projector, the projector based on the single-axis MEMS mirror has the advantages of compact structure, high scanning frequency and low cost. However, it can only project fringes in one direction, and the traditional three-dimensional measurement method based on the pinhole model of the DLP projector cannot be applied to the MEMS projector, which greatly limits the application of the MEMS projector in the field of fringe projection profilometry (FPP).

[0004] The prior art has the following technical problems:

[0005] 1) Projection direction limitation of the MEMS projector

[0006] The projector based on the single-axis MEMS mirror can only project fringe patterns in one direction and cannot project in multiple directions. This limitation makes the traditional three-dimensional measurement method based on the pinhole model (such as the DLP projector method) unable to be directly applied to the MEMS projector, resulting in that the measurement precision and efficiency cannot meet the actual requirements.

[0007] 2) Lack of standard three-dimensional reconstruction method suitable for the MEMS projector

[0008] Currently, most of the three-dimensional reconstruction methods of MEMS projectors are based on complex mathematical models, such as inverse camera model or multiple calculated multi-plane methods. These methods usually require multiple view images for three-dimensional reconstruction, which is computationally intensive and difficult to achieve real-time processing. Especially in large scenes or high-precision applications, existing methods often cannot effectively meet the dual requirements of precision and real-time performance.

[0009] 3) Complex calibration process

[0010] The calibration process of MEMS projectors usually requires multiple calibration positions and precise hardware support, such as high-precision displacement platforms or multiple image shooting. Existing calibration methods often rely on complex calculation steps, such as mapping between phase and height, interpolation between different planes, etc., resulting in time-consuming calibration process and susceptibility to environmental factors (such as light changes or surface reflection differences), which limits its application in actual industrial environments.

[0011] 4) Poor system adaptability

[0012] Most existing methods rely on specific hardware configurations or complex system adjustments, making these methods less adaptable and difficult to apply to a variety of different MEMS projectors or actual application scenarios. The poor universality of these methods results in frequent debugging and adjustment of system parameters in actual applications, reducing the operability and expandability of the system.

[0013] Based on the above problems, the existing technology needs to be improved, especially in simplifying the three-dimensional reconstruction and calibration process, expanding the system adaptability, new methods need to be proposed to meet the needs of industrial applications and real-time measurement. SUMMARY

[0014] The present application aims to solve the problems existing in the prior art and provides a high-precision structured light three-dimensional measurement and calibration method suitable for single-axis MEMS projectors. A new mathematical model of MEMS mirror is proposed. First, a MEMS coordinate system is established for the MEMS mirror, obtaining a simple constraint relationship of three-dimensional points in the MEMS coordinate system (MCS), and then a simple triangular stereo method expression is obtained to recover three-dimensional information. In addition, a simple and efficient MEMS mirror calibration method based on MCS is proposed, which only needs a chessboard image at one position to complete the model calibration, greatly simplifying the calibration process.

[0015] The present application is achieved by the following technical solutions:

[0016] The present application discloses a high-precision structured light three-dimensional measurement and calibration method suitable for single-axis MEMS projectors, comprising:

[0017] Obtain multiple chessboard calibration board images at different positions;

[0018] The chessboard sub-pixel corner point coordinates are extracted from the obtained chessboard calibration plate images in multiple different positions, camera calibration is performed through Zhang's calibration method, and camera intrinsic and distortion parameters are calculated;

[0019] A single-axis MEMS galvanometer projector is controlled to project a six-step multi-frequency phase shift fringe pattern to one chessboard calibration plate position, and a corresponding fringe image is synchronously collected by a camera;

[0020] According to a phase shift algorithm and a phase unwrapping method, an absolute phase distribution map is recovered from the collected fringe image, and a corresponding relationship between pixels and MEMS galvanometer rotation angles is established through angle pulse counting;

[0021] Chessboard sub-pixel corner point coordinates in the fringe image are extracted, and corresponding MEMS galvanometer rotation angle values are calculated using the absolute phase distribution map and the corresponding relationship between pixels and MEMS galvanometer rotation angles;

[0022] Combined with the camera intrinsic and distortion parameters, a triangulation model in a MEMS coordinate system (MCS) is established, the extracted chessboard sub-pixel corner point coordinates of the fringe image and the MEMS galvanometer rotation angle values are substituted into the equation, and three-dimensional coordinates of the chessboard calibration plate corners in the camera coordinate system are recovered by solving;

[0023] Through rotation axis fitting and reference plane (YOZ plane) fitting algorithms, an extrinsic matrix of the MEMS galvanometer projector relative to the camera is calculated.

[0024] As a further improvement, the present application obtains multiple chessboard calibration plate images in different positions, specifically at least 12 or more chessboard images in different positions to improve measurement accuracy and ensure sufficient corner point information for calibration and three-dimensional reconstruction.

[0025] As a further improvement, the present application extracts chessboard sub-pixel corner point coordinates from the obtained multiple chessboard calibration plate images in different positions, performs camera calibration through Zhang's calibration method, and calculates camera intrinsic and distortion parameters, specifically using a sub-pixel precision corner point extraction method to process at least 12 chessboard images in different positions, calculating a camera intrinsic matrix and distortion parameters through Zhang's calibration method, obtaining a camera intrinsic matrix, completing camera calibration, and accurately describing the imaging model of the camera to provide accurate geometric parameters for subsequent three-dimensional reconstruction.

[0026] As a further improvement, the present application controls a single-axis MEMS galvanometer projector to project a six-step multi-frequency phase shift fringe pattern to one chessboard calibration plate position, and synchronously collects a corresponding fringe image by a camera, specifically:

[0027] The rotation angle of the MEMS galvanometer is sinusoidally modulated using an FPGA, and the distorted fringe image captured by the camera is represented as:

[0028]

[0029] where I n is the nth fringe pattern, n∈[0,N), N is the number of phase shift steps (N≥3), (u,v) is the camera image coordinate, A is the average intensity, B is the intensity modulation, δ n =2πn / N is the phase shift, Φ=2πθ / T is the absolute phase, T is the fringe period, θ is the rotation angle of the MEMS mirror.

[0030] As a further improvement, the present application recovers the absolute phase distribution map from the collected fringe images according to the phase shift algorithm and the phase unwrapping method, and establishes the correspondence between the pixel and the MEMS mirror rotation angle through the angle pulse counting, specifically:

[0031] The wrapped phase φ is solved as:

[0032]

[0033] where arctan is the four-quadrant inverse tangent function, with a range of (-π,π];

[0034] The phase unwrapping method is adopted to obtain the continuous phase, which is divided into time phase unwrapping, stereo phase unwrapping, and double-frequency order phase unwrapping and geometric constraints are used to obtain the unwrapped phase Φ.

[0035] As a further improvement, the present application extracts the chessboard sub-pixel corner point coordinates in the fringe image, calculates the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between the pixel and the MEMS mirror rotation angle, specifically:

[0036] The relationship between the unwrapped phase Φ and the rotation angle is:

[0037]

[0038] where the MEMS mirror sends an angle mark pulse every 0.05 degrees of rotation, and the relationship between the camera image coordinates (u,v) and the MEMS mirror rotation angle θ is obtained by the phase shift method.

[0039] As a further improvement, the present application combines the camera intrinsic distortion parameters to establish a triangulation model under the MEMS coordinate system (MCS), substitutes the extracted chessboard sub-pixel corner point coordinates in the fringe image and the MEMS mirror rotation angle value into the equation, and solves to recover the three-dimensional coordinates of the chessboard corner points in the camera coordinate system, specifically:

[0040] The pinhole model is used to describe the camera mathematical model, and a point in the world coordinate system WCS is p w (xw ,y w ,z w ), which is converted to the camera coordinate system CCS, and then projected onto the camera image coordinate system (u c ,v c ), which is described as:

[0041]

[0042] where A 3×4 is the camera intrinsic parameter, M 4×4 is the camera extrinsic parameter, s is the scale factor, H is the product of the camera extrinsic parameter and the camera intrinsic parameter, and image distortion correction is needed due to lens distortion in the actual scene, (u c ,v c ) is the result of distortion correction.

[0043] When the laser light is incident on the center of the MEMS mirror, the single-axis MEMS projector is simplified as a light plane rotating around the MEMS rotation axis. The MEMS coordinate system MCS is established for the MEMS projector, the y-axis coincides with the MEMS rotation axis, the origin is at the axis point, and the z-axis is perpendicular to the y-axis and is set in the plane with a rotation angle of 0°. According to the geometric relationship, the relationship between the three-dimensional point and the rotation angle θ in the MEMS coordinate system is obtained:

[0044]

[0045] The world coordinate system and the MEMS coordinate system are coincided, and Eq(4) is rewritten as:

[0046]

[0047] where H is the projection matrix of MCS to the camera image coordinate system, which is obtained by pre-calibration, (u c ,v c ,θ), and the values to be solved are (x m ,y m ,z m ,s), so that:

[0048]

[0049] x m =tanθz m

[0050] By establishing the MEMS coordinate system for the MEMS mirror, the simple constraint relationship of the three-dimensional point in MCS is obtained, and the formula expression similar to the triangular measurement method of the DLP system is finally obtained. The method is convenient for transplantation from the DLP system, and pixel-to-pixel GPU acceleration is easy to perform.

[0051] As a further improvement, the application described by the fitting of the rotation axis fitting and the reference plane (YOZ plane) fitting algorithm, the calculation of the MEMS mirror projector relative to the camera of the external parameter matrix, specifically:

[0052] In order to obtain the transformation matrix of the camera coordinate system and the MEMS coordinate system, the MEMS mirror projector calibration process is divided into: y axis and origin calibration, z axis calibration, phase shift stripe and uniform illumination figure are projected on the chessboard, and the phase figure is obtained by phase extraction algorithm and phase unwrapping algorithm;

[0053] The three-dimensional coordinates of the white block center point on the chessboard in the camera coordinate system P={p0,…,p n} are obtained by PnP method, and the corresponding phase Φ={Φ0,…,Φ n} is obtained by projecting the white block center point to the camera image coordinate system and interpolating the phase figure, wherein the phase and the MEMS mirror rotation angle θ={θ0,…,θ n};

[0054] The three-dimensional coordinates of the chessboard corner point P and the MEMS rotation angle θ are one-to-one correspondence, in order to solve the rotation axis of the MEMS mirror, the angle relationship between the three-dimensional point P and the rotation axis is used to complete the calibration; n y (α y ,β y ,γ y ) T The direction vector of the rotation axis, o(x0,y0,z0) T The point on the axis, if any two points p j (x j ,y j ,z j ) T and p k (x k ,y k ,z k ) T , their corresponding rotation angles are θ j and θ k , p j , p k and o form the vector l j and l k :

[0055]

[0056] The normal vector of the plane where l j and n y are located is n j , l k and n yThe normal vector of the plane where the point is located is n k :

[0057]

[0058] The angle between the two planes is Δθ jk = |θ k - θ j | is obtained:

[0059]

[0060] Where [α y , β y , γ y , x0, y0, z0] are parameters to be solved, [x j , y j , z j , x k , y k , z k , θ j , θ k ] are known parameters, since the point on the shaft is not constrained on the normal of the shaft, let y0 = 0, n y There is no constraint on the vector module, let β y = 1; here since Δθ jk is a small value, the slope of cos(Δθ jk ) is low near Δθ jk = 0, the optimization function is constructed to solve the parameters by least squares method:

[0061]

[0062] Each two points on the checkerboard get a set of equations, and at least four points are needed to solve the equation set, so all the corner points on the checkerboard are used for least squares solution to fit the high-precision rotation axis;

[0063] The plane where the MEMS rotation angle is 0 is defined as the yoz plane, and the three-dimensional point p j is rotated counterclockwise by θ j around n y to obtain p′ j , so that these points all fall on the yoz plane, and the following is obtained by the Rodrigues formula:

[0064]

[0065] I is a 3 × 3 unit matrix, and the yoz plane is fitted by p' j and the y-axis to obtain two constraint conditions:

[0066]

[0067] Wherein n x =(α x ,β x ,γ x ) T It is the x-axis direction of the MEMS coordinate system, let alpha x =1 to eliminate the degree of freedom of vector length, get:

[0068]

[0069] At the same time, the least square method is constructed to fit the plane

[0070]

[0071] z-axis gets:

[0072] n z =n x ×n y

[0073] Get the conversion relationship of the MEMS coordinate system in the camera coordinate system:

[0074]

[0075] The present application has the following beneficial effects:

[0076] 1) Conveniently transplant three-dimensional measurement method

[0077] The present application combines the camera internal participation distortion parameter, establishes the triangulation model under the MEMS coordinate system (MCS), substitutes the sub-pixel corner point coordinates of the extracted fringe image and the MEMS mirror rotation angle value into the equation, solves and restores the three-dimensional coordinates of the chessboard calibration board corner point in the camera coordinate system. By establishing the MEMS coordinate system for the MEMS mirror, the simple constraint relationship of the three-dimensional point in the MCS is obtained, and finally the formula expression similar to the triangulation method of the DLP system is obtained. The method is convenient to transplant from the DLP system, and because pixel to pixel can be easily accelerated by GPU.

[0078] 2) Innovative calibration method suitable for single-axis MEMS mirror

[0079] The application utilizes the three-dimensional coordinates of the restored chessboard corner points in the camera coordinate system and the corresponding MEMS mirror rotation angle values to calculate the external parameter matrix of the projector relative to the camera through the fitting of the rotation axis and the reference plane. The application innovatively proposes a calibration method suitable for single-axis MEMS mirror three-dimensional measurement, establishes a MEMS coordinate system (MCS), and proposes a simplified constraint relationship, so that the three-dimensional point restoration process is realized through a simple triangular stereo method. The traditional three-dimensional reconstruction method based on a DLP projector cannot be directly applied to MEMS mirrors, but the application breaks through this technical bottleneck through a simple mathematical model and a triangular shape restoration method, and improves the application potential of the MEMS projector in three-dimensional measurement.

[0080] 3) Efficient MEMS mirror projector calibration method

[0081] The application controls the single-axis MEMS mirror projector to project a six-step multi-frequency phase shift fringe pattern to the position of a chessboard calibration board, and synchronously collects the corresponding fringe images by the camera. Only a single position chessboard image is needed to complete the calibration, which greatly simplifies the traditional calibration process. The traditional MEMS projector calibration method usually needs multi-position shooting and relies on complex hardware support. The unit position calibration method of the application not only reduces the number of shooting required for calibration, but also reduces the dependence on high-precision displacement platforms, so that the calibration process is more efficient and convenient.

[0082] 4) Improved measurement accuracy and stability

[0083] The application extracts the chessboard sub-pixel corner point coordinates in the fringe image, restores the absolute phase distribution map from the collected fringe images according to the phase shift algorithm and the phase unwrapping method, and establishes the corresponding relationship between the pixels and the MEMS mirror rotation angle through the angle pulse counting. By using the combination of sub-pixel extraction of the chessboard corner points and the phase shift technology, compared with the existing phase angle method, the application shows lower measurement error and higher stability, which can effectively reduce the error caused by environmental interference (such as uneven reflection and light change), so as to realize higher precision three-dimensional reconstruction.

[0084] 5) Reduced system cost and complexity

[0085] The application is a three-dimensional measurement system based on a single-axis MEMS mirror, and the hardware configuration adopted is more compact, and the cost is lower than that of the traditional DLP projector system. By simplifying the calibration process and three-dimensional reconstruction calculation, the application effectively reduces the overall complexity and implementation cost of the system, so that the MEMS projector is more widely applied in various application scenarios, and is especially suitable for small, low-cost and high-frequency measurement tasks. BRIEF DESCRIPTION OF DRAWINGS

[0086] Figure 1It is a mathematical model schematic diagram of a three-dimensional measurement system based on a single-axis MEMS mirror, in which, 1 is a MEMS mirror, and 2 is a camera;

[0087] Figure 2 It is a projection system calibration principle diagram based on a single-axis MEMS mirror;

[0088] Figure 3 It is a calibration and reconstruction flow chart of a three-dimensional measurement system based on a single-axis MEMS mirror. DETAILED DESCRIPTION

[0089] The technical scheme of the present application will be further described below by combining with the drawings of the specification and through specific implementation examples:

[0090] The purpose of the present application is to solve the problem that the existing DLP projector measurement method cannot be applied to the measurement of the single-axis MEMS mirror 1, and to propose a three-dimensional measurement method for the projector based on the single-axis MEMS mirror 1, and the device for realizing the present application comprises a laser (with a built-in Powell prism), a camera 2, a single-axis MEMS mirror 1, and a field programmable gate array (FPGA) controller. The device adopts a triangulation layout, the laser and the Powell prism are coaxial, the long side direction of the outgoing linear laser is parallel to the rotation axis (Y axis) of the MEMS mirror 1, and the laser is accurately irradiated on the center of the mirror surface of the MEMS mirror 1; the camera and the MEMS mirror 1 are separated in space, the optical centers of the two form a baseline with a certain length, and the optical axis of the camera and the initial reflected light path of the mirror form a certain angle, so as to form an effective triangulation structure and ensure the measurement sensitivity in the depth direction. When working, the FPGA controller controls the MEMS mirror 1 to periodically scan and synchronously modulates the intensity of the laser, so as to project the phase shift fringe pattern onto the surface of the object. The camera synchronously collects the deformed fringe pattern under the triggering of the FPGA. The calculation processing device calculates the absolute phase from the image by using the phase shift algorithm and the phase unwrapping technology, and finally reconstructs the three-dimensional appearance of the object through the triangulation model by combining the phase-angle mapping relationship determined by the angle pulse of the MEMS mirror 1.

[0091] Figure 1 It is a mathematical model schematic diagram of a three-dimensional measurement system of the present application;

[0092] The specific implementation method of the present application is as follows:

[0093] Obtain multiple images of checkerboard calibration boards at different positions. Place a high-precision checkerboard calibration board in the measurement field of view of the system, change the spatial posture of the calibration board, and ensure that it covers the entire field of view of the camera and has sufficient changes in inclination and rotation angle. Use the camera to collect clear checkerboard images from at least 12 different directions and store them. The purpose of this step is to provide sufficient sample data for subsequent camera parameter calibration.

[0094] The sub-pixel corner coordinates of the checkerboard are extracted from the images of the checkerboard calibration board at different positions, and the camera is calibrated by Zhang's calibration method to calculate the camera internal and distortion parameters. The corner detection function in the computer vision library (such as OpenCV) is used to process all the obtained checkerboard images to automatically extract the sub-pixel level precision coordinates of the corners in the checkerboard; then, the widely used Zhang's calibration method is used to perform nonlinear optimization calculation based on the corner coordinate data and the known physical size of the checkerboard. Finally, the internal parameter matrix of the camera and the lens distortion coefficient are solved, which will be used for subsequent image correction and three-dimensional reconstruction.

[0095] The single-axis MEMS mirror 1 projector projects a six-step multi-frequency phase shift fringe pattern to one checkerboard calibration board position, and the corresponding fringe images are synchronously captured by the camera. The checkerboard calibration board is fixed at a specific position in the measurement field. The single-axis MEMS mirror 1 projector is driven by the FPGA controller to project a set (preferably six steps) of sinusoidal phase shift fringe patterns with different frequencies (such as 32, 256, 1024 pixel periods) to the calibration board in turn, and the distorted fringe images captured by the camera are represented as:

[0096]

[0097] where I n is the nth fringe image, n ∈ [0, N); N is the number of phase shift steps (N ≥ 3), (u, v) is the camera image coordinate, A is the average intensity, B is the intensity modulation, δ n = 2πn / N is the phase shift, Φ = 2πθ / T is the absolute phase, T is the fringe period, and θ is the rotation angle of the MEMS mirror. At the same time, the FPGA sends a precise synchronization trigger signal to the camera to ensure that the camera synchronously captures each fringe pattern during stable projection, thereby obtaining a series of deformed fringe images modulated by the checkerboard surface.

[0098] According to the phase shift algorithm and the phase unwrapping method, the absolute phase distribution map is recovered from the collected fringe images, and the corresponding relationship between the pixels and the rotation angle of the MEMS mirror 1 is established by angle pulse counting. For the above collected phase shift fringe image sequence, the wrapped phase is calculated pixel by pixel using the standard multi-step phase shift algorithm (such as six-step phase shift method). Then, combined with the phase unwrapping algorithm such as the heterodyne principle, the wrapped phase φ is unwrapped as:

[0099]

[0100] where arctan is the four-quadrant inverse tangent function, and the range is (-π, π];

[0101] Phase unwrapping method is adopted to obtain continuous phase, which is divided into time phase unwrapping, stereo phase unwrapping, adopting double frequency order phase unwrapping and geometric constraint to obtain unwrapped phase Φ.

[0102] The wrapped phase is calculated as absolute phase value Φ, so as to obtain the absolute phase distribution map of the whole field. There is a relationship between the absolute phase value Φ and the physical rotation angle θ of the MEMS mirror 1 as follows:

[0103]

[0104] Where Δθ is the angle pulse interval, and T is the fringe period; according to the angle pulse number recorded by the FPGA, a one-to-one correspondence between the absolute phase value Φ of each pixel point on the image and the physical rotation angle θ of the MEMS mirror 1 is established.

[0105] The chessboard sub-pixel corner coordinates in the fringe image are extracted, and the corresponding MEMS mirror 1 rotation angle value is calculated by using the absolute phase distribution map and the correspondence between the pixel and the MEMS mirror 1 rotation angle. The chessboard corner detection is performed on a certain frame of fringe image collected in the front, and the high-precision sub-pixel corner coordinates are extracted again. Then, by using the obtained absolute phase distribution map, the absolute phase value Φ corresponding to each corner coordinate position is found by using the bilinear interpolation method, and the relationship between the absolute phase value Φ and the rotation angle is:

[0106]

[0107] Where the MEMS mirror sends an angle mark pulse every 0.05 degrees of rotation, and the relationship between the camera image coordinates (u, v) and the MEMS mirror 1 rotation angle θ is obtained by the phase shift method.

[0108] Combined with the camera internal participation distortion parameters, a triangulation model under the MEMS coordinate system (MCS) is established, and the sub-pixel corner coordinates of the extracted fringe image and the MEMS mirror 1 rotation angle value are substituted into the equation to solve and restore the three-dimensional coordinates of the chessboard corner in the camera coordinate system.

[0109] Pinhole model is widely used to describe the mathematical model of camera. A point in the world coordinate system (WCS) is p w (x w ,y w ,z w ), which is converted to the camera coordinate system (CCS), and then projected to the camera image coordinate system (u c ,v c ). The process is described as:

[0110]

[0111] Where A3×4 is camera intrinsic parameters, M 4×4 is camera extrinsic parameters, s is scale factor, H is the product of camera extrinsic parameters and intrinsic parameters. Due to lens distortion in actual scene, image distortion correction is needed, and (u c ,v c ) are considered as the results of distortion correction.

[0112] When the laser light is irradiated on the center of the MEMS mirror 1, the single-axis MEMS mirror 1 projector is simplified as a light plane rotating around the MEMS rotation axis, and the MEMS coordinate system (MCS) of the MEMS mirror 1 projector is established as shown in Figure 1 , the y-axis coincides with the MEMS rotation axis, the origin is at the axis point, and the z-axis is perpendicular to the y-axis and is set in the plane with a rotation angle of 0°. According to the geometric relationship, the relationship of the three-dimensional point in the MEMS coordinate system and the rotation angle θ is obtained.

[0113]

[0114] In order to simplify, the pinhole model formula above is rewritten as:

[0115]

[0116] where H is the projection matrix of MCS to the camera image coordinate system, and is obtained by pre-calibration, (u c ,v c , θ) is obtained by the relationship between the absolute phase value Φ and the physical rotation angle θ of the MEMS mirror 1, and the values to be solved are (x m ,y m ,z m , s). By combining the relationship of the three-dimensional point in the MEMS coordinate system and the rotation angle θ and the simplified pinhole model formula, the following formula is obtained:

[0117]

[0118] x m = tanθz m

[0119] By substituting the pixel coordinates and the corresponding rotation angle θ into the equation set, the three-dimensional coordinates of each chessboard corner point in the camera coordinate system (CCS) can be solved.

[0120] By establishing the MEMS coordinate system for the MEMS mirror 1, the simple constraint relationship of the three-dimensional point in the MCS is obtained, and the formula expression similar to the triangular measurement method of the DLP system is finally obtained. The method can be conveniently transplanted from the DLP system, and because the pixel-to-pixel can be easily accelerated by GPU.

[0121] In addition, in the case that the existing DLP projector calibration method cannot be applied to the single-axis MEMS mirror 1 projector calibration, a single-axis MEMS mirror 1 projector calibration method is proposed, and the external parameter matrix of the MEMS mirror 1 projector relative to the camera is calculated through the rotation axis fitting and the reference plane (YOZ plane) fitting algorithm.

[0122] Figure 2 It is the MEMS projection system calibration schematic diagram of the application;

[0123] The specific implementation method of the application is as follows:

[0124] The three-dimensional coordinates P of the white block center points on the camera coordinate system on the checkerboard are P={p0,…,p n} are obtained by PnP method, and the corresponding phase Φ is obtained by projecting the white block center points to the camera image coordinate system and interpolating the phase diagram, that is, Φ={Φ0,…,Φ n}, wherein the phase and the MEMS mirror 1 rotation angle θ are θ={θ0,…,θ n}, as shown in Figure 2

[0125] The y-axis and the origin are calibrated, the three-dimensional coordinates P of the checkerboard corner points and the MEMS rotation angle θ are one-to-one correspondence, and in order to solve the rotation axis of the MEMS mirror, the angle relationship between the three-dimensional points P and the plane formed by the rotation axis is used to complete the calibration.n y (α y ,β y ,γ y ) T is the direction vector of the rotation axis, and o(x0,y0,z0) T is the point on the axis, and it is assumed that any two points p j (x j ,y j ,z j ) T and p k (x k ,y k ,z k ) T on the checkerboard corner points correspond to the rotation angles θ j and θ k , the vectors formed by p j , p k and o are l j and l k :

[0126]

[0127] l j and n y ​The normal vector of the plane in which it lies is n j , l k and n y The normal vector of the plane in which it lies is n k :

[0128]

[0129] The angle between the two planes is Δθ jk =|θ k -θ j |Result:

[0130]

[0131] Where [α] y ,β y ,γ y [x0, y0, z0] are the parameters to be determined, [x j ,y j ,z j ,x k ,y k ,z k ,θ j ,θ k [The parameters are known. Since the points on the axis are not constrained on the normal to the axis of rotation, let y0 = 0, n] y There are no constraints on the vector magnitude, let β y =1. Here, due to Δθ jk It is a small value, cos(Δθ) jk ) in Δθ jk The slope is low near 0, so the optimization function is constructed and the parameters are solved using the least squares method:

[0132]

[0133] Every two points on the chessboard result in a set of equations, so at least four points are needed to complete the set of equations. Therefore, using all the corner points of the chessboard for least squares solution is sufficient to fit a high-precision rotation axis.

[0134] The z-axis and x-axis are calibrated, and the plane with a MEMS rotation angle of 0 is defined as the yoz plane. The three-dimensional point p... j around n y Rotate counterclockwise θ j Get p' j This ensures that all these points lie on the yoz plane, and they are obtained using the Rodriguez formula:

[0135]

[0136] I is a 3×3 identity matrix, obtained through p' jand y axis to fit the yoz plane, resulting in two constraint conditions:

[0137]

[0138] where n x = (α x , β x , γ x ) T is the x axis direction of the MEMS coordinate system, let α x = 1 to eliminate the degree of freedom of vector length, resulting in:

[0139]

[0140] At the same time, construct a least squares method to fit the plane:

[0141]

[0142] z axis to get:

[0143] n z = n x × n y

[0144] System parameters can be obtained by the transformation matrix of MCS in CCS, as shown below:

[0145]

[0146] where, is the normalized vector of n T , H describes the projection matrix from the 3D point of the MCS to the camera pixel coordinates.

[0147] Those skilled in the art understand that the above description is only a single example of the invention and is not intended to limit the invention, although the invention is described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples, or make equivalent replacement for part of the technical features. Any modification, equivalent replacement, etc. within the spirit and principles of the invention shall be included in the protection scope of the invention.

Claims

1. A method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector, characterized in that, include: Obtain images of the chessboard calibration board at multiple different locations; The coordinates of sub-pixel corner points of the chessboard are extracted from multiple chessboard calibration board images at different locations. The camera is calibrated using Zhang's calibration method, and the camera's internal distortion parameters are calculated. A single-axis MEMS galvanometer projector is controlled to project a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and the corresponding fringe image is simultaneously acquired by a camera. Based on the phase shift algorithm and phase unwrapping method, the absolute phase distribution map is recovered from the acquired stripe image, and the correspondence between pixels and MEMS mirror rotation angle is established by angular pulse counting. Extract the corner coordinates of the checkerboard subpixels in the stripe image, and calculate the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angle. By combining the distortion parameters of the camera's internal components, a triangulation model under the MEMS coordinate system (MCS) is established. The coordinates of the sub-pixel corner points of the extracted stripe image and the rotation angle value of the MEMS galvanometer are substituted into the equation to solve and recover the three-dimensional coordinates of the corner points of the checkerboard calibration plate under the camera coordinate system. The extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera is calculated using a rotation axis fitting and reference plane (YOZ plane) fitting algorithm.

2. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, characterized in that, The aforementioned method of obtaining multiple chessboard calibration board images at different locations specifically involves using at least 12 chessboard images at different locations to improve measurement accuracy and ensure sufficient corner information for calibration and 3D reconstruction.

3. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, characterized in that, The process involves extracting sub-pixel corner coordinates of the checkerboard from multiple different locations of the obtained checkerboard calibration board images, performing camera calibration using Zhang's calibration method, and calculating camera intrinsic parameters and distortion parameters. Specifically, this involves processing checkerboard images at least 12 locations using a sub-pixel precision corner extraction method, calculating the camera intrinsic parameter matrix and distortion parameters using Zhang's calibration method, obtaining the camera intrinsic parameter matrix, and completing the camera calibration to accurately describe the camera's imaging model and provide precise geometric parameters for subsequent 3D reconstruction.

4. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, 2, or 3, characterized in that, The control single-axis MEMS galvanometer projector projects a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and the camera simultaneously acquires the corresponding fringe images, specifically as follows: By using an FPGA to sinusoidally modulate the rotation angle of a MEMS galvanometer, the distorted fringe image captured by the camera is represented as follows: Among them I n Let n be the nth fringe pattern, n∈[0,N); N is the phase shift step number (N≥3), (u,v) are the camera image coordinates, A is the average intensity, B is the intensity modulation, and δ n =2πn / N is the phase shift, Φ=2πθ / T is the absolute phase, T is the fringe period, and θ is the rotation angle of the MEMS lens.

5. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 4, characterized in that, The method described above, which uses a phase-shifting algorithm and a phase unwrapping method to recover the absolute phase distribution map from the acquired stripe image and establishes the correspondence between pixels and the rotation angle of the MEMS galvanometer through angle pulse counting, specifically involves: The enclosed phase φ is solved as: Where arctan is the inverse tangent function in the fourth quadrant, with a range of (-π, π]. Phase unwrapping methods are used to obtain continuous phases and are divided into time phase unwrapping and three-dimensional phase unwrapping. Dual-frequency sub-phase unwrapping and geometric constraints are used to obtain the unwrapped phase Φ.

6. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 5, characterized in that, The extraction of checkerboard sub-pixel corner coordinates in the striped image, and the calculation of the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angles, specifically involves: The relationship between the unwrapping phase Φ and the rotation angle is as follows: The MEMS lens sends an angle marker pulse every 0.05 degrees of rotation. The relationship between the camera image coordinates (u,v) and the MEMS lens rotation angle θ is obtained by phase shift method.

7. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 4, characterized in that, The aforementioned method combines camera-integrated distortion parameters to establish a triangulation model in the MEMS coordinate system (MCS). The coordinates of the sub-pixel corner points of the extracted stripe image's checkerboard pattern and the rotation angle of the MEMS galvanometer are substituted into the equation to solve and recover the three-dimensional coordinates of the checkerboard calibration board corner points in the camera coordinate system. Specifically: The camera's mathematical model is described using a pinhole model, where a point is p in the world coordinate system (WCS). w (x w ,y w ,z w It is transformed to the camera coordinate system (CCS) and then projected onto the camera image coordinate system (u). c ,v c On the other hand, this process is described as follows: Where A 3×4 It's the camera's internal parameters, M. 4×4 Here, 's' is the camera's extrinsic parameter, 's' is the scale factor, and 'H' is the product of the camera's extrinsic and intrinsic parameters. In real-world scenarios, due to lens distortion, image distortion correction is necessary. c ,v c This is the result after distortion correction; When the laser beam illuminates the center of the MEMS lens, the single-axis MEMS projector is simplified as a plane of light rotating around the MEMS rotation axis. A MEMS coordinate system (MCS) is established for the MEMS projector, with the y-axis coinciding with the MEMS rotation axis, the origin at a point on the axis, and the z-axis perpendicular to the y-axis and set in a plane with a rotation angle of 0°. Based on geometric relationships, the relationship between the three-dimensional point in the MEMS coordinate system and the rotation angle θ is obtained as follows: By aligning the world coordinate system and the MEMS coordinate system, Eq(4) can be rewritten as: Where H is the projection matrix from the MCS to the camera image coordinate system, obtained through pre-calibration, (u c ,v c The value to be solved is (x, θ). m ,y m ,z m ,s), then we get: x m =tanθz m By establishing a MEMS coordinate system on the MEMS galvanometer, a simple constraint relationship of three-dimensional points in the MCS is obtained, and finally a formula expression similar to the triangulation method of the DLP system is obtained. The proposed method is easy to port from the DLP system, and pixel-to-pixel acceleration by the graphics processing unit (GPU) is easy.

8. The structured light three-dimensional measurement method based on a single-axis MEMS galvanometer projector according to any one of claims 1, 2, 3, 5, 6, or 7, characterized in that, The aforementioned algorithm, which uses rotation axis fitting and reference plane (YOZ plane) fitting, calculates the extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera, specifically as follows: To obtain the transformation matrix between the camera coordinate system and the MEMS coordinate system, the MEMS galvanometer projector calibration process is divided into: y-axis and origin calibration, z-axis calibration, phase shift fringe pattern and uniform illumination pattern are projected onto the checkerboard, and phase image is obtained through phase extraction algorithm and phase unwrapping algorithm; The three-dimensional coordinates of the center point of the white piece on the chessboard in the camera coordinate system are P = {p0,…,p...} n The PnP method is used to obtain the center points of these white blocks. These points are projected onto the camera image coordinate system, and the corresponding phases Φ = {Φ0,…,Φ1} are obtained by interpolating the phase map. n }, where the phase and the MEMS mirror rotation angle θ={θ0,…,θ n }; The three-dimensional coordinates P of the checkerboard corner points and the MEMS rotation angle θ have a one-to-one correspondence. To determine the rotation axis of the MEMS lens, the calibration is performed using the angle between the three-dimensional point P and the plane formed by the rotation axis. y (α y ,β y ,γ y ) T Let o(x0, y0, z0) be the direction vector of the rotation axis. T Let p be a point on the axis. If any two points p on the corner points of the chessboard are... j (x j ,y j ,z j ) T and p k (x k ,y k ,z k ) T Their corresponding rotation angles are θ j and θ k p j p k The vector formed by o is l j and l k : l j and n y The normal vector of the plane in which it lies is n j , l k and n y The normal vector of the plane in which it lies is n k : The angle between the two planes is Δθ jk =|θ k -θ j |Result: Where [α] y ,β y ,γ y [x0, y0, z0] are the parameters to be determined, [x j ,y j ,z j ,x k ,y k ,z k ,θ j ,θ k [The parameters are known. Since the points on the axis are not constrained on the normal to the axis of rotation, let y0 = 0, n] y There are no constraints on the vector magnitude, let β y =1; here due to Δθ jk It is a small value, cos(Δθ) jk ) in Δθ jk The slope is low near 0, so the optimization function is constructed and the parameters are solved using the least squares method: Every two points on the chessboard result in a set of equations. At least four points are needed to complete the set of equations. Therefore, using all the corner points on the chessboard for least squares solution is sufficient to fit a high-precision rotation axis. The plane with a MEMS rotation of 0 is defined as the yoz plane, and the three-dimensional point p... j around n y Rotate counterclockwise θ j Get p' j This ensures that all these points lie on the yoz plane, which can be obtained using the Rodriguez formula: I is a 3×3 identity matrix, obtained through p' j By fitting the yoz plane with the y-axis, two constraints are obtained: Where n x =(α x ,β x ,γ x ) T Let α be the x-axis direction of the MEMS coordinate system. x =1 to eliminate the degree of freedom of the vector length, resulting in: Simultaneously, a least squares method is constructed for plane fitting. The z-axis yields: n z =n x ×n y The transformation relationship between the MEMS coordinate system and the camera coordinate system is obtained:

Images