A high-precision structured light three-dimensional measurement and calibration method suitable for a single-axis MEMS projector

Abstract

This invention discloses a high-precision structured light 3D measurement and calibration method suitable for single-axis MEMS projectors. First, a MEMS coordinate system is established for the MEMS mirror, obtaining simple constraint relationships of 3D points within the MEMS coordinate system (MCS). Then, a simple triangulation method is used to recover the 3D information. Furthermore, a simple and efficient MEMS galvanometer calibration method is proposed based on the MCS. It only requires a checkerboard image at a single location to complete the model calibration, greatly simplifying the calibration process. This easily portable 3D measurement method is an innovative calibration method suitable for single-axis MEMS galvanometers and an efficient MEMS galvanometer projector calibration method. It improves measurement accuracy and stability while reducing system cost and complexity. This invention effectively reduces the overall system complexity and implementation cost, making MEMS projectors more widely applicable in various scenarios, especially suitable for measurement tasks requiring miniaturization, low cost, and high frequency.

Description

Technical Field

[0001] This invention relates to a high-precision structured light 3D reconstruction and calibration method, specifically, a high-precision structured light 3D measurement and calibration method applicable to single-axis MEMS projectors. Background Technology

[0002] 3D reconstruction has important applications in reverse engineering, virtual reality, and cultural relic preservation. Fringe projection profilometry has become one of the most commonly used methods due to its high precision and non-contact nature. It typically consists of a camera and a fringe generator. Sine fringes are projected onto the object, the camera captures the deformed fringes, and finally, the 3D shape is calculated using a decoding algorithm.

[0003] The most common stripe generators include Digital Light Processing (DLP) projectors, MEMS-based galvanometer projectors, and Liquid Crystal on Silicon (LCos) projectors. DLP projectors are widely used in 3D measurement due to their high brightness and high precision. However, because they require modulation of millions of micromirrors and rely on lenses for image projection, DLP projectors are expensive and large. A MEMS galvanometer is a tiny, drivable mirror, typically only a few millimeters in diameter. Projectors based on single-axis MEMS galvanometers usually consist of a line laser, a single-axis MEMS galvanometer, and an FPGA controller. The line laser beam illuminates the MEMS galvanometer lens, and the FPGA modulates the intensity of the line laser beam according to different rotation angles of the MEMS lens to achieve single-direction image projection. Compared to DLP projectors, projectors based on single-axis MEMS galvanometers have advantages such as compact structure, high scanning frequency, and low cost. However, since it can only project stripes along one direction, and the MEMS projection system is a non-lens imaging system, the three-dimensional measurement method of the traditional DLP projector based on the pinhole model cannot be applied to the MEMS projector, which greatly limits the application of the MEMS projector to fringe profilometry (FPP).

[0004] The existing technology has the following technical problems:

[0005] 1) Projection direction limitations of MEMS projectors

[0006] Projectors based on single-axis MEMS galvanometers can only project stripe patterns along one direction and cannot perform multi-directional projection. This limitation makes it impossible to directly apply traditional pinhole model-based 3D measurement methods (such as DLP projector methods) to MEMS projectors, resulting in measurement accuracy and efficiency that cannot meet practical requirements.

[0007] 2) Lack of a standard 3D reconstruction method suitable for MEMS projectors

[0008] Currently, most 3D reconstruction methods for MEMS projectors are based on complex mathematical models, such as inverse camera models or multi-plane methods involving multiple calculations. These methods typically require images from multiple viewpoints for 3D reconstruction, resulting in high computational costs and difficulty in real-time processing. Especially in large-scale or high-precision applications, existing methods often fail to effectively meet the dual requirements of accuracy and real-time performance.

[0009] 3) Complex calibration process

[0010] The calibration process of MEMS projectors typically requires multiple calibration locations and sophisticated hardware support, such as high-precision displacement platforms or multiple image captures. Existing calibration methods often rely on complex calculation steps, such as the mapping between phase and height, and interpolation between different planes, making the calibration process time-consuming and susceptible to environmental factors (such as changes in illumination or differences in surface reflection), which limits its application in practical industrial environments.

[0011] 4) Poor system adaptability

[0012] Existing methods mostly rely on specific hardware configurations or complex system adjustments, making them poorly adaptable and difficult to apply to various MEMS projectors or real-world application scenarios. This lack of versatility necessitates frequent debugging and adjustment of system parameters in practical applications, reducing system operability and scalability.

[0013] Based on the above problems, existing technologies urgently need to be improved, especially in simplifying the 3D reconstruction and calibration process and expanding system adaptability. New methods need to be proposed to meet the needs of industrial applications and real-time measurement. Summary of the Invention

[0014] This invention addresses the problems existing in current technologies by providing a high-precision structured light 3D measurement and calibration method suitable for single-axis MEMS projectors, where MEMS stands for Microelectromechanical Systems. A novel mathematical model for MEMS galvanometers is proposed. First, a MEMS coordinate system is established for the MEMS mirror, yielding simple constraint relationships of 3D points within the MEMS coordinate system (MCS). Then, a simple triangulation method is used to recover the 3D information. Furthermore, a simple and efficient MEMS galvanometer calibration method based on MCS is proposed, requiring only a checkerboard image at a single location to complete the model calibration, significantly simplifying the calibration process.

[0015] This invention is achieved through the following technical solution:

[0016] This invention discloses a method for high-precision structured light 3D measurement and calibration suitable for single-axis MEMS projectors, comprising:

[0017] Obtain images of the chessboard calibration board at multiple different locations;

[0018] The coordinates of sub-pixel corner points of the chessboard are extracted from multiple chessboard calibration board images at different locations. The camera is calibrated using Zhang's calibration method, and the camera's internal distortion parameters are calculated.

[0019] A single-axis MEMS galvanometer projector is controlled to project a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and the corresponding fringe image is simultaneously acquired by a camera.

[0020] Based on the phase shift algorithm and phase unwrapping method, the absolute phase distribution map is recovered from the acquired stripe image, and the correspondence between pixels and MEMS mirror rotation angle is established by angular pulse counting.

[0021] Extract the corner coordinates of the checkerboard subpixels in the stripe image, and calculate the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angle.

[0022] By combining the distortion parameters of the camera's internal components, a triangulation model under the MEMS coordinate system (MCS) is established. The coordinates of the sub-pixel corner points of the extracted stripe image and the rotation angle value of the MEMS galvanometer are substituted into the equation to solve and recover the three-dimensional coordinates of the corner points of the checkerboard calibration plate under the camera coordinate system.

[0023] The extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera is calculated using a rotation axis fitting and reference plane (YOZ plane) fitting algorithm.

[0024] As a further improvement, the present invention obtains multiple chessboard calibration board images at different positions, specifically by using at least 12 chessboard images at different positions to improve measurement accuracy and ensure sufficient corner information for calibration and 3D reconstruction.

[0025] As a further improvement, this invention extracts the sub-pixel corner coordinates of the checkerboard from multiple checkerboard calibration board images at different locations, performs camera calibration using the Zhang calibration method, and calculates the camera intrinsic parameters and distortion parameters. Specifically, it uses a sub-pixel precision corner extraction method to process checkerboard images at at least 12 locations, calculates the camera intrinsic parameter matrix and distortion parameters using the Zhang calibration method, obtains the camera intrinsic parameter matrix, and completes the camera calibration to accurately describe the camera's imaging model and provide precise geometric parameters for subsequent 3D reconstruction.

[0026] As a further improvement, the single-axis MEMS galvanometer projector described in this invention projects a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and a camera simultaneously acquires the corresponding fringe images, specifically as follows:

[0027] By using an FPGA to sinusoidally modulate the rotation angle of a MEMS galvanometer, the distorted fringe image captured by the camera is represented as follows:

[0028]

[0029] Where I n Let n be the nth fringe pattern, n∈[0,N); N is the phase shift step number (N≥3), (u,v) are the camera image coordinates, A is the average intensity, B is the intensity modulation, and δ n =2πn / N is the phase shift, Φ=2πθ / T is the absolute phase, T is the fringe period, and θ is the rotation angle of the MEMS lens.

[0030] As a further improvement, this invention recovers the absolute phase distribution map from the acquired stripe image based on the phase shift algorithm and the phase unwrapping method, and establishes the correspondence between pixels and the rotation angle of the MEMS galvanometer by counting angle pulses, specifically:

[0031] The enclosed phase φ is solved as:

[0032]

[0033] Where arctan is the inverse tangent function in the fourth quadrant, with a range of (-π, π].

[0034] Phase unwrapping methods are used to obtain continuous phases and are divided into time phase unwrapping and three-dimensional phase unwrapping. Dual-frequency sub-phase unwrapping and geometric constraints are used to obtain the unwrapped phase Φ.

[0035] As a further improvement, the present invention extracts the corner coordinates of checkerboard sub-pixels in the striped image and calculates the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angles, specifically as follows:

[0036] The relationship between the unwrapping phase Φ and the rotation angle is as follows:

[0037]

[0038] The MEMS lens sends an angle marker pulse every 0.05 degrees of rotation. The relationship between the camera image coordinates (u,v) and the MEMS lens rotation angle θ is obtained by phase shift method.

[0039] As a further improvement, the present invention combines the camera's intrinsic distortion parameters to establish a triangulation model in the MEMS coordinate system (MCS). The coordinates of the sub-pixel corner points of the extracted stripe image's checkerboard pattern and the MEMS galvanometer rotation angle are substituted into the equation to solve and recover the three-dimensional coordinates of the checkerboard calibration plate corner points in the camera coordinate system. Specifically:

[0040] The camera's mathematical model is described using a pinhole model, where a point is p in the world coordinate system (WCS). w (xw ,y w ,z w It is transformed to the camera coordinate system (CCS) and then projected onto the camera image coordinate system (u). c ,v c On the other hand, this process is described as follows:

[0041]

[0042] Where A 3×4 It's the camera's internal parameters, M. 4×4 Here, 's' is the camera's extrinsic parameter, 's' is the scale factor, and 'H' is the product of the camera's extrinsic and intrinsic parameters. In real-world scenarios, due to lens distortion, image distortion correction is necessary. c ,v c This is the result after distortion correction;

[0043] When the laser beam illuminates the center of the MEMS lens, the single-axis MEMS projector is simplified as a plane of light rotating around the MEMS rotation axis. A MEMS coordinate system (MCS) is established for the MEMS projector, with the y-axis coinciding with the MEMS rotation axis, the origin at a point on the axis, and the z-axis perpendicular to the y-axis and set in a plane with a rotation angle of 0°. Based on geometric relationships, the relationship between the three-dimensional point in the MEMS coordinate system and the rotation angle θ is obtained as follows:

[0044]

[0045] By aligning the world coordinate system and the MEMS coordinate system, Eq(4) can be rewritten as:

[0046]

[0047] Where H is the projection matrix from the MCS to the camera image coordinate system, obtained through pre-calibration, (u c ,v c The value to be solved is (x, θ). m ,y m ,z m ,s), then we get:

[0048]

[0049] x m =tanθz m

[0050] By establishing a MEMS coordinate system on the MEMS galvanometer, a simple constraint relationship of three-dimensional points in the MCS is obtained, and finally a formula expression similar to the triangulation method of the DLP system is obtained. The proposed method is easy to port from the DLP system, and pixel-to-pixel acceleration by the graphics processing unit (GPU) is easy.

[0051] As a further improvement, the extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera is calculated using the rotation axis fitting and reference plane (YOZ plane) fitting algorithm described in this invention, specifically as follows:

[0052] To obtain the transformation matrix between the camera coordinate system and the MEMS coordinate system, the MEMS galvanometer projector calibration process is divided into: y-axis and origin calibration, z-axis calibration, phase shift fringe pattern and uniform illumination pattern are projected onto the checkerboard, and phase image is obtained through phase extraction algorithm and phase unwrapping algorithm;

[0053] The three-dimensional coordinates of the center point of the white piece on the chessboard in the camera coordinate system are P = {p0,…,p...} n The PnP method is used to obtain the center points of these white blocks. These points are projected onto the camera image coordinate system, and the corresponding phases Φ = {Φ0,…,Φ1} are obtained by interpolating the phase map. n}, where the phase and the MEMS mirror rotation angle θ={θ0,…,θ n};

[0054] The three-dimensional coordinates P of the checkerboard corner points and the MEMS rotation angle θ have a one-to-one correspondence. To determine the rotation axis of the MEMS lens, the calibration is performed using the angle between the three-dimensional point P and the plane formed by the rotation axis. y (α y ,β y ,γ y ) T Let o(x0, y0, z0) be the direction vector of the rotation axis. T Let p be a point on the axis. If any two points p on the corner points of the chessboard are... j (x j ,y j ,z j ) T and p k (x k ,y k ,z k ) T Their corresponding rotation angles are θ j and θ k p j p k The vector formed by o is l j and l k :

[0055]

[0056] l j and n y The normal vector of the plane in which it lies is n j , l k and n yThe normal vector of the plane in which it lies is n k :

[0057]

[0058] The angle between the two planes is Δθ jk =|θ k -θ j |Result:

[0059]

[0060] Where [α] y ,β y ,γ y [x0, y0, z0] are the parameters to be determined, [x j ,y j ,z j ,x k ,y k ,z k ,θ j ,θ k [The parameters are known. Since the points on the axis are not constrained on the normal to the axis of rotation, let y0 = 0, n] y There are no constraints on the vector magnitude, let β y =1; here due to Δθ jk It is a small value, cos(Δθ) jk ) in Δθ jk The slope is low near 0, so the optimization function is constructed and the parameters are solved using the least squares method:

[0061]

[0062] Every two points on the chessboard result in a set of equations. At least four points are needed to complete the set of equations. Therefore, using all the corner points on the chessboard for least squares solution is sufficient to fit a high-precision rotation axis.

[0063] The plane with a MEMS rotation of 0 is defined as the yoz plane, and the three-dimensional point p... j around n y Rotate counterclockwise θ j Get p′ j This ensures that all these points lie on the yoz plane, which can be obtained using the Rodriguez formula:

[0064]

[0065] I is a 3×3 identity matrix, obtained through p' j By fitting the yoz plane with the y-axis, two constraints are obtained:

[0066]

[0067] Where n x =(α x ,β x ,γ x ) T Let α be the x-axis direction of the MEMS coordinate system. x =1 to eliminate the degree of freedom of the vector length, resulting in:

[0068]

[0069] Simultaneously, a least squares method is constructed for plane fitting.

[0070]

[0071] The z-axis yields:

[0072] n z =n x ×n y

[0073] The transformation relationship between the MEMS coordinate system and the camera coordinate system is obtained:

[0074]

[0075] The present invention has the following beneficial effects:

[0076] 1) A portable three-dimensional measurement method

[0077] This invention combines camera-intrinsic distortion parameters to establish a triangulation model in a MEMS coordinate system (MCS). Substituting the sub-pixel corner coordinates of the extracted stripe image with the MEMS mirror rotation angle values ​​into the equation, the three-dimensional coordinates of the checkerboard calibration board corners in the camera coordinate system are recovered. By establishing a MEMS coordinate system for the MEMS mirror, a simple constraint relationship of the three-dimensional points in the MCS is obtained, ultimately leading to a formulaic expression similar to the triangulation method in a DLP system. The proposed method is easily ported from DLP systems, and pixel-to-pixel processing facilitates GPU acceleration.

[0078] 2) An innovative calibration method applicable to uniaxial MEMS galvanometers

[0079] This invention utilizes the recovered 3D coordinates of the corner points of the checkerboard calibration board in the camera coordinate system and the corresponding MEMS galvanometer rotation angle values. By fitting the rotation axis to the reference plane, the extrinsic parameter matrix of the projector relative to the camera is calculated. An innovative calibration method suitable for 3D measurement of single-axis MEMS galvanometers is proposed, establishing a MEMS coordinate system (MCS) and proposing a simplified constraint relationship, enabling the 3D point recovery process to be achieved through a simple triangulation method. Traditional 3D reconstruction methods based on DLP projectors cannot be directly applied to MEMS galvanometers. However, this invention overcomes this technical bottleneck through a simple mathematical model and triangular shape recovery method, enhancing the application potential of MEMS projectors in 3D measurement.

[0080] 3) Efficient MEMS galvanometer projector calibration method

[0081] This invention controls a single-axis MEMS galvanometer projector to project a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, while a camera simultaneously acquires the corresponding fringe images. Calibration can be completed with only a checkerboard image from a single location, greatly simplifying the traditional calibration process. Traditional MEMS projector calibration methods typically require multiple image captures and rely on complex hardware support. The single-location calibration method of this invention not only reduces the number of images required for calibration but also reduces reliance on high-precision displacement platforms, making the calibration process more efficient and convenient.

[0082] 4) Improved measurement accuracy and stability

[0083] This invention extracts the sub-pixel corner coordinates of checkerboard grids in a striped image. Based on a phase-shifting algorithm and a phase unwrapping method, it recovers the absolute phase distribution map from the acquired striped image and establishes the correspondence between pixels and MEMS mirror rotation angles through angle pulse counting. By combining sub-pixel extraction of checkerboard grid corners with phase-shifting technology, compared with existing phase-angle methods, it exhibits lower measurement errors and higher stability, effectively reducing errors caused by environmental interference (such as uneven reflection and illumination changes), thereby achieving higher-precision 3D reconstruction.

[0084] 5) Reduced system cost and complexity

[0085] This invention relates to a 3D measurement system based on a single-axis MEMS galvanometer. It employs a more compact hardware configuration and is less expensive than traditional DLP projector systems. By simplifying the calibration process and 3D reconstruction calculations, this invention effectively reduces the overall system complexity and implementation cost, making MEMS projectors more widely applicable in various scenarios, especially suitable for measurement tasks requiring miniaturization, low cost, and high frequency. Attached Figure Description

[0086] Figure 1This is a schematic diagram of a mathematical model of a three-dimensional measurement system based on a single-axis MEMS galvanometer. In the diagram, 1 is the MEMS galvanometer and 2 is the camera.

[0087] Figure 2 This is a schematic diagram of the calibration principle of a projection system based on a single-axis MEMS galvanometer.

[0088] Figure 3 This is a flowchart of the calibration and reconstruction process for a three-dimensional measurement system based on a single-axis MEMS galvanometer. Detailed Implementation

[0089] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific implementation examples:

[0090] The purpose of this invention is to address the limitation of existing DLP projector measurement methods in applying to projector measurements using a single-axis MEMS galvanometer mirror 1. This invention proposes a three-dimensional measurement method based on a single-axis MEMS galvanometer mirror 1. The device implementing this invention includes a laser (with a built-in Powell prism), a camera 2, a single-axis MEMS galvanometer mirror 1, and a field-programmable gate array (FPGA) controller. The device employs a triangulation layout, with the laser and Powell prism coaxial. The long side of the emitted linear laser is parallel to the rotation axis (Y-axis) of the MEMS galvanometer mirror 1, precisely illuminating the center of the mirror surface. The camera and MEMS galvanometer mirror 1 are spatially separated, with their optical centers forming a baseline of a certain length. The camera's optical axis forms a certain angle with the initial reflected light path of the galvanometer mirror, thus forming an effective triangulation structure to ensure measurement sensitivity in the depth direction. During operation, the FPGA controller controls the MEMS galvanometer mirror 1 to periodically scan and synchronously modulates the intensity of the laser, thereby projecting a phase-shifted fringe pattern onto the object surface. The camera, triggered by the FPGA, synchronously acquires the deformed fringe pattern. The computing processing device uses phase-shifting algorithms and phase unwrapping techniques to calculate the absolute phase from the image, and then combines the phase-angle mapping relationship determined by the angle pulse of the MEMS galvanometer to finally reconstruct the three-dimensional shape of the object through a triangulation model.

[0091] Figure 1 This is a schematic diagram of the mathematical model of the three-dimensional measurement system of the present invention;

[0092] The specific implementation method of the present invention is as follows:

[0093] Obtain images of the checkerboard calibration board from multiple locations. Place the high-precision checkerboard calibration board within the system's measurement field of view, and change the spatial orientation of the calibration board to ensure it covers the entire field of view of the camera and has sufficient tilt and rotation angle variations. Use the camera to acquire and store clear checkerboard images from at least 12 different orientations. The purpose of this step is to provide sufficient sample data for subsequent camera parameter calibration.

[0094] Subpixel corner coordinates of the checkerboard were extracted from multiple images of the checkerboard calibration board at different locations. Camera calibration was performed using the Zhang calibration method, and camera intrinsic distortion parameters were calculated. Corner detection functions from computer vision libraries (such as OpenCV) were used to process all acquired checkerboard images, automatically extracting subpixel-level precision coordinates of the checkerboard corners. Subsequently, the widely used Zhang calibration method was employed, using these corner coordinate data and the known physical dimensions of the checkerboard to perform nonlinear optimization calculations. Finally, the camera's intrinsic parameter matrix and lens distortion coefficients were solved, and these parameters will be used for subsequent image correction and 3D reconstruction.

[0095] A single-axis MEMS galvanometer 1 projector projects a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, while a camera simultaneously acquires the corresponding fringe images. The checkerboard calibration plate is fixed at a specific position within the measurement field of view. The single-axis MEMS galvanometer 1 projector, driven by an FPGA controller, sequentially projects a set (preferably six steps) of sinusoidal phase-shift fringe patterns with different frequencies (e.g., 32, 256, and 1024 pixel periods) onto the calibration plate. The distorted fringe image captured by the camera is represented as follows:

[0096]

[0097] Where I n Let n be the nth fringe pattern, n∈[0,N); N is the phase shift step number (N≥3), (u,v) are the camera image coordinates, A is the average intensity, B is the intensity modulation, and δ n =2πn / N is the phase shift, Φ=2πθ / T is the absolute phase, T is the fringe period, and θ is the rotation angle of the MEMS lens. Simultaneously, the FPGA sends a precise synchronization trigger signal to the camera, ensuring that the camera synchronously acquires data when each fringe pattern is stably projected, thereby obtaining a series of deformed fringe images modulated by the checkerboard surface.

[0098] Based on the phase-shifting algorithm and phase unwrapping method, the absolute phase distribution map is recovered from the acquired fringe image, and the correspondence between pixels and the rotation angle of MEMS mirror 1 is established by counting angle pulses. For the acquired phase-shifted fringe image sequence, a standard multi-step phase-shifting algorithm (such as the six-step phase-shifting method) is used to calculate the wrapped phase pixel by pixel. Combined with phase unwrapping algorithms such as the multi-frequency heterodyne principle, the wrapped phase φ is solved as:

[0099]

[0100] Where arctan is the inverse tangent function in the fourth quadrant, with a range of (-π, π].

[0101] Phase unwrapping methods are used to obtain continuous phases and are divided into time phase unwrapping and three-dimensional phase unwrapping. Dual-frequency sub-phase unwrapping and geometric constraints are used to obtain the unwrapped phase Φ.

[0102] The wrapped phase is solved into an absolute phase value Φ, thus obtaining the absolute phase distribution map of the entire field. The absolute phase value Φ and the physical rotation angle θ of MEMS mirror 1 have the following relationship:

[0103]

[0104] Where Δθ is the angular pulse interval and T is the stripe period; based on the number of angular pulses recorded by the FPGA, a one-to-one correspondence is established between the absolute phase value Φ of each pixel on the image and the physical rotation angle θ of the MEMS galvanometer 1.

[0105] The sub-pixel corner coordinates of the checkerboard pattern in the striped image are extracted. The corresponding MEMS mirror 1 rotation angle value is calculated using the absolute phase distribution map and the correspondence between pixels and the rotation angle of MEMS mirror 1. Checkerboard corner detection is performed on a previously acquired frame of the striped image, and high-precision sub-pixel corner coordinates are extracted again. Then, using the obtained absolute phase distribution map, the absolute phase value Φ corresponding to each corner coordinate position is found through bilinear interpolation and other methods. The relationship between the absolute phase value Φ and the rotation angle is as follows:

[0106]

[0107] The MEMS lens sends an angle marker pulse every 0.05 degrees of rotation. The relationship between the camera image coordinates (u,v) and the rotation angle θ of the MEMS galvanometer 1 is obtained by phase shift method.

[0108] By combining the distortion parameters of the camera's internal components, a triangulation model under the MEMS coordinate system (MCS) is established. The subpixel corner coordinates of the extracted stripe image and the rotation angle value of MEMS galvanometer 1 are substituted into the equation to solve and recover the three-dimensional coordinates of the corner points of the checkerboard calibration plate under the camera coordinate system.

[0109] The pinhole model is widely used to describe camera mathematical models, where a point is p in the world coordinate system (WCS). w (x w ,y w ,z w It is transformed to the camera coordinate system (CCS) and then projected onto the camera image coordinate system (u). c ,v c On the other hand, this process is described as follows:

[0110]

[0111] Where A3×4 It's the camera's internal parameters, M. 4×4 Here, is the camera's extrinsic parameter, s is the scale factor, and H is the product of the camera's extrinsic and intrinsic parameters. In real-world scenarios, due to lens distortion, image distortion correction is necessary. The (u) in this invention... c ,v c This is considered to be the result after distortion correction.

[0112] When the laser beam illuminates the center of MEMS galvanometer 1, the single-axis MEMS galvanometer 1 projector is simplified as a plane of light rotating around the MEMS axis. This invention establishes a MEMS coordinate system (MCS) for the MEMS galvanometer 1 projector as follows: Figure 1 As shown, the y-axis coincides with the MEMS rotation axis, with the origin at a point on the axis. The z-axis is perpendicular to the y-axis and is set in a plane with a rotation angle of 0°. Based on geometric relationships, the relationship between the three-dimensional point in the MEMS coordinate system and the rotation angle θ is obtained as follows:

[0113]

[0114] To simplify, the world coordinate system and the MEMS coordinate system are aligned, and the above pinhole model formula is rewritten as follows:

[0115]

[0116] Where H is the projection matrix from the MCS to the camera image coordinate system, obtained through pre-calibration, (u c ,v c The relationship between the absolute phase value Φ and the physical rotation angle θ of MEMS mirror 1 is obtained from the above formula. The value to be solved is (x m ,y m ,z m By combining the relationships between the three-dimensional point in the MEMS coordinate system and the rotation angle θ with the simplified pinhole model formula, we obtain:

[0117]

[0118] x m =tanθz m

[0119] By substituting the pixel coordinates and their corresponding rotation angles θ into the system of equations, the three-dimensional coordinates of each checkerboard corner point in the camera coordinate system (CCS) can be obtained.

[0120] By establishing a MEMS coordinate system on MEMS galvanometer 1, a simple constraint relationship of 3D points in the MCS is obtained, ultimately leading to a formulaic expression similar to the triangulation method in the DLP system. The proposed method can be easily ported from the DLP system, and because it is pixel-to-pixel, it is easily accelerated using GPUs.

[0121] In addition, to address the issue that existing DLP projector calibration methods cannot be applied to the calibration of single-axis MEMS galvanometer 1 projectors, a calibration method based on single-axis MEMS galvanometer 1 projectors is proposed. By using rotation axis fitting and reference plane (YOZ plane) fitting algorithms, the extrinsic parameter matrix of the MEMS galvanometer 1 projector relative to the camera is calculated.

[0122] Figure 2 This is a schematic diagram of the calibration principle of the MEMS projection system of the present invention;

[0123] The specific implementation method of the present invention is as follows:

[0124] The three-dimensional coordinates of the center point of the white piece on the chessboard in the camera coordinate system are P = {p0,…,p...} n The PnP method is used to obtain the center points of these white blocks. These points are projected onto the camera image coordinate system, and the corresponding phases Φ = {Φ0,…,Φ1} are obtained by interpolating the phase map. n}, where the phase and the rotation angle θ of MEMS mirror 1 are {θ0,…,θ} n},like Figure 2 As shown;

[0125] The y-axis and origin are calibrated. The three-dimensional coordinates P of the checkerboard corner points and the MEMS rotation angle θ have a one-to-one correspondence. To solve for the rotation axis of the MEMS lens, the calibration is completed using the angle between the three-dimensional point P and the plane formed by the rotation axis. y (α y ,β y ,γ y ) T Let o(x0, y0, z0) be the direction vector of the rotation axis. T Let p be any two points on the corner of the chessboard. j (x j ,y j ,z j ) T and p k (x k ,y k ,z k ) T Their corresponding rotation angles are θ j and θ k p j p k The vector formed by o is l j and l k :

[0126]

[0127] l j and n yThe normal vector of the plane in which it lies is n j , l k and n y The normal vector of the plane in which it lies is n k :

[0128]

[0129] The angle between the two planes is Δθ jk =|θ k -θ j |Result:

[0130]

[0131] Where [α] y ,β y ,γ y [x0, y0, z0] are the parameters to be determined, [x j ,y j ,z j ,x k ,y k ,z k ,θ j ,θ k [The parameters are known. Since the points on the axis are not constrained on the normal to the axis of rotation, let y0 = 0, n] y There are no constraints on the vector magnitude, let β y =1. Here, due to Δθ jk It is a small value, cos(Δθ) jk ) in Δθ jk The slope is low near 0, so the optimization function is constructed and the parameters are solved using the least squares method:

[0132]

[0133] Every two points on the chessboard result in a set of equations, so at least four points are needed to complete the set of equations. Therefore, using all the corner points of the chessboard for least squares solution is sufficient to fit a high-precision rotation axis.

[0134] The z-axis and x-axis are calibrated, and the plane with a MEMS rotation angle of 0 is defined as the yoz plane. The three-dimensional point p... j around n y Rotate counterclockwise θ j Get p' j This ensures that all these points lie on the yoz plane, and they are obtained using the Rodriguez formula:

[0135]

[0136] I is a 3×3 identity matrix, obtained through p' jBy fitting the yoz plane with the y-axis, two constraints are obtained:

[0137]

[0138] Where n x =(α x ,β x ,γ x ) T Let α be the x-axis direction of the MEMS coordinate system. x =1 to eliminate the degree of freedom of the vector length, resulting in:

[0139]

[0140] Simultaneously, a least squares method is constructed for plane fitting:

[0141]

[0142] The z-axis yields:

[0143] n z =n x ×n y

[0144] System parameters can be obtained from the transformation matrix of the MCS in the CCS, as shown below:

[0145]

[0146] in, is n T The normalized vector H describes the projection matrix from 3D points in the MCS to camera pixel coordinates.

[0147] Those skilled in the art will understand that the above description is merely a single example of the invention and is not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector, characterized in that, include: Obtain images of the chessboard calibration board at multiple different locations; The coordinates of sub-pixel corner points of the chessboard are extracted from multiple chessboard calibration board images at different locations. The camera is calibrated using Zhang's calibration method, and the camera's internal distortion parameters are calculated. A single-axis MEMS galvanometer projector is controlled to project a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and the corresponding fringe image is simultaneously acquired by a camera. Based on the phase shift algorithm and phase unwrapping method, the absolute phase distribution map is recovered from the acquired stripe image, and the correspondence between pixels and MEMS mirror rotation angle is established by angular pulse counting. Extract the corner coordinates of the checkerboard subpixels in the stripe image, and calculate the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angle. By combining the distortion parameters of the camera, a triangulation model under the MEMS coordinate system MCS is established. The coordinates of the sub-pixel corner points of the extracted stripe image and the rotation angle value of the MEMS galvanometer are substituted into the equation to solve and recover the three-dimensional coordinates of the corner points of the checkerboard calibration plate under the camera coordinate system. The extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera is calculated using a rotation axis fitting and reference plane YOZ plane fitting algorithm. The aforementioned method combines camera-integrated distortion parameters to establish a triangulation model in the MEMS coordinate system (MCS). The extracted checkerboard subpixel corner coordinates of the striped image and the MEMS galvanometer rotation angle are substituted into the equation to recover the three-dimensional coordinates of the checkerboard calibration plate corners in the camera coordinate system. Specifically: The camera's mathematical model is described using a pinhole model, where a point is represented in the World Coordinate System (WCS). It is transformed to the camera coordinate system (CCS) and then projected onto the camera image coordinate system. The above process is described as follows: （1） in It's the camera's internal parameters. It is the camera's external parameters. As a scale factor, This is the product of the camera's extrinsic and intrinsic parameters. In real-world scenarios, due to lens distortion, image distortion correction is necessary. This is the result after distortion correction; When laser light shines on the center of the MEMS lens, the single-axis MEMS projector is simplified as a plane of light rotating around the MEMS rotation axis. A MEMS coordinate system (MCS) is established for the MEMS projector, with the y-axis coinciding with the MEMS rotation axis and the origin at a point on the axis. The z-axis is perpendicular to the y-axis and is set in a plane with a rotation angle of 0°. According to geometric relationships, the three-dimensional point lies in the MEMS coordinate system and the rotation angle... The relationship is obtained as follows: ； By aligning the world coordinate system and the MEMS coordinate system, equation (1) can be rewritten as: ； in The projection matrix from the MCS to the camera image coordinate system is obtained through pre-calibration. The value that needs to be solved is obtained. Then we get: ; ； By establishing a MEMS coordinate system on the MEMS galvanometer, a simple constraint relationship of three-dimensional points in the MCS is obtained, and finally a formula expression similar to the triangulation method of the DLP system is obtained. The proposed method is easy to port from the DLP system, and pixel-to-pixel acceleration by the graphics processing unit (GPU) is easy.

2. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, characterized in that, The aforementioned method of obtaining multiple chessboard calibration board images at different locations specifically involves using at least 12 chessboard images at different locations to improve measurement accuracy and ensure sufficient corner information for calibration and 3D reconstruction.

3. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, characterized in that, The process involves extracting sub-pixel corner coordinates of the checkerboard from multiple different locations of the obtained checkerboard calibration board images, performing camera calibration using Zhang's calibration method, and calculating camera intrinsic parameters and distortion parameters. Specifically, this involves processing checkerboard images at least 12 locations using a sub-pixel precision corner extraction method, calculating the camera intrinsic parameter matrix and distortion parameters using Zhang's calibration method, obtaining the camera intrinsic parameter matrix, and completing the camera calibration to accurately describe the camera's imaging model and provide precise geometric parameters for subsequent 3D reconstruction.

4. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, 2, or 3, characterized in that, The control single-axis MEMS galvanometer projector projects a six-step multi-frequency phase-shift fringe pattern onto a checkerboard calibration plate, and the camera simultaneously acquires the corresponding fringe images, specifically as follows: By using an FPGA to sinusoidally modulate the rotation angle of a MEMS galvanometer, the distorted fringe image captured by the camera is represented as follows: ； in For the nth stripe pattern, ; Number of phase shift steps , Here are the camera image coordinates, where A represents the average intensity and B represents the intensity modulation. For phase shift, The absolute phase is T, and the fringe period is T. The rotation angle of the MEMS lens.

5. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 4, characterized in that, The method described above, which uses a phase-shifting algorithm and a phase unwrapping method to recover the absolute phase distribution map from the acquired stripe image and establishes the correspondence between pixels and the rotation angle of the MEMS galvanometer through angle pulse counting, specifically involves: Wrap phase Interpreted as: ； in It is the inverse tangent function in the fourth quadrant, with a range of... ; Phase unwrapping methods are used to obtain continuous phase, and are divided into time phase unwrapping and three-dimensional phase unwrapping. Dual-frequency sub-phase unwrapping and geometric constraints are used to obtain the unwrapped phase. .

6. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 5, characterized in that, The extraction of checkerboard sub-pixel corner coordinates in the striped image, and the calculation of the corresponding MEMS mirror rotation angle value using the absolute phase distribution map and the correspondence between pixels and MEMS mirror rotation angles, specifically involves: Unwrapping phase The relationship between the rotation angle and the rotation angle is as follows: ； The MEMS lens sends an angular marker pulse every 0.05 degrees of rotation, and the camera image coordinates are obtained through the phase-shift method. and MEMS lens corner The relationship.

7. The method for high-precision structured light three-dimensional measurement and calibration of a single-axis MEMS projector according to claim 1, 2, 3, 5, or 6, characterized in that, The aforementioned algorithm, which uses rotation axis fitting and reference plane YOZ plane fitting, calculates the extrinsic parameter matrix of the MEMS galvanometer projector relative to the camera, specifically as follows: To obtain the transformation matrix between the camera coordinate system and the MEMS coordinate system, the MEMS galvanometer projector calibration process is divided into: y-axis and origin calibration, z-axis calibration, phase shift fringe pattern and uniform illumination pattern are projected onto the checkerboard, and phase image is obtained through phase extraction algorithm and phase unwrapping algorithm; The three-dimensional coordinates of the center point of the white piece on the chessboard in the camera coordinate system The center points of these white patches are obtained by projecting them onto the camera image coordinate system using the PnP method and interpolating the phase map to obtain the corresponding phase. The phase and the rotation angle of the MEMS galvanometer ; 3D coordinates of chessboard corner points and MEMS rotation angle To establish a one-to-one correspondence, and in order to solve for the rotation axis of the MEMS lens, three-dimensional points are used. The calibration is performed by the angle between the plane formed by the axis of rotation and the plane itself. Let be the direction vector of the rotation axis. Let A be a point on the axis. If any two points on the corner of the chessboard are... and Their corresponding rotation angles are and , , and The resulting vector is and : ； and The normal vector of the plane is , and The normal vector of the plane is : ; The angle between the two planes is get: ； in For the parameters to be determined, Given the parameters, since the points on the axis are not constrained on the normal to the axis of rotation, let , There are no constraints on the vector magnitude, let ; here because It is a small value. exist The slope is low in the vicinity, so the optimization function is constructed and the parameters are solved using the least squares method: ； Every two points on the chessboard result in a set of equations. At least four points are needed to complete the set of equations. Therefore, using all the corner points on the chessboard for least squares solution is sufficient to fit a high-precision rotation axis. The plane with a MEMS rotation of 0 is defined as the yoz plane, which is used to plot three-dimensional points. Around Rotate counterclockwise get This ensures that all these points lie on the yoz plane, which can be obtained using the Rodriguez formula: for identity matrix, through By fitting the yoz plane with the y-axis, two constraints are obtained: ； in Let x be the x-axis direction of the MEMS coordinate system. To eliminate the degree of freedom of the vector length, we get: ； Simultaneously, a least squares method is constructed for plane fitting. ； The z-axis yields: ； The transformation relationship between the MEMS coordinate system and the camera coordinate system is obtained: 。

Images