A real-time panoramic three-dimensional reconstruction method based on a virtual stereo unwrapping method

Abstract

This invention discloses a real-time panoramic 3D reconstruction method based on a virtual stereo unwrapping approach. The system comprises only one camera, a projector, and a turntable. The system projects and captures three-step phase-shift sinusoidal fringes at each angle to obtain the wrapped phase. Then, it utilizes a virtual camera-projector system formed at adjacent rotation angles to assist in unwrapping, ultimately synthesizing a 360-degree 3D model. The advantages of this method are that it uses only a single camera, reducing system cost compared to multi-camera SPU systems. Furthermore, the method provides more viewing angles for unwrapping during the virtual camera-projector system-assisted unwrapping process, significantly improving the stability of unwrapping.

Description

Technical Field

[0001] This invention relates to the field of optical measurement, specifically to a real-time panoramic 3D reconstruction method based on a virtual stereo unwrapping method. Background Technology

[0002] Optical three-dimensional (3D) topography reconstruction technology has been widely used in fields such as intelligent manufacturing and medicine [1-3]. Among them, fringe projection profilometry (FPP) [4-5] plays an important role because of its advantages such as high precision and full-field reconstruction. However, the three-dimensional reconstruction of a single-direction surface cannot obtain all the information of the target due to problems such as shadow occlusion, and gradually cannot meet the needs of reverse modeling, industrial inspection and other fields [6]. Fast, high-precision, 360-degree three-dimensional reconstruction is increasingly attracting the attention of academia and industry [7].

[0003] To achieve 360-degree three-dimensional reconstruction, it is necessary to collect point clouds of objects from different perspectives and stitch them together. The first method is the multi-angle stereo digital correlation (DIC) system. J.-J. Orteu et al. [8] first used a 4-camera system to measure the panoramic strain of cylindrical samples using multi-view DIC technology, but the system still has a limited field of view. DANA SOLAV et al. [9] used a 12-camera system to measure the full-field deformation and strain of the human lower limbs. However, the multi-camera system increases the equipment cost and the system calibration is difficult. Another method is to register the point clouds collected by a single measurement system from different perspectives. The point cloud registration method can be divided into algorithm registration and instrument-assisted registration. The point cloud registration technology relies on stitching together the repeated parts of the point cloud from different perspectives. Szymon et al.

[10] proposed a real-time 360-degree three-dimensional model acquisition technology based on iterative nearest neighbor (ICP), but this method skips coarse matching, resulting in low registration accuracy [7]. In 2019, Jiaming Qian et al. [7] proposed an improved registration strategy from coarse to fine, which greatly improved the accuracy while ensuring the registration speed. However, the point cloud registration technology is still very time-consuming and difficult to apply to real-time matching scenarios. Moreover, this method generally requires manual movement of the object to be measured, which is difficult to apply to automatic three-dimensional measurement scenarios. Instrument assistance includes mirror assistance and turntable assistance. Mirror assistance is to capture the target at three angles through the mirror to achieve panoramic three-dimensional measurement

[11] . However, the number of viewpoints provided by this method is limited and cannot meet the 360-degree reconstruction requirements of complex objects. Turntable assistance can obtain point clouds at any angle by rotating the turntable. The point clouds obtained at each angle only need to be rotated at the corresponding angle in the rotation axis coordinate system to complete the point cloud registration. For complex objects, a smaller rotation step can be selected to obtain a precise panoramic model. Meiling Dai et al.

[12] proposed a method to achieve automatic calibration of the rotation axis using a calibration plate. Xiaoqi Cai

[13] used an auxiliary camera to improve the calibration accuracy of the rotation axis. Xiaoli Liu et al.[6] used multiple 3D cameras and an automatically controlled turntable to achieve panoramic reconstruction of the model. However, since each sensor needs to be reconstructed in time, and the turntable needs to stop and wait for the reconstruction to finish at each angle, the panoramic reconstruction time is relatively long. Since FPP requires multiple frame projections, the reconstruction will be destroyed during the turntable rotation, which seriously limits the efficiency of panoramic reconstruction. In recent years, with the development of high-speed projectors, real-time three-dimensional reconstruction based on FPP[2,14,15] has begun to develop.Jiaming Qian[7] used a four-camera projector system to realize a real-time three-dimensional panoramic reconstruction based on a turntable. It only requires projecting single-cycle phase shift fringes to obtain the wrapped phase. The four cameras work together with an adaptive depth constraint (ADC) strategy to assist in unwrapping and realize real-time three-dimensional reconstruction. However, the four high-speed cameras based on the stereo phase unwrapping (SPU) method will increase the cost and make the system calibration difficult.

[0004] In summary, current multi-view DIC methods suffer from system complexity and high cost. Mirror-based 360-degree reconstruction methods have issues such as providing a limited number of viewpoints and a small field of view (FOV). Four-camera projector SPU systems also suffer from system complexity and high implementation costs. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention aims to provide a real-time panoramic 3D reconstruction method based on a virtual stereo unwrapping (VSPU) approach. This paper proposes a novel real-time 360-degree 3D reconstruction method based on VSPU. The system comprises only one camera, a projector, and a turntable. The system projects and captures three-step phase-shift sinusoidal fringes at each angle to obtain the wrapped phase. Then, it utilizes a virtual camera-projector system formed at adjacent rotation angles to assist in unwrapping, ultimately synthesizing a 360-degree 3D model. The advantages of this proposed method are that it uses only a single camera, reducing system cost compared to multi-camera VSPU systems. Furthermore, the method provides more viewing angles for unwrapping during the VSPU process, significantly improving the stability of the unwrapping process.

[0006] To achieve the above objectives, the technical method employed in this invention is as follows:

[0007] This invention discloses a real-time panoramic 3D reconstruction method based on a virtual stereo unwrapping method, comprising: constructing a 3D reconstruction system, the system comprising a computer, a projector, a camera, a turntable controller, a turntable connected to the turntable controller, and the projector and camera being connected to each other.

[0008] The projector, camera, and turntable are calibrated to obtain the internal and external parameters of the camera and projector, and the position of the turntable's rotation axis.

[0009] A rotation command is sent to the turntable controller, and the turntable rotates at a constant speed;

[0010] Perform a single virtual stereo unpacking (VSPU);

[0011] After a single virtual stereo unpacking process (VSPU) ends, there is a delay Δt. reg Proceed to the next VSPU process until the 360-degree scan is complete;

[0012] The specific process of a single virtual 3D unpacking unit (VSPU) includes:

[0013] The projector projects and captures 2m+1 (m≥1) multi-step phase-shifting images, with a delay of Δt for each step. vspu Each time a projection occurs, the camera captures a distorted phase-shifted image using hard triggering.

[0014] Based on the multi-step phase shift diagrams at different angles, calculate the wrap-around phase diagrams at positions {-m, -m+1, ..., m-1, m} respectively;

[0015] Based on the wrap-around phase map at position 0, a finite number of possible k values ​​are obtained through constant geometric constraints (CDC), and point clouds with different k values ​​are obtained through triangulation.

[0016] The obtained point cloud is projected onto the virtual camera target surface and the virtual projector target surface at the positions {-m, -m+1, -1, 1, ..., m-1, m}, respectively, to obtain the wrapping phase map of these points on the virtual camera and the virtual projector. The absolute value of the difference is used to obtain the wrapping phase error map at the positions {-m, -m+1, -1, 1, ..., m-1, m}.

[0017] By summing the package phase error maps, we obtain package phase error maps under different k values. We then take the k value with the smallest error to obtain the final unpacking k value map.

[0018] Based on the unpacked k-value map, point clouds at the 0 position angle are obtained through triangulation, and these point clouds are registered into the 360-degree panoramic model.

[0019] As a further improvement, this invention calculates the wrap-around phase map at positions {-m, -m+1, ..., m-1, m} based on the multi-step phase shift map at different angles, specifically as follows:

[0020] A projector projects phase-shifted fringes onto an object, and a camera captures the distorted fringe pattern. The fringe pattern captured by the camera can be represented as:

[0021]

[0022] I n The nth fringe image is projected, where n ∈ [0, N); N represents the number of phase shift steps. Generally, the larger the number of steps, the higher the measurement accuracy. This invention uses a three-step phase shift method. Represents the pixel coordinates of the i-th camera image (hereinafter, for simplicity, it will be referred to as...). (Indicates); A represents average intensity, B represents amplitude; φ represents phase; The phase shift, φ, can be obtained through a phase shift algorithm:

[0023]

[0024] atan2 is a signed arctangent operation that expands the wrapping phase φ to (-π, π) by changing the signs of the numerator and denominator. For a three-step phase shift, the wrapping phase φ is:

[0025]

[0026] atan2 is a signed arctangent operation that expands the wrapping phase φ to (-π, π) by the signs of the numerator and denominator.

[0027] As a further improvement, the present invention obtains a finite number of possible k values ​​based on the wrap-around phase map at position 0 through constant geometric constraints (CDC), specifically as follows:

[0028] In CDC, the system can set an approximate range Z based on the size of the object. min and Z max , is represented as:

[0029] Z min ≤Z w (o c K c )≤Z max ,

[0030] For 360° measurement systems, CDC excludes some k values ​​that are out of range. The depth range of the measured object will vary greatly at different angles, and phase ambiguity cannot be eliminated.

[0031] As a further improvement, the present invention projects the obtained point cloud onto the virtual camera target surface and the virtual projector target surface at positions {-m, -m+1, -1, 1, ..., m-1, m}, respectively, to obtain the wrapping phase map of these points on the virtual camera and the virtual projector. The absolute value of the difference is used to obtain the wrapping phase error map at positions {-m, -m+1, -1, 1, ..., m-1, m}. Specifically:

[0032] Using the object under test as a reference frame, a virtual camera-projector system is formed by rotating around the turntable axis. The phase difference θ between adjacent camera-projector systems is θ = v·Δt, where v represents the turntable rotation speed and Δt represents the interval between capturing phase-shifted images. Using the turntable rotation axis as the z-axis of the world coordinate system, the relationship between the camera and projector at different angles can be expressed as:

[0033]

[0034] Unlike spatial 3D unwrapping systems, the projector in the virtual system is also rotated. To use virtual camera-projector pairs at adjacent angles for assisted unwrapping, the possible points obtained by the main camera are projected onto adjacent virtual projectors and virtual cameras, respectively, as shown below:

[0035]

[0036]

[0037] in These represent the projections of the k-th possible point onto camera c. i and projector p i Coordinates of the target surface Let A represent the transformation matrices of the i-th camera and the projector in the world coordinate system, respectively. c A p These represent the intrinsic parameter matrices of the camera and projector, respectively. Let these represent the extrinsic parameter matrices of the i-th camera and the projector, respectively. This represents the phase of the k-th possible point of the main camera projected onto the camera target surface in the world coordinate system. The wrapping phase image obtained by Eq(3) at the corresponding rotation angle is used to... Two-dimensional interpolation reveals that phase compensation is required at the transition position. The phase value projected onto the projector target surface can be expressed as:

[0038]

[0039] in express The horizontal axis is the coordinate of the camera-projector limit constraint, and the U-axis is the coordinate of the camera-projector limit constraint. The range is [-π, π]. The possible points are filtered using the wrap-around phase error between the virtual projector and camera projections as the criterion. The error is expressed as:

[0040]

[0041]

[0042] in Let the package phase difference of the k-th possible point under the i-th virtual camera be denoted as . This indicates the corresponding discrimination error, and the phase shift is actively performed to ensure that the phase deviation is within [0, π].

[0043] As a further improvement, the present invention involves summing the package phase error maps to obtain package phase error maps under different k values, and selecting the k value with the smallest error to obtain the final unwrapped k-value map, specifically as follows:

[0044] The final chosen value of k is the minimum sum of the deviations of all virtual systems.

[0045]

[0046] k result =minIndex(Err(k)).

[0047] Where k result Here, k is the value obtained by the VSPU method, Err(k) is the phase error of different k values, minIndex(·) is the index function of the minimum value, m is the number of adjacent virtual systems selected, and the number of systems to assist in unpacking is 2m.

[0048] As a further improvement, the single VSPU process described in this invention includes five projections to capture multi-step phase-shift maps, with a delay of θ for each step. vspu After the VSPU process at a single angle is completed, there is a delay of θ. reg The next VSPU process will be performed as follows:

[0049] First, after the turntable starts, it rotates at a constant angular acceleration α until it reaches a uniform speed ω. turn The settling time is Δt = ω turn / α, after which the system begins projection imaging, each time passing t... reg A set of stripe patterns for the VSPU is captured at regular intervals, with the time interval determined by a set registration angle interval Δθ. reg The obtained Δt reg =Δθ reg / ω turn .

[0050] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0051] 1. Simple setup and low cost: 360° panoramic reconstruction can be achieved with only a single camera, projector, and turntable. Compared to the multiple camera systems of the SPU system, this method significantly reduces costs. Furthermore, this method eliminates the need for synchronized triggering of the projector and multiple cameras, making setup even simpler.

[0052] 2. Easy to implement: The calibration process of this method only involves single-camera-projector and camera-turntable axis calibration, which is simpler than the SPU method. During reconstruction, this method can flexibly select the number of virtual systems to assist in unwrapping as the object under test rotates, making it more stable than the SPU method.

[0053] 3. More Application Scenarios: The real-time 360° reconstruction system based on this method can be applied to real-time industrial inspection scenarios. For single-sided structured light inspection systems, issues such as shadow occlusion prevent complete defect detection of complex objects. This method provides a simple setup to achieve 360° panoramic defect detection. Additionally, the system can be applied to reverse engineering, cultural relic preservation, and other scenarios, obtaining a complete 360° 3D model in just 47 seconds. Since this method requires prior knowledge of the relative displacement between the object being measured and the measurement system, it can also be applied to real-time inspection of production lines. Due to the high speed of the production line, projecting multi-step phase-shift fringes can lead to phase destruction. This method can be used to capture single-cycle phase-shift fringes at different movement positions for unwrapping, achieving complete structured light reconstruction for real-time defect detection. Attached Figure Description

[0054] Figure 1 This is a diagram of the VSPU real-time 360° three-dimensional reconstruction system of the present invention;

[0055] In the diagram, 0 represents the camera, the dashed area represents the virtual camera during rotation, 1 represents the projector, the dashed area represents the virtual projector during rotation, 2 represents the turntable, 3 represents the geometric constraint measurement range of the system, and 4 represents the direction of rotation.

[0056] Figure 2 This is a data flow diagram of the method of the present invention;

[0057] Figure 3 This is the phase error diagram of the present invention; Detailed Implementation

[0058] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0059] Figure 1This is a diagram of the VSPU real-time 360° three-dimensional reconstruction system of the present invention; it includes a camera 0, a projector 1, an electric turntable 2, a turntable 2 controller, and a computer. The computer communicates with the camera 0, projector 1, and turntable 2 controller via serial ports. The turntable 2 controller and turntable 2 communicate via a 232 interface. The projector 1 is connected to the camera 0 via a trigger line for projection imaging. Each time the projector 1 projects a fringe pattern, it sends a trigger signal to the camera 0. The camera 0 captures the distorted fringe pattern for synchronous projection imaging in high-speed scenes. During VSPU three-dimensional reconstruction, the object under test is placed on the turntable 2. The control device sends a command to the turntable 2 to rotate uniformly along direction 4. Simultaneously, under precise timing control, the computer controls the projector 1 to project a set of single-cycle phase-shifted fringes at fixed intervals. The camera 0 acquires the distorted phase-shifted image through a trigger mechanism. The object under test rotates under the drive of the turntable 2. With the object under test as the reference frame, the reconstruction system rotates around the rotation axis to form a virtual system. Figure 1 (The part within the dashed box).

[0060] Figure 2 This is a data flow diagram of the method of the present invention; including:

[0061] The projector 1, camera 0, and turntable 2 are calibrated to obtain the intrinsic and extrinsic parameters of camera 0 and projector 1, and the position of the rotation axis of turntable 2; specifically:

[0062] The calibration process of the system includes the calibration of projector 1, camera 0 and turntable 2. The calibration process of camera 0 adopts Zhang's calibration method

[16] . The checkerboard is placed on the turntable 2 and rotated along direction 4 at a fixed angle. Camera 0 collects checkerboard images at different angles and extracts the checkerboard corner points to calculate the intrinsic parameters and distortion coefficients of camera 0. The calibration of projector 1 is carried out using Wei Gao's method

[17] . Red and blue checkerboards are placed at different angles. Projector 1 projects images with phase shift in the horizontal and vertical directions and maps the coordinates of the red and blue checkerboard corner points to projector 1. Projector 1 is used as the reverse camera 0 to calculate the intrinsic and extrinsic parameters and distortion coefficients of projector 1. The distortion correction of projector 1 is necessary. It will ensure the accuracy of the three-dimensional reconstruction system and the accuracy of the coordinates of the three-dimensional points projected onto the target surface of projector 1.

[0063] The rotation axis of turntable 2 was calibrated using a checkerboard pattern at different rotation angles. The corner points of the checkerboard pattern were extracted by camera 0, and the three-dimensional coordinates of camera 0 were calculated using the PnP algorithm. The checkerboard pattern coordinates at different angles were fitted with a circle using the SVD method

[18] , the rotation center axis was calculated, and the rotation center axis was used as the z-axis of the world coordinate system.

[0064] A rotation command is sent to the turntable 2 controller, and turntable 2 rotates at a constant speed;

[0065] Perform a single virtual stereo unpacking (VSPU);

[0066] The specific process of a single virtual 3D unpacking unit (VSPU) includes:

[0067] Projector 1 projects and captures three-step phase-shift diagrams 2m+1 (m≥1) times, with a delay of Δt for each step. vspu Each time a projection occurs, camera 0 captures a distorted phase-shifted image using hard triggering.

[0068] Based on the three-step phase shift diagrams at different angles, calculate the wrap-around phase diagrams at positions {-m, -m+1, ..., m-1, m} respectively; specifically:

[0069] Projector 1 projects phase-shifted fringes onto the object, and camera 0 captures the distorted fringe pattern. The fringe pattern captured by camera 0 can be represented as:

[0070]

[0071] I n The nth fringe image is projected, where n ∈ [0, N); N represents the number of phase shift steps. Generally, the larger the number of steps, the higher the measurement accuracy. This invention uses a three-step phase shift method. This represents the pixel coordinates of the i-th camera's 0-th image (hereinafter, for simplicity, it will be referred to as...). (Indicates); A represents average intensity, B represents amplitude; φ represents phase; The phase shift, φ, can be obtained through a phase shift algorithm:

[0072]

[0073] atan2 is a signed arctangent operation that expands the wrapping phase φ to (-π, π) by changing the signs of the numerator and denominator. For a three-step phase shift, the wrapping phase φ is:

[0074]

[0075] Based on the wrap-around phase map at position 0, a finite number of possible k values ​​are obtained by measuring range 3 using constant geometric constraints (CDC), and point clouds with different k values ​​are obtained by triangulation. Due to the limitations of projection brightness and focal depth of projector 1, the measurement system is set to a finite range. In CDC, the system can set an approximate range Z based on the size of the object. min and Z max , is represented as:

[0076] Z min ≤Z w (o c K c )≤Z max ,

[0077] For a 360° measurement system, CDC excludes some k values ​​that are out of range, but the depth range of the measured object will vary greatly at different angles, and phase ambiguity cannot be eliminated.

[0078] The obtained point cloud is projected onto the target surfaces of virtual camera 0 and virtual projector 1 at positions {-m, -m+1, -1, 1, ..., m-1, m}, respectively, to obtain the wrap-around phase maps of these points on virtual camera 0 and virtual projector 1. The absolute value of the difference is used to obtain the wrap-around phase error map at positions {-m, -m+1, -1, 1, ..., m-1, m}; in this invention, m = 2 is used, specifically:

[0079] Using the object under test as a reference frame, the camera 0-projector 1 system is considered to rotate around the axis of turntable 2 to form a virtual camera 0-projector 1 system. The phase difference angle θ between adjacent camera 0-projector 1 systems is θ = v·Δt, where v represents the rotation speed of turntable 2 and Δt represents the interval between capturing phase-shifted images. Using the rotation axis as the z-axis of the world coordinate system, the relationship between camera 0 and projector 1 at different angles can be expressed as:

[0080]

[0081] Unlike the spatial 3D unwrapping system, projector 1 in the virtual system is also rotated. In order to use the adjacent virtual camera 0-projector 1 for assisted unwrapping, the possible points obtained by the main camera 0 are projected onto the adjacent virtual projector 1 and virtual camera 0 respectively, which is represented as:

[0082]

[0083] in These represent the projections of the k-th possible point onto the camera 0c. i and projector 1p i Coordinates of the target surface Let A represent the transformation matrices of the i-th camera 0 and projector 1 in the world coordinate system, respectively. c A p These represent the intrinsic parameter matrices of camera 0 and projector 1, respectively. Let these represent the extrinsic parameter matrices of the i-th camera 0 and projector 1, respectively. Let represent the k-th possible point of the main camera 0 in the world coordinate system. For a correct possible point, the phase projected onto the virtual camera 0 at an adjacent angle should be the same as the phase of the corresponding projector 1. However, for an incorrect point, there will be a large deviation, and the phase projected onto the target surface of camera 0 will be... The wrapping phase image obtained by Eq(3) at the corresponding rotation angle can be used to... Two-dimensional interpolation reveals that phase compensation is required at the transition position. The phase value projected onto the target surface of projector 1 can be expressed as:

[0084]

[0085] in express The horizontal axis coordinate is given, and the limit constraint axis from camera 0 to projector 1 is the u-axis. The range is [-π, π]. The possible points are filtered using the wrap-around phase error of the possible points projected onto virtual projector 1 and camera 0 as the criterion. The error is expressed as:

[0086]

[0087]

[0088] in Let the phase difference of the package at the k-th possible point under the i-th virtual camera 0 be denoted as . This indicates the corresponding discrimination error. Due to the phase jump, the phases of camera 0 and projector 1 may fall on different k when checking the phase deviation. To avoid this ambiguity, a jump is actively performed to ensure that the phase deviation is within [0, π].

[0089] By summing the phase error maps of the package, we obtain the phase error maps of the package under different k values. The k value with the smallest error is then selected to obtain the final unwrapped k-value map. Specifically, the proposed method utilizes a virtual projector 1-camera 0 system for stereo unwrapping during the rotation process. The number of virtual systems can be flexibly selected; using more virtual systems for unwrapping ensures the stability of the method. The final selected k value is the minimum sum of the deviations of all virtual systems.

[0090]

[0091] k result =minIndex(Err(k))

[0092] Where k result Here, k is the value obtained by the VSPU method, Err(k) is the phase error of different k values, minIndex(·) is the index function of the minimum value, m is the number of adjacent virtual systems selected, and the number of systems to assist in unpacking is 2m.

[0093] Based on the unpacked k-value map, point clouds at the 0 position angle are obtained through triangulation, and these point clouds are registered into the 360-degree panoramic model.

[0094] After a single virtual stereo unpacking process (VSPU) ends, there is a delay Δt. regProceed to the next VSPU process until the 360-degree scan is complete.

[0095] A single VSPU process consists of five projection images of a three-step phase-shifting plot, with a delay of Δt for each step. vspu After the VSPU process at a single angle is completed, there is a delay of Δt. reg The next VSPU process will be performed as follows:

[0096] First, after turntable 2 starts, it rotates at a constant angular acceleration α until it reaches a uniform speed ω. turn The settling time is Δt = ω turn / α, after which the system begins projection imaging, each time after Δt... reg A set of stripe patterns for the VSPU is captured at regular intervals, with the time interval determined by a set registration angle interval Δθ. reg The obtained Δt reg =Δθ reg / ω turn In this invention, the rotational speed ω of turntable 2 turn =8° / s, turntable acceleration α = 4° / s 2 Registration interval Δt reg = 1.5s, registered angular interval Δθ reg =12°, VSPU angle interval Δθ vspu =2°, the present invention can ultimately achieve 360° three-dimensional reconstruction within 47s, and obtain point clouds at 30 angles.

[0097] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the core technical features of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0098] References

[0099] [1] J.Salvi, S.Fernandez, T.Pribanic,X.Llado, A state of the art instructured light patterns for surface profilometry, Pattern recognition, 43 (2010) 2666-2680.

[0100] [2]S.Zhang,Recent progresses on real-time 3D shape measurement usingdigital fringe projection techniques,Optics and lasers in engineering,48(2010)149-158.

[0101] [3]S.S.Gorthi,P.Rastogi,Fringe projection techniques:whither we are?,Optics and lasers in engineering,48(2010)133-140.

[0102] [4]S.Zhang,High-speed 3D shape measurement with structured lightmethods:A review,Optics and Lasers in Engineering,106(2018)119-131.

[0103] [5]S.Feng,C.Zuo,T.Tao,Y.Hu,M.Zhang,Q.Chen,G.Gu,Robust dynamic 3-dmeasurements with motion-compensated phase-shifting profilometry,Optics andLasers in Engineering,103(2018)127-138.

[0104] [6]X.Liu,X.Peng,H.Chen,D.He,B.Z.Gao,Strategy for automatic andcomplete three-dimensional optical digitization,Optics letters,37(2012)3126-3128.

[0105] [7]J.Qian,S.Feng,T.Tao,Y.Hu,K.Liu,S.Wu,Q.Chen,C.Zuo,High-resolutionreal-time 360 3d model reconstruction of a handheld object with fringeprojection profilometry,Optics letters,44(2019)5751-5754.

[0106] [8]J.-J.Orteu,F.Bugarin,J.Harvent,L.Robert,V.Velay,Multiple-camerainstrumentation of a single point incremental forming process pilot for shapeand 3D displacement measurements:methodology and results,Experimentalmechanics,51(2011)625-639.

[0107] [9]D.Solav,K.M.Moerman,A.M.Jaeger,K.Genovese,H.M.Herr,MultiDIC:Anopen-source toolbox for multi-view 3D digital image correlation,IEEE Access,6(2018)30520-30535.

[0108]

[10] S.Rusinkiewicz,O.Hall-Holt,M.Levoy,Real-time 3D modelacquisition,ACM Transactions on Graphics(TOG),21(2002)438-446.

[0109]

[11] W.Yin,S.Feng,T.Tao,L.Huang,S.Zhang,Q.Chen,C.Zuo,Calibrationmethod for panoramic 3D shape measurement with plane mirrors,Optics Express,27(2019)36538-36550.

[0110]

[12] M.Dai,L.Chen,F.Yang,X.J.A.o.He,Calibration of revolution axis for360deg surface measurement,52(2013)5440-5448.

[0111]

[13] X.Cai,K.Zhong,Y.Fu,J.Chen,Y.Liu,C.Huang,Calibration method forthe rotating axis in panoramic 3D shape measurement based on a turntable,Measurement Science and Technology,32(2020)035004.

[0112]

[14] J.Qian,T.Tao,S.Feng,Q.Chen,C.Zuo,Motion-artifact-free dynamic 3Dshape measurement with hybrid Fourier-transform phase-shifting profilometry,Optics express,27(2019)2713-2731.

[0113]

[15] S.Feng,Q.Chen,G.Gu,T.Tao,L.Zhang,Y.Hu,W.Yin,C.Zuo,Fringe patternanalysis using deep learning,Advanced Photonics,1(2019)025001.

[0114]

[16] Z.Zhang,A flexible new technique for camera calibration,IEEETransactions on pattern analysis and machine intelligence,22(2000)1330-1334.

[0115]

[17] W.Gao,L.Wang,Z.Hu,Flexible method for structured light systemcalibration,Optical Engineering,47(2008)083602.

[0116]

[18] T.-H.Lin,Automatic 3D color shape measurement system based on astereo camera,Applied Optics,59(2020)2086-2096.

Claims

1. A real-time panoramic 3D reconstruction method based on virtual stereoscopic unwrapping, characterized in that, include: A three-dimensional reconstruction system is constructed, the system including a computer, a projector and a camera respectively connected to the computer, a turntable controller, a turntable connected to the turntable controller, and the projector and camera being connected. The projector, camera, and turntable are calibrated to obtain the internal and external parameters of the camera and projector, and the position of the turntable's rotation axis. A rotation command is sent to the turntable controller, and the turntable rotates at a constant speed; Perform a single virtual stereo unpacking (VSPU); Delay after a single Virtual 3D Unpacking Unit (VSPU) is completed Proceed to the next VSPU process until the 360-degree scan is complete; The specific process of the single virtual stereo unpacking (VSPU) includes: Projector projection shooting 2m+1 ( () Multi-step phase shift diagram, with a delay of for each step. Each time a projection occurs, the camera captures a distorted phase-shifted image using hard triggering. Calculate the multi-step phase shift diagrams from different angles respectively. Package phase diagram under location; Based on the wrap-around phase map at position 0, a finite number of possible k values ​​are obtained through constant geometric constraints (CDC), and point clouds with different k values ​​are obtained through triangulation. The obtained point clouds are projected onto... By defining the virtual camera target surface and virtual projector target surface, and obtaining the wrap-around phase maps of these points on the virtual camera and virtual projector, the absolute value of the difference is obtained. Phase error map of the package at the location; By summing the package phase error maps, we obtain package phase error maps under different k values. We then take the k value with the smallest error to obtain the final unpacking k value map. Based on the unpacking k-value map, point clouds at the 0 position angle are obtained through triangulation, and these point clouds are registered into the 360-degree panoramic model. The aforementioned calculation based on multi-step phase shift diagrams at different angles is performed separately. The phase map of the package under the location is as follows: A projector projects phase-shifted fringes onto an object, and a camera captures the distorted fringe pattern. The fringe pattern captured by the camera can be represented as: ； Representing the projection number A striped image, ; Indicates the number of phase shift steps. Indicates the first Each camera image pixel coordinate (hereinafter referred to as...) express); Indicates average intensity. Indicates amplitude; Indicates phase; Indicates phase shift, phase This can be achieved through a phase-shifting algorithm: ; For the signed arctangent operation, the phase is enclosed by the signs of the numerator and denominator. Expand to .

2. The real-time panoramic 3D reconstruction method based on virtual stereoscopic unwrapping method according to claim 1, characterized in that, Based on the wrap-around phase diagram at position 0, a finite number of possible k values ​​are obtained through constant geometric constraints (CDC), specifically: In CDC, the system can set a general range based on the size of the object. and , is represented as: ； For 360° measurement systems, CDC excludes some that are outside the range. The depth range of the measured object varies greatly at different angles, making it impossible to eliminate phase ambiguity.

3. The real-time panoramic 3D reconstruction method based on virtual stereoscopic unwrapping method according to claim 2, characterized in that, The obtained point cloud is projected onto... By defining the virtual camera target surface and virtual projector target surface, and obtaining the wrap-around phase maps of these points on the virtual camera and virtual projector, the absolute value of the difference is obtained. The phase error map of the package at the location is as follows: Using the object under test as a reference frame, a virtual camera-projector system is formed by rotating the camera-projector system around the turntable axis. Adjacent camera-projector systems differ in angle. for ,in Indicates the rotational speed of the turntable. This indicates the time interval for capturing phase-shifted images, with the turntable rotation axis as the world coordinate system. The relationship between the camera and projector at different angles along the axis can be expressed as: ； Unlike spatial 3D unwrapping systems, the projector in the virtual system is also rotated. To use virtual camera-projector pairs at adjacent angles for assisted unwrapping, the possible points obtained by the main camera are projected onto adjacent virtual projectors and virtual cameras, respectively, as shown below: ； ； in They represent the first A possible point is projected onto the camera. and projector Coordinates of the target surface Representing the first in the world coordinate system Transformation matrices for each camera and projector. These represent the intrinsic parameter matrices of the camera and projector, respectively. They represent the first The extrinsic matrix of each camera and projector Indicates the first position of the main camera in the world coordinate system. One possible point, the phase projected onto the camera target surface. The wrapping phase image obtained by Eq(3) at the corresponding rotation angle is used to... Two-dimensional interpolation reveals that phase compensation is required at the transition positions. The phase value projected onto the projector target surface can be expressed as: ； in express The horizontal axis is the coordinate of the camera-projector limit constraint, and the U-axis is the coordinate of the camera-projector limit constraint. The range is The possible points are filtered using the wrap-around phase error between the virtual projector and the camera projection as the criterion. The error is expressed as: ； ； in For the k-th possible point in the th... Package phase difference under a virtual camera This indicates the corresponding discrimination error, and an active transition is performed to ensure that the phase deviation is within [the specified range]. Inside.

4. The real-time panoramic 3D reconstruction method based on virtual stereoscopic unwrapping method according to claim 1 or 3, characterized in that, The summation of the package phase error maps yields package phase error maps for different k values. The k value with the smallest error is selected to obtain the final unpacking k-value map. Specifically: The final chosen value of k is the minimum sum of the deviations of all virtual systems. ； ； in Obtained by the VSPU method value, For phase errors of different k values, The index function for the minimum value. To select the number of adjacent virtual systems, the number of systems assisting in unpacking is: .

5. The real-time panoramic 3D reconstruction method based on virtual stereoscopic unwrapping method according to claim 4, characterized in that, The described single virtual stereoscopic unwrapping process includes five projection images of multi-step phase-shifting maps, with a delay of [time value missing] for each step. After the VSPU process at a single angle is completed, there is a delay. The next VSPU process will be performed as follows: First, after the turntable starts, it accelerates at a constant angular acceleration. Rotation accelerates to a constant speed The stabilization time is Then the system began projecting images, each time passing through... A set of stripe patterns for the VSPU is captured at regular intervals, with the time interval determined by the set registration angle. To obtain .

Images