Moving object three-dimensional reconstruction error correction method

Abstract

The invention discloses a moving object three-dimensional reconstruction error correction method, which comprises the following steps that: a moving object three-dimensional measurement system based on a fringe projection technology is built, a projector circularly projects 2 + 1 phase-shift fringe patterns which are arranged in a predetermined manner to the surface of a moving object, and a camera synchronously acquires modulated 2 + 1 phase-shift fringe images of the moving object; extracting an initial truncated phase sequence of continuous frames by using a 2 + 1 phase shift method, and recovering an initial continuous phase sequence by using a minimum phase surface constraint method; calculating an average value of initial continuous phases of front and back frames to obtain a corrected continuous phase sequence; and calibrating geometric parameters of the fringe projection system, mapping the corrected continuous phase sequence to a three-dimensional physical space, and reconstructing the three-dimensional shape of the moving object. The phase information is preliminarily extracted by using the 2 + 1 phase shift method, the periodic phase error caused by object movement is corrected by calculating the average phase distribution of the adjacent frames, and the method has the advantages of high speed, high precision, strong robustness, wide applicability and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional measurement error correction, and in particular to a method for correcting errors in three-dimensional reconstruction of a moving object. Background Art

[0002] Fringe projection technology, with its advantages of non-contact measurement, high accuracy, and fast speed, is widely used in industrial inspection, biomedicine, virtual reality, and other fields. Fringe projection systems typically consist of a projector and a camera, forming a typical triangulation system: the projector projects pre-coded fringes onto the surface of the object being measured. The deformed fringes, modulated by the object, are synchronously captured by the camera. Digital fringe analysis technology extracts phase distribution information, and based on pre-calibrated system geometric parameters, a high-precision reconstruction of the object's three-dimensional topography is achieved. Phase shifting algorithms, which typically require N or more fringe patterns with varying phase shifts, offer the dual advantages of high precision and robustness, and have become the mainstream choice for industrial fringe projection systems.

[0003] However, when measuring moving objects, the inter-frame relative displacement between the object being measured and the fringe projection system will destroy the spatial consistency assumption of the phase shift algorithm: an unknown offset will be generated between the preset phase shift amount and the actual phase shift amount projected onto the object surface, which will lead to periodic errors in the phase solution results, directly affecting the accuracy of three-dimensional reconstruction.

[0004] To address the phase error correction problem for moving objects, existing methods use feature matching to extract the object's displacement information, then estimate the phase offset and correct the phase solution. However, this method relies on feature matching accuracy and is susceptible to interference from inter-frame modulation fringe variations, making it unsuitable for 3D measurement of low-texture moving objects. Therefore, overcoming the limitations of feature matching and correcting phase errors caused by moving objects remains a key issue that needs to be addressed in the field of dynamic 3D measurement using fringe projection technology, with significant practical significance and application value. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that the existing feature matching method extracts the displacement information of the moving object, estimates the phase shift offset and corrects the phase solution result, but is heavily dependent on the feature matching accuracy, is easily interfered by the difference in modulation stripes between frames, and is not suitable for the three-dimensional measurement of low-texture moving objects. Instead, a method for error correction of three-dimensional reconstruction of moving objects is proposed.

[0006] The technical solution of the present invention is a method for correcting errors in three-dimensional reconstruction of a moving object, comprising:

[0007] Build a three-dimensional measurement system for moving objects based on fringe projection technology. The projector, camera, and moving object form a triangulation relationship, and the projector and camera are triggered synchronously through hardware circuits.

[0008] The projector cyclically projects a predetermined 2+1 phase-shifted fringe pattern onto the surface of a moving object, and the camera synchronously captures the modulated 2+1 phase-shifted fringe image of the moving object.

[0009] Extract the initial truncated phase sequence of consecutive frames using the 2+1 phase shift method…,φ k ,φ k+1 ,…, using the minimum phase surface Φ min The constraint method recovers the initial continuous phase sequence...,Φ k ,Φ k+1 ,....;

[0010] Calculate the initial continuous phase of the previous and next frames...,Φ k ,Φ k+1 ,....the average value, obtain the corrected continuous phase sequence...,Φ′ k ,Φ′ k+1 ,....;

[0011] Calibrate the geometric parameters of the fringe projection system to correct the continuous phase sequence...,Φ′ k ,Φ′ k+1 ,....map to three-dimensional physical space and reconstruct the three-dimensional shape of the moving object.

[0012] Optionally, the predetermined arrangement of the 2+1 phase-shifted stripe pattern in the kth frame and the k+1th frame is expressed as: H sin,k (x,y)=0.5+0.5*sin(2πfx)

[0013] H cos,k (x,y)=0.5-0.5*cos(2πfx)

[0014] H avg,k (x,y)=0.5

[0015] H cos,k+1 (x,y)=0.5+0.5*cos(2πfx)

[0016] H sin,k+1 (x,y)=0.5-0.5*sin(2πfx)

[0017] H avg,k+1 (x,y)=0.5

[0018] Where k = 1, 2, 3, ... represents the serial number of the phase-shifted fringe pattern; (x, y) represents the pixel coordinates of the fringe image; f represents the fringe frequency along the X-axis; H sin,k 、H cos,k 、H avg,kRepresent the sine fringe, cosine fringe, and background light intensity patterns of the kth frame respectively; H sin,k+1 、H cos,k+1 、H avg,k+1 They represent the sine stripes, cosine stripes, and background light intensity patterns of the k+1th frame respectively.

[0019] Optionally, when the surface reflectivity change caused by object motion in the kth frame and the k+1th frame is ignored, the expression of the 2+1 phase-shifted fringe image captured by the camera is:

[0020] I sin,k (x,y)≈A(x,y)+B(x,y)*sin[φ(x,y)-δ sin,k ]

[0021] I cos,k (x,y)≈A(x,y)-B(x,y)*cos[φ(x,y)-δ sin,k ]

[0022] I avg,k (x,y)≈A(x,y)

[0023] I cos,k+1 (x,y)≈A(x,y)+B(x,y)*cos[φ(x,y)+δ cos,k+1 ]

[0024] I sin,k+1 (x,y)≈A(x,y)-B(x,y)*sin[φ(x,y)+δ sin,k+1 ]

[0025] I avg,k+1 (x,y)≈A(x,y)

[0026] Where A(x,y) represents the background intensity; B(x,y) represents the modulation intensity; φ(x,y) represents the cutoff phase; δ represents the additional phase shift introduced by the object motion; I sin,k , I cos,k , I avg,k I represents the image of sine fringe, cosine fringe and background light intensity of the kth frame respectively; sin,k+1 , I cos,k+1 , I avg,k+1 Represent the images of sine fringes, cosine fringes, and background light intensity of the k+1th frame respectively.

[0027] Optionally, the initial truncation phase φ of the kth frame and the k+1th frame k ,φ k+1 The calculation formula is as follows:

[0028]

[0029] Where tan -1 Represents the inverse tangent function.

[0030] Optionally, the initial truncated phase φ of the kth frame and the k+1th frame is k ,φ k+1 Convert from the value interval [-π,π) to the value interval [0,2π) as follows:

[0031] φ k ′(x,y)=mod[φ k (x,y),2π]

[0032] φ′ k+1 (x,y)=mod[φ k+1 (x,y),2π]

[0033] Where mod represents the remainder operation function.

[0034] Optionally, a fringe projection system is used to project a 2+1 phase-shifted fringe pattern of the same frequency onto a flat background plate. After phase unwrapping, the continuous phase image of the flat background plate is restored, which can be used as the minimum phase surface Φ min .

[0035] Optionally, calculate the initial truncation phase φ of the kth frame k The corresponding stripe levels are as follows:

[0036]

[0037] Where ceil represents the rounding function, then the initial continuous phase Φ of the kth frame is k The calculation formula is as follows:

[0038] Φ k (x,y)=φ k (x,y)+2πK(x,y)

[0039] Similarly, the initial continuous phase Φ of the k+1th frame can be calculated k+1 .

[0040] Optionally, the kth frame corrects the continuous phase Φ′ k The calculation formula is as follows:

[0041]

[0042] Similarly, the k+1th frame correction continuous phase Φ′ can be calculated k+1 .

[0043] Compared with the prior art, this application has at least one of the following beneficial technical effects:

[0044] The present invention uses the 2+1 phase shift method to preliminarily extract phase information, and corrects the periodic phase error caused by object motion by calculating the average phase distribution of adjacent frames. It has the advantages of fast speed, high accuracy, strong robustness and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 1 is the simulation experiment result of the 2+1 phase-shifted fringe image in the kth frame and the k+1th frame of the present invention.

[0046] Figure 2 These are the simulation results of the present invention for the initial truncated phase and the initial continuous phase at the kth frame and the k+1th frame, and the corrected continuous phase at the kth frame.

[0047] Figure 3 : These are the simulation results of the initial phase errors of the kth frame and the k+1th frame, and the corrected phase error of the kth frame.

[0048] Figure 4 are the initial 3D reconstruction results of the kth frame and the k+1th frame, and the corrected 3D reconstruction result of the kth frame. DETAILED DESCRIPTION

[0049] The technical solutions of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure claimed for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0050] Please refer to Figures 1-4 As shown in the figure, a method for correcting errors in 3D reconstruction of moving objects is proposed. This method uses the 2+1 phase shift method to initially extract phase information and corrects the periodic phase error caused by object motion by calculating the average phase distribution of adjacent frames. It has the advantages of fast speed, high accuracy, strong robustness, and wide applicability. The specific steps of this method are as follows:

[0051] Step S1: Build a three-dimensional measurement system for moving objects based on fringe projection technology. The projector, camera and moving object form a triangulation relationship, and the projector and camera are triggered synchronously through hardware circuits.

[0052] Step S2: The projector is responsible for cyclically projecting a predetermined 2+1 phase-shifted fringe pattern onto the surface of the moving object, and the camera is responsible for synchronously capturing the modulated 2+1 phase-shifted fringe image of the moving object. Figure 1 Taking the kth frame and the k+1th frame as examples, a 2+1 phase-shifted fringe image is shown.

[0053] The predetermined 2+1 phase-shifted fringe pattern at the kth frame and the k+1th frame is expressed as:

[0054] H sin,k (x,y)=0.5+0.5*sin(2πfx)

[0055] H cos,k (x,y)=0.5-0.5*cos(2πfx)

[0056] H avg,k (x,y)=0.5

[0057] H cos,k+1 (x,y)=0.5+0.5*cos(2πfx)

[0058] H sin,k+1 (x,y)=0.5-0.5*sin(2πfx)

[0059] H avg,k+1 (x,y)=0.5

[0060] Where k = 1, 2, 3, ... represents the serial number of the phase-shifted fringe pattern; (x, y) represents the pixel coordinates of the fringe image; f represents the fringe frequency along the X-axis; H sin,k 、H cos,k 、H avg,k Represent the sine fringe, cosine fringe, and background light intensity patterns of the kth frame respectively; H sin,k+1 、H cos,k+1 、H avg,k+1 They represent the sine stripes, cosine stripes, and background light intensity patterns of the k+1th frame respectively.

[0061] When the surface reflectivity changes caused by object motion in the kth and k+1th frames are ignored, the expression of the 2+1 phase-shifted fringe image captured by the camera is:

[0062] I sin,k (x,y)≈A(x,y)+B(x,y)*sin[φ(x,y)-δ sin,k ]

[0063] I cos,k (x,y)≈A(x,y)-B(x,y)*cos[φ(x,y)-δ sin,k ]

[0064] I avg,k (x,y)≈A(x,y)

[0065] I cos,k+1 (x,y)≈A(x,y)+B(x,y)*cos[φ(x,y)+δ cos,k+1 ]

[0066] I sin,k+1 (x,y)≈A(x,y)-B(x,y)*sin[φ(x,y)+δ sin,k+1 ]

[0067] I avg,k+1 (x,y)≈A(x,y)

[0068] Where A(x,y) represents the background intensity; B(x,y) represents the modulation intensity; φ(x,y) represents the cutoff phase; δ represents the additional phase shift introduced by the object motion; I sin,k , I cos,k , I avg,k I represents the image of sine fringe, cosine fringe and background light intensity of the kth frame respectively; sin,k+1 , I cos,k+1 , I avg,k+1 Represent the images of sine fringes, cosine fringes, and background light intensity of the k+1th frame respectively.

[0069] Step S3: Extract the initial truncated phase sequence of consecutive frames using the 2+1 phase shift method...,φ k ,φ k+1 ,...., using the minimum phase surface Φ min The constraint method recovers the initial continuous phase sequence...,Φ k ,Φ k+1 ,..... The initial truncated phase φ of the kth frame and the k+1th frame k ,φ k+1 The calculation formula is as follows:

[0070]

[0071] Where tan -1 In order to facilitate the operation, the initial truncated phase φ of the kth frame and the k+1th frame is k ,φ k+1 Convert from the value interval [-π,π) to the value interval [0,2π) as follows:

[0072] φ′ k (x,y)=mod[φ k (x,y),2π]

[0073] φ′ k+1(x,y)=mod[φ k+1 (x,y),2π]

[0074] Where mod represents the remainder operation function.

[0075] It should be noted that the fringe projection system is used to project a 2+1 phase-shifted fringe pattern of the same frequency onto a plane background plate. After phase unwrapping, the continuous phase image of the plane background plate is restored, which can be used as the minimum phase surface Φ min . Further, calculate the initial truncation phase φ of the kth frame k The corresponding stripe levels are as follows:

[0076]

[0077] Where ceil represents the rounding function, then the initial continuous phase Φ of the kth frame is k The calculation formula is as follows:

[0078] Φ k (x,y)=φ k (x,y)+2πK(x,y)

[0079] Similarly, the initial continuous phase Φ of the k+1th frame can be calculated k+1 .

[0080] like Figure 2 Shows the initial truncated phase φ of the kth frame and the k+1th frame k ,φ k+1 , the initial continuous phase Φ of the kth frame and the k+1th frame k ,Φ k+1 ; Figure 3 The initial phase error between the kth frame and the k+1th frame is shown, indicating the initial continuous phase Φ k ,Φ k+1 There is a large phase error.

[0081] Step S4: By calculating the initial continuous phase of the previous and next frames...,Φ k ,Φ k+1 ,....the average value, obtain the corrected continuous phase sequence...,Φ′ k ,Φ′ k+1 ,..., the kth frame corrects the continuous phase Φ′ k The calculation formula is as follows:

[0082]

[0083] Similarly, the k+1th frame correction continuous phase Φ′ can be calculated k+1 .

[0084] like Figure 2 Shows the k-th frame corrected continuous phase Φ′k , Figure 3 Shows the corrected k-th frame phase error, relative to the initial continuous phase Φ k ,Φ k+1 , correct the continuous phase Φ′ k The phase error is greatly reduced;

[0085] Step S5: calibrate the geometric parameters of the fringe projection system and calibrate the continuous phase sequence ..., Φ′ k ,Φ′ k+1 ,.Map to three-dimensional physical space and reconstruct the three-dimensional shape of the moving object. Figure 4 The initial k-th frame and k+1-th frame 3D reconstruction results, as well as the corrected k-th frame 3D reconstruction results are shown. The comparison results show that there are obvious periodic ripple errors in the initial k-th frame and k+1-th frame 3D reconstruction results, while the periodic ripple errors in the corrected k-th frame 3D reconstruction results are greatly reduced. Among them, the geometric parameter calibration method of the fringe projection system adopts the traditional phase-height mapping method. For details, please refer to the following literature Calibration of fringe projection profilometry: A comparative review [J]. Optics and Lasers in Engineering, 2021, 143: 106622, which will not be elaborated here.

[0086] The above specific embodiments are merely several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art may make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A method for correcting errors in three-dimensional reconstruction of a moving object, characterized in that: include: Build a three-dimensional measurement system for moving objects based on fringe projection technology. The projector, camera, and moving object form a triangulation relationship, and the projector and camera are triggered synchronously through hardware circuits. The projector cyclically projects a predetermined 2+1 phase-shifted fringe pattern onto the surface of a moving object, and the camera synchronously captures the modulated 2+1 phase-shifted fringe image of the moving object. Extract the initial truncated phase sequence of consecutive frames using the 2+1 phase shift method…,φ k ,φ k+1 ,…, using the minimum phase surface Φ min The constraint method recovers the initial continuous phase sequence...,Φ k ,Φ k+1 ,....; Calculate the initial continuous phase of the previous and next frames...,Φ k ,Φ k+1 ,....the average value, obtain the corrected continuous phase sequence...,Φ′ k ,Φ′ k+1 ,....; Calibrate the geometric parameters of the fringe projection system to correct the continuous phase sequence...,Φ′ k ,Φ′ k+1 ,....map to three-dimensional physical space and reconstruct the three-dimensional shape of the moving object.

2. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 1, wherein: The predetermined arrangement of the 2+1 phase-shifted fringe pattern in the kth frame and the k+1th frame is expressed as: H sin,k (x,y)=0.5+0.5*sin(2πfx) H cos,k (x,y)=0.5-0.5*cos(2πfx) H avg,k (x,y)=0.5 H cos,k+1 (x,y)=0.5+0.5*cos(2πfx) H sin,k+1 (x,y)=0.5-0.5*sin(2πfx) H avg,k+1 (x,y)=0.5 Where k = 1, 2, 3, ... represents the serial number of the phase-shifted fringe pattern; (x, y) represents the pixel coordinates of the fringe image; f represents the fringe frequency along the X-axis; H sin,k 、H cos,k 、H avg,k Represent the sine fringe, cosine fringe, and background light intensity patterns of the kth frame respectively; H sin,k+1 、H cos,k+1 、H avg,k+1 They represent the sine stripes, cosine stripes, and background light intensity patterns of the k+1th frame respectively.

3. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 2, wherein: When the surface reflectivity changes caused by object motion in the kth and k+1th frames are ignored, the expression of the 2+1 phase-shifted fringe image captured by the camera is: Yo sin,k (x,y)≈A(x,y)+B(x,y)*sin[φ(x,y)-δ sin,k ] Yo cos,k (x,y)≈A(x,y)-B(x,y)*cos[φ(x,y)-δ sin,k ] Yo avg,k (x,y)≈A(x,y) Yo cos,k+1 (x,y)≈A(x,y)+B(x,y)*cos[φ(x,y)+δ cos,k+1 ] Yo sin,k+1 (x,y)≈A(x,y)-B(x,y)*sin[φ(x,y)+δ sin,k+1 ] Yo avg,k+1 (x,y)≈A(x,y) Where A(x,y) represents the background intensity; B(x,y) represents the modulation intensity; φ(x,y) represents the cutoff phase; δ represents the additional phase shift introduced by the object motion; I sin,k , I cos,k , I avg,k I represents the image of sine fringe, cosine fringe and background light intensity of the kth frame respectively; sin,k+1 , I cos,k+1 , I avg,k+1 Represent the images of sine fringes, cosine fringes, and background light intensity of the k+1th frame respectively.

4. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 3, wherein: The initial truncated phase φ of the kth frame and the k+1th frame k ,φ k+1 The calculation formula is as follows: Where tan -1 Represents the inverse tangent function.

5. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 4, wherein: The initial truncated phase φ of the kth frame and the k+1th frame k ,φ k+1 Convert from the value interval [-π,π) to the value interval [0,2π) as follows: f′ k (x,y)=mod[φ k (x,y),2π] f′ k+1 (x,y)=mod[φ k+1 (x,y),2π] Where mod represents the remainder operation function.

6. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 1, wherein: A fringe projection system is used to project a 2+1 phase-shifted fringe pattern of the same frequency onto a flat background plate. After phase unwrapping, the continuous phase image of the flat background plate is restored, which can be used as the minimum phase surface Φ min .

7. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 1, wherein: Calculate the initial truncation phase φ of the kth frame k The corresponding stripe levels are as follows: Where ceil represents the rounding function, then the initial continuous phase Φ of the kth frame is k The calculation formula is as follows: Φ k (x,y)=φ k (x,y)+2πK(x,y) Similarly, the initial continuous phase Φ of the k+1th frame can be calculated k+1 .

8. The method for correcting errors in three-dimensional reconstruction of a moving object according to claim 1, wherein: The k-th frame corrects the continuous phase Φ′ k The calculation formula is as follows: Similarly, the k+1th frame correction continuous phase Φ′ can be calculated k+1 .

Images