A phase-shift profilometry method based on multi-angle sinusoidal fringe light field fusion

Abstract

Three-dimensional measurement based on light field has become one of the mainstream technologies for industrial three-dimensional detection due to its good accuracy, speed and detection range. Combining it with sinusoidal fringe phase shift profilometry can effectively improve the detection accuracy. The present invention proposes a phase shift profilometry based on multi-angle sinusoidal fringe light field fusion. The method first collects one or more groups of sinusoidal phase shift fringes based on the light field imaging system of the microlens array, calculates the absolute phase of each sub-aperture image after image calibration, selects the fused sub-aperture based on the phase contribution weight, and then combines the fused phase of angle, distance and similarity. After the phase is mapped to the height, it is filled into the point cloud data through correction, and finally filled into the point cloud matrix through multi-angle point cloud fusion. Furthermore, corresponding micro-lens array calibration, camera calibration and phase-height mapping calibration methods are proposed.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional optical detection technology, and in particular to a phase shift profilometry based on multi-angle sinusoidal fringe light field fusion, which can perform high-speed and high-precision three-dimensional detection of extremely small objects on diffuse reflection surfaces. Background Art

[0002] In order to better achieve quality process control in industrial production, optical inspection systems are a key part of many manufacturing and quality control processes. The cutting-edge manufacturing field currently has challenging requirements for optical inspection systems, namely high throughput and the ability to detect tiny defects, adaptability to complex inspection surfaces, and accurate 3D height measurement of moving objects. Light field 3D measurement has become one of the mainstream solutions for industrial 3D inspection due to its micron-level height measurement accuracy, millimeter-level height measurement range, and fast inspection speed.

[0003] Traditional light field uses light field images collected under natural light or uniform light illumination for 3D reconstruction, which is called passive light field. Passive light field technology has no special requirements for the environment and the system is relatively simple, but it is not ideal for structural reconstruction of weak texture surfaces. If structured light illumination technology is used to project structured light containing depth coding information onto the surface of objects and scenes, active coding can be generated regardless of the surface texture, thereby improving the effect of 3D reconstruction.

[0004] In the actual field of industrial three-dimensional detection, due to different light incident angles and different surface reflectivities, the image may be partially too bright or too dark, and shadows between detection targets may easily lead to detection blind spots and erroneous data. The information obtained based on the light field under monocular vision is not comprehensive enough, which may cause large measurement errors. Summary of the invention

[0005] The present invention aims to realize industrial three-dimensional high-speed and high-precision detection. First, one or more groups of phase-shifted fringes are projected, and a light field imaging system based on a micromirror array is used to take pictures. After image calibration, the relative phase and absolute phase of each sub-aperture are calculated. The sub-aperture to be fused is selected based on the sub-aperture selection algorithm based on the phase contribution weight. The sub-aperture fusion phase is combined with the angle, distance and similarity. After the phase is mapped to the height, it is filled into the point cloud data through correction. Finally, the final point cloud matrix is ​​formed through a multi-angle point cloud array. Furthermore, corresponding micromirror array calibration, camera calibration and phase-height mapping calibration methods are proposed, as shown in detail below.

[0006] S1. A sinusoidal stripe light field projection and shooting device provided by the present invention uses a projection mechanism to project one or more groups of sinusoidal stripes onto the surface of the object to be measured. After the light is reflected, it passes through a microlens array and is collected by a surface array camera. At this time, the camera not only collects the light intensity information of each light ray, but also collects the angle information of each light ray to obtain multiple sub-aperture stripe light images at different angles. Based on the dual-plane model, the four-dimensional light field is parameterized. A four-dimensional light field can be parameterized as L ux , where x = (x, y) T , u =(u, v) T Represent the spatial coordinates and angular coordinates of the light field respectively. The formula is characterized by the cross-section of the light in two planes.

[0007] S2. The present invention proposes a method for calibrating sub-aperture images. Since the microlens array cannot be completely parallel to the CCD plane, the sub-aperture image will have a certain offset. It is necessary to perform image calibration in advance to obtain calibration parameters, and correct the image based on the calibration parameters to facilitate subsequent processing.

[0008] S3. The present invention adopts the sinusoidal phase shift method to solve the phase. The correlation between multiple fringe images can effectively reduce the influence of noise and improve the precision and accuracy of phase acquisition. Assuming that N sinusoidal fringe images are projected, let I i (x,y) represents the light intensity of the i-th fringe image at pixel coordinate (x, y).

[0009]

[0010] Where a and b are the background radiation intensity and modulated radiation intensity of the recorded light field, respectively. is the phase information modulated by the object height, is the phase shift of the i-th fringe. The relative phase can be demodulated by fringe analysis technology. The formula is as follows.

[0011]

[0012] The range of the relative phase after the inverse tangent operation is (-π, π). If the phase exceeds one phase cycle, only the phase compressed within one phase cycle can be obtained. Here, the phase needs to be expanded to obtain the absolute phase. , phase unwrapping requires determining the fringe order K, as shown in the following formula.

[0013]

[0014] Among them, if a group of sinusoidal phase-shifted fringes is projected, that is, there is only one group of frequencies, a space-based phase unwrapping algorithm can be used; if multiple groups of sinusoidal phase-shifted fringes are projected, that is, there is more than one group of frequencies, a time-based phase unwrapping algorithm can be used.

[0015] In the process of calculating the absolute phase, the confidence of each pixel can be calculated as a basis for subsequent evaluation of the quality of the pixel. The confidence formula is as follows.

[0016]

[0017] S4. The present invention proposes a sub-aperture selection algorithm based on phase contribution weights, the purpose of which is to avoid the detection speed being reduced due to slow data processing caused by the participation of all sub-apertures in fusion. The model starts with the central sub-aperture, and the specific steps are as follows.

[0018] 1. Traverse the central sub-aperture and find all invalid phase inf pixel points set P. The invalid pixel point conditions meet A or B. The formula is as follows.

[0019]

[0020]

[0021]

[0022] Where A represents a set of pixels whose absolute phase is inf, B represents a set of pixels whose confidence is greater than a certain threshold M1, (x, y) represents the pixel coordinates of the subaperture, and (u, v) represents the angular coordinates of the subaperture.

[0023] 2. Construct a sub-aperture flag matrix C with an initial value of 0. Traverse each position coordinate in P and determine whether the pixel position (x', y') in other sub-apertures is legal after the coordinate mapping f (x, y). The formula is as follows.

[0024]

[0025]

[0026]

[0027] 3. Count the fusion weight of each sub-aperture. The fusion weight is measured by the inverse order of the sum in S. The sub-apertures finally selected can be a fixed number of sub-apertures at the front of the weight distribution or selected to be greater than a certain weight threshold. The formula is as follows.

[0028]

[0029] S5. The present invention proposes a sub-aperture fusion algorithm that combines angle, distance and similarity. Corresponding kernel functions are established based on the angle, distance and similarity of the sub-aperture, and the absolute phase of the central sub-aperture is reconstructed to obtain better imaging quality. At the same time, after fusion, shadow occlusion between targets can be suppressed to a certain extent. The specific steps are as follows.

[0030] 1. Construct the kernel function K1 based on the angular characteristics of the sub-aperture and construct the bilateral convolution window. The formula is as follows.

[0031]

[0032] Among them, u c , v c represents the central angle coordinate, is an input parameter.

[0033] 2. Construct the kernel function K2 based on the brightness variance. If the phase of a pixel is invalid, the brightness average value of the world coordinates mapped by the pixel position (x, y) at all sub-aperture positions (x', y') is required to replace it. The formula is as follows.

[0034]

[0035] in represents the absolute phase average value of the point (x',y'), is an input parameter.

[0036] 3. Based on the absolute phase map, the kernel function K3 is constructed to measure the similarity of the absolute phase.

[0037]

[0038] where u x ,u y represents the average phase value at (x,y), , represents the variance, represents covariance, C1, C2, C3 are constants.

[0039] 4. Combine all the above kernel functions into K.

[0040]

[0041] 5. Considering that sub-apertures have different angles, there will be a certain pixel offset in the sub-aperture images. When fusion is performed, the influence of different sub-aperture areas is taken into consideration. The formula is as follows.

[0042]

[0043] Where W dis the area window where point p is located, and D(p, q) is the distance kernel function.

[0044]

[0045] Where is the input parameter.

[0046] 6. The final fusion phase calculation formula is as follows.

[0047]

[0048] If the input data is changed from phase to confidence, the confidence after fusion can be obtained .

[0049]

[0050] S6. After height fusion, the absolute phase data in a single perspective is obtained, and then height mapping is performed to obtain a three-dimensional point cloud data based on the calibration parameters.

[0051] S7. The present invention proposes a phase-shift profilometry based on multi-angle sinusoidal fringe light field fusion, which can further avoid detection errors caused by occlusion between detection targets. A coaxial or approximately coaxial projection is used, and multiple light field imaging systems at different angles are installed on the side. These multi-angle light field imaging systems can be exposed synchronously and collect light field information at multiple angles at the same time. The three-dimensional point cloud data and confidence information at each viewing angle are calculated according to steps S1 to S6, and then filled into an overall three-dimensional point cloud matrix according to the mapping relationship. If there are points at the same time at multiple viewing angles, the height of the viewing angle with the best confidence is selected for filling; if there are no repeated points, they can be filled directly.

[0052] Among them, step S2 includes step S8. In order to calibrate the image of the micromirror array, the micromirror array needs to be calibrated in advance, specifically including using a camera to take multiple images of a color card at a fixed position through the micromirror array, and then averaging these images to obtain a sub-aperture image array. First, the image is binarized, and then the edge feature points of the sub-aperture image are extracted, and a straight line is fitted for the sub-aperture feature points in the same row / column. The rotation matrix and translation matrix of each sub-aperture are calculated through the fitted straight line as the calibration parameters of each sub-aperture image.

[0053] Among them, step S4 includes step S9. In order to be able to perform coordinate mapping between sub-aperture images and fill in the accurate position of height, camera calibration needs to be performed in advance. Camera calibration includes calibrating the intrinsic parameters and extrinsic parameters of the camera. Finally, a set of mapping parameters will be generated for each sub-aperture; if there are multiple viewing angles, it is necessary to synchronously expose and capture the camera calibration plate at the same position, and a set of sub-aperture mapping parameters will be generated at each viewing angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flow chart of phase shift profilometry based on sinusoidal fringe light field fusion provided by the present invention.

[0055] Figure 2 A schematic diagram of a sinusoidal fringe light field projection and shooting device provided by the present invention.

[0056] Figure 3 A microlens array image calibration flow chart provided by the present invention.

[0057] Figure 4 A schematic diagram of camera calibration based on phase-shift profilometry with multi-angle sinusoidal fringe light field fusion provided by the present invention.

[0058] Figure 5 A schematic diagram of phase-height mapping calibration of phase-shift profilometry based on multi-angle sinusoidal fringe light field fusion provided by the present invention.

[0059] Figure 6 A flow chart of phase shift profilometry based on multi-angle sinusoidal fringe light field fusion provided by the present invention.

[0060] Description of Figure Numbers.

[0061] S1 ~ S2: Phase shift profilometry flow chart of sinusoidal fringe light field fusion; 20: projection mechanism; 21: light field imaging system based on microlens array; 210: camera; 211: microlens array; 22: sinusoidal stripes; 30 ~ 35: microlens array image calibration flow chart; 40: camera calibration plate; 41: coaxial or quasi-coaxial projection mechanism; 42,43,44,45: light field imaging system based on microlens array at multiple different angles; 50: phase-height mapping calibration plate; 52,53,54,55: light field imaging system based on microlens array at multiple different angles; 60 ~ 64: Phase shift profilometry flow chart of multi-angle sinusoidal fringe light field fusion. DETAILED DESCRIPTION

[0062] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings to illustrate the technical solution of the present invention in detail. The main process is shown in the attached drawings. Figure 1 .

[0063] S1. A sinusoidal fringe light field projection and shooting device provided by the present invention is similar to the attached Figure 2 As shown: Specifically, a projection mechanism is used to project one or more groups of sinusoidal stripes onto the surface of the object to be measured. After the light is reflected, it is collected by a microlens array 211 and then by a surface array camera 210. At this time, the camera not only collects the light intensity information of each light, but also collects the angle information of each light, and obtains multiple sub-aperture stripe light images at different angles. Based on the dual-plane model, the four-dimensional light field is parameterized. A four-dimensional light field can be parameterized as L ux , where x = (x, y) T , u =(u, v) T Represent the spatial coordinates and angular coordinates of the light field respectively. The formula is characterized by the cross-section of the light in two planes.

[0064] S2. The present invention proposes a method for calibrating sub-aperture images. Since the microlens array cannot be completely parallel to the CCD plane, the sub-aperture image will have a certain offset. It is necessary to perform image calibration in advance to obtain calibration parameters, and correct the image based on the calibration parameters to facilitate subsequent processing.

[0065] S3. The present invention adopts the sinusoidal phase shift method to solve the phase. The correlation between multiple fringe images can effectively reduce the influence of noise and improve the precision and accuracy of phase acquisition. Assuming that N sinusoidal fringe images are projected, let I i (x,y) represents the light intensity of the i-th fringe image at pixel coordinate (x, y).

[0066]

[0067] Where a and b are the background radiation intensity and modulated radiation intensity of the recorded light field, respectively. is the phase information modulated by the object height, The relative phase can be demodulated by the fringe analysis technique, and the formula is as follows.

[0068]

[0069] The range of the relative phase after the inverse tangent operation is (-π, π). If the phase exceeds one phase cycle, only the phase compressed within one phase cycle can be obtained. Here, the phase needs to be expanded to obtain the absolute phase. , phase unwrapping requires determining the fringe order K, as shown in the following formula.

[0070]

[0071] Among them, if a group of sinusoidal phase-shifted fringes is projected, that is, there is only one group of frequencies, a space-based phase unwrapping algorithm can be used; if multiple groups of sinusoidal phase-shifted fringes are projected, that is, there is more than one group of frequencies, a time-based phase unwrapping algorithm can be used.

[0072] In the process of calculating the absolute phase, the confidence of each pixel can be calculated as a basis for subsequent evaluation of the quality of the pixel. The confidence formula is as follows.

[0073]

[0074] S4. The present invention proposes a sub-aperture selection algorithm based on phase contribution weights, the purpose of which is to avoid the detection speed reduction caused by too slow data processing due to the participation of all sub-apertures in fusion. The model starts with the central sub-aperture, and the specific steps are as follows:

[0075] 1. Traverse the central sub-aperture and find all invalid phase inf pixel points P. The invalid pixel point conditions meet A or B. The formula is as follows:

[0076]

[0077]

[0078]

[0079] Where A represents the set of pixels with an absolute phase of inf, B represents the set of pixels with a confidence greater than a certain confidence threshold M1, (x, y) represents the pixel coordinates of the subaperture, and (u, v) represents the angular coordinates of the subaperture;

[0080] 2. Construct the sub-aperture flag matrix C with an initial value of 0, traverse each position coordinate in P, and determine whether the pixel position is legal after the coordinate mapping f (x, y) in other sub-apertures (x', y'). The formula is as follows:

[0081]

[0082]

[0083]

[0084] 3. Count the fusion weight of each sub-aperture. The fusion weight is measured by the inverse order of the sum in S. The sub-apertures finally selected can be a fixed number of sub-apertures at the front of the weight distribution or selected to be greater than a certain weight threshold. The formula is as follows.

[0085]

[0086] S5. The present invention proposes a sub-aperture fusion algorithm that combines angle, distance and similarity. Corresponding kernel functions are established based on the angle, distance and similarity of the sub-aperture, and the absolute phase of the central sub-aperture is reconstructed to obtain better imaging quality. At the same time, after fusion, shadow occlusion between targets can be suppressed to a certain extent. The specific steps are as follows:

[0087] 1. Construct the kernel function K1 based on the angular characteristics of the sub-aperture and construct the bilateral convolution window. The formula is as follows:

[0088]

[0089] Among them, u c , v c represents the central angle coordinate, is the input parameter;

[0090] 2. Construct the kernel function K2 based on the brightness variance. If the phase of a pixel is invalid, the brightness average value of the world coordinates mapped to the pixel position (x, y) at all sub-aperture positions (x', y') is required to replace it. The formula is as follows:

[0091]

[0092] in represents the absolute phase average value of the point (x',y'), is the input parameter;

[0093] 3. Construct kernel function K3 based on the absolute phase map to measure the similarity of absolute phase;

[0094]

[0095] where u x ,u y represents the average phase value at (x,y), , represents the variance, represents covariance, C1, C2, C3 are constants;

[0096] 4. Combine all the above kernel functions into K;

[0097]

[0098] 5. Considering that sub-apertures have different angles, there will be a certain pixel offset in the sub-aperture images. When fusion is performed, the influence of different sub-aperture areas is taken into consideration. The formula is as follows:

[0099]

[0100] Where W dis the domain window where point p is located, and D(p, q) is the distance kernel function;

[0101]

[0102] in is the input parameter;

[0103] 6. The final fusion phase calculation formula is as follows:

[0104]

[0105] If the input data is changed from phase to confidence, the confidence after fusion can be obtained .

[0106]

[0107] S6. After height fusion, the absolute phase data in a single perspective is obtained, and then height mapping is performed to obtain a three-dimensional point cloud data based on the calibration parameters.

[0108] S7. The present invention proposes a phase shift profilometry based on multi-angle sinusoidal fringe light field fusion, which can further avoid the detection error caused by the occlusion between the detection targets. Figure 4 As shown: a coaxial or approximately coaxial projection 41 is used, and multiple light field imaging systems 42, 43, 44, and 45 at different angles are installed on the side. These multi-angle light field imaging systems can be exposed synchronously and collect light field information at multiple angles at the same time. According to steps S1 to S6, the three-dimensional point cloud data and confidence information at each viewing angle are calculated, and then filled into an overall three-dimensional point cloud matrix according to the mapping relationship. If there are points at multiple viewing angles at the same time, the height of the viewing angle with the best confidence is selected for filling; if there are no repeated points, they can be filled directly.

[0109] Wherein, step S2 includes step S8. In order to calibrate the image of the micromirror array, the micromirror array needs to be calibrated in advance, as shown in the attached Figure 3 As shown, a camera is used to take multiple images of a color card at a fixed position through a micromirror array 30, and these images are averaged to obtain a sub-aperture image array 31. The image is first binarized 32, and then edge feature points of the sub-aperture image are extracted 33. Straight line fitting is performed on the sub-aperture feature points in the same row / column 34, and the rotation matrix and translation matrix of each sub-aperture are calculated from the fitted straight line as calibration parameters 35 for each sub-aperture image.

[0110] Step S4 includes step S9. In order to map coordinates between sub-aperture images and fill in the accurate positions of heights, camera calibration needs to be performed in advance, similar to the attached Figure 4 As shown: the camera calibration includes calibrating the intrinsic parameters and extrinsic parameters of the camera, and finally each sub-aperture will generate a set of mapping parameters; if there are multiple viewing angles 42, 43, 44, 45, it is necessary to synchronously expose and capture the camera calibration plate 40 at the same position, and a set of sub-aperture mapping parameters will be generated at each viewing angle.

[0111] Step S4 includes step S9. In order to map the phase to the height for each light ray, a phase-height mapping calibration is required. Considering that the relative positions of the system projection mechanism and the imaging system are fixed, only the relationship between the absolute phase and the depth value needs to be calibrated, similar to the attached Figure 5 As shown: the projection mechanism 51 can be moved up and down, and the calibration block 50 can also be moved up and down. For each projection at each layer height, the light field imaging systems 52, 53, 54, and 55 of all viewing angles need to be exposed and imaged synchronously, and the phase is calculated. Then, the mapping of the phase to the height is calculated based on the linear / multiple term fitting. The linear fitting adopts a segmented fitting method. The number of segments n is determined by the height measurement range. The number of segments in the middle range can be more, and the number of segments at the two ends can be relatively less. Finally, a set of phase-to-height mapping parameters will be generated at each viewing angle.

Claims

1. A three-dimensional measurement method based on phase shift profilometry with multi-angle sinusoidal fringe light field fusion, characterized in that: The three-dimensional high-speed and high-precision detection of extremely small objects on diffuse reflective surfaces is achieved, as follows: S1. Sinusoidal stripe light field projection and shooting: A projection mechanism is used to project one or more groups of sinusoidal stripes onto the surface of the object to be measured. After the light is reflected, it is collected by a microlens array and then collected by a surface array camera. At this time, the camera not only collects the light intensity information of each light ray, but also collects the angle information of each light ray, and obtains multiple sub-aperture stripe light images at different angles. Based on the dual-plane model, the four-dimensional light field is parameterized. A four-dimensional light field can be parameterized as L ux , where x = (x, y) T ,u=(u,v) T Represent the spatial coordinates and angular coordinates of the light field respectively. The formula is characterized by the cross-section of the light in two planes; S2. Sub-aperture image calibration: Since the microlens array cannot be completely parallel to the CCD plane, the sub-aperture image will have a certain offset. Image calibration needs to be performed in advance to obtain calibration parameters. The image is corrected based on the calibration parameters for subsequent processing; S3. Use the sinusoidal phase shift method to solve the phase: Assume that N sinusoidal fringe images are projected, let Ii(x,y) represent the light intensity of the i-th fringe image at the pixel coordinate (x,y): Among them, a and b are the background radiation intensity and modulated radiation intensity of the recorded light field, respectively. φ(x, y) is the phase information modulated by the object height, δ i is the phase shift of the i-th fringe. The relative phase can be demodulated by fringe analysis technology. The formula is as follows: The range of the relative phase after the inverse tangent operation is (-π, π); if the phase exceeds one phase cycle, only the phase compressed within one phase cycle can be obtained. Here, the phase needs to be expanded to obtain the absolute phase Φ(x, y). Phase expansion requires determining the fringe order K, as shown in the following formula: Φ(x,y)=φ(x,y)+2πK(x,y) Among them, if a group of sinusoidal phase-shifted fringes is projected, that is, there is only one group of frequencies, a space-based phase unwrapping algorithm can be used; if multiple groups of sinusoidal phase-shifted fringes are projected, that is, there are more than one group of frequencies, a time-based phase unwrapping algorithm can be used; In the process of calculating the absolute phase, the confidence of each pixel is calculated as a basis for subsequent evaluation of the quality of the pixel. The confidence formula is as follows: M(x,y)=Imax(x,y)-Imin(x,y) S4, sub-aperture selection based on phase contribution weight: The purpose is to avoid all sub-apertures participating in the fusion, which causes data processing to be too slow and leads to a reduction in detection speed. The model starts based on the central sub-aperture. The specific steps are as follows; Traverse the central sub-aperture and find all invalid phase inf pixel points P. The invalid pixel point conditions meet A or B. The formula is as follows: TO u，v (x,y)={x,y|Φ(x,y)=inf} u，v B u，v (x,y)={x,y|M(x,y)<M1} u，v P=A u＝0，v＝0 ∪B u＝0，v＝0 Where A represents the set of pixels with an absolute phase of inf, B represents the set of pixels with a confidence greater than a certain threshold M1, (x, y) represents the pixel coordinates of the subaperture, and (u, v) represents the angular coordinates of the subaperture; Construct the sub-aperture flag matrix C with an initial value of 0, traverse each position coordinate in P, and determine whether the pixel position is legal after the coordinate mapping f(x,y) in other sub-apertures (x',y'). The formula is as follows: (x′,y′) u，v =f u，v (x,y) The fusion weight of each sub-aperture is counted. The fusion weight is measured by the reciprocal order of the sum function S. The sub-aperture finally selected can be a fixed number of sub-apertures at the front of the weight distribution, or a sub-aperture greater than a certain weight threshold. The formula is as follows: S5. Combining angle, distance and similarity for sub-aperture fusion: Establish corresponding kernel functions based on the angle, distance and similarity of the sub-aperture, reconstruct the absolute phase of the central sub-aperture, and obtain better imaging quality. At the same time, after fusion, shadow occlusion between targets can be suppressed to a certain extent. The specific steps are as follows: Based on the angular characteristics of the sub-aperture, the kernel function K1 is constructed to construct a bilateral convolution window. The formula is as follows: Among them, uc,vc represents the central angle,coordinate, and σ1 is the input parameter; The kernel function K2 is constructed based on the brightness variance. If the phase of a pixel is invalid, it needs to be replaced by the average brightness of the world coordinates mapped to the pixel position (x, y) at all sub-aperture positions (x', y'). The formula is as follows: in The absolute phase average value σ2 of the representative point (x', y') is the input parameter; The kernel function K3 is constructed based on the absolute phase map to measure the similarity of the absolute phase; where u x ,u y represents the average phase value σ at (x,y) x , σ y represents the variance, σ xy represents covariance, C1, C2, C3 are constants; Combine all the above kernel functions into K: K(x,y,u,v)=K1(u,v)K2(x,y) u，v K3(x,y) Considering that sub-apertures have different angles, there will be a certain pixel offset in the sub-aperture images. When fusion is performed, the influence of different sub-aperture areas is taken into consideration. The formula is as follows: Where W d is the area window where point p is located, and D(p,q) is the distance kernel function; Where σ3 is the input parameter, The final fusion phase calculation formula is as follows: If the input data is changed from phase to confidence, the confidence after fusion can be obtained S6, after height fusion, obtain the absolute phase data under a single perspective, then perform height mapping, and obtain a three-dimensional point cloud data based on the calibration parameters; S7. Use phase-shift profilometry based on multi-angle sinusoidal fringe light field fusion: use a coaxial or nearly coaxial projection, and install multiple light field imaging systems at different angles on the side. These multi-angle light field imaging systems can be exposed synchronously and collect light field information from multiple angles at the same time. According to steps S1 to S6, the three-dimensional point cloud data and confidence information at each viewing angle are calculated, and then filled into an overall three-dimensional point cloud matrix according to the mapping relationship. If there are points at the same time at multiple viewing angles, the height of the viewing angle with the best confidence is selected for filling; if there are no repeated points, they can be filled directly.

2. The method according to claim 1, characterized in that Step S2 includes: S2-1. To calibrate the image of the micromirror array, the micromirror array needs to be calibrated in advance. Specifically, a camera needs to be used to take multiple images of a color card at a fixed position through the micromirror array, and then these images are averaged to obtain a sub-aperture image array. First, the image is binarized, and then the edge feature points of the sub-aperture image are extracted, and a straight line is fitted for the sub-aperture feature points in the same row / column. The rotation matrix and translation matrix of each sub-aperture are calculated through the fitted straight line as the calibration parameters of each sub-aperture image.

3. The method according to claim 1, characterized in that Step S4 includes: S4-1. In order to map coordinates between sub-aperture images and fill in the accurate positions of heights, camera calibration needs to be performed in advance. Camera calibration includes calibrating the intrinsic and extrinsic parameters of the camera. Finally, a set of mapping parameters will be generated for each sub-aperture. If there are multiple viewing angles, the camera calibration plate at the same position needs to be exposed and imaged synchronously, and a set of sub-aperture mapping parameters will be generated for each viewing angle. Phase-height mapping calibration is performed to map the phase to the height for each light ray. Considering that the relative positions of the system projection mechanism and the imaging system are fixed, it is only necessary to calibrate the relationship between the absolute phase and the depth value. The distance between the projection mechanism and the calibration block is changed by moving the projection mechanism or the calibration block up and down. For each projection at each layer of height, the light field imaging system of all angles needs to be exposed and imaged synchronously, the phase is calculated, and then the phase to height mapping is calculated based on linear or multi-term fitting. The linear fitting adopts a segmented fitting method, and the number of segments n is determined by the height measurement range. There are relatively more segments in the middle range and relatively fewer segments at both ends. Finally, a set of phase to height mapping parameters will be generated at each viewing angle.

Images