Spectrally separated region adaptive fringe projection three-dimensional measurement method

Abstract

A kind of region adaptive stripe projection three-dimensional measurement method of spectral separation, comprising: the sinusoidal phase shift stripe pattern of transverse and longitudinal is projected to the surface of multiple measured objects, and the stripe sequence is collected with camera;Respectively, object contour mask and saturation mask are obtained;The white region coordinates of contour mask are converted to projector coordinates, and projection mask is obtained;Single object projection mask image is obtained using connected domain marking algorithm;Using the region marked and saturation mask, set two groups of mask region to generate longitudinal sinusoidal phase shift stripe pattern of adaptive intensity, and two groups of region stripe are projected to the surface of measured object in turn, while collecting spectral separation region stripe image;The stripe pattern of different channel color corresponding to the object is taken independently phase demodulation and phase fusion, and three-dimensional reconstruction is carried out according to the phase result after fusion;The present application significantly improves the measurement efficiency under the premise of ensuring to eliminate stripe aliasing interference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of structured light three-dimensional measurement, and in particular to a spectrum separation region adaptive fringe projection three-dimensional measurement method. BACKGROUND

[0002] In the modern industry and scientific research field, accurate three-dimensional measurement of objects is a crucial technology, especially in the manufacturing, medical, cultural heritage protection, virtual reality and other fields, the requirements for measurement accuracy and efficiency are increasing. In the field of three-dimensional measurement, when there are specular reflection objects and diffuse reflection objects in the scene at the same time, the traditional structured light measurement technology will face the following two challenges: 1) The specular reflection object will cause local light intensity saturation, which will destroy the structured light fringe and make it difficult to recover accurate phase information; 2) The specular reflection object may reflect the light to the diffuse reflection object surface twice, which will introduce other interference light on the diffuse reflection object surface, showing fringe aliasing phenomenon, causing measurement error. Some existing solutions all reduce measurement error through post-phase compensation, signal filtering and other methods, but cannot avoid the occurrence of this phenomenon from the source. Some other methods based on deep learning can realize aliasing fringe demixing and improve measurement accuracy. However, a large amount of training data set needs to be prepared, which is very tedious and not conducive to practical application.

[0003] Therefore, it is urgent to develop a flexible and simple method that can avoid multiple reflections and fringe aliasing from the source, and realize high-precision three-dimensional measurement of specular reflection and diffuse reflection objects at the same time. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provides a spectrum separation region adaptive fringe projection three-dimensional measurement method. The method designs a region adaptive projection strategy to alternately project red and blue fringe patterns of two different spectra in different measured object regions. During the time of projecting the red and blue fringe groups, the camera is used to capture images of two spectral colors at the same time through one-time exposure. After color channel separation, the phase extraction is carried out independently, which fundamentally avoids the reflection light interference between complex materials, thereby improving the three-dimensional measurement accuracy and reliability of the complex multi-material measurement scene.

[0005] To achieve the above purpose, the technical scheme provided by the present application is as follows:

[0006] A spectrum separation region adaptive fringe projection three-dimensional measurement method, comprising:

[0007] Projecting a fringe image with transverse and longitudinal sinusoidal phase shift to the surface of the measured object to obtain the transverse and longitudinal phase distribution results respectively;

[0008] According to the transverse and longitudinal sinusoidal phase-shifted fringe images, a YOLO network is used for contour segmentation of the measured object image to obtain a contour mask Mc;

[0009] In the transverse and longitudinal sinusoidal phase-shifted fringe images, a light intensity saturation region is detected, and a saturation mask Ms is obtained by threshold division;

[0010] According to the contour mask Mc and the saturation mask Ms, a projection mask image is generated;

[0011] A longitudinal sinusoidal phase-shifted fringe image is generated in the projection mask region;

[0012] The region fringe images are sequentially acquired, and phase demodulation and phase fusion are performed, and a phase-height mapping model is used for three-dimensional reconstruction.

[0013] The transverse and longitudinal sinusoidal phase-shifted fringe images are projected onto the surface of the measured object to obtain transverse and longitudinal phase distribution results, including:

[0014] The transverse and longitudinal phase-shifted fringe images acquired by the camera are respectively represented as:

[0015]

[0016] wherein, represents a transverse phase-shifted fringe image; represents a longitudinal phase-shifted fringe image; A and B represent background light intensity and fringe modulation, respectively; F is the fringe frequency; i is the phase shift sequence number, i = 1, 2, 3; (u c ,v c ) are the pixel horizontal and vertical coordinates of the camera-acquired image; The transverse phase-shifted fringe image sequence

[0017] and the longitudinal phase-shifted fringe image sequence

[0018] The transverse and longitudinal phase distribution results are calculated using the transverse and longitudinal fringe image sequences, respectively.

[0019] According to the transverse and longitudinal sinusoidal phase-shifted fringe images, a YOLO network is used for contour segmentation of the measured object image to obtain a contour mask Mc, including:

[0020] The transverse and longitudinal fringe images are input into the YOLO network, and the YOLO network outputs an object contour segmentation image;

[0021] The segmentation image is a binary mask image, which is used as the contour mask Mc.

[0022] The step of detecting saturated light intensity regions in the horizontally and vertically sinusoidally shifted stripe image and obtaining a saturation mask Ms by setting a threshold includes:

[0023] The texture image I of the measured object is obtained by averaging the acquired horizontal and vertical phase distribution results. bg ;

[0024] Based on a preset grayscale threshold, the texture image I of the object under test is processed. bg Pixel-by-pixel traversal: Pixels with a grayscale value greater than the preset grayscale threshold are identified as saturated pixels, until a saturated mask image M is obtained. s .

[0025] The step of generating a projection mask image based on the contour mask Mc and the saturation mask Ms includes:

[0026] The contour mask M c With the saturation mask M s The coordinates are transformed into the projector coordinate system to generate a projection mask image, and the coordinate transformation is performed according to the following formula:

[0027] v p =Φ hor (u c ,v c )×P / (2π)

[0028] u p =Φ ver (u c ,v c )×P / (2π)

[0029] Where, Φ hor (u c ,v c ), Φ ver (u c ,v c ) represents the horizontal and vertical phases of the white area points in the mask image Mc or Ms, and P represents the stripe frequency.

[0030] The step of generating a longitudinal sinusoidal phase-shifted fringe pattern in the projection mask region includes:

[0031] The projection mask M p The objects in the image are labeled with consecutive numbers, and the strong light projection mask M is arranged according to odd and even numbers. p The white area is divided into two groups. The first group consists of masks with odd-numbered indices, named M. pB The second group consists of masks with even-numbered indices, named M. pR ; respectively in M pB and M pRGenerate a three-step phase-shifting stripe sequence and As shown in the following formula:

[0032]

[0033] Among them, (u p ,v p ) represents the coordinates of the projected pixel; A and B are both constants; F represents the fringe frequency; i represents the i-th fringe pattern in the three-step phase-shifting method, i = 1, 2, 3;

[0034] The area stripe pattern is projected alternately in the following order: {P1 pB P1 pR P2 pB P2 pR P3 pB P3 pR};

[0035] In low-light mask M q Low-intensity three-step phase-shifting fringe sequence P is generated i q Parameters A and B are both stronger than photomask M. p Reduce by half;

[0036] will sequence and The stripes in the corresponding low-light mask area are replaced with P i q The updated adaptive intensity fringe sequences were obtained respectively. and

[0037] Adaptive intensity stripe sequence and All data are loaded into the projector and projected onto the surface of the object being measured.

[0038] Projection sequence When using a blue light source, the projection sequence Use red light source.

[0039] The sequential acquisition of regional fringe patterns, followed by phase demodulation and phase fusion, includes:

[0040] The camera exposure time is set to be greater than or equal to the projection time of two adjacent area stripe patterns, enabling the camera to simultaneously capture a sequence while acquiring one image. and Two regional stripe images;

[0041] The camera acquires images at least once in sequence, and saves at least one spectrally separated phase-shift fringe image for each acquisition;

[0042] Separate the collected spectral separation fringe pattern by RGB channel, and obtain high-quality phase results without multiple reflection light interference;

[0043] Obtain complete phase results by using the high-quality phase results, and perform three-dimensional reconstruction.

[0044] Separate the collected spectral separation fringe pattern by RGB channel, and obtain two images of R channel and B channel to independently perform the following phase extraction:

[0045]

[0046] wherein, Ф R and Ф B are the phases obtained by red and blue channels respectively, I 1R -I 3R is the red channel fringe image, I 1B -I 3B is the blue channel fringe image.

[0047] The three-dimensional reconstruction by using the phase-height mapping model comprises:

[0048] The three-dimensional reconstruction is performed by the following mapping formula:

[0049]

[0050] wherein, H(x,y) represents an actual depth value at coordinates (x,y), a(x,y), b(x,y), c(x,y) are phase-height mapping parameters, and are obtained by calibration; is a phase distribution of the object.

[0051] Compared with the prior art, the scheme has the following advantages:

[0052] The method can fundamentally avoid the fringe aliasing problem caused by secondary reflection between different materials. The measurement accuracy and robustness are significantly improved. At the same time, the adaptive intensity fringe can avoid light intensity saturation, further reducing the phase error. Moreover, the method fully utilizes the independent perception ability between different color channels, and uses two spectral channels far apart from each other, i.e., blue and red, to alternately project area fringes. At the same time, the camera exposure time is strictly matched with the fringe alternately projected time, so that the camera exposure is realized once, and two color fringe images are collected at the same time, and are independently processed. On the premise of eliminating the interference of fringe aliasing, the measurement efficiency is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the services required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of the measurement method of the present invention;

[0055] Figure 2 This is a flowchart of the method of the present invention;

[0056] Figure 3 This provides a general overview of the entire image processing process.

[0057] Figure 4 This describes the actual process and the final result of the photos captured by the camera when red and blue light alternate at different times.

[0058] Figure 5 For the acquired color spectral separation stripe image;

[0059] Figure 6 Extract the grayscale image of the R channel for color stripe separation;

[0060] Figure 7 Extract the grayscale image of the B channel for color stripe separation; Detailed Implementation

[0061] The present invention will be further described below with reference to specific embodiments:

[0062] Before proceeding further, the three-dimensional measurement system used in the spectral separation region adaptive fringe projection three-dimensional measurement method described in this embodiment of the invention will be specifically explained:

[0063] like Figure 1 As shown, the structured light projection 3D measurement system includes a computer 1, a region projection stripe system, and a 3D measurement system. Figure 2 3. Projector optical engine; 4. Measured scene; 5. Color industrial camera; 6. Camera captures color stripes. Figure 6 The region projection stripes are generated by computer 1. Figure 2 The signal is transmitted to the projection optical engine 3 and then projected onto the surface of the scene being measured 4. Subsequently, the color industrial camera 5 captures the color stripes. Figure 6 Then, the data is transmitted to computer 1 for data processing.

[0064] The working principle of the structured light projection technology based on regional spectral separation described in this embodiment is as follows:

[0065] S1, project the transverse and longitudinal sinusoidal phase-shifted fringe patterns to the surface of the measured object, and collect the fringe image sequence by using a camera, wherein the set of transverse and longitudinal three-step phase-shifted fringe images, a total of 9 images, can be respectively represented as:

[0066]

[0067] wherein represents the transverse phase-shifted fringe pattern, represents the longitudinal phase-shifted fringe pattern, A and B represent the background light intensity and the fringe modulation respectively, F is the fringe frequency, i is the phase-shift sequence number, i = 1, 2, 3; (u c ,v c ) are the pixel horizontal and vertical coordinates of the camera collected image;

[0068] S2, use the YOLO network to perform contour segmentation on the image of the measured object to obtain a contour mask Mc; pass the three-step phase image through the YOLO network to obtain a binary mask contour segmentation image of the object, that is, a contour mask M c ;

[0069] S3, detect the light intensity saturation area in the original image, and obtain a saturation mask M s by using a suitable threshold value;

[0070] S4, convert the coordinates of the contour mask M c and the saturation mask M s to the projector coordinate system respectively to generate a projection mask image. Use the transverse and longitudinal phases obtained in step S1 to perform coordinate conversion according to the following formula:

[0071] v p =Φ hor (u c ,v c )×P / (2π)

[0072] u p =Φ ver (u c ,v c )×P / (2π);

[0073] wherein Φ hor (u c ,v c ), Φ ver (u c ,v c ) represent the transverse phase and the longitudinal phase of the white area point in the mask image M c or M s , and P represents the fringe frequency.

[0074] S5, generate a longitudinal sinusoidal phase-shifted fringe pattern in the projection mask area;

[0075] S501. Using a connected component labeling algorithm, the projection mask image M... p The objects in the image are labeled with consecutive numbers. The strong light projection mask M is arranged according to odd and even numbers. p The white area is divided into two groups. The first group consists of masks with odd-numbered indices, named M. pB The second group consists of masks with even-numbered indices, named M. pR In M respectively pB and M pR Generate a three-step phase-shifting stripe sequence and As shown in the following formula:

[0076]

[0077] Where P i It can be and (u p ,v p ) represents the projection pixel coordinates, A and B are both set to 127.5, F represents the fringe frequency, and i represents the i-th fringe pattern in the three-step phase shift method, i = 1, 2, 3.

[0078] S502, Projection Blue light source is used during the sequence, projecting A red light source is used during the sequence. The two sets of region stripe patterns are projected alternately in the following order: {P1} pB P1 pR P2 pB P2 pR P3 pB P3 pR}

[0079] S503, in the low-light mask M q Low-intensity three-step phase-shifting fringe sequence P is generated i q , where parameters A and B are both reduced by half;

[0080] S504, will and The stripes in the corresponding low-light mask area are replaced with P i q The updated adaptive intensity fringe sequence is obtained. and Finally, and All data are loaded into the projector and projected onto the surface of the object being measured.

[0081] S6. The camera sequentially acquires regional stripe patterns and performs phase demodulation and phase fusion.

[0082] S601, set the camera exposure time greater than or equal to the time of adjacent two regional fringe pattern projection. For example, the projection time of adjacent two regional fringe patterns is 15 ms respectively, then the camera exposure time is set to be greater than or equal to 30 ms, so that the camera captures two regional fringe images at the same time in the process of collecting one image, realizing the existence of blue and red spectrum separated fringe images in the camera collected image at the same time; and two regional fringe images, realizing the existence of blue and red spectrum separated fringe images in the camera collected image at the same time;

[0083] S602, the camera collects three times in turn and saves three spectrum separated phase shift fringe images;

[0084] S603, separate the three collected spectrum separated fringe images in RGB channel, take two images of R channel and B channel to independently perform the following phase extraction:

[0085]

[0086] wherein, Ф R and Ф B are the phases obtained by red and blue channels respectively, I 1R ~ I 3R are red channel fringe images, I 1B ~ I 3B are blue channel fringe images.

[0087] S604, add Ф R and Ф B to obtain the complete phase result, which is used for three-dimensional reconstruction;

[0088] S7, three-dimensional reconstruction is performed by using the phase-height mapping model.

[0089] The mapping formula is as follows:

[0090]

[0091] wherein, H(x,y) represents the actual depth value at coordinate (x,y), a(x,y), b(x,y), c(x,y) are phase-height mapping parameters, which are obtained by calibration. is the phase distribution of the object.

[0092] In summary, the present application discloses a kind of spectral separation regional adaptive fringe projection three-dimensional measurement method, first, the transverse and longitudinal sinusoidal phase shift fringe pattern is projected to the surface of multiple measured objects, and the fringe sequence is collected using camera.Then, through YOLO network and saturation detection algorithm, the object contour mask and saturation mask are obtained respectively.Then, the white region coordinates of the contour mask are converted to the projector coordinates, and the projection mask is obtained.Then, the connected domain marking algorithm is used to mark each independent object in the projection mask, and the single object projection mask image is obtained.Then, using the marked region and saturation mask, set two groups of mask region to generate adaptive intensity longitudinal sinusoidal phase shift fringe pattern, and project the two groups of region fringe to the surface of measured object with different color light source in turn, the camera exposure time is set to be greater than the projection time of the two groups of different color fringe, and the spectral separation region fringe image is collected simultaneously.Finally, the corresponding different channel color fringe of the object is taken for independent phase demodulation and phase fusion, and three-dimensional reconstruction is carried out according to the fused phase result;The present application makes full use of the independent perception ability between different color channels, and uses blue and red two spectral channels far apart from each other to project region fringe alternately.Meanwhile, the camera exposure time and fringe projection time are strictly matched, the camera exposure is realized once, and two color fringe images are collected simultaneously for independent processing.Under the premise of eliminating fringe aliasing interference, the measurement efficiency is significantly improved.

[0093] The above-mentioned embodiments are only the preferred embodiments of the present application, and do not limit the scope of the present application, so any changes made according to the shape and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method of spectrally separated region-adaptive stripe projection three-dimensional measurement, characterized in that, The method comprises the following steps: projecting a transverse and longitudinal sinusoidal phase-shifted fringe image onto a surface of an object to be measured to obtain a transverse and longitudinal phase distribution result respectively; segmenting the object image by using a YOLO network according to the transverse and longitudinal sinusoidal phase-shifted fringe image to obtain a contour mask Mc; detecting a light intensity saturation region in the transverse and longitudinal sinusoidal phase-shifted fringe image, and dividing the saturation mask Ms by setting a threshold value; generating a projection mask image according to the contour mask Mc and the saturation mask Ms; generating a longitudinal sinusoidal phase-shifted fringe image in the projection mask region; sequentially collecting the region fringe image, performing phase demodulation and phase fusion, and performing three-dimensional reconstruction by using a phase-height mapping model; the step of generating a longitudinal sinusoidal phase-shifted fringe image in the projection mask region comprises: The objects in the projection mask image are marked as continuous serial numbers, and the strong light projection masks M p are divided into 2 groups according to odd and even serial numbers, wherein the first group is a mask composed of odd serial numbers, named M pB , and the second group is a mask composed of even serial numbers, named M pR ; three-step phase shift fringe sequences and are generated in M pB and M pR respectively as shown in the following formula: wherein (u p ,v p ) represents a projection pixel coordinate; A and B are both constants; F represents a fringe frequency; and i represents the i-th fringe pattern in a three-step phase-shifting method, i = 1, 2, 3. The area stripe pattern is projected in the following order alternately: In weak light mask M q generating low intensity three-step phase shifting fringe sequences where parameters A and B are both stronger than light mask M p by half; Replacing the stripes corresponding to the weak light mask regions in the sequence and with respectively gives the updated adaptive intensity stripe sequences and Adaptive intensity fringe sequence And are loaded into the projector and projected onto the surface of the object under test; projection sequence when using a blue light source, the projection sequence when using a red light source; the step of sequentially collecting the region fringe image and performing phase demodulation and phase fusion comprises: The camera exposure time is set to be greater than or equal to the time of projection of two adjacent area fringe patterns, so that the camera can capture two area fringe patterns simultaneously in the process of collecting one pattern and ​ the camera sequentially collects at least once, and at least one spectrum-separated phase-shifted fringe image is saved respectively; performing RGB channel separation on the collected spectrum-separated fringe image to obtain a high-quality phase result without multiple reflection light interference; obtaining a complete phase result by using the high-quality phase result to perform three-dimensional reconstruction.

2. The method of spectrally resolved, region-adaptive stripe projection three-dimensional measurement according to claim 1, characterized in that the step of projecting a transverse and longitudinal sinusoidal phase-shifted fringe image onto a surface of an object to be measured to obtain a transverse and longitudinal phase distribution result respectively comprises: the transverse and longitudinal phase-shifted fringe images collected by the camera are represented as: where I hor i represents the lateral phase-shifted fringe pattern; I ver i represents the longitudinal phase-shifted fringe pattern; A and B represent the background light intensity and the fringe modulation, respectively; F is the fringe frequency; i is the phase-shift sequence number, i = 1, 2, 3; (u c ,v c ) are the pixel horizontal and vertical coordinates of the camera captured image; the lateral phase-shifted fringe pattern sequence {I1 hor ,I2 hor ,I3 hor} and the longitudinal phase-shifted fringe pattern sequence {I1 ver ,I2 ver ,I3 ver} are captured, respectively; the transverse and longitudinal phase distribution results are calculated by using the transverse and longitudinal fringe image sequences respectively.

3. The method of spectrally resolved, region-adaptive stripe projection three-dimensional measurement according to claim 1, characterized in that the step of segmenting the object image by using a YOLO network according to the transverse and longitudinal sinusoidal phase-shifted fringe image to obtain a contour mask Mc comprises: inputting the transverse and longitudinal fringe images into the YOLO network, and the YOLO network outputs an object contour segmentation image; the segmentation image is a binary mask image, which is used as the contour mask Mc.

4. The method of spectrally resolved, region-adaptive stripe projection three-dimensional measurement according to claim 1, characterized in that the step of detecting a light intensity saturation region in the transverse and longitudinal sinusoidal phase-shifted fringe image, and dividing the saturation mask Ms by setting a threshold value comprises: Using the collected transverse and longitudinal phase distribution results, the texture image I of the measured object is obtained after averaging operation bg ; According to a preset gray threshold, judging the texture image I of the measured object bg Pixel by pixel traversal: judging the pixel greater than the preset gray threshold as a saturated pixel until a saturated mask image M is obtained s .

5. The method of claim 1, wherein the method further comprises: the step of generating a projection mask image according to the contour mask Mc and the saturation mask Ms comprises: The contour mask M c is converted into the projector coordinate system respectively, a projection mask image is generated, and the coordinate conversion is performed according to the following formula: The contour mask M s is converted into the projector coordinate system respectively, a projection mask image is generated, and the coordinate conversion is performed according to the following formula: v p = Φ hor (u c , v c ) x P / (2π) u p = Φ ver (u c , v c ) x P / (2π); where Φ hor (u c , v c ) and Φ ver (u c , v c ) represent the lateral and longitudinal phase of the white region points in the mask image Mcor Ms, respectively, and P represents the fringe frequency.

6. The spectrum-separated region adaptive fringe projection three-dimensional measurement method according to claim 1, wherein: the collected spectrum-separated fringe image is subjected to RGB channel separation, and two images of R and B channels are taken to independently perform the following phase extraction: where Φ R and Φ B are the phases obtained for the red and blue channels, respectively, I 1R - I 3R is the red channel fringe image, and I 1B - I 3B is the blue channel fringe image.

7. The method of claim 1, wherein the method further comprises: the step of performing three-dimensional reconstruction by using a phase-height mapping model comprises: three-dimensional reconstruction is performed by using the following mapping formula: where H(x, y) represents the actual depth value at coordinates (x, y), a(x, y), b(x, y), c(x, y) are phase-height mapping parameters obtained by calibration; is the phase distribution of the object.

Images