A high-reflective object monocular vision scanning method based on adaptive stripes

Abstract

The application discloses a high-reflective object monocular vision scanning method based on adaptive stripes, which comprises the following steps: acquiring a plurality of image groups, wherein the image groups comprise an orthogonal stripe image group and a gray image group; determining a high-reflective area, determining the high-reflective area according to the gray image projected on the object surface and establishing a projection mask; coordinate matching, establishing the coordinate correspondence relationship between the image and the projector image; generating adaptive stripes, generating the adaptive stripes according to the projection mask and the coordinate correspondence relationship; three-dimensional reconstruction, projecting the adaptive stripe pattern, and completing the three-dimensional reconstruction after the phase algorithm and system calibration; the method can quickly and accurately generate adaptive stripes, and has the advantages of fast scanning speed, high point cloud integrity and small error.

Description

Technical Field

[0001] This invention relates to the field of monocular vision scanning, and more particularly to a monocular vision scanning method for highly reflective objects based on adaptive stripes. Background Technology

[0002] Fringe projection profilometry has been widely used in practical applications of 3D shape measurement in industrial manufacturing, reverse engineering, and other fields. In a typical measurement system, a standard coded sinusoidal fringe pattern is projected onto the object surface by a digital projector. The coded fringe, modulated by the object's height, is captured by a camera. After computer processing, the absolute phase is obtained, and the phase-height mapping relationship is determined through system calibration, ultimately completing the 3D reconstruction of the object's surface topography. However, as an optical sensing technology, fringe projection profilometry requires the intensity of reflected light from the object to be within the camera's maximum grayscale range; otherwise, oversaturation will occur, leading to phase resolution errors. Currently, traditional fringe projection profilometry is not ideal for imaging objects with highly reflective surfaces such as ceramics and metals. Because the intensity of light reflected from the object's surface exceeds the camera's maximum grayscale range (255 for an 8-bit camera), the information exceeding this range is limited to this maximum grayscale level. Furthermore, the incident angle of the light is uncertain, resulting in serious phase errors in phase information calculation and ultimately leading to large errors in 3D reconstruction measurement. Therefore, reconstructing the surface topography of highly reflective surfaces remains a challenging task for fringe projection profilometry. Summary of the Invention

[0003] The technical problem to be solved by this invention is to address the pixel saturation of camera images when measuring highly reflective surfaces. This invention provides a monocular visual scanning method for highly reflective objects based on adaptive stripes, which can effectively measure the three-dimensional surface information of highly reflective objects.

[0004] The monocular vision scanning method for highly reflective objects based on adaptive stripes according to the present invention includes:

[0005] Step 1: Acquire multiple image groups, including orthogonal fringe image groups and grayscale image groups. The orthogonal fringe images are eight orthogonal fringe images projected onto the object surface and background plane. These eight orthogonal fringe images I1, I2, I3, I4, I5, I6, I7, and I8 are generated by computer, and their expressions are as follows:

[0006]

[0007] In formula (1), I1(x,y), I2(x,y), I3(x,y), I4(x,y), I5(x,y), I6(x,y), I7(x,y), and I8(x,y) are the light intensities of the stripe patterns I1, I2, I3, I4, I5, I6, I7, and I8, respectively, where I′(x,y) is the DC component of the image, I″(x,y) is the amplitude, and f is the number of periods contained in the stripe pattern;

[0008] Dephase the orthogonal fringe image group to obtain the absolute phase of the orthogonal fringe image group, based on multiple fringe images acquired by the camera. Solving for the wrapped phase:

[0009]

[0010]

[0011] In formula (2), To vertically wrap the phase, The horizontally wrapped phase is expanded by multi-frequency heterodyne phase to obtain the vertical absolute phase and the horizontal absolute phase, respectively.

[0012] Step 2: Identify high reflectivity areas. By projecting a uniform grayscale pattern with a grayscale value of 255, areas with a grayscale value of 255 in the image captured by the camera are identified as high reflectivity areas. A total of 25 uniform grayscale patterns with grayscale values ​​of 10, 20, 30, ..., 250 are projected onto the object surface to obtain camera images. The projected grayscale values ​​in the camera image that just do not exceed 255 are determined pixel by pixel, and a projection mask is created pixel by pixel.

[0013] Step 3, coordinate matching, involves establishing the coordinate correspondence between the image and the projector image. This is achieved by calculating the homography matrix based on the phase difference between the orthogonal absolute phase of the object and the orthogonal absolute phase of the background in the highly reflective region. This homography matrix is ​​then used to adjust the projection mask P. best The projection mask P in the projected image is adjusted according to the background orthogonal absolute phase. best The expression for the pixel coordinate correspondence between the camera and the projector is as follows:

[0014]

[0015] In formula (3), (x c ,y c (x) represents the camera image coordinates. p and y p These are the x and y coordinates of the projector image, Φ v (x c ,y c ) and Φ h (xc ,y c These are the background vertical absolute phase and the background horizontal absolute phase, respectively. P and H P These are the width and height of the projector image, respectively, and T is the number of fringe periods.

[0016] Step four: Generate adaptive fringes. Adaptive fringes are generated based on the correspondence between the projection mask and the coordinates. The adaptive fringes pattern... The expression is:

[0017]

[0018] In formula (4), Stripes in sequence Light intensity, P best Let f be the projection mask, and f be the number of periods contained in the stripe pattern.

[0019] Step 5, 3D reconstruction: Project the adaptive stripes onto the object surface, which can be represented by the image captured by the camera as follows:

[0020]

[0021] In formula (5), Stripes in sequence The light intensity, where s is the camera's photosensitivity coefficient, r(x) c ,y c ) represents the surface reflectance coefficient of the object, β1(x) c ,y c β2(x) represents the ambient light that enters the camera after being reflected from the object. c ,y c () represents ambient light directly entering the camera;

[0022] According to the four-step phase shift principle, the solution for the enclosed phase can be expressed as:

[0023]

[0024] In formula (6), To encapsulate the phase, the absolute phase information is obtained by unfolding the phase using the multi-frequency extrapolation principle. After system calibration, the point cloud of the highly reflective object surface is obtained.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] 1) The method proposed in this invention projects a uniform grayscale pattern onto the surface of an object and adjusts the projection mask pixel by pixel according to the grayscale value in the camera image. After the projection mask modulates the projected image, it can accurately reduce the projection intensity of highly reflective areas to avoid the grayscale value of the camera image pixels from exceeding 255.

[0027] 2) This invention first adjusts the projection mask by using background orthogonal phase information and object orthogonal phase information, and then maps it to the projector image to adjust the image grayscale value of the projector image, thus establishing the camera-projector pixel coordinate correspondence. Compared with the prior art, this invention optimizes the steps of establishing the camera-projector pixel coordinate correspondence and improves the accuracy of camera-projector pixel coordinate mapping. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a monocular vision scanning method for highly reflective objects based on adaptive stripes, provided in an embodiment of the present invention.

[0029] Figure 2 A reflective mouse image is provided for one embodiment of the present invention.

[0030] Figure 3 An adaptive stripe image is provided for one embodiment of the present invention.

[0031] Figure 4 The image shows the surface contour of an object scanned using an adaptive stripe method according to an embodiment of the present invention.

[0032] Figure 5 This is an example of an embodiment of the present invention, showing the surface contour of an object obtained by scanning using a conventional method. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0034] Reference Figure 1 The flowchart illustrates the monocular vision scanning method for highly reflective objects based on adaptive stripes, as described in this invention. The specific steps are as follows:

[0035] Step 1: Acquire multiple image groups, including orthogonal stripe image groups and grayscale image groups, and project orthogonal stripe patterns onto the object surface and background plane respectively. The images captured by the camera are used to establish the coordinate correspondence between the camera image pixels and the projector image pixels.

[0036] First, eight orthogonal fringe patterns I1, I2, I3, I4, I5, I6, I7, and I8 are generated by computer, and their expressions are as follows:

[0037]

[0038] In formula (1), I1(x,y), I2(x,y), I3(x,y), I4(x,y), I5(x,y), I6(x,y), I7(x,y), and I8(x,y) are the light intensities of the stripe patterns I1, I2, I3, I4, I5, I6, I7, and I8, respectively, where I′(x,y) is the DC component of the image, I″(x,y) is the amplitude, and f is the number of periods contained in the stripe pattern.

[0039] After projecting the eight orthogonal fringe patterns onto the object surface and background plane, a camera captures corresponding images. The captured orthogonal fringe image group is then dephased to obtain the absolute phase of the orthogonal fringe image group. Based on the multiple fringe patterns acquired by the camera... Solving for the wrapped phase:

[0040]

[0041]

[0042] In formula (2), To vertically wrap the phase, The phase is horizontally wrapped, but because the inverse tangent function is used when calculating the phase, the wrapped phase is a wrapped phase value wrapped within a periodic phase interval. Therefore, the phase needs to be expanded. After multi-frequency heterodyne phase expansion, the vertical absolute phase and the horizontal absolute phase are obtained respectively.

[0043] Step two: Identify highly reflective areas. First, by projecting a uniform grayscale pattern with a grayscale value of 255, areas with a grayscale value of 255 in the image captured by the camera are identified as highly reflective areas. Second, 25 uniform grayscale patterns with grayscale values ​​of 10, 20, 30, ..., 250 are projected onto the object surface to obtain camera images. The projected grayscale values ​​in the camera image that just do not exceed 255 are determined pixel by pixel, and a projection mask is created pixel by pixel. Figure 2 This is an image of a reflective mouse, and you can see that some highly reflective areas are overexposed.

[0044] Step three, coordinate matching, involves establishing the coordinate correspondence between the image and the projector image. To determine the coordinates of the highly reflective area in the projected image, a mapping relationship between the pixel coordinates of the camera image and the pixel coordinates of the projected image needs to be established. Therefore, this invention proposes a method for determining the mapping relationship between the highly reflective area and the projected area. This method involves adjusting the projection mask on the background plane and then performing mapping to determine the pixel coordinates in the projected image.

[0045] The homography matrix is ​​calculated based on the phase difference between the orthogonal absolute phase of the object and the orthogonal absolute phase of the background in the highly reflective region. This homography matrix is ​​used to adjust the projection mask P. best The projection mask P in the projected image is adjusted according to the background orthogonal absolute phase. best The expression for the pixel coordinate correspondence between the camera and the projector is as follows:

[0046]

[0047] In formula (3), (x c ,y c (x) represents the camera image coordinates. p and y p These are the x and y coordinates of the projector image, Φ v (x c ,y c ) and Φ h (x c ,y c These are the background vertical absolute phase and the background horizontal absolute phase, respectively. P and H P These are the width and height of the projected image, respectively, and T is the number of fringe periods. By reducing the (x) in the projected image... p ,y p The grayscale of ) can reduce the image captured by the camera (x c ,y c The grayscale value is used to avoid situations where the camera image pixel count exceeds 255.

[0048] Step four: Generate adaptive fringes. Adaptive fringes are generated based on the correspondence between the projection mask and the coordinates. The adaptive fringes pattern... The expression is:

[0049]

[0050] In formula (4), Stripes in sequence Light intensity, P best Let f be the projection mask, and f be the number of periods contained in the fringe pattern. Figure 3For the generated adaptive stripes.

[0051] Step 5, 3D reconstruction: Project the adaptive stripes onto the object surface, which can be represented by the image captured by the camera as follows:

[0052]

[0053] In formula (5), Stripes in sequence The light intensity, where s is the camera's photosensitivity coefficient, r(x) c ,y c ) represents the surface reflectance coefficient of the object, β1(x) c ,y c β2(x) represents the ambient light that enters the camera after being reflected from the object. c ,y c () represents ambient light directly entering the camera;

[0054] According to the four-step phase shift principle, the solution for the enclosed phase can be expressed as:

[0055]

[0056] In formula (6), To encapsulate the phase, the phase is unfolded using the multi-frequency extrapolation principle to obtain absolute phase information. After system calibration, the point cloud of the highly reflective object surface is obtained. Figure 4 For comparison, the mouse point cloud reconstructed using the adaptive stripe method in 3D is shown below. Figure 5 The mouse point cloud after 3D reconstruction using traditional methods.

[0057] The steps of the method proposed in this invention can be summarized as follows:

[0058] 1) Beforehand, two sets of orthogonal stripe patterns, totaling 24×2 patterns, are projected by a projector on a background plane without objects and after objects are placed. The images are then captured by a camera. After deconstructing the 48 stripe patterns, the vertical and horizontal absolute phases of the object surface, as well as the vertical and horizontal absolute phases of the background plane, are obtained.

[0059] 2) The projector projects a uniform grayscale pattern onto the surface of the object. By analyzing the relationship between the projected grayscale and the grayscale value of the pattern captured by the camera, the optimal projection light intensity of each pixel is determined, and the projection mask is obtained.

[0060] 3) Using the orthogonal absolute phase of the object and the orthogonal absolute phase of the background, the intensity mask at the camera end is adjusted on the background plane, and a coordinate mapping relationship between the camera image and the projector image is established.

[0061] 4) Design the adaptive stripe pattern based on the projection mask and coordinate mapping relationship.

[0062] 5) Project the adaptive stripe pattern onto the object's surface, and after phase resolution and system calibration, obtain the three-dimensional surface information of the object's highly reflective areas.

Claims

1. A monocular visual scanning method for highly reflective objects based on adaptive stripes, characterized in that, The method includes: Acquire multiple image groups, including orthogonal stripe image groups and grayscale image groups; Identify highly reflective areas by determining the highly reflective areas based on the grayscale image group projected onto the object surface and establishing a projection mask; Coordinate matching, wherein coordinate matching is to establish a coordinate correspondence between the image and the projector image; Generate adaptive stripes based on the correspondence between the projection mask and the coordinates. Three-dimensional reconstruction is performed by projecting the adaptive stripe pattern and completing the three-dimensional reconstruction after phase decomposition algorithm and system calibration. According to the aforementioned monocular visual scanning method for highly reflective objects based on adaptive stripes, the method is characterized by coordinate matching, which involves establishing a coordinate correspondence between the image and the projector image. A homography matrix is ​​calculated based on the phase difference between the orthogonal absolute phase of the object and the orthogonal absolute phase of the background in the highly reflective region. This homography matrix is ​​used to adjust the projection mask, which is adjusted based on the orthogonal absolute phase of the background. The expression for the pixel coordinate correspondence between the camera and the projector is as follows: In formula (1), For camera image coordinates, and These are the x and y coordinates of the projector image, respectively. and These are the background vertical absolute phase and the background horizontal absolute phase, and These are the width and height of the projected image, respectively. The number of stripe periods; The method for monocular visual scanning of highly reflective objects based on adaptive stripes is characterized by generating adaptive stripes, which is done according to the correspondence between the projection mask and the coordinates, and the adaptive stripe pattern... , , , The expression is: In formula (2), , , , Stripes in sequence , , , Light intensity, P best For the projection mask, The number of cycles contained in the stripe pattern.

2. The monocular vision scanning method for highly reflective objects based on adaptive stripes according to claim 1, characterized in that, Multiple image groups are acquired, including an orthogonal fringe image group and a grayscale image group. The orthogonal fringe image group consists of eight orthogonal fringe images projected onto the object surface and a background plane. The eight orthogonal fringe patterns are generated by a computer. , , , , , , , Their expressions are as follows: In formula (3), , , , , , , , Stripes in sequence , , , , , , , The light intensity, of which For the DC component of the image, For amplitude, The number of cycles contained in the stripe pattern; The eight fringe patterns are projected onto the object surface and background plane using a projector. The eight fringe patterns are captured by a camera. The wrapping phase is calculated based on the eight fringe patterns. Then, the phase is unwrapped using a multi-frequency heterodyne algorithm to obtain the orthogonal absolute phase of the object and the orthogonal absolute phase of the background. The pixel coordinate correspondence between the camera and the projector is established based on the orthogonal absolute phase of the object and the orthogonal absolute phase of the background.

3. The monocular vision scanning method for highly reflective objects based on adaptive stripes according to claim 1, characterized in that, High reflectivity areas are identified by projecting a uniform grayscale pattern with a grayscale value of 255. Areas with a grayscale value of 255 in the image captured by the camera are identified as high reflectivity areas. 25 uniform grayscale patterns with grayscale values ​​of 10, 20, 30, ..., 250 are projected onto the surface of the object to obtain camera images. The projection grayscale values ​​in the camera images that do not exceed 255 are determined pixel by pixel, and a projection mask is built pixel by pixel.

4. The monocular vision scanning method for highly reflective objects based on adaptive stripes according to claim 1, characterized in that, In 3D reconstruction, the adaptive fringes are projected onto the object's surface and represented by the image captured by the camera as follows: In formula (4), , , , Stripes in sequence , , , The light intensity, of which The photosensitive coefficient of the camera. The surface reflectance of the object. This represents the ambient light that enters the camera after being reflected from an object. This represents ambient light that directly enters the camera; Based on the four-step phase shift principle, the wrapped phase can be represented as: In formula (5), To encapsulate the phase, the absolute phase information is obtained by unfolding the phase using the multi-frequency extrapolation principle. After system calibration, the point cloud of the highly reflective object surface is obtained.

Images