High dynamic range object surface three-dimensional topography measurement method and system

Abstract

The application discloses a kind of high dynamic range object surface three-dimensional topography measurement method and system.The method is first built measurement system and calibration system parameters, then blue stripe image and blue uniform image are projected to the surface of the object to be measured by projector, and the deformed blue stripe image and the deformed blue uniform image modulated by the shape of the object surface are collected by color camera;Different color channel response of color camera to monochromatic stripe projection and color image color channel separation technology are used to separate the stripe image corresponding to blue and green channel from the deformed blue stripe image, and the deformed blue uniform image is used to assign the blue-green channel mask, and then a high dynamic stripe image is synthesized;Then, the phase solving method is applied to realize the measurement of high dynamic range object surface.The application reduces the number of projection images, improves the measurement efficiency, and can more effectively improve the measurement speed of dynamic object three-dimensional measurement system.

Description

Technical Field

[0001] This invention relates to the field of optical non-contact high dynamic range object three-dimensional topography measurement, specifically a method and system for measuring the three-dimensional topography of a high dynamic range object surface. Background Technology

[0002] Fringe projection profilometry is widely used in industrial manufacturing, cultural relic preservation, and biometrics due to its advantages such as fast measurement speed, high accuracy, and strong robustness. Fringe projection profilometry expects the measured object to have a uniform reflectance; however, the reflectance of many object surfaces is non-uniform, such as high dynamic range (HDR) objects. The surface reflectance of such objects is low in some areas and very high in others. The limited dynamic range of traditional cameras makes it difficult to obtain the correct phase distribution of fringe images from objects with large variations in optical reflectance. Ignoring the influence of large-scale surface reflectance can lead to pixel saturation in bright areas, resulting in measurement errors. Therefore, the accurate acquisition of 3D topographic data of HDR object surfaces has always been a research hotspot and challenge.

[0003] Measuring objects with high dynamic range is one of the major challenges in the field of optics. To overcome this problem, many scholars both domestically and internationally have conducted extensive research in recent years, as detailed below:

[0004] The paper "Zhang S, Yau S T. High dynamic range scanning technique[J]. Optical Engineering, 2009, 48(3): 033604" utilizes a high dynamic range scanning technique to capture fringe pattern sequences with different exposure times. By selecting the corresponding pixel with the highest gray level (unsaturated) from the original fringe pattern sequence, high-quality fringe patterns can be obtained. High reflectivity regions are extracted from short-exposure images, while low reflectivity regions are extracted from high-exposure images. This method effectively improves the dynamic range of the measurement. However, since the effect of exposure time cannot be quantified, a large number of images need to be generated at different exposure times when measuring complex HDR, which has the disadvantages of being time-consuming and having redundant image data.

[0005] The literature "Yoshinori Y, Hiroyuki M, Osamu N, et al. Shape measurement of glossy objects by range finder with polarization optical system [C]. Reports of the Technical Conference of the Institute of Image Electronics Engineers of Japan, 2003, 200: 43-50." uses the polarization optical method, while the literature "Feng SJ, Chen Q, Zuo C, et al. Fast three-dimensional measurements for dynamic scenes with shiny surfaces [J]. Optics Communications, 2017, 382: 18-27." uses multiple cameras to achieve high dynamic range object measurement. However, these methods require additional hardware to improve system performance, which may lead to increased costs or computational complexity.

[0006] As can be seen from the above literature, the measurement method of three-dimensional morphology of high dynamic range object surface has been extensively studied, but there are still many problems, such as the need for a large number of images and long measurement time, or high hardware cost and computational complexity. Therefore, there is an urgent need to develop a simple method for measuring the three-dimensional morphology of high dynamic range object surface that requires fewer images. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide a method and system for measuring the three-dimensional morphology of an object surface with high dynamic range.

[0008] The technical solution of this invention to solve the aforementioned technical problem is to provide a method for measuring the three-dimensional topography of an object surface with high dynamic range, characterized in that the measurement method includes the following steps:

[0009] (1) Set up the measurement system: The computer is connected to the projector and the color camera respectively; adjust the positional relationship between the projector and the color camera so that the projector, the color camera and the object being measured are in a triangular measurement relationship in space, and the object being measured is located in the measurement field where the projector and the color camera form a clear image within the depth of field.

[0010] (2) Calibrate system parameters;

[0011] (3) Replace the calibration plate with the object under test. The computer generates a uniform blue image with the maximum brightness value and a blue stripe image with the maximum brightness value. These images are projected onto the surface of the object under test by a projector. The color camera collects the deformed uniform blue image and deformed blue stripe image generated by the surface shape modulation of the object under test. The object under test is a high dynamic range object.

[0012] (4) Using color channel separation technology for color images, extract the brightness value of the blue channel of the deformed blue uniform image. Brightness values ​​of the green channel in a uniform blue image Then, a mask for the blue channel of the deformed blue uniform image is generated using equation (1). b (x,y), the mask for the green channel of the deformed blue uniform image is generated by equation (2). g (x,y);

[0013]

[0014]

[0015] Using color channel separation technology for color images, stripe images of the blue channel and the green channel of the deformed blue stripe image are extracted;

[0016] (5) Based on the blue channel of the deformed blue uniform image, a mask is applied. b (x,y), mask for the green channel of a deformed blue uniform image. g (x,y), the stripe image of the blue channel of the deformed blue stripe image and the stripe image of the green channel of the deformed blue stripe image are used to synthesize a high dynamic range stripe image using equation (3);

[0017]

[0018] In equation (3), I f (x,y) represents the brightness value of the high dynamic range stripe image; These represent the brightness values ​​of the blue channel of the distorted blue stripe image. The brightness value of the striped image in the green channel of the distorted blue striped image.

[0019] (6) Use the high dynamic range stripe image as the phase solution image to solve for the absolute phase;

[0020] (7) Based on the system parameters calibrated in step (2) and the absolute phase of the high dynamic stripe image obtained in step (6), recover the three-dimensional data of the surface of the object under test.

[0021] The technical solution of the present invention to solve the aforementioned system technical problem is to provide a measurement system for a high dynamic range object surface three-dimensional topography measurement method, characterized in that the measurement system includes a computer, a projector, and a color camera;

[0022] The computer is connected to the projector and the color camera respectively to control the projector to project images and the color camera to acquire images, and to store, display and process the acquired images; the projector is used to project images; the color camera is used to acquire images reflected by objects; the projector, the color camera and the object being measured are in a triangulation relationship in space, and the object being measured is located within the measurement field where the projector and the color camera form a clear image.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] (1) This method first sets up a measurement system and calibrates the system parameters. Then, a projector is used to project a blue stripe image and a blue uniform image onto the surface of the object being measured. A color camera acquires a deformed blue stripe image and a deformed blue uniform image modulated by the shape of the object's surface. Then, using the different color channel responses of the color camera to the monochrome stripe projection and the color channel separation technology of the color image, the stripe images corresponding to the blue and green channels are separated from the deformed blue stripe image. At the same time, the deformed blue uniform image is used to assign values ​​to the blue and green channel mask, thereby synthesizing a high dynamic range stripe image. Then, the phase calculation method is applied to realize the measurement of the object surface with high dynamic range.

[0025] (2) This method utilizes the different color channel responses of a color camera to the projection of monochrome stripes, and applies a mask of the blue-green channel to synthesize a high dynamic range stripe image, which reduces the number of projected images, improves measurement efficiency, and can more effectively improve the measurement speed of the dynamic object 3D measurement system.

[0026] (3) The algorithm of this invention is simple, the operation and device structure are simple, no additional hardware facilities are required, and it is easy to implement. In practical applications, it plays an important role in the accurate measurement of objects with high dynamic range in aerospace, automotive industry and other fields. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the measurement system structure of the present invention;

[0028] Figure 2 This is a uniform blue image generated by the encoding of this invention;

[0029] Figure 3 The blue stripe image generated by the encoding of this invention;

[0030] Figure 4 This is a deformed blue uniform image acquired in this invention;

[0031] Figure 5 The deformed blue stripe image acquired for this invention;

[0032] Figure 6 This is a mask for the blue channel of the deformed blue uniform image generated by the present invention;

[0033] Figure 7 The mask for the green channel of the deformed blue uniform image generated by this invention;

[0034] Figure 8 This is the stripe image of the blue channel of the deformed blue stripe image extracted by this invention;

[0035] Figure 9 This is the stripe image of the green channel of the deformed blue stripe image extracted by this invention;

[0036] Figure 10 This is a high dynamic range stripe image synthesized according to the present invention;

[0037] Figure 11 This is a diagram showing the three-dimensional shape measurement results of the object under test in Embodiment 1 of the present invention.

[0038] In the diagram, there is a computer (1), a projector (2), a color camera (3), and the object being measured (4). Detailed Implementation

[0039] Specific embodiments of the present invention are given below. These specific embodiments are only used to further illustrate the present invention and do not limit the scope of protection of the claims of this application.

[0040] This invention provides a method for measuring the three-dimensional topography of a high dynamic range object surface (hereinafter referred to as the measurement method), characterized in that the measurement method includes the following steps:

[0041] (1) Set up the measurement system: The computer 1 is connected to the projector 2 and the color camera 3 respectively; adjust the positional relationship between the projector 2 and the color camera 3 so that the projector 2, the color camera 3 and the object under test 4 are in a triangular measurement relationship in space, and the object under test 4 is located in the measurement field where the image is clear within the depth of field of the projector 2 and the color camera 3.

[0042] Preferably, in step (1), the optical axis angle between the projector 2 and the color camera 3 is 25-30°.

[0043] (2) Calibrate system parameters:

[0044] (2.1) Place the calibration plate with the circular mark at different positions in the measurement field; at each position, projector 2 projects a sine fringe pattern onto the surface of the calibration plate, solves the folded phase, and then solves the absolute phase of each pixel; at the same time, acquire a calibration plate texture image at each position and extract the center pixel coordinates of the circular mark of the calibration plate.

[0045] The color camera 3 is calibrated. Using the camera parameters of the calibrated color camera 3 and the center pixel coordinates of the circular mark on the calibration board, the spatial position of each pixel on the calibration board in the camera coordinate system is calculated, including the depth Z and XY coordinates.

[0046] (2.2) Establish the relationship between the absolute phase and depth Z of each pixel, establish the relationship between the pixel coordinates and XY coordinates of each pixel, obtain the system calibration parameters, and complete the three-dimensional calibration of the system;

[0047] Preferably, in step (2), the plane accuracy of the calibration plate is 1 μm.

[0048] Preferably, in step (2), the phase shift method is used to solve the folded phase, and the four-step phase shift method is more preferred; the optimal three-fringe selection method is used to solve the absolute phase, and the number of fringes in the three sets of fringe images are 64, 63 and 56 respectively.

[0049] (3) Replace the calibration plate with the object under test 4, and computer 1 generates a uniform blue image with all brightness values ​​at their maximum values. Figure 2 ) and the blue stripe image with the maximum brightness value ( Figure 3 The image is projected onto the surface of the object being measured 4 by the projector 2, and the color camera 3 captures a distorted blue uniform image generated by the surface shape modulation of the object being measured 4. Figure 4 ) and distorted blue stripe image ( Figure 5 The tested object 4 is a high dynamic range object;

[0050] (4) Using color channel separation technology for color images, extract the brightness value of the blue channel of the deformed blue uniform image. Brightness values ​​of the green channel in a uniform blue image Then, a mask for the blue channel of the deformed blue uniform image is generated using equation (1). b (x,y)( Figure 6 The mask for the green channel of the deformed blue uniform image is generated by equation (2). g (x,y)( Figure 7 );

[0051]

[0052]

[0053] In equations (1) and (2), mask b (x,y) represents the mask for the blue channel of the deformed uniform blue image. g (x,y) represents the mask for the green channel of the deformed blue uniform image. This represents the brightness value of the blue channel in a distorted, uniform blue image. This represents the brightness value of the green channel in a distorted blue uniform image;

[0054] Using color channel separation techniques in color images, the stripe image of the blue channel in a distorted blue stripe image is extracted. Figure 8 ) and the striped image of the green channel of the deformed blue striped image ( Figure 9 );

[0055] Preferably, in step (4), the mask for the blue channel of the deformed blue uniform image is... b The generation process of (x,y) is as follows: An initial mask for the blue channel is generated based on the size of the deformed uniform blue image; within the initial mask for the blue channel, the brightness value I of the blue channel of the deformed uniform blue image is... b The region where u(x,y) is less than the maximum value is assigned a value of 1, and the region where u(x,y) is equal to the maximum value is assigned a value of 0.

[0056] Preferably, in step (4), the mask for the green channel of the deformed blue uniform image is... g The generation process of (x,y) is as follows: An initial mask for the green channel is generated based on the size of the deformed blue uniform image; within the initial mask for the green channel, the brightness value of the blue channel of the deformed blue uniform image is... The brightness value of the green channel in a uniformly distorted blue image equal to the maximum value. Regions smaller than the maximum value are assigned a value of 1, and the remaining regions of the green channel in the distorted blue uniform image are assigned a value of 0.

[0057] (5) Based on the blue channel of the deformed blue uniform image, a mask is applied. b (x,y), mask for the green channel of a deformed blue uniform image. g (x,y), the stripe image of the blue channel of the deformed blue stripe image and the stripe image of the green channel of the deformed blue stripe image, and a high dynamic range stripe image is synthesized using equation (3). Figure 10 );

[0058]

[0059] In equation (3), I f (x,y) represents the brightness values ​​of a high dynamic range stripe image; mask i (x,y), i=b,g represent the mask of the blue channel of the deformed uniform blue image, respectively. b (x,y) and the mask for the green channel of the deformed blue uniform image. g (x,y); These represent the brightness values ​​of the blue channel of the distorted blue stripe image. The brightness value of the striped image in the green channel of the distorted blue striped image.

[0060] Preferably, in step (5), the specific process of synthesizing the high dynamic range stripe image is as follows: the brightness value of each pixel in the stripe image of the blue channel of the deformed blue stripe image is... Masks for the blue channel of the deformed blue uniform image, respectively. b Multiply the mask values ​​of the corresponding pixels in (x,y); similarly, multiply the brightness values ​​of each pixel in the stripe image of the green channel of the deformed blue stripe image. The mask for the green channel of the deformed blue uniform image is compared with the mask for the green channel. g The mask values ​​of the corresponding pixels in (x,y) are multiplied together; then the brightness values ​​of the blue channel stripe image of the deformed blue stripe image and the green channel stripe image of the deformed blue stripe image are added together to obtain the brightness value of the high dynamic range stripe image.

[0061] (6) Phase calculation of high dynamic range stripe image: The high dynamic range stripe image is used as the final phase calculation image. The folded phase is solved and then the absolute phase is solved.

[0062] Preferably, in step (6), the phase shift method is used to solve the folded phase, and the three-step phase shift method is more preferred; the optimal three-fringe selection method is used to solve the absolute phase. The number of fringes in the three sets of fringe images are 64, 63 and 56 respectively, and the phase shift of each set is 2π / 3. A total of 9 images are needed to solve the phase.

[0063] (7) Based on the system parameters calibrated in step (2) and the absolute phase of the high dynamic stripe image obtained in step (6), recover the three-dimensional data of the surface of the object under test 4.

[0064] The present invention also provides a high dynamic range object surface three-dimensional topography measurement system (hereinafter referred to as the measurement system), characterized in that the measurement system includes a computer 1, a projector 2 and a color camera 3;

[0065] Computer 1 is connected to projector 2 and color camera 3 respectively, and is used to control projector 2 to project images and color camera 3 to acquire images, and to store, display and process the acquired images; projector 2 is used to project images; color camera 3 is used to acquire images reflected by objects; the positional relationship between projector 2 and color camera 3 is adjusted so that projector 2, color camera 3 and the object under test 4 are in a triangulation relationship in space, and the object under test 4 is located in the measurement field where the object under test is clearly imaged within the depth of field of projector 2 and color camera 3.

[0066] Example 1

[0067] In this embodiment, the number of stripes in the three sets of projected stripe images are 64, 63, and 56, respectively.

[0068] In step (1), the projector 2 is a DLP digital projector of model lightcrafter4500 with a resolution of 912×1140, and is connected to computer 1 via an HDMI cable. The color camera 3 is a CCD camera of model AD-080GE with a resolution of 1024×768 pixels, and is connected to the gigabit network interface of computer 1.

[0069] In step (2), the calibration plate is a ceramic calibration plate produced by Shenzhen Kailinbo Optical Technology Co., Ltd., with 9 rows and 12 columns of circular markings on its surface. The spacing between adjacent markings in the horizontal and vertical directions is 15 mm, and its planar accuracy is 1 μm. The folded phase is solved using the three-step phase shift method; the absolute phase is solved using the optimal three-fringe selection method, and the number of fringes in the three sets of fringe images are 64, 63, and 56, respectively.

[0070] In step (3), the object under test 4 is a metal spherical part. The brightness value of the uniform blue image is 255, and the maximum brightness value of the blue striped image is 255.

[0071] In step (6), the three-step phase shift method is used to solve the folded phase; the optimal three-fringe selection method is used to solve the absolute phase. The number of fringes in the three sets of fringe images are 64, 63 and 56 respectively, and the phase shift of each set is 2π / 3.

[0072] In step (7), the recovered three-dimensional morphology measurement data of the metal spherical part surface is as follows: Figure 11 As shown.

[0073] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method of high dynamic range object surface three-dimensional topography measurement, characterized in that, The measurement method comprises the following steps: (1) building a measurement system: a computer is respectively connected with a projector and a color camera in communication; the positional relationship of the projector and the color camera is adjusted, so that the projector, the color camera and the measured object are in a triangulation relationship in space, and the measured object is located within the depth of field of the projector and the color camera to form a clear image in the measurement field; (2) calibrating system parameters; (3) replacing the calibration board with the measured object, the computer codes to generate a blue uniform image with a maximum brightness value and a blue stripe image with a maximum brightness value, which are projected onto the surface of the measured object by the projector, and the deformed blue uniform image and the deformed blue stripe image generated by the surface shape modulation of the measured object are collected by the color camera; the measured object is a high dynamic range object; (4) using color channel separation technique of color image, extracting the luminance value of blue channel of the deformed blue uniform image and the luminance value of green channel of the deformed blue uniform image then generating the mask of blue channel of the deformed blue uniform image by formula (1) b (x,y), generating the mask of green channel of the deformed blue uniform image by formula (2) g (x,y) The color channel separation technology of the color image is used to extract the stripe image of the blue channel of the deformed blue stripe image and the stripe image of the green channel of the deformed blue stripe image; (5) mask of the blue channel of the deformed blue uniform image b (x,y), mask of the green channel of the deformed blue uniform image g (x,y), stripe image of the blue channel of the deformed blue stripe image and stripe image of the green channel of the deformed blue stripe image, to synthesize a high dynamic stripe image using equation (3); In formula (3), I f (x, y) represents the luminance value of the high dynamic stripe image; respectively represent the luminance value of the stripe image of the deformed blue stripe image blue channel and the luminance value of the stripe image of the deformed blue stripe image green channel (6) taking the high dynamic stripe image as a phase calculation image to solve the absolute phase; (7) according to the system parameters calibrated in step (2) and the absolute phase of the high dynamic stripe image obtained in step (6), the three-dimensional data of the surface of the measured object is restored.

2. The high dynamic range object surface three-dimensional topography measurement method according to claim 1, characterized in that, In step (1), the included angle between the optical axes of the projector and the color camera is 25-30°.

3. The method of claim 1, wherein, Step (2) is specifically: (2.1) placing a calibration board with a circular ring mark at different positions of the measurement field; at each position, the projector projects a sinusoidal stripe image onto the surface of the calibration board to solve the folded phase and then the absolute phase of each pixel; at the same time, a texture image of the calibration board is collected at each position to extract the center pixel coordinates of the circular ring mark of the calibration board; The color camera is calibrated, and the spatial positions of each pixel point on the calibration board in the camera coordinate system, including the depth Z and the XY coordinates, are solved by using the camera parameters of the calibrated color camera and the center pixel coordinates of the circular ring mark of the calibration board; (2.2) establishing the relationship between the absolute phase of each pixel point and the depth Z, and the relationship between the pixel coordinates of each pixel point and the XY coordinates, to obtain the system calibration parameters and complete the three-dimensional calibration of the system.

4. The method of claim 1, wherein, The mask of the blue channel of the deformed blue uniform image in step (4) b The generating process of (x, y) is: generating the initial mask of the blue channel according to the size of the deformed blue uniform image; in the initial mask of the blue channel, the luminance value of the blue channel of the deformed blue uniform image is compared with the maximum value of the blue channel of the deformed blue uniform image The region less than the maximum value is assigned a value of 1, and the region equal to the maximum value is assigned a value of 0.

5. The method of claim 1, wherein, The mask of the green channel of the deformed blue uniform image in step (4) g The generating process of (x, y) is: generating the initial mask of the green channel according to the size of the deformed blue uniform image; in the initial mask of the green channel, the luminance value of the blue channel of the deformed blue uniform image is equal to the maximum value and the luminance value of the green channel of the deformed blue uniform image less than the maximum value is assigned as 1, and the remaining area is assigned as 0.

6. The method of claim 1, wherein, In step (5), the specific process of synthesizing the high dynamic fringe image is: multiplying the luminance value of each pixel point in the fringe image of the deformed blue fringe image blue channel with the mask value of the corresponding pixel point in the mask of the deformed blue uniform image blue channel b (x,y) respectively​ The luminance value of each pixel point in the fringe image of the deformed blue fringe image green channel is multiplied by the mask value of the corresponding pixel point in the deformed blue uniform image green channel, respectively g The luminance value of each pixel point in the fringe image of the deformed blue fringe image green channel is multiplied by the mask value of the corresponding pixel point in the deformed blue uniform image green channel, respectively g The luminance value of each pixel point in the fringe image of the deformed blue fringe image green channel is multiplied by the mask value of the corresponding pixel point in the deformed blue uniform image green channel, respectively 7. A measurement system for use in a high dynamic range three-dimensional surface topography measurement method according to any one of claims 1 to 6, characterized in that The measurement system comprises a computer, a projector and a color camera; The computer is respectively connected with the projector and the color camera in communication to control the projector to project images and the color camera to collect images, and to store, display and process the collected images; The projector is used to project images, and the color camera is used to collect images reflected by the object; The projector, the color camera and the measured object are in a triangulation relationship in space, and the measured object is located within the depth of field of the projector and the color camera to form a clear image in the measurement field.

Images