A method for fast grating projection profilometry based on prior information

Abstract

The present invention relates to a method for fast grating projection profilometry based on prior information. The method for fast grating projection profilometry based on prior information proposed by the present invention is aimed at the scenario of fixed product detection on industrial production lines. By using prior geometric information, the number of images to be projected is reduced to 5. At the same time, the method proposed by the present invention only performs phase calculation and three-dimensional reconstruction for each point inside the object to be detected. Therefore, compared with traditional methods, the computational amount is also significantly reduced. In summary, the present invention can significantly accelerate the three-dimensional reconstruction speed of grating projection in specific scenarios and can effectively perform three-dimensional reconstruction on dynamic objects to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional optical measurement, and relates to a three-dimensional reconstruction method based on grating projection, and particularly relates to a method for fast grating projection contour based on prior information. Background Technique

[0002] Nowadays, three-dimensional shape measurement technology based on structured light has been extensively studied and has a wide range of applications, including industrial inspection, consumer electronics, biomedicine, entertainment and other fields. Among them, the three-dimensional reconstruction method based on fringe projection profilometry (FPP) is often used in precision measurement fields such as industrial inspection due to its advantages of high measurement accuracy and fast speed. Digital fringe projection profilometry is an active triangulation method. Generally, a projector (such as digital light processing DLP, etc.) is used to project a series of sinusoidal gratings and rectangular gratings onto the surface of the object to be measured. The deformed grating image is obtained through a camera, and the phase value of each pixel point is calculated according to the change of the deformed grating image. Then, based on the system calibration parameters, the three-dimensional spatial coordinates of each point are calculated. The advantages of fringe projection profilometry are high measurement accuracy, fast speed, and the ability to perform three-dimensional reconstruction of the entire scene at one time, and its measurement accuracy can reach the micron level. One of the key technologies of fringe projection profilometry is the solution of the absolute phase value of each point. Currently, common phase solving methods include the Fourier transform method based on a single-frame grating image, the windowed Fourier transform method, and the phase shift method based on multiple-frame grating images. Compared with the transform domain analysis method, the phase shift method has higher accuracy and pixel-level resolution; however, since the phase shift method is based on multiple-frame grating images, its speed is relatively slow and it is not suitable for dynamic measurement.

[0003] 3D sensors based on fringe projection profilometry and phase-shift method have been extensively developed, manufactured, and widely used in various industrial precision inspection fields. In the phase-shift method, generally, several (3 - 5) sinusoidal gratings need to be projected first to calculate the wrapped phase, and then the wrapped phase is unwrapped. Currently, phase unwrapping methods can be divided into two categories: spatial phase unwrapping method and temporal phase unwrapping method. The spatial phase unwrapping method uses the information between pixel neighborhoods to perform phase unwrapping, but in the presence of steep cross-sections, local shadows, severe noise, etc., error propagation and incorrect decoding are likely to occur. The temporal phase unwrapping method can perform phase unwrapping for each pixel separately, effectively avoiding the error propagation between pixel points caused by the spatial phase unwrapping method. Therefore, the temporal phase unwrapping method is widely used in mature commercial 3D sensors. Common phase unwrapping methods include Gray code + phase-shift method, multi-frequency fitting method, multi-frequency heterodyne method, etc. In these methods, in addition to projecting 3 - 5 sinusoidal gratings to calculate the wrapped phase, several (4 - 8) sinusoidal gratings (multi-frequency method) or rectangular gratings (Gray code method) need to be projected for phase unwrapping. Therefore, usually, more than 10 grating images need to be projected in total.

[0004] For 3D sensors based on fringe projection profilometry and phase-shift method, it is often necessary to project several (10 - 20) sinusoidal gratings and rectangular gratings to obtain the absolute phase values of each point, resulting in the problems of low 3D reconstruction speed and inability to effectively reconstruct dynamic objects.

[0005] In view of the above defects, the inventor actively conducts research and innovation to create a method for fast fringe projection profilometry based on prior information, making it more valuable in industrial applications. Summary of the Invention

[0006] To solve the above technical problems, the objective of the present invention is to provide a method for fast fringe projection profilometry based on prior information.

[0007] To achieve the above objective, the present invention adopts the following technical solutions:

[0008] A method for fast fringe projection profilometry based on prior information comprises the following steps in sequence:

[0009] Step 1: When there is only a background and no object to be measured on the industrial production line, project a full-white image through a projector, capture the image with a camera and save it to record the background image on the production line.

[0010] Step 2: For the first object to be measured on the production line, and this object to be measured is a standard qualified product. First, project a full-white image using a projector, subtract the full-white image from the background image saved in Step 1 to obtain a difference image; set a reasonable threshold and perform binary processing on the difference image; then perform morphological processing and Blob analysis on the binary image, and screen through the area of the Blob. The largest Blob obtained is the object to be measured, and save the image of the object to be measured as a template image.

[0011] Step 3: For subsequent objects to be measured on the production line, first project a full-white image using a projector, subtract the full-white image from the background image saved in Step 1 to obtain a difference image; set a reasonable threshold and perform binary processing on the difference image; then perform morphological processing and Blob analysis on the binary image, and screen through the area of the Blob. The largest Blob obtained is the object to be measured; perform template matching between the current object image to be measured and the template image saved in Step 2 to obtain the translation amounts in the X and Y directions and the rotation angle, and then correspond each point of the current object to be measured with each point of the template image to obtain the standard height values of each point of the current object to be measured.

[0012] Step 4: Finally, based on the absolute phase image and the system calibration parameters, calculate the three-dimensional point cloud of the object to be measured.

[0013] As a further improvement of the present invention, in Step 2, for the first object to be measured on the production line, project four sine gratings further, calculate the wrapped phase using the four-step phase-shifting method, and then project several Gray-code-encoded gratings for phase unwrapping to obtain the absolute phase values of each point; then, calculate the height values of each point inside the object to be measured through the system calibration parameters and record them in a specified file.

[0014] As a further improvement of the present invention, in Step 3, for subsequent objects to be measured on the production line, project four sine gratings further to calculate the wrapped phase of each point inside the object to be measured; at the same time, for each point inside the object to be measured, inversely deduce the integer part of the absolute phase, that is, the fringe order, through the standard height value and the tolerance range obtained in Step 4; add the fringe order to the wrapped phase to obtain the final absolute phase value.

[0015] As a further improvement of the present invention, the template matching in Step 3 is the gray-scale method or the edge method.

[0016] As a further improvement of the present invention, the projector is a DLP projector.

[0017] As a further improvement of the present invention, the camera is a 2D industrial camera.

[0018] By means of the above solution, the present invention has at least the following advantages:

[0019] A method for fast grating projection profile based on prior information, aiming at the scenario of fixed product detection on industrial production lines, uses prior geometric information to reduce the number of images to be projected to 5; at the same time, the method proposed in the present invention only performs phase calculation and three-dimensional reconstruction on each point inside the object to be detected, so the calculation amount is also greatly reduced compared with the traditional method; in summary, the present invention can significantly accelerate the three-dimensional reconstruction speed of grating projection in specific scenarios and can effectively perform three-dimensional reconstruction on dynamic objects to a certain extent.

[0020] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and implement it according to the content of the specification, the following uses the preferred embodiments of the present invention to describe in detail as follows. Detailed implementation manners

[0021] The following combines embodiments to further describe in detail the specific implementation manners of the present invention. The following embodiments are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0022] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed present invention, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0023] Embodiment

[0024] A method for fast grating projection profile based on prior information includes the following steps in sequence:

[0025] Step 1: When there is only a background and no object to be measured on the industrial production line, project a full-white image through a projector, take and save the image through a camera, so as to record the background image on the production line.

[0026] Step 2: For the first object to be measured on the production line, and the object to be measured is a standard qualified product. First, project a full-white image through a projector, subtract the full-white image from the background image saved in Step 1 to obtain a difference image; set a reasonable threshold and perform binary processing on the difference image; then perform morphological processing and Blob analysis on the binary image, and screen through the area of the Blob. The largest Blob obtained is the object to be measured, and the image of the object to be measured is saved as a template image.

[0027] Step 3: For the subsequent objects to be measured on the production line, first project a full-white image using a projector, subtract the full-white image from the background image saved in Step 1 to obtain a difference image; set a reasonable threshold and perform binary processing on the difference image; then perform morphological processing and Blob analysis on the binary image, screen through the area of the Blob, and the largest Blob obtained is the object to be measured; perform template matching between the current object image to be measured and the template image saved in Step 2 to obtain the translation amounts in the X and Y directions and the rotation angle, and then correspond each point of the current object to be measured with each point of the template image to obtain the standard height values of each point of the current object to be measured.

[0028] Step 4: Finally, based on the absolute phase image and the system calibration parameters, calculate the three-dimensional point cloud of the object to be measured.

[0029] Preferably, in Step 2, for the first object to be measured on the production line, project four sinusoidal gratings further, calculate the wrapped phase using the four-step phase-shifting method, and then project several Gray-code-encoded gratings for phase unwrapping to obtain the absolute phase values of each point; then, calculate the height values of each point inside the object to be measured through the system calibration parameters and record them in a specified file.

[0030] Preferably, in Step 3, for the subsequent objects to be measured on the production line, project four sinusoidal gratings further to calculate the wrapped phase of each point inside the object to be measured; at the same time, for each point inside the object to be measured, inversely deduce the integer part of the absolute phase, i.e., the fringe order, from the standard height value and the tolerance range obtained in Step 4; add the fringe order and the wrapped phase to obtain the final absolute phase value.

[0031] Preferably, the template matching in Step 3 is the grayscale method or the edge method.

[0032] Preferably, the projector is a DLP projector.

[0033] Preferably, the camera is a 2D industrial camera.

[0034] First, the 3D structured light sensor projects a sequence of grating images by a DLP, the 2D industrial camera captures the image sequence, and transmits the images to the computer via USB.

[0035] Second, under the Windows platform, use the C++ and MFC frameworks to control the DLP and the industrial camera through software for three-dimensional reconstruction and graphics display, obtain 5 sequences of grating images for three-dimensional reconstruction, and then complete the extraction of the absolute phase and three-dimensional reconstruction based on the fast grating projection contour method proposed in the present invention.

[0036] In order to overcome the problems in the existing grating projection profilometry and phase-shifting method, such as excessive projection of grating images, low overall speed of three-dimensional reconstruction, and inability to effectively reconstruct dynamic objects, the present invention proposes a new fast grating projection three-dimensional reconstruction method based on prior information and applicable to industrial production lines.

[0037] The present invention effectively utilizes the prior geometric information of the object to be detected on the industrial production line, that is, the object to be detected on the industrial production line is usually a single and unchanging object, and the dimensions (heights) of each part of the object to be detected only vary within a very small range; using the method proposed by the present invention, the projector only needs to project 5 images to perform high-precision three-dimensional reconstruction of the specified object. Therefore, the reconstruction speed is greatly improved, and three-dimensional reconstruction of dynamic objects can be performed to a certain extent, thereby improving the production line efficiency and saving costs.

[0038] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for fast grating projection profilometry based on prior information, characterized in that, The steps are as follows: Step 1: When there is only background but no object to be measured on the industrial production line, a full white image is projected through a projector, and the image is captured and saved through a camera to record the background image on the production line; Step 2: For the first object to be tested on the production line, which is a qualified standard product, a full white image is projected using a projector, and a difference image is obtained by subtracting the full white image from the background image saved in step 1. A reasonable threshold is set and the difference image is binarized. The binarized image is then subjected to morphological processing and Blob analysis, and the Blob area is screened. The largest Blob obtained is the object to be tested, and the image of the object to be tested is saved as a template image. In step 2, four sinusoidal gratings are further projected for the first object to be measured on the production line, and the wrapped phase is calculated using the four-step phase shift method. Then, several Gray code-encoded gratings are projected for phase unfolding to obtain the absolute phase value of each point. Then, the height value of each point inside the object to be measured is calculated by the system calibration parameters and recorded in the specified file. Step 3: For the subsequent objects to be tested on the production line, first use a projector to project a full white image, subtract the full white image from the background image saved in step 1 to obtain a difference image; set a reasonable threshold and perform binarization on the difference image; then perform morphological processing and Blob analysis on the binarized image, screen the Blob by its area, and the largest Blob obtained is the object to be tested; perform template matching on the current object to be tested image and the template image saved in step 2, obtain the translation and rotation angle in the X and Y directions, and then correspond each point of the current object to be tested to each point of the template image to obtain the standard height value of each point of the current object to be tested; In step 3, four sinusoidal gratings are further projected for the subsequent measured objects on the production line to calculate the wrapped phase of each point inside the measured object; at the same time, for each point inside the measured object, the integer part of the absolute phase, that is, the fringe order, is inverted through the standard height value and tolerance range obtained in step 4; the fringe order is added to the wrapped phase to obtain the final absolute phase value; Step 4: Finally, based on the absolute phase image and system calibration parameters, the three-dimensional point cloud of the object under test is calculated.

2. The method for fast grating projection profilometry based on prior information according to claim 1, characterized in that, The template matching in step 3 is a grayscale method or an edge method.

3. The method for fast grating projection profilometry based on prior information according to claim 1, characterized in that, The projector is a DLP projector.

4. The method for fast grating projection profilometry based on prior information according to claim 1, characterized in that, The camera is a 2D industrial camera.