A digital grating projection 3D reconstruction method in mutual reflection environment

Abstract

The application discloses a digital grating projection three-dimensional reconstruction method under mutual reflection, which comprises the following steps: determining a first frequency and a second frequency of a phase shift image and a Gray code image of a mark stripe period, generating a first group of multi-step phase shift images and a second group of multi-step phase shift images respectively, and projecting to an object surface; capturing a stripe image reflected and modulated by the object surface, obtaining a first wrapped phase based on the first group of multi-step phase shift images, obtaining a second wrapped phase based on the second group of multi-step phase shift images, and judging whether two wrapped phases of the same position pixel conform to a wrapped phase relationship determined by the two frequencies; if not, the phase interference occurs at the position pixel and is filtered out; obtaining an absolute phase based on the wrapped phase of the remaining position pixel and the number of stripe periods determined by the Gray code image, combining a calibrated phase and height conversion model to calculate the height of the object, and generating an object point cloud. The application can effectively remove the error phase caused by mutual reflection.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a digital grating projection three-dimensional reconstruction method in an interreflection environment. Background Art

[0002] 3D reconstruction technology has a wide range of applications in modern industrial design, reverse engineering, medical imaging, cultural heritage preservation, and other fields. Grating projection 3D reconstruction technology, due to its non-contact, high-precision, and high-efficiency characteristics, has become a key method for 3D topography measurement. This technology projects a structured light pattern (such as a sinusoidal grating) onto the surface of the object being measured. A camera captures the modulated grating image of the object's surface, and then uses phase calculation and 3D reconstruction algorithms to recover the object's 3D topography.

[0003] In practical applications, grating projection 3D reconstruction technology faces many challenges, one of which is interreflection. Interreflection occurs when light is captured by a camera after multiple reflections on an object or scene. This phenomenon is particularly pronounced in complex geometries, highly reflective surfaces, or enclosed scenes. Interreflection causes the light intensity information received by the camera to contain a mixture of direct and multiple reflections, introducing phase measurement errors and reducing the accuracy and reliability of 3D reconstruction. Specifically, interreflection can cause false features, geometric distortion, or loss of detail in the reconstructed surface, severely impacting the quality of the measurement results.

[0004] Existing grating projection 3D reconstruction techniques typically assume that light undergoes only a single reflection, ignoring the effects of interreflection. Consequently, the measurement accuracy of traditional methods often falls short of practical requirements for highly reflective objects or complex scenes. To address this issue, a technical solution is needed that can remove phase errors caused by interreflection, thereby improving the accuracy and robustness of grating projection 3D reconstruction. Summary of the Invention

[0005] The purpose of the present invention is to provide a digital grating projection three-dimensional reconstruction method in an interreflection environment, and propose a phase resolution technology solution that can effectively remove the erroneous phase caused by interreflection, so as to improve the accuracy and robustness of grating projection three-dimensional reconstruction.

[0006] To achieve the above object, according to a first aspect of the present invention, a method for three-dimensional reconstruction using digital grating projection in an interreflective environment is provided, the method comprising:

[0007] Perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model;

[0008] Determining a first frequency and a second frequency of the phase-shifted image and a Gray code image marking a fringe period, generating a first set of multi-step phase-shifted images based on the first frequency, generating a second set of multi-step phase-shifted images based on the second frequency, and projecting them onto the surface of the object;

[0009] Capturing a fringe image reflected and modulated by the object surface, obtaining a first wrapping phase based on the fringe image corresponding to the first set of multi-step phase-shifted images, and obtaining a second wrapping phase based on the fringe image corresponding to the second set of multi-step phase-shifted images, and determining whether the first wrapping phase and the second wrapping phase of a pixel at the same position conform to the wrapping phase relationship determined by the first frequency and the second frequency; if not, the pixel at that position undergoes phase interference and is filtered out;

[0010] The absolute phase of the remaining pixels is obtained based on the wrapped phase of the remaining pixels and the number of fringe cycles determined by the Gray code image. The height of the object is calculated by combining the calibrated phase and height conversion model to generate the object point cloud.

[0011] In the above scheme, the phase-height calibration of the digital grating projection system based on the phase-height method is performed, and a phase-height conversion model is established, including:

[0012] With T as the period of the phase-shifted image, n phase-shifted images are generated for calculating the wrapped phase. The formula for generating the phase-shifted image is:

[0013]

[0014]

[0015] Where, is the pixel value at the horizontal coordinate x in the kth phase-shifted image, is the wrapping phase at the horizontal coordinate x, T is the fringe period of the phase-shifted image, and l is the horizontal coordinate length of the phase-shifted image;

[0016] When the phase-shift image is a four-step phase-shift image, the captured image grayscale is represented as:

[0017]

[0018] Where, The pixel coordinates are The reflectivity at , A is the background light intensity of the image, B is the modulated light intensity, For the wrapping phase;

[0019] Since the reflectivity at each pixel is different, the formula is transformed into:

[0020]

[0021]

[0022]

[0023] The above formula is expanded to become:

[0024]

[0025] Let the unknown term be , Let the unknown term , the captured four images are expressed in the form of overdetermined equations, then:

[0026] =

[0027] The solution of the equation is:

[0028]

[0029] =

[0030] =

[0031] Solve the equation of the wrapped phase :

[0032]

[0033] The following formula is used to obtain the absolute phase of the image:

[0034]

[0035] Use the phase height model based on the control equation to realize the conversion of absolute phase and height :

[0036]

[0037] Wherein, and are obtained by the absolute phase of the four groups of images with known height.

[0038] In the above scheme, the Gray code is used to mark and calculate the period of the stripe, including:

[0039] The number of stripe periods that need to be marked is calculated as , and the number of Gray code images required is determined as , and the Gray code image is generated based on the conventional Gray code generation rule to mark the period of the stripe; wherein, represents the upward rounding function; ​

[0040] The number of fringe periods of the Gray code marked fringe is is obtained by the following formula:

[0041]

[0042]

[0043] wherein, is the image gray value of the pixel position at the coordinate of the i-th Gray code image, is the image gray value of the pixel position at the coordinate of the i-th Gray code image, is the decimal Gray code corresponding to the pixel position at the coordinate of the i-th Gray code image. In the above scheme, one of the first frequency and the second frequency of the phase shift images is the frequency of the phase shift image in the calibration, the Gray code image is used to mark the fringe period of the phase shift image of the frequency, and the absolute phase of the remaining position pixels is obtained based on the wrapped phase of the phase shift image of the frequency and the number of fringe periods determined by the Gray code image.

[0044] In the above scheme, the second frequency is twice or half of the first frequency, that is, the fringe period of the second group of multi-step phase shift images is half or twice of the fringe period of the first group of multi-step phase shift images.

[0045] In the above scheme, when the fringe period of the second group of multi-step phase shift images is half of the fringe period of the first group of multi-step phase shift images, the wrapped phase relationship is:

[0046]

[0047] wherein,

[0048] is the first wrapped phase, is the second wrapped phase. In the above scheme, the digital light grid projection system based on the phase-height method includes a digital projector, a telecentric lens, a camera and a computer; the object is placed in the common field of view of the digital projector and the camera, the digital projector projects two groups of multi-step phase shift images and Gray code images to the object according to a preset phase sequence, the camera captures the fringe image reflected and modulated by the object surface through the telecentric lens; the computer unwraps the phase and maps the phase-height, and establishes a conversion model of the phase and the height.

[0049] According to the second aspect of the present application, a digital light grid projection three-dimensional reconstruction device in a mutual reflection environment is provided, which comprises:

[0050] a calibration unit configured to calibrate the phase-height of the digital light grid projection system based on the phase-height method, and establish a conversion model of the phase and the height;

[0051]

[0052] ​a projection unit, configured to determine a first frequency and a second frequency of the phase-shifted image and a Gray code image marking a fringe period, generate a first set of multi-step phase-shifted images based on the first frequency, generate a second set of multi-step phase-shifted images based on the second frequency, and project the images onto the surface of the object;

[0053] a filtering unit configured to capture a fringe image reflected and modulated by the surface of an object, obtain a first wrapping phase based on the fringe image corresponding to the first set of multi-step phase-shifted images, obtain a second wrapping phase based on the fringe image corresponding to the second set of multi-step phase-shifted images, and determine whether the first wrapping phase and the second wrapping phase of a pixel at the same position conform to a wrapping phase relationship determined by the first frequency and the second frequency; if not, the pixel at that position undergoes phase interference and is filtered out;

[0054] The generation unit is used to obtain the absolute phase of the remaining position pixels based on the wrapped phase of the remaining position pixels and the number of fringe cycles determined by the Gray code image, and calculate the height of the object in combination with the calibrated phase and height conversion model to generate the object point cloud.

[0055] According to a third aspect of the present invention, a computer device is provided, comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the digital grating projection three-dimensional reconstruction method in an interreflection environment described in any one of the first aspects are implemented.

[0056] According to a fourth aspect of the present invention, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the digital grating projection three-dimensional reconstruction method in an interreflection environment described in any one of the first aspects are implemented.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] The present invention proposes a digital grating projection three-dimensional reconstruction method in an inter-reflection environment. By projecting two sets of multi-step phase-shifted images with a specific frequency relationship and combining them with Gray code to mark the fringe period, efficient detection and filtering of phase interference in an inter-reflection environment is achieved. This method uses the theoretical relationship between phases wrapped at different frequencies to accurately identify and eliminate erroneous phase areas caused by multiple reflections, highlights or shadows, effectively solving the problems of phase jumps, false features and geometric distortion caused by inter-reflection in traditional grating projection three-dimensional reconstruction. At the same time, combined with a calibrated phase-height conversion model, the absolute phase information of the object can be accurately recovered from the remaining correct phase, generating a high-precision, low-noise three-dimensional point cloud, which significantly improves the robustness and reliability of three-dimensional morphology reconstruction in complex reflection scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1A schematic flow chart of a digital grating projection 3D reconstruction method in an interreflective environment provided by an embodiment of the present application;

[0060] Figure 2 An absolute phase calculation rule diagram of a traditional phase shift method + Gray code used to obtain absolute phase in phase height calibration provided in an embodiment of the present application;

[0061] Figure 3 A wrapping phase comparison diagram obtained under ideal conditions using two frequencies for detecting phase errors provided in an embodiment of the present application;

[0062] Figure 4 A wrapped phase map containing an erroneous phase obtained after an image is captured by a camera in an interreflection scene provided in an embodiment of the present application;

[0063] Figure 5 An error position map marked after filtering out error areas using an error phase filtering algorithm provided in an embodiment of the present application;

[0064] Figure 6 An embodiment of the present application provides an object point cloud image reconstructed without filtering errors in an interreflection scene;

[0065] Figure 7 An object point cloud image generated after filtering out erroneous areas in an interreflection scene provided in an embodiment of the present application;

[0066] Figure 8 A framework diagram of a digital grating projection 3D reconstruction device in an interreflective environment provided by an embodiment of the present application;

[0067] Figure 9 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0069] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present invention.

[0070] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0071] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0072] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0073] The present application provides a method for digital grating projection 3D reconstruction in an interreflective environment, specifically a method for filtering phase errors caused by interreflection in digital grating projection 3D reconstruction. The method comprises the following steps: first, calibrating the phase height of the digital grating projection system to establish a conversion model between absolute phase and object height; then, using a projector to project two specific frequency four-step phase shift patterns and a Gray code image, and capturing the image after the object is reflected by a camera; then, calculating the object's wrapping phase in a computer and applying an error phase filtering algorithm; finally, calculating the object's height using the calibrated phase height model and the absolute phase after filtering out the error phase, to obtain an object point cloud. This method can effectively remove error phases caused by shadows, highlights, and interreflections, and can improve the accuracy of traditional digital grating projection 3D reconstruction systems.

[0074] The digital grating projection 3D reconstruction method in the mutual reflection environment of the present application is as follows: Figure 1 As shown, the following steps are included:

[0075] S1. Perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model;

[0076] S2. Determine the two frequencies of the phase-shift image to be projected and the Gray code image marking the fringe period, generate n×2 multi-step phase-shift images and m Gray code images, and project m+2n images onto the surface of the object through a projector;

[0077] S3, using a camera to capture the deformed image after reflection and modulation of the object surface, calculating the object's wrapped phase, and using an error filtering algorithm to filter out the erroneous absolute phase caused by shadows, highlights, and mutual reflections;

[0078] S4. Calculate the height of the object using the calibrated phase height model and the absolute phase after filtering out the erroneous phase to obtain the object point cloud.

[0079] In some embodiments, a digital grating projection system based on the phase-height method includes a high-resolution camera, a digital projector, a computer, and a telecentric lens. The object to be measured is placed within the common field of view of the digital projector and the high-resolution camera. The digital projector projects multiple phase-shifted grating patterns and Gray code patterns onto the surfaces of multiple sets of platforms at different heights according to a preset phase sequence, and the high-resolution camera captures the modulated fringe images. The captured images are then phase-unwrapped and phase-to-height mapped by a computer to establish a conversion model between absolute phase and object height. The purpose of phase-height calibration is to determine the mapping relationship between absolute phase and height so that the height can be subsequently determined based on the absolute phase.

[0080] In some embodiments, step S2 is specifically as follows: on the basis of the four-step phase-shifted images and Gray code images originally used for phase height calibration, a group of grating stripe images with a frequency twice or half of the original frequency is added. Then, the total number of images used to reconstruct the interreflection scene is now two groups of n-step phase-shifted images and one group of Gray code images, and the image is projected again onto the object to be reconstructed using a projector.

[0081] This step requires generating 2n phase-shifted images (n is the number of phase-shift steps) with two frequencies that have a certain relationship, and m Gray codes used to mark the fringe period. The two parcel phases obtained from these frequency-related phase-shifted images will also have a certain relationship. This relationship allows us to determine which pixels have severely disturbed parcel phases and filter out those locations.

[0082] In some embodiments, step S3 specifically includes: adjusting parameters such as the camera exposure value, projecting the phase shift map of the two frequencies generated in the previous step and the Gray code image used to mark the fringe period onto the object, and capturing the improved multi-step phase shift map and Gray code image after being deformed by the object reflection, filtering the erroneous positions based on the error filtering algorithm, and obtaining the absolute phase of the object through the remaining correct wrapped phase and fringe period.

[0083] Specifically, the image modulated by the object is captured by a camera, and the error detection of the wrapping phase is performed in the computer. Since the frequencies of the two sets of phase-shifted images have a certain relationship, the obtained wrapping phases also have a certain relationship. If the pixel at a certain position does not conform to the specified wrapping phase relationship, it can be determined that the wrapping phase at that position is severely interfered with, and the position can be filtered out. After obtaining the accurate wrapping phase and the corresponding fringe period, the correct absolute phase on the object can be obtained.

[0084] In some embodiments, step S4 specifically involves combining the obtained absolute phase of the object after filtering out erroneous phases with the phase-height mapping relationship in step S1 to obtain a point cloud of the object. Specifically, after filtering out the erroneous wrapping phase in step S3, the remaining accurate wrapping phase is obtained. Combined with the number of fringe periods determined from the Gray code image, the absolute phase of the object can be determined. Using the phase-height conversion model calibrated in step S1, the obtained absolute phase can be converted into the corresponding object point cloud, ultimately achieving accurate point cloud reconstruction of the object in the interreflection scene.

[0085] Specifically, the digital grating projection 3D reconstruction method under the mutual reflection environment of the embodiment of the present application is as follows: Figure 1 As shown, the following steps are included:

[0086] S1. Perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model.

[0087] In this embodiment, a digital grating projection system based on the phase-height method includes a high-resolution camera, a digital projector, a computer, and a telecentric lens. This step is primarily used to determine the conversion relationship between absolute phase and height. The projector in this embodiment can project a maximum image size of 1140×912. With T as the period of the phase-shifted image, n phase-shifted images are generated for calculating the wrapped phase. The formula for generating the phase-shifted image is:

[0088]

[0089]

[0090] Where, is the pixel value at the x-axis in the k-th image, is the wrapping phase at the horizontal axis x, and T is the fringe period. Taking a four-step phase shift image (n=4) as an example, we explain how to obtain the absolute phase on the image through the four images captured by the camera:

[0091] Due to the different reflectivity of the object surface and the different background light intensities, when the four images generated above are projected four times by the projector and captured four times by the camera, the captured image grayscale can be expressed as:

[0092]

[0093] Where A is the background light intensity of the image, B is the modulated light intensity, The pixel coordinates are The reflectivity at each pixel is different, so the formula can be transformed into:

[0094]

[0095] Expanding the above formula becomes:

[0096]

[0097] The unknown term can be expressed in four phase-shifted images. Set as , Set as , the four captured images are expressed as overdetermined equations:

[0098] =

[0099] The solution to this equation is:

[0100]

[0101] =

[0102] =

[0103] Solving for the wrapped phase The equation is:

[0104]

[0105] This embodiment uses Gray code to mark and calculate the fringe period. Taking T=56 as an example, there are 1140 / 56≈21 periods that need to be marked, so at least 5 Gray code images are required ( ), the traditional Gray code generation rule for marking the stripe period is as follows Figure 2 As shown. The number of fringe periods of Gray code marking is . Obtained by the following formula:

[0106]

[0107]

[0108] Get the wrapped phase and the number of fringe cycles Then, the absolute phase of the image is obtained using the following formula:

[0109]

[0110] After research, the absolute phase of the image and height There is a linear relationship between them, and many methods can be used to fit the relationship between them. This experiment uses a phase height model based on the control equation to achieve absolute phase and height Conversion:

[0111]

[0112] in, and The absolute phase is obtained by taking four sets of images with known heights.

[0113] S2. Determine the two frequencies of the phase-shift image to be projected and the Gray code image marking the fringe period, generate n×2 multi-step phase-shift images and m Gray code images, and project m+2n images onto the object surface through a projector.

[0114] Since the images traditionally used for calibration include a set of four-step phase-shifted images and a set of Gray code images, in order to detect the wrong phase, this application adds a set of four-step phase-shifted images with half or twice the original fringe period. This experiment takes the original fringe period of 56 as an example and adds a set of four-step phase-shifted images with a period of 28. The specific principle is explained in the following step S3.

[0115] S3. Use a camera to capture the deformed pattern image after reflection and modulation on the object surface, calculate the absolute phase image of the object, and use an error filtering algorithm to filter out the erroneous absolute phase caused by shadows, highlights, and mutual reflections.

[0116] In order to detect the location where the wrapped phase is disturbed in the interreflection scene, we first analyze the reasons why interreflection affects the wrapped phase calculation. Taking the four-step phase shift as an example, the formula after expanding the unwrapped phase above is:

[0117]

[0118]

[0119] When mutual reflection occurs, the image captured by the camera is no longer the grayscale value obtained after one reflection of the object, but contains grayscale from other positions, that is:

[0120]

[0121]

[0122] in, is the reflectivity of the grating t projected on the object surface once, is the reflectivity of the grating due to multiple reflections on the surface of the object, The abnormal phase is caused by the light at other positions being captured by the camera after multiple reflections. The unwrapped phase formula is further simplified as follows:

[0123]

[0124]

[0125] At this time, the obtained There will be an offset from the correct wrapping phase, and the offset is affected by the reflectivity at other locations. and interference phase and The difference Δ Influence, and All from (- , ), the maximum offset can be close to 2 This also shows that mutual reflection has a huge impact on the calculation of the package phase, and because the interference source is uncontrollable, the traditional sinusoidal grating pattern cannot effectively deal with the mutual reflection scene. When the period T of the projected stripes is determined, and The difference Δ is a fixed value, so by changing the fringe period, the difference Δ is changed For example, the period of the first original fringe is T1, the period of the second fringe is T2, and T1 is twice T2, the first obtained wrapping phase is , the second obtained wrapping phase is When no phase interference occurs, the ideal phase relationship between the two packages is obtained at once as Figure 3 As shown, the wrapping phase relationship is:

[0126]

[0127] If phase interference occurs at this time, it can be divided into four cases. The first case is that both the interfering phase and the interfered phase become half of the original. Since the relative positions of the object, camera and projector have not changed, the reflectivity The error phase obtained by changing the fringe period is unchanged. for:

[0128]

[0129] The second and third are the correct phases and interference phase One side becomes half of the original, and the wrong phase is obtained after changing the fringe period. for:

[0130]

[0131] The fourth case is the correct phase and interference phase The error phase obtained after changing the fringe period is not changed to half of the original for:

[0132]

[0133] This embodiment uses a metal ring as the reconstruction object. After the camera captures the image and calculates the phase, the wrapping phase of the metal ring at two frequencies is as follows: Figure 4 During the reconstruction process, The value of is always within (0, 1), and the two wrapped phases obtained by interfering with the position before and after the change of the period are and It is very difficult to satisfy the twice-obtained wrapping phase in the normal position and The relationship satisfied is used to detect the obtained wrapped phase on this basis, which can filter out the erroneous phase caused by shadows, highlights and mutual reflections. The error areas marked after detection by the error detection algorithm are as follows Figure 5 As shown, Figure 5 The white point in the image is the point that needs to be filtered out. The phase of the shadow part is diffused from other positions, that is, The phase calculation error is unknown. The highlight part has too strong reflection and the pixel grayscale reaches the upper limit of the image. The phase calculation error is due to phase overlap and the reflectivity cannot be confirmed. Therefore, it is difficult to use compensation to restore the correct phase. In this experiment, whether the wrapped phases obtained from the two cycles satisfy the normal correspondence is used to judge whether the position has been phase-interfered, thereby removing the wrapped phases in the erroneous area.

[0134] For position pixels that meet the wrapping phase relationship, the absolute phase is obtained by combining the number of fringe periods determined by the Gray code image.

[0135] S4. Calculate the height of the object using the calibrated phase height model and the absolute phase after filtering out the erroneous phase to obtain the object point cloud.

[0136] Through the conversion model of the absolute phase obtained in step S3 and the phase and height obtained in step S1, the absolute phase of the object can be reconstructed into the point cloud of the object. The original point cloud of the object without filtering the error area is as follows: Figure 6 As shown, the original unfiltered point cloud has more noise, and the object point cloud after filtering the error area is as follows Figure 7 As shown in the figure, it can be seen that the erroneous point clouds in the shadow area, highlight area, and area seriously interfered by interreflection can be filtered out normally. The experiment shows the feasibility of the phase error filtering method caused by interreflection in digital grating projection 3D reconstruction.

[0137] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0138] The embodiments of the present application also provide a digital grating projection three-dimensional reconstruction device in an interreflective environment. These devices are used to implement the above-mentioned embodiments and preferred embodiments, and those that have been described will not be repeated. As used below, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0139] Figure 8 : is a structural block diagram of a digital grating projection 3D reconstruction device in a mutual reflection environment according to an embodiment of the present application. Figure 8 As shown, the device includes a calibration unit 201, a projection unit 202, a filtering unit 203 and a generation unit 204; wherein,

[0140] The calibration unit 201 is used to perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model;

[0141] A projection unit 202 is configured to determine a first frequency and a second frequency of the phase-shifted image and a Gray code image marking a fringe period, generate a first set of multi-step phase-shifted images based on the first frequency, generate a second set of multi-step phase-shifted images based on the second frequency, and project the images onto the surface of the object;

[0142] The filtering unit 203 is configured to capture the fringe image reflected and modulated by the object surface, obtain a first wrapping phase based on the fringe image corresponding to the first set of multi-step phase-shifted images, and obtain a second wrapping phase based on the fringe image corresponding to the second set of multi-step phase-shifted images, and determine whether the first wrapping phase and the second wrapping phase of a pixel at the same position conform to the wrapping phase relationship determined by the first frequency and the second frequency; if not, the pixel at that position undergoes phase interference and is filtered out.

[0143] The generating unit 204 is configured to obtain the absolute phase of the pixels at the remaining positions based on the wrapped phase of the pixels at the remaining positions and the number of fringe cycles determined by the Gray code image, and calculate the height of the object in combination with the calibrated phase-height conversion model to generate an object point cloud.

[0144] It should be noted that the above-mentioned units can be functional units or program units, and can be implemented by software or hardware. For units implemented by hardware, the above-mentioned units can be located in the same processor; or the above-mentioned units can be located in different processors in any combination.

[0145] In addition, combined Figure 1 The digital grating projection three-dimensional reconstruction method in a mutual reflection environment described in the embodiment of the present application can be implemented by a computer device. Figure 9Schematic diagram of the hardware structure of the computer device of the embodiment of the present application. Figure 9 As shown, the device may include a processor 301 and a memory 302 storing computer program instructions.

[0146] Specifically, the processor 301 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0147] Memory 302 may include a large-capacity memory for data or instructions. By way of example, and not limitation, memory 302 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the data processing device. In certain embodiments, memory 302 is non-volatile memory. In certain embodiments, memory 302 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0148] The memory 302 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 301 .

[0149] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any one of the digital grating projection three-dimensional reconstruction methods in the mutual reflection environment in the above embodiments.

[0150] In some embodiments, the computer device may further include a communication interface 303 and a bus 300. Figure 9 As shown, the processor 301 , the memory 302 , and the communication interface 303 are connected via a bus 300 and communicate with each other.

[0151] The communication interface 303 is used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication interface 303 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0152] Bus 300 includes hardware, software, or both, and couples components of a computer device to each other. Bus 300 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example, and not limitation, bus 300 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 300 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0153] The computer device can execute the digital grating projection three-dimensional reconstruction method under the mutual reflection environment in the embodiment of the present application based on the computer device, thereby realizing the combination of Figure 1 The digital grating projection 3D reconstruction method in an interreflection environment is described.

[0154] In addition, in conjunction with the digital grating projection 3D reconstruction method in an interreflective environment in the above-mentioned embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the digital grating projection 3D reconstruction method in an interreflective environment in any of the above-mentioned embodiments is implemented.

[0155] It should be noted that the various technical features of the above-described embodiments can be combined in any manner. To simplify the description, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there are no contradictions in the combination of these technical features, they should be considered to be within the scope of this specification. In addition, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0156] Those skilled in the art will readily understand that the above-described embodiments merely represent several implementation methods of the present application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make several variations and improvements without departing from the concept of the present application, and these variations and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be based on the appended claims.

Claims

1. A digital grating projection 3D reconstruction method in an interreflective environment, characterized in that: The method includes: Perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model; Determining a first frequency and a second frequency of the phase-shifted image and a Gray code image marking a fringe period, generating a first set of multi-step phase-shifted images based on the first frequency, generating a second set of multi-step phase-shifted images based on the second frequency, and projecting them onto the surface of the object; wherein the second frequency is twice or half of the first frequency, that is, the fringe period of the second set of multi-step phase-shifted images is half or twice the fringe period of the first set of multi-step phase-shifted images; Capturing a fringe image reflected and modulated by the object surface, obtaining a first wrapping phase based on the fringe image corresponding to the first set of multi-step phase-shifted images, and obtaining a second wrapping phase based on the fringe image corresponding to the second set of multi-step phase-shifted images, and determining whether the first wrapping phase and the second wrapping phase of a pixel at the same position conform to the wrapping phase relationship determined by the first frequency and the second frequency; if not, the pixel at that position undergoes phase interference and is filtered out; The absolute phase of the remaining pixels is obtained based on the wrapped phase of the remaining pixels and the number of fringe cycles determined by the Gray code image. The height of the object is calculated by combining the calibrated phase and height conversion model to generate the object point cloud. Gray code is used to mark and calculate the stripe period, including: Calculate the number of fringe cycles that need to be marked as , determine the minimum number of Gray code images required , based on the traditional Gray code generation rules, a Gray code image is generated to mark the fringe period; where T is the period of the phase-shifted image, l is the horizontal coordinate length of the phase-shifted image, represents the ceiling function; The number of fringe periods of the Gray code mark is , obtained by the following formula: Where, The coordinates of the i-th Gray code image are The image grayscale value of the pixel position at The coordinates of the image are The decimal Gray code corresponding to the pixel position at .

2. The digital grating projection 3D reconstruction method in an interreflective environment according to claim 1, characterized in that: Perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model, including: With T as the period of the phase-shifted image, n phase-shifted images are generated for calculating the wrapped phase. The formula for generating the phase-shifted image is: Where, is the pixel value at the horizontal coordinate x in the kth phase-shifted image, is the wrapping phase at the horizontal coordinate x, T is the fringe period of the phase-shifted image, and l is the horizontal coordinate length of the phase-shifted image; When the phase-shift image is a four-step phase-shift image, the captured image grayscale is represented as: Where, The pixel coordinates are The reflectivity at , A is the background light intensity of the image, B is the modulated light intensity, For the wrapping phase; Since the reflectivity at each pixel is different, the formula is transformed into: Expanding the above formula becomes: The unknown Set as , Set as , the four captured images are expressed as overdetermined equations: = The solution to this equation is: = = Solving for the wrapped phase The equation is: The absolute phase of the image is obtained using the following formula: Absolute phase is achieved using a phase height model based on the governing equations and height Conversion: in, and The absolute phase is obtained by taking four sets of images with known heights.

3. The digital grating projection 3D reconstruction method in an interreflective environment according to claim 1, characterized in that: One of the first frequency and the second frequency of the phase-shifted image is the frequency of the phase-shifted image during calibration. The Gray code image is used to mark the fringe period of the phase-shifted image at this frequency, and the absolute phase of the pixels at the remaining positions is obtained based on the wrapped phase of the phase-shifted image at this frequency and the number of fringe periods determined by the Gray code image.

4. The digital grating projection 3D reconstruction method in an interreflective environment according to claim 1, characterized in that: When the fringe period of the second set of multi-step phase-shifted images is half of the fringe period of the first set of multi-step phase-shifted images, the wrapped phase relationship is: Where, is the first wrapping phase, It is the second wrapping phase.

5. The digital grating projection 3D reconstruction method in an interreflection environment according to claim 1, characterized in that: The digital grating projection system based on the phase-height method includes a digital projector, a telecentric lens, a camera and a computer; the object is placed in the common field of view of the digital projector and the camera, and the digital projector projects two sets of multi-step phase-shifted images and Gray code images onto the object according to a preset phase sequence. The camera captures the fringe image reflected and modulated by the object surface through the telecentric lens; the computer performs phase unwrapping and phase-height mapping on the captured image to establish a phase and height conversion model.

6. A digital grating projection 3D reconstruction device in an interreflective environment, characterized in that: The device includes: A calibration unit is used to perform phase-height calibration on the digital grating projection system based on the phase-height method and establish a phase-height conversion model; a projection unit, configured to determine a first frequency and a second frequency of the phase-shifted image and a Gray code image marking a fringe period, generate a first set of multi-step phase-shifted images based on the first frequency, generate a second set of multi-step phase-shifted images based on the second frequency, and project the images onto the surface of the object; wherein the second frequency is twice or half the first frequency, i.e., the fringe period of the second set of multi-step phase-shifted images is half or twice the fringe period of the first set of multi-step phase-shifted images; a filtering unit configured to capture a fringe image reflected and modulated by the surface of an object, obtain a first wrapping phase based on the fringe image corresponding to the first set of multi-step phase-shifted images, obtain a second wrapping phase based on the fringe image corresponding to the second set of multi-step phase-shifted images, and determine whether the first wrapping phase and the second wrapping phase of a pixel at the same position conform to a wrapping phase relationship determined by the first frequency and the second frequency; if not, the pixel at that position undergoes phase interference and is filtered out; a generation unit, configured to obtain the absolute phase of the pixels at the remaining positions based on the wrapped phase of the pixels at the remaining positions and the number of fringe periods determined by the Gray code image, and calculate the height of the object in combination with a calibrated phase-height conversion model to generate a point cloud of the object; Gray code is used to mark and calculate the stripe period, including: Calculate the number of fringe cycles that need to be marked as , determine the minimum number of Gray code images required , based on the traditional Gray code generation rules, a Gray code image is generated to mark the fringe period; where T is the period of the phase-shifted image, l is the horizontal coordinate length of the phase-shifted image, represents the ceiling function; The number of fringe periods of the Gray code mark is , obtained by the following formula: Where, The coordinates of the i-th Gray code image are The image grayscale value of the pixel position at The coordinates of the image are The decimal Gray code corresponding to the pixel position at .

7. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the digital grating projection three-dimensional reconstruction method in an interreflection environment described in any one of claims 1 to 5 are implemented.

8. A readable storage medium, characterized in that: Programs or instructions are stored thereon, and when the programs or instructions are executed by a processor, the steps of the digital grating projection three-dimensional reconstruction method in a mutual reflection environment described in any one of claims 1 to 5 are implemented.

Images