A method for dynamic 3D scanning of a spatial object and a dynamic 3D scanner

The dynamic 3D scanning method using a variable-frequency stripe pattern and convolutional neural networks addresses the challenges of noise and coding frame requirements, achieving robust and accurate single-frame 3D image reconstruction for moving objects.

WO2025045591A1PCT designated stage expired Publication Date: 2025-03-06PIECHOCKI PIOTR +2
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2024/072884
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-08-28
Filing Date
2024-08-14
Publication Date
2025-03-06

AI Technical Summary

Technical Problem

Existing methods for dynamic 3D scanning of spatial objects are prone to errors and delays due to noise in input data and the need for coding frames, which affects accuracy and robustness, especially when objects are in motion.

Method used

A method using a dynamic 3D scanner that projects a variable-frequency stripe pattern formed with a periodic function, allowing for single-frame reconstruction of 3D images. This method employs convolutional neural networks to generate probability maps, filter out noise, and reconstruct 3D images with enhanced accuracy and robustness.

Benefits of technology

The method significantly improves the robustness of dynamic 3D scanning to noise in input data, enables accurate reconstruction of 3D images from a single frame, and allows for scanning of moving objects without delays, thereby enhancing overall scanning efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024072884_06032025_PF_FP_ABST
    Figure EP2024072884_06032025_PF_FP_ABST
Patent Text Reader

Abstract

A method for dynamic 3D scanning of a spatial object comprising the following steps: projecting (101) a stripe pattern (4) onto a spatial object (5) by a projector (2) of a dynamic 3D scanner (1), wherein the stripe pattern (4) is a variable-frequency pattern formed with the usage of a periodic function, and the stripe pattern (4) is arranged such that the width / height of its stripes corresponds to an oscillation period s i of eight pixels or its multiples, where a maximum value of the oscillation period s i , is limited to a maximum value of the resolution of the projector (2), wherein i = (0, 1,..., n); the stripes form a sequence S of said oscillation periods s i , wherein at most three of consecutive oscillation periods s i may have the same value, wherein i = (0, 1,...,n); the sum of the oscillation periods s i , forming the sequence S must be equal to or greater than the maximum value of the resolution of the projector (2), wherein i = (0, 1,...,n); and the method comprises further the steps of capturing (102) by a camera (3) of the dynamic 3D scanner (1) a frame of the spatial object (5) with the stripe pattern (4) onto its surface, generating (103) a map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object (5) from the captured frame, wherein said generation is done by a first convolutional network; thresholding (104) the map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object (5) to obtain (105) a shadow mask comprising pixels excluded from the reconstruction of a 3D image of the spatial object (5), correlating (106) the captured frame with the shadow mask to obtain (107) a frame comprising the spatial object (5) with the stripe pattern (4) onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object (5), generating (109) a map of the imaginary component of a signal with a variable frequency and a map of the real component of a signal with a variable frequency from the frame comprising the spatial object (5) with the stripe pattern (4) onto its surface with blanked out pixels excluded from said reconstruction, wherein said generation is done by a second convolutional network; generating (110) a wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency, generating (116) a pixel map with data comprising assigned oscillation period numbers and a map of probability values of assigning oscillation period numbers from the wrapped phase map with a variable frequency, wherein said generation is done by a third convolutional network; filtering out (117) from the pixel map with data comprising assigned oscillation period number areas having on the map of probability values of assigning the oscillation period numbers a value ≤ 0.99, reconstructing (118) the filtered out areas from the pixel map, generating (119) an unwrapped phase map with a variable frequency from the wrapped phase map with a variable frequency and the pixel map with reconstructed (118) filtered out areas, transforming (120) the unwrapped phase map with a variable frequency into an unwrapped phase map with a constant frequency so as to obtain a uniform oscillation period over the entire frequency domain of the unwrapped phase map, reconstructing (121) a 3D image of the spatial object based on the unwrapped phase map with a constant frequency.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] A method for dynamic 3D scanning of a spatial object and a dynamic 3D scanner

[0002] Technical field

[0003] The present invention relates to a method for dynamic 3D scanning of a spatial object and a dynamic 3D scanner.

[0004] Background of the invention

[0005] From Chinese patent CN111383234B there is known an online dense 3D reconstruction method based on machine learning that uses structured light. Said method comprises the steps of calibrating parameters of camera-projector system; acquiring an image using a projector to project coded structured light onto a surface of an object to be measured and using a camera to collect a structured light image modulated by the surface of the object; removing noise in the collected image to improve image quality; recognizing color using a neural network model; filtrating images based on recognition of light bar parameters; detecting a light bar center line and determining a corresponding index of the light bar center line in a light bar sequence; structured light decoding according to the brightness of each pixel point and the light bar index to obtain a phase corresponding to the images acquired by the camera and acquiring a set of corresponding points; based on a trigonometric intersection algorithm in photogrammetry, intersecting each group of matching point pairs in the set of corresponding point, and decomposing spatial three-dimensional coordinates of the matching point pairs, and then completing dense three-dimensional reconstruction of surfaces of the object. The object of the invention is to improve calculation efficiency and stability of said method comparing to the known solution.

[0006] From international PCT application W02020102814A1 there are known a method for processing interferograms or deformed fringes and a device for performing optical system design. The method comprising obtaining an input phase image based on the interferograms or the deformed fringes, wherein each pixel of the input phase image is associated with a phase value and the deformed fringes are obtained by projecting a set of pre-deformed fringes on an object; segmenting the input phase image by classifying the input phase image into multiple regions based on the phase value and a location of each pixel; assigning an integer value to each of the multiple regions, wherein the integer value indicates that a corresponding region has a phase offset equal to a product of 2p and the integer value; and constructing an output phase image based on the phase value of each pixel and the phase offset of each of the multiple regions. The multiple regions and the integer value corresponding to each of the multiple regions are determined by a convolutional neural network. Furthermore, the input phase image is denoised by using a neural network that is trained using only noisy data. In turn, the device configured to perform the steps of said method.

[0007] Further, from international PCT application W02020063013A1 there are known a fringe projection time phase unwrapping method based on deep learning. The method comprises the steps of projecting by a projector four sets of three-step phase-shifted patterns on a measured object, wherein frequencies of the four sets of the patterns are 1, 8, 32, and 64; acquiring a raster image to obtain a wrapped phase maps and calculating the wrapped phase maps of different frequencies; performing phase unwrapping on the wrapped phase maps to obtain a periodic level diagram and an absolute phase diagram of a phase with the frequency of 64; building a residual convolutional neural network, wherein the input data are the wrapped phase maps with the frequencies of 1 and 64 and the output data is the periodic level diagram of the phase with the frequency of 64; and finally a training set and a verification set are made to train and verify said network, wherein said network verifies the test set to output the periodic level diagram of the phase with the frequency of 64. As a result, the method is adopted and the wrapped phase maps with the frequency of 1 are used for unwrapping the wrapped phase maps with the frequency of 64. The purpose of the aforementioned solution is to provide a method for time-phase extension of fringe projection based on deep learning, while providing a highly accurate three-dimensional measurement without increasing the number of projection patterns.

[0008] Chinese patent application CN111523618A discloses a phase unwrapping method based on deep learning comprising the steps of providing fringe-encoded images by a projector; collecting the fringe- encoded images by a camera; obtaining a wrapped phase information of the fringe-encoded images by means of three-step phase algorithm, and adding Gaussian noise and random noise to it to generate a training data set; generating the training data set and building a fully connected neural network, wherein each layer of the fully connected neural network uses only one node such that the number of nodes is increased layer by layer and at the end of the network, the number of network nodes is decreased layer by layer; inputting corresponding pixel values into two wrapped phase images one by one to obtain a fringe level of the entire image, and then obtaining the absolute phase, wherein the absolute phase is described by the equation: wherein

[0009] 0(x,y) is the absolute phase,

[0010] 0(x, y) is the wrapped phase k(x,y) is the fringe order.

[0011] From the article by Pengcheng Yao, Shaoyan Gai, Yuchong Chen, Wenlong Chen, Feipeng Da, "A multicode 3D measurement technique based on deep learning", "Optics and Lasers in Engineering", Volume: 143, August 2021, 106623, there is known a multi-code deep learning method designed to obtain the high-slope absolute phase with high accuracy and robustness with usage of two patterns. Said method uses for this purpose two neural sub-networks. A convolutional neural network with adaptive linear (ALCNN) to obtain a wrapped phase from single cosine pattern and a multi-connected convolutional neural network (MCCNN) to obtain a fringe order from a multi-code pattern, wherein the fringe order is required to recover the absolute phase by unwrapping 2n discontinuities. The multicode pattern is expressed as follows: where:

[0012] Cs(x, y) is a coding scheme of the multi-code pattern.

[0013] The MCCNN network has functions of segmentation and recognition. The segmentation function is used to split codes located in different periods and extract coding scheme from the multi-code pattern. Then the fringe order is recognized from the coding scheme by the help of the recognition function and the feature of 3-connected-codes.

[0014] From Chinese patent application CN114777677A there is known a single-frame double-frequency multiplexing fringe projection three-dimensional surface type measurement method based on deep learning. The method comprises the steps of superposing two vertical sine fringes with different spatial frequencies and combining the two vertical sine fringes into a fringe pattern to generate a double-frequency spatial carrier frequency multiplexing fringe coding pattern; building a U-Net model; generating a training data set, including input set and truth value data set, with the usage of the double-frequency spatial carrier frequency multiplexing fringe coding pattern and training the U-Net model with the usage of the training data set.

[0015] From the article by Yixuan Li, Jiaming Qian, Shijie Feng, Qian Chen and Chao Zuo "Deep-learning- enabled dual-frequency composite fringe projection profilometry for single shot absolute 3D shape measurement", "Opto-Electronic Advances" 2022, Vol.5, No.5, DOI: 10.29026 / oeq.2022.210021, there is known a deep-learning-enabled dual-frequency composite fringe projection profilometry for single shot absolute 3D shape measurement, wherein a frame of a spatial object with a dual-frequency composite fringe pattern projected on its surface is captured by a camera, a map of the imaginary component of a signal with a dual-frequency and a map of the real component of a signal with a dualfrequency is generated from said frame. Further, a wrapped phase map with a variable frequency is generated from the map of the imaginary component of a signal with a dual-frequency and the map of the real component of a signal with a dual-frequency and an unwrapped phase map with a dualfrequency is generated from the wrapped phase map with a dual-frequency. Finally a 3D image of the spatial object is reconstructed based on the unwrapped phase map.

[0016] Although methods for dynamic 3D scanning of a spatial object known from the prior art discloses the use of a stripe pattern that allow reconstruction of a scanned spatial object from a single frame there is still the need to increase robustness of such methods and 3D dynamic scanners implementing such methods to the noise present in the input data. Therefore, an object of the invention is to provide a method for dynamic 3D scanning of a spatial object and a dynamic 3D scanner characterized by increased robustness to the noise present in the input data comparing to the solutions known from the prior art.

[0017] The Essence of the invention

[0018] The object of the invention is a method for dynamic 3D scanning of a spatial object comprising the following steps:

[0019] - projecting a stripe pattern onto a spatial object by a projector of a dynamic 3D scanner, wherein the stripe pattern is a variable-frequency pattern formed with the usage of a periodic function, and the stripe pattern is arranged such that

[0020] - the width / height of its stripes corresponds to an oscillation period stof eight pixels or its multiples, where a maximum value of the oscillation period stis limited to a maximum value of the resolution of the projector, wherein i = (0, 1, ...,ri);

[0021] - the stripes form a sequence S of said oscillation periods Sj, wherein at most three of consecutive oscillation periods may have the same value, wherein i = (0, 1, ... , ri);

[0022] - the sum of the oscillation periods forming the sequence S must be equal to or greater than the maximum value of the resolution of the projector, wherein i = (0, 1, ... , n);

[0023] - capturing by a camera of the dynamic 3D scanner a frame of the spatial object with the stripe pattern projected onto its surface, - generating a map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object from the captured frame, wherein said generation is done by a first convolutional network;

[0024] - thresholding the map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object to obtain a shadow mask comprising pixels excluded from the reconstruction of a 3D image of the spatial object,

[0025] - correlating the captured frame with the shadow mask to obtain a frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object,

[0026] - generating a map of the imaginary component of a signal with a variable frequency and a map of the real component of a signal with a variable frequency from the frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from said reconstruction, wherein said generation is done by a second convolutional network;

[0027] - generating a wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency,

[0028] - generating a pixel map with data comprising assigned oscillation period numbers and a map of probability values of assigning oscillation period numbers from the wrapped phase map with a variable frequency, wherein said generation is done by a third convolutional network;

[0029] - filtering out from the pixel map with data comprising assigned oscillation period number areas having on the map of probability values of assigning the oscillation period numbers a value < 0.99,

[0030] - reconstructing the filtered out areas from the pixel map,

[0031] - generating an unwrapped phase map with a variable frequency from the wrapped phase map with a variable frequency and the pixel map with reconstructed filtered out areas,

[0032] - transforming the unwrapped phase map with a variable frequency into an unwrapped phase map with a constant frequency so as to obtain a uniform oscillation period over the entire frequency domain of the unwrapped phase map,

[0033] - reconstructing a 3D image of the spatial object based on the unwrapped phase map with a constant frequency. The application of the disclosed stripe pattern allows to eliminate the need to use a coding frame during dynamic 3D scanning of a spatial object. This is due to the fact that the disclosed stripe pattern allows to comprise all the information needed to a reconstruction of scanned spatial object in a single frame. As a result a 3D image reconstruction is more resistant to errors when a scanned object moves quickly. Furthermore, the application of the disclosed stripe pattern eliminates the problem of delays occurring in acquisition of image sequences due to the way a camera and a projector, synchronized in a dynamic 3D scanner, work together.

[0034] The application of the disclosed steps of a method for dynamic 3D scanning of a spatial object and stripe pattern ensures increase of robustness of said method and a dynamic 3D scanner implementing such method to the noise present in the input data.

[0035] Preferably, the stripe pattern is generated with enhancement of overall stripe pattern brightness by increasing a periodic function constant component IDCof a stripe pattern pixel brightness level / (x) while decreasing a periodic function variable component IACof said level, wherein where:

[0036] I(x) - a stripe pattern pixel brightness level for a pixel coordinate value x in an 8-bit scale (0, 255),

[0037] IDC- a periodic function constant component,

[0038] IAC~aperiodic function variable component, x - a pixel coordinate value of the stripe pattern (4),

[0039] <p(x) - a function transforming a pixel coordinate value x of the stripe pattern to an angular value <p, pw(<p(x)) - a periodic function used to form the stripe patten.

[0040] The generation of the stripe pattern with enhancement of overall stripe pattern brightness by increasing the periodic function constant component IDCof the stripe patten pixel brightness level / (x) while decreasing the periodic function variable component IACof said level results in increased illumination of the captured frame what allows to include in said captured frame more information about a degree of shading and texture of the spatial object. As a consequence increasing the periodic function constant component IDCof the stripe pattern pixel brightness level / (x) while decreasing the periodic function variable component IACof said level facilitates distinguishing areas from the pixel map that are subject to reconstruction and those excluded from said reconstruction. Preferably, the transformation of the unwrapped phase map with a variable frequency into the unwrapped phase map with a constant frequency is done with the usage of the formula: where: and where:

[0041] 4>var ~anabsolute phase value associated with the unwrapped phase map with a variable frequency,

[0042] ^onst ~anabsolute phase value associated with the unwrapped phase map with a constant frequency, w(<j)var) - an auxiliary function that allows to extract from the absolute phase value associated with the unwrapped phase map with a variable frequency the relative phase value in a range (0,2TT), i - a number of an oscillation period st, wherein the number corresponds to an absolute phase value associated with the unwrapped phase map with a variable frequency <bvar,

[0043] Si - a value of an oscillation period of the sequence S formed by the stripes of the stripe pattern, wherein i = (0, 1, ... , n); f(i) - a relative phase value multiplier for period number i, m(i) - an oscillation period number modifier for period number i,

[0044] / (k) - a relative phase value multiplier for period number k.

[0045] Preferably, when reconstructing of the 3D image of the spatial object a projector coordinate value x(4>const) °f each reconstructed pixel of the frame captured by the camera is determined from the unwrapped phase map with a constant frequency according to formula where:

[0046] ^ccmst ~anabsolute phase value associated with the unwrapped phase map with a constant frequency, p - an oscillation period of the unwrapped phase map with a constant frequency expressed in pixels, in order to obtain a map of pixel coordinates, wherein the 3D coordinates of the spatial object are determined by converting the unwrapped phase map with a constant frequency into the map of projector pixel coordinates and taking into consideration relative location of the camera and the projector.

[0047] Preferably, after the step of obtaining the frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object and before the step of generating the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency from the frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from said reconstruction, said frame is divided into square fragments, and the generation of the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency is performed by generating for each of said square fragments of said frame square fragments of a map of the imaginary component of a signal with a variable frequency and square fragments of a map of the real component of a signal with a variable frequency, cutting out overlapping edges of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency, assembling of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency into, respectively, the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency.

[0048] The application of the steps of division of the frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object into square fragments, the generation of the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency performed by generating for each of said square fragments of said frame square fragments of a map of the imaginary component of a signal with a variable frequency and square fragments of a map of the real component of a signal with a variable frequency, cutting out overlapping edges of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency, assembling of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency into, respectively, the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency allows to obtain a higher maximum resulting resolution with the usage of the same computing memory resource as in the solutions known from the prior art.

[0049] Preferably, after the step of generating the wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency, and before the step of generating the pixel map with data comprising assigned period numbers and the map of probability values of assigning period numbers from the wrapped phase map with a variable frequency the spatial object in the wrapped phase map is divided into longitudinal fragments, and overlapping edges of the longitudinal fragments of the pixel map with data comprising assigned period numbers are cut out, overlapping edges of the longitudinal fragments of the map of probability values of assigning period numbers are cut out, the longitudinal fragments of the pixel map with data comprising assigned period numbers with the cut out overlapping edges are assembled into the pixel map with data comprising assigned period numbers, and the longitudinal fragments of the map of probability values of assigning period numbers with the cut out overlapping edges are assembled into the map of probability values of assigning period numbers.

[0050] The application of the steps of division of the spatial object in the wrapped phase map, cutting out overlapping edges of the pixel map with data comprising assigned period numbers and cutting out overlapping edges of the map of probability values of assigning period numbers obtained in a form of the longitudinal fragments, and assembling the pixel map with data comprising assigned period numbers from the longitudinal fragments with the cut out overlapping edges as well as assembling the map of probability values of assigning period numbers from the longitudinal fragments with the cut out overlapping edges allows to obtain even higher maximum resulting resolution, even up to 10 times, with the usage of the same computing memory resource as in the solutions known from the prior art. The application of the stripe pattern which is a variable-frequency pattern formed with the usage of the periodic function based on the arcsine function allows to reconstruct a 3D image of a spatial object with a higher level of accuracy compared to solutions known from the state of the art.

[0051] Preferably, the step of filtering out from the pixel map with data comprising assigned period number areas having on the map of probability values of assigning the period numbers is performed for a value <- 0.95.

[0052] Preferably, the step of filtering out from the pixel map with data comprising assigned period number areas having on the map of probability values of assigning the period numbers is performed for a value <- 0.90.

[0053] Preferably, the stripe pattern is a variable-frequency pattern formed with the usage of a periodic function based on the arcsine function where: where:

[0054] | . | - an absolute value operator, where: st- an element of an auxiliary sequence S of accumulated oscillation periods, wherein the auxiliary sequence S determines ranges of pixel coordinate values of the stripe pattern for each oscillation period in the sequence S such that Sj is an oscillation period value in a pixel range (Si, si+1), wherein i = (0, 1, ... , n);

[0055] <p(x, i) - a conditional function that allows to transform a pixel coordinate value x of the stripe pattern (4) to an angular value <p in a range (0,2TT) for an oscillation period stif the pixel coordinate value x of the stripe pattern fits in a determined range of pixel coordinate values of the oscillation period st, wherein i = (0, 1, ...,n).

[0056] The application of the stripe pattern which is a variable-frequency pattern formed with the usage of the periodic function based on the arcsine function allows to reconstruct a 3D image of a spatial object with a higher level of accuracy compared to solutions known from the state of the art.

[0057] The object of the invention is also a dynamic 3D scanner comprising a projector for projecting a stripe pattern onto a spatial object, a camera for capturing a frame of the spatial object with the stripe pattern on its surface, wherein the camera is pointed at non-zero angle to a display direction of the projector, the dynamic 3D scanner further comprising a processing unit, and wherein the processing unit is configured to performed the steps of the method according to the invention.

[0058] The invention also relates to a computer program comprising means of the program code for performing all steps of the method according to the invention, when said program is running on the dynamic 3D scanner according to the invention.

[0059] Furthermore, the invention relates to a computer-readable medium storing computer-implemented instructions for performing all the steps of the method according to the invention implemented on the dynamic 3D scanner according to the invention.

[0060] Advantages of the invention

[0061] The method according to the invention allows a reduction of manufacturing costs of a dynamic 3D scanner and use of hardware components such as a projector and camera with standard parameters, in particular the method according to the invention allows use of home projectors operating via HDMI interface. For example, the method according to the invention ensures use of Digital Light Processing projectors.

[0062] The method according to the invention ensures shortening of time of projecting a stripe pattern onto a spatial object subject to 3D reconstruction and as a result the method ensures shortening of time of capturing of a frame comprising the spatial object with the stripe pattern on its surface as in said method only one stripe pattern is projected. Said shortening is possible even when using home projectors operating via HDMI interface.

[0063] The method according to the invention ensures improvement of scanning by using convolutional neural networks.

[0064] The method according to the invention ensures increased robustness of a dynamic 3D scanner to the noise present in the input data, and as a result reconstruction of geometry of a spatial object with a higher accuracy.

[0065] The method according to the invention allows for displaying a static pattern, thus eliminating the problem of occurring delays in acquisition of image sequences.

[0066] The method according to the invention allows 3D scanning of a spatial object being in motion.

[0067] The method according to the invention eliminates the need to use a coding frame during dynamic 3D scanning of a spatial object.

[0068] The method according to the invention eliminates the problem of delays occurring in acquisition of image sequences due to the way a camera and a projector, synchronized in a dynamic 3D scanner, work together.

[0069] Description of the drawings

[0070] The subject of the invention is shown in the embodiments in a drawing, in which:

[0071] Fig. 1 presents a flowchart of a method for dynamic 3D scanning of a spatial object according to the invention,

[0072] Fig. 2 presents an operational scheme of a dynamic 3D scanner according to the invention,

[0073] Fig. 3 presents schematically a dynamic 3D scanner according to the first embodiment of the invention,

[0074] Fig. 4 presents an example of a stripe pattern for use in a method for dynamic 3D scanning of a spatial object according to the invention,

[0075] Fig. 5 presents an example of a captured frame of a spatial object with a stripe pattern on its surface,

[0076] Fig. 6 presents an example of a map of the probability of qualifying pixels for a reconstruction of a 3D image of a spatial object, Fig. 7 presents an example of a shadow mask comprising pixels excluded from a reconstruction of a 3D image of a spatial object,

[0077] Fig. 8 presents an example of a frame comprising the spatial object with the stripe pattern onto its surface with blanked out pixels excluded from a reconstruction of a 3D image of a spatial object,

[0078] Fig. 9A presents an example of a map of the imaginary component of a signal with a variable frequency,

[0079] Fig. 9B presents an example of a map of the real component of a signal with a variable frequency,

[0080] Fig. 10 presents an example of a wrapped phase map with a variable frequency,

[0081] Fig. 11A presents an example of a pixel map with data comprising assigned period numbers,

[0082] Fig. 11B presents an example of a map of probability values of assigning period numbers,

[0083] Fig. 12 presents an example of an unwrapped phase map with a variable frequency,

[0084] Fig. 13 presents an example of an unwrapped phase map with a constant frequency,

[0085] Fig. 14 presents an example of a reconstructed 3D image of a spatial object,

[0086] Fig. 15A presents an example of a captured frame with indicated overlapping edges,

[0087] Fig. 15B presents an example of a captured frame divided into square fragments,

[0088] Fig. 15C presents an example of square fragments of a map of the imaginary component of a signal with a variable frequency with cut out overlapping edges,

[0089] Fig. 15D presents an example of an assembled map of the imaginary component of a signal with a variable frequency,

[0090] Fig. 16A presents an example of square fragments of a map of the real component of a signal with a variable frequency with cut out overlapping edges,

[0091] Fig. 16B presents an example of an assembled a map of the real component of a signal with a variable frequency,

[0092] Fig. 17A presents an example of a spatial object in a wrapped phase map with indicated overlapping edges,

[0093] Fig. 17B presents an example of a spatial object in a wrapped phase map divided into longitudinal fragments, Fig. 17C presents an example of a pixel map with data comprising assigned period numbers with cut out overlapping edges,

[0094] Fig. 17D presents an example of an assembled a pixel map with data comprising assigned period numbers.

[0095] Detailed description of the invention

[0096] The first embodiment of the invention

[0097] A method for dynamic 3D scanning of a spatial object

[0098] In a method for dynamic 3D scanning of a spatial object according to the first embodiment of the invention a stripe pattern 4 is projected 101 onto a spatial object 5, for example a hand, by a projector 2 of a dynamic 3D scanner 1 and a frame of the spatial object 5 with the stripe pattern 4 onto its surface is captured 102 by a camera 3 of the dynamic 3D scanner 1. Said step of projecting 101 is performed using structured light. In the other embodiments said step of projecting 101 can be performed using halogen lighting, infrared light lasers, red lasers, blue lasers or LEDs.

[0099] The stripe pattern 4 is a variable-frequency pattern formed with the usage of a periodic function and the stripe pattern 4 is arranged such that

[0100] - the width / height of its stripes corresponds to an oscillation period of eight pixels or its multiples, where a maximum value of the oscillation period S[ is limited to a maximum value of the resolution of the projector 2, wherein i = (0, 1, ... , n);

[0101] - the stripes form a sequence S of said oscillation periods Sj, wherein at most three of consecutive oscillation periods stmay have the same value, wherein i = (0, 1, ... , ri);

[0102] - the sum of the oscillation periods stforming the sequence S must be equal to or greater than the maximum value of the resolution of the projector 2.

[0103] The stripe pattern 4 is generated 122 with enhancement of overall stripe pattern brightness by increasing a periodic function constant component IDCof a stripe patten pixel brightness level / (%) while decreasing a periodic function variable component IACof said level, wherein where: I(x) - a stripe pattern pixel brightness level for a pixel coordinate value x in a 8-bit scale (0, 255),

[0104] IDC~ a periodic function constant component,

[0105] IAC~aperiodic function variable component, x - a pixel coordinate value of the stripe pattern (4),

[0106] <p(x) - a function transforming a pixel coordinate value x of the stripe pattern (4) to an angular value (p in a range (0,27r), pw(<p(x)) - a periodic function used to form the stripe patten (4).

[0107] The generation of the stripe pattern 4 with enhancement of overall stripe pattern brightness by increasing the periodic function constant component IDCof the stripe patten pixel brightness level / (x) while decreasing the periodic function variable component IACof said level results in increased illumination of the captured frame what allows to include in said captured frame more information about a degree of shading and texture of the spatial object 5. As a consequence increasing the periodic function constant component IDCof the stripe pattern pixel brightness level / (x) while decreasing the periodic function variable component IACof said level facilitates distinguishing areas from a further generated pixel map that are subject to reconstruction and those excluded from said reconstruction.

[0108] The periodic function is based on the arcsine function. where: - a linear zigzag function defined in a range (0,2TT), where:

[0109] | . | - an absolute value operator, where:

[0110] §i - an element of an auxiliary sequence S of accumulated oscillation periods, wherein the auxiliary sequence S determines ranges of pixel coordinate values of the stripe pattern 4 for each oscillation period stin the sequence S such that sLis an oscillation period value in a pixel range (st, si+1), wherein i = (0, 1, ... , n);

[0111] <p(x, i) - a conditional function that allows to transform a pixel coordinate value x of the stripe pattern (4) to an angular value <p in a range (0,2TT) for an oscillation period sLif the pixel coordinate value x of the stripe pattern 4 fits in a determined range of pixel coordinate values of the oscillation period sz, wherein i = (0, 1, ... , ri).

[0112] An exemplary stripe pattern (Fig. 4) formed with the usage of the periodic function based on the arcsine function is arranged such that

[0113] - the width / height of its stripes corresponds to an oscillation period stof eight, sixteen and twenty four pixels, wherein i = (0, 1, ... , n);

[0114] - the stripes form a sequence S of said oscillation periods sizwherein i = (0, 1, ...,ri) and where

[0115] S = {Si}, i e (0, 63)

[0116] S = {24, 24, 8, 8, 8, 24, 24, 16, 8, 24, 8, 8, 24, 24, 16, 8, 8, 24, 24, 8, 8, 24, 24, 16, 24, 24, 8, 8, 16, 24, 24, 8, 24, 24, 8, 8, 8, 24, 24, 8, 8, 24, 24, 16, 24, 24, 8, 8, 24, 24, 8, 24, 24, 8, 24, 24, 16, 16, 24, 24, 8, 8, 8, 24},

[0117] - the sum of the oscillation periods szforming the sequence S is equal to the maximum value of the resolution of the projector 2, thus 1920 x 1080, wherein i = (0, 1, ... , n).

[0118] Furthermore,

[0119] S = {§;}, i e (0, 64)

[0120] S = {0, 24, 48, 56, 64, 72, 96, 120, 136, 144, 168, 176, 184, 208, 232, 248, 256, 264, 288, 312, 320, 328, 352, 376, 392, 416, 440, 448, 456, 472, 496, 520, 528, 552, 576, 584, 592, 600, 624, 648, 656, 664, 688, 712, 728, 752, 776, 784, 792, 816, 840, 848, 872, 896, 904, 928, 952, 968, 984, 1008, 1032, 1040, 1048, 1056, 1080}.

[0121] Thus, for example the first period s0of the sequence S of said oscillation periods applies for the first 24 pixels, that is, from the pixel number 0 to the pixel number 23 inclusive.

[0122] The application of the disclosed stripe pattern 4 allows to eliminate the need to use a coding frame during dynamic 3D scanning of a spatial object. This is due to the fact that the disclosed stripe pattern 4 allows to comprise all the information needed to a reconstruction of the scanned spatial object 5 in a single frame. As a result a 3D image reconstruction is more resistant to errors when the scanned object 5 moves quickly. Furthermore, the application of the disclosed stripe pattern 4 eliminates the problem of delays occurring in acquisition of image sequences due to the way the camera 3 and the projector 2, synchronized in the dynamic 3D scanner 1, work together.

[0123] Furthermore, the application of the stripe pattern 4 which is a variable-frequency pattern formed with the usage of the periodic function based on the arcsine function allows to reconstruct a 3D image of a spatial object 5 with a higher level of accuracy compared to solutions known from the state of the art.

[0124] In the next step a map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object 5 is generated 103 from the captured frame. Said generation is based on binary classification, wherein pixels excluded from the reconstruction are assigned to a negative class and pixels to be reconstructed are assigned to a positive class. Said binary classification is known from the prior art. Said generation is done by a first convolutional neural network. Input data of the first convolutional neural network constitutes the captured frame of the spatial object 5 with the stripe pattern 4 on its surface. Output data of the first convolutional neural network constitutes the map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object 5.

[0125] After said step the map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object 5 is thresholded 104 to obtain 105 a shadow mask comprising pixels excluded from the reconstruction of a 3D image of the spatial object 5. For example a brightness threshold is 50%. In the following step the captured frame is correlated 106 with the shadow mask to obtain 107 a frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object 5.

[0126] Further, a map of the imaginary component of a signal with a variable frequency (Fig. 9A) and a map of the real component of a signal with a variable frequency (Fig. 9B) are generated 109 from the frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from said reconstruction. Said generation is done by a second convolutional neural network. Input data of the second convolutional neural network constitutes the frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object 5. Output data of the second convolutional neural network constitute the map of the imaginary component of a signal with a variable frequency and a map of the real component of a signal with a variable frequency.

[0127] Next, a wrapped phase map with a variable frequency is generated 110 from the map of the imaginary component of a signal with a variable frequency and a map of the real component of a signal with a variable frequency.

[0128] Since in the disclosed embodiment the stripe pattern 4 is formed with the usage of a periodic function based on the arcsine function, the map of the imaginary component of a signal with a variable frequency is a map of a signal with a variable frequency associated with the arcsine function and the map of the real component of a signal with a variable frequency is a map of a signal with a variable frequency associated with the arccosine function.

[0129] As a result, for the purposes of this embodiment, the wrapped phase map with a variable frequency is generated 110 from the map of a signal with a variable frequency associated with the arcsine function and the map of a signal with a variable frequency associated with the arccosine function. The generation 110 of the wrapped phase map with a variable frequency is done according to formula where: and where:

[0130] <Pvar ~arelative phase value of each pixel forming the wrapped phase map with a variable frequency,

[0131] Vsinw~avalue of the map of a signal with a variable frequency associated with the arcsine function,

[0132] Vcosw- avalue of the map of a signal with a variable frequency associated with the arccosine function, atanw2(x,y) - a function that allows to determine relative phase values forming the wrapped phase map with a variable frequency based on values of the map of a signal with a variable frequency associated with the arcsine function and the map of a signal with a variable frequency associated with the arccosine function, atans2(y', x'~) - an intermediate function that allows to restore relative phase values in range (— n, n) from zigzac function components determined by converting signal values of periodic functions based on the arcsine function (y) and arccosine function (x) to y' and x' accordingly.

[0133] Afterthe step of generation 110 of the wrapped phase map with a variable frequency a pixel map with data comprising assigned oscillation period numbers and a map of probability values of assigning oscillation period numbers is generated 116 from the wrapped phase map with a variable frequency.

[0134] The generation 116 of the pixel map with data comprising assigned period numbers and the map of probability values of assigning period numbers is done by a third convolutional neural network. Input data of the third convolutional neural network constitutes the wrapped phase map with a variable frequency. Output data of the third convolutional neural network constitute the pixel map with data comprising assigned period numbers and the map of probability values of assigning period numbers.

[0135] In the next step areas having on the map of probability values of assigning the oscillation period numbers a value < 0.99 are filtered out 117 from the pixel map with data comprising assigned oscillation period number. Preferably, said value is <- 0.95 and more preferably said value is <- 0.90. The most preferably, said value is equal or greater than 0.5 and less than 1.0. The filtered out areas from the pixel map are reconstructed 118 and an unwrapped phase map with a variable frequency is generated 119 from the wrapped phase map with a variable frequency and the pixel map with reconstructed 118 filtered out areas.

[0136] Then the unwrapped phase map with a variable frequency is transformed 120 into an unwrapped phase map with a constant frequency so as to obtain a uniform oscillation period over the entire frequency domain of the unwrapped phase map.

[0137] The transformation 120 of the unwrapped phase map with a variable frequency into the unwrapped phase map with a constant frequency is done with the usage of the formula: where: and where

[0138] (j)var- anabsolute phase value associated with the unwrapped phase map with a variable frequency,

[0139] 4>COnSt- anabsolute phase value associated with the unwrapped phase map with a constant frequency, w(4jvar) - an auxiliary function that allows to extract from the absolute phase value associated with the unwrapped phase map with a variable frequency the relative phase value in a range (0,2TT), i - a number of an oscillation period st, wherein the number corresponds to an absolute phase value associated with the unwrapped phase map with a variable frequency <bvar,

[0140] Si - a value of an oscillation period of the sequence S formed by the strips of the stripe pattern 4, wherein i = (0, 1, ... , ri);

[0141] / (i) - a relative phase value multiplier for period number i, m(i) - an oscillation period number modifier for period number i, f(k) - a relative phase value multiplier for period number k.

[0142] Table 1 presents an exemplary set of values of oscillation period sitmultipliers / (i) and modifiers m(i) used in the first embodiment of the invention.

[0143] Finally, a 3D image of the spatial object 5 is reconstructed 120 based on the unwrapped phase map with a constant frequency. During said reconstruction 120 of the 3D image of the spatial object 5 a projector coordinate value x(4>Const) °f each reconstructed pixel of the frame captured 102 by the camera 3 is determined 121A from the unwrapped phase map with a constant frequency according to formula where: 4>e(0, co)

[0144] <j)ccmst ~anabsolute phase value associated with the unwrapped phase map with a constant frequency, p - an oscillation period of the unwrapped phase map with a constant frequency expressed in pixels, in order to obtain 121B a map of pixel coordinates. The 3D coordinates of the spatial object 5 are determined 121C by converting the unwrapped phase map with a constant frequency into the map of projector pixel coordinates and taking into consideration relative location of the camera 3 and the projector 2.

[0145] A dynamic 3D scanner

[0146] A dynamic 3D scanner 1 according to the first embodiment comprises a projector 2 for projecting 101 a stripe pattern 4 onto a spatial object 5. The projector 2 can be a home projector with a HDMI interface with a full HD resolution. The projector 2 uses structured light to project 101 the stripe pattern 4. In the other embodiments the projector 2 can project 101 said pattern 4 using halogen lighting, infrared light lasers, red lasers, blue lasers or LEDs. Further, the dynamic 3D scanner 1 comprises a camera 3 for capturing 102 a frame of the spatial object 5 with the stipe pattern 4 on its surface. The camera 3 is equipped with lens 6 and a spectral filter 7, wherein said filter 7 is designed to allow light of a specific wavelength to be captured. Said camera 3 is pointed at a non-zero angle to a display direction of the projector 2. The projector 2 and the camera 3 are fixed to a support structure 8, wherein said support structure 8 allows adjustment of mutual position of the projector 2 and camera 3 and as a consequence ensures that the proper angles between them are maintained. Further, the projector 2 is connected with a stand 9. Further, the dynamic 3D scanner 1 comprises a processing unit and said processing unit is configured to perform the steps of the method according to the invention. The processing unit is an integral component of the dynamic 3D scanner. The dynamic 3D scanner 1 is designed to be powered by an external power supply. In the other embodiments the processing unit can constitute an external device, for example a desktop computer. In such cases the dynamic 3D scanner 1 is designed to be connected to an external processing unit.

[0147] The dynamic 3D scanner disclosed in the indicated embodiment provides a disclosure that is also inherent in the other embodiments. A computer program

[0148] A computer program comprising means of the program code for performing all steps of the method according to the invention, when said program is running on the dynamic 3D scanner according to the invention.

[0149] The computer program disclosed in the indicated embodiment provides a disclosure that is also inherent in the other embodiments.

[0150] A computer-readable medium

[0151] A computer-readable medium storing computer-implemented instructions for performing all the steps of the method according to the invention implemented on the dynamic 3D scanner according to the invention.

[0152] The computer-readable medium disclosed in the indicated embodiment provides a disclosure that is also inherent in the other embodiments.

[0153] The second embodiment of the invention

[0154] A method for dynamic 3D scanning of a spatial object

[0155] The second embodiment of the method for dynamic 3D scanning of a spatial object as in the first embodiment, with the difference that after the step of obtaining 107 the frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object 5 and before the step of generating 109 the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency from the frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from said reconstruction, said frame is divided 108 into square fragments, for example having a resolution of 256 pixels x 256 pixels, and the generation 109 of the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency is performed generating 109Afor each of said square fragments of said frame square fragments of a map of the imaginary component of a signal with a variable frequency and square fragments of a map of the real component of a signal with a variable frequency, cutting out 109B overlapping edges of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency, assembling 109C of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency into, respectively, the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency. Since this embodiment like the first embodiment uses a periodic function based on the arcsine function, the map of the imaginary component of a signal with a variable frequency is a map of a signal with a variable frequency associated with the arcsine function and the map of the real component of a signal with a variable frequency is a map of a signal with a variable frequency associated with the arccosine function.

[0156] The application of the steps of division 108 of the frame comprising the spatial object 5 with the stripe pattern 4 onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object into square fragments, the generation 109 of the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency performed by generating 109A for each of said square fragments of said frame square fragments of a map of the imaginary component of a signal with a variable frequency and square fragments of a map of the real component of a signal with a variable frequency, cutting out 109B overlapping edges of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency, assembling 109C of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency into, respectively, the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency allows to obtain a higher maximum resulting resolution with the usage of the same computing memory resource as in the solutions known from the prior art.

[0157] Furthermore, after the step of generating 110 the wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency, and before the step of generating 116 the pixel map with data comprising assigned period numbers and the map of probability values of assigning period numbers from the wrapped phase map with a variable frequency the spatial object in the wrapped phase map is divided 111 into longitudinal fragments, for example having a longitudinal resolution of 128 pixels. Next, overlapping edges of the longitudinal fragments of the pixel map with data comprising assigned period numbers are cut out 112, overlapping edges of the longitudinal fragments of the map of probability values of assigning period numbers are cut out 113, the longitudinal fragments of the pixel map with data comprising assigned period numbers with the cut out overlapping edges are assembled 114 into the pixel map with data comprising assigned period numbers, and the longitudinal fragments of the map of probability values of assigning period numbers with the cut out overlapping edges are assembled 115 into the map of probability values of assigning period numbers.

[0158] The application of the steps of division 111 of the wrapped phase map into longitudinal fragments, cutting out 112 overlapping edges of the longitudinal fragments of the pixel map with data comprising assigned period numbers, cutting out 113 overlapping edges of the longitudinal fragments of the map of probability values of assigning period numbers, and assembling 114 the pixel map with data comprising assigned period numbers from the longitudinal fragments of the pixel map with data comprising assigned period numbers with the cut out overlapping edges as well as assembling 115 the map of probability values of assigning period numbers from the longitudinal fragments of the map of probability values of assigning period numbers with the cut out overlapping edges allows to obtain even higher maximum resulting resolution, even up to 10 times, with the usage of the same computing memory resource as in the solutions known from the prior art.

Claims

CLAIMS1. A method for dynamic 3D scanning of a spatial object comprising the following steps:- projecting (101) a stripe pattern (4) onto a spatial object (5) by a projector (2) of a dynamic 3D scanner (1), wherein the stripe pattern (4) is a variable-frequency pattern formed with the usage of a periodic function, and the stripe pattern (4) is arranged such that- the width / height of its stripes corresponds to an oscillation period Sj of eight pixels or its multiples, where a maximum value of the oscillation periodis limited to a maximum value of the resolution of the projector (2), wherein i = (0, 1, ... , n);- the stripes form a sequence S of said oscillation periods Sj, wherein at most three of consecutive oscillation periods stmay have the same value, wherein i = (0, 1, ... , n);- the sum of the oscillation periods Si forming the sequence S must be equal to or greater than the maximum value of the resolution of the projector (2), wherein i = (0, 1, ... , n);- capturing (102) by a camera (3) of the dynamic 3D scanner (1) a frame of the spatial object (5) with the stripe pattern (4) onto its surface,- generating (103) a map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object (5) from the captured frame, wherein said generation is done by a first convolutional network;- thresholding (104) the map of the probability of qualifying pixels for a reconstruction of a 3D image of the spatial object (5) to obtain (105) a shadow mask comprising pixels excluded from the reconstruction of a 3D image of the spatial object (5),- correlating (106) the captured frame with the shadow mask to obtain (107) a frame comprising the spatial object (5) with the stripe pattern (4) onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object (5),- generating (109) a map of the imaginary component of a signal with a variable frequency and a map of the real component of a signal with a variable frequency from the frame comprising the spatial object (5) with the stripe pattern (4) onto its surface with blanked out pixels excluded from said reconstruction, wherein said generation is done by a second convolutional network;- generating (110) a wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency,- generating (116) a pixel map with data comprising assigned oscillation period numbers and a map of probability values of assigning oscillation period numbers from the wrapped phase map with a variable frequency, wherein said generation is done by a third convolutional network;- filtering out (117) from the pixel map with data comprising assigned oscillation period number areas having on the map of probability values of assigning the oscillation period numbers a value < 0.99,- reconstructing (118) the filtered out areas from the pixel map,- generating (119) an unwrapped phase map with a variable frequency from the wrapped phase map with a variable frequency and the pixel map with reconstructed (118) filtered out areas,- transforming (120) the unwrapped phase map with a variable frequency into an unwrapped phase map with a constant frequency so as to obtain a uniform oscillation period over the entire frequency domain of the unwrapped phase map,- reconstructing (121) a 3D image of the spatial object based on the unwrapped phase map with a constant frequency.

2. The method according to claim 1, wherein the stripe pattern (4) is generated (122) with enhancement of overall stripe pattern brightness by increasing a periodic function constant component IDCof a stripe patten pixel brightness level I(x) while decreasing a periodic function variable component IACof said level, whereinKx) = IDC+ IAC * pwtvto) where:Z(x) - a stripe pattern pixel brightness level for a pixel coordinate value x in a 8-bit scale (0, 255),IDC~ a periodic function constant component,IAC~ a periodic function variable component, x - a pixel coordinate value of the stripe pattern (4),<p(x) - a function transforming a pixel coordinate value x of the stripe pattern (4) to an angular value <p in a range (0,2TT),- a periodic function used to form the stripe patten (4).

3. The method according to any of the preceding claims, wherein the transformation (120) of the unwrapped phase map with a variable frequency into the unwrapped phase map with a constant frequency is done with the usage of the formula: where: and where:4>var- anabsolute phase value associated with the unwrapped phase map with a variable frequency,^const- a nabsolute phase value associated with the unwrapped phase map with a constant frequency, w(<j>var) - an auxiliary function that allows to extract from the absolute phase value associated with the unwrapped phase map with a variable frequency the relative phase value in a range (0,2TT),i -a number of an oscillation period st, wherein the number corresponds to an absolute phase value associated with the unwrapped phase map with a variable frequency (j)var, st- a value of an oscillation period of the sequence S formed by the stripes of the stripe pattern (4), wherein i = (0, 1, ... , n); / (i) - a relative phase value multiplier for period number i, m(i) - an oscillation period number modifier for period number i, f(Ic) - a relative phase value multiplier for period number k.

4. The method according to any of the preceding claims, wherein when reconstructing (121) of the 3D image of the spatial object (5) a projector coordinate value x(4>const) of each reconstructed pixel of the frame captured (102) by the camera (3) is determined (121A) from the unwrapped phase map with a constant frequency according to formula where:4>const ~anabsolute phase value associated with the unwrapped phase map with a constant frequency, p - an oscillation period of the unwrapped phase map with a constant frequency expressed in pixels, in order to obtain (121B) a map of projector pixel coordinates, wherein the 3D coordinates of the spatial object (5) are determined (121C) by converting the unwrapped phase map with a constant frequency into the map of projector pixel coordinates and taking into consideration relative location of the camera (3) and the projector (2).

5. The method according to any of the preceding claims, wherein afterthe step of obtaining (107) the frame comprising the spatial object (5) with the stripe pattern (4) onto its surface with blanked out pixels excluded from the reconstruction of a 3D image of the spatial object (5)and before the step of generating (109) the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency from the frame comprising the spatial object (5) with the stripe pattern (4) onto its surfacewith blanked out pixels excluded from said reconstruction, said frame is divided (108) into square fragments, and the generation (109) of the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency is performed by- generating (109A) for each of said square fragments of said frame square fragments of a map of the imaginary component of a signal with a variable frequency and square fragments of a map of the real component of a signal with a variable frequency,- cutting out (109B) overlapping edges of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency,- assembling (109C) of the square fragments of the map of the imaginary component of a signal with a variable frequency and the square fragments of the map of the real component of a signal with a variable frequency into, respectively, the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency.

6. The method according to any of the preceding claims, wherein after the step of generating (110) the wrapped phase map with a variable frequency from the map of the imaginary component of a signal with a variable frequency and the map of the real component of a signal with a variable frequency, and before the step of generating (116) the pixel map with data comprising assigned period numbers and the map of probability values of assigning period numbers from the wrapped phase map with a variable frequency the spatial object in the wrapped phase map is divided (111) into longitudinal fragments, and- overlapping edges of the longitudinal fragments of the pixel map with data comprising assigned period numbers are cut out (112),- overlapping edges of the longitudinal fragments of the map of probability values of assigning period numbers are cut out (113), the longitudinal fragments of the pixel map with data comprising assigned period numbers with the cut out overlapping edges are assembled (114) into the pixel map with data comprising assigned period numbers, and- the longitudinal fragments of the map of probability values of assigning period numbers with the cut out overlapping edges are assembled (115) into the map of probability values of assigning period numbers.

7. The method according to any of the preceding claims, wherein the step of filtering out (117) from the pixel map with data comprising assigned period number areas having on the map of probability values of assigning the period numbers is performed for a value <= 0.95.

8. The method according to any of the preceding claims from 1 to 7, wherein the step of filtering out (117) from the pixel map with data comprising assigned period number areas having on the map of probability values of assigning the period numbers is performed for a value <= 0.90.

9. The method according to any of the preceding claims, wherein the stripe pattern (4) is a variable-frequency pattern formed with the usage of a periodic function based on the arcsine functionwhere: z(<p(x)) - a linear zigzag function defined in a range (0,2TT),where:| . | - an absolute value operator,where:Sj - an element of an auxiliary sequence S of accumulated oscillation periods, wherein the auxiliary sequence S determines ranges of pixel coordinate values of the stripe pattern (4) for each oscillation period S; in the sequence S such that S; is an oscillation period value in a pixel range (sj, Si+1), wherein i = (0, 1, ...,n);<p(x, i) - a conditional function that allows to transform a pixel coordinate value x of the stripe pattern (4) to an angular value (p in a range (0,2TT) for an oscillation period stif the pixel coordinate value x of the stripe pattern (4) fits in a determined range of pixel coordinate values of the oscillation period Si, wherein i = (0, 1, ... , n).

10. A dynamic 3D scanner (1) comprising:- a projector (2) for projecting (101) a stripe pattern (4) onto a spatial object (5),- a camera (3) for capturing (102) a frame of the spatial object (5) with the stripe pattern (4) on its surface, wherein the camera (3) is pointed at non-zero angle to a display direction of the projector (2),- the dynamic 3D scanner further comprising a processing unit, characterized in that- the processing unit is configured to performed the steps of the method according to any of the preceding claims 1-9.

11. A computer program comprising means of the program code for performing all steps of the method according to any of claims 1-9, when said program is running on the dynamic 3D scanner (1) according to claim 10.

12. A computer-readable medium storing computer-implemented instructions for performing all the steps of the method according to any of claims 1-9 implemented on the dynamic 3D scanner (1) according to claim 10.

Citation Information

Patent Citations

  • Structured light online dense three-dimensional reconstruction method based on machine learning

    CN111383234A

  • Phase unwrapping method based on deep learning

    CN111523618A

  • Fringe projection time phase unwrapping method based on deep learning

    WO2020063013A1

  • Phase unwrapping using segmentation

    WO2020102814A1

  • Single-frame double-frequency multiplexing fringe projection three-dimensional surface type measurement method based on deep learning

    CN114777677A