Depth measuring device, method and computer readable storage medium

Abstract

The application provides a depth measurement device, method and computer readable storage medium. The depth measurement device comprises: a projection module configured to project at least three frames of phase shift stripes and at least one floodlight beam to a target object; a receiving module comprising a TOF sensor configured to collect at least three frames of phase shift stripes and the floodlight beam reflected by the target object in different working modes; and a processing module configured to generate at least three frames of phase shift stripe images and at least one frame of depth image according to the phase shift stripes and the floodlight beam received by the receiving module, calculate the relative phase of each pixel in the TOF sensor by using the at least three frames of phase shift stripe images, obtain the first depth value of each pixel in the at least one frame of depth image, demodulate the relative phase of the corresponding pixel to obtain the absolute phase of each pixel, and calculate the target depth value of each pixel according to the absolute phase of each pixel. The application can improve the measurement speed while ensuring the depth measurement accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of depth measurement, and particularly relates to a depth measurement device, a depth measurement method and a computer readable storage medium. BACKGROUND

[0002] At present, industrial 3D cameras have been widely applied in many fields. Since industrial 3D measurement generally has a higher precision requirement, the current 3D measurement methods capable of meeting high precision mainly include phase shift structured light technology and 3D laser scanning technology. Since the phase shift structured light technology has the characteristics of fast measurement speed, no mechanical movement or scanning, etc., it is widely applied in industrial 3D measurement.

[0003] When the existing industrial 3D camera uses the phase shift structured light technology to measure the distance, in order to achieve higher precision, multiple phase shift stripes are often used to realize phase demodulation. However, this method can only obtain the relative phase, and the relative phase is discontinuous, so it is still necessary to unwrap the relative phase to obtain the continuous absolute phase. In order to solve the "phase wrapping" phenomenon in phase shift demodulation, the existing technologies such as multi-frequency, Fourier transform and fused Gray code are usually used. Since the multi-frequency and fused Gray code technologies need to increase the number of exposures, the measurement speed of the industrial 3D camera will be reduced. Although the Fourier transform can improve the measurement speed, the scene applicability is limited, and it cannot meet the high-precision measurement of non-continuous planes. SUMMARY

[0004] The present application provides a depth measurement device, a depth measurement method and a computer readable storage medium, which can improve the measurement speed on the basis of ensuring the measurement precision.

[0005] In a first aspect, the present application provides a depth measurement device, comprising: a projection module, configured to project at least three frames of phase shift stripes and at least one floodlight beam to a target object; a receiving module, comprising a TOF sensor, configured to respectively collect the phase shift stripes reflected by the target object in a 2D working mode and collect the floodlight beam reflected by the target object in a 3D working mode; a processing module, configured to respectively control the projection module and the receiving module, and further configured to respectively generate at least three frames of phase shift stripe images and at least one frame of depth image according to the phase shift stripes and the floodlight beam received by the receiving module; calculate the relative phase of each pixel in the TOF sensor by using the at least three frames of phase shift stripe images; obtain the first depth value of each pixel in the at least one frame of depth image, unwrap the relative phase of the corresponding pixel to obtain the absolute phase of each pixel, and calculate the target depth value of each pixel according to the absolute phase of each pixel.

[0006] In a second aspect, the embodiment of the present application provides a depth measurement method applied to the depth measurement device of the first aspect, the method comprising: driving the projection module to project at least three frames of phase shift stripes towards the target object, and controlling the TOF sensor to collect the phase shift stripes reflected by the target object; driving the projection module to project at least one floodlight beam towards the target object, and controlling the TOF sensor to collect the floodlight beam reflected by the target object; generating at least three frames of phase shift stripe images and at least one frame of depth image according to the phase shift stripes and the floodlight beam received by the TOF sensor, and calculating the relative phase of each pixel in the TOF sensor by using the at least three frames of phase shift stripe images; obtaining the first depth value of each pixel in the at least one frame of depth image, demodulating the relative phase of the corresponding pixel to obtain the absolute phase of each pixel, and calculating the target depth value of each pixel according to the absolute phase of each pixel.

[0007] In a third aspect, the embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions, when the processing module executes the computer execution instructions, the depth measurement method provided in the second aspect is realized.

[0008] The beneficial effects of the present application are as follows: compared with the prior art, the present application uses the high-speed depth calculation of iTOF to assist the phase wrapping recovery of phase shift structured light measurement, effectively improves the measurement speed of phase shift structured light. The present application not only combines the phase shift structured light technology and the iTOF technology, but also combines the characteristics of high precision of phase shift structured light and the characteristics of high-speed measurement of iTOF, so that the present application improves the measurement speed on the basis of ensuring high-precision measurement. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 FIG. 1 is a structural schematic diagram of a depth measurement device provided in the embodiment of the present application;

[0010] Figure 2 FIG. 2 is a step flowchart of a depth measurement method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical scheme and advantages of the embodiment of the present application more clear, the technical scheme of the embodiment of the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all the embodiments. Based on the embodiment in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. In addition, although the disclosure is introduced according to one or more examples, it should be understood that each aspect of the disclosure can also constitute a complete embodiment.

[0012] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of the application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0013] The terms "first", "second", and the like in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, for example, those other than the order given in the embodiment illustration or description of the application.

[0014] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not have to be limited to those components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0015] The term "module" used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or / and software code capable of performing functions related to the element.

[0016] When an industrial 3D camera uses a phase-shift structured light technique for measurement, in order to achieve higher accuracy, multiple phase-shift stripes are often used to achieve phase demodulation, such as three-step phase-shift method or four-step phase-shift method. However, based on the above method, only the relative phase The relative phase is discontinuous, and the value of the relative phase is in the range of (0, 2π), so the relative phase needs to be unwrapped to get the continuous absolute phase Wherein, k is the number of stripe period sequences (k = 0, 1, 2, …). The key to phase unwrapping is to quickly and correctly solve the stripe period sequence number k using known conditions.

[0017] However, in the prior art, in order to solve the "phase wrapping" phenomenon in phase shift demodulation, multi-frequency, Fourier transform, and fused Gray code techniques are often used. Among them, multi-frequency and fused Gray code techniques will reduce the measurement speed of the industrial 3D camera because they need to increase the number of exposures; while Fourier transform can improve the measurement speed, but the scene applicability is limited, and cannot meet the high-precision measurement of non-continuous planes.

[0018] Therefore, the present application proposes a depth measurement device and a depth measurement method to solve one or more of the above technical problems.

[0019] Figure 1A structural schematic diagram of a depth measuring device provided in an embodiment of the present application. The depth measuring device 10 comprises: a projection module 101, configured to project at least three frames of phase-shifted fringes and at least one floodlight beam to a target object 20; a receiving module 102, comprising a TOF sensor, configured to respectively collect at least three frames of phase-shifted fringes reflected by the target object in a 2D working mode and collect the floodlight beam reflected by the target object in a 3D working mode; a processing module 103, configured to control the projection module 101 and the receiving module 102, and further configured to respectively generate at least three frames of phase-shifted fringe images and at least one frame of depth image according to the phase-shifted fringes and the floodlight beam received by the receiving module, calculate the relative phase of each pixel in the TOF sensor by using the at least three frames of phase-shifted fringe images, obtain the first depth value of each pixel in the at least one frame of depth image, demodulate the relative phase of the corresponding pixel to obtain the absolute phase of each pixel, and calculate the target depth value of each pixel according to the absolute phase of each pixel.

[0020] In some embodiments, the projection module 101 comprises a driving unit, a laser, a DLP shaper and a lens; wherein the driving unit is configured to drive the laser to emit a pulse signal; the laser is configured to emit preset laser pulse signals of at least two different pulse widths and frequencies; the DLP shaper is configured to receive the at least two different preset laser pulse signals and generate at least three frames of phase-shifted fringes and at least one floodlight beam respectively; and the lens is configured to collimate and project the generated phase-shifted fringes and floodlight beam to the target object 20 respectively. Optionally, the lens can be a single collimating lens or a collimating lens group, which is not limited here.

[0021] In one embodiment, the DLP shaper comprises a plurality of micro-mirrors, each of which has a switching capability of independently controlling the light path and can control the time of turning on and off the light path of each micro-mirror by the pulse width of the preset laser pulse signal emitted by the laser, so as to realize the switching projection of the phase-shifted fringes and the floodlight.

[0022] In one embodiment, the receiving module 102 further comprises a lens and a filter, wherein the lens is configured to receive the light beam reflected by the target object and focus it on the pixels of the image sensor, and the filter is configured to limit the reflected light in the preset waveband from entering the image sensor to avoid the interference of stray light. Optionally, the lens can be a single focusing lens or a focusing lens group, which is not limited here.

[0023] In one embodiment, the image sensor is a TOF sensor, each pixel in the TOF sensor contains 3 or more taps (used to store and read or drain the charge signal generated by the reflected light pulse under the control of the corresponding electrode) compared with the traditional image sensor only used for taking pictures, when each pixel includes multiple taps, the taps are switched in a certain order to collect the electrons generated by the light signal received by the pixel reflected by the target to form a charge signal in a single frame period T (or within a single exposure time).

[0024] In one embodiment, the TOF sensor includes two working modes, namely 2D working mode and 3D working mode, wherein the 2D working mode is a single-frequency shuffle (rotation), multi-frame mode, and the TOF sensor in the 2D working mode is used to receive the stripe reflected by the target; the 3D working mode is a single-frequency noshuffle (non-rotation), single-frame mode, and the TOF sensor in the 3D working mode is used to receive the floodlight beam reflected by the target. It should be noted that shuffle / noshuffle is a working mode of the tap in the TOF sensor, shuffle means that each tap works multiple times when acquiring a single frame image, and noshuffle means that each tap only works once when acquiring a single frame image.

[0025] Specifically, the TOF sensor is controlled by the processing module to start the 2D working mode and synchronously generate a stripe trigger signal to the driving unit, so that the driving unit drives the laser to generate a laser pulse signal; the DLP shaper receives the laser pulse signal and projects at least three frames of phase-shifted stripes to the target through the lens, and the at least three frames of phase-shifted stripes are sequentially exposed in time; when the at least three frames of phase-shifted stripes are sequentially exposed, the TOF sensor sequentially collects different frames of phase-shifted stripes reflected by the target and transmits them to the processing module to generate a plurality of frames of stripe images.

[0026] In one embodiment, the TOF sensor collects different frames of phase-shifted stripes reflected by the target and transmits them to the processing module to generate a plurality of frames of stripe images, which specifically includes: at least 3 taps in each pixel of the TOF sensor collect at least 3 light signals in a single frame period, and convert the light signals into charge signals to obtain a rawphase image for transmission to the processing module 103, wherein the time and interval of each tap collection are the same; the processing module 103 calculates the intensity information of each pixel according to the rawphase image, and after traversing all pixels, a frame of stripe image can be obtained; wherein the rawphase image is the original data of the TOF sensor converting the collected light signal into a digital signal, and the pixel value of each pixel in the stripe image is the intensity information.

[0027] It should be noted that the above is the formation of a single frame of fringe image, and the above steps can be repeated to obtain multiple frames of fringe image, which is not limited here.

[0028] In one embodiment, the 3 taps in each pixel of the TOF image sensor collect 4 times of light signals in a single frame period, and the calculation of the intensity information of each pixel from the rawphase image includes: the amount of charges collected by each tap in each pixel in turn is C1, C2, C3 and C4, and the intensity information of a single pixel is calculated as follows:

[0029]

[0030] When there is an influence of ambient light, it is difficult to eliminate the influence of ambient light by the above method, resulting in a low signal-to-noise ratio of the final fringe image. Therefore, in another embodiment, assuming that the ambient light does not change in a very small time range, the intensity information of a single pixel can be calculated according to the following formula:

[0031]

[0032] It should be noted that when the number of times of collecting light signals by each pixel in a single frame period exceeds the number of taps included in each pixel, the amount of charges corresponding to the ambient light collected each time is recorded, and the intensity information of a single pixel can be obtained by calculating the amount of charges of the collected reflected light and the recorded amount of charges corresponding to the ambient light, which is not limited here.

[0033] Further, after the TOF sensor sequentially collects the fringe phase shift fringe reflected by the target object, the processing module controls the TOF sensor to start the 3D working mode and synchronously sends a floodlight trigger signal to the driving unit from the TOF sensor, so that the driving unit drives the laser to generate a laser pulse signal; the DLP shaper receives the laser pulse signal and projects a floodlight to the target through the lens, and at least 3 taps in each pixel of the TOF sensor collect at least 3 times of light signals in a single frame period and convert them into corresponding charge signals to obtain a rawphase image and transmit it to the processing module 103; wherein the time and interval of each tap are the same.

[0034] In one embodiment, the processing module 103 is further configured to process the rawphase image obtained in the 3D mode to obtain the corresponding floodlight time of flight of each pixel, and calculate the first depth value of each pixel according to the corresponding floodlight time of flight of each pixel, so as to obtain the depth image of the target by traversing all pixels.

[0035] Specifically, phase information of each pixel is calculated according to the rawphase image, preferably, taking the example that each pixel in the TOF sensor includes 3 taps to sequentially collect 3 times of light signals, the amount of charges obtained by each tap in each pixel is C1, C2 and C3, and the pulse width T of the pulse collection signal of each tap is h Then, the time of flight of a single pixel is:

[0036]

[0037] Further, a first depth value can be obtained according to d = c * t / 2, where c represents the speed of light. The first depth value of each pixel is obtained by traversing all pixels. It should be noted that if each pixel in the TOF sensor includes 4 taps, a four-phase sampling method can be used to calculate the corresponding time of flight; the present application can also obtain multiple frames of depth images, and the mean value of the corresponding depth values of each pixel in each depth image is taken as the first depth value, which is not limited here.

[0038] Therefore, the TOF sensor can be used to collect different phase-shifted stripes generated by the projection module, and can also realize the depth measurement function based on iTOF. Not only the cost is saved, but also both working modes are considered, thereby improving the measurement accuracy of the depth device.

[0039] Based on the content described in the above embodiments, as shown in Figure 2 The present application also provides a depth measurement method applied to the above depth measurement device.

[0040] S201, driving the projection module to project at least three frames of phase-shifted stripes towards the target object, and controlling the TOF sensor to collect the phase-shifted stripes reflected by the target object;

[0041] S202, driving the projection module to project at least one floodlight beam towards the target object, and controlling the TOF sensor to collect the floodlight beam reflected by the target object;

[0042] S203, generating at least three frames of phase-shifted stripe images and at least one frame of depth image according to the phase-shifted stripes and the floodlight beam received by the TOF sensor respectively, and calculating the relative phase of each pixel in the TOF sensor by using the at least three frames of phase-shifted stripe images;

[0043] S204, obtaining the first depth value of each pixel in the at least one frame of depth image to dephase the corresponding relative phase to obtain the absolute phase of each pixel, and calculating the target depth value of each pixel according to the absolute phase of each pixel.

[0044] In some embodiments, the application can perform phase unwrapping on the acquired at least three phase-shifted fringe images by phase-shifting method. In one embodiment, the phase-shifting method includes multi-step methods such as three-step method, four-step method and five-step method, and based on the consideration of error influence and calculation speed, the application preferably uses the four-step method for calculation, that is, in the present embodiment, phase unwrapping is performed by acquiring four phase-shifted fringe images.

[0045] Specifically, based on the light intensity formula The expression of each phase-shifted fringe image can be obtained as follows:

[0046]

[0047]

[0048]

[0049]

[0050] wherein a represents average brightness, b is the amplitude of the modulation signal, represents absolute phase.

[0051] Thus, the expression of relative phase can be obtained based on the above formula:

[0052]

[0053] Since the expression of absolute phase is:

[0054]

[0055] wherein the value range of relative phase is [-π, π]; k is a positive integer, representing the period number of the fringe. Since k in the above formula (3) is the period number of the fringe, the period number k cannot be determined by four fringe images, and therefore, to determine the absolute phase, the value of k must be determined first.

[0056] In one embodiment, based on the formula for calculating the time of flight wherein f m is the modulation frequency of the projection module, represents phase delay, that is, absolute phase, and thus, the absolute phase can be calculated by using the time of flight calculated in the TOF mode, and the corresponding fringe k level is calculated by formula (3).

[0057] In another embodiment, assuming that the projection image coordinates X p of a pixel are known, the absolute phase of the pixel can be calculated according to the following formula (4):

[0058]

[0059] wherein N is the number of fringes, W is the horizontal resolution of the projected image, represents the absolute phase. It is to be noted that the projected image is the fringe to be emitted on the projection module projection plane which is aligned with the imaging plane of the receiving module.

[0060] Further, the first depth value calculated by means of the 3D working mode is used to recover the phase wrapping of the fringe image acquired in the 2D working mode. Specifically, any pixel is selected. Since the same sensor is used to acquire the images in the present application, the first depth value corresponding to the pixel can be directly selected according to the coordinates of the pixel, and the projected image coordinates Xp corresponding to the pixel are calculated according to the corresponding first depth value. p The absolute phase of the pixel is calculated by formula (4) and the k value is solved by using the absolute phase and formula (3), so that a more accurate target depth value of the pixel can be calculated based on the absolute phase of the kth fringe.

[0061] In some embodiments, the projected image coordinates Xp of the pixel calculated by means of the first depth value can specifically include: assuming that the world coordinates corresponding to a pixel p in the depth image obtained based on the 3D working mode are (X, Y, Z), the world coordinates (X, Y, Z) of the point p are projected into the imaging plane of the receiving module and the projection plane of the projection module which is aligned with the imaging plane, respectively, wherein the coordinates projected into the imaging plane of the receiving module are marked as X C , and the coordinates projected into the projection plane are marked as X P , so that:

[0062] S C X C =K C [R e |T C ]X=P C X

[0063] S P X P =K P [R P |T P ]X=P P X (5)

[0064] wherein X represents the homogeneous coordinates of the world coordinates (X, Y, Z) of the point p, S C and S P represent scale factors, K P and K C represent the intrinsic matrices of the projection module and the receiving module, R P |T P and R C |T CThe projection matrix representing the projection module and the receiving module can be simplified as P C , P P In one embodiment, it is assumed that:

[0065]

[0066] Thus, according to formula (6), the world coordinates (X, Y, Z) of the point p can be expressed as:

[0067]

[0068] wherein x c , y c represent the coordinates of the point p on the imaging plane of the receiving module, and x p , y p represent the coordinates of the point p in the projection plane. Through the world coordinates (X, Y, Z) of the point p and the coordinates of the point p on the imaging plane, X P = (x p , y p ) can be calculated according to formula (7), and the absolute phase P of the point p can be calculated by substituting X into formula (4), and the value of k can be calculated by using formula (3). Since the depth of the point p is calculated based on the accurate phase value, and the depth value calculated by the phase shift method is more accurate than that calculated by the TOF principle, the more accurate target depth value of the point p can be calculated according to the absolute phase of the point p of the kth stripe. The target depth values of all the pixels can be obtained by traversing all the pixels according to the above method.

[0069] In one embodiment, the projection image coordinates X P of the pixel point p are calculated by using formula (4) according to the absolute phase of the point p of the kth stripe, and the target depth value of the point p can be calculated by using formula (7); or the target depth value of the pixel point p can be calculated by using the absolute phase of the point p of the kth stripe and by using the method of triangulation, which is not limited here.

[0070] In summary, the prior art generally uses the multi-frequency method to demodulate the phase of the phase shift stripe image, but it needs to continue to expose 4 times (double frequency) or 8 times (triple frequency) in addition to the original 4 times of exposure, which will greatly reduce the speed of 3D measurement. In the present application, the depth image collected by the TOF sensor is used to assist the phase shift stripe image to calculate the phase wrapping, compared with the existing multi-frequency demodulation method, it only needs to expose once in addition to the original exposure times, which can improve the speed of 3D measurement. In addition, the target depth is obtained by using the phase shift demodulation method of the phase shift stripe structure light, which also ensures the measurement accuracy.

[0071] Further, based on the content described in the above embodiments, the present embodiment further provides a computer readable storage medium, which stores computer execution instructions, when a processing module executes the computer execution instructions, to achieve the steps executed by the processing module in the above embodiments.

[0072] It should be understood that, in several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0073] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0074] In addition, the functional modules in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present alone, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware or in the form of hardware plus software function unit.

[0075] The integrated modules realized in the form of software function modules can be stored in a computer readable storage medium. The software function modules stored in the storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in each embodiment of the present application.

[0076] It should be understood that the above processing module can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0077] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0078] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The above-mentioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned storage medium includes ROM, RAM, magnetic disk or optical disk and various storage medium that can store program codes.

[0079] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A depth measuring device, characterized by, include: A projection module is used to project at least three phase-shifted fringes and at least one floodlight beam onto a target object. The receiving module includes a TOF sensor for acquiring the at least three frames of phase-shifted fringes reflected by the target object in 2D working mode and the floodlight beam reflected by the target object in 3D working mode. The processing module is configured to: when the processing module controls the TOF sensor to start a 2D working mode, the TOF sensor synchronously generates a stripe trigger signal to the projection module to drive the projection module to project at least three frames of phase-shifted stripes; when the processing module controls the TOF sensor to start a 3D working mode, the TOF sensor synchronously generates a floodlight trigger signal to the projection module to drive the projection module to project at least one floodlight beam; and generate at least three frames of phase-shifted stripe images and at least one frame of depth images based on the phase-shifted stripes and the floodlight beam received by the receiving module, respectively; calculate the relative phase of each pixel in the TOF sensor using the at least three frames of phase-shifted stripe images; obtain the first depth value of each pixel in the at least one frame of depth image, dephase the relative phase of the corresponding pixels to obtain the absolute phase of each pixel, and calculate the target depth value of each pixel based on the absolute phase of each pixel.

2. The apparatus of claim 1, wherein, The projection module includes a driving unit, a laser, a DLP shaper, and a lens; wherein: The driving unit is used to drive the laser to emit pulse signals; The laser is used to emit at least two preset laser pulse signals with different pulse widths and frequencies; The DLP shaper is used to receive the at least two different preset laser pulse signals and generate at least three frames of phase-shifted fringes and at least one floodlight beam, respectively. The lens is used to project the generated phase-shifting fringes and floodlight beam onto the target object, respectively.

3. The apparatus of claim 2, wherein, The DLP shaper includes multiple micromirrors, each of which has the ability to independently control the switching of the optical path. The timing of opening and closing the optical path of each micromirror can be controlled by the pulse width of the preset laser pulse signal emitted by the laser, thereby realizing the switching projection of phase-shifting fringes and floodlight beams.

4. The apparatus of claim 1, wherein, The TOF sensor includes at least one pixel, and each pixel includes at least three taps; wherein the 2D working mode is a single-frequency, multi-frame mode with rotating taps, and the 3D working mode is a single-frequency, single-frame mode with non-rotating taps.

5. The apparatus of claim 4, wherein, In the 2D working mode, at least three taps in each pixel of the TOF sensor are used to acquire light signals at least three times in a single frame period and convert them into charge signals to obtain a raw phase image and transmit it to the processing module. The processing module calculates the intensity information of each pixel based on the raw phase image, and obtains the corresponding phase-shifted stripe image after traversing all pixels.

6. The apparatus of claim 4, wherein, In the 3D working mode, at least three taps in each pixel of the TOF sensor are used to collect light signals at least three times in a single frame period and convert them into corresponding charge signals to obtain a raw phase image and transmit it to the processing module. The processing module processes the rawphase image to obtain a corresponding general light time of flight of each pixel and further calculates a first depth value of each pixel.

7. A method of measuring depth, characterized by, The depth measurement device of any one of claims 1-6, wherein the depth measurement method comprises: driving the projection module to project at least three frames of phase-shifted fringes toward the target object and controlling the TOF sensor to collect the phase-shifted fringes reflected by the target object; driving the projection module to project at least one general light beam toward the target object and controlling the TOF sensor to collect the general light beam reflected by the target object; generating at least three frames of phase-shifted fringe images and at least one frame of depth image from the phase-shifted fringes and the general light beam received by the TOF sensor, respectively, and calculating a relative phase of each pixel in the TOF sensor using the at least three frames of phase-shifted fringe images; obtaining a first depth value of each pixel in the at least one frame of depth image, demodulating the relative phase of the corresponding pixel to obtain an absolute phase of each pixel, and calculating a target depth value of each pixel according to the absolute phase of each pixel.

8. The method of claim 7, wherein, The obtaining a first depth value of each pixel in the at least one frame of depth image and demodulating the relative phase of the corresponding pixel to obtain an absolute phase of each pixel comprises: obtaining a first depth value of each pixel in the at least one frame of depth image, and obtaining a corresponding world coordinate of each pixel using the first depth value of each pixel; projecting the corresponding world coordinate of each pixel into an imaging plane of the receiving module and a projection plane of the projection module aligned with the imaging plane, and calculating a projection coordinate of each pixel in the projection plane using an imaging coordinate of each pixel in the imaging plane of the receiving module and the corresponding world coordinate of each pixel; calculating the absolute phase of each pixel point according to the projection image coordinate of each pixel.

9. The method of claim 7, wherein, The obtaining a first depth value of each pixel in the at least one frame of depth image and demodulating the relative phase of the corresponding pixel to obtain an absolute phase of each pixel comprises: obtaining a first depth value of each pixel in the at least one frame of depth image, and calculating a corresponding time of flight using the first depth value; Time-of-flight formula computing the absolute phase; wherein is the modulation frequency of the projection module, denotes the absolute phase.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and when the processing module executes the computer execution instructions, the depth measurement method of any one of claims 7-9 is implemented.

Images