A fast deep reconstruction method, system, device, and medium

Abstract

A fast depth reconstruction method comprises the following steps: step S1: obtaining data of a floodlight emission signal and demodulation signals; wherein one of the emission signals corresponds to at least two of the demodulation signals; step S2: in a first area of a depth map, simultaneously calculating the phase offset between the emission signal and the demodulation signal for multiple pixel points using a standard formula; wherein the standard formula uses absolute value for calculation; step S3: calculating the depth value of the pixel according to the phase offset; step S4: in a second area of the depth map, calculating the depth value of the pixel using disparity according to the corresponding demodulation signal; step S5: merging the depth value of the first area and the depth value of the second area to obtain the depth value of the depth map. The application can greatly improve the running speed of the application on a chip with parallel computing capability, fully utilize the computing capability of the chip, and improve the data acquisition speed.

Description

Technical Field

[0001] This invention relates to the field of depth measurement technology, and more specifically, to a rapid depth reconstruction method, system, device, and medium. Background Technology

[0002] Depth measurement technology can obtain depth data of a target object, thereby generating 3D images with depth information, such as RGBD. Compared to 2D images, 3D images contain more data, enabling better target recognition. Due to technological advancements, it has already found applications in fields such as facial recognition, 3D modeling, and VR. Compared to other depth measurement technologies, Time-of-Flight (TOF) measurement technology has a wider range and better consistency in accuracy, making it widely used in scenarios with a broad measurement scope.

[0003] Time-of-Flight (TOF) technology is further divided into iTOF and dTOF technologies based on their data acquisition principles. iTOF indirectly calculates depth data by emitting a laser signal of a specific waveform and calculating the phase difference between the received and emitted signals. dTOF, on the other hand, directly calculates depth data by receiving the time difference between the received and emitted signals. Due to the extremely high speed of light, the accuracy requirements for the transmitting and receiving devices are very high. Therefore, dTOF is mainly used for long-distance measurements, while iTOF is primarily used for medium-distance measurements. In enclosed spaces, iTOF technology is the dominant application.

[0004] Binocular vision technology uses the parallax of two images to calculate depth values. While it offers high accuracy at medium and close distances, it struggles to obtain accurate data for conditions with similar colors and also suffers from high computational complexity.

[0005] In enclosed spaces, such as cockpits, the mainstream computing chips are fixed-point DSP chips, while iTOF calculations mainly rely on floating-point operations, which means that current algorithms cannot be directly applied to fixed-point DSP chips. Summary of the Invention

[0006] Therefore, this invention uses standard formulas for calculation, eliminating the need for excessive classification and enabling rapid computation. This significantly improves the operating speed of the invention on chips with parallel computing capabilities, fully utilizing the chip's computing power, increasing data acquisition speed, and shortening response time. Simultaneously, this invention combines Time-of-Flight (TOF) technology with binocular vision technology and divides the depth map, thereby maximizing the use of the characteristics of both technologies and achieving a balance between speed and accuracy.

[0007] In a first aspect, the present invention provides a rapid depth reconstruction method, characterized by comprising the following steps:

[0008] Step S1: Obtain data of the floodlight's transmitted signal and demodulated signal; wherein, one transmitted signal corresponds to at least two demodulated signals;

[0009] Step S2: In the first region of the depth map, the phase offset between the transmitted signal and the demodulated signal is calculated simultaneously for multiple pixels using a standard formula; wherein, the standard formula uses absolute values ​​for calculation;

[0010] Step S3: Calculate the depth value of the pixel based on the phase offset;

[0011] Step S4: In the second region of the depth map, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal;

[0012] Step S5: Combine the depth values ​​of the first region and the second region to obtain the depth value of the depth map.

[0013] Optionally, the fast depth reconstruction method is characterized in that, during the calculation in step S2, the calculated data is multiplied by 2 to the power of n and converted into an integer, and then integer operations are performed; and after the operation is completed, it is divided by 2 to the power of n and converted into a floating-point number.

[0014] Optionally, the fast depth reconstruction method is characterized in that, in step S2, the phase offset is calculated using the formula atan2(y,x)=sgn(y)*sgn(x)*((sgn(x)-cond)*pi / 2+(2*cond-1)*atan(|min| / |max|)); where cond=|y|<|x|, atan(|min| / |max|)=atan(min(|x|,|y|),max(|x|,|y|)).

[0015] Optionally, the fast depth reconstruction method is characterized in that multiple pixels are divided into groups according to computing power, and the groups are processed sequentially.

[0016] Optionally, the fast depth reconstruction method is characterized in that step S2 includes:

[0017] Step S21: Mark the transmitted signals in sequence;

[0018] Step S22: Mark the received demodulated signals in sequence;

[0019] Step S23: Identify the winding portion based on the waveform of the demodulated signal;

[0020] Step S24: Recover the wound portion according to the order of the transmitted signal and the demodulated signal to obtain the recovered demodulated signal;

[0021] Step S25: Calculate the relative offset based on the recovered demodulated signal and the transmitted signal.

[0022] Optionally, the fast depth reconstruction method is characterized in that the image region is divided into several sub-regions, and a marker point is set for at least one of the sub-regions to determine whether the sub-region has a wrapped portion.

[0023] Optionally, the fast depth reconstruction method is characterized in that step S2 includes:

[0024] Step S21: Mark the transmitted signals in sequence;

[0025] Step S22: Mark the received demodulated signals in sequence;

[0026] Step S23: Identify the winding portion based on the waveform of the demodulated signal;

[0027] Step S26: For the pixel that has been wound up, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal.

[0028] Secondly, the present invention provides an iTOF fast depth reconstruction system for implementing the fast depth reconstruction method described in any of the preceding claims, characterized in that it includes:

[0029] An acquisition module is used to acquire data of the floodlight's transmitted signal and demodulated signal; wherein, one transmitted signal corresponds to at least two demodulated signals;

[0030] The bias module is used to calculate the phase bias between the transmitted signal and the demodulated signal simultaneously for multiple pixels in a first region of the depth map using a standard formula; wherein the standard formula uses absolute values ​​for calculation.

[0031] The first depth module is used to calculate the depth value of the pixel based on the phase offset;

[0032] The second depth module is used to calculate the depth value of a pixel in the second region of the depth map using parallax based on the corresponding demodulated signal.

[0033] The merging module is used to merge the depth values ​​of the first region and the second region to obtain the depth values ​​of the depth map.

[0034] Thirdly, the present invention provides an iTOF rapid depth reconstruction device, characterized in that it includes:

[0035] processor;

[0036] A memory in which executable instructions of the processor are stored;

[0037] The processor is configured to perform the steps of any of the preceding fast depth reconstruction methods by executing the executable instructions.

[0038] Fourthly, the present invention provides a computer-readable storage medium for storing a program, characterized in that, when the program is executed, it implements the steps of the fast depth reconstruction method described in any one of the preceding claims.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] This invention uses a standard formula to calculate the phase offset, eliminating the need for decision branches, enabling better parallel computing, improving calculation speed, and reducing response time.

[0041] This invention improves the calculation formula so that the calculation only needs to use absolute values, transforming the four cases that need to be classified into a standard case, simplifying the calculation logic and improving the calculation efficiency.

[0042] This invention also improves the calculation formula, enabling the chip to process multiple pixels simultaneously, thereby maximizing the chip's parallel computing capabilities, improving the chip's processing efficiency and capacity, shortening the iTOF data calculation time, and increasing efficiency.

[0043] This invention combines Time-of-Flight (TOF) technology with binocular vision technology, leveraging the advantages of both to achieve a fast and accurate calculation process. By pre-sensing a second region where entanglement might occur, calculations are performed directly using binocular vision, while the first region, where entanglement is unlikely, is calculated using TOF technology. This significantly reduces the amount of computation required later and improves efficiency. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1This is a flowchart illustrating the steps of a fast depth reconstruction method in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram illustrating time consumption in an embodiment of the present invention;

[0047] Figure 3 This is a flowchart illustrating the steps for calculating relative bias in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of a winding portion in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of image region division in an embodiment of the present invention;

[0050] Figure 6 This is a flowchart illustrating another step in calculating the relative bias in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of a module of an iTOF fast depth reconstruction system in an embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the structure of an iTOF fast depth reconstruction device according to an embodiment of the present invention; and

[0053] Figure 9 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0054] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0055] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0056] The present invention provides a rapid depth reconstruction method, which aims to solve the problems existing in the prior art.

[0057] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0058] The fast depth reconstruction method provided in this invention is applicable to fixed-point DSP chips. By standardizing the calculation formula and removing decision branches, the advantages of parallel computing are fully utilized, improving the chip's processing efficiency, thereby reducing calculation time, shortening response time, and improving efficiency.

[0059] In iTOF technology, periodically modulated signals are used for active emission, and then the received signal is demodulated with zero difference to obtain the phase shift of the reflected light. Taking a sinusoidal modulated transmitted signal as an example, it can be expressed by the following formula:

[0060] s(t)=A s cos(2πf mod t)+B s

[0061] Among them, A s B represents the amplitude of the signal. s f is the bias of the signal; mod This is the modulation frequency.

[0062] The received signal r(t) is obtained by delaying and attenuating the reflected signal:

[0063] r(t) = αs(t - ΔT)

[0064] Where 0≤α<<1, α is the attenuation coefficient, the value of α depends on the target distance and surface reflectivity, and ΔT is the delay time of the returned signal.

[0065] A continuous-wave time-of-flight sensor measures the distance to each pixel by sampling the phase difference function between the received signal r(t) and the demodulated signal g(t) at the same frequency s(t). Ideally, the demodulated signal is also a sine wave.

[0066] g(t) = A g cos(2πf mod t)+B g

[0067] The operations performed on the pixels are correlation operations:

[0068]

[0069] When both the transmitted and demodulated signals are sinusoidal, the correlation value is applied to the demodulated signal as a function of the delay T:

[0070] c(τ)=Acos(2πf mod t(ΔT-τ))+B

[0071] Among them, A=A g A s α, B = αB g B s .

[0072] Subsequently, within each quarter-cycle, the correlation function c(τ) is sampled (by changing the phase of the transmitted optical signal in 90° steps). The phase offset between the transmitted and demodulated signals is then considered. The following formula can be used to estimate:

[0073]

[0074] Depth is proportional to phase shift:

[0075]

[0076] Figure 1 This is a flowchart illustrating the steps of a fast depth reconstruction method according to an embodiment of the present invention. Figure 1 As shown, a fast depth reconstruction method in this embodiment of the invention includes the following steps.

[0077] Step S1: Obtain the data of the floodlight's transmitted signal and demodulated signal.

[0078] In this step, the transmitted and demodulated signals of the floodlight are obtained, including the shape, period, transmission time, and reception time of the transmitted and demodulated signals. One transmitted signal corresponds to at least two demodulated signals. The same transmitted signal is received by at least two receivers, resulting in at least two demodulated signals. These two demodulated signals constitute a binocular system, enabling the acquisition of depth data. A certain time difference exists between the transmitted and demodulated signals; however, due to the short measurement distance, this time difference is very short. Accurate measurement requires very high-precision equipment, which is economical. Therefore, by observing the overall changes in the transmitted and demodulated signal data, accurate data measurement can be achieved under relatively low-precision conditions.

[0079] Step S2: In the first region of the depth map, the phase offset between the transmitted signal and the demodulated signal is calculated simultaneously for multiple pixels using a standard formula.

[0080] In this step, the first region is the region where no winding is expected, such as a region with close proximity or a region without multipath effects. The standard formula uses absolute values ​​for calculation. The specific form of the standard formula varies depending on the transmitted signal pattern used. The standard formula enables phase offset... Calculations in different quadrants can be performed using the same formula without classifying input and output values, thus reducing decision branches and enabling the chip to process data from multiple pixels in batches, improving efficiency. When sampling the correlation function c(τ), it is necessary to sample and calculate separately every 1 / 4 cycle. However, by using the standard formula, the entire cycle no longer needs to be finely divided, allowing for overall sampling and calculation, improving consistency, and demonstrating the advantages of parallel computing. Figure 2 This is a schematic diagram illustrating time consumption in an embodiment of the present invention. From... Figure 2 It is clear from the data that branch judgment consumes too much time, while the standard formula can save time in branch judgment, thereby maximizing the utilization of the chip's processing power.

[0081] In some embodiments, during this calculation step, the calculated data is multiplied by 2 to the power of n, converted to an integer, and then subjected to integer operations. After the operation, it is divided by 2 to the power of n to convert it to a floating-point number. For example, the floating-point number 23.345346, when calculated with a precision of 3 decimal places, is multiplied by 2 to the power of 10, resulting in 23905.634304. The decimal part is discarded, resulting in 23905, which is then used for calculation. Assuming the calculated value is 6455454, dividing it by 2 to the power of 10 yields the floating-point number 6304.154296, which is the final calculation result. Furthermore, fixed-point multiplication and division by powers of 2 can be replaced by bit shifting operations to improve efficiency. By converting the values, floating-point operations are transformed into integer operations, improving processing efficiency while keeping the precision loss within a controllable range, thus ensuring accuracy. Floating-point numbers refer to the set of real numbers other than integers. In C, the function for floating-point numbers is `float`, and the function for integer arithmetic is `int`. Of course, other programming languages ​​also have corresponding functions; those skilled in the art simply need to choose the appropriate function for the language.

[0082] In some embodiments, applicable to sinusoidal waveforms, the phase offset is calculated using the formula atan2(y,x)=sgn(y)*sgn(x)*((sgn(x)-cond)*pi / 2+(2*cond-1)*atan(|min| / |max|)); where cond=|y|<|x|, atan(|min| / |max|)=atan(min(|x|,|y|),max(|x|,|y|)), and sgn() is the sign function. When x is negative, sgn(x) is "-"; when x is positive, sgn(x) is "+". When |y|<|x|, cond is 1; when |y|≥|x|, cond is 0. atan(y / x)=sgn(y)*sgn(x)*atan(|y| / |x|)=sgn(y)*sgn(x)*((1-cond)*pi / 2+(2*cond-1)*atan(|min| / |max|)). This formula standardizes the function in each of the four quadrants and obtains the calculation results for each quadrant, reducing the number of decision nodes under the sine waveform, which is beneficial for parallel computing and improves the calculation speed.

[0083] In some embodiments, multiple pixels are divided into groups based on computing power, and the groups are processed sequentially. For example, if the computing chip can process 64 pixels in parallel, then all pixels are divided into multiple groups of 64, and the calculation is performed on each group sequentially. The number of groups varies depending on the computing chip.

[0084] Step S3: Calculate the depth value of the pixel based on the phase offset.

[0085] In this step, the formula is used. The depth value of each pixel is calculated. By calculating the depth value of each pixel whose phase offset has been calculated, the depth value of all pixels can be obtained, thus producing a depth map.

[0086] In some embodiments, multiple pixels are divided into groups based on computing power, and the groups are processed sequentially. Unlike step S2, the number of pixels within a group in this step is greater than the number of pixels within a group in step S2, and the number of groups in this step is less than the number of pixels within a group in step S2.

[0087] Step S4: In the second region of the depth map, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal.

[0088] In this step, the second region is the area where entanglement may occur, such as areas that are too far away or areas with multipath effects. The first and second regions are fixed in many application scenarios, such as inside a smart cockpit or a VR experience zone. Known object positions at a preset shooting angle can be used to pre-define the first and second regions. Simultaneously, potential entanglement issues in the first region need to be addressed promptly. The acquired demodulated signals form a binocular system, which can utilize the parallax principle to obtain depth data. Since only the second region in the image needs to be calculated, the computational load is small, and data can be obtained in a short time. The second regions in the images generated by the two demodulated signals are not identical. When performing parallax calculation, only the pixels in the second region need to be matched. When the second region consists of multiple independent sub-regions, only the pixels within the corresponding sub-regions need to be calculated, without needing to match all pixels within the second region, thus minimizing the computational load.

[0089] Step S5: Combine the depth values ​​of the first region and the second region to obtain the depth value of the depth map.

[0090] In this step, the first and second regions constitute a complete depth map. The depth values ​​of the first and second regions are merged pixel-wise to obtain the complete depth map. At the junction of the first and second regions, smoothing is performed using neighborhood averaging. Smoothing is applied to N pixels at the boundaries of each of the first and second regions. N is proportional to the area of ​​either the first or second region. N is not a fixed value and can differ between the first and second regions. N can also differ for sub-regions within either the first or second region.

[0091] Figure 3 This is a flowchart illustrating the steps for calculating relative offset in an embodiment of the present invention, applicable to applications in enclosed spaces. For example... Figure 3 As shown, a method for calculating relative bias in an embodiment of the invention includes:

[0092] Step S21: Mark the transmitted signals in sequence.

[0093] In this step, the transmitted signal is emitted periodically. Typically, the emitted signal is infrared laser. In applications within enclosed spaces, especially smaller enclosed spaces such as inside a car, each emission results in a reflected signal being received, allowing for matching of the emitted and demodulated signals. The emitted signals are sequentially labeled, with each signal emitted within a cycle uniquely labeled, such as A1, A2, A3, A4… The labeling is independent of the shape of the emitted signal, depending only on the period.

[0094] Step S22: Mark the received demodulated signals in sequence.

[0095] In this step, due to the varying distances to the target object, different pixels receive the demodulated signal at different times. Depending on the application scenario, the receiving time can be divided to distinguish data within the effective distance. This step uniquely labels the received signal according to the pixel, such as BF1, BF2, BF3, BF4, etc. Unlike step S21, step S22 labels the signal according to the pixel, and labels the demodulated signal received by each pixel separately.

[0096] Step S23: Identify the winding portion based on the waveform of the demodulated signal.

[0097] In this step, the coiled portion is identified through waveform analysis. When the distance is too great or multipath effects are present, coiled portions are more likely to exist, causing distortion of the depth data. Coiling typically manifests as overlapping, coinciding, or excessively close waveforms. By detecting the waveform, the coiled portion can be identified. The identification effect is particularly good in relatively fixed, enclosed spaces. Figure 4 This is a schematic diagram of a wound section. As can be seen from the diagram, the waveform of the wound section is significantly different from the waveform in the normal state, and therefore can be identified by waveform detection.

[0098] Step S24: Recover the wound portion according to the order of the transmitted signal and the demodulated signal to obtain the recovered demodulated signal.

[0099] In this step, the wound portion is restored based on the waveform and overlap status. For the overlapping portion, it is decomposed based on the two overlapping waveforms to ensure that all overlapping waveforms are complete. Then, the overlapping waveforms are matched sequentially with the transmitted signal, and the correspondence between the transmitted and demodulated signals is readjusted. Figure 4 In the process, the overlapping portion is divided into m1 and m2, and these divisions are performed separately to ensure that both waveforms are complete. Then, based on the order of the split waveforms, the transmitted signal and the demodulated signal are re-matched.

[0100] Step S25: Calculate the relative offset based on the recovered demodulated signal and the transmitted signal.

[0101] In this step, due to the winding, the matching relationship between the initial demodulated signal and the transmitted signal is incorrect. Therefore, it is necessary to re-match the demodulated signal and the transmitted signal based on the recovered demodulated signal, and calculate the relative offset based on the final matching relationship. For example, if the transmitted signals corresponding to the continuous part of a demodulated signal are 5, 6, 7, 8, 9, 10, and there is winding in part 7, after recovery, it becomes two independent demodulated signals. Then the transmitted signals corresponding to this continuous part are 5, 6, (7, 8), 9, 10, 11, where (7, 8) are the transmitted signals corresponding to the recovered signals.

[0102] In some embodiments, when restoring an image, the image region is divided into several sub-regions, and a marker point is set for at least one of the sub-regions to determine whether the sub-region has a coiled portion. For example... Figure 5 As shown, when shooting a fixed scene from a fixed angle, the location of the winding is relatively fixed. Within region R, due to the greater distance, winding may occur, especially in variable states where winding may occur in one state and not in another. Therefore, several marker points are set in region R to determine whether winding will occur. If none of the marker points indicate winding, it is assumed that the current region will not experience winding, thus eliminating the need to judge other parts, saving computation and improving efficiency. If any marker point indicates winding, it is assumed that the current region may experience winding, requiring winding judgment within that region to improve accuracy.

[0103] In some embodiments, a temperature parameter is incorporated when calculating the depth value of the pixel to improve accuracy. Temperature variations are significant in enclosed spaces. For example, the temperature inside a cockpit can reach approximately 50°C in the summer sun, while it can drop below zero in the dead of winter. These large temperature differences severely impact the iTOF data results, necessitating temperature correction. Temperature correction requires consideration of both the target object and the camera. Temperature variations significantly affect the camera's thermal sensitivity error. Furthermore, temperature changes also affect the surface reflectivity of the target object; when the temperature difference is substantial, the impact on reflectivity cannot be ignored.

[0104] Figure 6 This is a flowchart illustrating another step in calculating the relative bias according to an embodiment of the present invention. Unlike the previous embodiments, this other step in calculating the relative bias includes:

[0105] Step S21: Mark the transmitted signals in sequence.

[0106] Step S22: Mark the received demodulated signals in sequence.

[0107] Step S23: Identify the winding portion based on the waveform of the demodulated signal.

[0108] Step S26: For the pixel that has been wound up, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal.

[0109] In this step, unlike the previous embodiments, binocular technology is used to calculate the entangled pixels, thereby directly obtaining accurate calculation results. When using a binocular system for calculation, it is only necessary to perform matching calculations within a certain range around the current pixel, without having to search and calculate the entire image.

[0110] After identifying the tangled pixel in this embodiment, the depth value of the pixel is recalculated using binocular technology. This allows for more accurate depth values ​​to be obtained using binocular technology, achieving dynamic integration of depth measurement with TOF and binocular technologies. This ensures data acquisition quality while maximizing calculation speed.

[0111] Figure 7 This is a schematic diagram of a module of an iTOF fast depth reconstruction system according to an embodiment of the present invention. Figure 7 As shown, an iTOF fast depth reconstruction system in an embodiment of the present invention includes:

[0112] The acquisition module 100 is used to acquire data of the floodlight's transmitted signal and demodulated signal; wherein, one transmitted signal corresponds to at least two demodulated signals.

[0113] The bias module 200 is used to simultaneously calculate the phase bias between the transmitted signal and the demodulated signal for multiple pixels in a first region of the depth map using a standard formula; wherein the standard formula uses absolute values ​​for calculation.

[0114] The first depth module 300 is used to calculate the depth value of the pixel based on the phase offset.

[0115] The second depth module 400 is used to calculate the depth value of a pixel in a second region of the depth map using parallax based on the corresponding demodulated signal.

[0116] The merging module 500 is used to merge the depth values ​​of the first region and the second region to obtain the depth values ​​of the depth map.

[0117] This invention also provides an iTOF fast depth reconstruction device, including a processor and a memory storing executable instructions for the processor. The processor is configured to execute steps of a fast depth reconstruction method by executing the executable instructions.

[0118] As described above, this embodiment can provide initial depth values ​​for structured light images using TOF depth data, thereby significantly reducing the number of pixels that need to be calculated during matching, reducing computational load, and improving efficiency. This embodiment fully utilizes the advantages of simple and fast TOF data acquisition, making structured light computation more efficient and enabling faster reconstruction without consuming structured light computing power.

[0119] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0120] Figure 8 This is a schematic diagram of the structure of an iTOF rapid depth reconstruction device according to an embodiment of the present invention. See below for reference. Figure 8 To describe an electronic device 600 according to this embodiment of the present invention. Figure 8 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0121] like Figure 8 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0122] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the above-described section on a fast depth reconstruction method according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0123] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0124] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0125] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0126] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 8 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0127] This invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of a fast deep reconstruction method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the foregoing section on a fast deep reconstruction method according to various exemplary embodiments of the invention.

[0128] As shown above, this embodiment can provide initial depth values ​​for structured light images using TOF depth data, thereby significantly reducing the number of pixels that need to be calculated during matching, reducing computational load, and improving efficiency. This embodiment fully utilizes the advantages of simple and fast TOF data acquisition, making structured light computation more efficient and enabling faster reconstruction without consuming structured light computing power.

[0129] Figure 9 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 9As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0130] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0132] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] This embodiment can provide initial depth values ​​for structured light images using Time-of-Flight (TOF) depth data, thereby significantly reducing the number of pixels that need to be calculated during matching, reducing computational load, and improving efficiency. This embodiment fully utilizes the advantages of simple and fast TOF data acquisition, making structured light computation more efficient and enabling faster reconstruction without consuming structured light computing power.

[0134] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0135] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A fast depth reconstruction method, applicable to Time-of-Flight (TOF) technology running on chips with parallel computing capabilities, characterized in that, Includes the following steps: Step S1: Obtain data of the floodlight's transmitted signal and demodulated signal; wherein, one transmitted signal corresponds to at least two demodulated signals; Step S2: In the first region of the depth map, simultaneously calculate the phase offset between the transmitted signal and the demodulated signal for multiple pixels using a standard formula; wherein, the standard formula uses absolute values ​​for calculation; during calculation, the calculated data is multiplied by 2 to the power of n, converted to an integer, and then subjected to integer arithmetic; after the calculation, it is divided by 2 to the power of n to convert to a floating-point number; the formula is then used. Calculate the phase offset; where, , ;when When, cond is 1; when When cond is 0; Step S3: Calculate the depth value of the pixel based on the phase offset; Step S4: In the second region of the depth map, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal; Step S5: Combine the depth values ​​of the first region and the second region to obtain the depth value of the depth map.

2. The rapid depth reconstruction method according to claim 1, characterized in that, Based on computing power, multiple pixels are divided into groups, and the groups are processed sequentially.

3. The rapid depth reconstruction method according to claim 1, characterized in that, Step S2 includes: Step S21: Mark the transmitted signals in sequence; Step S22: Mark the received demodulated signals in sequence; Step S23: Identify the winding portion based on the waveform of the demodulated signal; Step S24: Recover the wound portion according to the order of the transmitted signal and the demodulated signal to obtain the recovered demodulated signal; Step S25: Calculate the relative bias based on the recovered demodulated signal and the transmitted signal.

4. The rapid depth reconstruction method according to claim 3, characterized in that, The image region is divided into several sub-regions, and a marker point is set for at least one of the sub-regions to determine whether the sub-region has a curled part.

5. The rapid depth reconstruction method according to claim 1, characterized in that, Step S2 includes: Step S21: Mark the transmitted signals in sequence; Step S22: Mark the received demodulated signals in sequence; Step S23: Identify the winding portion based on the waveform of the demodulated signal; Step S26: For the pixel that has been wound up, calculate the depth value of the pixel using parallax based on the corresponding demodulated signal.

6. An iTOF fast depth reconstruction system for implementing the fast depth reconstruction method according to any one of claims 1 to 5, characterized in that, include: An acquisition module is used to acquire data of the floodlight's transmitted signal and demodulated signal; wherein, one transmitted signal corresponds to at least two demodulated signals; The bias module is used to calculate the phase bias between the transmitted signal and the demodulated signal simultaneously for multiple pixels in a first region of the depth map using a standard formula; wherein the standard formula uses absolute values ​​for calculation. The first depth module is used to calculate the depth value of the pixel based on the phase offset; The second depth module is used to calculate the depth value of a pixel in the second region of the depth map using parallax based on the corresponding demodulated signal. The merging module is used to merge the depth values ​​of the first region and the second region to obtain the depth values ​​of the depth map.

7. An iTOF rapid depth reconstruction device, characterized in that, include: processor; A memory in which executable instructions of the processor are stored; The processor is configured to perform the steps of the fast depth reconstruction method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium for storing a program, characterized in that, When the program is executed, it implements the steps of the fast depth reconstruction method according to any one of claims 1 to 5.

Images