Time-of-flight phase unfolding of the graph
By generating a point matrix and using a lookup table to project noisy phase points onto a low-dimensional plane, and selecting the most likely phase order set, the problem of inaccurate phase unfolding caused by noise is solved, thus improving the distance determination accuracy and computational efficiency of the time-of-flight camera.
Patent Information
- Application Number
- CN202280034621.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-05-11
- Filing Date
- 2022-04-09
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-04-09
AI Technical Summary
In phase-based time-of-flight cameras, noise causes inaccurate phase unfolding calculations, leading to imprecise distance determination.
By generating a point matrix and a lookup table, the noise phase points are projected onto a low-dimensional plane. The most likely phase order set is selected using an independent term reference lookup table, and the distance value is determined.
It improves the computational efficiency and accuracy of phase expansion, reduces memory usage, and decreases computational complexity.
Smart Images

Figure CN117413202B_ABST
Abstract
Description
Background Technology
[0001] Depth sensing systems, such as time-of-flight (ToF) cameras, can be used to generate depth images of the environment, where each pixel represents the distance to a corresponding point in the environment. In ToF imaging, the distance to a point on the imaging surface in the environment is determined based on the length of a time interval during which amplitude-modulated light emitted by the depth imaging system travels to that point and then returns to the sensor of the depth imaging system. Summary of the Invention
[0002] Examples relating to determining the phase order of phase data in a depth imaging system are disclosed for unfolding the phase data to determine the depth value of a pixel. One example provides a computational system including a depth sensor comprising a plurality of pixels, each pixel configured to sense light, an illumination source configured to output light, a logic machine, and a memory storing instructions executable by the logic machine to control the illumination source to output amplitude-modulated light at K modulation frequencies, where K is an integer greater than or equal to 2. The instructions are further executable to perform K phase measurements for each of the plurality of pixels at each of the K modulation frequencies to form a set of noisy phase measurements, determining the position in a K-dimensional phase space where a projection line passing through the set of noisy phase measurements crosses a low-dimensional plane in the K-dimensional phase space, the projection line being parallel to a noise-free phase evolution line, comparing this position with a plurality of independent entries in a predetermined point matrix in the low-dimensional plane, each point representing a corresponding predetermined set of noisy phase orders with known noise, and selecting a set of independent entries. The instruction can further be executed to locate a corresponding set of noise-free phase sequences by using a set of independent item reference lookup tables, determine a distance value for the pixel based on the corresponding set of noise-free phase sequences, and output the distance value for the pixel.
[0003] This summary is provided to present, in a simplified form, the selection of concepts further described below in specific embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that address any or all of the shortcomings pointed out in any part of this disclosure. Attached Figure Description
[0004] Figure 1A-Figure 1B An example electronic device including a time-of-flight (ToF) camera is shown.
[0005] Figure 2 An aspect of an example ToF camera system is shown.
[0006] Figure 3 Example ToF image data for multiple K modulation frequencies is illustrated schematically.
[0007] Figure 4 The linear relationship between phase and distance is described for an example set of signals, including phase measurements, for each of three different modulation frequencies.
[0008] Figure 5 This is a flowchart illustrating an example method for generating matrices and lookup tables for performing phase expansion.
[0009] Figure 6 A graph of an example reference phase vector is shown, which describes the evolution of phase with distance for a set of ToF phase measurements at three different modulation frequencies, and an example sequence of total phase triplets, representing possible expanded solutions obtained using a stochastic process.
[0010] Figure 7 A set of example parallel lines is shown, each with a point corresponding to a different total phase triplet obtained through a random process, and the slope is parallel to the reference phase vector.
[0011] Figure 8 An example point matrix is shown, representing the intersections of a set of parallel lines with a plane perpendicular to the reference phase vector.
[0012] Figure 9 It shows the source of the Figure 7 The parallel lines are projected onto three example point matrices on the YZ, XZ, and XY zero planes, respectively.
[0013] Figure 10 This shows how the dimension is reduced by changing the coordinate system. Figure 8 A matrix of points.
[0014] Figure 11 It shows the intersection at Figure 10 The point matrix has two sets of parallel lines, each set of lines providing independent terms for the phase order.
[0015] Figure 12 This is an example lookup table showing the phase order points as matrix elements, where rows and columns represent the corresponding... Figure 10 The independent terms of the line.
[0016] Figure 13 This is a flowchart describing an example method for performing experimental noisy phase unrolling using a matrix or tensor that includes phase-order elements.
[0017] Figure 14 The example of calculating independent terms of the noise phase by determining the intersection of the lines in the grid of the projection points with the vertical axis is illustrated graphically.
[0018] Figures 15A-15BA flowchart illustrating an example method for determining the distance value of a ToF pixel based on multiple noise phase measurements is shown.
[0019] Figure 16 This is a flowchart illustrating an example method for generating a point matrix for use in phase unrolling.
[0020] Figure 17 A block diagram of an example computing system is shown. Detailed Implementation
[0021] Phase-based optical Time-of-Flight (ToF) imaging is a variant of ToF imaging where depth is calculated based on the phase shift of amplitude-modulated light reflected from the object. The phase shift is proportional to the distance to the object in modulus to the wavelength of the modulation frequency.
[0022] However, due to the periodicity of the modulated light, the total phase of the measurement repeats (or wraps) once every 2π. For example, given the measurement phase... Total phase is Where n k It is an integer. Because n k Phase-based ToF pixels cannot be directly measured, therefore the total phase and the actual distance associated with the measurement are ambiguous. Since the measured phase ranges from 0 to 2π, the unambiguous range of the determined distance is from 0 to...
[0023] To address this issue, two or more different modulation frequencies k can be used to increase the unambiguous range, allowing phase information to be "unwrapped" for accurate distance determination. Phase unwrapping, or "de-aliasing," is a method of eliminating phase-shift data ambiguity and identifying distance values by modulating scene illumination with light at multiple different frequencies, since the distance ambiguity is different for each illumination light frequency. This extends the unambiguous range to the distance corresponding to a common beat frequency across all used frequencies.
[0024] However, phase unrolling calculations may not be straightforward due to noise in the measured phase, because the number of phases n measured at the modulation frequency k is limited. k The number of phase wraps may be uncertain. At wrap points, noise can cause the number of phases to appear as if it has already been wrapped into the next phase sequence, which can lead to inaccurate distance determination.
[0025] Therefore, an example is disclosed relating phase unrolling to determine distance values for depth image data by generating a point matrix and lookup table for use in ToF phase unrolling, and utilizing the point matrix and lookup table to determine the phase order set and the associated depth values for noisy phase points. In short, K-dimensional phase points representing K noisy phase measurements are projected onto a low-dimensional plane and compared with a predetermined point matrix to identify independent terms. The most probable phase order set is then selected using an independent term reference lookup table. The distance values are then determined and output based on the most probable phase order set. The point matrix is generated by projecting multiple total phase points—computationally representing—the derived noisy phase order—on a low-dimensional plane, which will be discussed in more detail below. The associated lookup table is further generated by plotting the independent terms from the point matrix. Compared to graphical phase unrolling methods that use optimization to determine the most probable phase order set, the example disclosed herein achieves improved efficiency through fast computation and low memory usage.
[0026] Before discussing these examples in detail, Figure 1A Figure B illustrates various example electronic devices 100A-D that can employ a phase-based ToF depth imaging system. First, refer to... Figure 1A Device 100A is a smartphone that includes camera 102A. Device 100B is a personal computer that includes webcam 102B. Device 100C is a video game system that includes peripheral 3D camera 102C. Device 100D is a virtual reality headset that includes 3D camera 102D. Figure 1B An example use environment 110 is shown, including a camera 112 that can employ phase-based ToF depth imaging. The camera 112 transmits data to an Internet of Things (IoT) endpoint computing device 120 via a communication hub 116, which is also connected to other IoT devices, such as a thermostat 114. The camera 112 may include an optional controller 118. Devices 100A-D and camera 112 can employ phase-based optical ToF imaging in industrial and / or consumer applications, including equipment automation applications, gaming and virtual reality applications, and biometric and facial recognition applications. Such applications can be performed locally on the depth imaging system, on a remote computing system (e.g., IoT endpoint computing device 120), and / or on an edge device at the edge of the local network, bringing cloud computing capabilities closer to the local network. It is understood that phase-based ToF depth imaging can be used in any other suitable electronic device.
[0027] Figure 2A schematic description of an exemplary phase-based ToF depth imaging system 200, including a ToF camera 202, is shown. The ToF camera 202 includes a sensor array 204 comprising a plurality of ToF pixels 206, each ToF pixel 206 configured to acquire light samples to capture phase data, a controller 208, and an objective lens system 210. In some examples, the objective lens system 210 may be omitted. The objective lens system 210 is configured to focus an image of at least one surface 220 of an object 222 onto the sensor array 204. The controller 208 is configured to collect and process data from the ToF pixels 206 of the sensor array 204 to construct a depth image of the object 222 and / or the environment. The controller 208 may include executable instructions (e.g., software, firmware, and / or hardware) to perform phase unrolling, as described below. In some examples, the controller 208 may be implemented across one or more computing devices, as referenced below. Figure 17 Examples of hardware implementations of controller 208 and other computing devices configured to perform phase unrolling are described in more detail.
[0028] The depth imaging system 200 also includes a modulated light emitter 230 and an electronic shutter 232 for analog and / or digital modulation of the sensor array 204 to control the integration of light by the sensor array 204. The modulated light emitter 230 and the sensor array 204 can be controlled by a controller 208. The modulated light emitter 230 can be configured to emit electromagnetic radiation of any frequency detectable by the ToF pixel 206. For example, the modulated light emitter 230 may include an infrared (IR) light-emitting diode (LED), a laser diode (LD), or any other suitable light source. The amplitude-modulated light can be modulated sequentially or simultaneously at different frequencies; for example, the modulated waveform may include multiple frequencies.
[0029] Sensor array 204 is configured to image light from modulated light emitter 230 as reflected light leaving surface 220 and returning to the camera. Each ToF pixel 206 of sensor array 204 may include one or more pixel taps operable to integrate the reflected light signal at different time intervals, whereby the phase shift can be determined. For each modulation frequency, sensor array 204 is controlled to sample light at multiple phase angles of the amplitude-modulated light from the light source and determine a phase sample for each modulation frequency from the multiple light samples used for the modulation frequency. The phase sample can then be unfolded, as described in the examples of this disclosure, to obtain a depth value for each pixel.
[0030] Figure 3An example ToF image data 300 for multiple K modulation frequencies is schematically illustrated. Data 300 represents data that can be acquired by a depth imaging system 200. In the example shown, the depth data comprises a data array M×N for each of the K modulation frequencies, resulting in an M×N grid of data 302a-c, where each pixel 304 in each grid represents a measurement acquired at the corresponding illumination light modulation frequency k among the K modulation frequencies. For example, the experimental signal collected by pixel 304 at (m,n) Represented as
[0031] in These are the phases, {m∈1,2…,M}, {n∈1,2…,N}, and {k∈1,2…,K}. A wavy accent on a variable indicates that the variable has been obtained experimentally and / or calculated, while no wavy accent indicates a variable corresponding to a noise-free condition.
[0032] Since the examples presented operate on a pixel-by-pixel basis, the symbols (m,n) will be omitted for simplicity. The terms "distance" and "depth value" can be used interchangeably when referring to the output value of a pixel. Although various examples are described in the context of three optical modulation frequencies, any number of frequencies K≥2 can be used.
[0033] Complex signal The phase can be calculated as:
[0034] in It is the imaginary part of the signal collected at frequency k, and This is the real part of the collected signal. The measured phase is used to calculate the depth value associated with the pixel. However, as mentioned above, in phase-based ToF imaging, the modulation frequency imposes a limitation on the measurable distance (called the unambiguous range). On the sensor, the relationship between the total phase φ of the amplitude-modulated light and the travel distance is proportional to the modulation frequency of the light, as shown in Equation 3:
[0035] Where d is the distance, f is the modulation frequency, and c is the speed of light. As mentioned above, due to the periodicity of the modulated light, the total phase of the measurement repeats (or wraps around) once every 2π. For example, given the phase of the measurement... Total phase is Where n k It is an integer. Because n kSince phase-based ToF pixels cannot be directly measured, the total phase is blurred. Because the measured phase ranges from 0 to 2π, the unambiguous range in the determined distance is from 0 to... Less than The distance can be determined without ambiguity, while distances outside this range cannot be determined without additional information. Therefore, higher frequencies will have shorter unambiguous ranges.
[0036] Figure 4 The linear relationship between phase and distance is described for three frequencies of amplitude-modulated light. For a given frequency, the measured phase “wraps” around to 0 as the total phase increases beyond 2π. For a single frequency f, the range is unambiguous. This is the distance to the first wrapping point. The phase order of the total phase is the number of times the phase wraps around the point. For example, line 402 corresponds to the phase of frequency f1 in the 0th phase order. Similarly, lines 404 and 406 correspond to the first and second phase orders of frequency f1.
[0037] The modulation frequency of the illumination light imposes a linear change on the accuracy of the phase measurement. This linear change in phase estimation leads to a frequency-based change in the depth value. Rearranging Equation 3 above to solve the distance problem results in:
[0038] in It is the modulation frequency f k The measured noise phase, and n k It's the phase order. Here, It represents the distance under ideal noise-free conditions and / or the most probable distance, which may be two or more noise sources. The weighted average of the values is as follows. As shown in Equation 4, the distance... The error depends on the measured phase. The error is inversely proportional to the frequency. Therefore, higher frequencies generally correspond to smaller errors. This can be seen in... Figure 4 As seen in the text, because when the slope is relatively steep (e.g., f1 in...), Figure 4 In the middle), the phase error corresponds to a relatively small distance error. Therefore, although higher frequencies are associated with lower frequencies (e.g., f3 at...), the phase error corresponds to a relatively small distance error. Figure 4 Compared to the previous method, the unambiguous range was reduced, but the higher frequency was also associated with larger errors in the depth values.
[0039] Therefore, a set of modulation frequencies k ≥ 2 can be used to increase the unambiguous range, allowing phase information to be unfolded for accurate distance determination. Phase unfolding, also known as "de-aliasing," is a method to eliminate phase shift data ambiguity and identify the correct distance value by using amplitude-modulated light of multiple different frequencies to illuminate the scene, since the distance ambiguity is different for each illumination light frequency. For example, in a multi-frequency approach, the amplitude-modulated light can include multiple frequencies. The waveform. The set of frequencies includes frequencies selected at different locations within an unambiguous range to enclose (see...). Figure 4 The unambiguous range is from Figure 4 The distance from zero extends to the point on the far right of the plotted graph, and all three frequencies are wrapped in a common location.
[0040] In the absence of noise, ideally, for each frequency f k (See Equation 4), the determined distances are the same. For example, in Figure 4 In the example scenario, distance 410 intersects the tenth segment or ninth wrapper at frequency f1 (at intersection point 412). Therefore, the phase order for this frequency is n1 = 9. Similarly, n2 = 6, n3 = 1, and the set of phase orders corresponding to distance 410 is... However, as mentioned above, accuracy varies with noise. Therefore, the disclosed example finds the most probable set of phase orders for determining the phase measurement in noisy conditions. (See again...) Figure 4 The phase of each optical modulation frequency evolves linearly with distance, and the slope of each line is proportional to the corresponding frequency. Therefore, in the first example implementation, the phase evolution is represented as a straight line in K-dimensional space:
[0041] Where t is the parameter corresponding to the distance. It's the slope. It is the initial phase, and This is the starting point. The line in Equation 5 corresponds to the phase evolution line of the ideal phase under noise-free conditions. Equation 3 indicates that the phase starts from 0. Based on this assumption, And the equation is simplified to:
[0042] Where R is the line and t is the distance parameter.
[0043] In an ideal, error-free scenario, the combination of the three measured phases will correspond to a unique depth value within the unambiguous range. However, due to noise, the number of phases n for measurements performed at the modulation frequency k is limited. kThe number of phase wraps may be uncertain. The most probable set of phase orders will be obtained as described in more detail below. The unfolding process can be based on comparing the projected noisy phase points with a predetermined point matrix in a low-dimensional projection plane. The noisy phase points in the K-dimensional phase space are projected along a direction parallel to a reference phase vector, which is a line representing the evolution of phase with distance. In one example, the noisy phase points are projected onto a plane perpendicular to the reference phase vector. In another example, the noisy phase points are projected onto a zero plane in K-dimensional space (e.g., a Cartesian plane with coordinates set to zero). The most probable set of phase orders is determined by comparing the projected noisy phase points with independent terms plotted through the point matrix, and by selecting the most probable set of phase orders using an independent term reference lookup table.
[0044] Figure 5 This is a flowchart illustrating an example method 500 for generating a point matrix and associated lookup table for performing phase unrolling. Below is a comparison with... Figures 6-12 The individual steps outlined in Method 500 are discussed in more detail. The example is depicted in three-dimensional phase space against the backdrop of three optical modulation frequencies. However, the method described herein can be applied to any suitable number of modulation frequencies.
[0045] Method 500 includes, at 502, generating a total phase sequence set representing a set of phase sequences that may be generated in the presence of noise. Each phase sequence set includes K total phases corresponding to K frequencies of the amplitude-modulated light. In some examples, at 504, a random process is used to generate points from a reference phase vector. In other examples, analytical methods or error propagation methods may be used. In examples where the noise probability is known, the total phase sequence set can be computed based on the probability. In some examples, at 506, the reference phase vector can be finely sampled by generating noisy phase points from points along the reference phase vector, as described in more detail below. Method 500 also includes, at 508, generating a set of parallel lines in a K-dimensional space, the K-dimensional space being run through the points associated with the total phase sequence set generated at 502.
[0046] At 510, method 500 includes locating the intersection of the parallel lines and the low-dimensional plane to form a point matrix. In some examples, at 512, the low-dimensional plane is perpendicular to the reference phase vector. In other examples, at 514, the low-dimensional plane is the zero plane. With the low-dimensional plane perpendicular to the reference phase vector, at 516, a rotation can be performed on the point matrix to reduce the dimension. Alternatively, a change in the coordinate system can be applied to reduce the dimension.
[0047] Method 500 also includes, at 518, drawing lines through a point matrix to group the points. A group of points comprises one or more points located above or near a line in a set of parallel lines. As discussed in more detail below, each set of parallel lines in the low-dimensional plane corresponds to a different independent term. Each independent term references a line within that set of parallel lines—and the corresponding group of points. Each point is located at the intersection of the lines, and thus each point is referenced by an independent term. Since each point is associated with a phase order set, at 520, method 500 includes generating a table referencing the phase order set through the independent terms, and at 522, storing the point matrix and lookup table, which can then be used at 524 to expand the noisy phase data.
[0048] As mentioned above, stochastic processes can be used to compute the data most likely to be generated by the data. The resulting phase order set. In one example employing a random method, the range of interest is finely sampled as described below, and the phase vectors... This is provided via Equation 3. The range of interest can include any subset up to and including the complete unambiguous range. Once the reference phase vector is... Random noise, compatible with the noise generated by the image sensor, is added to generate a noise phase vector. The generated noise can include anisotropic noise (frequency-dependent variance) or isotropic noise (frequency-independent variance). Phase order vector Calculated as:
[0049] The `floor` function outputs the largest integer less than or equal to a certain value. This process can be repeated an appropriate number of times to check possible combinations of phase orders. A unique phase order can be a set of phase orders. And were chosen.
[0050] Figure 6 A graph 600 in 3D space for the phase evolution line according to Equation 6 is shown, along with an example sequence of 59 total phase triplets generated by this random method. In this example, line 602 has slopes corresponding to three modulation frequencies: 54, 189, and 198 MHz. Other examples may use different frequencies and / or different numbers of frequencies (e.g., K-dimensional space, K>3). The total phase for each modulation frequency is represented on the x, y, and z axes, while the length of the line (from the origin) corresponds to the distance parameter (see Equation 6).
[0051] As mentioned above, Figure 6A sequence of total phase triples is generated by finely sampling a reference phase vector. In some examples, the sampling resolution is less than the theoretical resolution provided by the frequency set. For example, let the frequency vector... like Figure 6 As shown, and the collected phase vectors The collected phase vectors can be decomposed for an unambiguous range into Where 'a' is the greatest common divisor (gcd) of the frequency vectors, and also the difference frequency that determines the unambiguous range. Furthermore, They are coprime frequency vectors. In this example, a = 9 and exist Figure 6 In the example shown, the unambiguous range is sampled once every 1 millimeter and has a variance of 0.01 rad. 2 Gaussian isotropic noise is added. Within each distance interval, 10,000 noise phase vectors are generated, from which the phase order vectors are calculated, producing 59 vectors. Other examples can generate phase vector sets using different values.
[0052] Next, for each line, using a set of noise phase sequences as points and phase evolution lines as slopes, many parallel lines are generated. A set of L parallel lines... Described as:
[0053] in Is and point The corresponding line, t L It is a distance parameter, and It is a frequency vector. Figure 7 An example of a sequence of parallel lines 700 parallel to the phase evolution line 702 is depicted. The parallel lines correspond to... Figure 6 The subset of the total phase triplet depicted is omitted for clarity, as some points and lines are omitted. For example, point 704 is used to generate line 706 parallel to phase evolution line 702. Figure 7 In the three-dimensional example depicted,
[0054] A point matrix is obtained by extending parallel lines to intersect a low-dimensional plane, onto which the total phase points are projected. Each point in the point matrix corresponds to the intersection of the line and the low-dimensional plane. Furthermore, each point in the matrix represents a phase order set (i.e., the phase order set associated with the line). Any suitable low-dimensional plane can be used for the projection points. In some examples, a plane perpendicular to the phase vector is used (e.g., ...). Figure 8 In such examples, the plane can then be rotated to reduce its dimension. In other examples, a coordinate plane or a zero plane is used (e.g., Figure 11 If the set of parallel lines When projected onto a plane perpendicular to the phase evolution line R, the resulting point matrix can include a lattice, where the spacing between the points depends on the frequency.
[0055] In some examples, the normal plane P can be chosen to pass through the origin, which simplifies the calculation. With this choice, the normal plane P is defined as:
[0056] in And among them It is the phase vector. (The line...) Where l∈{1,2,…,L}, the distance propagated to plane P is calculated as a line. The intersection point with plane P. In three-dimensional space, this results in a deterministic system with 3 equations and 3 variables. The solution can be obtained in vector form:
[0057] Where t l Phase point in phase space Along the route The distance between the projection planes, and It is the transpose of the reference phase vector. The point projected onto plane P is calculated as follows:
[0058] in It is the position of the projection point in phase space.
[0059] Figure 8 An example is shown where the total phase triplet is projected along a line parallel to the reference phase vector to intersect a plane perpendicular to the reference phase vector. The location of the intersection between the parallel line and the normal plane gives a point matrix 800. In this example, the point matrix lies in a two-dimensional plane in three-dimensional space. The origin 802 corresponds to the projection of the noise-free phase evolution line. In this example, the normal plane passes through the origin 802. For example, the location of the points can be calculated using Equation 11 above.
[0060] In some examples, the set of parallel lines is projected onto a zero plane, which is a low-dimensional plane with zero values for a given axis. For example, in 3-dimensional phase space, the zero plane could be a coordinate plane, such as the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane. Figure 9Three figures, 902, 904, and 906, depict projections onto the zero plane. Figure 902 shows the point matrix produced by projecting a set of parallel lines onto the YZ Cartesian plane (i.e., the zero plane is set to X=0). Similarly, figure 904 shows the point matrix after projection onto the XZ Cartesian plane (i.e., Y=0), and figure 906 shows the point matrix after projection onto the XY Cartesian plane (i.e., Z=0).
[0061] The zero plane can be defined as P o ≡r o =0 with o={1,2,3} Equation 12
[0062] Where r o =0 indicates which coordinate is set to zero. For example, graph 902 corresponds to r1=0. (The last sentence appears to be incomplete and possibly refers to a line or plot.) Where l∈{1,2,…,L}, the distance propagated to plane P is calculated as a line. and one of the planes P o The intersection point (composed of two perpendicular lines). Since the three-dimensional case results in a definite system with three equations and three variables, the solution can be obtained in vector form as follows:
[0063] in It is the propagation distance along line l in dimension o. Therefore, it is projected onto plane P. o The point on is determined as:
[0064] in In the zero plane P o The position of the total phase point of the projection.
[0065] As mentioned above, in some examples, the point matrix can be rotated to reduce its dimensionality. Additionally or alternatively, dimensionality reduction can include changes in the coordinate system. For example, since the point matrix 800 comprises three-dimensional points in a tilted two-dimensional plane, rotation can be applied to reduce the points to two-dimensional points. Therefore, dimensionality reduction can be achieved by changing the coordinate system... This is achieved by multiplying by a three-dimensional rotation matrix, which is a normalized frequency vector. It consists of a coordinate system given by two perpendicular vectors. The perpendicular vectors are calculated by vector product: It is a univariate vector, k∈{1,2,3}, and A rotation matrix can be represented as a matrix ordered in either column or vector order. Given the rotation matrix, the rotated points are then solved:
[0066] in This represents the position of the intersection point after rotation. The result of the rotation is the projected point of the rotation. Univariate vector The corresponding dimension is 0.
[0067] Figure 10 The diagram shows a two-dimensional point matrix 1000 obtained by applying rotation to point matrix 800. Here, the dimension corresponding to the unary vector is discarded after rotation to reduce the points to two-dimensional points. In other examples, for instance... Figure 9 In the example shown, the rotation matrix can be omitted.
[0068] After obtaining the point matrix, the independent item points are grouped by plotting the matrix. As mentioned above, each independent item is grouped with reference to the phase order of the projection of the points along a parallel line in the low-dimensional plane. Figure 10 As shown, the phase order points of the projection are distributed along privileged directions, for example, aligned with the lattice vectors of the lattice forming the points. Therefore, two or more sets of parallel lines can be drawn to group points, such as... Figure 11 As shown. Figure 11 The 10 dashed lines shown correspond to the first independent term, and the 11 solid lines correspond to the second independent term. Figure 11 In the example shown, the independent items are not unique because they are determined by the orientation of the lattice. However, since the points are located at intersections, each point can be referenced by a unique combination of independent items.
[0069] The slope of a parallel line drawn from a point matrix can be obtained using any suitable method. For example, a random method can be used to find the most frequent slope among the projected points. Given two points in a point matrix, the slope is the ratio between the vertical and horizontal displacements. Similarly, for a set of points... The slope is
[0070] Where α l It's the slope. These are the coordinates in the first direction (e.g., perpendicular). α is the coordinate in the second direction (e.g., horizontal), and t indicates the matrix transpose. l It is a symmetric matrix where the diagonal elements are infinite because the divisor of these elements is zero. A symmetric matrix with diagonal elements can be created by applying the arctangent operation. θ l =arctan(α) l Equation 17
[0071] Where θ l It is a set of angles. Then, from the pattern of off-diagonal elements, such as an upper triangular matrix, find the two most frequent angles. In some examples, the angles can be rounded to make the process less sensitive to computational precision. In other examples, any other suitable method can be used to obtain the two most frequent angles (e.g., binning). Furthermore, the most frequent slope can be obtained from the angles by applying a tangent operation. Once the most frequent slope is found... Where k∈{1,2}, the independent terms are calculated as follows:
[0072] in It is an independent item.
[0073] For example, Figure 11 The line can be determined by finding the two most common slopes in the point matrix 1000 and then drawing the line through these points. In the example where the total phase point is projected onto the zero plane, the line can be drawn without rotation through the point matrix in the zero plane. Figure 902 shows the line corresponding to the two most common slopes drawn through the point matrix in the YZ Cartesian plane. Similarly, Figures 904 and 906 show the lines drawn through the point matrices in the XZ and XY Cartesian planes, respectively.
[0074] Assumption Indicates targeting those with and A unique set of independent terms in direction k, sorted in ascending order, forming a table with I rows and J columns. Furthermore, since the slopes are parallel, the distances between independent terms are the same, and there are two differences between consecutive independent terms. Next, these terms are used to generate a matrix with dimension (I,J):
[0075] Where Δ1 is for The difference between consecutive independent terms, Δ2 is for The difference between consecutive independent terms. The result of this operation is the elements of a matrix related to the phase order.
[0076] Figure 12 It shows from Figure 11 The point matrix and independent terms generate an example lookup table 1200. Each non-zero element of lookup table 1200 references a different phase order set. The position of the phase order set L can be determined using equation 19. In the example shown, the phase order sets are numbered. Here, the numbering pairs are obtained as described above to obtain a set The phase order set within the table is indexed. In other examples, each table element may include a phase order set instead of an index number. The lookup of element 0 in table 1200 corresponds to the position in a low-dimensional plane of the unprojected phase order (those infeasible due to random processes).
[0077] The line corresponding to the independent terms is in Figure 11 The points corresponding to the projection positions of the phase order set referenced by the table elements intersect at. For example, if the first independent item is 3 and the second independent item is 4, then the table element references phase order set 48, as in this example, where the first independent item line 3 and the second independent item line 4 intersect at phase order set 48. Next, the actual phase order set is looked up using index 48, for example, by... Search The dimension of lookup table 1200 corresponds to the number of unique independent terms found using equation 18. In other examples, the lookup table can have different sizes and can have different numbers of non-zero elements.
[0078] Once the phase sequence set The point matrix and lookup table are calculated, and they can be stored and used for phase unrolling. In some examples, the rotation matrix Rot, the line slope a... k Independent items The distance between lines and / or lines can also be stored.
[0079] Figure 13 This is a flowchart illustrating an example method 1300 for determining the most probable set of phase order and depth values for noise phase points using a point matrix and a lookup table. At 1302, the method includes receiving noise phase data comprising K noise phase measurements. As described above, the K noise phase measurements can be represented by noise phase points in a K-dimensional space. At 1304, the method includes projecting the noise phase points in the K-dimensional space along projection lines parallel to a reference phase vector to intersect the low-dimensional plane at the specified location.
[0080] In some examples, at position 1306, the low-dimensional plane is perpendicular to the reference phase vector. For example, the distance from the noise phase point to the low-dimensional plane P can be calculated as:
[0081] in It's distance. It is the noise phase vector of the noise phase point, and This is the transpose of the reference phase vector. Here, the wavy accent symbol indicates noise measurement. In the case of an M×N pixel array, the image array... The distance between noise phase points can be calculated as follows:
[0082] in This represents the distance to the noise phase point at pixel (m,n) in the image array. Therefore, the location of the projected noise point is calculated as follows:
[0083] in It is the position within the low-dimensional plane P.
[0084] In such an example, the method may also include applying rotations to the projected points to reduce dimensionality. In this example, the applied rotation is the one applied during the generation of the point matrix. For example, dimensionality can be reduced by using a rotation matrix:
[0085] Rot is the rotation matrix that is computed and applied during the generation of the point matrix (e.g., Equation 12). As mentioned above, the matrix Rot can be stored and used during phase unrolling.
[0086] Continuing with method 1300, in other examples, at 1310, the low-dimensional plane is the zero plane. In some such examples, when using 3 amplitude modulation frequencies (i.e., K = 3), the zero plane can be the XY, XZ, or YZ Cartesian plane.
[0087] Method 1300 also includes, at 1312, comparing the projected points with a predetermined point matrix and determining independent terms. Figure 14 A graphical representation illustrates an example of determining independent terms by comparing position 1400 with a predetermined point matrix 1401. In the example shown, position 1400 represents the intersection of the projection line and the low-dimensional plane. The predetermined point matrix 1401 also lies in the low-dimensional plane. The independent terms for the noise phase points of the projection are calculated as the intersections of the lines with the vertical axis:
[0088] in The horizontal component of the phase of the phase point representing the projection of pixel (m,n) is used. It is the vertical component of the position. It is the slope in direction k, and It is an independent term in direction k.
[0089] like Figure 14As seen, position 1400 is projected onto vertical line 1402 in two directions. In the first direction 1404, parallel to the dashed line, position 1400 is projected to point 1406, intersecting vertical line 1402. Similarly, position 1400 is projected along the second direction 1408, parallel to the solid line, to point 1410, intersecting vertical line 1402. In the example shown, the first direction can produce a first independent item 7 because point 1406 is closest to the 7th dashed line counting from the top of the figure. Similarly, the second direction can produce a second independent item 5 because point 1410 is closest to the 5th solid line counting from the top. In other examples, the parallel lines can be ordered or numbered in reverse order.
[0090] In other examples, any other suitable method can be used to find the independent terms. For example, the dot product between the 2D position 1400 and the vector orthogonal to the direction 1404 (or 1408) can be used to determine the first (or second) independent term. In another example, given an appropriate basis set, a change-of-basis matrix operation can be performed to project the noisy phase points onto a plane perpendicular to the reference phase vector and obtain the independent terms.
[0091] Refer again Figure 13 At 1314, the method involves projecting phase points onto a lookup table and using independent terms to determine the most likely set of phase orders. Once all independent terms for the projections are found, this value is subtracted from the previously calculated minimum independent value and divided by the distance between the parallel lines. The element (i,j) of the lookup table is then rounded. For example, when the point matrix is two-dimensional and has two independent terms, this element is calculated as:
[0092] Where Δ1 is the distance between the lines in the first group of parallel lines, and Δ2 is the distance between the lines in the second group of parallel lines. It is the lowest independent value of the first independent term, and This is the lowest independent value of the second independent term. The values i and j are then used to reference the lookup table (e.g., lookup table 1200) and retrieve the element (i, j), as described above regarding... Figure 12 The process is as described above. Then, if the lookup table contains an index, the location of the index is accessed to obtain the phase order set. In other examples, the lookup table may contain the phase order set itself, rather than an index to the storage location of the corresponding phase order set. In other examples, the lookup table may include a higher-dimensional table where each element is referenced by three or more independent entries.
[0093] After selecting the most probable phase order set, method 1300 further includes, at 1316, calculating the depth (or radial distance) value for the pixel. The most probable phase order set is added to the wrapped phases, and the distance is calculated as a weighted sum, which reduces jitter and improves accuracy. For the case of three modulation frequencies, the depth calculation is...
[0094] Where d is the depth value. It is the phase enveloping the frequency o. It is the most likely phase order at frequency o, and ξ o It is the normalization coefficient.
[0095] Figures 15A-15B A flowchart of an example method 1500 for determining the distance value of a ToF pixel is shown. Method 1500 includes, at 1502, performing K phase measurements, each phase measurement being performed at a different modulation frequency among K modulation frequencies, to form a set of noise phase measurements.
[0096] At 1504, method 1500 further includes determining the position in the K-dimensional phase space where a projection line passing through a set of noisy phase measurements intersects a low-dimensional plane in the K-dimensional phase space, the projection line being parallel to the noiseless phase evolution line. In some examples, at 1506, the projection line and the noiseless phase evolution line are perpendicular to the low-dimensional plane. In some such examples, at 1508, method 1500 includes performing a rotation on the predetermined point matrix to reduce the dimension. In other examples, at 1510, the low-dimensional plane is the zero plane. In some such examples, at 1512, K = 3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane.
[0097] continue Figure 15B Method 1500 further includes, at 1514, comparing the position with a plurality of independent terms in a predetermined point matrix in a low-dimensional plane, each point representing a corresponding predetermined set of noise phase orders with known noise, and selecting a set of independent terms. In some examples, at 1516, the predetermined point matrix in the low-dimensional plane comprises a lattice formed by projecting a plurality of total phase points onto the low-dimensional plane in a direction parallel to a noise-free phase evolution line, each of the plurality of total phase points representing a set of total phase orders in a K-dimensional phase space. In some examples, at 1518, the method includes determining each of the plurality of independent terms by projecting the position along a direction parallel to a set of lattice lines intersecting with an axis in the low-dimensional plane, the set of lattice lines corresponding to the independent terms.
[0098] Method 1500 further includes, at 1520, locating a corresponding set of noise-free phase sequences using the set of independent item reference lookup tables. As described above, in some examples, the elements of the lookup table can provide index numbers of a set of phase sequences within a sequence of phase sequences. Method 1500 further includes, at 1522, determining a distance value for the pixel based on the corresponding set of noise-free phase sequences, and at 1524, outputting the distance value for the pixel.
[0099] Figure 16 A flowchart of an example method 1600 for generating a point matrix and lookup table for use in phase unrolling is shown. At 1602, method 1600 includes generating multiple sets of phase orders from a reference phase vector in K-dimensional space via a stochastic process, the reference phase vector describing the phase evolution of K frequencies of amplitude-modulated light with distance. At 1604, the method includes generating multiple corresponding parallel lines in K-dimensional space based on the multiple sets of phase orders, each parallel line having points corresponding to the phase order set and a slope parallel to the reference phase vector.
[0100] Method 1600 further includes, at 1606, projecting multiple parallel lines onto a low-dimensional plane in K-dimensional space, thereby forming a point matrix based on the intersections of the parallel lines and the low-dimensional plane. In some examples, at 1608, the low-dimensional plane is perpendicular to the reference phase vector. In some such examples, at 1610, the method includes rotating the point matrix to reduce the dimension. In other examples, at 1612, the low-dimensional plane is a zero plane, and in some such examples, at 1614, K = 3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane.
[0101] Method 1600 further includes forming a point group at 1616 by drawing independent items from a point matrix. In some examples, at 1618, drawing independent items from a point matrix includes drawing a first set of parallel lines corresponding to the first independent item and drawing a second set of parallel lines corresponding to the second independent item, such that each point in the point matrix is located at the intersection of the lines in the first set of parallel lines and the lines in the second set of parallel lines.
[0102] Method 1600 also includes, at 1620, generating a lookup table that identifies a set of phase orders based on two or more independent items. At 1622, the method includes storing the point matrix and the lookup table for use in phase unrolling. In some examples, at 1624, multiple sets of phase orders, rotation matrices, multiple slopes, and / or values indicating distances between independent items are stored together with the point matrix and the lookup table.
[0103] In some examples, at 1626, method 1600 also includes performing phase unrolling using the stored point matrix and lookup table to calculate the depth value of the pixel. Therefore, method 1600 can utilize either method 1300 or method 1500.
[0104] Compared to other phase unrolling methods, the examples disclosed herein achieve improved efficiency through fast computation and low memory usage. Furthermore, the disclosed examples can leverage fast on-processor operations (such as multiplication and sorting) while avoiding interpolation. Therefore, these examples involve low-level computations that can be performed without high-power graphics processing units (GPUs).
[0105] In some embodiments, the methods and processes described herein can be tied to a computing system with one or more computing devices. In particular, such methods and processes can be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.
[0106] Figure 17 A non-limiting embodiment of a computing system 1700, which can implement one or more of the methods and processes described above, is illustrated schematically. The computing system 1700 is shown in a simplified form. The computing system 1700 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.
[0107] The computing system 1700 includes a logic machine 1702 and a memory machine 1704. The computing system 1700 may optionally include a display subsystem 1706, an input subsystem 1708, a communication subsystem 1710, and / or... Figure 17 Other components not shown.
[0108] Logic machine 1702 includes one or more physical devices configured to execute instructions. For example, a logic machine may be configured to execute instructions as part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transition the state of one or more components, achieve technical effects, or otherwise achieve desired results.
[0109] A logic machine may include one or more processors configured to execute software instructions. Additionally or alternatively, a logic machine may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The logic machine's processor may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logic machine may optionally be distributed across two or more independent devices, which may be located remotely and / or configured for coordinated processing. Aspects of the logic machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud computing configuration.
[0110] Storage machine 1704 includes one or more physical devices configured to store logically machine-executable instructions to implement the methods and processes described herein. When these methods and processes are implemented, the state of storage machine 1704 can be transformed—for example, to store different data.
[0111] Storage machine 1704 may include removable and / or built-in devices. Storage machine 1704 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic memory (e.g., hard disk drive, floppy disk drive, magnetic tape drive, MRAM, etc.). Storage machine 1704 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices.
[0112] It should be understood that the storage machine 1704 includes one or more physical devices. However, aspects of the instructions described herein can be propagated via a communication medium (e.g., electromagnetic signals, optical signals, etc.) that is not held by the physical device for a finite duration.
[0113] Various aspects of the logic machine 1702 and the memory machine 1704 can be integrated together into one or more hardware logic components. For example, such hardware logic components may include field-programmable gate arrays (FPGAs), programmable application-specific integrated circuits (PASICs / ASICs), programmable application-specific standard products (PSSPs / ASSPs), system-on-a-chip (SoCs), and complex programmable logic devices (CPLDs).
[0114] The terms "module," "program," and "engine" can be used to describe one aspect of the computing system 1700 that performs a specific function. In some cases, a module, program, or engine may be instantiated via a logic machine 1702 that executes instructions held by a memory machine 1704. It will be understood that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program, and / or engine may be instantiated from different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" can include a single or a group of executable files, data files, libraries, drivers, scripts, database records, etc.
[0115] It should be understood that, as used in this disclosure, a "service" is an application that can be executed across multiple user sessions. A service can be used for one or more system components, programs, and / or other services. In some implementations, a service can run on one or more server computing devices.
[0116] When included, display subsystem 1706 can be used to present a visual representation of data held by storage machine 1704. This visual representation may take the form of a graphical user interface (GUI). Just as the methods and processes described in this disclosure change the data held by the storage machine, thereby changing the state of the storage machine, the state of display subsystem 1706 can also be changed to visually represent the changes in the underlying data. Display subsystem 1706 may include one or more display devices utilizing virtually any type of technology. Such display devices may be integrated with logic machine 1702 and / or storage machine 1704 in a shared enclosure, or such display devices may be peripheral display devices.
[0117] When included, the input subsystem 1708 may include one or more user input devices, such as a keyboard, mouse, touchscreen, or game controller, or an interface thereto. In some embodiments, the input subsystem may include selected Natural User Input (NUI) components or an interface thereto. Such components may be integrated or peripheral, and the transduction and / or processing of input actions may be performed on-machine or off-machine. Example NUI components may include a microphone for speech and / or speech recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; and an electric field sensing component for assessing brain activity.
[0118] When included, the communication subsystem 1710 can be configured to communicatively couple the computing system 1700 to one or more other computing devices. The communication subsystem 1710 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem may allow the computing system 1700 to send and / or receive messages to other devices via a network such as the Internet.
[0119] Another example provides a computational system including a depth sensor comprising multiple pixels, each configured to sense light; an illumination source configured to output light; a logic machine; and a memory holding instructions executable by the logic machine to control the illumination source to output amplitude-modulated light at K modulation frequencies, where K is an integer greater than or equal to 2, and for each of the multiple pixels: K phase measurements are performed at each of the K modulation frequencies to form a set of noisy phase measurements; the position of a projection line in the K-dimensional phase space through the set of noisy phase measurements is determined, the projection line being parallel to a noiseless phase evolution line; the position is compared with multiple independent entries in a predetermined point matrix in the low-dimensional plane, each point representing a corresponding predetermined set of noisy phase sequences with known noise, and a set of independent entries is selected; the corresponding set of noiseless phase sequences is located using a reference lookup table of the set of independent entries; a distance value is determined for the pixel based on the corresponding set of noiseless phase sequences; and the distance value for the pixel is output. In some such examples, the projection line and the noiseless phase evolution line are perpendicular to the low-dimensional plane. Alternatively or alternatively, in some examples, the instructions are further executable to perform rotations on a predetermined point matrix to reduce the dimension. Alternatively or alternatively, in some examples, the low-dimensional plane includes a zero plane. Alternatively or alternatively, in some examples, K = 3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane. Alternatively or alternatively, in some examples, the predetermined point matrix in the low-dimensional plane includes a lattice formed by projecting a plurality of total phase points onto the low-dimensional plane in a direction parallel to the noiseless phase evolution line, each of the plurality of total phase points representing a set of total phase orderings in the K-dimensional phase space. Alternatively or alternatively, in some examples, the instructions are executable to determine each of a plurality of independent terms by projecting the location along a direction parallel to a set of lattice lines intersecting with an axis in the low-dimensional plane, the set of lattice lines corresponding to the independent terms.
[0120] Another example provides a method for generating a point matrix and a lookup table for use in phase unrolling, the method comprising: generating multiple phase order sets from a reference phase vector in a K-dimensional space via a random process, the reference phase vector describing the phase evolution of K frequencies of amplitude-modulated light with distance; generating multiple parallel lines in the K-dimensional space based on the multiple phase order sets, each parallel line having a point corresponding to a phase order set and a slope parallel to the reference phase vector; projecting the multiple parallel lines onto a low-dimensional plane in the K-dimensional space to form a point matrix from the intersections of the parallel lines and the low-dimensional plane; plotting independent terms from the point matrix; generating a lookup table that identifies the phase order sets based on two or more independent terms; and storing the point matrix and the lookup table for use in phase unrolling. In some such examples, the method further includes performing phase unrolling by receiving noisy phase points, which include K phase measurements from time-of-flight pixels; determining the position in which a projection line passing through the noisy phase points in the K-dimensional phase space intersects a low-dimensional plane in the K-dimensional phase space, the projection line being parallel to a reference phase vector; comparing the position with independent terms plotted by a point matrix and selecting a set of independent terms; determining a phase order set using a set of independent term reference lookup tables; determining a distance value based on the phase order set; and outputting the distance value for the time-of-flight pixels. Alternatively, in some examples, comparing the position with independent terms includes determining a first independent term by projecting the position along a first direction to intersect an axis, and determining a second independent term by projecting the position along a second direction to intersect an axis. Alternatively, in some examples, the low-dimensional plane is perpendicular to the reference phase vector. Alternatively, in some examples, the method also includes rotating the point matrix to reduce the dimensionality. Alternatively, in some examples, the low-dimensional plane is a zero plane. Alternatively or additionally, in some examples, K = 3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane. Additionally or additionally, in some examples, drawing independent terms using a point matrix includes drawing a first set of parallel lines corresponding to the first independent term and drawing a second set of parallel lines corresponding to the second independent term, such that each point in the point matrix lies at the intersection of the lines in the first set of parallel lines and the lines in the second set of parallel lines. Additionally or additionally, in some examples, the method also includes storing one or more of the following along with the point matrix and a lookup table: multiple sets of phase orders, a rotation matrix, multiple slopes corresponding to the independent terms, or values indicating the distance between the independent terms.
[0121] Another example provides a computing system including a depth sensor comprising a plurality of pixels, each configured to sense light; an illumination source configured to output light; a logic machine; and a memory storing instructions executable by the logic machine to control the illumination source to output amplitude-modulated light at K modulation frequencies, where K is an integer greater than or equal to 2, and for each of the plurality of pixels: K phase measurements are performed at each of the K modulation frequencies to form a set of noisy phase measurements; the position of a projection line in the K-dimensional phase space through the set of noisy phase measurements is determined to cross a low-dimensional plane in the K-dimensional phase space, the projection line being parallel to a noiseless phase evolution line; the position is compared with a plurality of independent entries in a predetermined point matrix in the low-dimensional plane, each point representing a corresponding predetermined set of noisy phase sequences with known noise, and a set of independent entries is selected; the corresponding set of noiseless phase sequences is located using a reference lookup table of the set of independent entries; a distance value is determined for the pixel based on the corresponding set of noiseless phase sequences; and the distance value for the pixel is output. In some such examples, the predetermined point matrix in the zero plane comprises a lattice formed by projecting a plurality of total phase points onto the zero plane in directions parallel to the noiseless phase evolution line, each of the plurality of total phase points representing a set of total phase orders in a K-dimensional phase space. Alternatively, in some examples, K = 3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane. Alternatively, in some examples, instructions may be executable to determine each of a plurality of independent terms by projecting positions along directions parallel to a set of lattice lines to intersect axes in a lower-dimensional plane, the set of lattice lines corresponding to that independent term, and selecting the set of independent terms based on the position of the axis intersection.
[0122] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the order shown and / or described, in another order, in parallel, or omitted. Similarly, the order of the above processes may be changed.
[0123] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of various processes, systems and configurations, as well as other features, functions, behaviors and / or properties disclosed herein, and any and all equivalents thereof.
Claims
1. A computing system, comprising: A depth sensor, comprising multiple pixels, each configured to sense light; The lighting source is configured to output light; Logic machine; as well as A storage unit that stores instructions executable by the logic machine to control the illumination source to output amplitude-modulated light at K modulation frequencies, where K is an integer greater than or equal to 2, and for each of the plurality of pixels: K phase measurements are performed at each of the K modulation frequencies to form a set of noise phase measurements; The position is determined where the projection lines passing through the set of noisy phase measurements in the K-dimensional phase space also pass through the lower-dimensional plane in the K-dimensional phase space, and these projection lines are parallel to the noise-free phase evolution lines. The position is compared with multiple independent entries in the predetermined point matrix in the low-dimensional plane, where each point represents a corresponding predetermined noise phase order set with known noise, and a set of independent entries is selected. By using the aforementioned set of independent item reference lookup tables, the corresponding set of noise-free phase sequences can be located. Based on the corresponding set of noise-free phase sequences, a distance value is determined for the pixel, and Output the distance value for the pixel.
2. The computing system according to claim 1, wherein the projection line and the noise-free phase evolution line are perpendicular to the low-dimensional plane.
3. The computing system of claim 2, wherein the instructions are further executable to perform rotations on the predetermined point matrix to reduce the dimension.
4. The computing system according to claim 1, wherein the low-dimensional plane is the zero plane.
5. The computing system according to claim 4, wherein, K=3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane.
6. The computing system of claim 1, wherein the predetermined point matrix in the low-dimensional plane comprises a lattice, the lattice being formed by projecting a plurality of total phase points onto the low-dimensional plane in a direction parallel to the noiseless phase evolution line, each of the plurality of total phase points representing a total phase order set in a K-dimensional phase space.
7. The computing system of claim 1, wherein the instructions are executable to determine each of the plurality of independent items by projecting the position along a direction parallel to a set of grid lines to intersect with an axis in the low-dimensional plane, the set of grid lines corresponding to the independent item.
8. A computing system, comprising: A depth sensor, comprising multiple pixels, each configured to sense light; The lighting source is configured to output light; Logic machine; as well as A storage unit that stores instructions executable by the logic machine to control the illumination source to output amplitude-modulated light at K modulation frequencies, where K is an integer greater than or equal to 2, and for each of the plurality of pixels: K phase measurements are performed at each of the K modulation frequencies to form a set of noise phase measurements; Determine the position in the K-dimensional phase space where the projection lines of the set of noise phase measurements pass through the zero plane in the K-dimensional phase space, and the projection lines are parallel to the noise-free phase evolution lines; The position is compared with multiple independent entries in a predetermined point matrix in the zero plane, where each point represents a corresponding predetermined noise phase order set with known noise, and a set of independent entries is selected. By using the set of independent item reference lookup tables, the corresponding set of noise-free phase sequences in the lookup tables are located; Based on the corresponding set of noise-free phase sequence, a distance value is determined for the pixel; as well as Output the distance value for the pixel.
9. The computing system of claim 8, wherein the predetermined point matrix in the zero plane comprises a lattice, the lattice being formed by projecting a plurality of total phase points onto the zero plane in a direction parallel to the noiseless phase evolution line, each of the plurality of total phase points representing a total phase order set in a K-dimensional phase space.
10. The computing system of claim 8, wherein K=3 and the zero plane is one of the XY Cartesian plane, the XZ Cartesian plane, or the YZ Cartesian plane.
11. The computing system of claim 8, wherein the instructions are executable to determine each of the plurality of independent items by projecting the position along a direction parallel to a set of grid lines to intersect with an axis in a low-dimensional plane, the set of grid lines corresponding to the independent item, and selecting the set of independent items based on the position of the intersection of the axes.
Citation Information
Patent Citations
Fast general multipath correction in time-of-flight imaging
CN105899969A
Image matching method and device
CN111582142A
