An Extended Ranging Method and System Based on Time of Flight

Abstract

The present invention provides an extended ranging method and system based on time of flight, which includes the following steps: performing probability modeling on the amplitude and distance of a target object q to obtain an amplitude-distance probability curve model of the target object q; constructing a reference function for judging amplitude-distance correlation; using a ToF camera to obtain a scene RGB image I1 at a fixed point O1 and a scene RGB image I2 at an offset point O2 offset from the fixed point O1 by a certain distance d; performing superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculating the feature vectors of each region in I1 and I2 respectively; matching the regions in I2 with the closest position information and feature vectors one by one with the corresponding regions in I1; performing amplitude correction processing on the matched regions; inputting the amplitude information after the correction processing into the amplitude-distance probability curve model to obtain the rough distances of each region in I1 and I2, then estimating the aliasing period k of the corresponding region, and calculating the true distance of the current region.

Description

Technical Field

[0001] The present invention relates to the technical field of spatial data processing, and more particularly, to an extended ranging method and system based on time of flight. Background Art

[0002] The time of flight (ToF) ranging imaging technology has become one of the most important and innovative research directions in research fields such as 3D imaging and multi-point ranging. Currently, the ToF imaging technology is mainly divided into direct time of flight imaging technology and indirect time of flight imaging technology. In the direct time of flight technology imaging scheme, since a high-precision timer is required to measure the time difference between the transmitted light pulse synchronized with the light source and the reflected pulse at the receiving end, this scheme has high requirements for the performance of the device and cannot be used on a large scale at present. In the indirect time of flight imaging scheme, the round-trip flight time of the light pulse signal is indirectly calculated mainly by measuring the phase difference between the transmitted signal and the received signal, thus avoiding the need for a high-precision timer. Therefore, the technical route of this scheme has also become the solution adopted by current mainstream ToF cameras.

[0003] The indirect time of flight measurement technology is also called the phase method technology. This technology is that the transmitting end emits a modulated continuous sine light signal, and then based on the measured phase difference between the output and the input signal, the time of flight is solved through the phase measurement formula. However, this technology also has some disadvantages, such as the phenomenon of phase cycle aliasing is likely to occur, resulting in a blurred and aliased result in the calculated distance. In other words, when the returned light wave exceeds 2π, it will overlap with the previous light wave, resulting in phase aliasing of the light wave, and then distance ambiguity and aliasing occur. To avoid this situation, the measurement distance of the ToF camera is limited to Where c is the speed of light and f is the frequency of the light wave, which results in a reduction in the actual detectable depth value. Currently, there are also literatures proposing to adopt a multi-frequency method to solve the phase aliasing and distance ambiguity problems of ToF cameras, that is, to emit signals of multiple different frequencies into the scene, measure the phase values of these signals, and then solve the distance ambiguity problem through the equation relationships established by them. However, this method not only requires high power consumption but also needs to take into account the cycle durations of different frequency signals to collect these round-trip signals, making it difficult to be applied to dynamic scenes with high real-time requirements. In addition, there are literatures proposing to use gradient edge detection algorithms and watershed segmentation algorithms to segment the depth map, then calculate the average amplitude of each segmented region, and finally manually select an amplitude threshold to determine whether distance ambiguity occurs in each region. Although this method is simple, it requires manual calculation of the amplitude at which the critical value of the ambiguous distance appears before measurement, and when the scene is relatively complex, the repair effect drops significantly, and the depth hole phenomenon is not considered during distance expansion, so its flexibility and robustness in actual scenes are both poor. There are literatures proposing to quantize the results of pixel point phase aliasing into multiple discrete values, and proposing an amplitude-distance probability model and a smoothing constraint model based on single-frequency measurement, and solving this problem in a global optimization manner by setting different weight coefficients of the probability model and the smoothing constraint model. However, the amplitude information is not only related to the distance but also related to the emissivity of the object and the incident angle of the light wave. Therefore, the matching relationship between the amplitude information and the distance is difficult to be represented by a single closed-form linear equation, but the amplitude-distance probability model proposed in this literature does not fully consider the influence of the above factors, making the robustness of this scheme not high in complex application scenarios; on the other hand, due to the use of the Belief Propagation (BP) algorithm during global optimization, there are a large number of iterative calculation processes during operation, making the time cost of this scheme extremely high. Summary of the Invention

[0004] The present invention aims to overcome the defects in the existing indirect time-of-flight measurement technology, such as the mismatch between amplitude information and distance, and poor flexibility and robustness in actual scenes, and provides an extended ranging method and system based on time of flight.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] An extended ranging method based on time of flight, comprising the following steps:

[0007] S1. According to the relationship between the amplitude and distance of the target object q, use the Dirac function to perform probability modeling on the amplitude and distance of the target object q to obtain the amplitude-distance probability curve model of the target object q;

[0008] S2. Set several different amplitudes, conduct simulation experiments on the amplitude-distance probability curve model, perform function fitting on each set of set amplitudes and their corresponding distance data combinations, and obtain a reference function for judging amplitude-distance correlation;

[0009] S3. Use a ToF camera to obtain the scene RGB image I1 at the fixed point O1, and obtain the scene RGB image I2 at the offset point O2 that is offset from the fixed point O1 by a certain distance d;

[0010] S4. Perform superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculate the feature vectors of each region in I1 and I2; according to the feature vectors of each region in I1 and I2, compare the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1, and match the region in I2 with the closest position information and feature vectors with the corresponding region in I1 one by one;

[0011] S5. Perform amplitude correction processing on the matched regions;

[0012] S6. For the matched regions, input the amplitude information after correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and then use the rough distance to estimate the aliasing period k of the corresponding region. Through d i +k·D max calculate the true distance of the current region, where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance;

[0013] For the regions that cannot be successfully matched and whose amplitudes are not 0, directly input the amplitude values of each pixel point in each region in I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare this distance information with the distance information measured by the current ToF camera: if the error result is less than the error distance threshold D T , then set the best k value to 0; otherwise, set this region as a hole and process it using the hole compensation algorithm.

[0014] Furthermore, the present invention also proposes an extended ranging system based on time of flight, applying the extended ranging method proposed by the present invention. Among them, the system includes:

[0015] A ToF camera for collecting the RGB images I1 and I2 of the fixed point O1 and the offset point O2 that is offset from the fixed point O1 by a certain distance d;

[0016] The region division module is used to perform superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculate the feature vectors of each region in I1 and I2 respectively;

[0017] The region matching module is used to compare the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1 according to the feature vectors of each region in I1 and I2, and match the region in I2 with the closest position information and feature vectors with the corresponding region in I1 one by one;

[0018] The amplitude correction module is used to perform amplitude correction processing on the matched regions;

[0019] The extended ranging module includes an amplitude-distance probability curve model constructed according to the relationship between the amplitude and distance of the target object q; for the regions that have completed matching, the extended ranging module is used to input the corrected amplitude information thereof into the amplitude-distance probability curve model to obtain the rough distances of each region in I1 and I2, and then estimate the aliasing period k of the corresponding region by using the rough distances, and obtain the true distance of the current region through d i +k·D max where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance; for the regions that cannot be successfully matched and have a non-zero amplitude, it is used to directly input the amplitude values of each pixel point in each region of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare the distance information with the distance information measured by the current ToF camera: if the error result is less than the error distance threshold D T , the optimal k value is set to 0; otherwise, the region is set as a hole and processed by the hole compensation algorithm.

[0020] Furthermore, the present invention also proposes a computer device, including one or more processors; a memory; and one or more application programs; wherein the one or more application programs are stored in the memory; the memory is configured to execute the operations of the extended ranging method based on time of flight proposed by the present invention by the processor.

[0021] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows: By constructing an amplitude-distance probability curve model, the present invention converts the actual amplitude information of all objects to be uniformly compared under the reference amplitude-distance curve, solving the problem of mismatch between amplitude information and distance; when distance aliasing occurs due to exceeding the original ranging range of the ToF camera, the present invention can extend the theoretical maximum measurement range of the ToF camera from 1 measurement cycle to k + 1 (k > 0) measurement cycles, thereby greatly expanding the measurement distance; during the process of extending the ranging, the present invention does not require obtaining the reflectivity and AoI of the objects in the scene as prior knowledge, thus effectively improving the robustness and applicability of the solution; the present invention also adds an amplitude correction operation, which can obtain more obvious performance gains and more accurate extended ranging effects in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flowchart of the extended ranging method based on time of flight for Embodiment 1.

[0023] Figure 2 It is a schematic diagram showing the change of AoI during the movement of the ToF camera in Embodiment 1.

[0024] Figure 3 It is a schematic diagram showing the relationship between amplitude and distance under different reflectivities before amplitude correction.

[0025] Figure 4 It is a schematic diagram showing the relationship between amplitude and distance under different reflectivities after amplitude correction.

[0026] Figure 5 It is a schematic diagram showing the relationship between amplitude and distance under different AoIs before amplitude correction.

[0027] Figure 6 It is a schematic diagram showing the relationship between amplitude and distance under different AoIs after amplitude correction.

[0028] Figure 7 It is a schematic diagram showing the relationship between amplitude and distance under different combinations of AoI and reflectivity before amplitude correction.

[0029] Figure 8 It is a schematic diagram showing the relationship between amplitude and distance under different combinations of AoI and reflectivity after amplitude correction.

[0030] Figure 9 It is a probability density curve graph under different amplitudes.

[0031] Figure 10 It is a reference curve graph of amplitude and distance.

[0032] Figure 11 It is a schematic diagram showing the relationship between amplitude and distance of different objects before amplitude correction.

[0033] Figure 12 Schematic diagram of the amplitude - distance relationship of different objects after amplitude correction.

[0034] Figure 13 Comparison chart of ranging results for objects made of foam material.

[0035] Figure 14 Comparison chart of ranging results for objects made of wood material.

[0036] Figure 15 RGB image after registration collected by the ToF camera.

[0037] Figure 16 Original depth aliasing image collected by the ToF camera.

[0038] Figure 17 Extended ranging depth map of each scheme.

[0039] Figure 18 Extended ranging depth map of the dual - frequency scheme.

[0040] Figure 19 Schematic diagram for comparing the mean square error performance of each scheme.

[0041] Figure 20 Schematic diagram for comparing the correct percentage of pixel point periods of each scheme.

[0042] Figure 21 Architecture diagram of the time - of - flight - based extended ranging system of Embodiment 3. Specific implementation mode

[0043] The attached drawings are only for illustrative purposes and should not be construed as limitations on this patent;

[0044] For those skilled in the art, it is understandable that the description of some well - known professional terms in the attached drawings may be omitted.

[0045] The technical solutions of the present invention will be further described below with reference to the attached drawings and embodiments.

[0046] Embodiment 1

[0047] This embodiment proposes a time - of - flight - based extended ranging method, as Figure 1 shown, which is the flowchart of the time - of - flight - based extended ranging method of this embodiment.

[0048] In the time - of - flight - based extended ranging method proposed in this embodiment, the following steps are included:

[0049] S1. According to the relationship between the amplitude and distance of the target object q, use the Dirac function to perform probability modeling on the amplitude and distance of the target object q, and obtain the amplitude - distance probability curve model of the target object q;

[0050] S2. Set several different amplitudes, conduct simulation experiments on the amplitude-distance probability curve model, perform function fitting on each set of set amplitudes and their corresponding distance data combinations, and obtain a benchmark function for judging amplitude-distance correlation;

[0051] S3. Use a ToF camera to obtain a scene RGB image I1 at a fixed point O1, and obtain a scene RGB image I2 at an offset point O2 that is offset from the fixed point O1 by a certain distance d;

[0052] S4. Perform superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculate the feature vectors of each region in I1 and I2 respectively; according to the feature vectors of each region in I1 and I2, compare the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1, and match the region in I2 with the closest position information and feature vectors with the corresponding region in I1 one by one;

[0053] S5. Perform amplitude correction processing on the matched regions;

[0054] S6. For the matched regions, input the amplitude information after correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and then use the rough distance to estimate the aliasing period k of the corresponding region. Through d i +k·D max calculate the true distance of the current region, where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance;

[0055] For regions that cannot be successfully matched and whose amplitude is not 0, directly input the amplitude values of each pixel point in each region of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare this distance information with the distance information measured by the current ToF camera: if the error result is less than the error distance threshold D T , then set the optimal k value to 0; otherwise, set the region as a hole and process it using a hole compensation algorithm.

[0056] In this embodiment, by constructing an amplitude-distance probability curve model, the actual amplitude information of all objects is converted to be uniformly compared under the reference amplitude-distance curve, solving the problem of mismatch between amplitude information and distance. At the same time, in the process of extended ranging in this embodiment, it is not necessary to obtain the reflectivity and AoI of the objects in the scene as prior knowledge, so the robustness and applicability of the solution are effectively improved. After completing the region matching, this embodiment also adds an amplitude correction operation, which can obtain more obvious performance gain and more accurate extended ranging effect under different scenarios. When distance aliasing occurs due to exceeding the original ranging range of the ToF camera, the present invention can extend the theoretical maximum measurement range of the ToF camera from 1 measurement cycle to k + 1 (k>0) measurement cycles, thus greatly expanding the measurement distance.

[0057] In this embodiment, considering that the amplitude information is affected by factors such as the reflectivity of the object and the Angle of Incidence (AoI), it often cannot truly reflect its relationship with the distance. There may be different amplitude information corresponding to the same distance, and thus the target distance cannot be uniquely determined by the amplitude information, resulting in the problem of distance ambiguity. If the spatial position of a pixel point cannot be uniquely determined, it can usually be quantized into multiple discrete values, and the possibility of the pixel point appearing at different positions in space is described by these discrete values. Therefore, the categorical distribution in the probability distribution can be used to model it to judge the ambiguity of this point.

[0058] In an alternative embodiment, the steps of constructing the amplitude-distance probability curve model of the target object q include:

[0059] S101. Obtain the reflectivity ρ of the target object q q , the prior pixel amplitude value A when the ToF camera measures an object with a reflectivity close to 1 at a unit distance, and the incident angle β of the target object q q ; Considering that most objects in general scenes conform to the Lambert radiation model, in this embodiment, based on the Lambert radiation model, the relationship expression between the amplitude and distance of the target object q is established:

[0060]

[0061] where B q represents the amplitude of the target object q, f R (ρ q ) represents the reflectivity function of the target object q, and D q is the distance between the ToF camera and the target object.

[0062] S102. With parameters A, ρ q , β qWhen it is known, the probability of the amplitude and distance of the target object q is modeled using the Dirac function; the expression is as follows:

[0063]

[0064] Among them, δ(·) represents the Dirac function.

[0065] S103. Since the probability estimate of the target object's distance is affected by amplitude, reflectivity, and AoI, in order to study the distance probability estimate of the target object only under the condition that the amplitude is a variable, in this embodiment, a double integral is performed on the reflectivity and incident angle of the target object q, and formula (2) is transformed to eliminate the influence of reflectivity and incident angle on the distance probability estimate of the target object; the expression is as follows:

[0066]

[0067] From formula (3), the amplitude-distance probability curve can be further obtained as:

[0068]

[0069] D m = argmax p(D q |B q ) (5)

[0070] Among them, p(D q |B q ) is the amplitude-distance probability curve model of the target object q; the distance corresponding to the maximum probability is obtained based on the amplitude-distance probability curve model, which is the optimal distance D m .

[0071] Furthermore, in an alternative embodiment, a number of different amplitudes are set, a simulation experiment is performed on the amplitude-distance probability curve model, and a function fitting is performed on each set of set amplitudes and their corresponding distance data combinations to obtain a reference function for judging the amplitude-distance correlation; the expression is as follows:

[0072]

[0073] Among them, B r represents the amplitude under the reference curve, d r represents the distance corresponding to the amplitude under the reference curve, f R (ρ r ) is the reflectivity function, ρ r represents ρ q converted to the reflectivity under the reference curve, β r represents β q converted to the incident angle under the reference curve; Γ(ρr , β r ) is the Joint Reflectivity - and - AoI (JRA) coefficient.

[0074] According to the amplitude - distance probability curve model shown in formula (4), select L appropriate amplitude values B q,l , l = 1, …, L, and then calculate the corresponding optimal distance q,l for the amplitude value B where D q,l|max is the D value corresponding to the maximum probability p(D q |B q,l ). q Value.

[0075] Then, let B r,l = B q,l and d r,l = D q,l|max to form L data pairs. Using the L d r values as independent variables, the L B r values as dependent variables, and the reference function shown in formula (6) as the fitting expression, with Γ(ρ r , β r ) as undetermined coefficients, use the MATLAB fitting toolbox to perform parameter fitting on the reference function to obtain the value of Γ(ρ r , β r ). Subsequently, this reference function can be used as a reference scale to convert the amplitude information of objects with various different reflectivities and AoIs to this scale for amplitude - distance relationship judgment.

[0076] In an optional embodiment, in the step S4, for the scene RGB images I1 and I2, the Simple Linear Iterative Clustering (SLIC) algorithm is respectively used for super - pixel segmentation and region division, and the feature vectors of each region in I1 and I2 are respectively calculated. The steps include:

[0077] S411: According to the ToF ranging scene situation, set the number of super - pixel region segments of I1 and I2 to M1 and M2, and adopt the SLIC super - pixel segmentation algorithm to perform region clustering segmentation on the obtained I1 and I2. Denote as the region with index a in I1 after segmentation, a = 1, 2, …, M1; as the region with index b in I2 after segmentation, b = 1, 2, …, M2.

[0078] S412: Calculate the first - order, second - order, and third - order color moments for each segmented region to form the corresponding region feature vectors Their expressions are as follows:

[0079]

[0080]

[0081]

[0082] where s ij represents the pixel value of the j-th pixel of the i-th image channel in region s of the digital image, and i = {R, G, B}, representing the R, G, and B channels respectively; N s represents the number of pixels in region s,

[0083] Furthermore, according to the feature vectors of each region in I1 and I2, the regions in I2 with the closest position information and feature vectors are matched one by one with the corresponding regions in I1. The steps include:

[0084] S421. Calculate the pixel center point of each region in I1 and the pixel center point of each region in I2 Calculate the two-norm of the difference between the feature vector of each region in I1 and the feature vector of each region in I2, and combine it with the region position weight factor w p to form the region similarity matching coefficient Its expression is as follows:

[0085]

[0086] where:

[0087]

[0088] where, and represent the feature vectors of region and region respectively, and δ d is the standard deviation of w p S422. According to the region similarity matching coefficient

[0089] calculate the region similarity matching coefficient formed by any region in I2 and all regions in I1 to search for the best matching region of each region in I2 with I1 Index; its expression is as follows:

[0090]

[0091] where t is the region Index of the best-matching region among all the divided regions of I1; when there is then the region with index b in I2 matches successfully with the region with index t in I1.

[0092] S423. Calculate the dispersion coefficient V of each region in I2 c , and its expression is as follows:

[0093]

[0094] where x i is the amplitude of the i-th pixel point in the corresponding region, is the average amplitude of each region.

[0095] S424. Set the dispersion threshold η according to the experimental scenario, and compare the dispersion coefficient V of each region in I2 c with the dispersion threshold η in turn, and regard the region that satisfies V c < η as belonging to the same overall surface, and the entire region s2 can be processed as a whole; for the region that satisfies V c ≥ η it is regarded as containing two or more different object surfaces, and the regions that do not meet the preset conditions need to be removed and then compensated by using traditional or specific hole compensation algorithms.

[0096] Since the feature vector of the region only considers the color information of the region, if the color information of two regions in the same picture is close, they still cannot be distinguished. Therefore, judging the similarity of regions from the feature vector of the region can only be limited to identifying objects with different color information. If there are many objects with similar colors in the scene, the feature vector of a certain region in I2 may be relatively close to the feature vectors of multiple regions in I1. At this time, it will be impossible to accurately find the region in I1 that matches this region in I2. Considering that when I1 and I2 perform region matching, if the offset is relatively small, the overall spatial positions of their corresponding regions deviate little. Therefore, when they cannot be distinguished by using the region feature vector, in this embodiment, the position information of this region in I2 is calculated and compared with the position information of multiple regions in I1 to find the region with the closest position information to this region in I2, so as to achieve a more accurate and unique matching.

[0097] Specifically, in this embodiment, the similarity matching coefficients between each region in I2 and all regions in I1 are first calculated, and then the region matching is assisted by adding a region position weight factor, so that the regions with similar attributes in the two images are associated, making the result of region matching more accurate. That is, if a region in I1 and a region in I2 both represent the same object in the scene, these two regions are associated and regarded as a successful match.

[0098] Further, in an alternative embodiment, the steps of performing amplitude correction processing on the matched regions in step S5 include: extracting the amplitude information of the ToF camera at the fixed point O1 and the offset point O2, during which the reflectivity and the angle of incidence of the target object q are regarded as approximately the same; then regarding the reflectivity and the angle of incidence of the target object q as a whole to characterize their influence on the amplitude, and comparing it with the JRA coefficient of the reference function to obtain the corrected amplitude value of the target object q under the reference function.

[0099] On the basis of completing the superpixel region segmentation and matching operations, this embodiment further performs amplitude correction processing on the successfully matched regions. In the actual application scenario, since the amplitude information is affected by factors such as the reflectivity of the object and the AoI, it often cannot truly reflect its relationship with the distance. In order to extract effective distance information from the amplitude information, it is necessary to correct the original amplitude information so that the corrected amplitude can more accurately reflect its relationship with the distance. During the above amplitude correction processing operation, there is no need to separately obtain the values of the reflectivity and the AoI of the object, so that the ranging process no longer depends on the prior knowledge of these parameters, effectively improving the robustness and applicability of the solution. As Figure 2 shown, it is a schematic diagram of the change of the AoI during the movement of the ToF camera in this embodiment.

[0100] Further, regarding the reflectivity and the angle of incidence of the target object q as a whole to characterize their influence on the amplitude, and comparing it with the JRA coefficient of the reference function to obtain the corrected amplitude value of the target object q; the steps include:

[0101] S501. Calculate the average amplitude values of each region in I1 and I2 respectively The expression is as follows:

[0102]

[0103] Among them, the average amplitude information of the regions successfully matched in I2 is expressed as:

[0104]

[0105]

[0106]

[0107] Among them, is the distance from the ToF camera to the area , is the JRA coefficient of the area , d is the offset distance between the fixed point O1 and the offset point O2, and λ is the amplitude information scaling factor.

[0108] S502. Normalize the amplitude information of the target object q at the positions of the fixed point O1 and the offset point O2 under the reference function for comparison, and calculate the actual amplitude information of the area after the light wave is reflected by the target object q, and convert it to the corrected amplitude value under the reference function The expression is as follows:

[0109]

[0110] Among them, Γ(ρ r ,β r ) is the JRA coefficient of the reference function, and A is the prior value of the pixel amplitude.

[0111] In this embodiment, an amplitude correction model is added, and more obvious performance gain and more accurate extended ranging effect can be obtained in different scenarios.

[0112] Furthermore, when the ToF camera moves from point O1 to O2, two cases need to be discussed:

[0113] (1) If a certain area is located at the central axis position of the ToF camera, that is, Figure 3 the area of point P in , then this area is in good alignment with the ToF camera. At this time, the AoI of this area is 0, and the AoI remains unchanged when moving from point O1 to point O2. Then and match successfully, and they correspond to the same object in the scene. Then Therefore, the average amplitude information that can be obtained by the area when the ToF camera is at O1 can be expressed as:

[0114]

[0115] Then the amplitude information scaling factor is expressed as:

[0116]

[0117] Among them, is the distance from the ToF camera to the area .

[0118] (2) If it is difficult to maintain a good alignment with the ToF camera in a certain area, that is, the AoI is not 0, such as Figure 3 the P1 or P2 area in Also, because At this time, the average amplitude information of this area at point O1 can be expressed as:

[0119]

[0120] where the superscript <p1>express Figure 3 In this case, the amplitude information scaling factor is:

[0121]

[0122] in

[0123] Comparing the amplitude information proportional factors in cases (1) and (2), after simulation investigation and derivation, it can be known that within a certain AoI value range, λ′ is approximately equivalent to λ, so case 2 can also be processed approximately according to case 1. Therefore, this embodiment adopts the amplitude information proportional factor of case (1) to express, and the above formulas are combined to obtain the area Expression for the JRA coefficient.

[0124] It can be seen from formula (13) that at the same distance, the amplitude information is positively correlated with the JRA coefficient. In order to facilitate the evaluation of the amplitude-distance relationship between different regions, it is necessary to normalize the amplitude information of different ranging target objects to the same scale for comparison, that is, convert it to the reference curve for comparison. Specifically, the region can be calculated from formula (6): The corresponding real distance Amplitude information under the reference curve It is expressed as:

[0125]

[0126] Among them, Γ(ρ r ,β r ) can be obtained by using formula (6) and Matlab fitting method. Since the original distance measured by the ToF camera is fuzzy, the real distance cannot be obtained. It is difficult to calculate Therefore, this embodiment combines formula (13) with the above formula to eliminate We can get:

[0127]

[0128] Substituting the JRA coefficient into the above formula, we can get the actual amplitude information of the light wave after it is reflected by the target object. Amplitude information converted to the reference scale As shown below:

[0129]

[0130] Further, in an optional embodiment, in the step S6, the steps include:

[0131] 1) For the area where the matching is completed, the amplitude value will be corrected Input into the amplitude-distance probability curve model to obtain the region where matching is completed of the rough distance and calculate the corrected amplitude value at the maximum probability of the amplitude-distance probability curve model corresponding to the distance D m ; its expression is as follows:

[0132]

[0133]

[0134] where, is the region where matching is completed corresponding to the distance between the ToF camera and the target object

[0135] Use the rough distance to solve the objective function of the aliasing period number k of the region where matching is completed, and its expression is as follows:

[0136] |(d i +k·D max )-D m |<D T (19)

[0137] In the formula, d i is the original measured distance from the ToF camera to the region D T is the error distance threshold, which is used to limit the error range of the period number of the current region, and its value is less than D max ; Substitute k = 0, 1,..., N into the above linear inequality. If the unique k value makes the objective function hold, then the aliasing period number k is output

[0138] Finally, calculate the true distance of the current region through d i +k·D max ; where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance

[0139] 2) For the regions that cannot be successfully matched and whose amplitudes are not 0, directly input the amplitude values of each pixel point in the corresponding regions of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare this distance information with the distance information measured by the current ToF camera: If the error result is less than D T , then set the optimal k value to 0; otherwise, set this region as a hole and then process it using the hole compensation algorithm

[0140] Since the ToF camera uses periodic signals for distance measurement, when a certain area in the scene exceeds the maximum unambiguous distance of the ToF camera, phase period aliasing will occur, resulting in fuzzy aliasing of the distance information collected by the ToF camera, that is, one-value ambiguity. In order to calculate the number of aliasing cycles in the area to eliminate ambiguity, this embodiment performs the amplitude correction operation, substitutes the corrected amplitude information into the amplitude-distance judgment model to obtain the rough distance D of each area m , and then use the rough distance D m The aliasing period k of the area is estimated, and then the real distance information of the area is restored. Through the method proposed in this embodiment, when the distance aliasing occurs due to exceeding the maximum distance measurement range of the ToF camera, the theoretical maximum measurement range of the camera can be extended from 1 measurement period to k+1 (k>0) measurement periods, thereby greatly expanding the measurement distance.

[0141] Example 2

[0142] This embodiment applies the extended ranging method based on flight time proposed in Embodiment 1 to conduct simulation experiments and analysis, and further illustrates the effectiveness and advancement of the present invention.

[0143] First, this example will combine simulation experiments to evaluate the effectiveness of the amplitude correction model scheme.

[0144] Since the amplitude value is related to the reflectivity and AoI of the object, in order to fully evaluate the performance of the amplitude correction model, the simulation experiment will study the amplitude correction when the reflectivity and AoI change separately and the amplitude correction when the reflectivity and AoI change jointly.

[0145] In this embodiment, the influence of amplitude and distance under various reflectivity is studied while keeping the AoI of the object at 45°. Figure 3 , 4 The following are schematic diagrams showing the relationship between amplitude and distance under different reflectivity before and after amplitude correction. In this embodiment, the reflectivity of the reference line (Ref) is set to 0.8. Figure 3 It can be seen from the figure that at the same distance, the light waves have different amplitude values ​​after being reflected by objects with different reflectivity. After the amplitude correction operation, Figure 4 It can be seen that the amplitude and distance curves of different reflectivities basically coincide with the baseline reference line, so that the amplitude values ​​of light waves after reflection by objects with different reflectivities are unified to the same scale, and the corrected amplitude values ​​are not affected by the reflectivity of the object.

[0146] Furthermore, this embodiment studies the influence of amplitude and distance under various AoIs while keeping the reflectivity of the object at 0.8. Figure 5 , 6 As shown, they are schematic diagrams of the amplitude-distance relationship at different AoIs before and after amplitude correction. In this embodiment, the AoI of the reference line (Ref) is set to 45°, and from Figure 5 it can be seen that at the same distance, the light wave has different amplitude values after being reflected by objects with different AoIs. After the amplitude correction operation, from Figure 6 it can be seen that the amplitude-distance curves of different AoIs basically coincide with the reference line, so that the amplitude values of the light wave reflected by objects with different AoIs are unified to the same scale, and the corrected amplitude value is not affected by the AoI of the object.

[0147] Furthermore, this embodiment also studies the influence of the amplitude and distance of objects under any combination of reflectivity and AoI. As Figure 7 、 8 shown, they are schematic diagrams of the amplitude-distance relationship under different combinations of AoI and reflectivity before and after amplitude correction. In this embodiment, the reflectivity of the reference line (Ref) is set to 0.8 and the AoI is 45°. From Figure 7 it can be seen that at the same distance, the light wave has different amplitude values after being reflected by objects with different combinations. After the amplitude correction operation, from Figure 8 it can be seen that the amplitude-distance curves of different combinations basically coincide with the reference line, so that the amplitude values of the light wave reflected by objects with different combinations of reflectivity and AoI are unified to the same scale, and the corrected amplitude value is related to the reference value and is not affected by the combined influence of the reflectivity and AoI of the object.

[0148] Secondly, this embodiment will evaluate the effectiveness of the extended ranging scheme by combining actual object experiments.

[0149] In this embodiment, according to the actual experimental conditions and scenario situations, different amplitude values are set, and several probability curves with different amplitudes are generated by using the amplitude-distance joint judgment probability model as Figure 9 shown. Then, the distance corresponding to the maximum probability is taken as the distance of this amplitude value, and based on these amplitude and distance data pairs, an amplitude-distance reference curve is generated, as Figure 10 shown. The subsequent amplitude correction operation will use this reference curve as the unified scale. Without loss of generality, this embodiment determines A = 5000 and several pairs of values through experiments, and uses formula (6) to fit the amplitude-distance relationship reference curve to obtain This value can be used as a prior parameter to calculate the results of various subsequent ranging experiments.

[0150] Furthermore, for the extended ranging experiment of a single object, this embodiment uses a ToF depth camera model TSP-V4 produced by Fanwei Technology Co., Ltd., and its main parameters are shown in Table 1.

[0151] Table 1 ToF camera parameters

[0152] Parameter Value Unit ToF Sensor Resolution 320*240 pixel RGB Image Sensor Resolution 1920*1080 pixel Modulation Frequency 36 MHz Exposure Time 1500 μs Frame Rate 30 fps

[0153] Set 11 evenly - spaced anchor points on the same straight line in the experimental site, as shown in Table 2.

[0154] Table 2 Experimental Anchor Point Position Table

[0155] Anchor Point P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 Distance [m] 0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5

[0156] Then, fix the ToF camera at the reference position P0, and move the foam (Bubble) and wooden objects along the straight - line path where the anchor points are located, and record the amplitude information of the light wave after being reflected by the objects. As Figure 11 、 12 shown, they are the schematic diagrams of the relationship between the amplitude and distance of different objects before and after correction respectively. Among them, Ref is the reference curve. It can be seen that after correction, the curves of the relationship between the amplitude and distance of the light wave reflected by the two objects are closer to the reference curve. Although there will still be a certain error when the distance reaches 6m and above, the overall error within the entire distance range is still much smaller than the error before correction.

[0157] As Figure 13 、 14 shown, they are the comparison diagrams of the ranging results of the foam - material object and the wooden - material object respectively. Among them: the camera curve is the original measured distance output by the camera; the real curve is the true value measured in advance offline; the original amplitude scheme is the distance directly obtained after correcting the original amplitude. It can be seen that the curve of this scheme is relatively close to the real curve, which to a certain extent shows the feasibility of using the distance obtained from the amplitude information to correct the original measured distance of the camera; the amplitude - corrected ranging scheme is the distance obtained after correcting the amplitude by the scheme proposed in the present invention, substituting it into the objective function given by formula (19), then finding the optimal value of k, and finally obtaining the measurement result. From Figure 13 、 14 it can be known that the measurement result of the ToF camera is limited to points P0 to P6, and the measurement results of points P7 to P11 are incorrect. The reason is that when the measurement frequency is 36MHz, the maximum unambiguous distance of this ToF camera is 4.17 meters. When the position of the object exceeds this measurement range, a distance aliasing phenomenon will occur, that is, the distance information of the object at a farther position is mixed with the distance information of the object at a closer position, and the true distance of the object at a farther position cannot be distinguished. However, the curve of the amplitude - corrected ranging scheme obtained by estimating the distance information using the corrected amplitude information and then performing auxiliary correction can basically coincide completely with the real curve, that is, it realizes the expansion of the ranging range of the ToF camera from P0 - P6 points to P0 - P11 points, and the measurement result can still maintain a high accuracy.

[0158] This embodiment also proposes to perform extended ranging for the full scene. In the experiment for the extended ranging requirement of the full scene, the following operations are performed:

[0159] (1) Fix the ToF camera at position O1 in the test site to collect data, then horizontally offset the ToF camera by dozens of centimeters to position O2, and then collect new data.

[0160] (2) Perform superpixel segmentation and region division on the full scene using step S4, process the RGB images obtained at the above two positions respectively, and then perform one-to-one matching on the regions segmented at positions O1 and O2.

[0161] (3) Use step S5 to perform amplitude correction processing on the regions that have completed the matching.

[0162] (4) Use step S6 to input the amplitude information after the correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and finally substitute the distance information into the objective function given by formula (5) to find the optimal value of k.

[0163] After the above operations, the measurement results of the full scene are finally obtained. The experimental results of ranging in the actual scene are as Figures 15 - 18 shown, where Figure 15 is the RGB image after registration collected by the ToF camera, Figure 16 is the original depth aliasing map collected by the ToF camera. It can be seen from this that the actual maximum measurable distance of this camera is about 4 meters, and the ranging results beyond 4 meters are completely incorrect. Figure 17 In Figure 17 (a) is the extended ranging depth map using the McClure scheme, Figure 17 (b) is the extended ranging depth map using the Crabb scheme, Figure 17 (c) is the extended ranging depth map using the AmpUncorrected scheme, Figure 17 (d) is the extended ranging depth map using the scheme of the present invention. Among them, AmpUncorrected is the result directly obtained from the amplitude-distance probability model without using the amplitude correction model; Figure 18 is the result obtained using the dual-frequency scheme. It is pointed out in the literature that the result obtained by the dual-frequency scheme is more accurate than that of the single-frequency scheme, so it can be used as an approximate reference true value for comparison with the scheme of the present invention. From Figure 16 , 17 Looking at the key areas within the middle circle, the McClure and Crabb schemes obtained incorrect results. In addition, the experimental results of the AmpUncorrected scheme showed that the ranging results at some positions in the scene deviated significantly from the actual results because the amplitude information at these positions could not reliably reflect the distance relationship. In contrast, the scheme proposed in the present invention can obtain results close to those of the dual-frequency reference scheme in most cases, but it has advantages such as simple equipment and low computational complexity compared to the dual-frequency scheme, and thus can achieve an overall better effect.

[0164] Finally, Figure 19 The mean square error performance comparison of the measurement results of each scheme with the measurement results of the dual-frequency scheme as the benchmark is given. It can be seen from this figure that the mean square error of the scheme of the present invention has been significantly reduced with respect to the initial depth data generated by the camera. Compared with the McClure scheme, the Crabb scheme, and the AmpUncorrected scheme, the mean square error of the scheme of the present invention is only about 50% of these two schemes. Further, Figure 20 The correct rate comparison of the pixel point period is given. This index is defined as the ratio of the phase measurement period of all pixel points in the depth image after algorithm processing to the approximate true phase measurement period obtained by back-calculation using the dual-frequency scheme. Since the measurement method based on the phase method will cause ambiguity in the measurement period corresponding to the pixel points in the depth map, that is, it is impossible to determine in which measurement period the pixel point is measured, the performance of the ranging scheme can be measured by counting the correct measurement periods corresponding to the pixel points. From Figure 20 it can be seen that in the original depth data generated by the camera, the correct period percentage of the pixel points is only about 30%, while the scheme of the present invention can reach about 90% and is better than other comparison schemes.

[0165] Embodiment 3

[0166] This embodiment proposes an extended ranging system based on time-of-flight, which is applied to the extended ranging method based on time-of-flight proposed in Embodiment 1. As Figure 21 shown, it is the architecture diagram of the extended ranging system of this embodiment.

[0167] In the extended ranging system proposed in this embodiment, it includes:

[0168] A ToF camera for collecting RGB images I1 and I2 of a fixed point O1 and an offset point O2 offset from the fixed point O1 by a certain distance d.

[0169] A region division module for performing superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculating the feature vectors of each region in I1 and I2 respectively.

[0170] A region matching module, which is used to compare the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1 according to the feature vectors of each region in I1 and I2, and match the region in I2 with the closest position information and feature vectors one by one with the corresponding region in I1.

[0171] An amplitude correction module, which is used to perform amplitude correction processing on the regions that have completed matching.

[0172] An extended ranging module, which includes an amplitude-distance probability curve model constructed according to the relationship between the amplitude and distance of the target object q; for the regions that have completed matching, the extended ranging module is used to input the amplitude information after its correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and then use the rough distance to estimate the aliasing period k of the corresponding region, and obtain the true distance of the current region through d i +k·D max where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance; for the regions that cannot be successfully matched and whose amplitude is not 0, it is used to directly input the amplitude values of each pixel point in each region in I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare the distance information with the distance information measured by the current ToF camera: if the error result is less than the error distance threshold D T , then set the optimal k value to 0; otherwise, set the region as a hole and process it using the hole compensation algorithm.

[0173] Embodiment 4

[0174] This embodiment provides a computer device, including one or more processors; a memory; and one or more applications; wherein the one or more applications are stored in the memory.

[0175] Wherein, the memory is configured to execute the operations of the extended ranging method based on time of flight proposed in Embodiment 1 by the processor.

[0176] Obviously, the above embodiments of the present invention are merely examples for clearly explaining the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. An extended ranging method based on time of flight, characterized in that, It includes the following steps: S1. According to the relationship between the amplitude and distance of the target object q, use the Dirac function to perform probability modeling on the amplitude and distance of the target object q, and obtain the amplitude-distance probability curve model of the target object q; S2. Set several different amplitudes, conduct simulation experiments on the amplitude-distance probability curve model, perform function fitting on each set of set amplitudes and their corresponding distance data combinations, and obtain a reference function for judging the amplitude-distance correlation; S3. Use a ToF camera to obtain the scene RGB image I1 at the fixed point O1, and obtain the scene RGB image I2 at the offset point O2 that is offset from the fixed point O1 by a certain distance d; S4. Perform superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculate the feature vectors of each region in I1 and I2 respectively; According to the feature vectors of each region in I1 and I2, compare the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1, and match the region in I2 with the closest position information and feature vectors one by one with the corresponding region in I1; S5. Perform amplitude correction processing on the matched regions; S6. For the regions where matching is completed, input the amplitude information after correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and then use the rough distance to estimate the aliasing period k of the corresponding region. Through d i + k·D max calculate the true distance of the current region, where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance; For the regions that cannot be successfully matched and whose amplitudes are not zero, directly input the amplitude values of each pixel point in each region of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare this distance information with the distance information measured by the current ToF camera: If the error result is less than the error distance threshold D T , then set the optimal k value to 0; otherwise, set this region as a hole and then process it using the hole compensation algorithm.

2. The extended ranging method based on time of flight according to claim 1, characterized in that, In the step S1, the steps of constructing the amplitude-distance probability curve model of the target object q include: S101. Obtain the reflectivity ρ of the target object q q 、The ToF camera measures the prior value A of the pixel amplitude when measuring an object with a reflectivity close to 1 at a unit distance, and the incident angle β of the target object q q , and establish a relational expression between the amplitude and distance of the target object q: Among them, B q represents the amplitude of the target object q, and f R (ρ q ) represents the reflectivity function with respect to the target object q, and D q is the distance between the ToF camera and the target object; S102. Use the Dirac function to perform probability modeling on the amplitude and distance of the target object q; its expression is as follows: wherein, δ(·) represents the Dirac function; S103. Perform a double integral on the reflectivity and incident angle of the target object q, and convert formula (2) to eliminate the influence of the reflectivity and incident angle on the probability estimation of the distance of the target object; its expression is as follows: D m = argmax p(D q | B q ) (5) where p(D q |B q ) is the amplitude-distance probability curve model of the target object q; the distance corresponding to the maximum probability obtained based on the amplitude-distance probability curve model is the optimal distance D m .

3. The extended ranging method based on time of flight according to claim 2, characterized in that, In the step S2, the expression of the reference function for judging the amplitude-distance correlation is as follows: Among them, B r represents the amplitude under the reference curve, d r represents the distance corresponding to the amplitude under the reference curve, f R (ρ r ) is the reflectivity function, ρ r represents the reflectivity after ρ q is converted to the reference curve, β r represents β q after being converted to the incident angle under the reference curve; Γ(ρ r , β r ) is the JRA coefficient; According to the amplitude-distance probability curve model, select L appropriate amplitude values B q,l , where l = 1, …, L, and then calculate the best distance corresponding to the amplitude value B q,l ​ Let B r,l = B q,l and d r,l = D q,l|max to form L data pairs. Using the L d r values as independent variables and the L B r values as dependent variables, with the reference function as the fitting expression, and Γ(ρ r , β r ) as undetermined coefficients, use the MATLAB fitting toolbox to perform parameter fitting on the reference function to obtain the value of Γ(ρ r , β r ).

4. The extended ranging method based on time of flight according to claim 1, characterized in that, In the step S4, the steps of performing superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculating the feature vectors of each region in I1 and I2 respectively include: S411. Set the number of superpixel region segmentations of I1 and I2 to M1 and M2 according to the ToF ranging scenario. Adopt the SLIC superpixel segmentation algorithm to perform regional clustering segmentation on the obtained I1 and I2, and denote as the region with index a after the segmentation of I1, where a = 1, 2,..., M1; as the region with index b after the segmentation of I2, where b = 1, 2,..., M2; S412. Calculate the first-order, second-order, and third-order color moments for each of the segmented regions to form the feature vector of the corresponding region. Their expressions are as follows: where s ij represents the pixel value of the j-th pixel in the i-th image channel of region s in the digital image, and i = {R, G, B}, representing the R, G, and B channels respectively; N s represents the number of pixels in region s, 5. The extended ranging method based on time of flight according to claim 4, characterized in that,In the step S4, according to the feature vectors of each region in I1 and I2, the steps of matching the region in I2 with the closest position information and feature vectors one by one with the corresponding region in I1 include: S421. Calculate the pixel center points of each region in I1 and the pixel center points of each region in I2 Calculate the two-norm of the difference between the eigenvectors of each region in I1 and the eigenvectors of each region in I2 Calculate the two-norm of the difference between the eigenvectors of each region in I1 and the eigenvectors of each region in I2, and combine it with the regional position weight factor w to form the regional similarity matching coefficient p Its expression is as follows: The expression is as follows: wherein: Among them, and respectively represent the eigenvectors of region and region , and δ d is the standard deviation of w p ; S422. Calculate the regional similarity matching coefficient for any region in I2 with all regions in I1 to search for the best matching region of each region in I2 with I1; the index is as follows: The expression is as follows: for the best matching region of I2 with I1 ; its expression is as follows: where t is the index of the best-matching region among all the segmented regions of I1; when there is a match, then the region with index b in I2 matches successfully with the region with index t in I1; ​ S423. Calculate the coefficient of variation V for each region in I2 c , and its expression is as follows: where x i is the amplitude of the i-th pixel in the corresponding region, is the average amplitude of each region; S424. Set the discrete threshold η, and compare the discrete coefficient V of each region in I2 c with the discrete threshold η in sequence. Consider the regions where V c < η as belonging to the same overall surface; for the regions where V c ≥ η are regarded as containing two or more different object surfaces. After removing the regions that do not meet the preset conditions, use traditional or specific hole compensation algorithms for compensation.

6. The time-of-flight based extended ranging method according to claim 4, wherein, In the step S5, the steps of performing amplitude correction processing on the matched regions include: Extract the amplitude information of the ToF camera at the fixed point O1 and the offset point O2, and keep the reflectivity and incident angle of the target object q approximately the same during this period; then regard the reflectivity and incident angle of the target object q as a whole to characterize its influence on the amplitude, and compare it with the JRA coefficient of the reference function to obtain the corrected amplitude value of the target object q under the reference function.

7. The time-of-flight based extended ranging method according to claim 6, wherein, Regard the reflectivity and incident angle of the target object q as a whole to characterize its influence on the amplitude, and compare it with the JRA coefficient of the reference function to obtain the corrected amplitude value of the target object q; Its steps include: S501. Calculate the average amplitude values of each region in I1 and I2 respectively The expression is as follows: wherein, the average amplitude information of the matched region in I2 is expressed as: Among them, is the distance from the ToF camera to the area , is the JRA coefficient of the area , d is the offset distance between the fixed point O1 and the offset point O2, and λ is the amplitude information scaling factor; S502. Normalize the amplitude information of the target object q at the fixed point O1 and the offset point O2 to compare it under the reference function, and calculate the area after the light wave is reflected by the target object q of the actual amplitude information to obtain the corrected amplitude value under the reference function through conversion Its expression is as follows: Among them, Γ(ρ r, β r ) is the JRA coefficient of the reference function; A is the prior value of the pixel amplitude.

8. The time-of-flight based extended ranging method according to claim 7, wherein, In the step S6, its steps include: 1) For the region where the matching is completed, the correction amplitude value is input into the amplitude-distance probability curve model to obtain the rough distance of the region where the matching is completed and the correction amplitude value is calculated at the distance D corresponding to the maximum probability of the amplitude-distance probability curve model m ; and its expression is as follows: Among them, is the area where the matching is completed corresponding distance between the ToF camera and the target object; Use the objective function for roughly solving the aliasing period number k of the matched region, and its expression is as follows: |(d i +k·D max )-D m |<D T (19) Among them, d i is the original measured distance from the ToF camera to the area , D T is the error distance threshold, which is used to limit the error range of the number of cycles in the current area, and its value is less than D max ; Substitute k = 0, 1,..., N into the above linear inequality. If the unique k value that makes the objective function hold is obtained, it is the output of the aliasing period number k; Finally, through d i + k·D max the true distance of the current area is obtained, where d i represents the original measured distance from the ToF camera to the corresponding area, and D max is the maximum unambiguous distance; 2) For the regions that cannot be successfully matched and have a non-zero amplitude, directly input the amplitude values of each pixel point in the corresponding regions of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare this distance information with the distance information measured by the current ToF camera: If the error result is less than D T , set the optimal k value to 0; otherwise, set this region as a hole and process it using the hole compensation algorithm.

9. An extended ranging system based on time of flight, applied to the extended ranging method based on time of flight according to any one of claims 1 to 8, characterized in that, Including: A ToF camera for collecting RGB images I1 and I2 of a fixed point O1 and an offset point O2 offset from the fixed point O1 by a certain distance d; A region division module for performing superpixel segmentation and region division on the scene RGB images I1 and I2 respectively, and calculating the feature vectors of each region in I1 and I2 respectively; A region matching module for comparing the position information and feature vectors of any region in I2 with the position information and feature vectors of all regions in I1 according to the feature vectors of each region in I1 and I2, and performing one-to-one matching of the region in I2 with the closest position information and feature vectors to the corresponding region in I1; An amplitude correction module for performing amplitude correction processing on the regions that have completed matching; An extended ranging module, which includes an amplitude-distance probability curve model constructed based on the relationship between the amplitude and distance of the target object q; for the regions that have completed matching, the extended ranging module is used to input the amplitude information after correction processing into the amplitude-distance probability curve model to obtain the rough distance of each region in I1 and I2, and then use the rough distance to estimate the aliasing period k of the corresponding region, and obtain the true distance of the current region through d i +k·D max where d i represents the original measured distance from the ToF camera to the corresponding region, and D max is the maximum unambiguous distance; for the regions that cannot be successfully matched and have a non-zero amplitude, it is used to directly input the amplitude values of each pixel point in each region of I1 and I2 into the amplitude-distance probability curve model, calculate the distance information corresponding to the current amplitude value, and then compare the distance information with the distance information measured by the current ToF camera: if the error result is less than the error distance threshold D T , then set the optimal k value to 0; otherwise, set the region as a hole and process it using the hole compensation algorithm.

10. A computer device, comprising one or more processors; a memory; and one or more applications; wherein the one or more applications are stored in the memory; characterized in that, The memory is configured to be executed by the processor to perform the operations of the time-of-flight based extended ranging method according to any one of claims 1 to 8.

Images