System and method for tracking objects beyond the unambiguous range

Abstract

A system for tracking a position of a target object includes circuitry configured to control an illumination source to periodically emit illumination pulses for illuminating the target object. The illumination reflected from the target object is detected to estimate a sequence of modulo-range distances to the target object wrapped by a pulse repetition period. The circuitry unwraps the sequence of modulo-range distances subject to one or more continuity constraints to output a continuous trajectory of the target object.

Description

Technical Field

[0001] This disclosure generally relates to object detection in a scene, and more particularly, to a single-photon LiDAR-based device and method for tracking objects beyond the unambiguous range of LiDAR. Background Technology

[0002] Object detection and tracking are fundamental components of many real-world applications. Several application areas, such as autonomous vehicles, industrial robots, navigation, aerospace, meteorological element measurement, and atmospheric environmental monitoring, require accurate object detection and / or tracking to perform critical functions. Several methods exist with varying degrees of complexity and accuracy for tracking objects in a scene. Image processing-based methods are suitable for object tracking over short distances but are limited by lighting requirements. On the other hand, remote sensing methods are well-suited for tracking distant objects without relying on the strength of the received signal or other parameters. For example, radar-based methods simply rely on the reception of reflected signals without delving into the complexities of processing those reflected signals. In this regard, the time-of-flight of the signal is considered sufficient to measure the distance to the object.

[0003] Time-of-flight (ToF) distance sensing is a technique that measures the time it takes for a signal to travel from a source to a target and back. This technique is commonly used for distance measurement in a variety of applications, including camera autofocus, gesture recognition, robotics, augmented and virtual reality, and autonomous driving. ToF sensors typically use low-frequency illumination, such as infrared light, as a source. The sensor emits short pulses of infrared light towards the target. The emitted light reflects off the target surface and travels back to the sensor. The sensor measures the time it takes for the light to make this round trip. Using the known speed of light, the sensor calculates the distance to the target by multiplying the time of flight by the speed of light and dividing by 2, as the light travels to the target and back.

[0004] Reliable object detection is typically performed using remote sensing techniques, such as light detection and ranging (LiDAR, or simply lidar). Remote sensing uses light in the form of pulsed lasers to measure the extent (distance) of objects in a scene. LiDAR can also be used to generate precise 3D information about the shape and characteristics of objects within its field of view. Several types of lidar are currently available for estimating the extent of such objects in a scene of interest. Different types of lidar systems exist, differing in their applications and components. An advanced type of lidar is single-photon lidar (SPL), also known as Geiger-mode lidar, which utilizes the properties of single-photon detectors for depth sensing and 3D mapping. It differs from traditional lidar systems in that it uses sensors capable of detecting individual photons, thus providing high sensitivity and accuracy. Various lidar systems rely on the principle of pulsed emission. SPL operates on the same basic lidar principle but employs a single-photon detector, typically a single-photon avalanche diode (SPAD). The lidar system emits laser pulses toward the target, and the SPAD detector measures the time it takes for each photon to travel to the target and return.

[0005] However, pulsed emission has its limitations in the context of Time-of-Flight (ToF) distance sensing. The illumination period (i.e., the time between pulses during pulsed emission) sets the maximum time of flight that can be measured without ambiguity. To measure distance over a range longer than this maximum unambiguous range, the pulse period can be increased, but this reduces the rate of light detection. Several methods exist to avoid this unambiguous range, including aperiodic pulse sequences or multiple repetition periods. However, using aperiodic pulse sequences or multiple repetition rates requires more complex illumination hardware and reduces the rate of photon detection.

[0006] Therefore, existing ToF distance sensing solutions are ineffective for tracking objects (especially at high speeds) without additional hardware to maximize the utilization of emitted and absorbed light pulses. For this reason, there is still a need to improve ToF distance sensing technology that utilizes pulse emission sensing. Summary of the Invention

[0007] Some implementations aim to provide improved techniques for determining the range of objects in a scene of interest. In this regard, some implementations also aim to provide systems and methods for performing range estimation of objects that exceed the typically unambiguous range of the sensing techniques used. Specifically, some example implementations relate to providing object tracking techniques for scenes where the maximum distance to be measured is greater than the unambiguous range of the underlying sensing system. Some example implementations aim to perform range estimation without requiring additional hardware. In this regard, some implementations involve maximizing the utilization of transmitted and received illumination pulses. To this end, some example implementations utilize modulus-based sensing systems and methods to supplement existing frameworks for range estimation without causing any degradation in the performance characteristics of such frameworks.

[0008] The unambiguous range of a remote sensing system is the maximum range within which a target can be located, ensuring that reflected signals / pulses from that target correspond to the most recently emitted pulse. In most sensing systems, such as those based on single-photon lidar (SPL) technology, the illumination pulse repetition rate setting is the maximum distance that can be unambiguously recovered. Some implementations recognize that one way to extend the maximum recordable depth is by reducing the pulse repetition rate (i.e., increasing the time between consecutive pulse transmissions). However, it is also recognized that slower acquisition speeds limit the number of photons received / detected, which may be insufficient for many critical applications such as tracking moving objects.

[0009] Some implementations are based on the understanding that distance ambiguity in Time-of-Flight (ToF) based object tracking techniques primarily stems from the assumption that the most recent measurement (photon detection) corresponds to the most recent transmitted pulse. Specifically, the light detection time is measured with respect to the emission time of the most recent pulse, enabling direct time-of-flight depth imaging from as few as one photon per scene pixel. However, if the target is outside the unambiguous range, the most recent measurement may not align with the most recent pulse transmission. Attempts to mitigate this by reducing the pulse repetition rate result in a trade-off: a higher repetition rate leads to more light detection, but reduces the depth range that can be measured unambiguously. Some implementations recognize that this trade-off becomes critical in tracking applications requiring high frame rates and long-range tracking capabilities, such as autonomous driving.

[0010] Some example implementations are based on the understanding that pulsed ToF measurements operate as modulus-based sensing systems. While "modulus" refers to the modulo operation of calculating the remainder when one number is divided by another, in the context of depth sensing, modulus-based sensing calculates the remainder with respect to the time of the most recent illumination measurement. Therefore, modulus-based sensing ignores the absolute time of photon detection and wraps the measured distance value within a specific range, thus providing cyclic or repetitive behavior. Given some constraints on the signal type, mathematical theories have been established to guarantee that the signal can be recovered solely from modulus measurements. Even when the conditions of the theory are not precisely met, some implementations recognize that the unambiguous trajectory of a moving target can be recovered solely from modulus (i.e., ambiguous) measurements.

[0011] Some implementations also recognize that treating ToF measurements as templates overcomes the unambiguous range problem, thereby allowing for the use of frequent pulse repetitions to track target objects over long distances. However, some implementations also recognize that template sensing introduces another problem: the need to unfold template distance measurements in applications that allow for discontinuous wrapping. Therefore, some implementations are based on the understanding that discontinuities can be addressed during unfolding by constraining the unfolded distance measurements to lie on a continuous trajectory. Consequently, some implementations leverage the smoothness of typical trajectories to achieve long-distance tracking of objects beyond the unambiguous range.

[0012] One understanding of some implementations is that sensing systems such as SPL naturally acquire modulo time-of-flight measurements. However, following experiments, some implementations have recognized that such SPL-based systems introduce challenges—including random sampling times, multiple noise sources, and absolute distance uncertainty—that have not been addressed by current modulo sensing techniques. Specifically, with modulo SPL measurements, trajectory samples are non-uniformly spaced in time. Furthermore, not all samples are informative due to ambient light, and informative samples themselves may be noisy. Additionally, modulo sensing methods expand the sequence to a constant ambiguity equal to an integer multiple of the modulo threshold.

[0013] To address these challenges, some implementations provide interpolation and denoising techniques that operate directly on the pattern, allowing for the recovery of uniformly denoised pattern samples for use with existing unfolding methods. According to some implementations, uncertainties in the absolute distance to the object are removed based on varying reflectance attenuation. Specifically, the absolute target position is recovered by finding a global offset that best fits the inverse squared decay of reflectance with distance to the distance.

[0014] Therefore, several implementations relate to achieving accurate distance estimation to objects in a scene, even when the object is located outside the unambiguous range of the underlying sensing system. The example implementations described herein are applicable to scenes with extremely low signal-to-background ratios and fast-moving objects. Thus, the framework proposed by these implementations can be integrated into conventional range estimation methods without requiring any special hardware or modifications to the existing architecture of such conventional systems.

[0015] To achieve the above objectives and advantages, some exemplary implementations provide systems, methods, and procedures for tracking the location of target objects.

[0016] For example, some exemplary implementations provide a system for tracking the position of a target object. The system includes circuitry configured to control an illumination source to periodically emit illumination pulses to illuminate the target object. The circuitry is also configured to detect illumination reflected from the target object to estimate a modulo distance sequence to the target object wrapped by the pulse repetition period, and to unroll the modulo distance sequence subject to one or more continuity constraints to output a continuous trajectory of the target object.

[0017] According to some implementations, the circuit includes an illumination source for illuminating a target object, one or more sensors for detecting reflected illumination from the target object, and a computer for performing unfolding and outputting a continuous trajectory of the object.

[0018] According to some implementations, the modulus distance sequence forms a discontinuous trajectory that is non-differentiable at least at one time, while the continuous trajectory of the target object is differentiable at all times. In some implementations, the circuit is also configured to recover the unfolded depth estimate from the modulus depth measurement at each measurement time based on the continuous trajectory of the target object.

[0019] According to some implementations, the expansion of the modulus distance sequence can be performed by any of the following: a first-order differential phase expansion algorithm, a convex optimization algorithm, or an extended Kalman filter that estimates the continuous trajectory from the modulus distance.

[0020] According to some embodiments, a continuous trajectory of a target object defines a change in the target object's distance relative to a lighting source. The circuit can also be configured to determine an offset for shifting the continuous trajectory of the target object to generate a shifted continuous trajectory that tracks the absolute distance of the target object relative to the lighting source. According to some embodiments, the offset can be determined based on prior knowledge of the absolute position of the target object at one or more times during the trajectory. According to some embodiments, the offset can be determined by selecting an offset that results in an optimal fit between the absolute trajectory and the change in reflectivity during the trajectory.

[0021] Some example implementations also provide a method for tracking the position of a target object. The method includes: controlling a lighting source to periodically emit lighting pulses to illuminate the target object, and detecting the lighting reflected from the target object to estimate a modulo distance sequence to the target object wrapped by the pulse repetition period. The method further includes: unpacking the modulo distance sequence subject to one or more continuity constraints to output a continuous trajectory of the target object.

[0022] Some example implementations also provide a non-transitory computer-readable medium having computer-executable instructions stored thereon for performing a method for tracking the position of a target object. The method includes: controlling a lighting source to periodically emit lighting pulses for illuminating the target object, and detecting lighting reflected from the target object to estimate a modulo distance sequence to the target object wrapped by the pulse repetition period. The method further includes: unwinding the modulo distance sequence subject to one or more continuity constraints to output a continuous trajectory of the target object.

[0023] The currently disclosed embodiments will be further explained with reference to the following figures. The figures are not necessarily drawn to scale, but generally focus on illustrating the principles of the currently disclosed embodiments. Attached Figure Description

[0024] [ Figure 1A ]

[0025] Figure 1A A pulse diagram illustrating ambiguity in range estimation is shown according to some example implementations; [ Figure 1B ] Figure 1B A schematic diagram of a system for tracking the location of a target, according to some example embodiments, is shown; [ Figure 1C ] Figure 1C The following are examples of implementations of the method. Figure 1B The system executes methods for tracking the location of objects; [ Figure 2A ] Figure 2A A framework for measuring the absolute depth of a target trajectory beyond the unambiguous range of a pulse-time-of-flight lidar system is shown, according to some example implementations. [ Figure 2B ] Figure 2B A schematic diagram of a pulse time-of-flight lidar system for generating mode measurements is shown according to some example embodiments; [ Figure 2C ] Figure 2CA schematic diagram of a pulsed single-photon lidar system for generating mode measurements is shown according to some exemplary embodiments; [ Figure 3 ] Figure 3 A timing diagram illustrating the relationship between the maximum unambiguous range and time between pulses in a pulse-time-of-flight lidar system, according to some exemplary embodiments, is shown. [ Figure 4A ] Figure 4A Example target trajectories, corresponding modulus measurements, and raw data for modulus measurements in a pulse-time-of-flight lidar system are depicted according to some exemplary implementations. [ Figure 4B ] Figure 4B Example target trajectories, corresponding mode measurements, and raw data for mode measurements in a pulsed single-photon lidar system are depicted according to some exemplary implementations. [ Figure 4C ] Figure 4C The sensitivity of noise mode measurements is shown according to some example implementations; [ Figure 4D ] Figure 4D The remaining global offset ambiguity after the unfolded model measurement is shown according to some example implementations; [ Figure 5A ] Figure 5A A detailed schematic diagram of a single-photon lidar system for generating modal measurements of moving targets, according to some exemplary embodiments, is shown. [ Figure 5B ] Figure 5B A model of a photon detection process for a single-photon lidar according to some example implementations is depicted; [ Figure 5C ] Figure 5C The workflow of an algorithm for estimating depth and reflectivity from single-photon detection using a subspace joint model, according to some example implementations, is shown. [ Figure 6 ] Figure 6 The following are examples of implementations for unfolding by Figure 5A The workflow of the first-order difference phase expansion algorithm for the modulo distance sequence estimated by the system; [ Figure 7 ] Figure 7The following are examples of implementations for unfolding by Figure 5A The workflow of the convex optimization algorithm for the system-estimated modulus distance sequence; [ Figure 8 ] Figure 8 The workflow for expanding a modulus-based distance sequence using an extended Kalman filter is illustrated according to some example implementations; [ Figure 9 ] Figure 9 A block diagram of a system for implementing a system for tracking objects, according to some example embodiments, is shown; and [ Figure 10 ] Figure 10 The scenario illustrates an example use case of a system for tracking objects in a scene, based on some example implementations. Detailed Implementation

[0026] While the accompanying drawings illustrate the embodiments currently disclosed, other embodiments are contemplated, as pointed out in the discussion. This disclosure presents illustrative embodiments by way of representation and not limitation. Those skilled in the art can devise many other modifications and embodiments falling within the scope and spirit of the principles of the embodiments currently disclosed.

[0027] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the following description of the exemplary embodiments will provide those skilled in the art with enabling descriptions for implementing one or more of the exemplary embodiments. Various changes to the function and arrangement of the elements will be contemplated without departing from the spirit and scope of the subject matter disclosed as set forth in the appended claims.

[0028] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that embodiments can be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments. Furthermore, similar reference numerals and names in the various figures indicate similar elements.

[0029] Furthermore, various implementations can be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. Although flowcharts can describe operations as sequential processes, many operations can be performed in parallel or simultaneously. Moreover, the order of operations can be rearranged. A process may terminate upon completion of its operations, but may have additional steps not discussed or included in the diagrams. Furthermore, not all operations in any particular described process may occur in all implementations. A process may correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, the termination of the function may correspond to the function returning to the calling function or the main function.

[0030] Object detection and tracking technologies are widely used in several application areas because understanding the environment surrounding an object is crucial for making informed decisions. For example, in remote sensing applications, it is often desirable to describe the physical characteristics of an area as clearly as possible. In other applications, such as autonomous and semi-autonomous vehicles, timely decision-making is essential for the operation of these vehicles. For all such applications, accurate range estimation of objects within the scene of interest must be performed.

[0031] LiDAR is an increasingly popular sensing modality used in ranging applications ranging from autonomous driving to industrial robotics and lunar navigation. An advanced type of LiDAR is single-photon LiDAR (SPL), also known as Geiger-mode LiDAR, which utilizes the properties of single-photon detectors for depth sensing and 3D mapping. It differs from traditional LiDAR systems in that it uses sensors capable of detecting individual photons, thus providing high sensitivity and accuracy. Various LiDAR systems rely on the principle of pulse emission. SPL operates on the same basic LiDAR principle but employs a single-photon detector, typically a single-photon avalanche diode (SPAD). The LiDAR system emits laser pulses toward the target, and the SPAD detector measures the time it takes for each photon to travel to the target and return.

[0032] However, like any lidar system, pulsed emission has its limitations for SPL in the context of Time-of-Flight (ToF) distance sensing. The illumination period (i.e., the time between pulses during pulsed emission) setting is the maximum time of flight that can be measured without ambiguity.

[0033] Existing solutions fail to provide reliable range estimation for tracking objects beyond the unambiguous range. Therefore, some example implementations specifically address providing object tracking techniques for scenes where the maximum distance to be measured exceeds the unambiguous range of the underlying sensing system. Some example implementations aim to perform range estimation without requiring additional hardware. In this regard, some implementations involve maximizing the utilization of emitted and absorbed illumination pulses.

[0034] The unambiguous range of a remote sensing system is the maximum range at which a target can be located so as to guarantee that the reflected signal / pulse from that target corresponds to the most recently transmitted pulse. The maximum unambiguous range ( ) is the longest range that a transmitted pulse can travel outward and return again between successive transmitted pulses. In other words, is the maximum distance that light can travel back and forth between pulses and still produce reliable information.

[0035] The pulse repetition frequency (PRF) or its reciprocal value, the inter-pulse period T (PRT), and the relationship between them determine the unambiguous range of the radar. Figure 1A shows a pulse diagram illustrating a second scan echo that leads to ambiguity in range estimation according to some example embodiments. A sensing system such as radar or LiDAR transmits a series of pulses towards a target. It can be seen that pulses 101, 103, and 105 are evenly spaced, which means they have a uniform inter-pulse period (e.g., T).

[0036] Let t denote the round-trip time for a pulse to return from transmission to the detector after hitting a target. If t<T (i.e., the time of flight (ToF) of the pulse is shorter than the inter-pulse period), the returned (reflected) signal arrives before the next pulse has been transmitted. For example, if pulse 101 returns to the detector within a distance d1, it corresponds to a scenario where the ToF of pulse 101 is shorter than T (corresponding to a distance of d2). In this case, as shown in Figure 1A , the reflection of pulse 101 arrives before the next pulse 103 is transmitted. In another case, if t=T, the return signal of pulse 101 arrives exactly when the next pulse 103 has been transmitted. In each of these scenarios (i.e., t < T ), the return signal arriving at the detector at or before the transmission of the next pulse 103 can be definitely regarded as the reflection of pulse 101. Therefore, when t < T , there is no ambiguity when estimating the distance from the ToF t.

[0037] However, if t>T, the return signal (reflection) of pulse 101 arrives after the next pulse 103 has been transmitted. For example, referring to Figure 1A , if the reflection of pulse 101 is at If the signal returns to the detector within a certain range, it corresponds to a scenario where the Time-of-Flight (ToF) of pulse 101 is longer than the inter-pulse period T. Therefore, under the common assumption that the detected signal corresponds to the most recently emitted pulse, the returned signal will be considered a reflection of pulse 103 rather than pulse 101. This leads to ambiguity; that is, without any additional information, the system cannot infer whether the returned signal comes from the first pulse 101 or the second pulse 103. When determining the range, some systems may assume that within a certain range... The returned signal detected is a short-range echo of the next pulse 103. Therefore, in this case, the estimated range based on ToF t is incorrect.

[0038] In other words, the maximum range that can be unambiguously estimated by such a system is the range where t is less than T. This maximum range is called the unambiguous range and is given by the following equation:

[0039] in The unfuzzy range in [m] = speed of light [3.10] 8 m / s] T = Pulse Repetition Time (PRT) [s] =Length of the transmitted pulse [s].

[0040] The factor 2 in the above equation takes into account the pulse's journey to the target and back to the detector. The length (pulse width) of the transmitted pulse in the above equation... The signal indicates that the complete echo pulse must be received. Typically, the entire pulse length must be processed first to detect the target. If the transmitted pulse is very short relative to the pulse period, it can be ignored. Ignoring the pulse length, the maximum unambiguous range of any pulse signal radar or lidar can be calculated using the following formula:

[0041] Among them, f p =Pulse Repetition Frequency (PRF) [Hz or s] -1 ].

[0042] Figure 1B A schematic diagram of a system 100 for tracking the position of a target 108, according to some example embodiments, is shown. System 100 includes memory storing different types of data, instructions, and software modules used by the processor 104 of system 100 to track the target 108. System 100 may also include interfaces configured to perform data exchange with other components of system 100 or with other systems.

[0043] It is conceivable that tracking an object such as target 108 can be equivalent to estimating the continuous range of target 108. In this respect, either or both of system 100 and target 108 can be in motion. In some embodiments, neither target 108 nor system 100 can be in motion, and tracking of target 108 can correspond to a periodic estimation of the range to target 108. According to some embodiments, system 100 can obtain a map of the space where target 108 is located and determine the coordinates of target 108 based on the estimated range. Therefore, system 100 can be used in various applications where range estimation is a basic or primary requirement.

[0044] Next reference Figure 1C Describe the operations performed by system 100. Figure 1C A method 150 for tracking target 108 is shown. System 100 controls illumination sources 152 (such as...). Figure 1B An illumination source 107 is configured to periodically emit illumination pulses to illuminate a scene containing a target 108. The illumination source 107 includes an illuminator, such as a laser or a light-emitting diode (LED). The illumination source 107 can provide any suitable illumination option and can be configured accordingly. For example, the illumination source 107 can be a laser source that emits periodic laser pulses. The illumination source 107 has a pulse repetition rate from which it emits illumination pulses.

[0045] The emitted illumination pulse travels to target 108 and is reflected back to system 100, where detector 109 detects the reflected illumination 154. For this purpose, detector 109 can be implemented using one or more sensors and associated optics and electronics. For example, detector 109 may include focusing optics to collect the reflected illumination from target 108 onto one or more photodiodes. According to some embodiments, the target range estimation may not include signal processing for the received reflections, and therefore detector 109 may only count the arrival time of the reflected signal towards the detection of the reflected signal. According to some embodiments, detector 109 may have a focused field of view to filter out noise or scattered illumination arriving at detector 109. Furthermore, in some embodiments, detector 109 may apply one or more filters to the detected signal to remove outliers or noise. Since the emitted pulses are sequential, detector 109 also performs a detection sequence of reflected illumination.

[0046] Processor 104 can be communicatively coupled to illumination source 107 and detector 109. Detected reflections are obtained by processor 104 via interface 106 and processed to estimate a modulo distance sequence of the target object enveloped by the pulse repetition period of illumination source 107. The modulo distance sequence forms a discontinuous trajectory that is nondifferentiable at least at one moment. The processor then expands 156 the modulo distance sequence to target 108 enveloped by the pulse repetition period of illumination source 107. The expansion can be performed in any suitable manner known in the art. For example, processor 104 can utilize a suitable expansion algorithm (such as a first-order differential phase expansion algorithm or a convex optimization algorithm) to expand the enveloped modulo distance sequence. Alternatively, processor 104 can utilize an extended Kalman filter to expand the enveloped modulo distance sequence.

[0047] By performing unfolding mode measurements, processor 104 obtains a continuous trajectory of target 108, which can be output 158 ​​for further processing. The continuous trajectory defines the variation in distance of target 108 with respect to illumination source 107 or detector 109. The continuous trajectory of the target is differentiable at all times. According to some embodiments, processor 104 can use the continuous trajectory to recover an estimate of the unfolding depth of target 108 at each measurement / detection time. According to some embodiments, processor 104 can also perform offset estimation to shift the continuous trajectory of target 108. The shift results in a shifted continuous trajectory that tracks the absolute distance of target 108 with respect to illumination source 107.

[0048] Pulsed ToF lidar measures distance based on the time difference between the emission of an illumination pulse and the detection of the back-reflected echo. Modulated pulse lidar uses the same optical mechanism but employs additional processing to reconstruct the absolute distance measurement of a continuous trajectory. Figure 2AA framework for absolute depth measurement of a target trajectory beyond the unambiguous range of a pulse-time-of-flight lidar system, according to some example embodiments, is shown. This framework describes the process of estimating a target trajectory using a modal-pulse ToF lidar system 200. A pulsed light source 202 illuminates a target 201 with a sequence of laser pulses. In this case, a detector timing clock is triggered to start so that the detection time can be measured. Detector 204 detects 203 backreflected light from the target to generate an electrical signal related to the intensity of the detected light, which is then sampled and digitized 205 by an analog-to-digital converter (ADC) 206. Based on the digitized signal, processor 208 estimates 207 the delay relative to the most recent pulse emission time and stores this estimate in memory 210. In this regard, any suitable technique for calculating the delay from the digitized light intensity can be utilized, including finding the peak of the backreflected pulse, calculating the centroid of the backreflected pulse, or using a matched filter to calculate the maximum likelihood delay. Based on the delay estimation sequence stored in memory 210, processor 208 can further estimate the full trajectory by unfolding 209 the modal measurement and recovering the global offset. Finally, the estimated trajectory is output 211 via interface 212. For example, the estimated trajectory can be displayed on the display interface.

[0049] Figure 2B A schematic diagram of a pulse time-of-flight (ToF) lidar system 230 for generating mode measurements is shown according to some example embodiments. In the pulsed ToF lidar system 230, the raw measurement is time-resolved light intensity. Measurement begins when a pulse is emitted from a pulse source 232, and the light intensity measured by a detector 234, such as a photodiode, is recorded over time. In principle, a back-reflected pulse from a target 231 causes a peak in intensity measured by an ADC 236. Assuming the recorded peak is due to reflection from the irradiated target 231, ToF estimation requires peak fitting or other peak-finding methods to obtain a trajectory estimate 241.

[0050] More advanced lidar systems allow for different modes of measurement. For example, a single-photon lidar (SPL) emits laser pulses toward a target, and a SPAD detector measures the time it takes for each photon to travel to the target and return. However, even with SPL, pulse emission has its limitations in the context of Time-of-Flight (ToF) distance sensing. The illumination period (i.e., the time between pulses during pulse emission) is set as the maximum time of flight that can be measured without ambiguity.

[0051] Figure 2CA schematic diagram of a pulsed single-photon lidar (SPL) system 250 for generating mode measurements, according to some example embodiments, is shown. A pulsed laser 252 generates periodic pulses of laser illumination to illuminate a target 251. A single-photon detector (SPD) 254 may include one or more of a single-photon avalanche diode (SPAD), a superconducting nanowire photodetector (SNSPD), a photomultiplier tube (PMT), or other SPDs. Instead of directly measuring time-resolved light intensity, the pulsed SPL system 250 records the timestamps of each photon detection using a time-to-digital converter (TDC) 256. The TDC can be implemented by suitable circuitry that generates a digital output corresponding to each photon detection timestamp. The TDC 256, together with the SPD 254, thus generates raw data as a sequence of photon detection timestamps. In the case of multiple illuminations, and assuming that the target 251 is static or moves slowly enough, the SPL can collect enough photon detections to statistically recover the light intensity equivalent to the full waveform from a single pulse. Then, peak lookup methods or other statistical estimators are used for the set of photon detections to produce trajectory estimates 253.

[0052] The advantages of SPL are twofold. First, the sensitivity to individual photons means that even weaker light can be detected from reflections from lower-power laser sources or weaker (e.g., more distant or darker) targets. Second, the timestamps of individual photons can be recorded at levels as low as picoseconds, allowing for shorter pulse durations and thus achieving higher precision than typically achieved with classical ToF lidar.

[0053] One property of all pulse ToF systems (including classical and single-photon lidar) is that the maximum unambiguous range is determined by the time interval between pulses. Figure 3 A timing diagram illustrating the relationship between the maximum unambiguous range and time between pulses in a pulse-time-of-flight lidar system, according to some example embodiments, is shown. Figure 3 As shown, the measurement of the real ToF 310 begins at the moment of pulse emission and records the delay of pulse emission time 302 relative to the first pulse (the peak position of classical ToF or the photon detection time of SPL). However, when multiple pulses are emitted before the reflection of the first pulse is recorded 304, a delay measurement is still performed relative to the emission time of the most recent pulse 308 (observed ToF 306). Therefore, the time between pulses sets the unambiguous range of the lidar.

[0054] Let the time between pulses (the period between pulses) be... This makes the pulse repetition frequency of 1 / 2. Pulse duration Significantly shorter than the time between pulses. The maximum Time-of-Flight (ToF) that can be measured without ambiguity is... The division by 2 takes into account the fact that light must travel to the target and return. Given the speed of light... m / s, the maximum distance that can be measured without fuzziness (i.e., the unfuzzy range) is .

[0055] Therefore, the simultaneous presence of multiple pulses during flight leads to range ambiguity, and most pulsed lidar systems thus limit the pulse repetition rate to prevent ambiguity for a specific maximum range of interest. However, reducing the pulse rate also reduces the rate at which range is sampled, which is important when tracking moving objects. Furthermore, for SPL, more pulses mean more photon detection, resulting in more accurate depth estimation. Therefore, there is a trade-off between better depth accuracy and unambiguous depth estimation.

[0056] Due to the periodic illumination sequence, pulsed lidar systems such as SPL naturally acquire mode time-of-flight measurements. If the distance z to the object is greater than the unambiguous range (i.e., If ), then another pulse will be emitted before the previous pulse is detected. Let ,in Then, the distance z exceeds the unfuzzy range. The delay relative to the most recent pulse time measurement will correspond to the distance. It is a modulo measurement. In fact, for any integer k, if the distance is... The delay will then correspond to the distance. This indicates the inherent ambiguity caused by modulus measurement.

[0057] Figure 4A An example target trajectory 400, corresponding mode measurements, and raw data for mode measurements in a pulse-time-of-flight lidar system are depicted according to some exemplary implementations. As an example, if the repetition frequency... Then the non-fuzzy range is It can be observed that the target distance changes over time, and when its trajectory crosses 400... When the delay is an integer multiple of the time, the observed measurements are wrapped because the delay is estimated relative to the most recent pulse emission. At T=19s, the original measurement 401 shows the delay when trajectory 400 crosses the time. When the light intensity is an integer multiple of the time, it appears to wrap around the object. This wrapping also causes an object at time T=58s shown in measurement 403 to appear slightly closer than at T=80s shown in measurement 405, even though the actual distance 402 is closer at 80s than at 58s. In fact, at T=80s, the modulus position 404 overlaps with the actual position 402.

[0058] Figure 4BIt shows the relationship with Figure 4A The same example trajectory is used, but the original measurements were performed using a single-photon lidar system, and the photon timestamps were accumulated in the photon counting histograms 421, 423, and 425 over short periods of time.

[0059] An important note about pulsed ToF is that, in addition to measuring distance, the raw measurement also encodes information about the target's reflectivity, which varies with distance. Specifically, such as... Figure 4A As shown, the height of the pulse waveform encodes how much attenuation has occurred; that is, the intensity is inversely proportional to the square of the distance from the target. At closer points in the target trajectory 400, the received signal amplitude is higher. Similarly, in Figure 4B In this case, the rate of photon detection is inversely proportional to the squared distance from the target.

[0060] Within the scope of this disclosure, modulo measurement means that for any integer k, if the distance is Then the same modulus measurement will be recorded. Solving the uncertainty of a modulus measurement requires estimating the unknown integer k. Although this information may be lost individually, for a series of modulus measurements, the unknown integer for each measurement can be recovered. Recovering the signal from the modulus measurement requires two types of considerations: 1) local and 2) global.

[0061] (Partial restoration - expanded)

[0062] The local assumption effectively enforces a smoothing constraint, i.e., how quickly the modal signal can change. For example, most studies on modal sampling assume a band-limited, finite-energy signal. For modal lidar, the local assumption constrains how quickly the trajectory of a moving target can change, which is limited by physics. Since an object cannot instantly jump from one location to another, it must move along a continuous trajectory with some velocity constraints.

[0063] Given its similarity to phase expansion, applying local assumptions for mode recovery is a process called "expansion." Various methods exist for expanding mode signals. For example, the classical method of phase expansion can be applied, which assumes that the first-order difference between adjacent samples is less than the dynamic range. Half of it. The infinite sampling framework summarizes the phase expansion to use higher-order differences, which is theoretically guaranteed for noise-free band-limited signals, even if the first-order difference is greater than half. Other methods that enforce band-limiting conditions in the spectral domain and in consistency with time-domain measurements via optimization problems.

[0064] Figure 4C The unfolding process and an example of how it is sensitive to noise are shown. The true target trajectory 452 is a smooth function of distance over time. Again, the repetition frequency... produce The unambiguous range is determined by estimating the delay of light intensity relative to the time of the most recent illumination pulse. The mode measurement 454 of the true trajectory 452 is determined by estimating the delay of light intensity relative to the time of the most recent illumination pulse. This estimation is inherently noisy due to pulse width, background light, and non-ideal detectors; therefore, the mode measurement is affected by both the mode in the delay and the uncertainty. To remove the mode effect, an expansion algorithm such as first-order differential phase expansion can be used, which sequentially adds or subtracts based on the relative difference between adjacent samples. The unfolding method can be sensitive to noise, as shown in the unfolded trajectory 456. The unfolded trajectory 456 typically follows the true trajectory up to T=58s. However, at T=58s, noise causes the unfolding algorithm to incorrectly add... Exceeding the necessary level leads to incorrect shifts in all subsequent unfolding measurements. Simultaneously, forcing trajectory smoothness for denoising and unfolding can improve unfolding results and avoid incorrect jumps.

[0065] (Global Recovery - Offset Estimation)

[0066] Even if the analog signal is correctly expanded, the expanded measurement may still have global integer ambiguity. Figure 4D Multiple possible signals 474, 476, and 478 are shown, each resulting in a different offset from the same modulus measurement 472. Given only the modulus measurement 472, it is impossible to determine which of the signals 474, 476, or 478 represents the true trajectory. Therefore, additional constraints or measurement information are required to recover the global offset and thus the absolute distance.

[0067] (Trajectory recovery from LiDAR measurement)

[0068] Trajectory recovery from mode lidar measurements first involves expanding the mode measurements by applying local constraints, and then resolving the remaining integer offset uncertainties using global assumptions.

[0069] In this regard, existing expansion methods can effectively expand mode lidar measurements. Alternatively, expansion methods can be tailored to specific problems of trajectory estimation. For example, the motion of a target in one dimension can be described by a motion model such as a constant velocity model or a Singer model, which assumes motion with a constant velocity unless manipulation occurs. Manipulation is characterized by acceleration and assumes that the deviation from normal motion is a Gaussian random process. Estimating the target's state vector (position, velocity, acceleration) can be performed using a Kalman filter, which predicts the next state given the current state and updates it with new measurements. Because the measurements are nonlinear due to the mode, variations on the Kalman filter can be used. For example, the mode measurement can be embedded in a periodic signal such as a sine or cosine, and then an expanded Kalman filter (EKF) can be used to linearize the measurement. Alternatively, unscented Kalman filters or particle filters can be used to track how nonlinearity affects the prediction. If the parameters of the motion model are unknown, a single Kalman filter may not be able to accurately predict the trajectory from the sample. Conversely, in addition to performing the expansion, methods such as interactive multiple models or variational Bayesian Kalman filters can be used to estimate the parameters of the motion model.

[0070] For global offset recovery, various constraints or additional information can be used to recover absolute distance measurements. One possibility is that the object's initial position is at a known distance, for example, the first sampling time is within the unambiguous range. Then, even if the object later in the trajectory exceeds the unambiguous range, the constraint can be used to resolve integer ambiguities. Another approach is to utilize reflectivity information embedded in the raw data. Specifically, the height of the pulse waveform or the rate of photon detection encodes how much attenuation occurs due to the travel distance of light scattered from the target. Radial light attenuation is a function of the absolute distance to the object (independent of the modulus distance), so the reflectivity variation relative to a reference measurement at a known distance can be used to resolve global integer uncertainties. Furthermore, reflectivity information can be used even if no reference measurement is taken at a known distance. Conversely, since reflectivity varies across the entire target trajectory, integer offsets can be solved, leading to a best-fit between the reflectivity variation and possible position variations.

[0071] Figure 5AA detailed schematic diagram of an example mode system 500 using a single-photon lidar (SPL) is shown. According to some embodiments, the SPL may also be referred to as a Geiger-mode lidar, which detects as few as one photon count at a time. A pulsed laser 501 illuminates a target object 502 that can move in one dimension (such as on a stage 504). A SPAD detector 505 observes the same optical axis 507 as the pulsed laser 501 via a beam splitter 503. The light arriving at the SPAD detector 505 is focused by a lens 509 and spectrally filtered by a filter 511 to allow only the laser wavelength to reach the detector 505. Measurements of the pulse sequence are time-series photon detection data recorded using a time-to-digital converter (TDC) 513. The TDC 513 thus provides raw data as a sequence of photon detection timestamps.

[0072] Typically, the photons detected by the SPAD detector 505 are random observations of the full waveform signal, including back-reflected pulses, noise, and other degradations such as interference or multipath. Due to the statistical nature of single-photon detection, not every pulse must follow a photon detection. Alternatively, depending on the single-photon detector configuration, several pulses may be followed by multiple photon detections. The computer 515 accumulates photon detections over time, resulting in a time histogram of approximate full waveform measurements. In lidar acquisition of static targets, the depth estimator can use the entire photon detection timestamp sequence. However, for moving targets, individual depth estimates are required for smaller time units to accurately track target motion. Therefore, the acquisition time that acquires all photon detection timestamps can be subdivided into sub-acquisition sequences corresponding to portions of all emitted laser pulses. Each sub-acquisition further comprises multiple photon detection times.

[0073] For each sub-acquisition, the modulus depth estimate of the target can be determined based on the photon detection time within the corresponding sub-acquisition. In this regard, according to some embodiments, the modulus depth estimate corresponding to a sub-acquisition may include the centroid of the photon detection time within that sub-acquisition. According to some embodiments, the modulus depth estimate corresponding to a sub-acquisition may include a maximum likelihood depth estimate, which is implemented as a log-matched filter of the photon detection time within that sub-acquisition. In still other embodiments, the modulus depth estimate in a sub-acquisition is determined by the union of subspace decompositions, which finds the best fit of the detection time to the model, assuming that the detected light consists of a uniform background light and a single reflection peak.

[0074] Figure 5B Described Figure 5A The system's 500 acquisition model, and will be combined with Figure 5A Described. The pulsed laser 501 constitutes the emission time... Total of repetitions A lighting system with one pulse, of which Therefore, the time between pulses is And the pulse repetition frequency is Pulse duration Significantly shorter than the time between pulses. The laser pulse shape is... And it is usually distributed by a Gaussian distribution (with root mean square duration) The Gaussian distribution is approximated.

[0075] Each photon detected by the SPAD detector 505 is measured using an absolute detection time. ( Add a timestamp, which contains the total number of photons detected. It is a random variable. The absolute detection time of this group is expressed as... However, it is impossible to determine which illumination pulse generated the detected photon. In the absence of depth aliasing (i.e., at the target depth...), Under the conventional assumptions, the relative detection time Compared to the most recent pulse irradiation time (i.e., (It was recorded.)

[0076] Target

[0077] Target 502 occupies a position at time t. The object is assumed to move along one dimension. This means the same object is visible throughout the entire measurement duration (i.e., no occlusion occurs). Time-of-Flight (ToF) It is the time it takes for pulse n to reach the object. and location Illuminate the target.

[0078] Assuming negligible beam divergence, equal laser power impacts target 502 regardless of its distance from laser 501. The reflectivity of the object (including attenuation effects from surface albedo, viewing angle, detector efficiency, and attenuation) is inversely proportional to the target distance.

[0079] Considering the reference distance Reference reflectance measured at [location] For a target with Lambertian surface albedo, the reflectivity decreases inversely with the square of the target distance. Therefore, for the nth pulse, the reflectivity is... .

[0080] (Measurement)

[0081] Single-photon lidar detection is a stochastic process. As a realization of a Poisson process with a rate function given by the following equation, the photon arrives at the detector.

[0082] Where b is the background intensity of the detection, which is a combination of ambient light and detector dark count. Assuming approximately one (possibly wrapped) pulse in each repetition cycle, the photon detection rate is given as...

[0083] Assume the signal rate is defined as The background rate is defined as Furthermore, the signal-to-background ratio (SBR) is defined as... Note that, due to mode behavior, the photons detected in period n may have initially been generated by laser 501 in pulses. Launch. It can be assumed that SPL has... Operating at low throughput allows non-ideals such as dead time to be ignored, and the detection sequence can be considered to have a rate function. The non-uniform Poisson process.

[0084] (Single-photon lidar recovery)

[0085] Each individually detected photon can be a signal photon 521 caused by a signal or a background photon 523 caused by a background, but it is not possible to inform the true source in isolation. Instead, according to some implementations, if it is assumed that the target motion occurs over a period of time... of If the detections are negligible within a short time frame (sub-acquisition), multiple detections can be accumulated into histogram 525 within each time frame. A single modulus depth measurement is performed for each time frame via preprocessing, then the modulus depth sequence is unfolded and the global integer offset is estimated.

[0086] The m-th frame includes There are 10 photons, which are roughly the same in distribution, and It is a random variable. Modulus depth estimation is performed by applying an approximate log-matched filter.

[0087] in, It contains the index of the detection in the m-th frame.

[0088] Estimated sequence This represents a set of modulus measurements, with an estimate for each time frame (sub-acquisition).

[0089] Reflectance can also be estimated based on detections in each time frame / sub-acquisition. For example, some implementations estimate reflectance based on photon counts. According to some implementations, reflectance estimation in each sub-acquisition can be performed based on photon detection time. If a constrained maximum likelihood estimator for reflectance is obtained by calibrating with a known background rate B, then...

[0090] in, Frame time The number of lighting cycles in the circuit.

[0091] Figure 5C This demonstrates how reflectivity and depth can be jointly estimated alternatively via a hypothetical 1-sparse signal plus a subspace union method with an unknown background level. Consider a signal dictionary containing N possible mode-shifted illumination pulses on a uniform grid. Then, the histogram of the detection time within the sub-acquisition... The intervals are distributed as follows:

[0092] in It is the length of 1 - Vector. Since it is assumed that the target is opaque and quasi-static within the sub-acquisition, v is 1-sparse, i.e., it has a single non-zero element. The goal is then to identify the union of the subspaces that best represent the 1-sparse signal and the uniform background. Definition , And select some tolerance As a stopping criterion. Threshold operator or hinge function. Set negative values ​​to zero. yes With support, and Only keep the maximum value of x. Then, Algorithm 550 can be used to recover... The first N elements correspond to a 1-sparse signal vector, and the last element is an estimate of the background b. The modulo depth estimation corresponds to... The shift of non-zero elements, and the reflectivity is the magnitude of non-zero elements.

[0093] Based on these modulus depth measurements, local recovery can be achieved using any suitable unrolling algorithm (e.g., first-order phase unrolling, higher-order difference, out-of-bandwidth residual reconstruction (B2R2), etc.) or a custom trajectory recovery algorithm using a Kalman filter. Estimated sequence This represents a set of expanded measurements.

[0094] Global integer offset recovery can be achieved by finding the best-fitting global trajectory that takes into account the change in reflectivity as a function of distance. For example...

[0095] in, It finds the integer k that best accounts for radial decay when using the squared error loss.

[0096] value It could be that the target is at a known distance. Calibration detection rate at the location. Alternatively, value... It can be estimated from the data itself. .

[0097] Figure 6 The following are examples of implementations for unfolding by Figure 5A The workflow of the first-order difference phase expansion algorithm 600 for estimating the modulus distance sequence of the system is described. For each sample m, the difference in the sample is calculated. .if If the value is greater than half of the unambiguous range, then an encapsulation is assumed to have occurred. An integer shift j is updated to counteract the encapsulation, and the unwrap measurement is given as... First-order phase expansion is extremely simple, requiring neither prior knowledge of trajectory characteristics nor uniform temporal intervals for samples, and it has no hyperparameters.

[0098] Figure 7 The following are examples of implementations for unfolding by Figure 5A A convex optimization algorithm for the system-estimated modulo distance sequence ( ) The workflow of 700. Algorithm 700 assumes that the input trajectory u is band-limited and has a known bandwidth. And having finite energy, this means the trajectory will begin and end at a distance of 0. The pattern can be written as... ,in This is referred to as the residual signal. Algorithm 700 is designed to extract from the sample... The first-order difference is used to recover the original signal. Recovery utilizes both spectral and temporal information. In the Fourier domain, any frequency content of the pattern exceeding the known bandwidth can only be attributed to the residual signal, leading to linear equations. , where V is the partial Discrete Fourier Transform (DFT) operator for frequencies outside the signal bandwidth. From recover It is a linear inverse problem, which can be solved by encouraging... The sparsity penalty is regularized during recovery. The full residual can then be recovered via cumulative summation, and then subtracted from the sample to generate the unfolded trajectory. A soft thresholding operator is used to achieve 700. It promotes sparsity An approximation operator for the norm. The algorithm also uses tolerance. The maximum number of iterations, maxIter, is used as the stopping criterion.

[0099] Figure 8 The workflow for expanding a modulo-based distance sequence using the extended Kalman filter (EKF) algorithm 800, according to some example implementations, is illustrated. The expansion can be performed using the extended Kalman filter if some properties of the target motion are known in advance. It is assumed that the target motion, given by position z(t), velocity v(t), and acceleration a(t)s, is described by a Singer trajectory, which assumes the target has a constant velocity unless manipulation occurs. The target manipulation variable is then the acceleration. This leads to a deviation from normal motion and is assumed to be in the target state vector. time Location and autocorrelation function Time-dependent. The time between a given measurement. The equation of motion is defined as

[0100] Wherein, the target state transition matrix is

[0101] And non-uniform drive input It is a discrete-time white noise sequence, whose statistics depend on .

[0102] The extended Kalman filter can be used to predict a sequence of state vectors based on previous state vector estimates, updated by measurements, and taking into account the modulus. Let the state transition and measurement model be as follows:

[0103] Here, the model parameter vector is ; It depends on , , covariance Zero-mean multivariate Gaussian; yes The position of the element; about Application model; noise measurement It has variance The zero-mean univariate Gaussian; and the measurement is embedded in a pair of sine curves, such as

[0104] Note that because sine waves have a period... Therefore, embedding is unknown for the module, such as .

[0105] The EKF algorithm 800 embeds the modulus measurement into a sine curve, which preserves the approximate Gaussian error but also reflects the periodicity of the modulus measurement. It utilizes each new modulus measurement... EKF updates the previous estimate of the unfolding at the current time step. The estimate of the unfolding... It is a state vector Positional elements.

[0106] Figure 9 A block diagram of a system for implementing a lidar system for tracking the position of an object, according to some example embodiments, is shown. The controller 911 includes a processor 940, a computer-readable storage device 912, a storage device 958, and a user interface 949 connected via a bus 956, and optionally a display 952 and a keyboard 951. For example, the user interface 964, communicating with the processor 940 and the computer-readable storage device 912, acquires photon detection data and stores it in the computer-readable storage device 912 upon receiving control input from the user or input from the surface of the user interface 957 or the keyboard 953.

[0107] Computer 911 may include a power supply 954, which may optionally be located externally to computer 911, depending on the application. A user input interface 957, suitable for connection to a display device 948, may be linked via bus 956. The display device 948 may include a computer monitor, television, projector, or mobile device, etc. A network interface controller (NIC) 934 is adapted to be connected to network 936 via bus 956, wherein output data or other data may be presented to external devices, such as third-party display devices, third-party imaging devices, and / or third-party printing devices external to computer 911.

[0108] Still referencing Figure 9Output data or other electronic data can be transmitted through the communication channel of network 936 and / or stored in storage system 958 for storage and / or further processing. Furthermore, time-series data or other data can be received wirelessly or hardwired from receiver 946 (or external receiver 938), or transmitted wirelessly or hardwired via transmitter 947 (or external transmitter 939), both of which are connected via bus 956. Computer 911 can be connected to external sensing device 944 and external input / output device 941 via input interface 908. For example, external sensing device 904 may include sensors that collect data. Computer 911 can be connected to other external computers 942. Output interface 909 can be used to output processed data from processor 940. Note that user interface 949, communicating with processor 940 and non-transitory computer-readable storage medium 912, acquires various types of data upon receiving input from the user on the surface of user interface 949 and stores it in non-transitory computer-readable storage medium 912.

[0109] Figure 10 A depiction of some example implementations is shown, such as Figure 5A The example use case scenario for the SPL system of system 500 is as follows: A vehicle 1002 moving on road link 1004 can connect to base station 1006. The road link may have one or more obstacles 1008, and the goal may be to cross the road link 1004 without colliding with the obstacles 1008. For this, it is necessary to determine an accurate estimate of the depth / range of the obstacles 1008 so that the route of vehicle 1002 can be planned to avoid the obstacles 1008.

[0110] Vehicle 1002 can be a manually driven vehicle, a semi-autonomous vehicle, or a fully autonomous vehicle, and can be configured to communicate with base station 1006. In this regard, vehicle 1002 can be equipped with suitable components to perform remote sensing, data communication, and data processing. For this purpose, it is conceivable that the vehicle can be equipped with an onboard single-point (low photon count) lidar system, such as the lidar system previously described in this disclosure. The lidar system's transmitter emits radiation pulses (shown as solid lines) toward obstacle 1008 and receives reflections of the emitted pulses from the scene ahead (shown as dashed lines). Since vehicle 1002 may be moving, conventional techniques may not be able to accurately infer the depth estimate of obstacle 1008.

[0111] The controller of vehicle 1002 can invoke the system to perform position tracking of obstacle 1008. The system may be entirely or partially on vehicle 1002. In embodiments where the system is partially on vehicle 1002, the calculation steps leading to range estimation may be performed via edge computing on base station 1006. In some example embodiments, vehicle 1002 may determine the correct range of obstacle 1008 via an SPL system according to example embodiments previously described in this disclosure, thereby determining the geographical location of obstacle 1008, for example, using map data. Thus, the onboard controller of vehicle 1008 may determine maneuvers to divert vehicle 1008 to avoid a collision with obstacle 1008. In some example embodiments, vehicle 1002 may additionally or alternatively transmit at least the geographical location of obstacle 1008 to base station 1006 to update the map of the area where road link 1004 exists.

[0112] Although the example use cases are described with reference to road vehicles, the example implementations described herein can be applied to any type of vehicle, such as railcars or spacecraft moving in one dimension. Other examples of some implementations use industrial robots, computer vision systems, and camera autofocus, among others.

[0113] The modal SPL method described in this disclosure is superior to other methods known in the art for range and trajectory estimation. For example, some embodiments recognize that the trajectory estimation of the modal SPL system described in several embodiments is more accurate than that of the full-range SPL system because the full-range system emits fewer pulses and is therefore more sensitive to background noise, and it samples the trajectory less frequently.

[0114] It can be noted that reflectance estimates are typically highly noisy, and therefore may be insufficient for estimating trajectories. However, some implementations recognize that reflectance is important for estimating which integer... The optimal correspondence is to the estimated reflectance value and the uncertainty. The unfolded trajectory is useful for both. For example, when the reflectivity is known at a certain distance... place as When k is used, different values ​​of k produce significantly different trajectories, so choosing the correct k that best matches the reflectance estimate is easy.

[0115] For different operating conditions, various implementations achieve the following: for a sufficiently high SBR, the modulo SPL achieves better trajectory estimation than a full-range lidar system. However, as the SBR decreases, the higher relative noise level degrades the unfolding and offset recovery performance.

[0116] The above description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with enabling descriptions for implementing one or more exemplary embodiments. Various changes to the function and arrangement of the elements will be contemplated without departing from the spirit and scope of the subject matter disclosed as set forth in the appended claims.

[0117] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that embodiments can be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary details. In other cases, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, the same reference numerals and names in the various figures indicate the same elements. Furthermore, the various embodiments may be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. Although flowcharts may describe operations as sequential processes, many operations may be performed in parallel or simultaneously. Furthermore, the order of operations may be rearranged. A process may terminate upon completion of its operations, but may have additional steps not discussed or included in the figures. Furthermore, not all operations in any particular described process may occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, the termination of the function may correspond to the function returning to the calling function or the main function.

[0118] Furthermore, implementations of the disclosed subject matter can be carried out, at least partially, manually or automatically. They can be performed, or at least assisted by, using machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, for manual or automatic implementation. When implemented in software, firmware, middleware, or microcode, program code or code segments performing the necessary tasks can be stored in a machine-readable medium. A processor can perform the necessary tasks. The various methods or processes outlined herein can be encoded as software executable on one or more processors employing any of various operating systems or platforms. Additionally, such software can be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine. Typically, in various implementations, the functionality of program modules can be combined or distributed as needed.

[0119] Embodiments of this disclosure can be specifically implemented as a method, examples of which have been provided. The actions performed as part of the method can be ordered in any suitable manner. Therefore, embodiments in which actions are performed in a different order than those shown can be constructed, which may include performing some actions simultaneously, even if they are shown as sequential actions in the illustrative embodiments. Furthermore, the use of ordinal terms such as “first” or “second” to modify claim elements in the claims does not imply any priority, precedence, or order of one claim element relative to another claim element, or the temporal order of the actions of the method, but is merely used as labels to distinguish one claim element with a specific name from another element with the same name (but using ordinal terms) to differentiate claim elements. Although this disclosure has been described with reference to certain preferred embodiments, it should be understood that various other adjustments and modifications can be made within the spirit and scope of this disclosure. Therefore, aspects of the appended claims cover all such variations and modifications that fall within the true spirit and scope of this disclosure.

Claims

1. A system for tracking the position of a target object, the system comprising: The circuit is configured as follows: The lighting source is controlled to periodically emit lighting pulses to illuminate the target object; Detect the illumination reflected from the target object to estimate the modulo distance sequence to the target object, which is wrapped by the pulse repetition period; as well as Expand the modulo distance sequence subject to one or more continuity constraints to output the continuous trajectory of the target object.

2. The system according to claim 1, wherein, The modulo distance sequence forms a discontinuous trajectory that is not differentiable at least at one moment, wherein the continuous trajectory of the target object is differentiable at all moments.

3. The system according to claim 2, wherein, The circuit is also configured to recover an unfolded depth estimate from the modulus distance sequence at each measurement time based on the continuous trajectory of the target object.

4. The system according to claim 3, wherein, In order to expand the modulus distance sequence, the circuit is configured to perform a first-order differential phase expansion algorithm.

5. The system according to claim 3, wherein, To expand the modulus distance sequence, the circuit is configured to perform convex optimization Lasso- algorithm.

6. The system according to claim 3, wherein, To expand the modulus distance sequence, the circuit is configured to execute an extended Kalman filter that estimates the continuous trajectory based on the modulus distance.

7. The system according to claim 1, in, The continuous trajectory of the target object defines the change in the distance of the target object with respect to the lighting source, and The circuit is further configured to determine an offset for shifting the continuous trajectory of the target object to generate a continuous trajectory that tracks the shift of the target object with respect to the absolute distance of the illumination source.

8. The system according to claim 7, wherein, The circuit is configured to determine the offset based on prior knowledge of the absolute position of the target object at one or more moments during the continuous trajectory.

9. The system according to claim 7, wherein, The circuit is configured to determine the offset by selecting an offset that matches the change in radial attenuation observed during the continuous trajectory.

10. The system according to claim 9, wherein, The circuit is also configured to collect prior knowledge of the absolute position of the target object as supplementary data.

11. The system according to claim 1, wherein, The circuit includes a single-photon detector to detect the illumination reflected from the target object and to generate raw data as a photon detection timestamp sequence over the total acquisition time.

12. The system according to claim 11, wherein, The total acquisition time is divided into sub-acquisition sequences, each of which is defined by multiple photon detection times. The circuit is configured to generate a mode depth estimate based on the photon detection time within each sub-acquisition sequence to estimate the mode distance sequence of the target object.

13. The system according to claim 12, wherein, The modulus depth estimation in the sub-acquisition of the sub-acquisition sequence includes at least one of the following: The centroid of the photon detection time; or The maximum likelihood depth estimation is implemented as a log-matched filter for the photon detection time.

14. The system according to claim 12, wherein, The modulus depth estimate in the sub-acquisition is determined by the union of the subspace optimization models.

15. The system according to claim 11, wherein, The total acquisition time is divided into sub-acquisition sequences, each of which is defined by multiple photon detection times. The circuit is configured to generate a reflectance estimate based on the photon detection time within each of the sub-acquisition sequences.

16. The system according to claim 15, wherein, The reflectance estimate in each sub-acquisition of the sub-acquisition sequence includes at least one of the following: Maximum likelihood estimator of the number of photon detections; Based on the maximum likelihood estimator of the photon detection time; or The magnitude of the sparse vector recovered using subspace union optimization.

17. A method for tracking the position of a target object, the method comprising the following steps: The lighting source is controlled to periodically emit lighting pulses to illuminate the target object; Detect the illumination reflected from the target object to estimate the modulo distance sequence to the target object, which is wrapped by the pulse repetition period; as well as Expand the modulo distance sequence subject to one or more continuity constraints to output the continuous trajectory of the target object.

18. The method according to claim 17, wherein, The modulo distance sequence forms a discontinuous trajectory that is not differentiable at least at one moment, wherein the continuous trajectory of the target object is differentiable at all moments.

19. The method of claim 18, further comprising recovering an unfolded depth estimate from the modulus distance sequence at each measurement time based on the continuous trajectory of the target object.

20. A non-transitory computer-readable medium storing instructions executable by a computer to perform a method for tracking the location of a target object, the method comprising: The lighting source is controlled to periodically emit lighting pulses to illuminate the target object; Detect the illumination reflected from the target object to estimate the modulo distance sequence to the target object, which is wrapped by the pulse repetition period; as well as Expand the modulo distance sequence subject to one or more continuity constraints to output the continuous trajectory of the target object.

Images