METHOD AND SYSTEM FOR OPTIMIZING THE DISTANCE DETECTION OF LiDAR CAMERAS USING A MACHINE LEARNING MODEL

Abstract

A method (300) and a system (100) for optimizing distance sensing using a light detection and distance measurement (LiDAR) camera (102) are described. The method (300) comprises receiving a time-correlated single-photon counting histogram (TCSPC) representing the distance of an object within a field of view of the LiDAR camera (102), extracting a set of feature vectors from the TCSPC histogram, filtering the extracted set of feature vectors based on at least one predefined criterion, and further comprising training a machine learning (ML) model (108) at least on the basis of the filtered set of feature vectors to generate a trained ML model, and implementing the generated trained ML model to optimize distance sensing in real time.

Description

[0001] The present invention relates to the field of LiDAR (Light Detection and Ranging) technology, and more particularly to the use of machine learning (ML) to optimize the range detection of LiDAR cameras based on time-correlated single-photon counting (TCSPC) histograms.

[0002] LiDAR (Light Detection and Ranging) technology has become a crucial component in various applications, particularly in autonomous driving. LiDAR technology provides important depth information for environmental perception and decision-making. LiDAR technology uses LiDAR cameras that operate on the principle of Time-of-Flight (TOF) and Continuous Wave (CW) LiDAR. The TOF and CW LiDAR principles play a crucial role in scanning the environment and generating three-dimensional (3D) data for applications such as vehicle guidance. Typically, there are several types of LiDAR cameras, including those with rotating or tilting mirrors and flash LiDAR. Flash LiDAR illuminates an entire line or scene with a laser and captures the reflections in a single measurement.Flash LiDAR is preferred in the automotive industry due to its mechanical robustness against vibrations. Furthermore, LiDAR cameras can be implemented using MEMS (micro-electro-mechanical systems) components, a process known as MEMS scanning.

[0003] However, LiDAR technology has several limitations, such as the lack of color information, and its performance can be affected by environmental factors such as weather and ambient light. Two common LiDAR variants, Direct Time-of-Flight (dTOF) and Continuous Wave (CW) LiDAR, are widely deployed, with each variant having its own advantages and disadvantages. Furthermore, the measurements taken by LiDAR cameras are susceptible to various parasitic interactions and environmental problems. Extraneous light sources, particle interference, ambient light, jitter, noise, differential nonlinearity (DNL), pile-up, and dark counting are all factors that can affect the accuracy and reliability of ranging with LiDAR cameras. Regarding data processing, LiDAR cameras generate extensive data, requiring efficient data reduction and ranging techniques.Challenges include the interference caused by retroreflectors and the need for advanced data processing methods to improve depth resolution.

[0004] Conventionally, the center of mass method is used for LiDAR ranging, but this method suffers from a lack of absolute accuracy. Furthermore, there is growing interest in leveraging machine learning (ML) techniques to optimize the performance of LiDAR cameras. Researchers have explored the application of neural networks to improve accuracy and noise reduction in ranging. However, existing techniques often face challenges related to the computational intensity of neural networks, which makes real-time processing on conventional hardware impractical.Therefore, there is a need for an improved method and system to address these challenges by exploring the use of machine learning (ML), particularly neural networks, to optimize the range detection capabilities of LiDAR cameras and improve accuracy, reduce noise, and achieve real-time processing without hardware manipulation.

[0005] According to one embodiment, a method for optimizing range detection using a light detection and ranging (LiDAR) camera is disclosed. The method includes receiving a time-correlated single-photon count (TCSPC) histogram representing the distance of an object within a field of view of the LiDAR camera. The TCSPC histogram includes data indicative of the distance of the object, an environmental condition, and one or more parameters. The method further includes extracting a set of feature vectors from the TCSPC histogram. The method further includes filtering the extracted set of feature vectors based on at least one predetermined criterion. The method further includes training a machine learning (ML) model based at least on the filtered set of feature vectors to generate a trained ML model.The method then involves implementing the generated trained ML model to optimize distance detection in real time.

[0006] In some embodiments, the set of feature vectors from the TCSPC histogram includes a partial histogram at an interval of a maximum, a partial histogram of the nearest one or more neighbors, a differential nonlinearity modulus (DNL), a relative reflectance of the object, an index of a maximum, the temperature of the (LiDAR) camera, and a result of a center of mass (CoM) method.

[0007] In some embodiments, the predetermined criterion corresponds to a threshold for determining an erroneous or inconsistent distance. In some embodiments, the environmental conditions include at least one of a temperature condition and an ambient light condition.

[0008] In some embodiments, the one or more parameters include at least one of the signal-to-noise ratio (SNR) or LiDAR camera calibration parameters. In some embodiments, the LiDAR camera (102) is equipped with a sensor element, and the sensor element includes a single-photon avalanche diode (SPAD).

[0009] In some embodiments, the method further comprises validating the trained ML model using a test dataset.

[0010] In accordance with another embodiment, a system is disclosed. The system includes a memory and at least one processor communicatively coupled to the memory. The at least one processor is configured to receive a time-correlated single-photon counting (TCSPC) histogram representing the distance of an object within the field of view of a light detection and ranging (LiDAR) camera. The TCSPC histogram includes data indicative of the distance of the object, environmental conditions, and one or more parameters. The at least one processor is configured to extract a set of feature vectors from the TCSPC histogram. The at least one processor is configured to filter the extracted set of feature vectors based at least on a predetermined criterion.The at least one processor is configured to train a machine learning (ML) model (108) based at least on the filtered set of feature vectors to generate a trained ML model. The at least one processor is then configured to implement the generated trained ML model to optimize distance detection in real time.

[0011] In some embodiments, the set of feature vectors from the TCSPC histogram includes a partial histogram at an interval of a maximum, a partial histogram of the nearest one or more neighbors, a differential nonlinearity modulus (DNL), a relative reflectance of the object, an index of a maximum, the temperature of the (LiDAR) camera, and a result of a center of mass (CoM) method.

[0012] In some embodiments, the predetermined criterion corresponds to a threshold for determining an erroneous or inconsistent distance. In some embodiments, the environmental conditions include at least one of a temperature condition and an ambient light condition.

[0013] In some embodiments, the one or more parameters include at least one of the signal-to-noise ratio (SNR) or LiDAR camera calibration parameters. In some embodiments, the LiDAR camera (102) is equipped with a sensor element, and the sensor element includes a single-photon avalanche diode (SPAD). In some embodiments, the at least one processor is configured to validate the trained ML model against a test dataset.

[0014] In accordance with another embodiment, a non-transitory computer-readable medium is disclosed.The non-transitory computer-readable medium comprises instructions to cause a processor to perform functions including receiving a time-correlated single-photon counting (TCSPC) histogram representing a distance of an object within a field of view of the LiDAR camera, and the TCSPC histogram comprising data indicative of the distance of the object, an environmental condition, and one or more parameters; extracting a set of feature vectors from the TCSPC histogram; filtering the extracted set of feature vectors based on at least one predetermined criterion; training a machine learning (ML) model based on at least the filtered set of feature vectors to generate a trained ML model; and implementing the generated trained ML model to optimize ranging in real time.

[0015] In accordance with another embodiment, a method for determining distance information using a light detection and ranging (LiDAR) camera is disclosed. The method includes receiving a time-correlated single-photon count (TCSPC) histogram representing a distance of an object within a field of view of the LiDAR camera; determining distance information based on the received TCSPC histogram using a first method and a second method. The first procedure includes examining at least one entire interval from the received TCSPC histogram; obtaining a region containing a larger spatial value to determine the distance information; and determining at least one interval from the entire interval for examination by the second procedure.Furthermore, the second method comprises examining the at least one interval provided by the first method; obtaining a region containing a smaller spatial value to determine the distance information; and summing the distance information obtained by both the first method and the second method to obtain final distance information.

[0016] In some embodiments, the method further comprises determining the distance information using the first method, and the second method is performed independently.

[0017] In some embodiments, the method further comprises providing one or more indices from the first method to determine the at least one interval for examination by the second method.

[0018] In some embodiments, the method of determining the distance information using the second procedure comprises extracting a set of feature vectors from the at least one interval determined by the first procedure, and using a pre-trained machine learning model to determine either a plurality of probabilities corresponding to a distance class or a continuous value corresponding to continuous distance information.

[0019] The method further includes using the second procedure for an index and an interval selection to determine the distance information of the first procedure when a plurality of intervals are determined by the first procedure using the pre-trained machine learning model.

[0020] The above summary merely serves to summarize some embodiments to provide a basic understanding of some aspects of the present disclosure. Accordingly, it is to be understood that the embodiments described above are merely examples and should not be construed to limit the scope or spirit of the present disclosure in any way. It is recognized that the scope of the present disclosure encompasses many possible embodiments in addition to those summarized here, some of which are described further below.

[0021] To achieve the foregoing and related purposes, one or more aspects comprise the features fully described below and particularly pointed out in the claims. In the following description and the accompanying drawings, certain illustrative aspects are set forth in detail, indicating only a few of the various ways in which the principles of the aspects may be applied. Other advantages and novel features will become apparent from the following detailed description when considered in conjunction with the drawings, and the disclosed aspects are intended to encompass such aspects and their equivalents.

[0022] Further embodiments, details, advantages and modifications of the present embodiments will become apparent from the following detailed description of the embodiments, which is to be seen in conjunction with the accompanying drawings, in which: Fig. 1 shows a block diagram of a system according to an exemplary embodiment of the subject matter described herein; Fig. 2A shows a diagram for the derivation of an error Δd according to an exemplary embodiment of the subject matter described herein; Fig. 2B is a graph illustrating curvature error in meters as a function of pixel spacing in the x, y, and diagonal xy directions, according to one embodiment of the subject matter described herein; Fig. 2C shows a circuit diagram of a relay board according to an embodiment of the subject matter described herein; Fig. 3 shows a method for optimizing range detection with a LiDAR (Light Detection and Ranging) camera according to an embodiment of the subject matter described herein; Fig. 4A is a diagram illustrating a differential nonlinearity (DNL) measurement with a deactivated laser according to one embodiment of the subject matter described herein; Fig. 4B shows a histogram with the frequencies of an offset between the measured real distances and the distances determined from the indices of a maximum according to an exemplary embodiment of the subject matter described herein; Fig. 5 shows a fully connected linear neural network according to an embodiment of the subject matter described herein; Fig. 6 shows a flowchart of one or more errors associated with the LiDAR camera according to an embodiment of the subject matter described herein; Fig. 7 shows a method for determining distance information using the LiDAR camera according to another embodiment of the subject matter described herein; and Fig. 8 shows another fully connected linear neural network according to an embodiment of the subject matter described herein.

[0023] Some embodiments of this disclosure, illustrating their features, will now be discussed in detail. The words "consisting of," "having," "containing," and "including," and other forms thereof, are equivalent in meaning and are to be understood as open-ended, such that an element or elements following any of these words are not intended to be an exhaustive list of such element or elements or to be limited to the listed element or elements.

[0024] It should also be noted that, as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. While any apparatus and method similar or equivalent to those described herein may be used in practicing or testing embodiments of the present disclosure, the apparatus and method will now be described.

[0025] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings, in which like numerals refer to like elements throughout the several figures and in which exemplary embodiments are shown. However, the embodiments recited in the claims may be embodied in many different forms and should not be construed as limited to the embodiments shown herein. The examples shown here are non-limiting examples and merely represent examples among other possible examples.

[0026] An exemplary embodiment of the present disclosure and its potential advantages are described by reference to the Fig. 1 to 8 of the drawings, with like numbers being used for like and corresponding parts of the different drawings.

[0027] Fig. 1 shows a block diagram of a system (100) in accordance with an exemplary embodiment of the present disclosure. The system (100) may include a LIDAR camera (102), at least one processor (104), a memory (106), an ML model (108), an input / output circuit (110), a communication circuit (112), and a LiDAR light source (114).

[0028] In some embodiments, the LiDAR camera (102) may be configured to scan an object (not shown) within a field of view of the LiDAR camera (102). In some embodiments, the scanning of the object may be realized by other mechanical means, such as through the use of galvanometers. Alternatively, chip-based control techniques may be employed, e.g., through the use of microchips that employ one or more microelectromechanical system (MEMS)-based reflectors, e.g., a digital micromirror device (DMD) or a digital light processing device (DLP). In some embodiments, the scanning may be performed by non-mechanical means, e.g., through the use of electronic signals to control one or more optical phased arrays.

[0029] In some embodiments, the object in the field of view may reflect portions of the light pulses emitted by the LiDAR light source (114). Furthermore, one or more reflected portions of the light pulses from the object are detected by the LiDAR camera (102). In some embodiments, the light pulses reflected from the object may be detected by one or more sensors. The one or more sensors include, but are not limited to, the LiDAR camera (102) and a single-photon avalanche diode (SPAD) that detects individual photons of light.

[0030] In some embodiments, the system (100) further comprises the at least one processor (104). In some embodiments, the at least one processor (104) may be communicatively coupled to the LiDAR camera (102) and the memory (106). In some embodiments, the at least one processor (104) may be configured to receive a time-correlated single-photon count (TCSPC) histogram representing a distance of the object within the field of view of the LiDAR camera (102). Further, the at least one processor (104) may be configured to process the received TCSPC histogram. The at least one processor (104) may include suitable logic, circuitry, and / or interfaces operable to execute one or more instructions stored in the memory (106).In one embodiment, the at least one processor (104) may be configured to decode and execute all instructions received from one or more other electronic devices or servers.

[0031] The at least one processor (104) may be configured to execute one or more computer-readable program instructions, such as program instructions for performing any of the functions described in this specification. Further, the at least one processor (104) may be implemented using one or more processor technologies known in the art. Examples of the at least one processor (104) include, but are not limited to, one or more general-purpose processors (e.g., INTEL ® - or Advanced Micro Devices ® (AMD) microprocessors) and / or one or more special-purpose processors (e.g. digital signal processors or Xilinx ®System On Chip (SOC) Field Programmable Gate Array (FPGA) processors).

[0032] In some embodiments, the memory (106) may be configured to store a set of instructions and data executed by the at least one processor (104). Further, the memory (106) may include one or more instructions executable by the at least one processor (104) to perform certain operations. The memory (106) may be configured to include the instructions to receive a time-correlated single-photon count histogram (TCSPC) representing a distance of the object within the field of view of the LiDAR camera (102). One of ordinary skill in the art will appreciate that the instructions stored in the memory (106) enable the hardware of the system (100) to perform the specified operations.Commonly known memory implementations include, but are not limited to, hard disk drives, magnetic tapes, floppy disks, optical disks, compact disc read-only memories (CD-ROMs) and magneto-optical disks, semiconductor memories such as ROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other types of media / machine-readable media suitable for storing electronic instructions.

[0033] In some embodiments, the system (100) may further include the ML model (108). The ML model (108) may be further trained for optimized range detection in real time. The ML model (108) may include, but is not limited to, support vector machines (SVMs), random forests, and neural networks (e.g., autoencoders, convolutional machines, recurrent networks, long / short term memory (LSTMs), Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial networks, liquid state machines, etc.) and / or other types of machine learning models. In some embodiments, the ML model (108) may further use data from the LiDAR camera (102) as input to determine range information.

[0034] In some embodiments, the system (100) may further include input / output circuitry (110). The input / output circuitry (110) may enable a user to communicate or interface with the system (100) through one or more user devices (not shown). The one or more user devices may include N number of user devices. In some embodiments, the input / output circuitry (110) may serve as a medium for transmitting inputs from the interface to and from the system (100). In some embodiments, the input / output circuitry (110) may refer to the hardware and software components that facilitate the exchange of information between one or more user devices and the system (100).In one example, the system (100) may include a graphical user interface (GUI) (not shown) as input circuitry to allow the one or more users to enter data. The input / output circuitry (110) may include various input devices such as keyboards, barcode scanners, and the GUI to allow the one or more users to enter data, and various output devices such as displays and printers to allow the one or more users to receive data. In another example, the input / output circuitry (110) may include various output circuitry such as a display (not shown) to display the distance information based on the TCSPC histogram.

[0035] In some embodiments, the system (100) may further comprise the communication circuit (112). The communication circuit (112) may enable the system (100) to exchange data or information with other systems or devices. Furthermore, the communication circuit (112) may comprise network interfaces, protocols, and software modules responsible for sending and receiving data or information. In some embodiments, the communication circuit (112) may comprise Ethernet ports, Wi-Fi adapters, or communication protocols such as HTTP or MQTT for connecting to other systems. The communication circuit (112) may further comprise components such as communication modules (e.g., Wi-Fi, Ethernet, cellular), transceivers, antennas, and protocols (e.g., TCP / IP, MQTT, SNMP) for exchanging data with other systems or network devices.The communication circuits (112) may enable the system (100) to stay up to date and accurately track the collected distance information.

[0036] In some embodiments, the input / output circuitry (110) and the communication circuitry (112) can be configured to integrate the acquired distance information into other systems, such as third-party monitoring platforms, to enable centralized monitoring, analysis, and control by operators and automated processes. One of ordinary skill in the art will appreciate that the above components of the system (100) have been provided for illustrative purposes only, without departing from the scope of the disclosure.

[0037] Fig. 2A shows a diagram (200) for the derivation of an error Δd according to an embodiment of the subject matter described here. Fig. Figure 2B shows a graph (202) illustrating a curvature error in meters as a function of the spacing of the pixels in the x, y, and diagonal xy directions, d 2.5 m, f 5 mm, according to an exemplary embodiment of the subject matter described herein. Fig. 2C shows a circuit diagram (204) of a relay board according to an exemplary embodiment of the subject matter described herein. Fig. 2A-2C are used in conjunction with Fig. 1 described.

[0038] In some embodiments, the system (100) may also utilize a comprehensive measurement setup with measurements between the LiDAR camera (102) and a flat panel display to control various parameters such as distance, reflectance, temperature, ambient light, and signal-to-noise ratio (SNR). Furthermore, the measurements may be performed using retroreflectors located within the field of view of the LiDAR camera (102). Note that during the measurements, a measurement script may be written that performs the measurements in the following hierarchy (described as pseudocode):

[0039] Since a center point (e.g., pixel (40,129)) is assumed, the error Δd due to the flat screen can be detected, as pixels at the edge of the flat screen are farther away from the LiDAR camera (102), resulting in inaccurate measurements. In addition, the TCSPC histograms located within a certain distance around the center point can be considered reliable. The TCSPC histograms within a wider range can also be considered for further analysis. The wider range can be defined, for example, as rows 10-70 and columns 10-250. To calculate the distance in the measurement arrangement of the flat screen around the center point for reliable measurements, the magnitude of the error Δd is calculated in relation to the distance from the center point (P x ) of the flat screen. The following relationship is in Fig. 2A, ie, cos(α)=dΔd+d

[0040] Obtained through transformation: Δdx=d(1−cos(αx))cos(αx) Δdy=d(1−cos(αy))cos(αy)

[0041] α depends on the pixel pitch in the x or y direction and the braking distance of the number of rows or columns. αx=tan−1(px⋅ncolumnf) αy=tan−1(py⋅nrowsf)

[0042] In the measurements, the distance between two measurements is 5 mm, so a maximum error of 2.5 mm is permissible. Also, substitute equation (4) into equation (2) and equation (5) into equation (3) to calculate the error as a function of n. Spalten and n rows to determine.

[0043] The same applies to the diagonal pixel width, which is pxy=px2+py2 results, as in Fig. 2B. In some embodiments, a maximum of 4 pixels can be used in a square around the center point, as otherwise the error may be too large. This results in a square of 9x9 pixels from which the TCSPC histograms can be extracted for further processing. The entire depth of the histograms is not used for data reduction. For data reduction, not the entire depth of the histograms of 448 bins can be stored, but only 50 bins, since 50 bins correspond to a range of approximately 5 m that is not fully utilized during measurement. This leads to data reduction.

[0044] Furthermore, a halogen lamp, which can be used as an equivalent LiDAR light source (114) for the ambient light, can be used for the additional ambient light for measurement, which is controlled by a computer device (not shown) via a programmable controller. The LiDAR camera (102) can be disassembled and mounted on a structure that allows the LiDAR camera (102) to be cooled or heated. Furthermore, a structure made of aluminum plates can be used as a heat conductor, which is connected to a Peltier element (not shown), which in turn can exchange thermal energy with the environment via a heat sink. In some embodiments, the Peltier element is in turn controlled by a programmable voltage source. In some embodiments, the voltage or other parameters of the programmable voltage source can be controlled externally via an RS232 interface.

[0045] Fig. Figure 2C shows the circuit diagram (204) of a relay board with a connector (J1). The connector has pins labeled Terminal A, Terminal B, GND, +12V supply, and a relay control. The connector (J1) can serve as an interface for connecting external devices to the relay board. Furthermore, the circuit diagram (204) includes a relay (K1). The relay (K1) switches the current flow through the Peltier element. Furthermore, the circuit diagram (204) includes a resistor (R1) suitable for controlling the relay or ensuring proper current flow. The circuit diagram (204) further includes a transistor (Q1) that can control the components of the circuit diagram (204). In some embodiments, using the self-built relay board, as shown in Fig. As shown in Figure 2C, the current flow through the Peltier element can be reversed via terminals A and B, allowing the Peltier element to either heat or cool the LiDAR camera (102). The relay board is controlled by a microcontroller (not shown), which receives a command to turn the relay on and off via a virtual COM port. A temperature sensor (not shown) can also be present on the LiDAR camera (102) so that the temperature can be measured via an application programming interface (API).

[0046] Fig. Figure 3 shows a method (300) for optimizing distance detection with the LiDAR camera (102) according to one embodiment. The method (300) includes a series of steps designed to utilize machine learning techniques in conjunction with time-correlated single-photon counting (TCSPC) histograms. Fig. 4A shows a diagram (400) illustrating a differential nonlinearity (DNL) measurement with the laser deactivated, according to an embodiment of the subject matter described herein. Fig. 4B shows a histogram (402) with the frequencies of the offset between the measured real distances and the distances determined from the indices of the maxima, according to an exemplary embodiment of the subject matter described herein. Fig. 5 shows a fully connected linear neural network (500) according to an embodiment of the present disclosure, corresponding to an exemplary embodiment of the subject matter described herein. Fig. 6 illustrates a diagram (600) showing one or more errors associated with the LiDAR camera, according to an example embodiment of the subject matter described herein.

[0047] The method (300) begins in step (302) by receiving the TCSPC histogram representing the distance of the object within the field of view of the LiDAR camera (102). In some embodiments, the TCSPC histogram captures the arrival times of the photons and provides insights into the distance of objects based on the time-of-flight principle. The TCSPC histogram includes data indicative of the distance of the object, the environmental conditions, and one or more parameters. The environmental conditions include at least one of a temperature condition and an ambient light condition. Furthermore, the one or more parameters include at least one of the signal-to-noise ratio (SNR) or calibration parameters of the LiDAR camera (102).

[0048] In some embodiments, the TCSPC histogram is a data structure representing a series of values ​​over discrete time intervals. Each interval, or bin, in the histogram may be associated with a value. For example, the TCSPC histogram may be configured to record the number of photodetectors activated during specific time intervals within one or more acquisition periods. Alternatively, the TCSPC histogram may represent the digital conversion of an analog signal at different points in time. The TCSPC histogram may include both the signal, e.g., the pulses, and the accompanying noise. Therefore, the TCSPC histogram serves as a record of both the signal and the noise and is similar to a photon time series or flux.The original form of the TCSPC histogram, often referred to as the raw or digitized histogram, contains the unfiltered signal and noise as recorded in memory (106).

[0049] In some embodiments, after recording all data and the corresponding data sets for the inclined surfaces in step (304), a set of feature vectors is extracted from the TCSPC histogram. As described above, the pixels to be generated lie in a 9x9 square around the center point. The center point is at position p m = ([80 / 2)],(258 / 2))] = (40,129). The TCSPC histograms are extracted, and the first and second maximum in the histograms are searched for, and the corresponding indices i max,1 and i max,2 are stored, where |i max,1 - i max,2|> 5 can be satisfied to avoid overlaps. This calculation is used to compensate for the blooming effect of a retroreflector, since in this case two maxima occur and only one of the maxima contains the desired distance.

[0050] In addition, a first filter can be implemented directly. To avoid the generation of feature vectors from false detections, the median med = Median (i max,1,2,pm-(j,k), ) starting with j = [-4, -3, ... 3, 4] and k = [-4, -3, ... 3, 4]. The shift along the rows and columns is denoted by j and k. In the next step, only those histograms are completed that satisfy the condition |imax ,l,pm-(j,k) -med| <=1 with I = [1, 2]. Furthermore, outliers that would otherwise negatively impact training can be removed. The remaining TCSPC histograms are then used for further extraction of the set of feature vectors.

[0051] In some embodiments, the set of feature vectors from the TCSPC histogram includes a partial histogram in an interval with a maximum. The partial histogram is extracted and further normalized to ensure the distance independence of the histogram profiles. Note that first, the dependence of a luminous flux (F1-2) on the distance between the LiDAR camera (102) and the illuminated surface is investigated. Furthermore, the irradiance (E) with which the object is illuminated by the laser with power or radiant flux (F) emitting a light beam with intensity (I) at a distance must be calculated. Furthermore, the calculation is performed approximately assuming small solid angles according to a photometric distance law. In the following, d is replaced by R and an isotropic radiator is assumed. The first result is therefore: le=d⋅Φe.dΩ

[0052] Here, dΩ describes the angle of propagation at which the transmitter radiates the light. Assuming that the reflecting object is a Lambertian radiator, the radiance in the half-space can be calculated as follows. A simplification is made for a vertical surface element dA1 with ε=π2. It is assumed that the entire radiant power of the laser hits the surface element dA1 of the object and is reflected with the reflectance R. L=le⋅RdA1cos(ε)=le⋅RdA1

[0053] The luminous flux radiated from the surface element of the object dA1 to the surface element of the receiver dA2 is defined in the photometric distance law as: Φ1→2=L.cosε1.cosε2r2.dA1⋅dA2 which can be simplified by assuming rectangular surfaces: Φ1→2=L⋅1r2⋅dA1⋅dA2=Φe⋅ler2

[0054] If equation (6) is inserted into equation (7) and equation (7) into equation (9), we get Φ1→2=ler2⋅dA2

[0055] The calculation of dA2 corresponds to the area of ​​the aperture: dA2=2∏r2=∏.d22

[0056] The aperture diameter d is calculated according to d=f / k

[0057] Then insert equation (12) into equation (11) and obtain dA2: dA2=∏.f22.k2

[0058] Also insert equation (13) into equation (8): Φ1→2=le.∏.f2r2.k2=Φe.∏.f2r2.k2.dΩ

[0059] From equation (14) it follows that the luminous flux Φ 1→2 quadratically with the distance (r). To make the partial histograms distance-independent, in some embodiments each entry of the histogram is labeled with the inverse distance r max - r normalized to compensate for the effect: norm(r)=1(dmax−d)2

[0060] However, the distance r is not yet known at the time of measurement, so it is calculated beforehand from the index of the maximum i max ∈ N estimated. As ilimit ∈ N the end of the selected measuring range is used as bin index: norm(imax)=1(ilimit−imax)2

[0061] For the partial histogram pH, an interval of the TCSPC histogram H containing the maximum or the second maximum is used. To determine the interval, the index of the maximum i max ∈ N is used as a reference. The parameters pre and post are also used, which specify how many elements before or after the maximum must be removed. The partial histogram is calculated for the first and second maximum. p.H1,0(j+pre)=H(imax,l+j).norm(imax) with j=[−pre,−pre+1,….0….,post]and l=[1,2]

[0062] In some embodiments, the partial histogram from the previous measurements is identified to obtain further information. The measurement is performed before the current measurement (r) so that partial histograms pH can be obtained, which can be added to the set of feature vectors. The maximum of the two previous measurements r-1 and r-2 is used to avoid excessive dependence on the previous measurements. p.Hl,r−k(i+pre)=H(imax,l,r−k+i).norm(imax)with i=[−pre,−pre+1,…0…,post]and l=[1,2]and k=[1,2]

[0063] In some embodiments, the set of feature vectors from the TCSPC histogram includes an index of a maximum. The index of the first two maxima is determined to check whether the current element contains valid data or is an outlier. It is further processed and added to the set of feature vectors as i max,1 , and i max,2added.

[0064] In some embodiments, the set of feature vectors from the TCSPC histogram includes a subhistogram of the nearest one or more neighbors. In this subhistogram, a mean value of one or more similar neighbors is calculated. Furthermore, additional information can be provided for determining the distance without creating a time dependency. Similar neighbors are used; for example, a neighbor with a significantly different distance does not significantly distort the current result. Furthermore, the indices of the maxima are again used as a reference.

[0065] In some embodiments, if a neighbor in a 3x3 grid around the current pixel p = (0,0) has the same i maxThe partial histogram can be added to a list of previously identified partial histograms. Furthermore, the mean of all partial histograms in this list can be calculated, the mean is calculated from the N added values, and the resulting partial histogram (mnH) is added to the feature vector after normalization. mnH(i+pre)=1 / N∑k=−11∑|imax,1−imax,k,l|<11l=−1Hk,l(imax,1+i)norm(imax) with i=[−pre,−pre+1,…0…,post]and k=[−1,0,1],l=[−1,0,1]

[0066] In some embodiments, the set of feature vectors from the TCSPC histogram includes the temperature of the LiDAR camera (102). Since temperature has a strong influence on the receiver side, the temperature is recorded during measurements and added to the set of feature vectors.

[0067] In some embodiments, the set of feature vectors from the TCSPC histogram includes a differential non-linearity (DNL) modulo. The DNL modulo of the TCSPC histogram can be measured to evaluate data quality and accuracy. To measure DNL, ​​multiple measurements can be performed without a laser, so that only ambient light causes events. The ambient light can be assumed to be quasi-continuous during the measurement period. Accordingly, the TCSPC histogram can show virtually no difference between the counts of individual bins recorded over an extended period. If DNL is present, the count of the respective bin deviates from the histogram mean, as shown in Fig. 4A. Furthermore, the count values ​​differ significantly from each other. Note that the DNL has a periodicity of 14, meaning that deviations occur every 14 bins and a factor of 2 between the minimum count and the maximum count (2500 to 4500). This characteristic is abbreviated as dm: dm1=imax,lmod14 with l=[1,2]

[0068] Therefore, the resulting DNL errors can be taken into account in further implementation. Furthermore, the measurement setup can be designed to obtain sufficient, high-quality data. This data can then be used to test different methods and compare their performance.

[0069] In some embodiments, the set of feature vectors from the TCSPC histogram includes a relative reflectance of the object. The signal contained in the histogram can be strongly influenced by the reflectance of the reflecting object. This reflectance can be determined from the distance and radiant power. The sum of the histogram can therefore be used to obtain information about the reflectance of the object, since the reflectance is influenced by the illumination. Therefore, the relative reflectance can be calculated and added to the set of feature vectors, which can provide an indication of the extent of the pile-up effect: Rrel=∑n=1∞pH1,0(i)

[0070] In some embodiments, the set of feature vectors from the TCSPC histogram includes a result of a center of mass (CoM) method, where the distance of the object can be measured using the center of mass method. delta_d=d_com⋅−d_com−1

[0071] The result of the CoM procedure with respect to the partial histogram can be added to the set of feature vectors as an additional feature. Furthermore, the partial histogram in the marker vector can potentially be truncated, since the result of the CoM procedure already contains information from the partial histogram. di,ML,llR,CoM=α.di,ML+(1−α).di−1,ML,llR,CoM+β.Δdi,CoM

[0072] The set of feature vectors can be extracted to encapsulate key features of the TCSPC histogram for further analysis. In some embodiments, the set of feature vectors from the TCSPC histogram can include a partial histogram at an interval of a maximum, a partial histogram of the nearest one or more neighbors, a differential nonlinearity modulus (DNL), a relative reflectance of the object, an index of a maximum, the temperature of the (LiDAR) camera, and a result of a center of mass (CoM) method.

[0073] In step (306), the extracted set of feature vectors is filtered based on at least one predetermined criterion to improve the accuracy and reliability of the data. Such filtering can eliminate noise, outliers, and irrelevant data points to ensure that only high-quality feature vectors are retained for subsequent analysis. The predetermined criteria correspond to a threshold for determining an erroneous or inconsistent distance. The extracted set of feature vectors indicating distances beyond a predefined threshold could be considered inconsistent or erroneous and therefore filtered out during filtering. Thus, one or more vectors whose value lies above the predefined threshold, which can range from 5 to 10 mm, can be removed.

[0074] The filtering of the extracted set of feature vectors also involves the use of an index of maxima i ,max1+2 to estimate a range in which the result is likely to lie, with the step size here being approximately 10 cm. Furthermore, the histogram has a camera-specific offset that must be determined. Two pieces of data are required for filtering: a target vector y, which contains the measured distance, and two vectors d 1+2 , which respectively represent the index of the first and second maximum i max,1+2 . In the first step, the corresponding distance is calculated from the index of the maxima. The offset vectors o 1+2 result from the difference between the determined distance d and the measured distance y o1+2→=d1+2→−y→

[0075] Furthermore, the histogram is created from all values ​​of o 1+2created, then the range of the permissible offset, the boundaries around the two peaks, is read. In one case, interval I corresponds to the offset from 3.38 m to 4.21 m. These values ​​are significantly larger than the measurement range. The reason for this is the delay between the start of the laser pulse and the start of recording the measurement data in the histogram. As in Fig. As shown in Figure 4B, two peaks can be seen. Note that the spacing between the peaks corresponds to a bin. The bin refers to a specific time interval within the TCSPC histogram.

[0076] The reason for two peaks is that if the distance lies in the transition between two bins, the index of the maximum fluctuates slightly between the two bins and is therefore too early in the next bin, and the offset is correspondingly one bin length too long. If neither the first nor the second determined distance from d 1+2lies within the determined interval I, the target vector y is marked. This represents an erroneous determination from the feature vector. If the vector d 1+2 determined distance in interval I, the target distance y can be determined from the feature vector v. To distinguish whether the first or second maximum lies in the permissible interval I, the target vector is marked by multiplication by -1 at the point where the second maximum lies in the permissible interval, but not the first. The reason for this procedure is that the neural network can use different parts of the feature vector v. The result is the set of feature vectors v. i and the correspondingly marked target values ​​y.

[0077] Furthermore, in step (308), a machine learning (ML) model (108) is trained at least based on the filtered set of feature vectors. The ML model (108) is trained using the extracted features and the corresponding distance labels, so that the ML model (108) can make informed predictions about the distance based on the input TCSPC histogram. In some embodiments, a machine learning library can be used to train the neural networks. Using the machine learning library, the neural networks can be very easily composed of multiple layers. Furthermore, a list can be created that specifies the order of the layers to be connected. The list begins, for example, with a linear network, which must first be initiated as an independent class. Here, two parameters are sufficient for initiation: the number of inputs and the number of nodes.It should be noted that a linear network can be followed by an activation function.

[0078] In addition, a class is created again with the activation function to be initiated, e.g., Rectified Linear Unit (ReLU) function. The ReLU activation function is then added to the list. This process can be repeated until the planned network is complete. The machine learning library can also implement metrics, such as entropy and Rooted Squared Mean Error (RSME), to evaluate the training and test results obtained, which can be passed on to the training or testing algorithm. Training neural networks can be easily achieved using pre-built examples. The machine learning library can also be used to create and test various neural network architectures. Neural network architectures can be divided into three groups, as shown in Fig. 5 shown.

[0079] A group can contain linear networks with a total of 6 layers. Fig. 5 represents a number of input layers (502) I1 to I n , a number of hidden layers (504) H1 to H m , H1 to H o , H1 to H p , H1 to H q and an output layer (506) O1. Another group may include linear networks with CNNs—starting with a convolutional layer with m channels and a step size of o, followed by a max-pooling layer along the channels with length p and step size q, and 4 linear layers, where the first linear layer is adapted to the output layer (506) of the max-pooling layer. Another group may include linear networks with an auto-encoder.

[0080] The RSME cost function can be used as the cost function. The last layer represents the node corresponding to the target distance y. In some embodiments, the random forest is trained. New features are gradually added to determine the influence of each feature on the detection probability for both models. Starting with the partial histogram of length 5, a feature can be added and removed again if the detection probability has not increased.

[0081] In addition, a random forest class named clf is first set up and parameterized. Entropy can be used as the cost function. The minimum number of elements required to form a new leaf in the decision tree can be set to 25. The leaf size here is very small compared to the size of the dataset. Furthermore, the size of the forest is varied across different measurements. Further variations of the random forest parameters are not necessary. Training is performed using the fit method of the clf class, where the vectors (x i ) and the discretized target classes d d,c of the training dataset as a parameter. The classification result is then determined by the Random Forest t using a predict method and the set of feature vectors of the training dataset.

[0082] In some embodiments, the SVM is trained. Similar to Random Forest, an SVM class called clf is initiated. Several parameters are passed. First, it is necessary to define a kernel to be used. For example, an RBF kernel can be chosen here. This determines the distance k between two feature vectors x. i and x j calculated according to the rule: k(x→lx→j)=exp) where d is the Euclidean distance and Y is a scaling parameter yet to be defined. If Y is too large, only the training vectors themselves are used as support vectors, which would result in overfitting. If Y is too small, the support vectors from the training data and the individual discretized classes d d,ccan no longer be distinguished from each other. Another parameter is the parameter C, which can be used to control the distance between the hyperplanes and the support vectors when creating the hyperplanes. These parameters are first calculated using the complete feature vectors x i calculated.

[0083] Furthermore, various SVMs are used with the help of an optimization algorithm. Classes clf are initiated with different combinations of y and C and are then tested with the reduced training data set, which contains the discretized classes d d,c The fitting procedure is trained. The results are again determined by classifying the test dataset and using the RSME metrics, and the relative frequency can be compared. The optimization algorithm is part of the SCIPY package, for example. 4 and tries to calculate the relative frequency with the parameters Y and C.

[0084] In the next step, as with Random Forest, the number of features is gradually increased during training of the SVM and the influence of feature determination is increased.

[0085] In various embodiments, after being trained at least based on the filtered set of feature vectors, the ML model (108) is validated against a test dataset to evaluate its performance and generalization ability. Validation helps ensure that the ML model (108) can accurately predict distances for unknown data instances, thereby confirming its reliability for real-world applications.

[0086] In step (310), the trained machine learning model is implemented for optimized distance detection in real time. The model can now process incoming TCSPC histogram data in real time and provide accurate distance information for objects in the field of view of the LiDAR camera (102). In some embodiments, some errors that may occur in the LiDAR camera (102) are analyzed to adjust the measurement. The expected errors can be divided into four different classes, as shown in Fig. 6. For example, uncorrectable (602) - the error cannot be corrected. Uncorrectable errors include, among others, contamination of the lens of the LiDAR camera (102), optical distortions, and power supply disturbances. Uncorrectable errors cannot be compensated or detected and therefore cannot be corrected. Another example: Partially correctable (604) - the error cannot be eliminated, but its effects can be reduced.

[0087] Partially correctable errors include, among others, extraneous light, ionizing radiation, weather influences (fog, rain, etc.), dark counting, and vignetting. Errors such as extraneous light, ionizing radiation, and dark counting cause interference events and can lead to incorrect distance determination at a low SNR, which is why measurements can be performed at low SNR to develop suitable mitigation measures. In another example, probably correctable (606) - the error can probably be corrected by suitable measures. Probably correctable errors include, among others, errors caused by the retroreflector, DNL, ​​temperature-related time-of-flight differences, temperature-related pulse deformations, and inhomogeneous illumination. In another example, negligible (608) - the error has only a minor effect and does not require correction.Negligible errors include multiple reflections and after-pulse effects.

[0088] Measurements are planned across various temperature ranges and reflectances. Errors caused by multiple reflections are highly unlikely to lead to incorrect evaluations, as the pulses lose intensity due to reflections and are filtered out during data processing because they are weaker than the original peak. The erroneous events caused by afterpulses are also relatively small compared to the initial peak. Correcting retroreflector errors is likely to be difficult to compensate for. Measurements with retroreflectors within the field of view are currently being prepared.

[0089] Fig. 7 shows a method (700) for determining distance information using the LiDAR camera (102) according to an exemplary embodiment. Fig. 8 shows another fully connected linear neural network (800) according to an embodiment of the present disclosure and according to an exemplary embodiment of the subject matter described herein. Fig. 7 is in connection with Fig. 8 shown.

[0090] The method (700) begins in step (702) with the receipt of the TCSPC histogram, which represents the distance of the object within the field of view of the LiDAR camera (102). The TCSPC histogram records the arrival times of the photons and provides information about the distance of objects based on the time-of-flight principle. Furthermore, in step (704), the distance information is determined based on the received TCSPC histogram using a first and a second method. In some embodiments, the distance information is determined independently of each other using the first method and the second method.

[0091] The first method comprises examining at least one entire interval or a larger interval from the received TCSPC histogram. Furthermore, a region containing a larger spatial value is determined to determine the distance information. Furthermore, the first method comprises determining at least one interval for examination by the second method. The second method comprises examining the at least one interval provided by the first method. Furthermore, a region containing a smaller spatial value is determined to determine the distance information.

[0092] In some embodiments, one or more indices may be provided by the first procedure to determine the at least one interval from which the second procedure extracts values ​​and uses the values ​​to determine the distance information. In some embodiments, a machine learning model of the second procedure may be trained using previously recorded reference signals with subtraction of the portion related to the first procedure.

[0093] For example, the rough distance d is first estimated from at least one index of the maximum and the temporal size of a bin A: dc=imax⋅c2⋅Δtbin

[0094] In the next step, the coarse distance is subtracted from the real distance t. In addition, the LiDAR camera (102) subtracts the conditional offset o, which gives the fine distance d f results in: df=t−dc+c

[0095] Since the spatial resolution of a bin Δt bin is about 10 cm, the f exhibit similar behavior. However, a larger range of approximately 80 cm can also be used. Each of the values ​​of the maximum index, starting from hanging peaks, extends to the sides up to a width of approximately 80 cm. This results in approximately 80 different classes of d,c in cm intervals: dd,c=⌊df,100⌋

[0096] The feature vector x contains two different maxima Imax,1 and I max,2 . In addition, during filtering, an estimate is already made in the target vector as to which data are invalid and in which case the second maximum lies within the permissible range. If the first or second maximum lies within the permissible range, ie, y is positive, d c calculated from the index of the first maximum. If only the second maximum lies within the permissible range, ie, y is negative, d ccalculated from the index of the second maximum. Accordingly, d d,c multiplied by -1. If none of the maxima lies within the permissible range, ie, y is marked at that point, d d,c defined as -100, which corresponds to incorrect detection.

[0097] If the recognition is again unsuccessful, the feature vectors x i and the corresponding target vector d d,cmixed and divided into a training data set and a test data set. The training data set comprises 90% and the test data set 10% of the data. Furthermore, the distance information determined using the second method comprises extracting the set of feature vectors from the at least one interval. Furthermore, a pre-trained machine learning model, preferably the neural networks, is used to determine either a plurality of probabilities corresponding to a distance class or a continuous value corresponding to continuous distance information.

[0098] The values ​​generated by the neural networks are not only able to continuously determine the values ​​y, but also the multitude of probabilities that indicate a probability that the current input vector corresponds to a class c. The multitude of probabilities represent the discrete distance values ​​d d,cand influence the structure of the neural network. Instead of a node in the output layer, a corresponding number of output nodes O1 to O n The corresponding output value of an individual node then corresponds to the probability with which the current input vector belongs to the class assigned to the node. During classification, the class with the highest probability is then selected.

[0099] In contrast to the new networks with continuous output, in the fully connected linear neural network (800), as in Fig. 8, only linear networks were trained. Fig. 8 shows a number of input layers (802) I1 to I n , a number of hidden layers (804) m, o, p and q and a number of output layers (806) O1 to O n. Furthermore, the ReLU activation function can be used in each node. Each node is assigned to class d dc During classification, the node with the highest value is selected. The assigned class is then the result of the classification.

[0100] In some embodiments, if the distance detection is performed using neural networks, it can also be performed using random forests and SVMs. The input data can be the same data used for the neural networks. The input data can then be processed so that the pre-trained model can determine the distance from the data using the second method.

[0101] In exemplary embodiments, the second procedure may further be used for index and interval selection to determine the distance information of the first procedure when multiple intervals are determined by the first procedure using a pre-trained machine learning model. In some exemplary embodiments, the distance information using the first method includes determining the distance information from the selected index and the interval. To obtain final distance information, the distance determined by the first method and the second method are added in step 706.

[0102] In exemplary embodiments, the results are also evaluated. The main criterion for evaluating the results should be the error squared (RSME) between the target value t and the output y. The error squared with n ∈ N + and i ∈ N0 +different results of the model can be calculated as follows: RSME=1n∑i=0n(ti−yi)2

[0103] Another criterion is the relative probability h n (A) ensure that the event A(∈) occurs, that the target value and the output of the model y i within an error state |t i - y i |< ∈ with ∈ € R+ and i € N + In one case, n is the total number of observations and H n (A) the absolute frequency with which the condition A(∈) is satisfied. hn(A(ϵ))=Hn(A(ϵ)) / n

[0104] In some embodiments, hardware is tested. The test hardware is the LiDAR camera (102), which operates entirely without a mirror and instead operates in flash mode.

[0105] By appropriate conversions, you get the distance between the pixels in the x and y directions: px=2.tan−1(FOVx⋅f)ncolumns≈42μm py=2.tan−1(FOVy⋅f)nrows≈35μm

[0106] The LiDAR camera (102) is controlled either via an existing graphical user interface (GUI) or the application programming interface (API). The transmitter and receiver registers can be configured, and the measurement data is retrieved via a USB 3.0 interface. In one case, both the distance and intensity data as well as the full TCSPC histogram data can be accessed.

[0107] Embodiments of the present disclosure may be provided as a computer program product, which may include a computer-readable medium having tangibly embodied thereon instructions that can be used to program a computer (or other electronic device) to perform a process. The computer-readable medium may include, but is not limited to, hard disk drives, magnetic tapes, floppy disks, optical disks, compact disc read-only memories (CD-ROMs) and magneto-optical disks, semiconductor memories such as ROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other types of media / machine-readable media suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).

[0108] Furthermore, embodiments of the present disclosure may also be downloaded as one or more computer program products, where the program may be transmitted from a remote computer to a requesting computer via data signals embodied in a carrier wave or other transmission medium over a communications connection (e.g., a modem or network connection).

[0109] The detailed description of the application should point out that the order of the method steps is not critical. Such mentions would later support the argument that the order of steps in a method claim is not critical or fixed. Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments and / or in combination with or instead of the features of the other embodiments.

[0110] While the above embodiments have been illustrated and described, as noted above, many changes may be made without departing from the spirit and scope of the example embodiments. For example, aspects of the subject matter disclosed herein may be practiced on other operating systems. Accordingly, the scope of the example embodiments is not limited by the disclosure of the embodiment. Instead, the embodiments should be determined entirely by reference to the following claims. List of reference symbols 100 systems 102 LIDAR camera 104 processor 106 memory 108 ML model 110 Input / output circuit 112 Communication circuit 114 LiDAR light source 200 Diagram for the derivation of an error Δd 202 Diagram for a curvature error 204 Circuit diagram of a relay board 300 methods for optimizing distance detection 302 Reception of a time-correlated single-photon counting histogram (TCSPC) 304 Extraction of a set of feature vectors from the TCSPC histogram 306 Filtering the extracted set of feature vectors 308 Training a Machine Learning (ML) Model 310 Implementation of the trained machine learning model 400 Measurement of a differential nonlinearity (DNL) with deactivated laser 402 frequencies of the offset between the measured real distances and the distances determined from the indices of the maxima 500 fully connected linear neural network 502 Input layer 504 hidden layer 506 Output layer 600 errors related to the LiDAR camera 602 uncorrectable error 604 partially correctable error 606 probably correctable error 608 negligible errors 700 Methods for determining distance information 702 Reception of the TCSPC histogram 704 Determining the distance information based on the received TCSPC histogram using a first and a second method 706 Adding the distance determined by the first method and the second method to obtain the final distance information 800 additional fully connected linear neural networks 802 Input layer 804 hidden layer 806 Output layer

Claims

[1] A method (300) for optimizing range detection using a light detection and ranging (LiDAR) camera (102), the method comprising: - receiving a time-correlated single-photon counting (TCSPC) histogram representing a distance of an object within a field of view of the LiDAR camera (102), the TCSPC histogram comprising data indicative of a distance of the object, environmental conditions, and one or more parameters; - Extraction of a set of feature vectors from the TCSPC histogram; - filtering the extracted set of feature vectors based on at least one given criterion; - training a machine learning (ML) model (108) at least based on the filtered set of feature vectors to generate a trained ML model; and - Implementation of the generated, trained ML model to optimize distance detection in real time. [2] The method (300) of claim 1, wherein the set of feature vectors from the TCSPC histogram comprises a partial histogram at an interval of a maximum, a partial histogram of the nearest one or more neighbors, a differential nonlinearity modulus (DNL), a relative reflectance of the object, an index of a maximum, the temperature of the (LiDAR) camera, and a result of a center of mass (CoM) method. [3] The method (300) of any preceding claim, wherein the predetermined criterion corresponds to a threshold value for determining an erroneous or inconsistent distance, and wherein the ambient conditions comprise at least one of a temperature condition and an ambient light condition. [4] The method (300) of any preceding claim, wherein the one or more parameters comprise at least one of signal-to-noise ratio (SNR) or LiDAR camera calibration, and wherein the LiDAR camera (102) is equipped with a sensor element, and the sensor element comprises a single-photon avalanche diode (SPAD). [5] The method (300) according to any one of the preceding claims further comprises validating the trained ML model using a test data set. [6] A system (100) comprising: - a memory (106); - at least one processor (104) communicatively connected to the memory (106), wherein the at least one processor (104) is configured to: • receiving a time-correlated single-photon counting histogram (TCSPC) representing a distance of an object within a field of view of a light detection and ranging camera (LiDAR) (102), the TCSPC histogram comprising data indicative of a distance of the object, environmental conditions, and one or more parameters; • extract a set of feature vectors from the TCSPC histogram; • Filtering the extracted set of feature vectors based on at least one given criterion; • Training a machine learning (ML) model (108) based at least on the filtered set of feature vectors to generate a trained ML model; and • implement the created trained ML model to optimize distance detection in real time. [7] The system (100) of claim 6, wherein the set of feature vectors from the TCSPC histogram comprises a partial histogram in an interval of a maximum, a partial histogram of the nearest one or more neighbors, a differential nonlinearity modulus (DNL), a relative reflectance of the object, an index of a maximum, the temperature of the (LiDAR) camera (102), and a result of a center of mass (CoM) method. [8] The system (100) of any preceding claim, wherein the predetermined criterion corresponds to a threshold for determining an erroneous or inconsistent distance, and wherein the environmental conditions comprise at least one of a temperature condition and an ambient light condition. [9] The system (100) of any preceding claim, wherein the one or more parameters comprise at least one of a signal-to-noise ratio (SNR) or LiDAR camera (102) calibration parameter, and wherein the LiDAR camera (102) is equipped with a sensor element, and the sensor element comprises a single-photon avalanche diode (SPAD). [10] The system (100) according to any one of the preceding claims further comprises validating the trained ML model using a test data set. [11] A method (700) for determining distance information using a light detection and ranging (LiDAR) camera (102), the method (700) comprising: - receiving a time-correlated single photon count (TCSPC) histogram representing the distance of an object within the field of view of the LiDAR camera (102); - determining distance information based on the received TCSPC histogram using a first method and a second method, - the first method comprising: • Checking at least one entire interval of the received TCSPC histogram; • to obtain an area containing a larger spatial value to determine the distance information; and • Determining at least one interval from the entire interval for testing by the second method; and - the second method comprising: • Checking the at least one interval provided by the first method; • Obtaining an area containing a smaller spatial value to determine the distance information; and • Adding the distance information obtained from both the first and second methods to obtain a final distance information. [12] The method (700) of claim 11, wherein the determination of the distance information is performed using the first method and the second method independently of each other. [13] The method (700) according to any one of the preceding claims further comprises providing one or more indices from the first method to determine the at least one interval for examination by the second method. [14] The method (700) of any preceding claim, wherein determining the distance information using the second procedure comprises: - extracting a set of feature vectors from the at least one interval determined by the first procedure; and - Using a pre-trained machine learning model to determine either a plurality of probabilities corresponding to a distance class or a continuous value corresponding to continuous distance information. [15] The method (700) of any preceding claim further comprises using the second procedure for an index and an interval selection to determine the distance information of the first procedure when a plurality of intervals are determined by the first procedure using the pre-trained machine learning model.

Images