Glare detection method, and device and computer program product

Abstract

The content of the present disclosure relates to a glare detection method, and a device and a computer program product. The glare detection method for a light detection and ranging device comprises: obtaining corresponding multi-echo data of a plurality of pixels by means of a light detection and ranging device; on the basis of the corresponding multi-echo data of the plurality of pixels, determining a high-reflectivity object region, wherein the high-reflectivity object region comprises one or more high-reflectivity object pixels; on the basis of the high-reflectivity object region and the lighting logic of the light detection and ranging device, determining a potential glare region, wherein the potential glare region comprises one or more potential glare pixels; and on the basis of the depth and width characteristics of one or more echoes of pixels in the high-reflectivity object region and the potential glare region, classifying the one or more potential glare pixels, so as to determine glare pixels and categories thereof.

Description

Methods, apparatuses, and computer program products for glint detection

[0001] Priority claim

[0002] This application claims priority to Chinese Patent Application No. 202410864914.6, filed on June 28, 2024, entitled “Methods, apparatuses, and computer program products for glint detection,” the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates generally to the field of signal processing, and more specifically to methods, apparatuses, and computer program products for glint detection of light detection and ranging devices. BACKGROUND

[0004] Light detection and ranging devices are devices for detecting and locating objects, whose working process includes emitting light through an emitter and receiving light reflected back after colliding with a target object through a receiver. The distance to the target object is measured according to the time difference between light emission and reception (i.e., time of flight, ToF). The receiver records the number of photons received at different times, which is called echo data. After the light is reflected back by the object, an echo is formed on the echo data, and the distance to the target object can be determined through the position of the echo.

[0005] Light detection and ranging devices can have multiple pixels, and corresponding echo data can be obtained for each pixel. When the scene contains a high-reflective object (i.e., a high-reflectivity or regenerative reflective object), the energy of the reflected light can be too strong, which can cause the current pixel and its surrounding pixels to receive the light energy. Therefore, if the surrounding pixels are in an open state, even if there is no real object corresponding to the depth, a current echo can be generated at the current time, resulting in an echo of an incorrect depth (referred to as a glint echo). This phenomenon is referred to as glint of the light detection and ranging device.

[0006] It is desirable to efficiently and accurately detect glint of light detection and ranging devices. SUMMARY

[0007] A first aspect of the present disclosure relates to a method for glint detection of a light detection and ranging device, comprising: obtaining, by the light detection and ranging device, respective multiple echo data of a plurality of pixels; determining, based on the respective multiple echo data of the plurality of pixels, a high-reflective object region, wherein the high-reflective object region comprises one or more high-reflective object pixels; determining, based on the high-reflective object region and a glinting logic of the light detection and ranging device, a potential glint region, wherein the potential glint region comprises one or more potential glint pixels; and classifying, based on depth and width characteristics of one or more echoes of pixels in the high-reflective object region and the potential glint region, the one or more potential glint pixels to determine glint pixels and their types.

[0008] A second aspect of the present disclosure relates to an electronic device for performing various methods according to embodiments of the present disclosure. For example, the methods include the method for glint detection of a light detection and ranging device.

[0009] A third aspect of the present disclosure relates to a computer program product comprising one or more instructions. The one or more instructions, when executed by a processor, cause implementation of various methods according to embodiments of the present disclosure. For example, the methods include the method for glint detection of a light detection and ranging device.

[0010] The foregoing summary is provided to summarize some example embodiments and to provide a basic understanding of aspects of the subject matter described herein. Thus, the foregoing summary is not intended to be a comprehensive description of the subject matter described herein and should not be interpreted as limiting the scope or spirit of the subject matter described herein in any way. Other features, aspects, and advantages of the subject matter described herein will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] A better understanding of the present disclosure can be obtained from the following detailed description in conjunction with the following drawings, in which like or similar designations refer to like or similar elements. The following detailed description includes specific details for the purpose of providing a thorough understanding of the innovative teachings. However, it will be apparent to those skilled in the art that the innovative teachings described herein can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring the concepts of the innovative subject matter.

[0012] FIG. 1 illustrates an example of a light detection and ranging device according to embodiments of the present disclosure.

[0013] FIG. 2A illustrates a schematic diagram of an application scenario for light detection and ranging according to embodiments of the present disclosure.

[0014] FIG. 2B illustrates a schematic diagram of echo data obtained in the application scenario of FIG. 2A according to embodiments of the present disclosure.

[0015] FIG. 3A illustrates a schematic diagram of an example scenario where a glint phenomenon occurs according to embodiments of the present disclosure.

[0016] FIG. 3B shows a schematic diagram of echo data obtained in the scenario of FIG. 3A, according to an embodiment of the present disclosure.

[0017] FIG. 3C schematically shows example influence ranges of a high-reflectivity object and glare.

[0018] FIG. 4 shows a schematic diagram of multi-echo data of different types of pixels that are subject to glare, according to an embodiment of the present disclosure.

[0019] FIG. 5 shows a flowchart of an example method 500 for determining a high-reflectivity object region, according to an embodiment of the present disclosure.

[0020] FIG. 6 shows a schematic diagram of an example process for adjusting a high-reflectivity object region, according to an embodiment of the present disclosure.

[0021] FIG. 7 shows one example of a determined high-reflectivity object region and a potential glare region, according to an embodiment of the present disclosure.

[0022] FIG. 8 shows a schematic diagram of regression results of a glare intensity model using an adaptive local regression method, according to an embodiment of the present disclosure.

[0023] FIGS. 9 and 10 respectively show example application scenarios of a glare detection method, according to embodiments of the present disclosure.

[0024] FIG. 11 shows an example method for glare detection for a light detection and ranging device, according to an embodiment of the present disclosure.

[0025] FIG. 12 shows an example block diagram of an electronic device for implementing various methods according to embodiments of the present disclosure.

[0026] While the embodiments described in the present disclosure can be susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the claims. DETAILED DESCRIPTION

[0027] The following description of representative applications of aspects of devices and methods according to the present disclosure is not meant to be an exhaustive description of all applications of the described embodiments. The descriptions of these examples are only to increase the understanding of the described embodiments. Thus, it will be apparent to one of ordinary skill in the art that the described embodiments can be practiced without some or all of the specific details set forth herein. In other instances, well known process steps have not been described in detail in order to avoid unnecessarily obscuring the described embodiments. Other applications are possible, and the schemes of the present disclosure are not limited to the examples described.

[0028] Light detection and ranging device example

[0029] FIG. 1 illustrates an example of a light detection and ranging device according to embodiments of the present disclosure.

[0030] In the example of FIG. 1, the light detection and ranging device 100 can include a transmitter 110, a receiver 120, a processor 130, and a memory 140. The light detection and ranging device 100 can be configured to detect and range objects in the surrounding environment by transmitting and receiving light signals, and even to perform three-dimensional modeling. For example, the transmitter 110 can be configured to transmit light signals 115 by a light source. The transmitted light signals 115 propagate through a medium to reach a target object 160 in the environment and reflect from the object 160. The reflected light signals 115’ propagate through the medium back to the light detection and ranging device 100 and are received by the receiver 120. In one embodiment, the transmitter 110 and the receiver 120 can each include an optical lens (not shown). The receiver 120 can include a detector to detect the received light signals 115’. In one embodiment, the light source is configured to transmit laser signals, the laser having a plurality of specific sequences of pulses. In this way, the light detection and ranging device 100 becomes a LiDAR (Light Detection And Ranging).

[0031] The light detection and ranging device 100 can have a plurality of pixels 125a, 125b, …, 125n. For example, the receiver 120 can include an array of detectors, each of which can correspond to a pixel. It should be noted that hereinafter, when it is not necessary to distinguish the plurality of pixels 125a, 125b, …, 125n from each other, they can be collectively referred to as pixels 125. Each pixel 125 can receive and detect light signals respectively. The detector corresponding to each pixel can be turned on or off according to the lighting logic of the light detection and ranging device 100, etc. In other words, from the perspective of the pixels, each pixel can be in an on or off state according to the lighting logic of the light detection and ranging device 100, etc. In some embodiments, the receiver 120 can include an array of single-photon avalanche diodes (SPADs). A SPAD is a pixel structure that can amplify the electrons of a single incident photon using avalanche multiplication technology, forming an avalanche-like superposition, so as to be able to detect a single photon. A LiDAR that utilizes a SPAD array can be referred to as a SPAD LiDAR.

[0032] In this example, processor 130 may be configured to be coupled to transmitter 110, receiver 120, and memory 140. Processor 130 may execute one or more modules and / or processes to enable the light detection and ranging device 100 to perform various functions. These functions include controlling the transmission and reception of optical signals (e.g., laser signals) and various other functions described below. For example, processor 130 may be configured to perform these functions by reading and executing computer programs, code, or executable instructions stored in memory 140. In some embodiments, processor 130 may include a microprocessor, microcontroller, digital signal processor, central processing unit (CPU), graphics processing unit (GPU), etc.

[0033] Memory 140 may be a non-transitory computer-readable storage medium, including but not limited to electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multipurpose disc (DVD), memory sticks, floppy disks, mechanical encoding devices, and any suitable combination thereof.

[0034] Figure 1 shows only one target object 160. The target object can be any type of object to be detected, including but not limited to trees, furniture, roadblocks, vehicles, pedestrians, workpieces, road signs, etc. In reality, there are usually multiple objects of various types in the environment, which makes the detection, ranging, and 3D modeling of target objects complex.

[0035] Figure 2A illustrates a schematic diagram of an application scenario for light detection and ranging according to an embodiment of the present disclosure. Those skilled in the art should understand that the application scenario shown in Figure 2A is merely an example, and the application scenarios for light detection and ranging disclosed herein are not limited thereto. Figure 2B illustrates a schematic diagram of echo data obtained in the application scenario of Figure 2A according to an embodiment of the present disclosure. This example application scenario will be described below with reference to the light detection and ranging device 100 in Figure 1.

[0036] As shown in FIG. 2A, in a light detection and ranging scheme, the transmitter 110 is configured to emit light signals 210 and 220. The light signal 210 propagates through the medium to reach the surface of the front-side object 260 and reflects off the object. The reflected light signal 210' propagates back through the medium to the light detection and ranging device 100 and is received by the pixel 125a. The light signal 220 propagates through the medium to reach the edge of the front-side object 260. Then, a portion of the light signal 220 reflects off the edge of the front-side object 260, and the reflected light signal 220' returns to the light detection and ranging device 100 and is received by the pixel 125b. Another portion of the light signal 220 continues to propagate through the medium to reach the back-side object 270 and reflects off the surface of the object. The reflected light signal 220" returns to the light detection and ranging device 100 and is received by the pixel 125b. Each pixel 125 of the receiver 120 can be configured to record the number of received photons at different times, thereby generating return data. The generated return data can be used by the light detection and ranging device 100 for ranging.

[0037] As mentioned above, the light detection and ranging device 100 can have multiple pixels 125a, 125b,..., 125n. Here, the meaning of pixel is similar to that in image sensing, but the data recorded for a pixel is not an RGB color component, but the number of received photons (i.e., return data). Each pixel can correspond to one or more return data. In the example of FIG. 2A, the light signal 210 is received by 125a after hitting the A1 point on the front-side object, thereby generating one return data for 125a. The light signal 220 is received by the pixel 125b after hitting the A2 point on the front-side object and the B2 point on the back-side object, thereby generating two return data for the pixel 125b.

[0038] For simplicity of illustration, only 2 pixels of the light detection and ranging device 100 are shown in FIG. 2A. In one or more embodiments, there can be a larger number of pixels depending on the configuration of the light detection and ranging device 100.

[0039] In FIG. 2A, only the light signals emitted by the transmitter 110 and the real object returns are shown. In this disclosure, the real object returns belong to the signal returns (may also be referred to as valid returns) that can be used for further processing. However, in general, each pixel 125 of the receiver 120 will also receive light signals that originate from the propagation of ambient light (e.g., sunlight). These light signals constitute noise returns with respect to the real object returns. The noise returns can interfere with the detection of the signal returns, thereby affecting the detection accuracy of the signal returns. Methods of filtering out the signal returns will be described later.

[0040] The multiple echo data corresponding to pixels 125a and 125b are shown in FIG. 2B by way of examples (1) and (2), respectively. In the examples of FIG. 2B, the horizontal coordinate represents time, thus can indicate the time of flight of the echoes, and further indicate the distance or depth in various application scenarios. The vertical coordinate represents the number of received photons, thus can indicate the intensity of the echoes. In the examples, there can also be some lower peaks. Since these lower peaks do not meet the judgment criteria (e.g. intensity and duration conditions) of the echoes, they can not be considered. It should be noted that in the following, each echo is schematically shown as a simple geometric shape (e.g. triangle or trapezoid), but those skilled in the art can understand that the shape is only an example for ease of illustration and does not limit the present disclosure, and in practice the received echo waveform can have a more complex shape, and is also applicable to embodiments according to the present disclosure.

[0041] Referring to example (1), echo data a corresponds to the light signal reflected by point A1 of the front object 260, thus is a signal echo. Echo data b and c both correspond to noise echoes. Referring to example (2), echo data d corresponds to the light signal reflected by point A2 of the front object 260, and echo data e corresponds to the light signal reflected by point B2 of the rear object 270, thus are both signal echoes. Echo data f corresponds to a noise echo.

[0042] It should be understood that in the case of higher peaks, noise echoes can be mistakenly identified as signal echoes, thus interfering with the detection of signal echoes. In examples (1) and (2), noise echoes b, c or f can all affect the detection accuracy of the corresponding signal echoes. This can be disadvantageous for the detection and ranging of target objects and three-dimensional modeling.

[0043] Types of multiple echo data

[0044] In the context of the present disclosure, data including relevant information of multiple echoes can be referred to as multiple echo data, such as the multiple echo data including echoes a, b, c described in example (1) of FIG. 2B and the multiple echo data including echoes d, e, f described in example (2).

[0045] It should be noted that although FIG. 2B illustrates multi-echo data including a complete waveform of echo intensity versus time, multi-echo data is not limited to this. Multi-echo data can include histogram data, echo-pattern multi-echo data, ranging-pattern multi-echo data, or other data including multiple echoes. Histogram data is raw data measured by a light detection and ranging device, which conveys richer information through a larger amount of data. For example, histogram data can include the number of photons received at each time point. Echo-pattern multi-echo data is data obtained by processing raw data, which retains only a portion of information in raw data related to echoes. For example, echo-pattern multi-echo data can include intensities at all time points within the width of each echo. Ranging-pattern multi-echo data is data obtained by further processing echo-pattern multi-echo data, which retains only information necessary for ranging, with the least amount of information. For example, ranging-pattern multi-echo data can include the start time, end time, and peak position and intensity of each echo, so that the depth, width, and intensity of each echo can be determined. In embodiments of the present disclosure, a flare detection operation can be performed based only on the depth, width, and intensity of an echo, and further flare elimination can be implemented. That is, even ranging-pattern multi-echo data with a smaller amount of data is sufficient for implementing flare detection and elimination according to the present disclosure.

[0046] Generation of high-reflective objects and flares

[0047] As described above, a pixel of a light detection and ranging device can receive light reflected back from a real object to generate echo data, and determine a distance (depth) to the object from the position of the echo. However, when a high-reflective object (i.e., a high-reflectivity or regenerative reflective object) is included in a scene, the light energy reflected back can be too strong, which can cause the current pixel and its surrounding pixels to all receive the light energy. Thus, if the surrounding pixels are in an on state, an echo can be generated at the current time point even if there is no real object corresponding to the depth, resulting in an echo with an erroneous depth. Such an echo with an erroneous depth can be referred to as a flare.

[0048] FIG. 3A illustrates a schematic diagram of an example scenario in which a glint phenomenon occurs, according to an embodiment of the present disclosure. As shown in FIG. 3A, the example scenario includes a normal object 360 and a high-reflective object 370. The high-reflective object 370 is a high-reflectivity or retro-reflective object. Typical high-reflective objects can include road signs, street lamps, reflective tapes, etc. in a road. In the example scenario, it is assumed that the normal object 360 is farther (deeper) than the high-reflective object 370. In the example scenario, the emitter 110 is configured to emit light signals 310a and 320a. The light signal 310a propagates through the medium to reach the surface of the normal object 360 and reflects from the object. The reflected light signal 310b propagates back through the medium to the LIDAR device 100 and is received by the pixel 125c. The light signal 320a propagates through the medium to reach the high-reflective object 370 and reflects from the surface of the object. The reflected light signal 320b returns to the LIDAR device 100 and is received by the pixel 125d, which is adjacent to the pixel 125c. However, because the object 370 is a high-reflective object, the reflected light signal 320b is too strong in energy, causing crosstalk such that pixels surrounding the pixel 125d also receive a portion of the energy of the reflected light signal 320b. For example, in the example scenario, the pixel 125c, which is adjacent to the adjacent pixel 125d, receives the light signal 320c as a result of the crosstalk. This causes the pixel 125c to produce a return at a time corresponding to the depth of the high-reflective object 370 even though there is no real object at that depth. This false depth return is referred to as a glint return. Moreover, it is found that, according to the principle of the glint generation, the farther the distance from the pixel that receives the reflected light from the high-reflective object (i.e., the high-reflective object pixel), the lower the degree of the crosstalk effect. Therefore, the strength of the glint return is inversely related to the distance from the high-reflective object pixel to the pixel at which the glint occurs.

[0049] Moreover, because the light signal 320b is too strong in energy, the light signal 320b is reflected by the LIDAR device 100 when it reaches the LIDAR device 100, resulting in a light signal 320d. The light signal 320d propagates through the medium again to reach the same high-reflective object 370 and reflects from the surface of the object. The reflected light signal 320e returns to the LIDAR device 100 again and is received by the pixel 125d. Therefore, for the pixel 125d, another return is produced at a time corresponding to approximately twice the distance of the depth of the high-reflective object 370. This additional return can be referred to as a ghost return. It is noted that if the reflected light signal 320e is still strong enough in energy, this reflection phenomenon can continue, resulting in additional returns at times corresponding to three times the distance, four times the distance, and so on.

[0050] FIG. 3B shows a schematic diagram of the multi-bounce data obtained in the scenario of FIG. 3A, according to an embodiment of the present disclosure. FIG. 3B shows the multi-bounce data obtained by pixel 125d and 125c respectively through examples (1) and (2). It should be noted that in the present disclosure, both ghost bounce and flare bounce are considered as signal bounces. In the examples of FIG. 3B, the description of noise bounces will be omitted.

[0051] The multi-bounce data of pixel 125d is first described with reference to example (1), bounce 380 corresponds to light signal 320b, and its depth corresponds to the depth of high-reflective object 370. As schematically depicted in the figure, due to the very high intensity of light signal 320b, both the intensity and width of bounce 380 are large, and even the saturation of the waveform occurs. Bounce 382 corresponds to light signal 320e, and can be referred to as a ghost bounce. As described above, according to the principle of generation of ghost bounces, bounce 382 occurs at a position approximately 2 times the time of bounce 380. That is, assuming the depth of bounce 380 is t, the depth of bounce 382 is approximately 2t. It should be noted that although not shown, additional bounces at time 3t, 4t, and so on can also exist.

[0052] Then, the multi-bounce data of pixel 125c is described with reference to example (2), bounce 384 corresponds to crosstalk signal 320c, and can be referred to as flare. Due to the principle of generation of flares, the depth of bounce 384 is substantially consistent with the depth of bounce 380, i.e., the depth of bounce 384 is also approximately t. However, for pixel 125c, there can be no real object at depth t. Bounce 386 corresponds to light signal 310b, and its depth corresponds to the depth of real object 360. The existence of flare bounce 384 can cause the depth of ordinary object 360 to be erroneously estimated as the depth of high-reflective object 370.

[0053] FIG. 3C schematically shows example impact ranges of high-reflective objects and flares. As shown in FIG. 3C, in one example, it is assumed that the pixels in the gray solid area in the figure are the pixels corresponding to the high-reflective object, i.e., these pixels receive high-energy reflected light from the high-reflective object. It is assumed that according to the lighting logic (e.g., horizontal lighting) of the light detection and ranging device, the pixels in the area shown by the dashed line frame are simultaneously in an on state. Therefore, the pixels in the area shown by the dashed line frame can be affected by flares. In one example, it is assumed that the pixels in the hatched area experience flare phenomenon. As a result, the depths of these pixels can be estimated to be consistent with the depth of the high-reflective object, even though there is no real object at this depth for these pixels.

[0054] Types of flares

[0055] Depending on the presence of the real object echo and its positional relationship with the glint echo, the pixels where glint occurs can be classified into different types. In some embodiments, the pixels where glint occurs can be classified into a double-echo glint type, a fusion glint type, and a single-echo glint type.

[0056] FIG. 4 shows a schematic diagram of multi-echo data of different types of pixels where glint occurs, according to an embodiment of the present disclosure. It should be noted that in this example, the description of noise echoes will be omitted.

[0057] Example (0) of FIG. 4 depicts the multi-echo data of the pixels corresponding to the high-reflective object (i.e., the pixels receiving high-intensity reflected light from the high-reflective object) as a reference. Example (0) can be the multi-echo data of pixel 125d. For example, echoes 402 and 404 can correspond to echoes 380 and ghost echo 382 in FIG. 3B, respectively.

[0058] Examples (1) to (3) of FIG. 4 depict the multi-echo data of the double-echo glint type, the fusion glint type, and the single-echo glint type, respectively. For the double-echo glint type, its echo data includes both glint echo 412 and real object echo 414, and the real object echo 414 is after and mutually separated from the glint echo 412. For the fusion glint type, its echo data also includes glint echo 422a and real object echo 422b, and the real object echo 422b is also after the glint echo 422a. However, the glint echo 422a and the real object echo 422b partially overlap together, thus forming one wider fused echo 422. For the single-echo glint type, its multi-echo data can only include one signal echo 432. The signal echo 432 can be a glint-only echo, or can also be a glint echo and a real object echo completely overlapping together. As mentioned above, according to the principle of echo generation, the depths of echoes 412, 422, and 432 in examples (1) to (3) substantially correspond to the high-reflective object. Examples (1) to (3) can be the multi-echo data of pixel 125c.

[0059] Glint can cause an impact on the accuracy of ranging. For example, assuming that the ordinary object 360 is located at a more distant (deeper) position relative to the high-reflective object 370, in ranging or depth map creation prioritizing short-distance responses, the glint phenomenon can cause the depth of the high-reflective object 370 to be wrongly estimated as the depth of the ordinary object 360, or cause the real object 360 in the rear to disappear in the depth map. In addition, data with glint can also cause an impact on downstream object detection tasks. Therefore, it is necessary to effectively detect glint for elimination.

[0060] There are methods that use histogram data to detect glint based on mathematical models. However, due to the large amount of histogram data, current light detection and ranging device vendors often use range mode multi-echo data, which does not have the information necessary for these methods. In addition, glint detection methods based on mathematical models often rely on historical data or simulated data in the laboratory, and cannot adaptively estimate according to the actual data detected by the light detection and ranging device, so they may not perform well in many scenarios.

[0061] In embodiments of the present disclosure, a high-reflective object region is determined based on the depth, width and intensity of the pixel-based multi-echo data, a potential glint region is determined and classified, so as to detect glint pixels. On the one hand, this can make the method of the present disclosure applicable to various types of multi-echo data including range mode multi-echo data, thereby improving the versatility. On the other hand, the demand for computing and storage resources can be reduced.

[0062] In embodiments of the present disclosure, the parameters are adaptively estimated from the data obtained from actual measurements, which can improve the accuracy of glint detection, thereby improving the accuracy of ranging, while also improving the versatility of the method.

[0063] Glint detection processing example

[0064] Still referring to FIG. 1, according to embodiments of the present disclosure, the memory 140 of the light detection and ranging device 100 can include modules, such as an echo data acquisition module 142, a high-reflective object region determination module 144, a potential glint region determination module 146, and a pixel classification module 148, in order to improve the performance of the light detection and ranging device 100 in the above and other aspects.

[0065] As an example, the echo data acquisition module 142 can be configured to obtain respective multi-echo data for a plurality of pixels. The high-reflective object region determination module 144 can be configured to determine a high-reflective object region based on the respective multi-echo data for the plurality of pixels, wherein the high-reflective object region includes one or more high-reflective object pixels. The potential glint region determination module 146 can be configured to determine a potential glint region based on the high-reflective object region and a lighting logic of the light detection and ranging device, wherein the potential glint region includes one or more potential glint pixels. The pixel classification module 148 can be configured to classify the one or more potential glint pixels based on depth and width characteristics of one or more echoes of the pixels in the high-reflective object region and the potential glint region, to determine glint pixels and their types. The detailed operations of each module can be understood in conjunction with the further description of embodiments of the present disclosure.

[0066] In embodiments, the term module is used to represent an example partitioning of executable instructions for ease of discussion. It is noted that the module partitioning can be done differently, and one or more functions can be arranged differently (e.g., merged into a smaller number of modules, partitioned into a larger number of modules, etc.). Furthermore, the functions and modules described herein can be implemented in whole or in part by software and / or firmware executable on a processor, or can be implemented in whole or in part by hardware (e.g., special-purpose processing circuitry, etc.).

[0067] The specific steps of the processing of each module will be described in more detail below.

[0068] Acquisition of multi-echo data

[0069] As described above, first, the respective multi-echo data of the plurality of pixels can be obtained by the lidar device 100. For example, the plurality of pixels can correspond to the plurality of pixels 125a, 125b, …, 125n in FIG. 1.

[0070] Since the glint detection method according to embodiments of the present disclosure only uses the depth, width, and intensity of the echoes, the multi-echo data can be obtained from any one of the histogram data, the multi-echo data of the echo mode, or the multi-echo data of the ranging mode. Therefore, in some embodiments, the multi-echo data can include the histogram data, the multi-echo data of the echo mode, or the multi-echo data of the ranging mode. This makes the glint detection method of the present disclosure applicable to the lidar devices actually produced by manufacturers, thereby improving the practicality and universality. Furthermore, since the operation only involves a small amount of data, the demand for computing and storage resources can be reduced.

[0071] It should be noted that the acquired multi-echo data is not limited to the multi-echo data described above, and can be any type of multi-echo data as long as the depth, width, and intensity of each echo can be determined from the multi-echo data.

[0072] Determination of high-reflective object region

[0073] After the multi-echo data is acquired, a high-reflective object region including one or more high-reflective object pixels can be determined based on the respective multi-echo data of the plurality of pixels. Here, the high-reflective object pixel refers to a pixel considered to receive reflected light from a high-reflective object, or can be said to be a pixel considered to correspond to a high-reflective object, and the high-reflective object region is a region formed of such pixels.

[0074] FIG. 5 shows a flowchart of an example method 500 for determining a high-reflective object region according to embodiments of the present disclosure. The example method 500 can be performed by various lidar devices, and example operations of the method 500 are described below in connection with the lidar device 100.

[0075] As shown in Figure 5, method 500 may include, for a first pixel among a plurality of pixels, obtaining one or more signal echoes of the first pixel based on the intensity of a plurality of echoes of the first pixel (510). In other words, the plurality of echoes of the first pixel may be filtered according to the intensity of the echoes to filter out noise echoes and select signal echoes. For example, the first pixel may be any one of a plurality of pixels (e.g., 125a, 125b, ..., 125n).

[0076] In some embodiments, to obtain one or more signal echoes, echoes that meet the first threshold can be identified as signal echoes based on a comparison of the intensity of a single echo with a first threshold. For example, the first threshold can be a fixed threshold, such as half of the maximum echo intensity max_value of the photodetector and ranging device 100. For example, suppose the first pixel has 5 echoes, the intensity of each echo is denoted as x0, ..., x4, and the intensity of the i-th echo is denoted as x_i. i As an example, for each echo of the first pixel, if the intensity of that echo x i If the value is greater than 0.5max_value, the echo is considered a signal echo; otherwise, it is considered a noise echo. However, when ambient light (e.g., sunlight) is strong, the peak value of noise echoes may be high. In this case, using this method based on comparing the intensity of a single echo with a threshold may cause noise echoes to be incorrectly identified as signal echoes, thus affecting subsequent processing.

[0077] Therefore, alternatively, echoes that meet a second threshold can be identified as signal echoes based on a comparison of the relative intensity relationships between multiple echoes with a second threshold, where the second threshold includes a set threshold or an adaptive threshold. For example, also assuming the last echo (i.e., the echo with intensity x4) is a noise echo, then the signal echo can be identified based on the intensity x of each echo. i The signal echo is determined by comparing it with the intensity x4 of the last echo. In one example, if x... i If x4 > 0.1max_value, then the echo is considered a signal echo. For example, in another embodiment, the second threshold can be adaptively calculated based on x4, denoted as d(x4). In one example, based on simulation results, d(x4) can take its maximum value when x4 = 0.5max_value, and decrease as x4 moves away from 0.5max_value. In this example, if x... i If -x4 > 3d(x4), then the echo is considered a signal echo. In some embodiments, the signal echo can also be determined based on a comparison of the relative intensity relationship between two consecutive echoes and a second threshold. In one example, if x i+1 -x ithreshold k, it is considered that the i-th and previous echoes are signal echoes, wherein the threshold k can be fixed or adaptively determined similarly as described above. Although the difference between the intensities of two echoes is taken as the relative intensity relationship between the two echoes above, the ratio between the intensities of two echoes (e.g., x i / x4or x i / x i+1 ) can also be taken as the relative intensity relationship between the two echoes. In addition, the method of screening signal echoes (e.g., the determination method of the second threshold) can be appropriately selected according to actual conditions.

[0078] As shown in FIG. 5, the method 500 can further include determining the first pixel as a high-reflective-object pixel based on the presence of a ghost echo in one or more signal echoes (520). As described with reference to FIGS. 3A and 3B, for a pixel receiving reflected light from a high-reflective object, a ghost echo will appear in its multi-echo data at a position corresponding to approximately twice the depth of the high-reflective object. Therefore, this rule can be utilized to inversely determine whether a pixel is a high-reflective-object pixel. That is, a pixel in which a ghost echo exists in its signal echoes can be determined as a high-reflective-object pixel. Specifically, the determination of the presence of a ghost echo in one or more signal echoes can be made by the condition that the one or more signal echoes include at least two signal echoes, and at least one signal echo with a larger depth (denoted as t2) is between twice and twice plus a small threshold of another signal echo with a smaller depth (denoted as t1, and t1 < t2). When the condition is satisfied, it is determined that a ghost echo exists in the one or more signal echoes (i.e., at depth t2), and at this time, the signal echo with the smaller depth (t1) can be referred to as a potential high-reflective-object echo. It should be noted that the determination condition of the ghost echo can also be t thre , where t thre is a very small threshold. thre1 <t2-2t1<t thre2 , where t thre1 and t thre2 are both very small thresholds.

[0079] In addition, as described with reference to FIGS. 3A and 3B, for a pixel receiving reflected light from a high-reflective object, the width and intensity of the echo corresponding to the depth of the high-reflective object (e.g., echo 380) are both large due to the very strong energy of the reflected light. This characteristic can be utilized to improve the accuracy of determining a high-reflective-object pixel. That is, on the basis of the presence of a ghost echo in one or more signal echoes, when the width of the potential high-reflective-object echo is greater than a threshold w thre , and the intensity of the potential high-reflective-object echo is greater than a threshold y threIn some embodiments, the first pixel can be determined to be a high-reflective object pixel based on the second echo in the one or more signal echoes of the first pixel constituting a ghost echo of the first echo and the intensity and / or width of the first echo being greater than a respective threshold.

[0080] In some embodiments, the determination based on the comparison of the width and intensity of the echo with the respective threshold can be omitted, i.e. the determination of whether a pixel is a high-reflective object pixel can be made based on the ghost echo only. In this case, the potential high-reflective object echo directly becomes a high-reflective object echo. In embodiments involving the high-reflective object region adjustment as described below, the first pixel determined to be a high-reflective object pixel in step 520 can be referred to as an exact high-reflective object pixel, and the region constituted by the exact high-reflective object pixels can be referred to as an exact high-reflective object region.

[0081] In some embodiments, the method 500 can be performed for each pixel in the plurality of pixels. In some embodiments, in the case that no high-reflective object pixel is determined by the method 500 for all pixels 125 (i.e. the plurality of pixels does not include an exact high-reflective object pixel), it can be determined that there is no high-reflective object and no glare, and the subsequent steps can be skipped.

[0082] Adjustment of high-reflective object region

[0083] In some embodiments, the current high-reflective object region (e.g. the exact high-reflective object region) can also be adjusted by adding or removing high-reflective object pixels. Since there can be false positives or false negatives of high-reflective object pixels, the high-reflective object region can be optimized by the adjustment.

[0084] Figure 6 shows a schematic diagram of an example process for adjusting a high-reflective object region according to embodiments of the present disclosure. As shown in Figure 6, (a) of Figure 6 shows a high-reflective object region determined by, for example, the method 500 of Figure 5. In (b) of Figure 6, high-reflective object regions with too small an area can be removed, which can be done by a connected component area threshold or a morphological operation. Since in real situations, high-reflective object regions are generally unlikely to be too small (e.g. only one or two pixels) and noise is also unlikely to be falsely detected as high-reflective object in large areas, this removal operation helps to prevent false positives of high-reflective object pixels. In addition, gaps and holes in the high-reflective object region can also be eliminated, which can be done by a morphological operation. This step helps to prevent false negatives of high-reflective object pixels.

[0085] Then, all sub-regions of the high-reflective object region can be obtained using connected component detection, and in (c) of FIG. 6, the features of each pixel can be calculated. The features of a pixel referred to herein can refer to the depth, intensity, and / or width of the echo of the pixel. For example, the features can be for a certain echo in the pixel (e.g., the echo corresponding to the high-reflective object), or can be for multiple echoes in the pixel (e.g., a combined or weighted average, etc.). Further, the features of a sub-region can be calculated for each sub-region of the high-reflective object region. For example, the features of a region or sub-region can include statistical computed values of the features of all pixels in the region or sub-region, such as the mean, median, standard deviation, quantile, etc. of the depth, intensity, and / or width of the echoes.

[0086] In (d) of FIG. 6, pixels adjacent to the current high-reflective object region and having features matching those of the high-reflective object pixels in the current high-reflective object region can be determined as potential high-reflective object pixels, and the current high-reflective object region and the potential high-reflective object pixels can be combined to obtain an adjusted high-reflective object region. This step is also referred to as region growing. Matching the features of the high-reflective object pixels in the current high-reflective object region can refer to matching the features of a certain pixel in the high-reflective object region (e.g., a neighboring pixel or a pixel at the center of the region), or can also refer to matching the features of the entire high-reflective object region or a certain sub-region. As an example, assuming the mean depth of the high-reflective object region is μ and the standard deviation is σ, pixels adjacent to the pixels in the current high-reflective object region and having depths within the range of μ ± nσ can be considered as potential high-reflective object pixels, e.g., where n = 3. This step can be iteratively repeated until there are no more pixels adjacent to the current high-reflective object region that satisfy the feature matching condition. If two high-reflective object sub-regions are connected together through region growing, they are considered as one high-reflective object sub-region. The growing of the high-reflective object region can help prevent the miss-detection of high-reflective object pixels. Further, in real-world scenarios, many high-reflective objects (e.g., a street sign) often have non-high-reflective parts (e.g., the text in the street sign), which can separate the high-reflective object region into multiple sub-regions, potentially causing trouble for subsequent processing. Thus, the growing of the high-reflective object region can help re-connect these sub-regions that belong to the same high-reflective object. It should be noted that the high-reflective object pixels determined through region growing (e.g., the pixels shown in cross-hatched lines in (d) of FIG. 6) are referred to as potential high-reflective object pixels to indicate their likelihood of being high-reflective object pixels and to distinguish them from the exact high-reflective object pixels. In some cases, it is not excluded that a potential high-reflective object pixel is an ordinary object pixel or a glint pixel.

[0087] It should be noted that although (b) to (d) of FIG. 6 shows a continuous process, the high-reflective object region adjustment of the present disclosure is not limited to the process shown in FIG. 6. For example, only some of these steps can be included, such as only performing the growth of the high-reflective object region, or not performing the growth of the high-reflective object region.

[0088] Determination of potential glare region

[0089] After the high-reflective object region is determined, a potential glare region including one or more potential glare pixels can be determined based on the high-reflective object region and the lighting logic of the light detection and ranging device. Here, a potential glare pixel refers to a pixel that can be affected by glare from the high-reflective object, or in other words, a non-high-reflective object pixel that can have a return corresponding to the high-reflective object. In embodiments of the present disclosure, actual glare pixels can be determined from the potential glare pixels. The lighting logic of the light detection and ranging device is related to its design. For example, the lighting logic can include horizontal direction lighting, vertical direction lighting, and combinations thereof.

[0090] As described above with reference to FIGS. 3A to 3C, according to the principle of generation of glare, pixels that are turned on at the same time as pixels receiving high-energy reflected light from the high-reflective object are prone to glare phenomenon. Therefore, in some embodiments, a pixel region that is turned on at the same time as one or more high-reflective object pixels in the high-reflective object region can be determined as a potential glare region.

[0091] FIG. 7 shows one example of a determined high-reflective object region and potential glare region according to embodiments of the present disclosure.

[0092] Suppose that the high-reflective object region 710 is determined by the method of FIG. 5 (and optionally, FIG. 6), which includes one or more high-reflective object pixels 715. Suppose that the pixels in the region 705 shown by the dashed box are turned on at the same time according to the lighting logic of the light detection and ranging device 100, then the region outside the high-reflective object region 710 in the region 705 (as shown by the vertical hatched line) is determined as a potential glare region 720, and the pixels therein are all potential glare pixels 725.

[0093] Classification of pixels

[0094] After the potential glare region is determined, one or more potential glare pixels can be classified to determine glare pixels and their types based on the depth and width characteristics of one or more returns of the pixels in the high-reflective object region and the potential glare region. Here, a glare pixel refers to a pixel that is affected by glare. For example, multi-return data can be obtained for one or more returns by the return data acquisition module 142, and one or more signal returns can be obtained by step 510.

[0095] In some embodiments, the one or more potential glint pixels can be classified into one of a plurality of pixel types: high-reflective object pixel, double-bounce glint pixel, fusion glint pixel, single-bounce glint pixel, or non-glint pixel. Glint pixels can include pixels of type double-bounce glint pixel, fusion glint pixel, or single-bounce glint pixel. Here, high-reflective object pixel, double-bounce glint pixel, fusion glint pixel, single-bounce glint pixel can correspond to high-reflective object, double-bounce glint, fusion glint, single-bounce glint type described above with reference to FIG. 4, respectively. Non-glint (also referred to as normal object) pixel can correspond to pixels other than high-reflective object pixel and glint pixel. It should be noted that the present disclosure is not limited to the pixel types described above, and one or more potential glint pixels can be classified into different types as needed. For example, in alternative embodiments, one or more potential glint pixels can be classified into glint pixel and non-glint pixel only according to whether the depth of the echo is close to that of high-reflective object region.

[0096] In some embodiments, a separate pixel classification model can be established for each pixel type, and pixel classification can be performed using the pixel classification model. For example, the pixel classification model can be a probabilistic model. In some embodiments, parameters of a plurality of pixel classification models corresponding to a plurality of pixel types can be calculated based on the depth and width of the echo of one or more high-reflective object pixels in a high-reflective object region. In embodiments where exact high-reflective object pixels and potential high-reflective object pixels are distinguished, the high-reflective object region here can refer to exact high-reflective object region. The echo of one or more high-reflective object pixels can refer to high-reflective object echo of one or more high-reflective object pixels. The calculation based on the depth and width of the echo of one or more high-reflective object pixels in a high-reflective object region can include some statistical calculation (e.g., mean and variance) based on the depth and width of the echo of all or part of the high-reflective object pixels in the high-reflective object region, or can include the calculation based on the depth and width of the echo of a representative pixel (e.g., a pixel adjacent to a potential glint pixel, a center pixel) in the high-reflective object region.

[0097] In some embodiments, for a first potential glint pixel in a potential glint region, a corresponding score or probability can be calculated for the first potential glint pixel using a plurality of pixel classification models based on the depth, width of the echo of the first potential glint pixel and / or the distance from the first potential glint pixel to a high-reflective object region. Then, the pixel type of the first potential glint pixel can be determined based on the calculated score or probability. The first potential glint pixel can be any potential glint pixel in the potential glint region. The distance of a pixel to a high-reflective object region can refer to the distance (e.g., Euclidean distance, Manhattan distance, etc.) from the pixel to a representative pixel (e.g., a pixel adjacent to the pixel, a center pixel, etc.) in the high-reflective object region, or can refer to the statistics (e.g., mean) of the distance from the pixel to a plurality of pixels (e.g., all pixels) in the high-reflective object region.

[0098] Below, the step of the above classification is described in more detail, taking the case of classification as high-reflective object pixel, double-bounce glint pixel, fusion glint pixel, single-bounce glint pixel or non-glare pixel as an example. Assume that the multiple echoes of the first pixel are x1, x2,... ; x id iw represent the depth and width of echo i, respectively; and the mean and variance of the depth and the mean and variance of the width of the echoes of the pixels in the exact high-reflective object region are denoted as μ d and μ w Then, they can be modeled by the following expressions.

[0099] The model corresponding to the high-reflective object pixel is:

[0100] P(x1 is signal echo) I (potential high-reflective pixel) Sim(x1) (1 - P(x2 is signal echo)) (1)

[0101] wherein, Sim(x1) represents the product of the probabilities that the depth and width of echo x1 are similar to the corresponding depth and width of the pixels in the high-reflective object region. I() is a discriminant function, which is 1 if the condition is satisfied, and 0 otherwise. In some embodiments, the condition of the discriminant function can be derived by using historical data to train. P(x i is signal echo) represents the probability that x i is signal echo. If the adaptive threshold is used in step 510 of the method 500, the probability can be calculated using a normal distribution function with x4 as the mean and d(x4) as the standard deviation, which can be, for example, 1 - Φ(k - (x i - x4) / d(x4) 2 ). Wherein Φ is the standard Gaussian cumulative probability function, and the larger x i - x4 is, the larger the probability is. k is a preset threshold, and the larger k is, the smaller the probability is. Preferably, k = 3. The discriminant function can also be used to calculate the probability that x i is signal echo, i.e. P(x i is signal echo) = I(x i is signal echo). (1 - P(x2 is signal echo)) represents the probability that x2 is not signal echo, which corresponds to the fact that if x2 is signal echo, the first potential glint pixel will be identified as high-reflective object echo in step 520, instead of being identified as potential glint pixel. Expression (1) can represent the probability that the first pixel is potential high-reflective pixel, its echo x1 is signal echo, its echo x2 is not signal echo, and the characteristics of echo x1 are similar to the high-reflective object region. ​​​

[0102] The model corresponding to the double-echo glint pixel is:

[0103] (x1 is signal echo) Rel d (x1) Rel w (x1) P (x2 is signal echo) (2)

[0104] wherein,

[0105] h0 and h1 are preset thresholds. Since the depth of glint is likely to be larger than that of the high-reflective object, and the bandwidth is likely to be smaller than that of the high-reflective object, h0≥1 and h1≤1 can be taken. In an example, h0=10, x d -μ d The probability of x d -μ d = -5 is greater than that of x d The calculation of Rel

[0106] f(s) can be a monotonically increasing function, can indicate the relationship between the difference between the depth of glint and the depth of the high-reflective object and s, and can be estimated by experimental measurement data. Preferably, f(s) can be represented using a piecewise function, a quadratic function about s can be used when s is relatively small (s<10), and a linear function can be used when s is relatively large (s>=10).

[0107] The model corresponding to the fusion glint pixel is:

[0108] P(x1 is signal echo) Rel d (x1) Lrg w (x1) (3)

[0109] wherein,

[0110] wherein, l>0 is a preset threshold, and Φ is a Gaussian cumulative probability function. As described above, in the case of fusion glint, the echo of glint and the echo of signal will be fused into one echo. Therefore, the bandwidth thereof will be wider than that of a general echo. Lrg w (x1) can indicate whether the echo x1 has a larger width, i.e., conforms to the generation principle of fusion glint.

[0111] The model corresponding to the single-echo glint pixel is:

[0112] P(x1 is signal echo) Rel d (x1) Nrw w(x1) (1 - P(x2 is a signal echo)) (4)

[0113] wherein,

[0114] wherein, r > 0 is a pre-set threshold, Nrw w (x1) can indicate whether the echo x1 is narrower (such as narrower than μ w by at least r), i.e. the fact that glint echoes of single-echo glint pixels tend to be narrower than echoes of real objects.

[0115] The model corresponding to non-glint pixels is:

[0116] wherein, U is a uniform distribution function, μ s , is the mean and variance of signal echoes of non-high-reflective object pixels.

[0117] The scores or probabilities (in this example, probabilities) of each type can be calculated using the above expressions (1) to (5), respectively. In some embodiments, the type with the highest calculated score or probability can be taken as the type of the first potential glint pixel, and / or only the types with a calculated score or probability above a certain threshold can be taken as the type of the first potential glint pixel. In alternative embodiments, no model calculation can be used for non-glint pixels, but the model scores or probabilities of high-reflective object pixels, double-echo glint pixels, fusion glint pixels, single-echo glint pixels are calculated. If the maximum of the calculated scores or probabilities are all smaller than a certain threshold, the pixel is classified as a non-glint pixel.

[0118] It should be noted that in embodiments where exact high-reflective object pixels and potential high-reflective object pixels are distinguished, the potential high-reflective object pixels can be similarly classified in addition to the potential glint pixels, to more accurately detect high-reflective objects and glints. In some embodiments, only potential high-reflective object pixels can be classified into the high-reflective object pixel category.

[0119] Thus, the category of each potential glint pixel (and potential high-reflective object pixel) can be determined, thereby determining the glint region where glint pixels exist and the type of each glint pixel therein. The determined glint region and glint type can be helpful for subsequent glint intensity estimation and / or glint removal.

[0120] Estimation of glint intensity

[0121] In embodiments of the present disclosure, the intensity of glare can be estimated. For example, for a first glare pixel in a potential glare region, a distance of the first glare pixel to the high-reflective object region can be calculated. The first glare pixel can be any glare pixel determined through the classification of the potential glare pixels. As mentioned above, the distance of a pixel to the high-reflective object region can refer to a distance (e.g., Euclidean distance, Manhattan distance, etc.) of the pixel to a representative pixel (e.g., a pixel adjacent to the first glare pixel, a center pixel, etc.) in the high-reflective object region, or can refer to a statistic (e.g., mean) of distances of the first glare pixel to multiple pixels (e.g., all or part of the pixels) in the high-reflective object region. Then, the intensity of glare corresponding to the first glare pixel can be estimated based on the calculated distance and a glare intensity model.

[0122] In some embodiments, the glare intensity model can include a glare intensity model determined based on respective echo intensities of one or more glare pixels in a potential glare region and respective distances of the one or more glare pixels to the high-reflective object region. In this example, the echo intensity and distance data used to determine the glare intensity model are data of multiple pixels in the potential glare region that are classified as glare type (i.e., glare pixels). As an alternative or supplement to experimental data or historical data, the echo intensity and distance data used to determine the glare intensity model can be obtained in real time by the light detection and ranging device. It should be understood that determining the glare intensity model by the echo intensity and distance data obtained in real time, and estimating the glare intensity of a glare pixel obtained almost simultaneously using the model, can improve the accuracy of the glare intensity estimation on one hand, and improve the generality of the estimation method on the other hand.

[0123] For example, the above-mentioned glare intensity model can be determined using a regression algorithm. In other words, the glare intensity model can be regressed according to the intensity of the glare echo of a glare pixel and its distance to the high-reflective object region. For fused glare pixels, it can be difficult to distinguish the glare echo and the real object echo. Therefore, in some embodiments, the regression method can be used only for double-echo glare pixels and single-echo glare pixels and their echo data. For example, the regression method can use an adaptive local regression method. Specifically, the data is first segmented using a sliding window, and the data smaller than the median of the last sliding window (or the median of the last sliding window plus a certain threshold) is used to calculate the median of the current sliding window, and then the medians of all sliding windows are calculated in turn as the local regression result. According to the principle of the generation of glare, the intensity of glare decreases as the distance to the high-reflective object region increases. Therefore, such an adaptive local regression method can effectively filter out false noise data (e.g., data that does not decrease in intensity as the distance increases). After the model is regressed, the respective distances of the glare pixels to the high-reflective object region can be substituted into the regressed glare intensity model, so that the intensity of glare of the glare pixels can be estimated.

[0124] The regression algorithm is explained in more detail below as an example. Let the data points of the input glare pixels be represented as (X, Y), where X is the distance of the glare pixel to the high-reflective object region (e.g., the center), and Y is the intensity of the glare echo of the glare pixel. Let B be the width of the sliding window, N be the maximum number of sliding windows, and k be the adjustable parameter. In some embodiments, the pseudo code of the regression algorithm can be represented as follows:

[0125] The process of the regression algorithm is explained in more detail below with reference to a numerical example. It should be noted that the example is for ease of description only and is not intended to limit the scope of the present disclosure. Assume that X = [1, 2, 2.1, 3, 4, 4.5, 5, 6.1, 6.5, 6.9, 7.5], Y = [100, 95, 90, 88, 80, 75, 70, 90, 72, 70, 68], B = 2, N = 4, and k = 1, then initially y med = [100, 0, 0, 0].

[0126] First, between [2, 4), the X in the range is [2, 2.1, 3], and the corresponding Y is [95, 90, 88], all of which are less than y med [0] = 100, so the median 90 is taken as y med [1]. Next, between [4, 6), the X in the range is [4, 4.5, 6], and the corresponding Y is [80, 75, 70], all of which are less than y med [1] = 90, so the median 75 is taken as y med [2]. Then, between [6, 8), the X in the range is [6.1, 6.5, 6.9, 7.5], and the corresponding Y is [90, 72, 70, 68], of which 90 > y med [2], so this data is excluded as an outlier since it deviates from the rule that the glare intensity is negatively correlated with the distance. Therefore, the median 70 of [72, 70, 68] is taken as y med [3]. After regression, assume that the distance x of a certain input glare pixel to the high-reflective object region is 5.5, since x is within the second sliding window, i.e., between [6, 8), the estimated glare intensity y using the regression model is y med [2] = 75.

[0127] Further, in embodiments using regression algorithm, a confidence interval can also be calculated based on the error of the regressed glare intensity model and actual data points, and the calculated confidence interval can be further combined when calculating the glare intensity. In some embodiments, when calculating the glare intensity, the confidence interval can be added to the intensity value obtained by using the regressed model as the final glare intensity. As an example but not limitation, 3 times of the error standard deviation can be used as the confidence interval. Further, for double-bounce glare pixels, because it can be determined to some extent that there is a real object echo behind the glare echo, the glare intensity can be estimated slightly higher to some extent. Therefore, in some embodiments, a larger confidence interval can be used for double-bounce glare pixels than for single-bounce glare pixels.

[0128] FIG. 8 shows a schematic diagram of the regression result of the glare intensity model using the adaptive local regression method according to embodiments of the present disclosure. As shown in FIG. 8, the lower dashed line shows the local median, and the upper solid line shows the final estimated glare intensity of the model with the confidence interval superimposed. As shown, the points in the lower part of FIG. 8 generally satisfy the rule that the glare intensity decreases as it moves away from the high-reflective object area. However, as shown in the upper part of FIG. 8, there are some data points that do not satisfy this rule, which can be due to the fact that in these data points the glare echo can be completely superimposed with the real object echo. Using these points for regression can erroneously overestimate the intensity of the glare. Embodiments of the present disclosure exclude these data points as noise by using the adaptive local regression method described above, so that only the points near the local median are used for regression, so that the intensity of the glare can be more accurately estimated.

[0129] Additionally or alternatively, in some embodiments, the glare intensity model can include a theoretical glare intensity model determined based on the respective echo intensity of one or more high-reflective object pixels in the high-reflective object area and the respective distance of the one or more high-reflective object pixels to the high-reflective object area. The distance of the high-reflective object pixel to the high-reflective object area can be similarly understood. For example, the distance can refer to the distance (e.g., Euclidean distance, Manhattan distance, etc.) of the high-reflective object pixel to a representative pixel (e.g., center pixel, etc.) in the high-reflective object area, or can refer to the statistics (e.g., mean) of the distance of the high-reflective object pixel to multiple pixels (e.g., specific partial pixels) in the high-reflective object area. In this example, the decay law within the high-reflective object area can be used to estimate the decay law within the glare area. In this example, the echo intensity and distance data in the high-reflective object area used to determine the glare intensity model can be obtained in real time by the light detection and ranging device, or can include laboratory data, historical data, or simulation data.

[0130] In some embodiments, multiple glint intensity models can be used simultaneously to calculate the glint intensity. For example, the regression-based glint intensity model and the theoretical glint intensity model described above can be used simultaneously. In this example, the intensity regressed with the confidence interval (or just the regressed intensity) can be compared with the intensity estimated by the theoretical glint intensity model, and the smaller or larger one can be selected as the final calculated glint intensity. When removing the glint based on the glint intensity, to avoid deleting the original information in the echo due to over-removal, the smaller calculated result can be selected as the final calculated glint intensity.

[0131] It should be noted that for a light detection and ranging device with a lower dynamic range and / or a smaller range of pixels in an on state at the same time, the resulting echo data can not have a significant downward trend as shown in FIG. 8. In this case, since the estimated glint intensity can always be a larger value, it can not be meaningful to estimate the glint intensity. Accordingly, in some embodiments, the estimation of the glint intensity can be omitted.

[0132] Glint removal

[0133] After the glint pixels and their types are determined, the glint can be removed according to the type of the glint pixel. In other words, in the case where the first potential glint pixel described above has the pixel type of the glint pixel (e.g., including a double-echo glint pixel, a fusion glint pixel, or a single-echo glint pixel), the glint removal can be performed based on the pixel type of the first potential glint pixel.

[0134] In some embodiments, for a double-echo glint pixel or a single-echo glint pixel, without estimating the glint intensity, the intensity of the echo corresponding to the high-reflective object (i.e., the glint echo) of the glint pixel can be set to zero, i.e., assuming that the entire echo is due to the glint; or, with the estimation of the glint intensity, the estimated glint intensity can be subtracted from the intensity of the glint echo of the glint pixel. For a fusion glint pixel, the end position of the echo corresponding to the high-reflective object can still be considered as the position of the real object echo. Therefore, in some embodiments, the echo data of the pixel can be adjusted based on the end position of the echo corresponding to the high-reflective object of the pixel. For example, the depth value can be adjusted to the end position of the echo minus a preset threshold. For example, the threshold can be the mean value of the width of the signal echo of the actually measured non-high-reflective object pixel, or can be the width of the real object echo calculated theoretically through laboratory data, simulation data, historical data, etc.

[0135] In addition, if only non-glint pixels and glint pixels are distinguished in the pixel classification, the echo intensity of the glint pixel can be directly subtracted by the corresponding glint intensity value. Without estimating the glint intensity, the echo intensity of the glint pixel can be set to zero.

[0136] In some embodiments, the multiple signal echoes of the glint pixel can be sorted based on the intensity of the signal echoes after glint elimination, and the depth of the signal echo with the highest intensity can be considered as the depth of the glint pixel after glint elimination.

[0137] With the above method, the glint can be eliminated in different ways according to the type of glint, so that the glint can be more reliably eliminated, and the accuracy of depth / distance measurement can be improved.

[0138] Example application scenarios

[0139] FIGS. 9 and 10 respectively show example application scenarios of the glint detection method according to embodiments of the present disclosure. As shown in FIGS. 9 and 10, the technology according to embodiments of the present disclosure can be used for construction of a depth map, and thus can be applied in the field of autonomous driving, etc.

[0140] First, referring to FIG. 9, in this scenario, the two road signs shown by the dashed boxes in the scene diagram constitute high-reflective objects. This makes the depth of the area around the high-reflective objects in the depth map be wrongly estimated as the same as the high-reflective objects due to the influence of glint. The light detection and ranging device used in the scenario of FIG. 9 is a light detection and ranging device with a small dynamic range (e.g., 0-144), and according to its lighting logic (e.g., vertical lighting), the glint can only affect the area within 12 pixels in the vertical direction. Therefore, in this example scenario, it is desirable not to estimate the glint intensity.

[0141] According to the scheme of the present disclosure, by screening the signal echoes, a ghost echo is found in the echoes (e.g., at the position shown by the black dashed box in the second echo depth map), and the high-reflective object position is determined accordingly. In this example, the potential glint area is within 12 pixels in the vertical direction of the high-reflective object. Then, the pixels in this range are classified as glint. For the fused glint pixel, the end position of the echo is subtracted by the average signal echo width of the non-high-reflective object. For the double-echo glint pixel or the single-echo glint pixel, the glint echo intensity is directly set to 0. Finally, the signal echoes are sorted according to the echo intensity, and the depth of the echo with the strongest intensity is taken as the updated depth.

[0142] As shown by the dashed box in the lowermost glint-eliminated depth map in FIG. 9, by using the method according to embodiments of the present disclosure, even if the estimation of the glint intensity is not performed, the influence of the glint can be effectively eliminated, and thus the accuracy of the depth map can be improved.

[0143] Referring next to FIG. 10, in this scenario, it can be seen that there are two adjacent and close depth road signs in the left side of the scene, and the right road sign constitutes a high-reflective object. This makes the depth of the area around the right road sign in the depth map be wrongly estimated as the same as the right road sign due to the effect of glare, so that the two road signs appear to be connected together. The light detection and ranging device used in the scenario of FIG. 10 is a light detection and ranging device with a large dynamic range (e.g., 0-3006), and according to its lighting logic (e.g., horizontal lighting), the glare can affect the area within 96 pixels in the horizontal direction. Therefore, in this example scenario, the estimation of the intensity of the glare can be made.

[0144] According to the scheme of the present disclosure, by screening the signal echoes, the ghost echoes are found in the signal echoes, and the position of the high-reflective object is determined accordingly. In this example, the potential glare area is within 96 pixels in the horizontal direction of the high-reflective object. For example, as shown, the area corresponding to the right road sign is detected as the high-reflective object area, and the area around it (including the area corresponding to the left road sign) is determined as the potential glare area. Then, the pixels in this range are classified as glare. For the fused glare pixels, the end position of the echo is subtracted by the average width of the signal echoes of non-high-reflective objects. For the double-echo glare pixels or single-echo glare pixels, the intensity of the glare is estimated using local median regression, and the glare echo intensity is subtracted by the intensity of the glare calculated according to its distance to the high-reflective object. Finally, the signal echoes are sorted according to the echo intensity, and the depth of the echo with the strongest intensity is taken as the updated depth.

[0145] As shown in the bottommost glare-eliminated depth map in FIG. 10, by using the method according to the embodiments of the present disclosure, the effect of the glare can be effectively eliminated, thereby improving the accuracy of the depth map.

[0146] FIG. 11 shows an example method for glare detection of a light detection and ranging device according to embodiments of the present disclosure. It should be understood that the example method can be performed by various light detection and ranging devices.

[0147] As shown in FIG. 11, the method 1100 can include obtaining, by a light detection and ranging device, respective multi-bounce data for a plurality of pixels (1110). The method 1100 can include determining a high-reflective object region based on the respective multi-bounce data for the plurality of pixels (1120). The high-reflective object region can include one or more high-reflective object pixels. The method 1100 can include determining a potential glare region based on the high-reflective object region and a lighting logic of the light detection and ranging device (1130). The potential glare region can include one or more potential glare pixels. The method 1100 can further include classifying the one or more potential glare pixels to determine glare pixels and their types based on depth and width characteristics of one or more bounces of the pixels in the high-reflective object region and the potential glare region (1140).

[0148] In one embodiment, the multi-bounce data includes histogram data, multi-bounce data of a bounce pattern, or multi-bounce data of a ranging pattern.

[0149] In one embodiment, determining the high-reflective object region can include, for a first pixel of the plurality of pixels, obtaining one or more signal bounces of the first pixel based on intensities of the plurality of bounces of the first pixel; and determining the first pixel as a high-reflective object pixel based on a presence of a ghost bounce in the one or more signal bounces.

[0150] In one embodiment, the one or more signal bounces are obtained based on at least one of: a comparison of intensities of individual bounces to a first threshold value to determine a bounce that meets the first threshold value as a signal bounce; or a comparison of relative intensity relationships between the plurality of bounces to a second threshold value to determine a bounce that meets the second threshold value as a signal bounce, wherein the second threshold value includes a set threshold value or an adaptive threshold value.

[0151] In one embodiment, the first pixel is determined as a high-reflective object pixel based on a second bounce of the one or more signal bounces constituting a ghost bounce of a first bounce and an intensity and / or a width of the first bounce being greater than a respective threshold value.

[0152] In one embodiment, the method further includes adjusting the current high-reflective object region by adding or removing high-reflective object pixels.

[0153] In one embodiment, adjusting the current high-reflective object region includes: determining pixels adjacent to the current high-reflective object region and whose characteristics match characteristics of the high-reflective object pixels in the current high-reflective object region as potential high-reflective object pixels; and combining the current high-reflective object region and the potential high-reflective object pixels to obtain an adjusted high-reflective object region. For example, the characteristics can include depths, intensities, and / or widths of bounces of the pixels.

[0154] In one embodiment, determining the potential glint region comprises determining a region of pixels that are turned on at the same time as one or more high-reflective object pixels in the high-reflective object region as the potential glint region.

[0155] In one embodiment, classifying the one or more potential glint pixels comprises classifying the one or more potential glint pixels as one of a plurality of pixel types, including a high-reflective object pixel, a double-bounce glint pixel, a fusion glint pixel, a single-bounce glint pixel, or a non-glint pixel. In one example, the pixel types of glint pixels include double-bounce glint pixels, fusion glint pixels, and single-bounce glint pixels.

[0156] In one embodiment, classifying the one or more potential glint pixels comprises: calculating parameters of a plurality of pixel classification models corresponding to the plurality of pixel types based on depths and widths of echoes of the one or more high-reflective object pixels of the high-reflective object region. For a first potential glint pixel in the potential glint region: calculating scores or probabilities for the first potential glint pixel using the plurality of pixel classification models based on a depth, a width of an echo of the first potential glint pixel, and / or a distance of the first potential glint pixel to the high-reflective object region; determining a pixel type of the first potential glint pixel based on the calculated scores or probabilities.

[0157] In one embodiment, the method further comprises, in a case that the first potential glint pixel has a pixel type of a glint pixel, performing glint removal based on the pixel type of the first potential glint pixel; and ranking a plurality of signal echoes of the first potential glint pixel based on intensities of the signal echoes after the glint removal, wherein a depth of a signal echo with a highest intensity corresponds to a depth of the first potential glint pixel after the glint removal.

[0158] In one embodiment, performing glint removal based on the pixel type of the first potential glint pixel comprises: in response to the pixel type of the first potential glint pixel being a double-bounce glint pixel or a single-bounce glint pixel, setting an intensity of an echo of the first potential glint pixel corresponding to the high-reflective object to zero, or subtracting an estimated glint intensity from the intensity of the echo of the first potential glint pixel corresponding to the high-reflective object.

[0159] In one embodiment, performing glint removal based on the pixel type of the first potential glint pixel comprises: in response to the pixel type of the first potential glint pixel being a fusion glint pixel, adjusting echo data of the first potential glint pixel based on an ending position of an echo of the first potential glint pixel corresponding to the high-reflective object.

[0160] In one embodiment, the method further comprises, for a first glint pixel in the potential glint region, calculating a distance of the first glint pixel to the high-reflective object region; and estimating a glint intensity corresponding to the first potential glint pixel based on the calculated distance and a glint intensity model.

[0161] In one embodiment, the glint intensity model is determined based on at least one of: respective return intensities of one or more glint pixels in the potential glint region and respective distances of the one or more glint pixels to the high-reflective object region, or respective return intensities of one or more high-reflective object pixels in the high-reflective object region and respective distances of the one or more high-reflective object pixels to the high-reflective object region.

[0162] In one embodiment, the glint intensity model is determined using a regression algorithm, the method further comprising: calculating a confidence interval based on error values of the regressed glint intensity model and actual data points; and using a larger confidence interval for double-return glint pixels than for single-return glint pixels.

[0163] In one embodiment, the light detection and ranging device comprises a single-photon avalanche diode LiDAR (SPAD LiDAR).

[0164] Embodiments of the present disclosure also provide an electronic device. The electronic device includes one or more processors and one or more memories having stored thereon one or more instructions. The one or more instructions, when executed by the one or more processors, cause the one or more processors to perform the method for glint detection and elimination for a light detection and ranging device according to embodiments of the present disclosure.

[0165] Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon one or more instructions. The one or more instructions, when executed by a processor, cause the processor to perform the method for glint detection and elimination for a light detection and ranging device according to embodiments of the present disclosure.

[0166] Embodiments of the present disclosure also provide a computer program product including one or more instructions. The one or more instructions, when executed by a processor, cause the processor to perform the method for glint detection and elimination for a light detection and ranging device according to embodiments of the present disclosure.

[0167] It should be understood that the instructions in the computer-readable storage medium according to embodiments of the present disclosure can be configured to perform operations corresponding to the above-mentioned device and method embodiments. When referring to the above-mentioned device and method embodiments, the embodiments of the computer-readable storage medium are clear to those skilled in the art, and therefore are not repeatedly described. The computer-readable storage medium for carrying or including the above-mentioned instructions also falls within the scope of the present disclosure. Such computer-readable storage medium can include, but is not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

[0168] It should be noted that the various components or units in the present disclosure are merely logical modules divided according to the specific functions implemented by them, and are not intended to limit the specific implementation manner, for example, can be implemented in software, hardware or a combination of software and hardware. In actual implementation, the above-mentioned various components or units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). For example, the multiple functions included in one unit in the above embodiment can be implemented by separate devices. Alternatively, the multiple functions implemented by multiple units in the above embodiment can be respectively implemented by separate devices. In addition, one of the above functions can be implemented by multiple units.

[0169] In addition, it should be understood that the above series of processes and devices can also be implemented by software and / or firmware. In the case of implementation by software and / or firmware, the program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware structure, such as the electronic device 1300 shown in FIG. 12, which can perform various functions when various programs are installed. FIG. 12 shows an example block diagram of an electronic device for implementing various methods according to embodiments of the present disclosure.

[0170] In FIG. 12, a central processing unit (CPU) 1301 performs various processes according to a program stored in a read only memory (ROM) 1302 or a program loaded from a storage section 1308 to a random access memory (RAM) 1303. In the RAM 1303, data required when the CPU 1301 performs various processes, etc. is also stored as necessary.

[0171] The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output interface 1305 is also connected to the bus 1304.

[0172] The following components are connected to the input / output interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet.

[0173] A drive 1310 is also connected to the input / output interface 1305 as necessary. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 1310 as necessary, so that a computer program read therefrom is installed in the storage section 1308 as necessary.

[0174] When the above series of processes are implemented by software, the program constituting the software is installed from a network such as the Internet or a storage medium such as removable media 1311.

[0175] Those skilled in the art will understand that such storage media are not limited to the removable medium 1311 shown in FIG. 12, which stores programs and is distributed separately from the device to provide programs to users. Examples of removable media 1311 include magnetic disks (including floppy disks (registered trademark)), optical disks (including optical disc read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini-discs (MD) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be ROM 1302, a hard disk included in storage section 1308, etc., which stores programs and is distributed to users along with the device containing them.

[0176] It should be understood that the technical solutions of this disclosure can be implemented through the following example implementation methods.

[0177] 1. A method for glare detection in a light detection and ranging device, comprising:

[0178] The optical detection and ranging device obtains corresponding multi-echo data for multiple pixels.

[0179] Based on the corresponding multi-echo data of the plurality of pixels, a high-reflectivity object region is determined, wherein the high-reflectivity object region includes one or more high-reflectivity object pixels;

[0180] Based on the highly reflective object region and the lighting logic of the light detection and ranging device, a potential glare region is determined, wherein the potential glare region includes one or more potential glare pixels; and

[0181] Based on the depth and width characteristics of one or more echoes from pixels in the highly reflective object region and the potential glare region, the one or more potential glare pixels are classified to determine the glare pixels and their types.

[0182] 2. The method according to Example 1, wherein the multi-echo data includes histogram data, multi-echo data in echo mode, or multi-echo data in ranging mode.

[0183] 3. According to the method described in Example 1, determining the high-reflectivity object region includes:

[0184] For a first pixel among the plurality of pixels, one or more signal echoes of the first pixel are obtained based on the intensity of the plurality of echoes of the first pixel; and

[0185] The first pixel is identified as a highly reflective object pixel based on the presence of ghost echoes in the one or more signal echoes.

[0186] 4. The method of example 3, wherein the one or more signal echoes are obtained by at least one of:

[0187] determining an echo that meets a first threshold as a signal echo based on a comparison of an intensity of the single echo to the first threshold; or

[0188] determining an echo that meets a second threshold as a signal echo based on a comparison of a relative intensity relationship between a plurality of echoes to the second threshold, wherein the second threshold comprises a set threshold or an adaptive threshold.

[0189] 5. The method of example 3, wherein a first pixel is determined as a high-reflecting object pixel based on a second echo of the one or more signal echoes constituting a ghost echo of a first echo and an intensity and / or a width of the first echo being greater than a respective threshold.

[0190] 6. The method of example 3, further comprising adjusting a current high-reflecting object region by adding or subtracting high-reflecting object pixels.

[0191] 7. The method of example 6, wherein adjusting the current high-reflecting object region comprises:

[0192] determining pixels adjacent to the current high-reflecting object region and having characteristics matching characteristics of the high-reflecting object pixels in the current high-reflecting object region as potential high-reflecting object pixels; and

[0193] combining the current high-reflecting object region and the potential high-reflecting object pixels to obtain an adjusted high-reflecting object region,

[0194] wherein the characteristics comprise a depth, an intensity, and / or a width of an echo of a pixel.

[0195] 8. The method of example 1, wherein determining the potential glare region comprises determining a region of pixels that are simultaneously turned on with the one or more high-reflecting object pixels in the high-reflecting object region as the potential glare region.

[0196] 9. The method of example 1, wherein classifying the one or more potential glare pixels comprises classifying the one or more potential glare pixels as one of a plurality of pixel types: a high-reflecting object pixel, a double-echo glare pixel, a fusion glare pixel, a single-echo glare pixel, or a non-glare pixel,

[0197] wherein the pixel types of the glare pixels comprise the double-echo glare pixel, the fusion glare pixel, and the single-echo glare pixel.

[0198] 10. The method of example 9, wherein the one or more potential glare pixels are classified by:

[0199] based on the depth and width of the echo of the one or more high-reflective-object pixels of the high-reflective-object region, compute parameters of a plurality of pixel classification models corresponding to the plurality of pixel types;

[0200] for a first potential glint pixel in the potential glint region:

[0201] based on the depth, width, and / or distance of the first potential glint pixel to the high-reflective-object region, use the plurality of pixel classification models to compute a score or probability for the first potential glint pixel;

[0202] based on the computed score or probability, determine a pixel type of the first potential glint pixel.

[0203] 11. The method of example 10, further comprising:

[0204] in a case that the first potential glint pixel has the pixel type of the glint pixel, perform glint removal based on the pixel type of the first potential glint pixel; and

[0205] based on intensities of signal echoes of the first potential glint pixel after the glint removal, rank a plurality of signal echoes of the first potential glint pixel, wherein a depth of a signal echo with a highest intensity corresponds to a depth of the first potential glint pixel after the glint removal.

[0206] 12. The method of example 11, wherein performing the glint removal based on the pixel type of the first potential glint pixel comprises:

[0207] in response to the pixel type of the first potential glint pixel being the double-echo glint pixel or the single-echo glint pixel, set an intensity of an echo of the first potential glint pixel corresponding to the high-reflective-object to zero or subtract an estimated glint intensity from the intensity of the echo of the first potential glint pixel corresponding to the high-reflective-object.

[0208] 13. The method of example 11, wherein performing the glint removal based on the pixel type of the first potential glint pixel comprises:

[0209] in response to the pixel type of the first potential glint pixel being the fusion glint pixel, adjust echo data of the first potential glint pixel based on an ending position of an echo of the first potential glint pixel corresponding to the high-reflective-object.

[0210] 14. The method of example 1, further comprising:

[0211] for a first glint pixel in the potential glint region, compute a distance of the first glint pixel to the high-reflective-object region; and

[0212] Based on the computed distances and the glare intensity model, estimate a glare intensity corresponding to the first glare pixel.

[0213] 15. The method of example 14, wherein the glare intensity model is determined based on at least one of:

[0214] a respective return intensity of one or more glare pixels in the potential glare region and a respective distance of the one or more glare pixels to the high-reflective object region, or

[0215] a respective return intensity of the one or more high-reflective object pixels in the high-reflective object region and a respective distance of the one or more high-reflective object pixels to the high-reflective object region.

[0216] 16. The method of example 15, wherein the glare intensity model is determined using a regression algorithm, the method further comprising:

[0217] computing a confidence interval based on an error value of the regressed glare intensity model and actual data points; and

[0218] using a larger confidence interval for double-return glare pixels than for single-return glare pixels.

[0219] 17. The method of example 1, wherein the light detection and ranging device comprises a single-photon avalanche diode LiDAR (SPAD LiDAR).

[0220] 18. An electronic device, comprising:

[0221] at least one processor; and

[0222] at least one memory having instructions stored thereon, wherein the at least one memory and the instructions are configured to, with the at least one processor, cause the electronic device to perform the method of any of examples 1-17.

[0223] 19. A computer program product comprising one or more instructions that, when executed by a processor, cause performance of the method of any of examples 1-17.

[0224] The exemplary embodiments of the present disclosure are described above with reference to the accompanying drawings, but the present disclosure is of course not limited to the above examples. Various changes and modifications can be made by those skilled in the art within the scope of the appended claims, and it should be understood that such changes and modifications naturally fall within the technical scope of the present disclosure.

[0225] For example, a plurality of functions included in one unit in the above embodiments can be implemented by separate apparatuses. Alternatively, a plurality of functions implemented by a plurality of units in the above embodiments can be implemented by one unit. In addition, one of the above functions can be implemented by a plurality of units. Needless to say, such a configuration is included in the technical scope of the present disclosure.

[0226] In this specification, steps described in a flowchart describe not only processing performed in time series according to the order described in the flowchart, but also processing performed in parallel or individually rather than in time series. Further, even in a step that is processed in time series, needless to say, the order can be changed as appropriate.

[0227] While the present disclosure and its advantages have been disclosed in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present disclosure is not intended to be limited to the particular embodiments of the disclosure described herein, but is intended to include all embodiments falling within the scope of the appended claims. Additionally, it is contemplated that various embodiments of the present disclosure include any combination of the components and features described herein. In addition, the term "comprising" or "comprises" as used herein is used in the sense of "including" and / or "containing" but not limited to "consisting only of."

Claims

1. A method for glint detection of a light detection and ranging device, comprising: obtaining, by the light detection and ranging device, respective multi-return data of a plurality of pixels; determining, based on the respective multi-return data of the plurality of pixels, a high-reflective object region, wherein the high-reflective object region comprises one or more high-reflective object pixels; determining, based on the high-reflective object region and a glinting logic of the light detection and ranging device, a potential glint region, wherein the potential glint region comprises one or more potential glint pixels; and classifying, based on depth and width characteristics of one or more returns of pixels in the high-reflective object region and the potential glint region, the one or more potential glint pixels to determine glint pixels and their types. The multi-return data comprises histogram data, multi-return data of a range mode, or multi-return data of a ranging mode.

2. The method of claim 1, wherein, Determining the high-reflective object region comprises:

3. The method of claim 1, wherein, for a first pixel of the plurality of pixels, obtaining one or more signal returns of the first pixel based on intensities of a plurality of returns of the first pixel; and determining the first pixel as a high-reflective object pixel based on a presence of a ghost return in the one or more signal returns. The one or more signal returns are obtained based on at least one of:

4. The method of claim 3, wherein, determining, based on a comparison of intensities of individual returns to a first threshold, a return that meets the first threshold as a signal return; or determining, based on a comparison of relative intensity relationships between a plurality of returns to a second threshold, a return that meets the second threshold as a signal return, wherein the second threshold comprises a set threshold or an adaptive threshold. determining the first pixel as a high-reflective object pixel based on a second return of the one or more signal returns constitutes a ghost return of a first return and an intensity and / or a width of the first return is greater than a respective threshold.

5. The method of claim 3, wherein, Adjusting a current high-reflective object region by adding or removing high-reflective object pixels.

6. The method of claim 3, further comprising: Adjusting the current high-reflective object region comprises:

7. The method of claim 6, wherein, determining, as potential high-reflective object pixels, pixels that are adjacent to the current high-reflective object region and whose characteristics match characteristics of high-reflective object pixels in the current high-reflective object region; and combining the current high-reflective object region and the potential high-reflective object pixels to obtain an adjusted high-reflective object region, wherein the characteristics comprise depth, intensity, and / or width of returns of the pixels. Determining the potential glint region comprises determining, as the potential glint region, a region of pixels that are turned on simultaneously with the one or more high-reflective object pixels in the high-reflective object region.

8. The method of claim 1, wherein, Classifying the one or more potential glint pixels comprises classifying the one or more potential glint pixels as one of a plurality of pixel types: a high-reflective object pixel, a double-return glint pixel, a fusion glint pixel, a single-return glint pixel, or a non-glint pixel, 9. The method of claim 1, wherein, wherein the pixel types of glint pixels comprise the double-return glint pixel, the fusion glint pixel, and the single-return glint pixel. Classifying the one or more potential glint pixels comprises:

10. The method of claim 9, wherein, calculating, based on depth and width of returns of the one or more high-reflective object pixels of the high-reflective object region, parameters of a plurality of pixel classification models corresponding to the plurality of pixel types; and for a first potential glint pixel in the potential glint region: ​ based on a depth, a width, and / or a distance of the first potential glint pixel to the high-reflectivity object region, calculate a score or a probability for the first potential glint pixel using the plurality of pixel classification models; based on the calculated score or probability, determine a pixel type of the first potential glint pixel.

11. The method of claim 10, further comprising: in a case that the first potential glint pixel has the pixel type of the glint pixel, perform glint removal based on the pixel type of the first potential glint pixel; and and sort a plurality of signal echoes of the first potential glint pixel based on intensities of the signal echoes after the glint removal, wherein a depth of a signal echo with a highest intensity corresponds to a depth of the first potential glint pixel after the glint removal.

12. The method of claim 11, wherein, performing the glint removal based on the pixel type of the first potential glint pixel comprises: in response to the pixel type of the first potential glint pixel being the double-echo glint pixel or the single-echo glint pixel, setting an intensity of an echo of the first potential glint pixel corresponding to the high-reflectivity object region to zero or subtracting an estimated glint intensity from the intensity of the echo of the first potential glint pixel corresponding to the high-reflectivity object region.

13. The method of claim 11, wherein, performing the glint removal based on the pixel type of the first potential glint pixel comprises: in response to the pixel type of the first potential glint pixel being the fusion glint pixel, adjusting echo data of the first potential glint pixel based on an ending position of an echo of the first potential glint pixel corresponding to the high-reflectivity object region.

14. The method of claim 1, further comprising: for a first glint pixel in the potential glint region, calculating a distance of the first glint pixel to the high-reflectivity object region; and and based on the calculated distance and a glint intensity model, estimating a glint intensity corresponding to the first glint pixel.

15. The method of claim 14, wherein, the glint intensity model is determined based on at least one of: echo intensities of one or more glint pixels in the potential glint region and respective distances of the one or more glint pixels to the high-reflectivity object region, or echo intensities of the one or more high-reflectivity object pixels in the high-reflectivity object region and respective distances of the one or more high-reflectivity object pixels to the high-reflectivity object region.

16. The method of claim 15, wherein, the glint intensity model is determined using a regression algorithm, the method further comprising: calculating a confidence interval based on error values of the regressed glint intensity model and actual data points; and using a larger confidence interval for a double-echo glint pixel than for a single-echo glint pixel.

17. The method of claim 1, wherein the light detection and ranging device comprises a single-photon avalanche diode lidar (SPAD LiDAR).

18. An electronic device, comprising: at least one processor; and at least one memory having instructions stored thereon, wherein the at least one memory and the instructions are configured to, with the at least one processor, cause the electronic device to perform the method of any of claims 1-17.

19. A computer program product comprising one or more instructions that, when executed by a processor, cause performance of the method of any of claims 1-17.

Images