Scanning system and scanning method

Abstract

The present disclosure relates to a scanning system and a scanning method. For example, a scanning system includes a scanning structure configured to rotate around at least one first scanning axis, a driver configured to drive the scanning structure around the at least one first scanning axis and to detect a position of the scanning structure relative to the at least one first scanning axis during movement of the scanning structure, a segmented pixel sensor including a plurality of sub-pixel elements arranged in a pixel area, and a controller configured to selectively activate and deactivate the plurality of sub-pixel elements into at least one active cluster and at least one inactive cluster, to form at least one active pixel from the at least one active cluster, to receive first position information indicative of the detected position of the scanning structure from the driver, and to dynamically change the clustering of the activated sub-pixel elements and the clustering of the deactivated sub-pixel elements based on the first position information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of photodetection, and more specifically, to a scanning system and a scanning method. BACKGROUND

[0002] Light Detection and Ranging (LIDAR) is a remote sensing method that uses light in the form of pulsed laser to measure the distance (variable distance) to one or more objects in a field of view. Specifically, light is emitted toward the object. A single photodetector or a photodetector array receives the reflection from the object illuminated by the light and determines the time required for the reflection to reach individual sensors in the photodetector array. This is also known as measuring the time of flight (ToF). The LIDAR system forms a depth measurement and makes a distance measurement by calculating the mapping of the distance to the object based on the time of flight. Thus, the time of flight calculation can create a distance and depth map, which can be used to generate an image.

[0003] Generally, the pixel structure in a conventional photodetector array is typically static. Moreover, oversampling and averaging are highly desirable features in order to improve the signal-to-noise ratio (SNR) and improve the receiver’s robustness to typical noise (e.g., shot noise, white noise) such as interference from other LIDAR systems / transmitters. In addition to interfering with other LIDAR systems / transmitters, laser pulse power limitations related to eye safety can cause a demand for averaging.

[0004] While a one-dimensional (ID) scanning method supports a high frame rate, enabling oversampling and averaging, a 2x ID Lissajous scanning method supports only a low frame rate of, for example, 10-25 Hz. Thus, oversampling and averaging are considered infeasible solutions because they effectively further reduce the frame rate. Moreover, if the laser shots need to be averaged over multiple frames, a high data rate occurs and the storage requirement increases. This is because of the need to store all the data of the photodetector array over multiple frames to perform averaging. Moreover, averaging over two or more frames is affected by motion blur of the scene.

[0005] If averaging is performed in a manner that uses several fast consecutive laser pulses, the storage requirement is greatly reduced because there is no need to store the data over multiple frames. Moreover, the motion blur of the scene is reduced. A quasi-static scanner (laser shot timing is not affected by any scanner oscillation) supports this averaging method (move to a position, shoot a few times, move to the next position, shoot a few times, and so on). In contrast, an oscillating scanner (oscillating motion of the scanner is continuous), introduces motion blur of the moving scanner (e.g., due to ~3us time of flight of a single pulse, limited pulse repetition frequency of the laser transmitter (e.g., 100 kHz)). Thus, it is almost impossible to average over consecutive laser shots while using a high frequency oscillating scanner.

[0006] Furthermore, the scene can be illuminated by LIDAR emitters using a sequence of pulses of laser light or by a frequency modulated continuous wave (FMCW) beam. Since the pixel structure in the photodetector array is static, an oscillating / moving scanner using these scene illumination methods will cause a moving projection on the photodetector array (e.g., 3us time of flight of a single pulse, limited pulse repetition frequency of the laser (e.g., 100 kHz)). This moving projection on the static pixels will result in a compromised perception of the environment. Thus, in the case of a moving / oscillating scanner, it is nearly impossible to oversample and average the respective static pixels with fast consecutive laser pulses because the scanner moves too much between consecutive laser shots and between laser emission and reception.

[0007] Accordingly, a dynamically reconfigurable non-static photodetector array that supports dynamic re-clustering of sub-sub-pixels and sub-pixels to form pixels for a LIDAR receiving system can be desirable. SUMMARY

[0008] One or more embodiments provide a scanning system, comprising: a first scanning structure configured to rotate around at least one first scanning axis; a driver configured to drive the first scanning structure around the at least one first scanning axis and detect a position of the first scanning structure relative to the at least one first scanning axis during movement of the first scanning structure; a segmented pixel sensor comprising a plurality of sub-pixel elements arranged in a pixel area; and a controller configured to selectively activate and deactivate the plurality of sub-pixel elements into at least one active cluster and at least one inactive cluster to form at least one active pixel from the at least one active cluster, receive first position information indicative of the detected position of the first scanning structure from the driver, and dynamically change clustering of the activated sub-pixel elements and clustering of the deactivated sub-pixel elements based on the first position information.

[0009] One or more embodiments provide a scanning method, comprising: driving a first scanning structure around at least one first scanning axis; detecting a position of the first scanning structure relative to the at least one first scanning axis during movement of the first scanning structure; selectively activating and deactivating a plurality of sub-pixel elements of a segmented pixel sensor into at least one active cluster and at least one inactive cluster to form at least one active pixel from the at least one active cluster; and dynamically changing clustering of the activated sub-pixel elements and clustering of the deactivated sub-pixel elements based on first position information indicative of the detected position of the first scanning structure. BRIEF DESCRIPTION OF DRAWINGS

[0010] Embodiments are described herein with reference to the accompanying drawings.

[0011] Figure 1 is a schematic diagram of a LIDAR scanning system according to one or more embodiments;

[0012] Figure 2 is a schematic block diagram of a LIDAR scanning system according to one or more embodiments;

[0013] Figure 3A is a schematic diagram of a SiPM pixel (i.e., a 2D SiPM pixel) according to one or more embodiments;

[0014] Figure 3B is a schematic diagram of a 2D SiPM pixel array according to one or more embodiments;

[0015] Figures 4A-4H various ways of clustering sub-pixels into pixels on a segmented pixel sensor are shown according to one or more embodiments;

[0016] Figures 5A-5D dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments;

[0017] Figure 6 is shown Figures 5A-5D pixel averaging of pixel A represented in

[0018] Figures 7A-7F dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments;

[0019] Figure 8 is shown Figures 7A-7F pixel averaging of pixel A represented in

[0020] Figures 9A-9D dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments;

[0021] Figure 10 is shown Figures 9A-9D pixel averaging of pixel A represented in

[0022] Figure 11A and Figure 11B is a schematic block diagram of a receiver circuit according to one or more embodiments;

[0023] Figure 12A is a schematic diagram of a Lissajous LIDAR scanning system according to one or more embodiments;

[0024] Figure 12B is a schematic block diagram of a Lissajous LIDAR scanning system including similar to Figure 2the additional circuitry shown;

[0025] Figure 13A is Figure 12A a schematic diagram of a LIDAR scanning system of the Lissajous type, but additionally showing two successive laser shots into the field of view; and

[0026] Figure 13B and Figure 13C different possible pixel positions on a segmented pixel sensor at which a laser shot can be received based on the time of flight of each laser shot are shown. DETAILED DESCRIPTION

[0027] In the following, various embodiments will be described in detail with reference to the attached drawings. It should be noted that these embodiments are only for illustrative purposes and should not be construed as limiting. For example, although an embodiment can be described as including a plurality of features or elements, this should not be construed as indicating that all of these features or elements are required in order to implement the embodiment. Rather, in other embodiments, some features or elements can be omitted, or replaced with alternative features or elements. Furthermore, other features or elements can be provided in addition to those explicitly shown and described, for example, conventional components of a sensor device.

[0028] Features from the different embodiments can be combined to form further embodiments unless specifically stated otherwise. Changes or modifications to one embodiment that would be apparent to a person of ordinary skill in the art are also considered to be within the scope of the other embodiments. In some cases, well-known structures and devices are not shown or described in detail in order to avoid obscuring the embodiments.

[0029] Furthermore, in the following description, equivalent or similar elements are denoted by equivalent or similar reference numbers. Since the same reference numbers are given to the same or functionally equivalent elements in the drawings, repeated description of the elements with the same reference numbers can be omitted. Thus, the description provided for elements with the same or similar reference numbers is mutually exchangeable.

[0030] Unless otherwise stated, connections or couplings between elements shown in the drawings or described herein can be wire-based connections or wireless connections. Furthermore, such connections or couplings can be direct connections or couplings without additional intermediate elements, or indirect connections or couplings with one or more additional intermediate elements, as long as the general purpose of the connection or coupling (e.g., transmitting a particular kind of signal or transmitting a particular kind of information) is substantially maintained.

[0031] In the disclosure, expressions including ordinal numbers, such as "first," "second," etc., can modify various elements. However, the elements are not limited by the above expressions. For example, the above expressions do not limit the order and / or importance of the elements. The above expressions are used merely to distinguish an element from another. For example, a first block and a second block represent different blocks although both are blocks. For another example, a first element can be termed as a second element, and similarly, a second element can be also termed as a first element without departing from the scope of the disclosure.

[0032] Embodiments relate to optical sensors and optical sensor systems and acquiring information about optical sensors and optical sensor systems. A sensor can refer to a component that converts a physical quantity to be measured into an electrical signal (e.g., a current signal or a voltage signal). For example, the physical quantity can include electromagnetic radiation, such as visible light, infrared (IR) radiation, or other types of illumination signals, electric current, or voltage, but is not limited thereto. For example, an image sensor can be a silicon chip inside a camera that converts photons of light from a lens into a voltage. The larger the active area of the sensor, the more light can be collected for creating an image.

[0033] A sensor device as used herein can refer to a device that includes a sensor and further components (e.g., biasing circuitry, an analog-to-digital converter, or a filter). The sensor device can be integrated on a single chip, although in other embodiments, multiple chips or off-chip components can be used to implement the sensor device.

[0034] In a light detection and ranging (LIDAR) system, a light source emits a pulse sequence of light pulses or a frequency modulated continuous wave (FMCW) beam into a field of view, and light is reflected from one or more objects by backscattering. A pixel array, such as an array of photodetectors, detects and measures the reflected light beams received from the objects illuminated by the emitted light. Specifically, a LIDAR can use either a direct ToF sensor, which emits a short light pulse that lasts only a few nanoseconds and then measures the time required for some of the emitted light to return, or an indirect ToF sensor, which emits continuous modulated light and measures the phase of the reflected light to calculate the distance to an object. Either type can be used in embodiments described herein.

[0035] In embodiments described herein, the array of photodetectors is a segmented pixel sensor, where pixels are flexibly and dynamically formed by combining sub-sub-elements to form sub-elements and combining the formed sub-elements to form pixels. Thus, pixels are clustered from a collection of sub-elements (i.e., sub-pixels). The combination of sub-sub-elements and the combination of sub-elements are programmable and can be flexibly and dynamically changed during a scan operation while a pulse sequence or a FMCW beam is being emitted into a field of view. Depending on how the sub-sub-elements and sub-elements are combined to form pixels, either a 1D pixel array or a 2D pixel array can be created.

[0036] The smallest unit of the photodetector array is a sub-sub-element, which can be a single diode such as an avalanche photodiode (APD) or a single-photon avalanche diode (SPAD). Hereinafter, the sub-sub-element will be referred to as a sub-sub-pixel.

[0037] Sub-pixels (hereinafter referred to as sub-pixels) cluster from a dynamically selected set of sub-sub-pixels, and each pixel clusters from a dynamically selected set of sub-pixels.

[0038] For example, a sub-sub-pixel can be a single SPAD (or APD). An 8x8 cluster of SPADs or APDs can form a sub-pixel. In the case of using SPADs, the sub-pixel can be referred to as a silicon photomultiplier (SiPM). A 4x4 cluster of sub-pixels (e.g., 32x32 SPADs or APDs) can form a pixel. One or more pixels can be formed at any given time.

[0039] The sub-sub-pixels and sub-pixels used to form a pixel can be dynamically selected by a system controller implementing a clustering algorithm. The number of sub-sub-pixels and sub-pixels used to form a pixel is fully configurable and programmable. Furthermore, the selection of sub-sub-pixels and sub-pixels used to form a pixel can be formed by a single contiguous grouping, two or more contiguous groupings themselves spaced apart (i.e., non-contiguous) from each other by at least one or more sub-sub-pixels. Furthermore, each grouping of sub-sub-pixels and sub-pixels can be grouped in any shape and need not be in the form of a square.

[0040] Each sub-sub-pixel can be read out as a sub-sub-measurement signal in the form of raw analog or digital data. Each sub-sub-measurement signal from a pixel (i.e., from the sub-sub-pixels of a pixel) can be summed and / or averaged together by a readout processing chain to generate a pixel measurement signal.

[0041] The time difference of return of each light pulse across multiple pixels of the pixel array can then be used to make a digital 3D representation of the environment or to generate other sensor data. For example, in a direct ToF system, a light source can emit a single light pulse, and a receiver circuit electrically coupled to the pixel array can count from the time of the emitted light pulse (corresponding to a start signal) until the time of receipt of the reflected light pulse at the receiver (i.e., at the pixel array) (corresponding to a stop signal). The “time of flight” of the light pulse is then converted to a distance.

[0042] For example, a light source can emit a single light pulse, and a time-to-digital converter (TDC) electrically coupled to the pixel array can count from the time of the emitted light pulse (corresponding to a start time) and record until the time of receipt of the reflected light pulse at the receiver (i.e., at the pixel array) (i.e., a ToF hit time) (corresponding to a ToF hit signal).

[0043] Alternatively, an analog-to-digital converter (ADC) can be electrically coupled to the pixel array (e.g., indirectly coupled with intermediate elements between them) for pulse detection and ToF measurements. For example, the ADC can be used to estimate the time interval between the start and ToF hit signals using appropriate algorithms.

[0044] Scans such as oscillating horizontal scans (e.g., from left to right and from right to left of the field of view) can illuminate the scene in a continuous scanning fashion. Each time the light source emits a laser beam, it creates a scan line in the “field of view”. By emitting successive light pulses or FMCW light beams in different scanning directions, an area called the field of view can be scanned and objects within that area can be detected and imaged. Thus, the field of view represents a scanning plane with a projection center. Raster scanning or Lissajous scanning for orthogonal scanning axes can also be used.

[0045] Figure 1 is a schematic illustration of a LIDAR scanning system 100 according to one or more embodiments. The LIDAR scanning system 100 is an optical scanning device comprising a transmitter comprising an illumination unit 10, a transmitter optics 11 and a one-dimensional (1D) MEMS mirror 12 (1D MEMS scanner), and a receiver comprising a main optics 14 and a light receiver 15. The light receiver 15 in the figure is a segmented pixel sensor 15. The receiver can further comprise receiver circuitry such as data acquisition / readout circuitry and data processing circuitry, which will be further described below. Figure 2

[0046] While this arrangement represents one example of a LIDAR system type, it should be understood that other types of LIDAR systems can also be used, such as those used in flash LIDAR. In addition, the LIDAR scanning system 100 can be rotatable to scan in different scanning directions. For example, the LIDAR scanning system 100 can be rotated by 90° to scan in a vertical direction instead of a horizontal direction. Thus, the embodiments described herein are not limited to a particular type of light emitter or ToF system and can also be applied to other types of ToF systems.

[0047] Returning to Figure 1 The segmented pixel sensor 15 is arranged in a way to map the intended field of view vertically onto the vertical extension of the segmented pixel sensor 15. Depending on the vertical angle of the received light beam, the received light beam will only hit certain rows or groups of rows of the detector array. The intended field of view can be further mapped horizontally on the horizontal extension of the segmented pixel sensor 15.

[0048] ​At its smallest unit, a segmented pixel sensor 15 can be formed by an array of analog or digital sub-sub-pixel elements. In the case of forming a SiPM (i.e., sub-pixel), each SiPM comprises a plurality of microcells, each microcell comprising a SPAD. Thus, each SiPM comprises a collection of two or more SPADs.

[0049] Like an avalanche photodiode (APD), a SPAD triggers an avalanche current with incident radiation on a p-n junction under reverse bias. The fundamental difference between a SPAD and an APD is that a SPAD is specifically designed to operate at a reverse bias voltage much higher than its breakdown voltage. This operation is also known as Geiger mode (as opposed to the linear mode of an APD).

[0050] In this example, the illumination unit 10 comprises a plurality of light sources (e.g., laser diodes or light emitting diodes) that are linearly aligned in a single row and configured to emit light for scanning a field of view of the object. The light emitted by the light sources is typically infrared light, although other wavelengths of light can also be used. It can be seen from the embodiment of Figure 1 that the shape of the light emitted by the light sources expands in a direction perpendicular to the direction of emission to form a beam of light having a rectangular shape perpendicular to the direction of emission. The illumination light emitted from the light sources is directed towards the emitter optics 11, which are configured to focus each laser onto a one-dimensional MEMS mirror 12. For example, the emitter optics 11 can be a lens or a prism.

[0051] When reflected by the MEMS mirror 12, the light from the light sources is vertically aligned to form a one-dimensional vertical scan line SL of infrared light or a vertical strip of infrared light for each emitted laser beam. Each light source of the illumination unit 10 contributes to a different vertical area of the vertical scan line SL. Thus, the light sources can be activated and deactivated simultaneously to obtain a pulse of light having a plurality of vertical segments, each corresponding to a respective light source. However, each vertical area or segment of the vertical scan line SL can also be independently activated or not activated by turning on or off a corresponding one of the light sources of the illumination unit 10. Thus, a portion or the entire vertical scan line SL of light can be output from the system 100 towards the field of view.

[0052] Thus, the emitter of the system 100 is an optical arrangement configured to generate a laser beam based on a laser pulse, the laser beam having a rectangular shape extending in a direction perpendicular to the direction of emission of the laser beam. Figure 1 It can be seen that each light source is associated with a different vertical area in the field of view, such that each light source only illuminates the vertical scan line into the vertical area associated with the light source. For example, a first light source illuminates into a first vertical area and a second light source illuminates into a second vertical area different from the first vertical area.

[0053] Further, although three laser sources are shown, it should be understood that the number of laser sources is not limited thereto. For example, the vertical scan line SL can be generated by a single laser source, two laser sources, or more than three laser sources.

[0054] The MEMS mirror 12 is a mechanically moving mirror (i.e., a MEMS micro-mirror) integrated on a semiconductor chip (not shown). The MEMS mirror 12 according to the present embodiment is used in a ID scanning system and is configured to rotate about a single scan axis. Unlike a 2D-MEMS mirror (2D MEMS scanner), a ID MEMS mirror has only one degree of scanning freedom, through which a single scan axis is fixed to a non-rotating substrate, thus maintaining its spatial orientation during oscillation of the MEMS mirror. Due to this single scan rotation axis, the MEMS mirror 12 is referred to as a ID MEMS mirror or a ID MEMS scanner.

[0055] It can be appreciated that a 2D scanning system can also be used, such as a 2D MEMS mirror oscillating about two orthogonal scan axes or two ID MEMS mirrors each oscillating about a single scan axis orthogonal to each other (i.e., a 2x ID scanning system). The latter can be referred to as a Lissajous scanning system.

[0056] Typically, a ID scanning system emits a scan line of laser light into a field of view. In contrast, a 2D scanning system emits a scan spot (e.g., a circular or square spot) of laser light into a field of view. Thus, for a ID scanning system, the scan line SL and the receive line RL shown in FIG. 1A can be replaced by a scan spot and a receive spot, respectively, for a 2D scanning system. Figure 1 The scan line SL and the receive line RL shown in FIG. 1A can be replaced by a scan spot and a receive spot, respectively, for a 2D scanning system.

[0057] Figure 1 The MEMS mirror 12 in FIG. 1A is configured to continuously oscillate “side-to-side” about a single scan axis 13 such that light reflected from the MEMS mirror 12 (i.e., a vertical scan line of light) oscillates back and forth in a horizontal scan direction. The movement of the MEMS mirror 12 is continuous throughout a scan period (e.g., for a full scan of the field of view). For example, a scan period, an oscillation period, or a frame is defined by one complete oscillation from a first edge (e.g., left side) of the field of view to a second edge (e.g., right side) of the field of view and then back to the first edge. A mirror period or a frame of the MEMS mirror 12 corresponds to a scan period. Thus, a frame is one complete scan of the field of view by one complete mirror period.

[0058] Thus, by changing the angle of the MEMS mirror 12 over its scan axis 13, the field of view is scanned in horizontal direction by the vertical light stripes. For example, the MEMS mirror 12 can be configured to oscillate between + / - 15 degrees in the horizontal scan direction to steer the light over + / - 30 degrees (i.e. 60 degrees), constituting the horizontal scan range of the field of view. Thus, the field of view can be continuously scanned line by line by the MEMS mirror 12 with its degree of motion rotation. One such sequence of degrees of motion is referred to as a single scan (e.g. from -15 degrees to +15 degrees, and vice versa). Thus, two scans are used per scan cycle. Multiple scans can be used to generate distance and depth maps and 3D images by the processing unit. The horizontal resolution of the depth map and images depends on the size of the incremental steps of the rotation angle of the MEMS mirror 12 taken between scans.

[0059] While the transmitting mirror is described in the context of a MEMS mirror, it is understood that other ID mirrors or even 2D mirrors can be used as well. Further, the degree of rotation is not limited to + / - 15 degrees and the field of view can be increased or decreased depending on the application. Thus, the one-dimensional scanning mirror is configured to oscillate around a single scan axis and direct the laser beam of different directions into the field of view. Thus, the transmission technique comprises transmitting the light beam from the transmitting mirror oscillating around a single scan axis into the field of view such that the light beam is projected into the field of view as a vertical scan line SL which moves horizontally across the field of view as the transmitting mirror oscillates around the single scan axis. In comparison to 2D scanning mirrors which scan the field of view using a laser point (which requires more transmissions from the transmitter to scan the field of view), the LIDAR system using the ID scanning mirror can use a more relaxed transmission rate of the illumination unit 10 (i.e. the transmitter). Further, in comparison to 2D scanning mirrors, the LIDAR system using the ID scanning mirror is generally more resistant to impacts and vibrations and is thus well suited for automotive applications.

[0060] Upon hitting one or more objects, the transmitted stripe of vertical light is reflected by backscattering towards the LIDAR scanning system 100, where the second optical component 14 (e.g. a lens or a prism) receives the reflected light. The second optical component 14 directs the reflected light onto the segmented pixel sensor 15, which receives the reflected light as a receiving line RL and is configured to generate an electrical measurement signal. The segmented pixel sensor 15 generates a digital measurement signal based on the received light. The digital measurement signal can be used to generate a 3D map of the environment and / or other object data (e.g. via TOF calculations and processing) based on the reflected light.

[0061] The receiving line RL is shown as a column of vertical light extending along one of the pixel columns in the longitudinal direction of the pixel columns. The receiving line has a length in the longitudinal direction of the pixel columns which is equal to the length of the pixel column in the longitudinal direction of the pixel columns. The receiving line has a length in the transverse direction of the pixel columns which is equal to the length of the pixel column in the transverse direction of the pixel columns. Figure 1The vertical regions of the illustrated vertical scan lines SL correspond to three vertical regions. As the vertical scan lines SL move horizontally across the field of view, the vertical light columns RL incident on the 2D segmented pixel sensor 15 also move horizontally across the 2D segmented pixel sensor 15. The reflected light beams RL move from a first edge of the photodetector array 15 to a second edge of the photodetector array 15 as the receiving direction of the reflected light beams RL changes. The receiving direction of the reflected light beams RL corresponds to the emitting direction of the scan lines SL.

[0062] The segmented pixel sensor 15 receives the reflected light pulses as receiving lines RL and generates digital electrical signals in response thereto. Since the time of emission of each light pulse from the illumination unit 10 is known, and since light travels at a known speed, the distance of the object from the segmented pixel sensor 15 can be determined using time-of-flight calculations of the electrical signals. Alternatively, for indirect measurement, the phase difference between the emitted light and the received reflected light can be measured to calculate the distance to the object. A depth map can be plotted of the distance information.

[0063] In one example, for each distance sample, the microcontroller triggers a laser pulse from each light source of the illumination unit 10 and also starts a timer in a time-to-digital converter (TDC) integrated circuit (IC). The laser pulse propagates through the emission optics, is reflected by the target field, and is captured by one or more receiving sub-sub-pixels of the segmented pixel sensor 15. Each receiving sub-sub-pixel emits a short electrical pulse that is read out by the readout circuit.

[0064] The comparator IC identifies the pulse and sends a digital signal to the TDC to stop the timer. The TDC uses a clock frequency to calibrate each measurement. The TDC sends serial data of the differential time between the start and stop digital signals to the microcontroller, which filters out any erroneous readings, averages multiple time measurements, and calculates the distance to the target at a particular field location. By emitting successive light pulses in different directions established by the MEMS mirror 12, the region (i.e., field of view) can be scanned, a three-dimensional image can be generated, and objects within the region can be detected.

[0065] The signal processing chain of the receiver can also include an ADC for each photodiode. The ADC is configured to convert the analog electrical signal from the photodiode into a digital signal for further data processing.

[0066] Furthermore, instead of using a TDC method, an ADC can be used for signal detection and ToF measurement. For example, each ADC can be used to detect the analog electrical signal from one or more photodiodes to estimate the time interval between the start signal (i.e., corresponding to the timing of the emitted light pulse) and the ToF hit signal (i.e., corresponding to the timing of receiving the analog electrical signal at the ADC) with an appropriate algorithm.

[0067] In one example, all light sources of the illumination unit 10 can be used to generate the scan line SL / receiving line RL. In this case, the receiving line RL can extend along the full length of the segmented pixel sensor 15. In another example, only a subset of the light sources can be used to generate the scan line SL / receiving line RL. In this case, the receiving line can only extend in the longitudinal direction along a portion of the segmented pixel sensor 15.

[0068] The position and / or shape of the activated pixels of the segmented pixel sensor 15 can be dynamically changed during the scanning operation to follow the received light (i.e., the receiving line or receiving point) as it moves across the segmented pixel sensor 15 in synchronization with the scanning motion of the emitted light. Thus, those sub-sub-pixels located in the position of the segmented pixel sensor 15 where the light is expected to be received can be selectively activated, while those sub-sub-pixels located outside of this expected position can be selectively deactivated.

[0069] Figure 2 is a schematic block diagram of a LIDAR scanning system 200 in accordance with one or more embodiments. In particular, Figure 2 Additional features of the LIDAR scanning system 200 are shown, including example processing and control system components, such as MEMS drivers, receiver circuitry, and a system controller.

[0070] The LIDAR scanning system 200 includes a transmitter unit 21 responsible for the transmitter path of the system 200 and a receiver unit 22 responsible for the receiver path of the system 200. The system also includes a system controller 23 configured to control the components of the transmitter unit 21 and the receiver unit 22, and to receive raw analog or digital data from the receiver unit 22 and perform processing (e.g., via analog and / or digital signal processing) thereon to generate object data (e.g., point cloud data). Thus, the system controller 23 includes at least one processor and / or processor circuitry (e.g., comparators and digital signal processors (DSPs)) for a signal processing chain to process the data, and control circuitry (such as a microcontroller) configured to generate control signals. The LIDAR scanning system 200 can also include sensors 26, such as temperature sensors, that provide sensor information to the system controller 23.

[0071] The transmitter unit 21 comprises the illumination unit 10, the MEMS mirror 12 and a MEMS driver 25 configured to drive the MEMS mirror 12. The MEMS driver 25 can continuously drive the MEMS mirror 12 during the full scan of the field of view. In particular, the MEMS driver 25 actuates and senses the rotational position of the mirror and provides the system controller 23 with position information of the mirror (e.g. the tilt angle or the rotational degree around the rotational axis). Based on this position information, the laser source of the illumination unit 10 is triggered by the system controller 23 and the photodiode is activated to sense and thereby measure the reflected light signal. Thus, the higher the position sensing accuracy of the MEMS mirror, the more precise the control of the other components of the LIDAR system.

[0072] The receiver unit 22 comprises the segmented pixel sensor 15 and a receiver circuit 24 comprising a digital readout circuit. As will be described in more detail below, the pixels of the segmented pixel sensor 15 can be coupled to readout channels of the receiver circuit 24 from which electrical signals are received. Pixels (more specifically, their sub-sub-pixels) coupled to the receiver circuit 24 can be referred to as active, while sub-sub-pixels not coupled to the receiver circuit 24 can be referred to as inactive.

[0073] The readout circuit comprises N output channels (e.g. 32 channels) configured to read out measurement signals received from selected pixels of the segmented pixel sensor 15. Furthermore, multiple pixels can be formed and read out. The acquisition of pixel data (i.e. pixel measurement signals) from the segmented pixel sensor 15 on the output channels can be referred to as sampling, and each output channel can be used to acquire different samples from different pixels. Each sample can further correspond to a sampling time at which the pixel measurement signals are read out from one or more pixels.

[0074] Thus, the receiver circuit 24 can receive digital electrical signals from the pixels of the segmented pixel sensor 15 and transmit the electrical signals as raw analog or digital data to the system controller 23 for ToF measurements and generation of object data (e.g. 3D point cloud data).

[0075] The receiver circuit 24 can further receive trigger control signals from the system controller 23 to trigger the activation of one or more sub-sub-pixels or, conversely, to disable one or more sub-sub-pixels. Thus, the system controller 23 can control which sub-sub-pixels are enabled and which are disabled. In turn, the readout circuit 24 can be configured to activate or deactivate specific sub-sub-pixels of the segmented pixel sensor 15. The receiver circuit 24 can further receive gain setting control signals for controlling the gain of one or more sub-sub-pixels.

[0076] Figure 3A is a schematic illustration of a SiPM sub-pixel 1 according to one or more embodiments. Figure 3Bis a schematic view of an example pixel 5 of a segmented pixel sensor 15 according to one or more embodiments. Thus, Figure 3B only represents a portion of the entire segmented pixel sensor 15.

[0077] In particular, the segmented pixel sensor 15 is composed of an array of SPADs, in this case arranged in a 4x4 array of SiPM sub-pixels 1 to form a pixel 5. However, as will be explained, this configuration of SiPM sub-pixels can be adjusted in shape and size to form a pixel 5. Each SiPM sub-pixel 1 comprises an array of micro-cells 2, each micro-cell 2 comprising a SPAD 3 in series with a quenching circuit 4 (e.g. a resistor or a transistor). Thus, each SiPM sub-pixel 1 comprises a selectable cluster of sub-sub-pixels (i.e. SPADs) selected from the complete array of sub-sub-pixels in the segmented pixel sensor 15. In other words, a single SiPM sub-pixel can be referred to as an array of SPADs, and each SPAD can be referred to as a SPAD sub-sub-pixel.

[0078] Each SPAD is essentially a binary device (either a photon hits it or not). Once a photon is received, the SPAD 3 generates an electrical pulse. The strength of the signal generated by a SiPM is obtained by counting the number of output pulses generated by its activated SPADs within a measurement time slot (photon counting), or by detecting the cumulative current of all SPADs that do not resolve each photon event, while the time-dependent waveform of the signal is obtained by measuring the time distribution of the output signal (photon timing). The latter can be obtained by operating the SPAD detector in time-correlated single-photon counting (TCSPC).

[0079] In particular, a SPAD is a solid-state photodetector in which a photon-generated carrier (via an internal photoeffect) can trigger a short but relatively large avalanche current. This avalanche is produced through a mechanism called impact ionization, whereby the carriers (electrons and / or holes) are accelerated to high kinetic energy by a large potential gradient (voltage). If the kinetic energy of the carriers is sufficient (as a function of the ionization energy of the bulk material), more carriers are freed from the atomic lattice. Thus, in some cases, the number of carriers grows exponentially compared to a single carrier.

[0080] The avalanche current rises rapidly [sub-nanosecond rise time] to a macroscopic steady level in the milliamp range. If the primary carrier is photon-generated, the leading edge of the avalanche pulse marks the arrival time of the detected photon [with picosecond time jitter]. The current continues until the avalanche is quenched by the bias voltage V BIAS is reduced or lowered below the breakdown voltage V BD and the avalanche dies out. The internal capacitance is the stray or parasitic capacitance of the SPAD and is given by Figures 5A-5Dinternal capacitance C D is represented.

[0081] When this happens, the lower electric field is no longer able to accelerate the carriers to collide with the lattice atoms to ionize, so the current stops. In order to be able to detect another photon, the bias voltage at the internal capacitance must be raised again (i.e., charged) above the breakdown voltage. This charging time makes the SPAD be blinded or inactive until the internal capacitance is charged above the breakdown voltage. The circuit responsible for quenching the avalanche current and subsequently recharging the internal capacitance is called quenching circuit 4. The quenching circuit 4 can be an active quenching-recharging circuit including active components (such as transistors) that is actively triggered by a clock signal or a passive quenching-recharging circuit including passive components (such as resistors) that is not actively triggered.

[0082] This quenching and recharging operation requires a suitable circuit that senses the front of the avalanche current, generates a standard output pulse synchronized with the avalanche accumulation, quenches the avalanche by lowering or below the breakdown voltage, and restores the photodiode to the operating level (i.e., above the breakdown voltage).

[0083] In addition, each SPAD can be selectively activated and deactivated. For example, this can be done by selectively coupling (activating) or decoupling (deactivating) the SPAD to the output of the segmented pixel sensor 15 through a multiplexer or selectively activating or deactivating its corresponding quenching-recharging circuit 4 so that the SPAD is no longer charged to the operating level. However, it should be understood that the activation and deactivation of the SPAD are not limited to these example techniques. In the case of using APDs as sub-sub-pixels, selectively coupling (activating) or decoupling (deactivating) the APD to the output of the segmented pixel sensor 15 can be used for the activation and deactivation of the APD.

[0084] In Figure 3A and Figure 3BIn the example of FIG. 1, the SiPM 1 includes 12 microcells 2. Thus, the SiPM 1 has 12 SPADs arranged in an array. The output of a pixel 5 formed from a 4x4 SiPM array is accumulated from the electrical signals generated by the SPADs 3. For example, if only one SPAD in the pixel detects a photon during a measurement period, the output of the pixel 5 can have an intensity I. On the other hand, if five SPADs in the pixel each detect a photon during the measurement period, the output of the pixel 5 can have an intensity 5I. If all of the SPADs in the array detect a photon during the measurement period, the output of the pixel 5 can have an intensity 192I (i.e., the total number of SPADs that make up the pixel 5). Thus, the contributions of all of the SPADs 3 in the formed pixel 5 are added together by a summing circuit to generate a pixel output signal (i.e., a pixel measurement signal). The number of SiPM subpixels and the number of SPAD sub-subpixels within each pixel 5 is fully configurable and can be actively changed during a scan operation according to a clustering algorithm implemented by the system controller 23.

[0085] The following example embodiments can use dynamic pixel clustering for a segmented pixel sensor and can dynamically adapt its pixel clustering for non-stationary beam steering so that the adapted pixels follow the projection of the laser beam on the segmented pixel sensor as it is steered into the field of view. Thus, by enabling pulsed sequences or continuous (FMCW) illumination with a continuously moving / oscillating scanner (i.e., a non-quasi-static scanner), oversampling and averaging can be achieved over multiple consecutive laser shots, and motion blur-free long-range scanning can be achieved since the motion of the continuously oscillating mirror is compensated for as the laser enters and exits the field of view.

[0086] According to the following example embodiments, sub-sub-pixels (e.g., SPADs or APDs) are freely grouped (i.e., dynamically) into configurable sub-pixels (e.g., SiPMs), and sub-pixels are freely grouped (i.e., dynamically) into configurable pixels to effectively define a “pixel.” These “pixels” are not limited to a fixed raster, but can vary in size, shape, and location in the segmented pixel sensor 15. This enables the sensor to more accurately match the pixels to the shape and expected location of the received laser. It also enables flexibility to increase or decrease resolution by reducing or increasing the size of the “pixels.” This applies to both 2D scanning LIDAR that receives a circular spot at the segmented pixel sensor 15, as well as 1D LIDAR that works with a line of reception of light. Since the location of the “pixels” is freely variable, the sensor can also compensate for distortion or blur caused by a moving MEMS mirror.

[0087] Effectively, only a small fraction of the array of microcells can be in an active state at any given time. This reduces power consumption, but equally importantly, it also reduces the amount of data transferred. It would not be possible to transfer data away from the receiver chip at a frequency of 1 GHz from the entire segmented pixel sensor 15.

[0088] Figures 4A-4H Various ways of clustering sub-pixels 1 into pixels 5 on the segmented pixel sensor 15 are shown, in accordance with one or more embodiments. Each sub-pixel 1 is formed from a selected cluster of sub-sub-pixels (not shown).

[0089] In Figure 4A , a single pixel 5 is formed from a cluster of sub-pixels 1. The pixel 5 lies in the projection of the illumination field of view 16 projected onto the segmented pixel sensor 15.

[0090] In Figures 4B-4H , multiple pixels 5 are formed in different arrangements based on different clustering of sub-pixels 1. For example, in Figure 4B , Figure 4C and Figure 4E , square pixels 5 are formed in a way that gaps are formed between them. To form the gaps, two adjacent pixels 5 are spatially separated by at least one deactivated sub-sub-pixel or deactivated sub-pixel. In Figure 4D , Figure 4F , Figure 4G and Figure 4H , all pixels 5 are adjacent to at least one adjacent pixel. In Figure 4D , four square pixels 5 are combined to form a sensing area, maximizing the covered area on the segmented pixel sensor 15. In Figures 4F-4H , sub-pixels 1 are clustered to form non-square (e.g., cross-shaped) pixels 5. Figure 4F The pixels 5 in Figure 4G are combined to form a sensing area that is circular with a gap near the center. Figure 4H The pixels 5 in Figure 4H are combined to form a sensing area that is offset from an X / Y grid. Finally, The pixels 5 in

[0091] are shaped and combined to form a non-rectangular area. Here, in Figures 4A-4H , the illumination area is circular, and the pixels 5 are combined in a way that approximately conforms to the shape of the illumination area (i.e., approximately circular sensing area).In addition to the pixel structure shown, the illumination of the pixels can also be different. For example, the illuminated sensor area can be larger than, can match, or can be smaller than the respective utilized sensor area. Furthermore, the average number can be fixed for x-scan or y-scan, while for Lissajous scan the average number can vary, depending on the frequency of the pixel rows being illuminated or even depending on the proportion of partially illuminated pixels. In the following, different embodiments are provided showing the dynamic activation and averaging of the pixels.

[0092] Figures 5A-5D Dynamic formation of pixels moving across a segmented pixel sensor is shown in accordance with one or more embodiments. Figures 5A-5D The sequence of laser shot events representing a Lissajous scan operation (e.g., four consecutive laser shots or four steps of a four-step laser shot). Thus, in step N (N being an integer), a first laser is emitted into the field of view in a first emission direction at a first time event (i.e., in a first laser shot), in step N+1, a second laser is emitted into the field of view in a second emission direction at a second time event (i.e., in a second laser shot), in step N+2, a third laser is emitted into the field of view in a third emission direction at a third time event (i.e., in a third laser shot), and in step N+3, a fourth laser is emitted into the field of view in a fourth emission direction at a fourth time event (i.e., in third laser shot). Similar principles can apply to FMCW beams, where each step corresponds to a different consecutive frequency ramp of four consecutive frequency ramps of the beam.

[0093] Here, an illumination area 16 larger than the respective utilized sensor area of the segmented pixel sensor 15 is used. The shape of the illumination area 16 can be different (e.g., square, circular, etc.). Active pixels 5a, inactive pixels 5b, and inactive sub-pixels arranged between active and / or inactive pixels are provided. Those pixels fully within the sensor area of the segmented pixel sensor 15 (e.g., a 4x4 area including sub-pixels) are activated, while those pixels partially within the sensor area due to extending outside the sensor area of the segmented pixel sensor 15 are deactivated. The pixels (including active and inactive pixels) move within the sensor area in order to follow the trajectory of the projected field of view or scene (i.e., the trajectory of the emitted laser beam). The spatial orientation of the pixels relative to each other remains fixed. Thus, when the inactive pixels move fully within the sensor area, they are activated. Similarly, when the active pixels move fully or partially outside the sensor area, they are deactivated.

[0094] If a large number of pixels are in the active state at the same time, the pixels lost at the border will have little effect on the illumination efficiency. For example, a 4x4 pixel area requires an illumination area of 5x5 pixels and will result in an illumination efficiency of 64%.

[0095] InFigure 5A In this case, pixels A, B, C and D are activated. Pixels E and F (not shown) are completely outside the sensor area.

[0096] In this case, pixels A, B, C and D are activated. Pixels E and F (not shown) are completely outside the sensor area. Figure 5B In this case, pixels A, B, C and D are moving in the direction of the projected scene trajectory. As they remain completely within the sensor area, they remain activated. Due to the movement of the pixel arrangement as a whole, pixels E and F are partially moved into the sensor area, but remain deactivated as they are not completely within the sensor area.

[0097] In this case, pixels A, B, C and D are moving in the direction of the projected scene trajectory. As they remain completely within the sensor area, they remain activated. Due to the movement of the pixel arrangement as a whole, pixels E and F are partially moved into the sensor area, but remain deactivated as they are not completely within the sensor area. Figure 5C In this case, pixels A, B, C and D are moving in the direction of the projected scene trajectory. As pixels A and B remain completely within the sensor area, they remain activated. However, pixels C and D are no longer completely within the sensor area and are deactivated. On the other hand, pixels E and F are completely moved within the sensor area and are activated.

[0098] Figure 5D The dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments. As with the previous example, the projected scene trajectory is shown by the dashed line 10.

[0099] , Figure 6 The pixel average of pixel A is shown. The system controller 23 receives a pixel measurement signal from pixel A in each of the sequential steps N to N+3 and applies an equal-weight average to generate an averaged pixel measurement signal. The same average can be applied to each of pixels A-E to generate averaged pixel data for each pixel across multiple frames. Figures 5A-5D

[0100] The dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments. As with the previous example, the projected scene trajectory is shown by the dashed line 10. Figures 7A-7F , Figures 5A-5D , Figures 7A-7F The sequence of laser shot events representing a Lissajous scan operation is shown. Here, six steps N to N+5 of six consecutive laser shots are shown. Here, the illumination area 16 matches the pixel area of the segmented pixel sensor 15, where the pixels are partially averaged to achieve higher efficiency, particularly for illuminations with a low number of pixels in the illumination area. In this case, the partial of pixels 5a and 5c within the illumination area are activated, while pixels 5b or partial of pixels 5d located outside the illumination area are deactivated.

[0101] Again, the formed pixels follow the movement of the projected scene trajectory across the sequential steps (i.e., sequential frames).

[0102] Figure 8 The dynamic formation of pixels moving across a segmented pixel sensor is shown according to one or more embodiments. As with the previous example, the projected scene trajectory is shown by the dashed line 10.Figures 7A-7F Averaging of the pixels of the illustrated pixel A. Here, the pixel A or parts thereof are only activated during steps N+1, N+2, N+3 and N+4. Thus, the average pixel measurement signal of the pixel A is generated from the pixel measurement signals taken from the pixel A for these four steps. The averaging algorithm can weight all pixels at least partially represented on the segmented pixel sensor 15 with their corresponding activation area, i.e. according to the number of activated sub-pixels of a given pixel. The averaging algorithm can weight the pixels according to their illuminated area. If a pixel is only illuminated by 50%, i.e. 50% of the sub-pixels of this pixel are activated, the weighting algorithm mathematically takes this fact into account in the averaging. For example, a pixel illuminated by 50% in one step can be assigned half the weight of a fully activated pixel.

[0103] Figures 9A-9D A dynamic formation of pixels moving across the segmented pixel sensor is illustrated according to one or more embodiments. As with the previous embodiment, the laser shot events represent a Lissajous scan operation. Here, four steps N to N+3 of four consecutive laser shots are illustrated. Here, the illumination area 16 matches the pixel area of the segmented pixel sensor 15. In this case, the entire pixel within the illumination area is activated, while the pixels outside the illumination area are deactivated. The time between the laser shots is exactly the time to move an integer number N of pixels. For example, N=1 and 2x averaging can be used for each pixel. Figures 5A-5D Figures 9A-9D The sequence of laser shot events representing a Lissajous scan operation is illustrated. Here, four steps N to N+3 of four consecutive laser shots are illustrated. Here, the illumination area 16 matches the pixel area of the segmented pixel sensor 15. In this case, the entire pixel within the illumination area is activated, while the pixels outside the illumination area are deactivated. The time between the laser shots is exactly the time to move an integer number N of pixels. For example, N=1 and 2x averaging can be used for each pixel.

[0104] Again, the formed pixels follow the movement of the projected scene trajectory across the sequence of steps, i.e. the sequence of frames.

[0105] Figure 10 A dynamic formation of pixels moving across the segmented pixel sensor is illustrated according to one or more embodiments. As with the previous embodiment, the laser shot events represent a Lissajous scan operation. Here, four steps N to N+3 of four consecutive laser shots are illustrated. Here, the illumination area 16 matches the pixel area of the segmented pixel sensor 15. In this case, the entire pixel within the illumination area is activated, while the pixels outside the illumination area are deactivated. The time between the laser shots is exactly the time to move an integer number N of pixels. For example, N=1 and 2x averaging can be used for each pixel. Figures 9A-9D Averaging of the pixels of the illustrated pixel A. The averaging algorithm only weights the entire pixel. In this case, the pixel A is only present within the illumination area in steps N+1 and N+2, and the average pixel measurement signal of the pixel A is generated from the pixel measurement signals taken from the pixel A in these two steps.

[0106] Furthermore, illumination areas smaller than the corresponding utilized sensor area can be used. It is assumed that pixels are at the edge of the utilized sensor area, and typical lasers supporting non-rectangular illumination profiles do not have sharp laser edges. In this case, at the edge of the illumination area, the pixels only take up a small fraction of the laser. Thus, the SNR at these pixels is reduced. However, the partially covered pixels and their pixel information can be used, for example, in a weighted manner for averaging in order to improve the overall laser efficiency.

[0107] Figure 11A and Figure 11B A schematic block diagram of the receiver circuit 24 according to one or more embodiments is illustrated. In particular,​Figure 11A For analog pixel multiplexing and data conversion, and Figure 11B For digital pixel multiplexing and pixel data computation. In each case, the receiver circuit 24 is selectively coupled to the sub-pixels 1 according to configuration control signals received from the system controller 23. The configuration control signals provide sub-sub-pixel and sub-pixel clustering information for selectively grouping the sub-sub-pixels into sub-pixels and for selectively grouping the sub-pixels into pixels.

[0108] In Figure 11A , the sum of the electrical signals (e.g., currents) from each sub-sub-pixel of the corresponding sub-pixel is received as a sub-pixel current. The sub-pixel currents are received at selected selection and summation circuits 31-1, 31-2, …, and 31-n, each of which corresponds to an assigned pixel based on the sub-pixel clustering information. Thus, the sub-pixel currents are grouped into the assigned pixels based on the assignment of the sub-pixel currents to particular selection and summation circuits 31-1, 31-2, …, and 31-n. Additional multiplexer and summation circuits (not shown) can be used to group and sum the sub-sub-pixels into sub-pixel currents based on the sub-sub-pixel clustering information and direct the sub-pixel currents to the assigned selection and summation circuits 31-1, 31-2, …, and 31-n according to the sub-pixel clustering information.

[0109] Each selection and summation circuit 31-1, 31-2, …, and 31-n selects which sub-pixel currents to add in the received sub-pixel currents and applies a summation algorithm to add the selected sub-pixel currents together to generate a pixel current for the corresponding pixel. The respective pixel current is transmitted to a respective transimpedance amplifier (TIA) 32-1, 32-2, …, and 32-n, which converts the pixel current to a voltage signal. The voltage signal is then provided to a respective ADC 33-1, 33-2, …, and 33-n, which converts the analog voltage signal to a digital signal. The digital signal is then transmitted from the receiver circuit 24 to the system controller 23 on a readout channel as pixel data for further processing.

[0110] In Figure 11B , the sub-sub-pixels (e.g., SPADs) provide sub-sub-pixel signals (e.g., currents) to corresponding 1-bit ADC detector circuits 41. That is, each sub-sub-pixel is electrically coupled to its own 1-bit ADC detector circuit 41. The 1-bit ADC detector circuit 41 detects whether a sub-sub-pixel is received and outputs a binary signal (i.e., 1 or 0) based on the detection. Thus, the binary signal is a digital sub-sub-pixel signal. This detection can be synchronized with all of the 1-bit ADC detector circuits 41 based on a clock signal.

[0111] Based on the sub-subpixel clustering information, the digital sub-subpixel signals are provided to assigned adders 42. In particular, each adder 42 corresponds to a different digital SiPM and generates the subpixel data for that SiPM by adding the digital sub-subpixel signals together. Multiplexers (not shown) can be used to direct the digital sub-subpixel signals to their assigned adders 42. Each adder 42 corresponds to a different subpixel and is configured to add the received digital sub-subpixel signals to the subpixel data for its corresponding subpixel. A multiplexer 43 receives the subpixel data for each subpixel and directs the subpixel data to the assigned pixel circuit according to the subpixel clustering information.

[0112] The pixel circuit arrangement includes weighting circuits 44 and adders 45. Each weighting circuit 44 applies an assigned weight (e.g., 0 to Wn) according to one of the aforementioned weighting schemes (e.g., equal weighting as shown in Figure 6 and Figure 10 pixel weighting as shown in Figure 8 etc.). Configuration control signals can be provided to program the weighting information for the weighting circuits 44.

[0113] Each adder 45 receives the weighted subpixel data from its respective weighting circuit 44 and adds the weighted subpixel data to generate the pixel data for its respective pixel. The digital pixel signals are then emitted as pixel data from the receiver circuit 24 on the readout channel to the system controller 23 for further processing.

[0114] The mirror position at the time of the laser shot is only related to a specific area of the illuminated field of view. While the laser pulse is typically very short (once ns) and thus not affected by continuous mirror movement, the mirror position at the time of receiving the light varies based on the time of flight. That is, the position of the receiver mirror can be at different angles depending on when the reflected beam is received. Thus, the position of the illuminated area on the segmented pixel sensor 15 can change based on the time of flight due to continuous mirror movement. In other words, the scene representation on the segmented pixel sensor 15 depends on the time delay (up to once us) and the mirror speed.

[0115] Short and long distance objects on the reflected laser road will appear at different positions on the segmented pixel sensor 15. Adaptive pixel clustering can also be used to compensate for this effect.

[0116] Figure 12A is a schematic diagram of a LIDAR scanning system 1200 according to one or more embodiments. Figure 12B is a schematic block diagram of a LIDAR scanning system 1200 including additional circuitry similar to that shown in Figure 2 .

[0117] The LIDAR scanning system 1200 comprises a first 1D MEMS mirror 12y that performs beam steering in the y-direction and a second 1D MEMS mirror 12x that performs beam steering in the x-direction. Both MEMS mirrors make up a 2x1D mirror scanning system for scanning a field of view. Both mirrors are used for both transmitting and receiving a light beam. Additionally, the LIDAR scanning system 1200 comprises an illumination unit 10, optics 14, and a segmented pixel sensor 15.

[0118] Figure 12A It is shown how short and long range objects appear at different locations on the segmented pixel sensor 15, where due to the difference in rotational position of the continuously oscillating MEMS mirrors 12x and 12y, a close range object (short flight time) is projected via reflected laser light onto one area of the segmented pixel sensor 15, while other objects (longer flight time) are projected via reflected laser light onto another area of the segmented pixel sensor 15.

[0119] The MEMS drivers 25y and 25x not only drive the oscillation of their respective MEMS mirrors 12y and 12x, but also sense the rotational position (i.e. angular position or tilt angle) of their respective mirrors around their rotational axis. The position information of each mirror is provided to the system controller 23. The system controller 23 uses the position information of each mirror to predict the projected area 16 of the received light beam on the segmented pixel sensor 15.

[0120] As the flight time increases, the rotational position of the mirrors 12y and 12x continues to change. Thus, the projected area 16 can start at a starting area based on the rotational position of the mirrors 12y and 12x when the light beam is emitted by the illumination unit 10 (i.e. at time To). The projected area 16 changes as the rotational position of the mirrors 12y and 12x changes and further deviates from the starting position as the flight time increases. A short flight time can correspond to time T1, while a longer flight time can correspond to time T2.

[0121] As the projected area 16 changes, the system controller 23 changes the pixel position of the active pixel 5a to remain within the expected projected area 16. In this example, the active pixel 5a remains at the center of the expected projected area 16. To change the pixel position of the active pixel 5a, the system controller 23 updates the sub-subpixel and subpixel cluster information and transmits the updated configuration control signal to the receiver circuit 24. In response to the updated configuration control signal, the receiver circuit 24 changes the sub-subpixel and subpixel clusters to change the position of the active pixel 5a while deactivating the remaining sub-subpixels. Figures 5A-5D One example of changing the pixel position based on the expected trajectory of the received laser light is shown and similarly applies to compensate for mirror movement during the flight time.

[0122] Thus, after a laser shot, the active pixel position follows the projection on the pixel sensor 15 to compensate for the mirror movement during the time of flight. This compensation technique is especially important for distant objects and very fast (kHz) moving mirrors.

[0123] For example, the pixel position that can vary with the sub-pixel resolution can be moved at any time to the one that is best suited for the intended projection (x and y movements can be different and need to be switched at different times). For example, for a field of view x*y = 60°*60°, with 120 pixels in the x direction, the pixel size represents 0.5° and is composed of 4x4 sub-pixels (1 sub-pixel corresponds to approximately 0.125°), the laser is moving at a speed of 0.25° / us in the x direction and at a speed of 0.2° / us in the y direction, so that every 0.5us the pixel is moved by 1 sub-pixel in the x direction and every 0.625us by 1 sub-pixel in the y direction to compensate for the movement of the projection on the pixel sensor 15.

[0124] It should be noted that the switching between sub-pixels can potentially cause steps in the received signal during the reception of a single laser shot response (pixel signal) due to the different gain of the sub-pixels. However, this is not critical as a symmetric filter (i.e. matched filter) can be used to suppress such steps.

[0125] For pulsed laser beams (i.e. pulsed sequence LIDAR), the LIDAR scanning system can shoot the laser when the moving / oscillating MEMS mirrors 12y and 12x reach their desired position. The desired position is reached when the illuminated area in the field of view coincides with the active pixel position within the sub-pixel resolution.

[0126] Alternatively, for pulsed laser beams, the LIDAR scanning system can take a shot at any time and dynamically adjust the pixel clustering in a way that the illuminated area in the field of view coincides as closely as possible with the active pixel position within the sub-pixel resolution.

[0127] Alternatively, for FMCW LIDAR and pulsed sequence LIDAR, the LIDAR scanning system can dynamically adjust the active pixel position depending on the beam that is controlled and moved over the field of view, for example as Figures 5A-5D shown.

[0128] The above techniques for fast successive laser shots and pixel position compensation for average or time-of-flight mirror movement can be used separately or in combination. In one possible embodiment, both techniques can be combined. In this embodiment, after a laser pulse shot, the pixel position is moved to compensate for mirror movement during the time-of-flight. When the receiving phase of the laser shot, including signal averaging, is completed, the laser pulse can be shot again from the new pixel position. Thus, the receiving phase, as well as the time for the mirror to move one pixel, can be in line with the laser shot repetition rate of the illumination unit 10.

[0129] Figure 13A is Figure 12A A schematic diagram of the Lissajous LIDAR scanning system 1200 is shown, but additionally two successive laser shots 17 and 18 into the field of view are shown. The two successive laser shots 17 and 18 can correspond to any two successive steps (e.g., N and N+1, N+1 and N+2, etc.) described above. However, for this example, the two successive laser shots 17 and 18 correspond to steps N and N+1. As can be seen from the field of view, the two successive laser shots 17 and 18 overlap by one pixel. In this case, pixel C is shared by both successive laser shots 17 and 18. In other words, for both laser shots 17 and 18, pixel C will be in the active state on the segmented pixel sensor 15.

[0130] Figure 13B and Figure 13C Different possible pixel positions on the segmented pixel sensor 15 to receive the laser shots 17 and 18 based on the time-of-flight of each laser shot 17 and 18, or more specifically, based on the position change of the MEMS mirrors 12y and 12x, are shown based on the time-of-flight of the laser shots 17 and 18. Figure 13B Different receiving phases are shown, each phase corresponding in turn to a range of longer time-of-flight and a different range of mirror positions. Figure 13C Movement or change of the position of the active pixels of the current receiving phase as a function of the increasing time-of-flight is shown according to Figure 13B

[0131] For example, the laser shot 17 can be received in one of four preconfigured receiving phases M, M+1, M+2, and M+3, each phase corresponding in turn to a range of longer time-of-flight and a different range of mirror positions. M is an integer. Each of the preconfigured receiving phases M, M+1, M+2, and M+3 is assigned a different pixel position of the active pixels A, B, C, and D for receiving the laser shot 17. As the time-of-flight progresses through the different receiving phases, the system controller 23 updates the sub-sub-pixel and sub-pixel clustering information to change the pixel position to accommodate the current receiving phase.

[0132] ​In other words, in one of the plurality of discrete areas of the pixel area, one or more pixels are selectively activated. Each discrete area is assigned to a different one of the plurality of position ranges of the MEMS mirror 12. For example, the reception phase M corresponds to a first range of time-of-flight (i.e. a first time period) which corresponds to a first position range of angular positions or rotational positions of the MEMS mirror 12. The first position range is the range of angular positions in which the MEMS mirror 12 is expected to travel from the beginning of the reception phase M to the end of the reception phase M. Thus, in a first discrete area of the pixel area, the projection of the received laser light deflected from the MEMS mirror 12 onto the segmented pixel sensor 15 can be expected. As such, during the reception phase M, one or more pixels within this first discrete area are activated.

[0133] If no laser light is received in the reception phase M, the system will enter a reception phase M+1. The reception phase M+1 corresponds to a second range of time-of-flight (i.e. a second time period) which corresponds to a second position range of angular positions or rotational positions of the MEMS mirror 12. The second position range is the range of angular positions in which the MEMS mirror 12 is expected to travel from the beginning of the reception phase M+1 to the end of the reception phase M+1. Thus, in a second discrete area of the pixel area, the projection of the received laser light deflected from the MEMS mirror 12 onto the segmented pixel sensor 15 can be expected. As such, during the reception phase M+1, the position of one or more pixels within this second discrete area is changed and one or more pixels are activated.

[0134] If no laser light is received in the reception phase M+1, the system will enter a reception phase M+2 and if no laser light is received in the reception phase M+2, the system will enter M+3.

[0135] At the beginning of the reception phase M+4, the illumination unit 10 emits a laser shot 18 which also has different specified pixel positions for the activated pixels C, E, F and G. Similar to the laser shot 17, the laser shot 18 can be received in one of four preconfigured reception phases M+4, M+5, etc.

[0136] The system controller 23 is configured to determine a position range of the plurality of position ranges of the MEMS mirror 12 based on the position information received from the driver 25 and to change the cluster of activated sub-pixel elements and the cluster of deactivated sub-pixel elements to form at least one activated pixel based on the determined position range.

[0137] In most practical cases, the mirror movement time will be higher than the reception phase (also the mirror speed varies, e.g. via a sinusoidal mirror movement) and the laser shot repetition time will also be higher than the reception phase to simplify the post (subsequent) processing (as only one laser pulse is received in the respective reception phase).

[0138] However, in another embodiment, the reception phase can also be covered by having multiple laser pulses in the air for parallel processing of the multiple pulses that fire before the end of the reception phase of the first laser pulse.

[0139] In this case, a pixel clustering approach based on digital SIPM can be useful as it allows for parallel clustering and reception of different but partially overlapping pixels at the same time. In contrast, the analog current of a subpixel cannot be easily added for different overlapping pixels.

[0140] The embodiments described herein are represented by a receiver structure that uses dynamic pixel clustering for segmented pixel sensors that dynamically adapt their pixel clustering for non-static beam control. Thus, the pixels follow the projection of the laser beam onto the pixel sensor that is directed to the scene.

[0141] Pixel clustering for APDs can come at the cost of reduced area efficiency and photon detection efficiency.

[0142] Pixel clustering is very useful for receiving elements that already cluster, such as analog and digital SiPMs. In particular, digital SiPMs would benefit from pixel clustering as the sub-subpixel data is digitally available and thus can be more easily added in different clusters.

[0143] Although the embodiments described herein relate to LIDAR systems, it should be understood that the above-described embodiments are applicable to other applications and are not limited to LIDAR.

[0144] Furthermore, although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware apparatus, like, for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more method steps can be performed by such an apparatus.

[0145] Depending on certain implementation requirements, embodiments provided herein can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0146] Instructions can be executed by one or more processors, such as one or more central processing units (CPU)s, digital signal processors (DSP)s, general purpose microprocessors, application-specific integrated circuits (ASIC)s, field programmable logic arrays (FPGA)s, or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein can refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0147] The foregoing exemplary embodiments are merely illustrative. It will thus be appreciated that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. It is intended that the invention embrace all such modifications and variations as fall within the scope of the appended claims and their equivalents.

Claims

1. A scanning system comprising: a first scanning structure configured to rotate around at least one first scanning axis; a driver configured to drive the first scanning structure around the at least one first scanning axis and to detect a position of the first scanning structure relative to the at least one first scanning axis during movement of the first scanning structure; a segmented pixel sensor comprising a plurality of sub-pixel elements arranged in a pixel area; and a controller configured to: selectively activate and deactivate the plurality of sub-pixel elements to obtain at least one active cluster and at least one inactive cluster to form at least one active pixel from the at least one active cluster; receive, from the driver, first position information indicative of the detected position of the first scanning structure; and dynamically change a cluster of activated sub-pixel elements and a cluster of deactivated sub-pixel elements based on the first position information, wherein a position of an illuminated area on the segmented pixel sensor changes based on time of flight due to continuous mirror movement.

2. The scanning system of claim 1, wherein the controller is configured to change a position of the at least one active cluster and a position of the at least one inactive cluster within the pixel area as the position of the first scanning structure changes.

3. The scanning system of claim 1, wherein the controller is configured to change a position of the at least one active pixel within the pixel area as the position of the first scanning structure changes.

4. The scanning system of claim 1, wherein: the at least one active pixel is selectively activated in one discrete area of a plurality of discrete areas of the pixel area, wherein each discrete area of the plurality of discrete areas is assigned to a different position range of a plurality of position ranges of the first scanning structure, and the controller is configured to determine a position range of the plurality of position ranges of the first scanning structure based on the first position information and to change the cluster of activated sub-pixel elements and the cluster of deactivated sub-pixel elements based on the determined position range to form the at least one active pixel.

5. The scanning system of claim 1, wherein the controller is configured to selectively group sub-pixel elements of the at least one active cluster into a plurality of active pixels.

6. The scanning system of claim 5, wherein the controller is configured to change a position of the plurality of active pixels within the pixel area as the position of the first scanning structure changes.

7. The scanning system of claim 1, wherein the position of the first scanning structure continuously changes during a full scan of a field of view.

8. The scanning system of claim 1, wherein sub-pixel elements of the at least one active cluster are configured to generate an electrical signal based on detected light and sub-pixel elements of the at least one inactive cluster are deactivated based on detected light so as not to generate an electrical signal.

9. The scanning system of claim 1, further comprising: a readout circuit coupled to the segmented pixel sensor, ​ wherein sub-pixel elements of the at least one active cluster are coupled to the readout circuit, and sub-pixel elements of the at least one inactive cluster are decoupled from the readout circuit.

10. The scanning system of claim 1, further comprising: a readout circuit configured to receive electrical signals from the sub-pixel elements of the at least one active cluster, and combine the electrical signals according to the at least one active pixel to generate a pixel signal for each of the at least one active pixel.

11. The scanning system of claim 10, wherein the readout circuit is configured to add electrical signals of active pixels of the at least one active pixel together according to assigned weights to generate a pixel signal for the active pixel.

12. The scanning system of claim 11, wherein: the controller is configured to change a position of the active pixel within the pixel area with a change in position of the first scanning structure, and the readout circuit is configured to generate a first pixel signal for the active pixel arranged in a first position of the pixel area, generate a second pixel signal for the active pixel arranged in a second position of the pixel area, and generate an average pixel signal based on the first pixel signal and the second pixel signal.

13. The scanning system of claim 1, wherein the controller is configured to form at least one inactive pixel from the at least one inactive cluster.

14. The scanning system of claim 1, wherein: the segmented pixel sensor comprises a plurality of sub-sub-pixel elements for forming the plurality of sub-pixel elements, the controller is configured to selectively group the plurality of sub-sub-pixel elements into the plurality of sub-pixel elements.

15. The scanning system of claim 14, wherein the sub-pixel elements are activated by activating their corresponding sub-sub-pixel elements, and are deactivated by deactivating their corresponding sub-sub-pixel elements.

16. The scanning system of claim 14, wherein each of the plurality of sub-sub-pixel elements is a photodiode, an avalanche diode, or a single-photon avalanche diode.

17. The scanning system of claim 1, further comprising: a second scanning structure configured to rotate around a second scanning axis, wherein the driver is configured to drive the second scanning structure around the second scanning axis, and detect a position of the second scanning structure relative to the second scanning axis during movement of the second scanning structure, wherein the controller is configured to receive second position information indicative of the detected position of the second scanning structure from the driver, and dynamically change the clustering of the active sub-pixel elements and the clustering of the inactive sub-pixel elements based on the first position information and the second position information.

18. The scanning system of claim 1, wherein the driver continuously drives the first scanning structure during a full scan of a field of view.

19. A scanning method, comprising: driving a first scanning structure around at least one first scanning axis; detecting a position of the first scanning structure relative to the at least one first scanning axis during movement of the first scanning structure; selectively activating and deactivating a plurality of sub-pixel elements of a segmented pixel sensor to obtain at least one active cluster and at least one inactive cluster to form at least one active pixel from the at least one active cluster; and dynamically changing the cluster of activated sub-pixel elements and the cluster of deactivated sub-pixel elements based on first position information indicative of the detected position of the first scanning structure, wherein a position of an illuminated area on the segmented pixel sensor is changed based on time of flight due to continuous mirror movement.

20. The method of claim 19, further comprising: changing a position of the at least one active cluster and a position of the at least one inactive cluster to coincide with a change in position of the first scanning structure.

21. The method of claim 19, further comprising: changing a position of the at least one active pixel with a change in position of the first scanning structure.

22. The method of claim 19, further comprising: selectively activating the at least one active pixel in one discrete region of a plurality of discrete regions of a pixel area, wherein each discrete region of the plurality of discrete regions is assigned to a different position range of a plurality of position ranges of the first scanning structure; determining a position range of the plurality of position ranges of the first scanning structure based on the first position information; and changing the cluster of activated sub-pixel elements and the cluster of deactivated sub-pixel elements based on the determined position range to form the at least one active pixel.

23. The method of claim 19, wherein the position of the first scanning structure is continuously changed during a full scan of a field of view.

24. The method of claim 19, further comprising: combining electrical signals generated by sub-pixel elements of the at least one active cluster according to the at least one active pixel to generate a pixel signal for each active pixel of the at least one active pixel.

25. The method of claim 19, further comprising: driving a second scanning structure about a second scanning axis; detecting a position of the second scanning structure relative to the second scanning axis during movement of the second scanning structure; and dynamically changing the cluster of activated sub-pixel elements and the cluster of deactivated sub-pixel elements based on the first position information and second position information indicative of the detected position of the second scanning structure.

Images