Receiver for laser radar, laser radar and terminal device

Abstract

The invention relates to a receiver for a laser radar, the laser radar and terminal equipment. The receiver comprises an SPAD chip, the SPAD chip comprises a pixel array, the pixel array extends along a first direction and a second direction, the pixel array comprises a first pixel array, the first pixel array comprises first pixels and is configured to receive echoes, reflected by an object, of detection light of the laser radar and output echo signals; and a second pixel array including second pixels and configured to receive visible light and output an image signal; wherein the first pixel and the second pixel are aligned in the second direction, the pixel array further comprises a dummy pixel, and the dummy pixel is located between the first pixel and the second pixel in the first direction.

Description

Technical Field

[0001] This disclosure relates to the field of photoelectric detection, and more specifically to receivers, lidar, and terminal devices for lidar. Background Technology

[0002] LiDAR (Light Detection and Ranging) is a radar system that uses emitted laser beams to detect the position, velocity, and other characteristics of objects. Due to its advantages such as high resolution, strong anti-interference capability, good detection performance, small size, and light weight, LiDAR is widely used in fields such as autonomous driving, transportation communication, drones, and intelligent robots.

[0003] In fields such as autonomous driving and robotics, multi-sensor fusion technology has become a core solution for environmental perception. The combination of cameras and LiDAR has attracted much attention because it can take into account both image information and depth data. However, the receiving devices of cameras and LiDAR are usually separate or independent, which leads to problems such as difficulties in spatiotemporal synchronization of multi-source data, high calibration complexity, and increased system integration costs. Summary of the Invention

[0004] According to a first aspect of this disclosure, a receiver for a lidar is provided, comprising: a SPAD chip, the SPAD chip including a pixel array extending along a first direction and a second direction, the pixel array including: a first pixel array including a first pixel and configured to receive an echo of probe light from the lidar reflected from an object and output an echo signal; and a second pixel array including a second pixel and configured to receive visible light and output an image signal; wherein the first pixel and the second pixel are aligned in the first direction, and the pixel array further includes a virtual pixel located between the first pixel and the second pixel in the first direction.

[0005] Optionally, the virtual pixel includes a first virtual pixel and a second virtual pixel, the first virtual pixel is adjacent to the first pixel, the second virtual pixel is adjacent to the second pixel, and along the first direction, the interval between the first virtual pixel and the first pixel is the same as the interval between the second virtual pixel and the second pixel.

[0006] Optionally, the lidar includes multiple detection cycles. In one detection cycle, the first pixel in the second direction is activated in a round-robin fashion, and the second pixel in the second direction is activated in a round-robin fashion. The lidar is configured to determine depth information based on the echo signals of the multiple first pixels and to determine image information based on the image signals of the multiple second pixels.

[0007] Optionally, the plurality of detection cycles include a first detection cycle and a second detection cycle, wherein the first pixel determines depth information in the first detection cycle, and the second pixel determines image information in the second detection cycle, and the first detection cycle and the second detection cycle do not overlap.

[0008] Optionally, during the first detection period, the depth information includes multiple depth data in the first direction.

[0009] Optionally, the second pixel includes at least one of the following: a first sub-pixel including a first SPAD and a first filter disposed on one side of the first SPAD, the first filter being configured to allow light of a first wavelength range to pass through and be incident on the first SPAD; a second sub-pixel including a second SPAD and a second filter disposed on one side of the second SPAD, the second filter being configured to allow light of a second wavelength range to pass through and be incident on the second SPAD; a third sub-pixel including a third SPAD and a third filter disposed on one side of the third SPAD, the third filter being configured to allow light of a third wavelength range to pass through and be incident on the third SPAD; and a fourth sub-pixel including a fourth SPAD, the fourth SPAD being configured to receive light of a fourth wavelength range.

[0010] Optionally, the second pixel array includes the first sub-pixel, the second sub-pixel, and the third sub-pixel in the first direction.

[0011] Optionally, the second pixel array includes the first sub-pixel, the second sub-pixel, the third sub-pixel, and the fourth sub-pixel in the first direction.

[0012] Optionally, the receiver further includes a fourth filter disposed in the receiving optical path of the first pixel array and configured to allow the echo to pass through and be incident on the first pixel array.

[0013] According to a second aspect of this disclosure, a lidar is provided, comprising: a transmitter configured to emit probe light; a receiver as described above; and a controller configured to determine depth data based on the echo signal and to determine image data based on the image signal.

[0014] Optionally, the lidar further includes a scanner configured to oscillate or rotate about an axis to change the transmission direction of the probe light, the echo, and the visible light in the first direction.

[0015] Optionally, the optical paths of the probe light, the echo, and the visible light at least partially overlap.

[0016] According to a third aspect of this disclosure, a terminal device is provided, including a lidar as described in a second aspect of this disclosure. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be introduced as examples below. The drawings described below are merely embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain this disclosure and do not constitute a limitation of this disclosure. In the drawings: Figure 1 An example structural block diagram of an exemplary lidar consistent with some embodiments of this disclosure is shown.

[0018] Figure 2 A structural block diagram of an exemplary vehicle system consistent with some embodiments of this disclosure is shown.

[0019] Figure 3 A schematic block diagram of a LiDAR receiver consistent with some embodiments of this disclosure is shown.

[0020] Figure 4 A schematic block diagram of a LiDAR receiver consistent with some embodiments of this disclosure is shown. Figure 5 A schematic block diagram of a LiDAR receiver consistent with some embodiments of this disclosure is shown.

[0021] Figure 6 A schematic diagram is shown illustrating a lidar receiver detecting at different sampling angles, consistent with some embodiments of this disclosure.

[0022] Figure 7 A schematic block diagram of a LiDAR receiver consistent with some embodiments of this disclosure is shown.

[0023] Figure 8 Several examples of a second pixel consistent with some embodiments of this disclosure are shown.

[0024] Figure 8 Example A illustrates a schematic diagram of a second pixel consistent with some embodiments of this disclosure.

[0025] Figure 8 Example B illustrates a schematic diagram of a second pixel consistent with some embodiments of this disclosure.

[0026] Figure 8 Example C illustrates a schematic diagram of a second pixel consistent with some embodiments of this disclosure.

[0027] Figure 8 Example D illustrates a schematic diagram of a second pixel consistent with some embodiments of this disclosure.

[0028] Figure 9 A schematic block diagram of a lidar consistent with some embodiments of this disclosure is shown.

[0029] Figure 10 A schematic diagram is shown illustrating a lidar detection at different sampling angles, consistent with some embodiments of this disclosure.

[0030] It should be understood that the accompanying drawings are for illustrative purposes and should not be considered to be drawn to scale unless specifically stated otherwise. Furthermore, as schematic diagrams, the drawings are provided to aid understanding and may not include all aspects or information compared to a realistic representation. Detailed Implementation

[0031] The embodiments of this disclosure will be described below. It should be noted that, in order to provide a concise description of these embodiments, this specification cannot provide a detailed description of all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, changes may occur from one embodiment to another to achieve specific objectives. Furthermore, it is also understood that, although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content of this disclosure, some design, manufacturing, or production modifications based on the technical content disclosed in this disclosure are merely conventional technical means and should not be construed as insufficient content of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in the claims and description shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure and the claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The terms “connected,” “coupled,” or “linked” and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0033] Unless otherwise specified, all embodiments mentioned herein can be combined to form new technical solutions. Furthermore, unless otherwise specified, all technical features and preferred features mentioned herein can be combined to form new technical solutions.

[0034] In this disclosure, the terms "or" and "and / or" describe the relationship between related objects and indicate a non-exclusive inclusion. For example, "A and / or B" and "A or B" can include: the presence of only "A", the presence of only "B", and the presence of both "A" and "B", where "A" and "B" can be singular or plural. As another example, "A, B, and / or C" and "A, B, or C" can include: the presence of only "A", the presence of only "B", the presence of only "C", the presence of both "A" and "B", the presence of both "A" and "C", the presence of both "B" and "C", and the presence of both "A", "B", and "C", where "A", "B", and "C" can be singular or plural. Furthermore, the symbol " / " in this disclosure indicates an "or" relationship between the related objects before and after the symbol. In this disclosure, the term "at least one A or B" has the same meaning as "A or B" described above. The term "at least one A, B or C" has the same meaning as "A, B or C" above.

[0035] LiDAR (Light Detection and Ranging) is a type of remote sensing technology. LiDAR uses lasers to measure distances and create three-dimensional (3D) images of objects or landscapes. During object detection, the LiDAR emits a laser beam. This laser beam is reflected from the object's surface. The reflected light (called the echo) is received by the LiDAR and converted into an electrical signal. The LiDAR processes this electrical signal to determine information about the object, such as its distance, position, or velocity. LiDAR can also be configured to create real-time 3D models of the environment, which can be represented as point clouds. A point cloud is a collection of 3D data points representing the surfaces of objects, structures, and environments within a specific area. Each data point in the point cloud can be defined by its X, Y, and Z coordinates in space, representing its position in 3D space. Using point clouds, vehicles can accurately identify the positions of objects on the road, such as cars, pedestrians, and / or cyclists.

[0036] In some examples, LiDAR can generate point clouds, which simplifies and simplifies the processing of driver assistance algorithms. LiDAR provides vehicles, such as intelligent vehicles, with high-resolution 3D vision, working in conjunction with cameras and other types of radar to enhance vehicle perception and handle more complex road conditions, such as dark environments or unknown objects on highways. LiDAR can further provide high-performance automotive-grade LiDAR solutions, ensuring safer and smarter driver assistance, such as Level 2+ assisted driving. When configured, LiDAR can be widely used in passenger cars and commercial vehicles equipped with advanced driver assistance systems (ADAS) and / or autonomous driving (e.g., automated traffic). LiDAR can also be applied to any suitable end device, such as drones or robots. For example, LiDAR can support robotic applications such as delivery robots and logistics robots.

[0037] In some examples, LiDAR can be a long-range LiDAR sensor with a long detection range, such as hundreds of meters. Long-range LiDAR sensors can detect objects at a distance. They can be mounted on the roof of a vehicle (e.g., the front and / or rear roof) to provide an unobstructed view of the road ahead and / or behind, and to detect objects at greater distances. This is extremely useful for highway driving and for detecting distant objects as early as possible.

[0038] In some examples, lidar can be configured as a short-range lidar sensor. The detection range is relatively short, for example, within a few meters to tens of meters around the lidar. However, the field of view (FOV) is wide, for example, from 60 degrees to 360 degrees horizontally. The wider FOV allows for the detection of nearby objects and provides a more comprehensive view of the surrounding environment / objects. Short-range lidar sensors can be mounted near the headlights or sides of the vehicle, or near the rearview mirrors, improving blind spot awareness and assisting in lane keeping and / or lane changing maneuvers.

[0039] In some examples, the lidar can be configured as a mid-range lidar sensor. Mid-range lidar sensors strike a balance between long-range and short-range lidar sensors in terms of detection range (e.g., from a few meters to several hundred meters) and field of view (e.g., from 30 degrees to 180 degrees horizontally). Mid-range lidar sensors can be mounted on the front bumper, above the windshield, inside the windshield, on the side panel, or on the rear bumper. Mid-range lidar is suitable for detecting objects on the road or near the vehicle during assisted or autonomous driving processes.

[0040] In some examples, a lidar system with multiple lidar sensors is deployed around the vehicle. These multiple lidar sensors can have different detection ranges and fields of view to cover the area around the vehicle. In some embodiments, the lidar system includes one or more short-range lidar sensors and one or more mid-range lidar sensors. By combining lidar sensors located at different positions on the vehicle, the lidar system can provide a comprehensive view of the environment. Data from these lidar sensors can be processed with data from other sensors, such as cameras and / or millimeter-wave radar, to make real-time decisions for safe and efficient autonomous driving. The combination of lidar sensors with different detection ranges, fields of view, and locations allows for a balance between long-range visibility and near-range object detection. This also benefits aesthetics and helps control overall vehicle costs.

[0041] In some examples, multiple lidar sensors are activated in a lidar system. In some embodiments, multiple lidar sensors are activated or deactivated depending on different scenarios or requirements. For example, when the vehicle is traveling at high speeds (e.g., above 40 mph), one or more short-range lidar sensors may be deactivated, while one or more long-range and / or mid-range lidar sensors may be activated. As another example, when the vehicle is traveling at lower speeds (e.g., below 40 mph), one or more long-range lidar sensors may be deactivated, while one or more short-range and mid-range lidar sensors may be activated. This effectively saves energy and extends the lifespan of the lidar system.

[0042] Figure 1 An example structural block diagram of an exemplary lidar consistent with some embodiments of this disclosure is shown. In some examples, reference is made to... Figure 1 The lidar 100 includes a laser emitting system 110, a laser receiving system 120, and a control and processing system 130. Optionally, the lidar 100 also includes a scanning system 140. The scanning system 140 may include a rotating optomechanical system, a rotating reflector, a reciprocating oscillating mirror or galvanometer (e.g., a MEMS mirror, a Galvo mirror, etc.), and other components that can direct the laser beam to different locations in the environment (e.g., prisms, gratings, phased arrays, etc.).

[0043] Laser emitting system 110 emits a laser. When the laser encounters object 10, it is reflected from the surface of object 10, forming an echo. The echo returns to lidar 100. Laser receiving system 120 receives the reflected echo and converts it into an electrical signal. This electrical signal, after preprocessing, is used to determine echo data, such as the echo reception time or time of flight. The echo data is provided to control and processing system 130. Control and processing system 130 processes the echo data to determine information about object 10, such as its distance, position, or velocity. This process is repeated multiple times to create an accurate, real-time 3D environment map, such as a point cloud. Computers in terminal devices such as vehicles can then perform safe navigation based on the point cloud.

[0044] In some examples, the laser emitting system 110 includes a driving circuit 112, a laser 114, and emitting optics 116. The laser 114 emits laser light under the drive of the driving circuit 112. The laser light exits through the emitting optics 116. In some embodiments, the laser 114 may include a semiconductor laser, such as a vertical-cavity surface-emitting laser (VCSEL), an edge-emitting laser (EEL), or other semiconductor lasers capable of generating laser light. In other embodiments, the laser 114 may also include a solid-state laser or a fiber laser. The laser 114 emits laser light at a wavelength of 905 nm, 940 nm, or 1550 nm; other wavelengths may also be emitted. The driving circuit 112 may include a driver integrated circuit, such as an analog chip or a digital-analog hybrid chip.

[0045] In some examples, the laser receiving system 120 includes receiving optics 122 and a receiver 124. The receiving optics 122 collects echoes reflected from an object and focuses the echoes onto the receiver 124. The receiver 124 uses the photoelectric effect to convert the echoes into electrical signals. In some embodiments, the receiver 124 may include a photon detector, such as an avalanche photodiode (APD), a single-photon avalanche diode (SPAD), or a silicon photomultiplier (SiPM). The lidar 100 may also include a preprocessing circuit 150. Alternatively, the laser receiving system 120 may also include a preprocessing circuit 150. This preprocessing circuit 150 may include digitization circuitry, such as an analog-to-digital converter (ADC), to convert analog signals into digital signals for use with the control and processing system 130. As another example, the preprocessing circuit 150 may include a time-to-digital converter (TDC). The echoes are detected and converted into electrical signals by the receiver 124, and these electrical signals are provided to the TDC. Based on the received electrical signal, the TDC can determine the timing information (e.g., timestamp) of the echo and convert the timing information into a digital signal for the control and processing system 130. The preprocessing circuit 150 may also include analog front-end circuitry for channel selection and analog signal amplification. In some embodiments, the preprocessing circuit 150 may be implemented as a system-on-chip (SOC) or an application-specific integrated circuit (ASIC). Exemplarily, the chip of the preprocessing circuit 150 is packaged together with the chip of the receiver 124.

[0046] For example, the emitting optics 116 may include one or more optical components such as a lens / lens group, a mirror, a filter, a beam splitter, an aperture, and a homogenizer. Similarly, the receiving optics 122 may include one or more optical components such as a lens / lens group, a mirror, a filter, a beam splitter, an aperture, and a homogenizer. The emitting optics 116 and the receiving optics 122 may be independently configured optical components, or they may be fully or partially multiplexed (the multiplexed portion may be referred to as "transmitting and receiving optics"). For example, the emitting optics 116 and the receiving optics 122 may include at least one of a common lens, a common lens group, a common mirror, a common aperture, and a common beam splitter. The laser emitted by the laser and the echo reflected by the object may pass through at least one of the common lens, the common lens group, the common mirror, the common aperture, and the common beam splitter.

[0047] In some examples, the control and processing system 130 may include an information processing circuit 132 and a light source control circuit 134. The information processing circuit 132 processes electrical signals to determine information about the object. For example, the information processing circuit 132 may include circuits implemented using an ASIC or a programmable logic device (PLD), such as a field-programmable gate array (FPGA), a microcontroller unit (MCU), or a digital signal processor (DSP). Alternatively, the information processing circuit 132 may include a central processing unit (CPU). The light source control circuit 134 sends control signals to the excitation source to control the excitation source to drive the laser to emit light, achieving pulsed laser emission. For example, the light source control circuit 134 may send timing signals to control the laser emission timing. Furthermore, the light source control circuit 134 may control one or more of the pulse interval, pulse intensity, and pulse width. Adding pulse coding functionality can enhance the anti-interference capability of the lidar. Optionally, the light source control circuit 134 and the information processing circuit 132 can be integrated together, for example, integrated into a main control chip, or they can each be independent or partially independent chips. In some embodiments, when the lidar 100 includes a scanning system 140, the control and processing system 130 may further include a scanning control circuit 136 for controlling the scanning system. The scanning control circuit 136 can be integrated with one or all of the light source control circuit 134 and the information processing circuit 132. For example, the scanning control circuit 136, the light source control circuit 134, and the information processing circuit 132 can be integrated into a main control chip; or they can each be independent or partially independent chips. In some embodiments, the control and processing system 130 can be implemented in the form of a SOC or an ASIC.

[0048] In some examples, lidar can be installed on terminal devices to transmit the detected sensing data. The terminal devices can then utilize this sensing data to perform one or more functions, such as analysis, decision-making, or control. Examples of terminal devices include vehicles, ships, aircraft (e.g., flying vehicles or drones), and robots (e.g., industrial robots or home robots).

[0049] Figure 2 A structural block diagram of an exemplary vehicle system consistent with some embodiments of this disclosure is shown. In some examples, reference is made to... Figure 2The vehicle system 200 includes a sensor system 202, a perception system 204, a planning system 206, and a control system 208. The vehicle system 200 may have autonomous capabilities, for example, possessing at least one function, characteristic, device, and / or similar device that enables the vehicle to operate partially or fully without human intervention, including but not limited to fully autonomous vehicles (e.g., abandoning vehicles dependent on human intervention), highly autonomous vehicles (e.g., abandoning vehicles dependent on human intervention in certain situations), and / or similar devices. The sensor system 202 includes one or more devices, such as lidar 202a, radar 202b, camera 202c, sonar 202d, global positioning system (GPS) 202e, and inertial measurement unit (IMU) 202f. Lidar 202a may include lidar sensors, such as long-range lidar sensors, mid-range lidar sensors, or short-range lidar sensors. In some embodiments, the sensor system 202 may use one or more devices included in the sensor system 202 to generate environment-related data. The data generated by the sensor system 202 can be used by one or more systems to observe the environment in which the vehicle is located.

[0050] In some examples, the perception system 204 can receive data associated with objects in the environment and classify those objects. In some examples, the perception system 204 can receive image data (e.g., point clouds) associated with objects captured by a lidar system. In such examples, the perception system 204 can classify objects based on groupings of objects (e.g., bicycles, vehicles, traffic signs, pedestrians, and / or similar objects). In some embodiments, the perception system 204 can transmit data related to object classification to the planning system 206.

[0051] In some examples, the planning system 206 may receive destination-related data and generate route or trajectory-related data. A vehicle may then travel along this route or trajectory towards its destination. In some embodiments, the planning system 206 may periodically or continuously receive data from the sensing system 204 and update the route or trajectory based on the data generated by the sensing system 204.

[0052] In some examples, control system 208 can receive trajectory-related data from planning system 206, and control system 208 can control the operation of the vehicle. In some embodiments, control system 208 includes steering control system 208a and powertrain control system 208b. Control system 208 can control the operation of steering control system 208a and powertrain control system 208b according to the received trajectory. In some embodiments, powertrain control system 208b can receive control signals from control system 208 to start, stop, accelerate, decelerate, turn left, turn right, or perform similar operations on the vehicle. Steering control system 208a can receive control signals from control system 208 to turn one or more wheels of the vehicle. In some examples, when the trajectory includes a left turn, control system 208 transmits control signals to cause steering control system 208a to adjust the direction.

[0053] According to some embodiments of this disclosure, a receiver for a lidar is provided. The receiver may include a SPAD chip. The SPAD chip includes a pixel array. The pixel array extends along a first direction (e.g., the X direction) and a second direction (e.g., the Y direction). The second direction is perpendicular to the first direction. The pixel array includes a first pixel array and a second pixel array. The first pixel array includes a first pixel and can receive the echo of the lidar's probe light reflected from an object and output an echo signal. The second pixel array includes a second pixel and can receive visible light and output an image signal. The first and second pixels are aligned in the second direction. The pixel array also includes a virtual pixel. The virtual pixel is located between the first and second pixels in the first direction. This disclosure proposes an integrated architecture for a lidar receiver. The receiver uses a SPAD chip to simultaneously receive the echoes of visible light and lidar, which helps reduce the complexity and system cost of fusing visual information and point cloud information.

[0054] See Figure 3 The diagram shows a schematic block diagram of a lidar receiver 300 consistent with some embodiments of the present disclosure.

[0055] In some embodiments, refer to Figure 3 The receiver 300 includes one or more SPAD chips 30. Each SPAD chip 30 includes a pixel array. A pixel may include one or more detectors. For example, a pixel may include one or more SPADs. When a photon is incident on a pixel, it triggers a SPAD within the pixel. The SPAD generates an electrical signal. The pixel is coupled to a preprocessing circuit. The preprocessing circuit can determine the number of SPADs triggered within the pixel and the triggering time. Based on the number of triggered SPADs and the triggering time, the photon intensity and arrival time can be determined. The lidar can determine object information based on the photon intensity and arrival time.

[0056] The pixel array includes a first pixel array 310. The first pixel array 310 can receive the echo of the detection light from the lidar reflected by an object and output the echo signal. In some embodiments, the first pixel array 310 includes a first pixel 311. The first pixel 311 includes a first number of SPADs.

[0057] The pixel array also includes a second pixel array 320. The second pixel array 320 can receive visible light and output image signals. In some embodiments, the second pixel array 320 includes second pixels 321. The second pixel 321 can include a second number of SPADs. The first number and the second number are both positive integers not less than 1. The first number and the second number can be the same or different. For example, the first number can be less than, equal to, or greater than the second number. In some embodiments, the first number is greater than the second number. When a first pixel 311 can generate one data point and a second pixel 321 can generate one data point, the resolution of the image data generated by the receiver 300 is higher than the resolution of the point cloud data. In some embodiments, the first number is equal to the second number. For example, the SPADs in the pixel array can be independently addressed or gated. A SPAD can be either a first pixel 311 or a second pixel 321.

[0058] For example, along the second direction, the first pixel array 310 includes the same number of SPADs as the second pixel array 320. Along the first direction, the second pixel array includes a greater number of SPADs than the second pixel array 320.

[0059] In some embodiments, the first pixel 311 and the second pixel 321 may be aligned in a second direction (e.g., the Y direction). In this disclosure, "alignment of the first pixel and the second pixel in a second direction" can be characterized as the first pixel and the second pixel corresponding to substantially the same field of view in the second direction. The first pixel 311 is offset from the second pixel 321 in a first direction (e.g., the X direction). In this disclosure, "displacement of the first pixel from the second pixel in a first direction" can be characterized as the first pixel and the second pixel corresponding to different field of view in the first direction at the same time. Thus, at the same time, point cloud data determined based on the first pixel 311 and image data determined based on the second pixel 321 may correspond to different field of view in the first direction and the same field of view in the second direction. In some LiDAR systems, the receiver rotates about an axis in the Y direction (i.e., the second direction), and the motion trajectories of the first pixel 311 and the second pixel 321 may substantially overlap.

[0060] In some embodiments, the first direction may include one of a horizontal direction and a vertical direction. The second direction may include the other of a horizontal direction and a vertical direction.

[0061] The SPAD chip 30 may also include one or more virtual pixels. The one or more virtual pixels may be located between the first pixel 311 and the second pixel 321 in a first direction (e.g., the X direction). Virtual pixels may include one or more detectors. For example, virtual pixels may include one or more SPADs. In this disclosure, a "virtual pixel" refers to a pixel that is not used to output echo signals or image signals. Exemplarily, the virtual pixel includes a SPAD structure. The SPAD of the virtual pixel does not have photosensitive capabilities. For example, the photosensitive surface of the SPAD of the virtual pixel is obscured.

[0062] In some embodiments, one or more virtual pixels include a first virtual pixel 331 and a second virtual pixel 332. The first virtual pixel 331 is adjacent to a first pixel 311. The second virtual pixel 332 is adjacent to a second pixel 321. Along a first direction, the spacing between the first virtual pixel 331 and the first pixel 311 is the same as the spacing between the second virtual pixel 332 and the second pixel 321.

[0063] See Figure 4 The diagram shows a schematic block diagram of a lidar receiver 400 consistent with some embodiments of the present disclosure. Several details of the receiver 400 are similar to those of the receiver 300.

[0064] In some embodiments, refer to Figure 4 The receiver 400 includes one or more SPAD chips 40. Each SPAD chip 40 includes a pixel array. A pixel may include one or more detectors. For example, a pixel may include one or more SPADs.

[0065] The pixel array includes a first pixel array 410. The first pixel array 410 can receive the echo of the detection light from the lidar reflected by an object and output an echo signal. In some embodiments, the first pixel array 410 includes a first pixel 411. The pixel array also includes a second pixel array 420. The second pixel array 420 can receive visible light and output an image signal. In some embodiments, the second pixel array 420 includes a second pixel 421. In some embodiments, the first pixel 411 and the second pixel 421 can be aligned in a second direction (e.g., the Y direction). The first pixel 411 is offset from the second pixel 421 in a first direction (e.g., the X direction).

[0066] In some embodiments, the SPAD chip 40 may include a substrate and a first SPAD array and a second SPAD array formed on the substrate. The first SPAD array may be used to provide a first pixel array 410. The second SPAD array may be used to provide a second pixel array 420. A gap may exist between the first SPAD array and the second SPAD array. For example, the size d of the gap may be larger than the spacing between adjacent SPADs in the SPAD array. In some embodiments, the gap between the first SPAD array and the second SPAD array may at least partially expose the substrate of the SPAD chip 40.

[0067] See Figure 5 The diagram shows a schematic block diagram of a lidar receiver 500 consistent with some embodiments of the present disclosure. Several details of receiver 500 are similar to those of receiver 300 or receiver 400.

[0068] In some embodiments, refer to Figure 5 The receiver 500 includes one or more SPAD chips 50. Each SPAD chip 50 includes a pixel array. Each pixel may include one or more detectors. For example, a pixel may include one or more SPADs.

[0069] The pixel array includes a first pixel array 510. The first pixel array 510 can receive the echo of the detection light from the lidar reflected by an object and output an echo signal. In some embodiments, the first pixel array 510 includes a plurality of first pixels 511. The pixel array also includes a second pixel array 520. The second pixel array 520 can receive visible light and output an image signal. In some embodiments, the second pixel array 520 includes a plurality of second pixels 521. In some embodiments, one of the first pixels 511 and a corresponding second pixel 521 among the plurality of second pixels 521 can be aligned in a second direction (e.g., the Y direction) and offset in a first direction (e.g., the X direction).

[0070] In some embodiments, a lidar may include multiple detection cycles. One detection cycle may correspond to one sampling angle. In some embodiments, a lidar may include a first detection cycle and a second detection cycle. The first detection cycle corresponds to a first sampling angle. The second detection cycle corresponds to a second detection angle. The first sampling angle is different from the second sampling angle. Exemplarily, the sampling angle includes a sampling angle in a first direction.

[0071] See Figure 6The diagram illustrates a LiDAR receiver 500 performing detection at different sampling angles, consistent with some embodiments of this disclosure. For ease of understanding, the receiver 500 at the first sampling angle is labeled 500A, and the receiver 500 at the second sampling angle is labeled 500B.

[0072] During the first detection period, multiple first pixels (e.g., a column of first pixels) 511A in the first pixel array 510A of receiver 500A receive echoes from different field-of-view angles in the second direction (e.g., the Y direction). The lidar generates a first column of point clouds in the second direction based on the received echoes. This first column of point clouds corresponds to a first field-of-view angle 601 in the first direction (e.g., the X direction). During the first detection period, multiple second pixels (e.g., a column of second pixels) 521A in the second pixel array 520A of receiver 500A receive visible light from different field-of-view angles in the second direction. The lidar generates a first column of image points in the second direction based on the received visible light. This first column of image points corresponds to a second field-of-view angle 602 in the first direction.

[0073] During the second detection cycle, multiple first pixels (e.g., a column of first pixels) 511B in the Y-direction of the first pixel array 510B of receiver 500B receive echoes from different field-of-view angles in the second direction. The lidar generates a second column of point clouds in the second direction based on the received echoes. This second column of point clouds corresponds to a third field-of-view angle 603 in the first direction. The third field-of-view angle 603 is different from the first field-of-view angle 601. During the second detection cycle, multiple second pixels (e.g., a column of second pixels) 521B in the second pixel array 520B of receiver 500B receive visible light from different field-of-view angles in the second direction. The lidar generates a second column of image points in the second direction based on the received visible light. This second column of image points corresponds to a fourth field-of-view angle 604 in the first direction. The fourth field-of-view angle 604 is different from the second field-of-view angle 602.

[0074] Figure 6 The first field of view 601, the second field of view 602, the third field of view 603, and the fourth field of view 604 are indicated by dashed lines. It can be understood that the first field of view 601, the second field of view 602, the third field of view 603, and the fourth field of view 604 can have a certain range in the first direction. Figure 6 The dotted line in the diagram can indicate the center of the first field of view 601, the second field of view 602, the third field of view 603, and the fourth field of view 604 in the first direction.

[0075] In some embodiments, see Figure 5In a detection cycle (e.g., a first detection cycle or a second detection cycle), the first pixel 511 in the second direction can be activated in rotation. The second pixel 521 in the second direction can also be activated in rotation. The lidar can determine depth information based on the echo signals of multiple first pixels 511 and determine image information based on the image signals of multiple second pixels 521.

[0076] In this disclosure, "round-robin activation" can be characterized as time-division activating one or more pixels (e.g., first pixel 511 or second pixel 521) along a second direction (e.g., the Y direction) within a detection cycle, causing them to receive echo or visible light and output echo signals or image signals. See also Figure 5 In the second direction, a column of first pixels 511 can be activated sequentially to receive the echo. One first pixel 511 can be activated at a time, or several first pixels 511 can be activated at once. See also... Figure 5 In the second direction, a column of second pixels 521 can be activated sequentially to receive visible light. One second pixel 521 can be activated at a time, or several second pixels 521 can be activated at once.

[0077] In some embodiments, see Figure 6 The first field of view 601 can be substantially equal to the fourth field of view 604. For example, in the first detection cycle, a first column of point clouds in a second direction can be generated based on the echoes received by multiple first pixels 511A in the second direction of the first pixel array 510A of receiver 500A. This first column of point clouds corresponds to the first field of view 601 in the first direction. In the second detection cycle, a second column of image points in a second direction can be generated based on the visible light received by multiple second pixels 521B in the second direction of the second pixel array 520B of receiver 500B. This second column of image points corresponds to the fourth field of view 604 in the first direction. "The first field of view is substantially equal to the fourth field of view" means that the first column of point clouds and the second column of image points can correspond to substantially the same field of view in the first direction (e.g., the X direction). In this way, the lidar can generate fused data corresponding to this field of view (first field of view / fourth field of view) based on the first column of point clouds and the second column of image points, realizing pixel-level data fusion of echo signals and image signals.

[0078] In some embodiments, a first pixel 511A can determine first depth information during a first detection period. A second pixel 521A can determine first image information during the first detection period. A first pixel 511B can determine second depth information during a second detection period. A second pixel 521B can determine second image information during the second detection period. The first and second detection periods do not overlap. In some embodiments, the first depth information and the second image information may correspond to the same field of view in a first direction and the same field of view in a second direction.

[0079] For example, the angle difference in the first direction corresponding to the first detection period and the second detection period is related to the angular resolution of the lidar in the first direction. The time difference between the first detection period and the second detection period is related to the angular resolution of the lidar in the first direction, the field of view, and the frame rate of the lidar.

[0080] In some embodiments, the first pixel array may include a plurality of first pixels staggered in a first direction. In one detection cycle, the plurality of first pixels can provide depth data at different field angles in the first direction. For example, in a first detection cycle, first depth information may include multiple depth data points in the first direction. For example, in a second detection cycle, second depth information may include multiple depth data points in the first direction.

[0081] See Figure 7 The diagram shows a schematic block diagram of a lidar receiver 700 consistent with some embodiments of the present disclosure. Several details of the receiver 700 are similar to those of receiver 300, receiver 400, or receiver 500.

[0082] In some embodiments, refer to Figure 7 The receiver 700 includes one or more SPAD chips 70. Each SPAD chip 70 includes a pixel array. Each pixel may include one or more detectors. For example, a pixel may include one or more SPADs.

[0083] The pixel array includes a first pixel array 710. The first pixel array 710 can receive the echo of the detection light from the lidar reflected by an object and output an echo signal. In some embodiments, the first pixel array 710 includes a plurality of first pixels 711. The pixel array also includes a second pixel array 720. The second pixel array 720 can receive visible light and output an image signal. In some embodiments, the second pixel array 720 includes a plurality of second pixels 721. In some embodiments, one of the first pixels 711 and a corresponding second pixel 721 among the plurality of second pixels 721 may be aligned in a second direction (e.g., the Y direction) and offset in a first direction (e.g., the X direction).

[0084] In some embodiments, the first pixel array 710 includes a plurality of first pixels 711 staggered in a first direction. In one detection cycle, the plurality of first pixels 711 can provide depth data for different field angles in the first direction. See, for example... Figure 7The first pixel array 710 may include a first column of first pixels 711 and a second column of first pixels 711. The first column of first pixels 711 may include a plurality of first pixels 711 distributed along a second direction. The second column of first pixels 711 may also include a plurality of first pixels 711 distributed along the second direction. At the same time, the first column of first pixels 711 may correspond to a first field of view 701 in the first direction, and the second column of first pixels 711 may correspond to a fifth field of view 705 in the first direction. The first field of view 701 is different from the fifth field of view 705. Thus, in one detection cycle, the depth information may include multiple depth data in the first direction.

[0085] In some embodiments, the first pixel array 710 may be arranged in a two-dimensional matrix. Each first pixel 711 in this two-dimensional matrix can be addressed individually. Each first pixel can be independently controlled and its signals acquired. For example, the first pixel array 710 may include M×N first pixels 711. M is a positive integer that represents the number of first pixels 711 included in the first pixel array 710 in the Y direction. N is a positive integer that represents the number of first pixels 711 included in the first pixel array 710 in the X direction.

[0086] It should be noted that, although Figure 7 The illustration shows that a plurality of first pixels in the first column of first pixels 711 and a plurality of first pixels in the second column of first pixels 711 are offset from each other in a second direction, but the present disclosure is not limited thereto. In some embodiments, at least one first pixel in the first column of first pixels 711 and at least one first pixel in the second column of first pixels 711 may have overlapping areas in the second direction, or they may be aligned in the second direction.

[0087] In some embodiments, the second pixel array 720 may include multiple columns of second pixels 721. The multiple columns of second pixels 721 are arranged along the X direction. In one detection cycle, the image information determined by the multiple columns of second pixels 721 corresponds to different field of view angles. For example, in a first detection cycle, the image information determined by the first column of second pixels may correspond to field of view angle A, and the image information determined by the second column of second pixels may correspond to field of view angle B. In a second detection cycle, the image information determined by the second column of second pixels may correspond to field of view angle A. The image information determined by the first column of second pixels in the first detection cycle and the image information determined by the second column of second pixels in the second detection cycle can be superimposed to generate image data with field of view angle A. This can effectively extend the exposure time and improve the quality of the obtained image data, such as increasing the signal-to-noise ratio or reducing noise.

[0088] It should be noted that, Figure 3-7The illustration shows a first pixel comprising 2×2 SPADs and a second pixel comprising 2×2 SPADs for illustrative purposes only. A first pixel may include more or fewer SPADs. A second pixel may also include more or fewer SPADs. For example, the first pixel and the second pixel may include different numbers of SPADs. A first pixel may include a 3×3 SPAD subarray. A second pixel may include one SPAD. Alternatively, the first pixel and the second pixel may include the same number of SPADs. Exemplarily, the SPADs in the pixel array can be independently addressed or gated. A first pixel may include one SPAD. A second pixel may include one SPAD. A LiDAR can superimpose the echo data from multiple first pixels into a single detection data set. This can improve the signal-to-noise ratio of the detection.

[0089] In some embodiments, the second pixel may include one or more sub-pixels. A sub-pixel may include one or more SPADs.

[0090] The SPAD of the sub-pixels of the first and second pixels can be configured with filters. The filters may include filters for lidar echoes (e.g., infrared narrowband filters) and filters for color image data (e.g., visible light filters). In some embodiments, the filters may be attached as separate devices to the SPAD of the sub-pixels of the first and second pixels. The pixel array includes dummy pixels. The dummy pixels can provide physical isolation between adjacent lidar echo filters and color image data filters to avoid edge interference. The dummy pixels may be located between the first and second pixels in a first direction (e.g., the X-direction). In other embodiments, the filters may be integrated onto the SPAD chip. In this case, the pixel array may not include dummy pixels.

[0091] In some embodiments, at least one second pixel in the second pixel array may include a first sub-pixel. The first sub-pixel may include a first SPAD and a first filter disposed on one side of the first SPAD. The first filter may allow light of a first wavelength range to pass through and be incident on the first SPAD. In some embodiments, the first wavelength range may be, for example, between 585 nm and 720 nm. For example, the first sub-pixel may receive red light. For example, the first filter may be disposed in the receiving optical path of the first SPAD that receives red light and allow red light to pass through.

[0092] In some embodiments, at least one second pixel in the second pixel array may include a second sub-pixel. The second sub-pixel may include a second SPAD and a second filter disposed on one side of the second SPAD. The second filter may allow light of a second wavelength range to pass through and be incident on the second SPAD. In some embodiments, the second wavelength range may be, for example, between 490 nm and 580 nm. For example, the second sub-pixel may receive green light. For example, the second filter may be disposed in the receiving optical path of the second SPAD that receives green light and allow the green light to pass through.

[0093] In some embodiments, at least one second pixel in the second pixel array may include a third sub-pixel. The third sub-pixel may include a third SPAD and a third filter disposed on one side of the third SPAD. The third filter may allow light of a third wavelength range to pass through and be incident on the third SPAD. In some embodiments, the third wavelength range may be, for example, between 430 nm and 485 nm. For example, the third sub-pixel may receive blue light. For example, the third filter may be disposed in the receiving optical path of the third SPAD that receives blue light and allow blue light to pass through.

[0094] In some embodiments, the receiver may further include a fourth filter. The fourth filter may be disposed in the receiving optical path of the first pixel array. The fourth filter allows light in a fourth wavelength range to pass through and be incident on the first pixel array. The fourth wavelength range may include the wavelength of the echo from the lidar. The wavelength of the echo light may include 905 nm, 940 nm, or 1550 nm. This facilitates the first pixel array receiving the echo without receiving unwanted light (e.g., ambient light of other wavelengths). The lidar environment may contain a large amount of ambient light (e.g., from the sun, vehicle headlights, streetlights, etc.), which may enter the first pixel array and cause signal interference, thereby reducing ranging accuracy. Distributing a fourth filter in the receiving optical path of the first pixel array can effectively filter out light signals in non-target wavelength bands, such as visible light. The fourth filter allows the echo of the target laser wavelength band to pass through and enter the first pixel array. This can improve the signal-to-noise ratio (SNR) of the first pixel array, reduce ambient light interference, and improve the ranging stability and accuracy of the lidar in strong light or complex lighting environments.

[0095] In some embodiments, the fourth filter may be a narrowband filter. The transmission bandwidth of the fourth filter is matched to the emission wavelength of the lidar. The emission wavelength of the lidar can be determined based on the center wavelength and temperature drift range of the laser. For example, when the emission wavelength of the lidar is 905 nm, the transmission range of the fourth filter can be limited to 895 nm to 915 nm to ensure that the echo signal enters the first pixel array. In some embodiments, the fourth filter may be integrated with receiving optics (e.g., lenses, mirrors).

[0096] In some embodiments, at least one second pixel in the second pixel array may include a fourth sub-pixel. The fourth sub-pixel may include a fourth SPAD. The fourth SPAD may receive light in a fourth wavelength range. In some embodiments, the fourth wavelength range includes the wavelength range of visible light. The fourth wavelength range may be, for example, between 380 nm and 750 nm. The fourth sub-pixel can be used to increase luminous flux, which is beneficial for imaging performance in low light. In some embodiments, the fourth sub-pixel may not include a filter. In some embodiments, the fourth sub-pixel may include a fifth filter disposed on one side of the fourth SPAD. The fifth filter may allow light in the fourth wavelength range to pass through and be incident on the fourth SPAD. For example, the fourth sub-pixel may receive white light. For example, the fifth filter may be disposed in the receiving optical path of the fourth SPAD that receives white light and allow white light to pass through.

[0097] See Figure 8 This illustrates several examples of a second pixel consistent with some embodiments of this disclosure. In some embodiments, see [link to relevant documentation]. Figure 8 In Example A, the second pixel array 810 may include a plurality of sub-pixels extending along a first direction. The plurality of sub-pixels may include a first sub-pixel 812, a second sub-pixel 814, and a third sub-pixel 816. In this example, pixel 811 includes the first sub-pixel 812, the second sub-pixel 814, and the third sub-pixel 816. The first sub-pixel 812, the second sub-pixel 814, and the third sub-pixel 816 may correspond to the same field of view in a second direction. Thus, pixel 811 may contain three monochromatic light pixels (e.g., a red light pixel, a green light pixel, and a blue light pixel). In some embodiments, by combining these three monochromatic light pixels, pixel 811 can generate color image data points in the second direction consistent with the number of sub-pixels, without interpolation to increase the data volume or resolution.

[0098] For example, the first pixel and the second pixel are aligned in the second direction. The number of color image data points generated by the second pixel array is the same as the number of second pixels in the second direction. The lidar can superimpose data from multiple first pixels in the second direction to generate a single detection data point. The resolution of the point cloud formed by the lidar in the second direction can be lower than the resolution of the color image.

[0099] In some embodiments, pixel 811 may also use an interpolation algorithm based on the sub-pixels of adjacent pixels to generate corresponding color image pixels.

[0100] In some embodiments, see Figure 8 In Example B, the second pixel array 820 may include a 2×2 array of subpixels. For example, the first row of subpixels includes a first subpixel 822 and a second subpixel 824, corresponding to a second directional field of view Y1. The second row of subpixels includes a second subpixel 824 and a third subpixel 826, corresponding to a second directional field of view Y2. In this example, the second pixel array 820 may include pixels 821A and 821B. Pixel 821A may include a first subpixel 822 and a second subpixel 824 corresponding to the second directional field of view Y1. Pixel 821B may include a second subpixel 824 and a third subpixel 826 corresponding to the second directional field of view Y2. Pixel 821A contains two monochromatic light pixels, and color image data corresponding to the second directional field of view Y1 can be obtained by interpolation based on a third monochromatic light pixel in a neighboring pixel. For example, color image data corresponding to the second direction field of view Y1 can be obtained based on the first sub-pixel 822 and the second sub-pixel 824 in pixel 821A and the third sub-pixel 826 in pixel 821B. Pixel 821B contains two monochromatic light pixels, and color image data corresponding to the second direction field of view Y2 can be obtained by interpolation based on the third monochromatic light pixel in the neighboring pixels. For example, color image data corresponding to the second direction field of view Y2 can be obtained based on the first sub-pixel 822 in pixel 821A and the second sub-pixel 824 and the third sub-pixel 826 in pixel 821B.

[0101] In some embodiments, see Figure 8 In Example C, the second pixel array 830 may include a first sub-pixel 832, a second sub-pixel 834, a third sub-pixel 836, and a first sub-pixel 838 extending along a second direction. The first sub-pixel 832 may correspond to a second-direction field of view Y1. The second sub-pixel 834 may correspond to a second-direction field of view Y2. The third sub-pixel 836 may correspond to a second-direction field of view Y3. The first sub-pixel 838 may correspond to a second-direction field of view Y4. The second pixel array 830 may include pixels 831A, 831B, 831C, and 831D. Pixel 831A may include a first sub-pixel 832 corresponding to a second-direction field of view Y1. Pixel 831B may include a second sub-pixel 834 corresponding to a second-direction field of view Y2. Pixel 831C may include a third sub-pixel 836 corresponding to a second-direction field of view Y3. Pixel 831D may include a first sub-pixel 838 corresponding to a second-direction field of view Y4.

[0102] For example, pixel 831A contains a monochromatic light pixel, and color image data corresponding to the second direction field of view Y1 can be obtained by interpolation based on two other monochromatic light pixels among its neighboring pixels. For instance, color image data corresponding to the second direction field of view Y1 can be obtained based on the first sub-pixel 832 in pixel 831A, the second sub-pixel 834 in pixel 831B, and the third sub-pixel 836 in pixel 831C.

[0103] For example, pixel 831B contains a monochromatic light pixel, and color image data corresponding to the second-direction field of view Y2 can be obtained by interpolation based on two other monochromatic light pixels among its neighboring pixels. For instance, color image data corresponding to the second-direction field of view Y2 can be obtained based on the first sub-pixel 832 in pixel 831A, the second sub-pixel 834 in pixel 831B, and the third sub-pixel 836 in pixel 831C. Alternatively, color image data corresponding to the second-direction field of view Y2 can be obtained based on the first sub-pixel 838 in pixel 831D, the second sub-pixel 834 in pixel 831B, and the third sub-pixel 836 in pixel 831C.

[0104] For example, pixel 831C contains a monochromatic light pixel, and color image data corresponding to the second-direction field of view Y3 can be obtained by interpolation based on two other monochromatic light pixels among its neighboring pixels. For instance, color image data corresponding to the second-direction field of view Y3 can be obtained based on the first sub-pixel 832 in pixel 831A, the second sub-pixel 834 in pixel 831B, and the third sub-pixel 836 in pixel 831C. Alternatively, color image data corresponding to the second-direction field of view Y3 can be obtained based on the second sub-pixel 834 in pixel 831B, the third sub-pixel 836 in pixel 831C, and the first sub-pixel 838 in pixel 831D.

[0105] For example, pixel 831D contains a monochromatic light pixel, and color image data corresponding to the second direction field of view Y4 can be obtained by interpolation based on two other monochromatic light pixels among its neighboring pixels. For instance, color image data corresponding to the second direction field of view Y4 can be obtained based on the second sub-pixel 834 in pixel 831B, the third sub-pixel 836 in pixel 831C, and the first sub-pixel 838 in pixel 831D.

[0106] It should be noted that this disclosure... Figure 8 The examples shown are illustrative and not limiting. In cases where a pixel corresponding to a certain second-direction field of view Yi does not contain three monochromatic light pixels, the pixel can be interpolated based on the data of its neighboring sub-pixels to obtain color image data corresponding to that second-direction field of view Yi.

[0107] In some embodiments, see Figure 8 In Example D, pixel array 840 may include multiple rows of subpixels. A row of subpixels may include one or more subpixels and a fourth subpixel 848. The one or more subpixels may include some or all of the first subpixel 842, the second subpixel 844, and the third subpixel 846. The first subpixel 842 may be used to output a monochromatic light intensity signal. The fourth subpixel 848 may be used to output a white light intensity signal. In some embodiments, in low-light environments, the exposure time of the RGB subpixels may be insufficient to obtain a sufficient monochromatic light intensity signal. A grayscale image can be generated using the white light intensity signal received by the fourth subpixel 848. In some embodiments, the white light intensity signal received by the fourth subpixel 848 may be combined or compensated with the monochromatic light intensity signals output by the first subpixel 842, the second subpixel 844, and the third subpixel 846. For example, the monochromatic light compensation intensity of each color in the white light intensity signal can be determined based on the white light intensity signal, and the monochromatic light compensation intensity can be superimposed on the monochromatic light intensity signal to improve the imaging quality in low-light environments.

[0108] See Figure 9 The diagram shows a schematic block diagram of a lidar 900 that is consistent with some embodiments of the present disclosure.

[0109] In some embodiments, refer to Figure 9 The lidar 900 includes a transmitter 910, a receiver 920, and a scanner 930.

[0110] Emitter 910 can emit probe light 911. Emitter 910 may include a laser that emits the probe light. In some embodiments, emitter 910 may include a laser emitting circuit or various types of lasers, including but not limited to VCSELs, EELs, etc. In some embodiments, emitter 910 may include one or more laser arrays. The laser array may be a two-dimensional array or a one-dimensional array. The individual lasers in the laser array may emit light in a time-division manner (e.g., sequentially) or simultaneously.

[0111] The scanner 930 can oscillate or rotate about its axis, changing the transmission direction of the probe light 911, the echo 912, and the visible light 913 in the first direction. The scanning range of the scanner 930 in the first direction corresponds to the field of view (FOV) of the lidar 900 in the first direction.

[0112] Receiver 920 can receive the echo 912 generated after the probe light 911 is reflected by object 10. Receiver 920 may include a detector. Receiver 920 may employ a photodetector circuit or various types of detectors, including but not limited to SPAD, APD, SiPM, etc. In some embodiments, receiver 920 may be any one of receiver 300, receiver 400, receiver 500 or receiver 700 described above.

[0113] The receiver may include one or more SPAD chips. The SPAD chip may include a pixel array. The pixel array may include a first pixel array and a second pixel array. The first pixel array can receive the echo 912 reflected from the object 10 by the lidar's detection light 911 and output an echo signal. The second pixel array can receive visible light 913 and output an image signal.

[0114] In some embodiments, the optical paths of the probe light 911, the echo 912, and the visible light 913 may at least partially overlap.

[0115] In some embodiments, the transmitter 910 may include one or more light-emitting areas. The echo generated by the detection light emitted by the light-emitting areas after reflection by the object 10 can form a light spot on the receiver 920. The light spot can be received by one or more first pixels to detect information from the echo. The detection light emitted by all the light-emitting areas of the transmitter 910 enables the lidar 900 to perform FOV scanning in a second direction. For example, the detection light emitted by multiple light-emitting areas can be staggered in the second direction to be emitted simultaneously or time-divisionally to different angular regions distributed along the second direction.

[0116] In some embodiments, the scanner 930 can change the direction of the detection light in a first direction with a preset step size, deflecting the detection light to different angles in the first direction. In some embodiments, the scanner 930 can guide echoes and visible light at different angles in the first direction to the receiver in the first direction with a preset step size. In this disclosure, "preset step size" can represent the amount of change in the angle by which the scanner 930 deflects the detection light in the first direction after a preset time period. The preset time period can include the detection window of the receiver 920 (i.e., the time period during which multiple first pixels can sense the detection light). For example, when the laser performs a first scan, the scanner deflects the detection light emitted by the laser to a first angle in the first direction. When the laser performs an adjacent next scan (e.g., a second scan), the scanner deflects the detection light emitted by the laser to a second angle in the first direction. The preset step size can represent the angular interval between the first angle and the second angle. For example, when the laser performs a first scan, the scanner deflects the detection light emitted by the laser to 0° in the horizontal direction. During the second adjacent scan, the scanner deflects the probe light emitted by the laser to a horizontal angle of 0.1°. During the third adjacent scan, the scanner deflects the probe light to a horizontal angle of 0.2°. The preset step size can be 0.1°. It is understood that in two adjacent scans, the angle between the two probe beams in the horizontal direction is twice the angle the scanner rotates in the horizontal direction. From the scanner's perspective, the preset step size can also be expressed as 0.05°. For example, within one preset step size, multiple lasers and multiple first pixels can complete one detection process of the lidar 900 at a first field of view in the first direction for a second direction FOV. Multiple second pixels can complete one detection process of the lidar 900 at a second field of view in the first direction for a second direction FOV. For example, during the process of the scanner deflecting the probe light to a first angle and then to a second angle, multiple lasers in the transmitter can emit probe light sequentially or in parallel, and multiple first pixels in the receiver can receive the echo sequentially or in parallel, so as to complete a detection process of the second direction field of view.

[0117] In some embodiments, the lidar 900 may include multiple detection processes. These multiple detection processes may include adjacent first and second detection processes. In the first detection process, the scanner 930 deflects detection light to a first directional field of view A1. A column of first pixels in the receiver 920 receives echoes from different field of view angles in a second direction. The lidar generates a first column of point clouds in the second direction based on the received echoes. This first column of point clouds corresponds to the first directional field of view A1. A column of second pixels in the receiver 920 receives visible light from different field of view angles in the second direction. The lidar generates a first column of image points in the second direction based on the received visible light. This first column of image points corresponds to the first directional field of view B1. During the second detection process, the scanner 930 changes the direction of the detection light by a preset step size to deflect the detection light to the first direction field of view A2; a column of first pixels of the receiver 920 receives echoes from different field of view angles in the second direction, and the lidar generates a second column of point cloud in the second direction based on the received echoes, which corresponds to the first direction field of view A2; a column of second pixels of the receiver 920 receives visible light from different field of view angles in the second direction, and the lidar generates a second column of image points in the second direction based on the received visible light, which corresponds to the first direction field of view B2.

[0118] In some examples, the preset step size can be between 0.05° and 1°, for example, it can be 0.05°, 0.08°, 0.1°, 0.15°, 0.2°, 0.3°, 0.4°, 0.5°, 0.6°, 0.7°, 0.8°, 0.9°, or 1.0°. In some LiDAR systems, the "preset step size" corresponds to the angular resolution of the LiDAR in the first direction. The preset step size can represent the minimum angular spacing between adjacent points in the first direction generated by the LiDAR. For example, if the preset step size is 0.2°, the angular spacing between adjacent points in the LiDAR point cloud in the first direction is 0.2°.

[0119] See Figure 10 The diagram shows a lidar 900 performing detection at different sampling angles, consistent with some embodiments of the present disclosure.

[0120] The lidar 900 may include multiple detection cycles described above. One detection cycle may correspond to one sampling angle. In some embodiments, the lidar may include a first detection cycle and a second detection cycle. The first detection cycle corresponds to a first sampling angle, and the second detection cycle corresponds to a second detection angle. The first sampling angle is different from the second sampling angle. For ease of understanding, the receiver 920 at the first sampling angle is labeled 920A, and the receiver 920 at the second sampling angle is labeled 920B.

[0121] In some embodiments, the receiver 920 may include a first pixel array 91A and a second pixel array 92A. The first pixel array 91A may include a first pixel 911A for receiving the echo signal generated after the probe light is reflected by an object, for determining depth information. The second pixel array 92A may include a second pixel 921A for receiving visible light and outputting an image signal, for determining image information.

[0122] During the first detection period, multiple first pixels 911A in the first pixel array 91A of receiver 920A receive echoes from different field-of-view angles in the second direction. The lidar generates a first column of point clouds in the second direction based on the received echoes. This first column of point clouds corresponds to a field-of-view angle 901 in the first direction. During the first detection period, multiple second pixels 921A in the second pixel array 92A of receiver 920A receive visible light from different field-of-view angles in the second direction. The lidar generates a first column of image points in the second direction based on the received visible light. This first column of image points corresponds to a field-of-view angle 902 in the first direction.

[0123] During the second detection cycle, multiple first pixels 911B in the first pixel array 91B of receiver 920B receive echoes from different field-of-view angles in the second direction. The lidar generates a second column of point clouds in the second direction based on the received echoes. This second column of point clouds corresponds to a field-of-view angle 903 in the first direction. Field-of-view angle 903 is different from field-of-view angle 901. During the second detection cycle, multiple second pixels 921B in the second pixel array 92B of receiver 920B receive visible light from different field-of-view angles in the second direction. The lidar generates a second column of image points in the second direction based on the received visible light. This second column of image points corresponds to a field-of-view angle 901 in the first direction.

[0124] In some embodiments, n scan cycles can be performed between the first detection cycle and the second detection cycle. During the first detection cycle, multiple first pixels 911A in the second direction can receive the echo from the field of view 901 in the first direction. During the second detection cycle, the probe light, echo, and visible light are oriented by the scanner 930 relative to the first detection cycle by n preset steps. Consequently, multiple second pixels 921B in the second direction can receive the visible light from the field of view 901 in the first direction. Thus, both the first column of point cloud and the second column of image points correspond to the field of view 901 in the first direction. In this way, the lidar 900 can generate fused data corresponding to the field of view 901 based on the first column of point cloud and the second column of image points, achieving pixel-level data fusion of the echo signal and the image signal.

[0125] In some embodiments, n can be determined by the following formula:

[0126] in, The preset step size for the lidar (corresponding to the angular resolution of the point cloud generated by the first pixel 911A in the first direction). This is the offset of the field of view angle between the first pixel 911A and the second pixel 921B in the first direction (i.e., the angular difference between field of view angles 901 and 902). For example, when the preset step size... The offset is 0.1°. When the angle is 1°, then n=10. After the first pixel 911A acquires depth data in the first scan, the image data acquired by the second pixel 921B in the tenth scan can be fused with it.

[0127] In some embodiments, the timing of the first pixel array receiving the echo of the probe light from the lidar reflected by the object and outputting the echo signal can be synchronized or asynchronous with the timing of the second pixel array receiving visible light and outputting the image signal.

[0128] In some embodiments, the first pixel array and the second pixel array can complete one detection process in the same step size. Thus, in the first direction, the point cloud resolution generated by the first pixel of the lidar can be equal to the image resolution generated by the second pixel.

[0129] In some embodiments, the first pixel array can complete one detection within one preset step, and the second pixel array can complete one detection within multiple (e.g., two) preset steps. Thus, in the first direction, the point cloud resolution generated by the first pixel of the lidar can be smaller than the image resolution generated by the second pixel. For example, the point cloud resolution generated by the first pixel of the lidar can be twice the image resolution generated by the second pixel.

[0130] In some embodiments, different monochromatic light pixels (e.g., a first sub-pixel, a second sub-pixel, and a third sub-pixel) in the second pixel array can have the same or different timing sequences. For example, the first sub-pixel, the second sub-pixel, and the third sub-pixel can have the same integration duration to simplify timing control. Alternatively, the integration durations of the first sub-pixel, the second sub-pixel, and the third sub-pixel can be different from each other. For example, the integration duration of the first sub-pixel can be shorter than the integration duration of the second sub-pixel. For example, the integration duration of the second sub-pixel can be shorter than the integration duration of the third sub-pixel.

[0131] According to another exemplary embodiment of this disclosure, a terminal device is also provided. The terminal device may include the lidar described above.

[0132] In some embodiments, the lidar can be installed on a terminal device to transmit the detected sensing data to the terminal device. The terminal device then uses the sensing data to perform one or more functions such as analysis, decision-making, or control. Terminal devices may include, for example, vehicles, ships, aircraft (e.g., flying vehicles or drones), and robots (e.g., industrial robots or home robots).

[0133] This concludes the description of the lidar and terminal device according to the present disclosure. The receiver proposed in this disclosure facilitates pixel-level data fusion of echo signals and image signals.

[0134] It should be understood that the above description is illustrative and not restrictive. For example, the above embodiments (and / or aspects thereof) can be used in combination with each other. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of the various embodiments of this disclosure without departing from the scope of this disclosure. While the dimensions and types of materials described herein are used to define parameters of the various embodiments of this disclosure, the embodiments are not intended to be restrictive but are exemplary. Many other embodiments will become apparent to those skilled in the art upon reading the above description. Therefore, the scope of the various embodiments of this disclosure should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A receiver for a lidar system, characterized in that, The receiver includes: A SPAD chip, the SPAD chip including a pixel array extending along a first direction and a second direction, the pixel array including: A first pixel array, including a first pixel and configured to receive the echo of the probe light from the lidar reflected from an object and output an echo signal; and The second pixel array includes a second pixel and is configured to receive visible light and output an image signal; Wherein, the first pixel and the second pixel are aligned in the second direction. The pixel array further includes a virtual pixel located between the first pixel and the second pixel in the first direction.

2. The receiver as described in claim 1, characterized in that, The virtual pixel includes a first virtual pixel and a second virtual pixel. The first virtual pixel is adjacent to the first pixel, and the second virtual pixel is adjacent to the second pixel. Along the first direction, the interval between the first virtual pixel and the first pixel is the same as the interval between the second virtual pixel and the second pixel.

3. The receiver as described in claim 1, characterized in that, The lidar includes multiple detection cycles. In one detection cycle, the first pixel in the second direction is activated in a round-robin fashion, and the second pixel in the second direction is activated in a round-robin fashion. The lidar is configured to determine depth information based on the echo signals of the multiple first pixels and to determine image information based on the image signals of the multiple second pixels.

4. The receiver as described in claim 3, characterized in that, The plurality of detection cycles include a first detection cycle and a second detection cycle. The first pixel determines depth information in the first detection cycle, and the second pixel determines image information in the second detection cycle. The first detection cycle and the second detection cycle do not overlap.

5. The receiver as described in claim 4, characterized in that, During the first detection period, the depth information includes multiple depth data in the first direction.

6. The receiver as claimed in claim 1, characterized in that, The second pixel includes at least one of the following: The first sub-pixel includes a first SPAD and a first filter disposed on one side of the first SPAD, the first filter being configured to allow light of a first wavelength range to pass through and be incident on the first SPAD; The second sub-pixel includes a second SPAD and a second filter disposed on one side of the second SPAD, the second filter being configured to allow light in a second wavelength range to pass through and be incident on the second SPAD; The third sub-pixel includes a third SPAD and a third filter disposed on one side of the third SPAD, the third filter being configured to allow light in a third wavelength range to pass through and be incident on the third SPAD; and The fourth sub-pixel includes a fourth SPAD, which is configured to receive light in a fourth wavelength range.

7. The receiver as claimed in claim 6, characterized in that, The second pixel array includes the first sub-pixel, the second sub-pixel, and the third sub-pixel in the first direction.

8. The receiver as claimed in claim 6, characterized in that, The second pixel array includes the first sub-pixel, the second sub-pixel, the third sub-pixel, and the fourth sub-pixel in the first direction.

9. The receiver as claimed in claim 1, characterized in that, Also includes: A fourth filter is disposed in the receiving optical path of the first pixel array and configured to allow the echo to pass through and be incident on the first pixel array.

10. A lidar, characterized in that, The lidar includes: The transmitter is configured to emit a probe beam; The receiver as claimed in any one of claims 1-9; and The controller is configured to determine depth data based on the echo signal and to determine image data based on the image signal.

11. The lidar as described in claim 10, characterized in that, It also includes a scanner configured to oscillate or rotate about an axis, changing the transmission direction of the probe light, the echo, and the visible light in the first direction.

12. The lidar as described in claim 10, characterized in that, The optical paths of the probe light, the echo, and the visible light overlap at least partially.

13. A terminal device, characterized in that, Including the lidar as described in any one of claims 10-12.

Images