Lidar system for a vehicle including multiplexed point clouds

Abstract

An example light detection and ranging (Lidar) system for a vehicle includes at least one emitter configured to emit Lidar signals at a plurality of frequencies, an omnidirectional transmittable Lidar configured to emit signals via the at least one emitter, and a plurality of receivers configured to receive the reflected Lidar signals from the environment surrounding the vehicle. Each of the plurality of receivers is configured to receive reflected Lidar signals at a different frequency. A vehicle control module is configured to combine frequency data from the reflected Lidar signals received at the plurality of receivers into a point cloud array, generate a multiplexed point cloud from the point cloud array, provide the multiplexed point cloud to a perception stack configured to process three-dimensional images, and automatically control steering, acceleration, and braking of the vehicle from the multiplexed point cloud provided to the perception stack.

Description

[0001] introduce

[0002] The information provided in this section is for the purpose of generally presenting the context of this disclosure. The work of the currently named inventors, within the scope described in this section, and in aspects of this specification that might not otherwise conform to the prior art at the time of submission, is neither expressly nor implicitly acknowledged as prior art to this disclosure.

[0003] This disclosure relates to LiDAR systems for vehicles, including systems that use multiplexed point clouds.

[0004] Some vehicles use LiDAR systems to generate images of their surroundings for automated vehicle control, such as autonomous driving. LiDAR systems determine distances by aiming a laser at an object or surface and measuring the time it takes for the reflected light to return to the receiver. Summary of the Invention

[0005] An example optical detection and ranging (Lidar) system for a vehicle includes: at least one transmitter configured to transmit Lidar signals at multiple frequencies, wherein the at least one transmitter is coupled to the vehicle; an omnidirectional Lidar transmitter configured to transmit signals via the at least one transmitter; a plurality of receivers configured to receive Lidar signals reflected from the vehicle's surrounding environment, wherein each of the plurality of receivers is configured to receive reflected Lidar signals at different frequencies; and a vehicle control module configured to: combine frequency data from the reflected Lidar signals received at the plurality of receivers into a point cloud array; generate a multiplexed point cloud based on the point cloud array; provide the multiplexed point cloud to a perception stack configured to process three-dimensional images; and automatically control the vehicle's steering, acceleration, and braking based on the multiplexed point cloud provided to the perception stack.

[0006] In some examples, each of the plurality of receivers is coupled to the vehicle at a different location on the vehicle. In some examples, the vehicle control module is configured to index the frequency and scan angle of the reflected LiDAR signal received from each of the plurality of receivers.

[0007] In some examples, the vehicle control module is configured to embed the indexed frequency and scan angle into the point cloud. In some examples, the vehicle control module is configured to: generate an array including the point cloud scan angle based on the scan angle of the indexed point cloud; and iterate the point cloud scan angle to combine the frequency data into the point cloud array.

[0008] In some examples, the vehicle control module is configured to: detect at least one object in the point cloud array; index the at least one object across multiple scan angles of the point cloud array; and track the position of the object across the multiple scan angles of the point cloud array.

[0009] In some examples, the vehicle control module is configured to: measure the point density of the point cloud array at the current index; and compare the point density with a specified point density threshold.

[0010] In some examples, the vehicle control module is configured to provide the multiplexed point cloud to the perception stack in response to the point density being greater than the specified point density threshold.

[0011] In some examples, the vehicle control module is configured to: obtain the next scan angle in the point cloud array in response to the point density being less than the specified point density threshold, and update the multiplexed point cloud according to the next scan angle in the point cloud array.

[0012] In some examples, updating the reused point cloud includes removing duplicate objects from the reused point cloud. In some examples, updating the reused point cloud includes: normalizing the point density using a specified consistent resolution; and removing duplicate points in the reused point cloud resulting from point cloud reuse.

[0013] In some examples, the vehicle control module is configured to track the positions of objects in the multiplexed point cloud to determine the final object destination. In some examples, the multiplexed point cloud does not include any blind spots, the point density of which is below a specified point density threshold.

[0014] An example method for operating a light detection and ranging (Lidar) system for a vehicle includes: transmitting Lidar signals at multiple frequencies via at least one Lidar transmitter coupled to the vehicle, and positioning a parabolic reflector to reflect the Lidar signals emitted by the at least one Lidar transmitter; receiving Lidar signals reflected from the vehicle's surrounding environment via a plurality of Lidar receivers, wherein each of the plurality of Lidar receivers receives reflected Lidar signals at different frequencies; combining frequency data from the reflected Lidar signals received at the plurality of Lidar receivers into a point cloud array; generating a multiplexed point cloud based on the point cloud array; providing the multiplexed point cloud to a perception stack configured to process three-dimensional images; and automatically controlling the vehicle's steering, acceleration, and braking based on the multiplexed point cloud provided to the perception stack.

[0015] In some examples, each of the plurality of Lidar receivers is coupled to the vehicle at a different location on the vehicle. In some examples, the method includes indexing the frequency and scan angle of reflected Lidar signals received from each of the plurality of Lidar receivers.

[0016] In some examples, the method includes embedding the frequency and scan angle of the index into the point cloud. In some examples, the method includes: generating an array including the scan angles of the point cloud based on the scan angles of the indexed point cloud; and iterating over the scan angles of the point cloud to combine the frequency data into the point cloud array.

[0017] In some examples, the method includes: detecting at least one object in the point cloud array; indexing the at least one object across a plurality of scan angles of the point cloud array; and tracking the position of the object across the plurality of scan angles of the point cloud array.

[0018] An example optical detection and ranging (Lidar) system for a vehicle includes: at least one transmitter configured to transmit Lidar signals at multiple frequencies, wherein the at least one transmitter is coupled to the vehicle; multiple receivers configured to receive Lidar signals reflected from the vehicle's surrounding environment, wherein each of the multiple receivers is configured to receive reflected Lidar signals at different frequencies; and a vehicle control module configured to: combine frequency data from the reflected Lidar signals received at the multiple receivers into a point cloud array; generate a multiplexed point cloud based on the point cloud array; provide the multiplexed point cloud to a perception stack configured to process three-dimensional images; and automatically control the vehicle's steering, acceleration, and braking based on the multiplexed point cloud provided to the perception stack.

[0019] This disclosure provides the following examples:

[0020] Example 1. A light detection and ranging (Lidar) system for a vehicle, the Lidar system comprising:

[0021] At least one transmitter is configured to transmit LiDAR signals at multiple frequencies, wherein the at least one transmitter is coupled to the vehicle;

[0022] An omnidirectional transmittable LiDAR is configured to transmit signals via the at least one transmitter;

[0023] A plurality of receivers configured to receive LiDAR signals reflected from the vehicle's surrounding environment, wherein each of the plurality of receivers is configured to receive reflected LiDAR signals at different frequencies; and

[0024] The vehicle control module is configured as follows:

[0025] Frequency data from reflected Lidar signals received at the multiple receivers are combined into a point cloud array;

[0026] Generate a multiplexed point cloud based on the point cloud array;

[0027] The multiplexed point cloud is provided to a perception stack configured to process 3D images; and

[0028] The vehicle's steering, acceleration, and braking are automatically controlled based on the multiplexed point cloud provided to the perception stack.

[0029] Example 2. The Lidar system according to Example 1, wherein each of the plurality of receivers is coupled to the vehicle at a different location on the vehicle.

[0030] Example 3. The Lidar system according to Example 1, wherein the vehicle control module is configured to index the frequency and scan angle of the reflected Lidar signal received from each of the plurality of receivers.

[0031] Example 4. The Lidar system according to Example 3, wherein the vehicle control module is configured to embed the frequency and scan angle of the index into the point cloud.

[0032] Example 5. According to the Lidar system described in Example 4, wherein the vehicle control module is configured as follows:

[0033] An array including the scanning angles of the point cloud is generated based on the scanning angles of the indexed point cloud; and

[0034] Iterate the point cloud scanning angle to combine the frequency data into the point cloud array.

[0035] Example 6. The Lidar system according to Example 1, wherein the vehicle control module is configured as follows:

[0036] Detect at least one object in the point cloud array;

[0037] Indexing the at least one object across multiple scan angles of the point cloud array; and

[0038] The position of the object is tracked across the multiple scanning angles of the point cloud array.

[0039] Example 7. The Lidar system according to Example 6, wherein the vehicle control module is configured as follows:

[0040] Measure the point density of the point cloud array at the current index; and

[0041] The point density is compared with a specified point density threshold.

[0042] Example 8. The Lidar system according to Example 7, wherein the vehicle control module is configured to provide the multiplexed point cloud to the perception stack in response to the point density being greater than the specified point density threshold.

[0043] Example 9. The Lidar system according to Example 7, wherein the vehicle control module is configured to: obtain the next scan angle in the point cloud array in response to the point density being less than the specified point density threshold and update the multiplexed point cloud according to the next scan angle in the point cloud array.

[0044] Example 10. The Lidar system according to Example 9, wherein updating the multiplexed point cloud includes removing duplicate objects from the multiplexed point cloud.

[0045] Example 11. According to the Lidar system described in Example 10, updating the multiplexed point cloud includes:

[0046] Normalize the point density using the specified uniform resolution; and

[0047] Remove duplicate points in the reused point cloud caused by point cloud reuse.

[0048] Example 12. The Lidar system according to Example 11, wherein the vehicle control module is configured to track the positions of objects in the multiplexed point cloud to determine the final object destination.

[0049] Example 13. The Lidar system according to Example 1, wherein the multiplexed point cloud does not include any blind spots, the point density of which is below a specified point density threshold.

[0050] Example 14. A method for operating a light detection and ranging (Lidar) system for a vehicle, the method comprising:

[0051] Lidar signals are transmitted at multiple frequencies via at least one Lidar transmitter, wherein the at least one Lidar transmitter is coupled to the vehicle, and a parabolic reflector is positioned to reflect the Lidar signals transmitted by the at least one Lidar transmitter;

[0052] Lidar signals reflected from the vehicle's surrounding environment are received via multiple Lidar receivers, each of which receives reflected Lidar signals at a different frequency.

[0053] Frequency data from reflected Lidar signals received at the plurality of Lidar receivers are combined into a point cloud array;

[0054] Generate a multiplexed point cloud based on the point cloud array;

[0055] The multiplexed point cloud is provided to a perception stack configured to process 3D images; and

[0056] The vehicle's steering, acceleration, and braking are automatically controlled based on the multiplexed point cloud provided to the perception stack.

[0057] Example 15. The method according to Example 14, wherein each of the plurality of Lidar receivers is coupled to the vehicle at a different location on the vehicle.

[0058] Example 16. The method according to Example 14 further includes indexing the frequency and scan angle of the reflected Lidar signal received from each of the plurality of Lidar receivers.

[0059] Example 17. The method described in Example 16 further includes embedding the frequency and scan angle of the index into the point cloud.

[0060] Example 18. The method according to Example 17 further includes:

[0061] An array including the scanning angles of the point cloud is generated based on the scanning angles of the indexed point cloud; and

[0062] The point cloud scanning angle is iterated to combine the frequency data into the point cloud array.

[0063] Example 19. The method according to Example 14 further includes:

[0064] Detect at least one object in the point cloud array;

[0065] Indexing the at least one object across multiple scan angles of the point cloud array; and

[0066] The position of the object is tracked across the multiple scanning angles of the point cloud array.

[0067] Example 20. A light detection and ranging (Lidar) system for a vehicle, the Lidar system comprising:

[0068] At least one transmitter is configured to transmit LiDAR signals at multiple frequencies, wherein the at least one transmitter is coupled to the vehicle;

[0069] A plurality of receivers are configured to receive LiDAR signals reflected from the vehicle's surrounding environment, wherein each of the plurality of receivers is configured to receive reflected LiDAR signals at different frequencies; and

[0070] The vehicle control module is configured as follows:

[0071] Frequency data from reflected Lidar signals received at the multiple receivers are combined into a point cloud array;

[0072] Generate a multiplexed point cloud based on the point cloud array;

[0073] The multiplexed point cloud is provided to a perception stack configured to process 3D images; and

[0074] The vehicle's steering, acceleration, and braking are automatically controlled based on the multiplexed point cloud provided to the perception stack.

[0075] Further applications of this disclosure will become apparent from the detailed description, claims, and accompanying drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0076] This disclosure will be more fully understood through detailed description and accompanying drawings.

[0077] Figure 1 This is a functional block diagram of an example vehicle according to the examples of this disclosure, which includes a Lidar system having multiple Lidar receivers.

[0078] Figure 2 This is an example block diagram of a multi-frequency Lidar transmission system for a vehicle, including multiple Lidar receivers, according to the present disclosure.

[0079] Figure 3 This is a flowchart illustrating an example process for controlling a Lidar system for a vehicle having multiple Lidar receivers, according to an example of this disclosure.

[0080] Figure 4 It is based on the example of this disclosure for use in Figure 3 The flowchart illustrates an example process of acquiring point cloud frequency information based on the received signal.

[0081] Figure 5 It is based on the example of this disclosure for use in Figure 3 The flowchart illustrates an example process for generating a reusable point cloud during the process.

[0082] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation

[0083] In some example embodiments, a light detection and ranging (Lidar) system for a vehicle may include a multi-frequency Lidar transmitter and multiple Lidar receivers configured to receive reflected Lidar signals at different frequencies. Due to the presence of objects, some Lidar technologies face limitations in generating complete and gapless point clouds. In some example embodiments described herein, Lidar systems using multiplexed point clouds can optimize and remove these shadows (e.g., blind spots where point cloud data is not obtained or is not obtained at a sufficiently high point density to detect objects).

[0084] For example, multiplexed multi-frequency point clouds can be used with a single Lidar system having multiple receiver modules or a single Lidar transmitter. The Lidar system can capture data from different frequencies and angles, allowing for a more comprehensive and accurate representation of the environment surrounding the vehicle.

[0085] The example point cloud multiplexing technique described in this paper enables LiDAR systems to collect data from various perspectives, thereby reducing or minimizing the effects of shadows and gaps in point cloud data caused by objects. By effectively combining information from multi-frequency point clouds and utilizing multiple LiDAR receiver modules, the resulting multiplexed point cloud can provide a more complete and detailed representation of the vehicle's surrounding environment.

[0086] In some examples, vehicle LiDAR system architectures combine multiple wavelengths and utilize multiplexing techniques to reduce or eliminate object shadows, generating point clouds with few or zero blind spots. This can be achieved by separating the LiDAR signal transmission and reception functions into independent transmission units and multiple receivers at different locations on the vehicle. This design facilitates the generation of coherent, full 3D images from point cloud data.

[0087] Now for reference Figure 1 Vehicle 10 includes wheels 12 and 13, which can shift between front and rear wheels depending on the direction of movement of vehicle 10 (e.g., an autonomous vehicle may not have a defined front side). Figure 1 In this configuration, drive unit 14 selectively outputs torque to wheels 12 and / or wheels 13 via drive lines 16 and 18, respectively. Vehicle 10 may include different types of drive units. For example, the vehicle may be an electric vehicle (such as a battery electric vehicle (BEV), a hybrid vehicle, or a fuel cell vehicle), a vehicle including an internal combustion engine (ICE), or other types of vehicles.

[0088] Some examples of drive unit 14 may include any suitable electric motor, power inverter, and motor controller, the motor controller being configured to control power switches within the power inverter to adjust the motor speed and torque during propulsion and / or regeneration. The battery system supplies power to or receives power from the motor of drive unit 14 via the power inverter during propulsion or regeneration.

[0089] Although Figure 1 The vehicle 10 includes a drive unit 14, but the vehicle 10 may have other configurations. For example, two separate drive units may drive wheels 12 and 13, or one or more individual drive units may drive individual wheels, etc. It is understood that other vehicle configurations and / or drive units may be used.

[0090] The vehicle control module 20 can be configured to control the operation of one or more vehicle components (such as drive unit 14) (e.g., by commanding the torque setting of the electric motor of drive unit 14). The vehicle control module 20 can receive inputs for controlling vehicle components, such as signals received from the steering wheel, accelerator paddles, etc. The vehicle control module 20 can monitor vehicle telematics for safety purposes, such as vehicle speed, vehicle position, vehicle braking, and acceleration.

[0091] The vehicle control module 20 can receive signals from any suitable component to monitor one or more aspects of the vehicle, including one or more vehicle sensors (e.g., cameras, LiDAR transmitters and LiDAR receivers, microphones, pressure sensors, wheel position sensors, position sensors (e.g., GPS antennas, etc.). Some sensors can be configured to monitor the vehicle's current motion, vehicle acceleration, steering wheel position, etc.

[0092] like Figure 1 The vehicle 10 shown includes a Lidar system 22, which includes a multi-frequency transmitter 24 and a parabolic reflector 26. The multi-frequency transmitter can be configured to transmit Lidar signals at multiple frequencies, for example, by using a laser to aim at an object or surface around the vehicle and measuring the time it takes for the reflected light to return to the receiver to determine distance. Example frequencies may include, but are not limited to, 10Hz, 100Hz, and nm wavelength signals. The multi-frequency transmitter 24 can be configured to change the scanning angle of different signals or different frequency signals.

[0093] Vehicle 10 includes a first Lidar receiver 28 and a second Lidar receiver 30, each configured to receive reflected Lidar signals emitted by a multi-frequency transmitter 24 and reflected from objects or surfaces in the vehicle's surrounding environment. In some examples, each Lidar receiver may be configured to receive reflected Lidar signals of different frequencies and may be positioned at different locations on vehicle 10 (e.g., the Lidar receivers may be decoupled from the multi-frequency transmitter 24). Vehicle control module 20 may store the position and / or distance of each Lidar receiver relative to the multi-frequency transmitter 24 for triangulation of the reflected signals received by each Lidar receiver. The Lidar signals may be emitted along a fixed direction or may scan multiple directions around vehicle 10.

[0094] Optional parabolic reflector 26 can reflect signals emitted by multi-frequency transmitter 24 to increase the number of angles or the range of the signal, thereby providing greater coverage of the area surrounding vehicle 10. Parabolic reflector 26 can have any suitable shape for dispersing the reflection of the emitted signal, such as a ring mirror, parabolic sphere, inverted mushroom, etc., to provide a 360-degree field of view. Parabolic reflector 26 can cover other sensors below it. Although Figure 1 The illustration shows a Lidar transmitter and two Lidar receivers, but other example embodiments may include more transmitters and / or receivers. In some embodiments, an omnidirectional transmittable Lidar may be configured to transmit signals via at least one transmitter.

[0095] In some examples, reflected Lidar signals received by the first Lidar receiver 28 and the second Lidar receiver 30 can be combined into a multiplexed point cloud for 3D imaging control of vehicle functions such as autonomous driving. For example, multiplexed point cloud data based on Lidar signals received by the Lidar receivers can be provided to a perception stack that automatically controls the steering, acceleration, and braking of the vehicle 10 to provide an autonomous advanced driver assistance system (ADAS).

[0096] The vehicle control module 20 can communicate with another device via a wireless communication interface, which may include one or more wireless antennas for transmitting and / or receiving wireless communication signals. For example, the wireless communication interface can communicate via any suitable wireless communication protocol, including but not limited to vehicle-to-everything (V2X) communication, Wi-Fi communication, wireless local area network (WAN) communication, cellular communication, personal area network (PAN) communication, short-range wireless communication (e.g., Bluetooth), etc. The wireless communication interface can communicate with remote computing devices through one or more wireless and / or wired networks. Regarding vehicle-to-vehicle (V2X) communication, vehicle 10 may include one or more V2X transceivers (e.g., V2X signal transmitting and / or receiving antennas).

[0097] In some examples, Lidar systems can have the ability to overlay object shadows in Lidar point clouds (due to the reuse of point cloud data from different frequencies or scan angles), thus producing point clouds without blind spots. Lidar systems can capture data from all angles and viewpoints to provide a more comprehensive and accurate representation of the environment.

[0098] By integrating multiple Lidar receivers into a Lidar system, the detection range can be extended. For example, the Lidar system can detect objects at greater distances, thereby enhancing its overall performance and capabilities. Using multiple Lidar receivers can improve the probability of detection (PoD) of objects around a vehicle. As more Lidar receivers capture data, the Lidar system can achieve a higher level of accuracy in detecting and identifying objects in its surrounding environment.

[0099] In some examples, point cloud data can be converted into accurate images more quickly, while also improving the image resolution. Lidar systems can process and analyze data more efficiently, thus providing a clearer and more detailed representation of the environment.

[0100] The example Lidar system can generate 360-degree images using a single non-rotating Lidar. This reduces or eliminates the need for multiple sensors to cover the entire field of view, simplifying setup and lowering costs. Integration of multiple receivers allows for a smaller Lidar package size. Lidar systems can be more compact and lighter, making them easier to integrate into different applications and platforms.

[0101] In some examples, improved Lidar systems can reduce or eliminate the need for heat sinks because the computational and energy requirements of a Lidar system can be lower due to the use of a single module. This reduces system complexity and cost while also improving its overall efficiency. The example Lidar systems also address signal attenuation issues, ensuring that the Lidar system maintains a strong and reliable signal throughout its operation. This allows the Lidar system to accurately capture and analyze data without quality loss or degradation.

[0102] Now for reference Figure 2 The Lidar system 200 can be a multi-frequency Lidar transmission system with multiple receivers. Figure 2 The Lidar system 200 can be corresponding to Figure 1 The Lidar system 22 includes a first Lidar receiver 28 and a second Lidar receiver 30.

[0103] like Figure 2 As shown, the multi-frequency LiDAR transmitter 204 transmits LiDAR signals at multiple frequencies (such as a first frequency 208, a second frequency 212, and a third frequency 216). Although Figure 2 The illustration shows Lidar signals at three frequencies, but other examples could include more or fewer frequencies.

[0104] Signals of different frequencies can be reflected by an optional parabolic reflector 220 to increase the signal coverage area around the vehicle. The transmitted signals are then received by multiple LiDAR receivers after being reflected off objects or surfaces in the vehicle's surrounding environment. Figure 2 The diagram shows a first Lidar receiver 224, a second Lidar receiver 228, and a third Lidar receiver 232, each of which can be configured to receive a reflected signal at one of the corresponding frequencies of a first frequency 208, a second frequency 212, and a third frequency 216.

[0105] Other example embodiments may include more or fewer Lidar receivers. As further explained below, the signals received by each of the first Lidar receiver 224, the second Lidar receiver 228, and the third Lidar receiver 232 can be converted into point cloud information, which includes indices of frequencies and scan angles corresponding to the received signals, to generate a multiplexed point cloud.

[0106] Figure 3 This is a flowchart of an example process for controlling a Lidar system for a vehicle with multiple Lidar receivers. Figure 3 The process can be, for example Figure 1This is achieved through the vehicle control module 20. At 304, the process begins by transmitting LiDAR signals at multiple frequencies. For example, Figure 1 The multi-frequency transmitter 24 can be configured to transmit Lidar signals at different specified frequency values.

[0107] At point 308, the vehicle control module is configured to reflect the emitted LiDAR signal using a parabolic reflector. For example, Figure 1 The multi-frequency LiDAR signal transmitted by the multi-frequency transmitter 24 can be generated by... Figure 1 The parabolic reflector 26 reflects the Lidar signal to disperse it over a larger area or angular range around the vehicle.

[0108] At 312, the vehicle control module is configured to receive LiDAR signals reflected from the surrounding environment at multiple decoupled LiDAR receivers on the vehicle. For example, after the LiDAR signals emitted from the multi-frequency transmitter 24 and the parabolic reflector 26 are reflected away from objects in the environment surrounding the vehicle 10, Figure 1 The first Lidar receiver 28 and the second Lidar receiver 30 can receive Lidar signals of different frequencies.

[0109] At point 316, the vehicle control module is configured to acquire point cloud frequency information based on the received LiDAR signal. Further details regarding the acquisition of point cloud frequency information are provided below. Figure 4 discuss.

[0110] The vehicle control module is configured to generate a multiplexed point cloud at location 320. This multiplexed point cloud may not include blind spots (e.g., the 3D image of the multiplexed point cloud does not have sufficient point cloud data or point density to determine the location of an object). Further details regarding the generation of the multiplexed point cloud are referenced below. Figure 5 Further discussion.

[0111] At point 324, the vehicle control module is configured to provide 3D imaging to the vehicle perception and viewing stack based on multiplexed point clouds. For example, the vehicle control module can be configured to generate one or more 3D images based on multiplexed point clouds for use by one or more vision control algorithms of the vehicle.

[0112] At point 328, the vehicle control module is configured to automatically control the vehicle's steering, acceleration, and braking based on 3D imaging. For example, an Advanced Driver Assistance System (ADAS) can use multiplexed point clouds without blind spots to perform autonomous driving control of the vehicle.

[0113] Figure 4 It is used in Figure 3 The flowchart illustrates an example process of acquiring point cloud frequency information based on the received signal. Figure 4The process can be, for example Figure 1 This is implemented by the vehicle control module 20. At 404, the process begins by indexing the frequency and scan angle of the Lidar signals received by multiple vehicle Lidar receivers. For example, the vehicle control module can index with... Figure 1 The first Lidar receiver 28 and the second Lidar receiver 30 receive the frequency and scan angle values ​​corresponding to the reflected Lidar signals.

[0114] At position 408, the vehicle control module is configured to embed frequency and scan angle data into the point cloud. Then, at position 412, the vehicle control module generates an array that includes the scan angle of the point cloud.

[0115] At position 416, the vehicle control module is configured to select a first scan angle from 1 to N scan angles. For example, the array may include point cloud scan angles indexed from 1 to N according to different Lidar receivers, and the vehicle control module may be configured to iterate over each point cloud scan angle value.

[0116] At 420, the vehicle control module is configured to combine frequency data into a point cloud array. For example, the frequency data may correspond to different Lidar signal frequencies received by different Lidar receivers, each tuned to a different frequency, and these Lidar receivers may be mounted at different locations on the vehicle.

[0117] The vehicle control module is configured to determine at point 424 whether the last scan angle in the array has been reached. For example, in an array of 1 to N scan angles, the control can determine whether the Nth scan angle has been processed.

[0118] If not, control proceeds to 428 to select the next scan angle in the array, and then at 420, frequency data is combined into the point cloud array based on the next selected scan angle. Once the last scan angle in the array is reached at 424, control proceeds to 432 to store the point cloud array including the combined frequency data from each scan angle.

[0119] Figure 5 It is used in Figure 3 The flowchart illustrates an example process for generating a reusable point cloud during the process. Figure 5 The process can be, for example Figure 1 This is implemented by the vehicle control module 20. At 504, the process begins by performing object recognition and detection on a point cloud that includes frequency information. For example, one or more automatic object detection algorithms for LiDAR point cloud image processing can be used to detect objects in the environment surrounding the vehicle.

[0120] At point 508, the vehicle control module is configured to perform object indexing. For example, an index value can be assigned to each object detected in the point cloud to track objects across different point cloud frequency portions or scan angles.

[0121] At point 512, the vehicle control module is configured to track object positions based on object recognition and indexing. For example, the control can use an index assigned to a specific detected object to track the position of that object across various point cloud frequency portions or scan angles.

[0122] At point 516, the vehicle control module is configured to measure the point density at the current index. For example, the vehicle control module can select a first scan angle index for the point cloud and measure the point density at that scan angle index. This point density can be measured using any suitable algorithm and can indicate the point density of the entire point cloud, the point density at a specific location in the point cloud, the point density of different parts of the point cloud, etc. The point density can indicate whether there are any blind spots in the point cloud where there is insufficient data to determine whether an object exists at that location.

[0123] For example, at 520, the vehicle control module is configured to determine whether the measured point density is less than a specified point density threshold, which can be calibrated. The point density threshold can be set to indicate the presence of any blind spots in the point cloud where there is insufficient point data to determine the presence of an object at that location.

[0124] If the measured point density is below a threshold at 524, control continues to 532 to obtain the next scan angle and begin multiplexing the point cloud. For example, control can obtain point cloud data from another scan angle, which may correspond to one or more different frequencies of reflected LiDAR signals received from different receivers among multiple LiDAR receivers. Control can then multiplex the point cloud data corresponding to the selected scan angle with the point cloud data processed for the previous scan angle.

[0125] At point 536, the vehicle control module is configured to remove duplicate objects from the reused point cloud. For example, if an object identified in the current point cloud scan angle index is a duplicate of the same object identified in a previous point cloud scan angle index, the control can remove the duplicate object data so that the same object is identified only once in the reused point cloud.

[0126] At point 540, the vehicle control module is configured to normalize the point density using a consistent resolution in the multiplexed point cloud. For example, the control can remove duplicate points generated by multiplexed point cloud data from different scan angle indices, thereby maintaining a consistent resolution of points when adding additional point cloud data from different scan angle indices.

[0127] At point 544, the vehicle control module is configured to re-track the object positions in the multiplexed point cloud to determine the final object destination. For example, after adding supplementary data from different scan angle indices to the multiplexed point cloud, the control can use the object index to update or confirm the position of objects in the multiplexed point cloud.

[0128] After updating the reused point cloud, control then returns to index 516 to measure the point density at that index. Once the measured point density at index 524 is higher than a specified point density threshold, the vehicle control module is configured to transmit the reused point cloud to the perception stack (e.g., for autonomous driving control or other vehicle vision systems). Because the reuse of point cloud data covers any blind spots that may appear in the individual indices of the scan angle index of the point cloud data, the reused point cloud may not include any blind spots.

[0129] The foregoing description is merely illustrative in nature and is in no way intended to limit this disclosure, its application, or use. The broad teachings of this disclosure can be implemented in many forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent upon examination of the drawings, specification, and appended claims. It should be understood that one or more steps within a method may be performed in a different order (or simultaneously) without altering the principles of this disclosure. Furthermore, while each embodiment is described above as having specific features, any one or more of those features described with respect to any embodiment of this disclosure may be implemented in any other embodiment and / or combined with features of any other embodiment, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and the arrangement of one or more embodiments with respect to each other remains within the scope of this disclosure.

[0130] Spatial and functional relationships between components (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including “connection,” “joint,” “coupled,” “proximity,” “adjacent,” “on top of,” “above,” “below,” and “set.” Unless explicitly stated as “direct,” the relationship between the first and second components described in the above disclosure can be a direct relationship, where no other intermediate components exist between the first and second components, but it can also be an indirect relationship, where one or more intermediate components exist (spatially or functionally) between the first and second components. As used herein, the phrase “at least one of A, B, and C” should be interpreted as meaning logically (A or B or C) using the non-exclusive logical “OR,” and should not be interpreted as meaning “at least one of A, at least one of B, and at least one of C.”

[0131] In the accompanying drawings, the direction of the arrows, as indicated by the arrows, typically illustrates the flow of information (e.g., data or instructions) of interest to the illustration. For example, when components A and B exchange various types of information, but the information transmitted from component A to component B is relevant to the illustration, the arrow can point from component A to component B. This unidirectional arrow does not imply that no other information is transmitted from component B to component A. Furthermore, for information sent from component A to component B, component B can send a request for or confirmation of receipt of that information to component A.

[0132] In this application, including the following definitions, the term "module" or "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or combinations of some or all of the foregoing, such as in a system-on-a-chip.

[0133] This module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module of this disclosure can be distributed among multiple modules connected via the interface circuits. For example, multiple modules can allow for load balancing. In a further example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0134] The term "code" as used above can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" covers a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" covers a processor circuit that, in conjunction with additional processor circuitry, executes some or all of the code from one or more modules. References to multiple processor circuits cover multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" covers a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" covers a memory circuit that, in conjunction with additional memory, stores some or all of the code from one or more modules.

[0135] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not cover transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); therefore, the term "computer-readable medium" can be considered tangible and non-transitory. Non-limiting examples of non-transitory, tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0136] The apparatus and methods described in this application can be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to execute one or more specific functions embodied in a computer program. The aforementioned function blocks, flowchart components, and other elements serve as a software specification that can be routinely translated into a computer program by a skilled technician or programmer.

[0137] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may encompass a basic input / output system (BIOS) that interacts with the hardware of a special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0138] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code for execution by an interpreter; (v) source code for compilation and execution by a just-in-time (JIT) compiler; and so on. As an example only, source code can be written using syntax from languages ​​including: C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, etc. Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language 5th Edition), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Lua, MATLAB, SIMULINK and

Claims

1. A light detection and ranging (Lidar) system for a vehicle, the Lidar system comprising: At least one transmitter is configured to transmit LiDAR signals at multiple frequencies, wherein the at least one transmitter is coupled to the vehicle; An omnidirectional transmittable LiDAR is configured to transmit signals via the at least one transmitter; A plurality of receivers configured to receive LiDAR signals reflected from the vehicle's surrounding environment, wherein each of the plurality of receivers is configured to receive reflected LiDAR signals at different frequencies; and The vehicle control module is configured as follows: Frequency data from reflected Lidar signals received at the multiple receivers are combined into a point cloud array; Generate a multiplexed point cloud based on the point cloud array; The multiplexed point cloud is provided to a perception stack configured to process 3D images; and The vehicle's steering, acceleration, and braking are automatically controlled based on the multiplexed point cloud provided to the perception stack.

2. The Lidar system according to claim 1, wherein, Each of the plurality of receivers is coupled to the vehicle at a different location on the vehicle.

3. The Lidar system of claim 1, wherein the vehicle control module is configured to index the frequency and scan angle of reflected Lidar signals received from each of the plurality of receivers.

4. The Lidar system of claim 3, wherein the vehicle control module is configured to embed the frequency and scan angle of the index into the point cloud.

5. The Lidar system according to claim 4, wherein the vehicle control module is configured as follows: An array including the scanning angles of the point cloud is generated based on the scanning angles of the indexed point cloud; and Iterate the point cloud scanning angle to combine the frequency data into the point cloud array.

6. The Lidar system according to claim 1, wherein the vehicle control module is configured as follows: Detect at least one object in the point cloud array; Indexing the at least one object across multiple scan angles of the point cloud array; and The position of the object is tracked across the multiple scanning angles of the point cloud array.

7. The Lidar system according to claim 6, wherein the vehicle control module is configured to: Measure the point density of the point cloud array at the current index; and The point density is compared with a specified point density threshold.

8. The Lidar system of claim 7, wherein the vehicle control module is configured to provide the multiplexed point cloud to the perception stack in response to the point density being greater than the specified point density threshold.

9. The Lidar system of claim 7, wherein the vehicle control module is configured to: obtain the next scan angle in the point cloud array in response to the point density being less than the specified point density threshold and update the multiplexed point cloud according to the next scan angle in the point cloud array.

10. The Lidar system of claim 9, wherein updating the multiplexed point cloud includes deleting duplicate objects from the multiplexed point cloud.

Images