Aggregated-based lidar data alignment

Abstract

A LIDAR-to-vehicle alignment system includes a memory and an alignment and autonomous driving module. The memory stores data points provided based on output of one or more LIDAR sensors and positioning data. The alignment module performs an alignment process including: determining, based on the positioning data, whether the host vehicle is turning; responsive to the host vehicle turning, selecting a portion of the data points; aggregating the selected portion to provide aggregated data; selecting a target based on the aggregated data; and based on the selected target, iteratively reducing a loss value of a loss function to provide a synthetic LIDAR-to-vehicle transformation matrix. The autonomous driving module: converts at least the selected portion into at least one of vehicle coordinates or world coordinates based on the synthetic LIDAR-to-vehicle transformation matrix to provide synthetic data; and performs one or more autonomous driving operations based on the synthetic data.

Description

[0001] introduction The information provided in this section is for the purpose of presenting the general background of this disclosure. To the extent described in this section, the work of the currently named inventors, and aspects of the description that may otherwise not conform to the prior art at the time of submission, are neither explicitly nor implicitly considered to be prior art to this disclosure. Technical Field

[0002] This disclosure relates to vehicle object detection systems and more particularly to vehicle light detection and ranging (LIDAR) systems. Background Technology

[0003] The vehicle may include various sensors for detecting the surrounding environment and objects within that environment. Sensors may include cameras, radio detection and ranging (radar) sensors, LiDAR sensors, etc. The vehicle controller may perform various operations in response to the detected surrounding environment. These operations may include performing partially and / or fully autonomous vehicle operations, collision avoidance operations, and information reporting operations. The accuracy of the performed operations may be based on the accuracy of the data collected from the sensors. Summary of the Invention

[0004] A LiDAR-to-vehicle alignment system is disclosed, comprising a memory, an alignment module, and an autonomous driving module. The memory is configured to store data points provided by outputs and positioning data from one or more LiDAR sensors. The alignment module is configured to perform an alignment process including: based on the positioning data; determining whether a master vehicle is steering; responding to the master vehicle steering; selecting a portion of the data points; aggregating the selected portion of the data points to provide aggregated data; selecting a target based on the aggregated data; and iteratively reducing the loss value of a loss function based on the selected target to provide a synthetic LiDAR-to-vehicle transformation matrix. The autonomous driving module is configured to: transform at least a selected portion of the data points to at least one of vehicle coordinates or world coordinates based on the synthetic LiDAR-to-vehicle transformation matrix to provide synthetic data; and perform one or more autonomous driving operations based on the synthetic data.

[0005] Among other features, the alignment module is configured to receive positioning data from one or more of a global position sensor or an inertial measurement sensor.

[0006] Among other features, the alignment module is configured to suppress the execution of data selection and aggregation, as well as target selection, when the master vehicle is not steering.

[0007] Among other features, the alignment module is configured to select data points and targets based on the steering point.

[0008] Among other features, the alignment module is configured to select an area based on the turning point, remove ground points based on the selected area, identify targets, and perform a data clustering algorithm to select one of the targets to be monitored.

[0009] Among other features, the alignment module is configured to accumulate LiDAR data from multiple scenes and minimize the loss value based on the LiDAR data from multiple scenes.

[0010] Among other features, the alignment module is configured to perform a continuity check to determine which data points to select and which to discard.

[0011] Among other features, the alignment module is configured to: perform a LiDAR scan of data in a predefined space; perform target selection based on the aggregated LiDAR scan of the data; and after performing target selection, downsample each frame of data associated with the aggregated LiDAR scan of the data using an intensity filter and a random filter to provide synthetic target data.

[0012] Among other features, the alignment module is configured to: obtain the z-value of each of a plurality of LiDAR scans; obtain a normal distribution of the aggregated data associated with the LiDAR scans to extract a first mean of the aggregated data associated with the LiDAR scans; select data around the first mean; obtain another normal distribution of the selected data around the first mean to obtain a second mean not based on ground points; extract ground points around the second mean; label data other than ground points as target data; and, when a predetermined amount of target data has been collected, perform a clustering algorithm to select a portion of the target data.

[0013] Among other features, the alignment module is configured to, when iteratively reducing the loss value,: load optimization parameters and an initial estimated LiDAR-to-vehicle transformation matrix; and iteratively (i) change the initial estimated LiDAR-to-vehicle transformation matrix or the final determined LiDAR-to-vehicle transformation matrix to provide a synthetic LiDAR-to-vehicle transformation matrix, (ii) evaluate the changes in the synthetic LiDAR-to-vehicle transformation matrix, including determining the loss value, and (iii) based on the loss value, determine whether to perform another iteration.

[0014] Among other features, the alignment module is configured to, when evaluating the loss function,: compute the LiDAR-to-world transformation matrix using the synthesized LiDAR-to-vehicle transformation matrix; project the aggregated data onto world coordinate data based on the LiDAR-to-world transformation matrix; apply a binary tree to the world coordinate data; provide distances by computing the distances to neighboring data points for each LiDAR data point; and determine the loss value based on the average of the distances.

[0015] Among the other features, the loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets.

[0016] Among other features, the autonomous vehicle driving method includes: storing data points provided by output and positioning data from one or more LiDAR sensors; performing a LiDAR-to-vehicle alignment process, including (i) determining, based on the positioning data, whether the master vehicle is turning; (ii) in response to the master vehicle turning, selecting a portion of the data points and aggregating the selected portion of the data points to provide aggregated data; (iii) selecting a target based on the aggregated data; and (iv) iteratively reducing the loss value of a loss function based on the selected target to provide a synthetic LiDAR-to-vehicle transformation matrix; transforming at least a selected portion of the data points to at least one of vehicle coordinates or world coordinates based on the synthetic LiDAR-to-vehicle transformation matrix to provide synthetic data; and performing one or more autonomous driving operations based on the synthetic data.

[0017] Among other features, the autonomous vehicle driving method further includes: selecting an area based on a steering point; removing ground points based on the selected area; identifying targets; and performing a data clustering algorithm to select one of the targets to be monitored.

[0018] Among other features, the autonomous vehicle driving method further includes: performing a continuity check to determine which data points to select and which to discard; and accumulating LiDAR data from multiple scenarios, and minimizing the loss value based on the LiDAR data from multiple scenarios.

[0019] Among other features, the autonomous vehicle driving method further includes: a LiDAR scan of data in a predefined space; target selection based on the aggregated LiDAR scan of the data; and, after performing target selection, downsampling each frame of data associated with the aggregated LiDAR scan of the data using intensity filters and random filters to provide synthetic target data.

[0020] Among other features, the autonomous vehicle driving method further includes: obtaining the z-value of each of a plurality of LiDAR scans; determining at least one average value based on aggregated data associated with the LiDAR scans; extracting ground points based on the at least one average value; labeling data other than the ground points as target data; and performing a clustering algorithm to select a portion of the target data when a predetermined amount of target data has been collected. Among other features, the autonomous vehicle driving method further includes: obtaining the z-value of each of a plurality of LiDAR scans; obtaining a normal distribution of aggregated data associated with the LiDAR scans to extract a first average value of the aggregated data associated with the LiDAR scans; selecting data surrounding the first average value; obtaining another normal distribution of the selected data surrounding the first average value to obtain a second average value not based on ground points; extracting ground points surrounding the second average value; labeling data other than the ground points as target data; and performing a clustering algorithm to select a portion of the target data when a predetermined amount of target data has been collected.

[0021] Among other features, the autonomous vehicle driving method further includes, while iteratively reducing the loss value: loading optimization parameters and an initial estimated LiDAR-to-vehicle transformation matrix; and iteratively (i) changing the initial estimated LiDAR-to-vehicle transformation matrix or the finally determined LiDAR-to-vehicle transformation matrix to provide a synthetic LiDAR-to-vehicle transformation matrix, (ii) evaluating the change in the synthetic LiDAR-to-vehicle transformation matrix, including determining the loss value, and (iii) determining, based on the loss value, whether to perform another iteration.

[0022] Among other features, the autonomous vehicle driving method further includes, when evaluating the loss function: calculating the LiDAR-to-world transformation matrix using a synthetic LiDAR-to-vehicle transformation matrix; projecting the aggregated data onto world coordinate data based on the LiDAR-to-world transformation matrix; providing distances based on the world coordinate data by calculating the distances to neighboring data points for each LiDAR data point; and determining a loss value based on the average of the distances.

[0023] Among the other features, the loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets.

[0024] This invention provides the following technical solutions: 1. A LiDAR-to-vehicle alignment system, comprising: The memory is configured to store data points based on the output and positioning data of one or more LiDAR sensors; The alignment module is configured to perform the alignment process, including... Based on location data, determine whether the main vehicle is turning. In response to the main vehicle's steering, a subset of data points is selected and aggregated to provide aggregated data. Target selection based on aggregated data, and Based on the selected objective, the loss value of the loss function is iteratively reduced to provide the synthetic LiDAR-to-vehicle transformation matrix; and The autonomous driving module is configured as follows: Based on the synthetic LiDAR-to-vehicle transformation matrix, at least a selected portion of the data points is transformed to at least one of vehicle coordinates or world coordinates to provide synthetic data, and Perform one or more autonomous driving operations based on synthetic data.

[0025] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to receive positioning data from one or more of a global position sensor or an inertial measurement sensor.

[0026] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to suppress the execution of data selection and aggregation as well as target selection when the master vehicle is not turning.

[0027] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to select data points and targets based on steering points.

[0028] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to select an area based on a turning point, remove ground points based on the selected area, identify multiple targets, and perform a data clustering algorithm to select one of the multiple targets for monitoring.

[0029] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to accumulate LIDAR data from multiple scenarios and minimize the loss value based on the LIDAR data from the multiple scenarios.

[0030] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to perform a continuity check to determine which data points to select and which data points to discard.

[0031] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to: LiDAR scans that aggregate data in a predefined space; Data-driven clustered LiDAR scanning is used to perform target selection; and After target selection is performed, each frame of data associated with the aggregated LiDAR scan of the data is downsampled using intensity and random filters to provide synthetic target data.

[0032] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to: Obtain the z-value of each of multiple LiDAR scans; Obtain the normal distribution of the aggregated data associated with the LiDAR scan to extract the first mean of the aggregated data associated with the LiDAR scan; Select data around the first average; Obtain another normal distribution of selected data around the first mean to obtain a second mean that is not based on the ground point; Extract ground points around the second average value; Mark data other than ground points as target data; and Once a predetermined amount of target data has been collected, a clustering algorithm is executed to select a subset of the target data.

[0033] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to, when iteratively reducing the loss value: Load the optimized parameters and the initially estimated LIDAR into the vehicle transformation matrix; and Iterative The initial estimated LiDAR-to-vehicle transformation matrix or the final determined LiDAR-to-vehicle transformation matrix can be changed to provide a synthesized LiDAR-to-vehicle transformation matrix. Evaluate the changes in the synthetic LiDAR to the vehicle transformation matrix, including determining the loss value, and Based on the loss value, determine whether to perform another iteration.

[0034] According to the LIDAR-to-vehicle alignment system of technical solution 1, the alignment module is configured to, when evaluating the loss function: The LiDAR to world transformation matrix is ​​calculated using the synthesized LiDAR to vehicle transformation matrix. The aggregated data is projected onto world coordinates based on the LIDAR-to-world transformation matrix. Applying binary trees to world coordinate data; Distance is provided by calculating the distance to neighboring data points for each LiDAR data point; and The loss value is determined based on the average of the multiple distances.

[0035] According to the LIDAR-to-vehicle alignment system of technical solution 1, the loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets.

[0036] An autonomous vehicle driving method, comprising: Stores data points based on the output and positioning data of one or more LiDAR sensors; Performing the LIDAR-to-vehicle alignment process includes Based on location data, determine whether the main vehicle is turning. In response to the main vehicle's steering, a subset of data points is selected and aggregated to provide aggregated data. Target selection based on aggregated data, and Based on the selected objective, the loss value of the loss function is iteratively reduced to provide the synthetic LIDAR to vehicle transformation matrix; Based on the synthetic LiDAR-to-vehicle transformation matrix, at least a selected portion of the data points is transformed to at least one of vehicle coordinates or world coordinates to provide synthetic data; and Perform one or more autonomous driving operations based on synthetic data.

[0037] The autonomous vehicle driving method according to technical solution 13 further includes: Select the region based on the turning point; Remove ground points based on the selected area; Identify multiple targets; and A data clustering algorithm is executed to select one of the plurality of targets for monitoring.

[0038] The autonomous vehicle driving method according to technical solution 13 further includes: Perform continuity checks to determine which data points to select and which to discard; and Accumulate LiDAR data from multiple scenarios and minimize the loss value based on the LiDAR data from multiple scenarios.

[0039] The autonomous vehicle driving method according to technical solution 13 further includes: LiDAR scans that aggregate data in a predefined space; Data-driven clustered LiDAR scanning is used to perform target selection; and After target selection is performed, each frame of data associated with the aggregated LiDAR scan of the data is downsampled using intensity and random filters to provide synthetic target data.

[0040] The autonomous vehicle driving method according to technical solution 13 further includes: Obtain the z-value of each of multiple LiDAR scans; Determine at least one average value based on aggregated data associated with LIDAR scans; Ground points are extracted based on the at least one average value; Mark data other than ground points as target data; and Once a predetermined amount of target data has been collected, a clustering algorithm is executed to select a subset of the target data.

[0041] The autonomous vehicle driving method according to technical solution 13 further includes, when iteratively reducing the loss value: Load the optimized parameters and the initially estimated LiDAR to vehicle transformation matrix; and Iterative The initial estimated LiDAR-to-vehicle transformation matrix or the final determined LiDAR-to-vehicle transformation matrix can be changed to provide a synthesized LiDAR-to-vehicle transformation matrix. Evaluate the changes in the synthetic LiDAR to the vehicle transformation matrix, including determining the loss value, and Based on the loss value, determine whether to perform another iteration.

[0042] The autonomous vehicle driving method according to technical solution 13 further includes, when evaluating the loss function: The LiDAR to world transformation matrix is ​​calculated using the synthesized LiDAR to vehicle transformation matrix. Based on the LIDAR-to-world transformation matrix, aggregated data is projected onto world coordinate data; Distance is provided based on world coordinate data by calculating the distance to neighboring data points for each LiDAR data point; and The loss value is determined based on the average distance.

[0043] According to the autonomous vehicle driving method of technical solution 13, the loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets. Further applicability areas of this disclosure will become clear from the detailed description, claims, and drawings. The detailed description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0044] This disclosure will be more fully understood based on the detailed description and accompanying drawings, wherein: This disclosure will be more fully understood based on the detailed description and accompanying drawings, wherein: Figure 1 This is a functional block diagram of an example vehicle system including an alignment verification module according to this disclosure; Figure 2 This is a functional block diagram of an example alignment system including vehicles and a back office, based on this disclosure; Figure 3 An example alignment method according to this disclosure is illustrated; Figure 4 A more detailed illustration is provided. Figure 3 Alignment method; Figure 5 The illustration shows an example buffer management and data filtering method according to this disclosure; Figure 6 The illustration shows an example feature extraction and filtering algorithm according to this disclosure; Figure 7 An example target selection algorithm according to this disclosure is illustrated; Figure 8 An example optimization algorithm according to this disclosure is illustrated; Figure 9 This is an example plot of the loss versus translation distance using the first density characterization method according to this disclosure; and Figure 10 This is an example plot of the translation distance using the loss of the second density characterization method according to this disclosure.

[0045] In the accompanying drawings, reference numerals may be repeated to identify similar and / or identical elements. Detailed Implementation

[0046] The autonomous driving module can perform sensor alignment and fusion operations, perception and localization operations, and path planning and vehicle control operations. The described operations can be performed based on data collected from various sensors, such as LiDAR sensors, radar sensors, cameras, and inertial measurement sensors (or inertial measurement units). Sensor alignment and fusion can include aligning the coordinate system of each sensor with a reference coordinate system such as the vehicle coordinate system. Fusion can refer to the collection and combination of data from various sensors.

[0047] Perception refers to the monitoring of the vehicle's surroundings and the detection and identification of various features and / or objects within those features. This can include determining aspects of features and objects. As used herein, the term "feature" refers to one or more detection points that can be reliably used to determine the location of an object. This differs from other detected data points that do not provide reliable information about the object's location, such as points on leaves or branches. The determined aspects can include the object's distance, position, size, shape, orientation, trajectory, etc. This can include determining the type of object detected, such as whether the object is a traffic sign, vehicle, utility pole, pedestrian, ground, etc. Lane marking information can also be detected. Features can refer to the surface, edge, or corner of a building. Localization refers to the determination of information about the primary vehicle, such as its position, speed, heading, etc. Path planning and vehicle control (e.g., braking, steering, and acceleration) are performed based on the collected perception and localization information.

[0048] Vehicles may include multiple LiDAR sensors. LiDAR sensor alignment, including LiDAR-to-vehicle alignment and LiDAR-to-LIDAR alignment, affects the accuracy of the determined perception and localization information, which includes feature and object information, as described above. GPS measurements are used for vehicle localization, mapping, and LiDAR alignment.

[0049] Data collected from different LiDAR sensors is transformed (or projected) to the same coordinate system before being evaluated. For example, LiDAR data can be transformed to a vehicle coordinate system or a world coordinate system. The accuracy of this transformation depends on the alignment of the alignment system. Alignment refers to the transformation matrix T used to transform LiDAR coordinate data into vehicle coordinate data. LtoV Are the entries in it valid? Transformation matrix T LtoV It can be used to transform coordinates in the vehicle coordinate system to LiDAR coordinate data. Due to various reasons, alignment may deteriorate or become incorrect over time, causing the transformation matrix T to... LtoV The entry is no longer valid. Alignment may deteriorate due to sensor misalignment, sensor vibration, accidents, etc. LiDAR sensor realignment can be performed offline at an alignment service center by trained service technicians using specialized equipment. Offline alignment of the vehicle's LiDAR sensors is typically not performed by the vehicle's customer.

[0050] The aggregation of LIDAR frames includes the use of positioning data, such as GPS data and data from the inertial measurement unit (IMU), LIDAR data, and the LIDAR-to-vehicle coordinate alignment transformation matrix T. LtoV (Hereinafter referred to as "transformation matrix T") LtoV Transformation matrix T LtoVUsed to provide composite data in the vehicle coordinate system. If alignment is not performed, then due to the transformation matrix T... LtoV Misalignment can cause errors in the synthesized image. When performing tasks such as autonomous vehicle maneuvering, object detection, and collision avoidance, misalignment can introduce these errors.

[0051] Examples described in this paper include online misalignment detection and correction. An alignment algorithm is provided, which includes determining an effective transformation matrix T for transforming data between LiDAR and vehicle coordinates. LtoV An alignment algorithm is disclosed that maximizes the point density of a clustered LiDAR point cloud of a selected target. The amount of alignment error tends to increase as the volume of the point cloud increases (or the point cloud density decreases). By increasing the point density associated with the selected target and / or decreasing the point volume, image sharpness is improved, and the amount of alignment error is reduced.

[0052] The alignment algorithm includes: a sub-algorithm for dynamically processing LiDAR data to compute alignment results; a sub-algorithm for automatically selecting one or more targets during cornering maneuvers; the use of one or more loss functions; a sub-algorithm for optimizing the one or more loss functions used; and a sub-algorithm for generating results using data from multiple scenes. The loss function can be a density-based loss function used to characterize the severity of misalignment. The sub-algorithm for optimizing one or more loss functions accelerates the corresponding iterative process being executed, as explained further below. Each data scene refers to a set of LiDAR data images collected for a specific area traversed by the vehicle. Data collected for multiple scenes is used to improve the robustness of the synthesized deterministic transformation. The alignment algorithm performs target selection and is applicable to targets of any shape used for alignment purposes.

[0053] Transformation matrix T LtoV This can be represented by Equation 1, where R is a 3×3 rotation matrix and T is a translation vector. The rotation matrix R is generated based on roll, pitch, and yaw angles, as shown in Equation 2. The translation vector T can be represented by Equation 3 and is a 3×1 vector used for translation, where T... x T y T z These are the translation values ​​in the x, y, and z directions. R and T depend on T. x T y T z And roll, pitch, and yaw angles.

[0054] Transformation matrix (1) (2) (3).

[0055] The data in the vehicle coordinates is equal to the transformation matrix TLtoV The product of the collected LIDAR sensor data and the transformation matrix T. LtoV The inverse can be used to transform vehicle coordinates to LiDAR coordinates.

[0056] Examples described in this article include LiDAR-to-vehicle alignment systems and algorithms that generate the transformation matrix T "online". LtoV Transformation matrix T LtoV Used to provide a dynamic linear mapping between LIDAR and vehicle coordinates. This provides dynamic calibration for individual vehicles as well as large fleets, while minimizing and / or eliminating the need for alignment at the manufacturer, dealership, and / or service station.

[0057] Figure 1 An example vehicle system 100 including an online alignment module 104 is shown. Reference is made below. Figure 1-10 The operations performed by the online alignment module 104 are further described. Vehicle system 100 may include an autonomous driving module 105, a body control module (BCM) 107, a telematics module 106, a propulsion control module 108, a power steering system 109, a braking system 111, a navigation system 112, an infotainment system 114, an air conditioning system 116, and other vehicle systems and modules 118. Autonomous driving module 105 includes the online alignment module 104, a fusion module 113, a perception module 115, a mapping and positioning module 117, and a path planning module 121. Mapping and positioning module 117 may include a GPS correction module. The operation of these modules is further described below.

[0058] Modules and systems 104-108, 112-115, 118, and 121 can communicate with each other via a Controller Area Network (CAN) bus, Ethernet, Local Area Network (LIN) bus, another bus or communication network, and / or wirelessly. Item 119 may refer to and / or include a CAN bus, Ethernet, LIN bus, and / or other bus and / or communication network. This communication may include other systems, such as systems 109, 111, and 116. Power supply 122 may be included and power autonomous driving module 105 and other systems, modules, devices, and / or components. Power supply 122 may include an accessory power module, one or more batteries, a generator, and / or other power sources.

[0059] The telematics module 106 may include a transceiver 130 and a telematics control module 132. The propulsion control module 108 may control the operation of the propulsion system 136, which may include an engine system 138 and / or one or more electric motors 140. The engine system 138 may include an internal combustion engine 141, a starter motor 142 (or a starter), a fuel system 144, an ignition system 146, and a throttle system 148.

[0060] The autonomous driving module 105 can control modules and systems 106, 108, 109, 111, 112, 114, 116, 118, and other devices and systems based on data from sensors 160. Other devices and systems may include door and window actuators 162, interior lights 164, exterior lights 166, a trunk motor and lock 168, a seat position motor 170, a seat temperature control system 172, and a vehicle rearview mirror motor 174. Sensors 160 may include temperature sensors, pressure sensors, flow rate sensors, position sensors, etc. Sensors 160 may include a LiDAR sensor 180, a radar sensor 182, a camera 184, an inertial measurement sensor 186, a GPS sensor 190, and / or other environmental and feature detection sensors. The GPS sensor 190 may be implemented as part of the navigation system 112. The LiDAR sensor 180, the inertial measurement sensor 186, and the GPS sensor 190 can provide LiDAR data points, inertial measurement data, and GPS data as mentioned below.

[0061] The autonomous driving module 105 may include a memory 192, which may store sensor data, historical data, alignment information, etc. The memory 192 may include a dedicated buffer mentioned below.

[0062] Figure 2 An example alignment system 200 is shown, which includes a first (or master) vehicle (e.g., Figure 1 The main vehicle includes vehicle 102 and / or other vehicles, a distributed communication network 202, and a back office 204. The main vehicle includes an autonomous driving module 105, vehicle sensors 160, a telematics module 106, and actuators 210. Actuators 210 may include motors, drivers, valves, switches, etc.

[0063] The back office 204 can be a central office providing services to the vehicle, including data collection and processing services. The back office 204 may include a transceiver 211 and a server 214 with a control module 216 and a memory 218. Alternatively, the vehicle may communicate with other cloud-based network devices besides the server.

[0064] The autonomous driving module 105 may include a sensor data collection module 212, an online alignment module 104, a fusion module 113, a perception module 115, a map creation and localization module 117, and a path planning module 121. The sensor data collection module 212 can collect data from sensor 160, including LiDAR data from multiple LiDAR sensors. The online alignment module 104 can perform sensor alignment operations based on the output of sensor 160 (e.g., sensors 180, 182, 184, 186, 190), as further described below. The fusion module 113 aggregates the data received from the difference sensors after being transformed into, for example, vehicle or world coordinates. Aggregation may be time-based.

[0065] The online alignment module 104 determines whether the LIDAR sensor and / or other sensors are aligned, meaning that the differences in information provided by the LIDAR sensor and / or other sensors for the same one or more features and / or objects are within predetermined ranges from each other. The online alignment module 104 can determine the differences in the six degrees of freedom of the LIDAR sensor, including roll, pitch, yaw, x, y, and z differences, and determine whether the LIDAR sensor is aligned based on this information. The x-coordinate can refer to the lateral horizontal direction. The y-coordinate can refer to the forward and backward or longitudinal direction, and the z-coordinate can refer to the vertical direction. The x, y, and z coordinates can be interchanged and / or defined differently. If misalignment occurs, one or more LIDAR sensors can be recalibrated and / or an alignment process as further described below can be performed.

[0066] The perception module 115 can perform perception operations based on transformed and aggregated sensor data provided by the fusion module 113. The received data can be used to determine the corresponding master vehicle (e.g., Figure 1 The mapping and positioning module 117map performs GPS mapping and positioning operations, including real-time GPS positioning, which refers to providing GPS information about the current location of the main vehicle 102.

[0067] The path planning module 121 can determine the vehicle's path based on the output of the map drawing and positioning module 117. The path planning module 121 can perform autonomous vehicle operations, including controlling the vehicle 102 based on the determined path, including controlling the power steering system 109, the propulsion control module 108, the braking system 111, and / or... Figure 1Operation of other systems and equipment. This may include controlling the operation of actuator 210 and / or performing reporting information, generating alarm messages, alerting nearby vehicles, etc. Autonomous vehicle operation may include collision avoidance operations and / or information reporting operations.

[0068] The autonomous driving module 105 can operate in either offline or online mode. Offline mode refers to the time when the back office 204 collects data for the autonomous driving module 105 and performs data processing. This may include, for example, collecting GPS data from vehicle 102, performing GPS positioning correction and LiDAR alignment for data annotation, and providing the corrected GPS data and data annotations back to the autonomous driving module 105. The neural network of the autonomous driving module 105 can be trained based on the data annotations. GPS position correction can be performed before data annotation. Although... Figure 2 Not shown, but the control module 216 of server 214 may include one or more of modules 212, 104, 113, and / or perform operations similar to one or more of modules 212, 104 and / or 113.

[0069] During offline mode, server 214 processes data previously collected over an extended period. During online mode, autonomous driving module 105 performs GPS positioning correction and / or LiDAR alignment. This can be achieved with or without the aid of cloud-based network devices such as server 214. During online mode, autonomous driving module 105 performs real-time GPS positioning and LiDAR alignment using collected and / or historical data. This can include data collected from other vehicles and / or infrastructure devices. Cloud-based network devices can provide historical data, historical results, and / or perform other operations to assist in real-time GPS positioning and LiDAR alignment. Real-time GPS positioning refers to providing GPS information about the current location of the main vehicle. LiDAR alignment information is generated for the current state of one or more LiDAR sensors.

[0070] Modules 105, 212, 104, 113, 115, 117, and 121 can access memory 192. Memory 192 may include, for example, a raw data buffer 220, a target data buffer 222, a result transformation matrix buffer 224, and other buffers 226. The raw data buffer 220 may store data from sources such as... Figure 1 The raw data collected by the LIDAR sensor 180. Target data buffer 222 can store data associated with one or more selected targets. Result transformation matrix buffer 224 can store data used for... Figure 3-8 The transformation matrix is ​​determined by the method and algorithm.

[0071] Figure 3 An example alignment method is shown, which can be derived from... Figure 1-2 The online alignment module 104 is implemented. The alignment method may include... Figure 4-8 The methods and algorithms that can be derived from Figure 1-2 The system implementation involves performing an alignment method to determine the optimal alignment transformation matrix T. LtoV It is a 4×4 matrix. The transformation matrix is ​​a linear mapping that can be used for dynamic transformations from vehicle to LiDAR coordinates or from LiDAR to vehicle coordinates. The alignment method can be performed iteratively and can begin at 300. At 302, LiDAR data and positioning data can be loaded into the raw data buffer 220. The timestamp of the LiDAR data... i This can be represented by Equation 4. Positioning data can be represented by Equation 5 and can be provided by GPS sensor 190 and / or inertial measurement sensor 186. Sensors 190 and 186 provide vehicle orientation and translation information. Equation 5 provides the vehicle's ( V ) to the world ( W Transformation matrix T VtoW，i .

[0072] (4) ,in (5).

[0073] At 304, the online alignment module 104 can determine whether the vehicle 102 is turning. Turning (or steering) refers to when the vehicle turns left or right. This may include making left-hand and right-hand turns, U-turns, driving on roundabout entrance or exit ramps, etc. If the vehicle is turning, operation 306 is performed; otherwise, operation 302 is performed. This ensures that the vehicle is moving when alignment is performed. If the vehicle is not moving, alignment is not performed. In one embodiment, if the vehicle is not turning, alignment is not performed.

[0074] At 306, the online alignment module 104 selects and aggregates data associated with turning (or steering) events. World coordinates L W The aggregated LiDAR data in the equation can be represented by Equation 6, where X It is matrix multiplication. Furthermore, it selects a specific point based on the turning point, as described further below. Figure 3 The method is an optimization method that uses the transformation matrix T LtoV It begins with initial estimates and / or default values, and then includes optimization of the entries in the matrix.

[0075] (6).

[0076] At 308, the online alignment module 104 performs target selection based on the aggregated data. The online alignment module 104 determines a turning point and, based on this turning point, determines a selected area to be monitored. The perimeter of the selected area can be circular and at a predetermined distance from the turning point. Ground points are removed from the selected area, and one or more targets within the selected area are selected for monitoring. Ground points j Selected from Equation 7 is true. Ground points are removed, and then a data clustering algorithm, such as Density-Based Noisy Applied Spatial Clustering (DBSCAN), is applied to identify and select one or more targets. Each target (e.g., light pole, traffic sign, tree, etc.) is selected with the maximum number of points associated with the ground points.

[0077] At 310, the online alignment module 104 determines whether the target data buffer 222 is full. If so, operation 312 is executed; otherwise, operation 302 can be executed. Once sufficient target data exists (e.g., data associated with 3-10 targets), operation 312 is executed.

[0078] In step 312, the online alignment module 104 performs loss function optimization, which can be represented by equation 7. The online alignment module 104 searches for the optimal alignment transformation matrix T. LtoV This allows for the use of gradient-based variable step size optimization to minimize the loss function.

[0079] (7).

[0080] The loss function is optimized to minimize the distance of a point relative to a reference point of the target, and thus maximize the point density of each target, resulting in a sharpened image of one or more targets. This method can terminate at 314 after performing operation 312. The method described above provides the overall flow and includes (i) a dynamic target selection algorithm and (ii) a fast optimization algorithm.

[0081] Figure 4 Showing more details Figure 3 The alignment method can begin at 400. At 402, the online alignment module 104 loads LIDAR frames and inertial navigation system (INS) data from sensors 186 and 190, for example. The LIDAR frames and INS data can be loaded into a queue, for example, that of the autonomous driving module 105.

[0082] At 404, the online alignment module 104 accumulates N frames of data to construct the scan. At 406, the online alignment module 104 uses interpolation to calculate INS system data corresponding to the positioning data.

[0083] At 408, the online alignment module 104 determines whether the vehicle speed is greater than a predetermined speed threshold. If so, operation 410 is performed; otherwise, operation 402 is performed. At 410, the online alignment module 104 stores the LIDAR data and positioning data in the raw data buffer 220.

[0084] In 412, the online alignment module 104 operates as follows: Figure 5 The buffer management and data filtering algorithms (or methods) shown are illustrated. At 414, the online alignment module 104 operates as follows: Figure 6 The feature extraction and filtering algorithm shown is illustrated.

[0085] At 416, the online alignment module 104 accumulates data from multiple scenes. This accumulation of data from multiple different scenes (e.g., scenes associated with multiple turning events) aims to improve robustness. Each scene can be associated with one or more targets. At 418, the online alignment module 104 runs an optimization algorithm and stores the results in the result transformation matrix buffer 224.

[0086] At 420, the online alignment module 104 determines whether all raw data stored in the raw data buffer 220 is used in the current iteration of the method. If so, operation 422 is performed; otherwise, operation 402 is performed. At 422, the online alignment module 104 reports the alignment result. This may include reporting the transformation matrix. T LtoV For future use. The method can end at 424 after performing operation 422.

[0087] Figure 5 The illustration depicts an example buffer management and data filtering method. This method can be executed iteratively and can begin at 500. At 502, the online alignment module 104 converts the vehicle-to-world transformation data into 6-parameter data, which includes T... x T y T z Roll (α), pitch (β), and yaw (ɣ) data.

[0088] At 504, the online alignment module 104 appends LiDAR data, vehicle-to-world transformation data, and 6-parameter data to one or more buffers (e.g., one or more of raw data buffer 220 and other buffers 226). In one embodiment, three raw data buffers are used to store LiDAR data, vehicle-to-world transformation data, and 6-parameter data, respectively. At 506, the online alignment module 104 loads the last two buffered events corresponding to the 6-parameter data for each of the 6 parameters.

[0089] At 508, the online alignment module 104 determines whether the difference between the data values ​​of two consecutive events is greater than a predetermined difference threshold. If so, operation 510 is performed; otherwise, operation 502 is performed. Operations 506 and 508 are performed to perform a continuity check and detect discontinuities (or errors). For example, if the vehicle is moving, and the data from two consecutive events indicate that the vehicle is stationary or moving in the wrong direction, then a discontinuity and / or error exists. If a discontinuity and / or error exists, operation 510 is performed to remove the corresponding data record from one or more buffers storing the data record. If no discontinuity and / or error exists, operation 502 is performed.

[0090] At 512, the online alignment module 104 determines whether the amount of data stored in one or more buffers exceeds a maximum threshold. At 514, the online alignment module 104 removes the initial data point (or the oldest stored data point) from one or more buffers and sets the data ready flag to true. After performing operation 514, the method can end at 516.

[0091] Figure 6 The illustration shows an example feature extraction and filtering algorithm. This algorithm can be executed iteratively and can begin at 600. At 602, the online alignment module 104 determines whether the ready flag is true. If it is, operation 604 is performed; otherwise, the method can end at 624.

[0092] At 604, the online alignment module 104 calculates the yaw angle change during the buffer period. At 606, the online alignment module 104 determines whether the yaw angle change is greater than a predetermined angle threshold (e.g., 80° or 90°). If so, operation 608 is performed; otherwise, the method can end at 624.

[0093] At 608, the online alignment module 104 determines information about the steering position. This information may include the steering angle (e.g., a yaw angle change of 45°) and / or whether the steering exceeds a predetermined distance threshold (e.g., 20 meters).

[0094] At 610, the online alignment module 104 determines whether the steering position information is available. If yes, operation 612 is performed; otherwise, the method can end at 624. At 612, the online alignment module 104 calculates vehicle T for the sequentially collected data points. x and T y Changes in value.

[0095] At 614, if the online alignment module 104 detects discontinuities in the data and the changes exceed a predetermined threshold, the method can terminate at 624; otherwise, operation 616 can be performed. At 616, the online alignment module 104 aggregates a LiDAR scan of data within a predefined region for data selection. For example, it focuses on a region near a reference point x, y, such as within the range of x+20 to x+40 and y+30 to y+60. Operation 616 is performed for region selection, where a target is selected in subsequent operations. At 618, the online alignment module 104 performs target selection, as described below regarding... Figure 7 The method described herein.

[0096] At 620, the online alignment module 104 downsamples each data frame using an intensity filter and a random filter to minimize computation. At 622, the target data is saved to the target data buffer 222, the original data in one or more buffers is cleared (e.g., the last 800 frames are cleared), and the data ready flag is set to false. Setting the data ready flag to false causes the online alignment module 104 to wait for another predetermined amount of data to be collected in one or more buffers (e.g., in the original data buffer). After performing operation 622, Figure 6 The method can end at 624.

[0097] In the above method, a predetermined number of data frames (e.g., 800 frames) are collected in a first one or more buffers (raw data buffers). After a target is selected and the corresponding target data obtained from the frame data is stored in a second one or more buffers (target data buffers), the predetermined number of data frames (e.g., 800 frames) are cleared. A target is selected, and the target data is stored in the second one or more buffers until optimization is performed, at which point the second one or more buffers are cleared.

[0098] Figure 7 The illustration shows an example target selection algorithm. This algorithm can be executed iteratively and can begin at 700. At 702, the online alignment module 104 obtains the z-value for each LiDAR scan. Most LiDAR data points are ground points. For this reason, the calculation is performed in the z-direction.

[0099] At 704, the online alignment module 104 obtains a normal distribution of the LiDAR data to extract the average. At 706, the online alignment module 104 selects data around the average plus or minus a first threshold (or predetermined value). At 708, the online alignment module 104 obtains a normal distribution of the selected data to determine another average not based on ground points.

[0100] At 710, the online alignment module 104 extracts another average value plus or minus a second threshold (or predetermined value) around the ground points. This can be implemented using Equation 8. The ground points are identified from the LIDAR point cloud and removed in the vertical dimension z, where ε is the threshold for the average value of the selected data points.

[0101] (8).

[0102] In 712, the online alignment module 104 marks other data points as target data.

[0103] At 714, the online alignment module 104 determines whether the amount of target data stored in the target data buffer 222 is greater than a third threshold. If so, the method can end at 724; otherwise, operation 716 can be performed.

[0104] At 716, the online alignment module 104 performs clustering on the target data (e.g., performs DBSCAN to find clusters). At 718, the online alignment module 104 selects the N targets with the most points. At 720, the online alignment module 104 selects ground points based on predefined spatial filters.

[0105] At 722, the online alignment module 104 aggregates all selected target data and ground data, and saves the aggregated data to the target data buffer 222. After performing operation 720, the method can terminate at 724.

[0106] Figure 8 The diagram illustrates the optimized alignment result (or transformation matrix T). LtoV An example optimization algorithm is provided. This can be achieved by reducing and minimizing the loss, such as that calculated using Equation 8. The algorithm can be executed iteratively and can start at 800. At 802, the online alignment module 104 loads 5 optimization parameters (Tx, Ty, α, β, ɣ) and the alignment matrix T. LtoV The initial estimate is set, and the step size is set to 0. Parameter Tz is not loaded.

[0107] At 804, the online alignment module 104 determines whether the step size value is greater than a predetermined step size threshold. If so, the method can end at 826; otherwise, operation 806 is executed.

[0108] In 806, the online alignment module 104 adjusts the alignment based on translation (e.g., 0.1 meters or The alignment result can be increased or decreased by changing one or more of the five parameters (Tx, Ty, α, β, ɣ) by 0.2 degrees.

[0109] At 808, the online alignment module 104 evaluates the loss function. This evaluation may include: (i) using the current alignment result (T) LtoV Calculate the LiDAR to world transformation matrix T LtoW (ii) Project the aggregated LIDAR data onto world coordinates (T). VtoW,I XT LtoV XL i (iii) Apply the KD-tree to the world coordinate data; (iv) for each point, calculate the distance to neighboring points; and (v) use the average of all distances as the loss (or loss value) provided by the loss function, similar to Equation 8. The KD-tree is used to organize data and quickly determine distances. A KD-tree is a binary tree where each node is a K-dimensional point. In 810, if the loss decreases, the online alignment module 104 updates the alignment result. If the loss decreases (i.e., improves), the changes made in 806 are maintained.

[0110] In step 812, the online alignment module 104 determines whether the alignment result changes in the same direction for the last two iterations of the method. For example, the last two iterations may have the same increase or decrease in one or more parameters.

[0111] At 814, the online alignment module 104 determines whether the change is within a first predefined range. If so, operation 816 is performed; otherwise, operation 818 is performed. At 816, the online alignment module 104 increases the change made at 806. For example, this change could increase by 0.2 meters and / or 0.4 degrees. At 818, the online alignment module 104 maintains the change made at 806.

[0112] At 820, the online alignment module 104 determines whether the change is within a second predefined range. If so, operation 822 is performed; otherwise, operation 818 is performed. At 822, the online alignment module 104 reduces the change made at 806. For example, the change may be reduced by 0.5 meters and / or 0.1 degrees.

[0113] At 824, the online alignment module 104 determines whether the change in loss is less than a predetermined loss threshold. If so, the method can end at 826; otherwise, operation 804 is executed.

[0114] Figure 8 The optimization algorithm (or method) includes searching for the optimal LiDAR-to-vehicle transformation matrix T. LtoVIt utilizes gradient-based variable step-size optimization to minimize the loss function. The optimization algorithm can additionally or alternatively implement stochastic gradient descent (SGD), root mean square propagation and adaptive gradient algorithm (or adaptive momentum (ADAM) algorithm), particle swarm optimization (PSO), and general algorithm (GA) to optimize the alignment result (or transformation matrix T). LtoV ).

[0115] exist Figure 8 One or more different loss functions can be used to characterize point density in the method. Increasing the point density of the selected target improves image sharpness and the accuracy of synthetic decisions. Actions are performed based on the synthetic image. As a first example, a first density characterization method can be implemented, which includes, for each selected data point, determining the count of how many other points are within a predetermined radius (e.g., 0.5 meters) of the selected point. Figure 9 An example plot of the loss versus translation distance using the first density characterization method is shown. As another example, a second density characterization method can be implemented, which involves calculating the average distance of the 20 nearest points for each point. This could include using Equation 8. Figure 10 An example plot of the loss versus translation distance using the second density representation method is shown. As a third example, a third density representation method can be implemented, which involves, for each point, using point cloud alignment (PCA) on nearby points to compute the sum of three eigenvalues.

[0116] Figure 3-8 The above operations are intended as illustrative examples. Depending on the application, operations may be performed sequentially, synchronously, simultaneously, continuously, during overlapping time periods, or in different orders. Furthermore, depending on the implementation method and / or the order of events, some operations may not be performed or may be skipped.

[0117] The foregoing description is merely illustrative in nature and is by no means intended to limit this disclosure, its application, or its use. The broad teachings of this disclosure can be implemented in many forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become clear upon examination of the drawings, specification, and appended claims. It should be understood that one or more steps within the method may be performed in a different order (or simultaneously) without altering the principles of this disclosure. Furthermore, although each embodiment is described above as having certain features, any one or more of those features described with respect to any embodiment of this disclosure may be implemented in and / or combined with features of any other embodiment, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and substitutions of one or more embodiments for each other remain within the scope of this disclosure.

[0118] Spatial and functional relationships between components (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including “connection,” “joint,” “coupled,” “adjacent,” “next to,” “on top,” “above,” “below,” and “set.” Unless explicitly described as “direct,” when describing the relationship between first and second components in the above disclosure, the relationship can be a direct relationship where no other intervening components exist between the first and second components, but it can also be an indirect relationship (spatial or functional) between the first and second components with one or more intervening components. As used herein, at least one of the phrases A, B, and C should be interpreted as meaning logical (A “or”, B “or”, C) using the non-exclusive logical “or,” and should not be interpreted as meaning “at least one of A, at least one of B, and at least one of C.”

[0119] In a diagram, the direction of the arrows, as indicated by the arrows, typically indicates the flow of information (such as data or instructions) of interest to the diagram. For example, when components A and B exchange various types of information, but the information transmitted from component A to component B is relevant to the diagram, the arrow can point from component A to component B. This unidirectional arrow does not imply that no other information is transmitted from component B to component A. Furthermore, for information sent from component A to component B, component B can send a request for or confirmation of receipt of that information to component A.

[0120] In this application, including the following definitions, the term "module" or "controller" may be replaced by the term "circuit". The term "module" may refer to or include some of the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the aforementioned functionality; or combinations of some or all of the above, such as in a system-on-a-chip.

[0121] This module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module in this disclosure may be distributed across multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0122] The term "code" as used above can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" includes a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" includes processor circuitry combined with additional processor circuitry to execute some or all of the code from one or more modules. References to multiple processor circuitry include multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" includes a single memory circuitry that stores some or all of the code from multiple modules. The term "group memory circuit" includes memory circuitry combined with additional memory to store some or all of the code from one or more modules.

[0123] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); therefore, the term "computer-readable medium" can be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0124] The apparatus and methods described in this application can be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The function blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs by the routine work of a skilled technician or programmer.

[0125] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may include a basic input / output system (BIOS) for interacting with the hardware of a special-purpose computer, device drivers for interacting with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0126] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code executed by an interpreter; (v) source code compiled and executed by a real-time compiler, etc. As an example only, source code may be written using syntax from languages ​​including: C, C++, C#, Objective C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language, Fifth Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.

Claims

1. A LiDAR-to-vehicle alignment system, comprising: The memory is configured to store data points based on the output and positioning data of one or more LiDAR sensors; The alignment module is configured to perform the alignment process, including... Based on location data, determine whether the main vehicle is turning. In response to the main vehicle's steering, a subset of data points is selected and aggregated to provide aggregated data. Target selection based on aggregated data, and Based on the selected objective, the loss value of the loss function is iteratively reduced to provide a synthetic LIDAR-to-vehicle transformation matrix; and The autonomous driving module is configured as follows: Based on the synthetic LiDAR-to-vehicle transformation matrix, at least a selected portion of the data points is transformed to at least one of vehicle coordinates or world coordinates to provide synthetic data, and Perform one or more autonomous driving operations based on synthetic data.

2. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to receive positioning data from one or more of a global position sensor or an inertial measurement sensor.

3. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to suppress the execution of data selection and aggregation, as well as target selection, when the master vehicle is not turning.

4. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to select data points and targets based on the turning point.

5. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to select an area based on a turning point, remove ground points based on the selected area, identify multiple targets, and perform a data clustering algorithm to select one of the multiple targets for monitoring.

6. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to accumulate LiDAR data from multiple scenes and minimize the loss value based on the LiDAR data from the multiple scenes.

7. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to perform a continuity check to determine which data points to select and which to discard.

8. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to: LiDAR scans that aggregate data in a predefined space; Data-driven clustered LiDAR scanning is used to perform target selection. and After target selection is performed, each frame of data associated with the aggregated LiDAR scan of the data is downsampled using intensity and random filters to provide synthetic target data.

9. The LIDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to: Obtain the z-value of each of multiple LiDAR scans; Obtain the normal distribution of the aggregated data associated with the LiDAR scan to extract the first mean of the aggregated data associated with the LiDAR scan; Select data around the first average; Obtain another normal distribution of selected data around the first mean to obtain a second mean that is not based on the ground point; Extract ground points around the second average value; Mark data other than ground points as target data; and Once a predetermined amount of target data has been collected, a clustering algorithm is executed to select a subset of the target data.

10. The LIDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to, when iteratively reducing the loss value: Load the optimized parameters and the initially estimated LIDAR to the vehicle transformation matrix; and Iterative The initial estimated LiDAR-to-vehicle transformation matrix or the final determined LiDAR-to-vehicle transformation matrix can be changed to provide a synthesized LiDAR-to-vehicle transformation matrix. Evaluate the changes in the synthetic LiDAR to the vehicle transformation matrix, including determining the loss value, and Based on the loss value, determine whether to perform another iteration.

11. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The alignment module is configured to, when evaluating the loss function: The LiDAR to world transformation matrix is ​​calculated using the synthesized LiDAR to vehicle transformation matrix. The aggregated data is projected onto world coordinates based on the LIDAR-to-world transformation matrix. Applying binary trees to world coordinate data; Distance is provided by calculating the distance to neighboring data points for each LIDAR data point; and The loss value is determined based on the average of the multiple distances.

12. The LiDAR-to-vehicle alignment system according to claim 1, wherein, The loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets.

13. An autonomous vehicle driving method, comprising: Stores data points based on the output and positioning data of one or more LiDAR sensors; Performing the LIDAR-to-vehicle alignment process includes Based on location data, determine whether the main vehicle is turning. In response to the main vehicle's steering, a subset of data points is selected and aggregated to provide aggregated data. Target selection based on aggregated data, and Based on the selected objective, the loss value of the loss function is iteratively reduced to provide the synthetic LIDAR to vehicle transformation matrix; Based on the synthetic LiDAR to vehicle transformation matrix, at least a selected portion of the data points is transformed to at least one of vehicle coordinates or world coordinates to provide synthetic data; as well as Perform one or more autonomous driving operations based on synthetic data.

14. The autonomous vehicle driving method according to claim 13, further comprising: Select the region based on the turning point; Remove ground points based on the selected area; Identify multiple targets; and A data clustering algorithm is executed to select one of the plurality of targets for monitoring.

15. The autonomous vehicle driving method according to claim 13, further comprising: Perform a continuity check to determine which data points to select and which to discard; and Accumulate LiDAR data from multiple scenarios and minimize the loss value based on the LiDAR data from multiple scenarios.

16. The autonomous vehicle driving method according to claim 13, further comprising: LiDAR scans that aggregate data in a predefined space; Data-driven clustered LiDAR scanning is used to perform target selection. and After target selection is performed, each frame of data associated with the aggregated LiDAR scan of the data is downsampled using intensity and random filters to provide synthetic target data.

17. The autonomous vehicle driving method according to claim 13, further comprising: Obtain the z-value of each of multiple LiDAR scans; Determine at least one average value based on aggregated data associated with LIDAR scans; Ground points are extracted based on the at least one average value; Mark data other than ground points as target data; and Once a predetermined amount of target data has been collected, a clustering algorithm is executed to select a subset of the target data.

18. The autonomous vehicle driving method according to claim 13, further comprising, when iteratively reducing the loss value: Load the optimized parameters and the initially estimated LiDAR to vehicle transformation matrix; and Iterative The initial estimated LiDAR-to-vehicle transformation matrix or the final determined LiDAR-to-vehicle transformation matrix can be changed to provide a synthesized LiDAR-to-vehicle transformation matrix. Evaluate the changes in the synthetic LiDAR to the vehicle transformation matrix, including determining the loss value, and Based on the loss value, determine whether to perform another iteration.

19. The autonomous vehicle driving method according to claim 13, further comprising, when evaluating the loss function: The LiDAR to world transformation matrix is ​​calculated using the synthesized LiDAR to vehicle transformation matrix. Based on the LIDAR-to-world transformation matrix, aggregated data is projected onto world coordinate data; Distance is provided by calculating the distance to neighboring data points for each LiDAR data point, based on world coordinate data. and The loss value is determined based on the average distance.

20. The autonomous vehicle driving method according to claim 13, wherein, The loss function is a density-based loss function that characterizes at least one of the severity of LIDAR misalignment or point density relative to one of the selected targets.

Images