Perception data geometry calibration system

Abstract

A perception data geometry calibration system calibrates perception data geometry captured by one or more vehicles located within a predefined geographic area. In one embodiment, the perception data geometry system includes one or more central computers that receive, over a communication network, local map data and corresponding high definition map data from a particular vehicle representing a predefined geographic area. The local map data includes a plurality of perception data geometry features defining objects within an environment surrounding the particular vehicle. The one or more central computers determine a unique calibration scaling ratio for each segment that is part of a vehicle trajectory based on one or more non-linear optimization algorithms by minimizing a cost function between a road segment point representing a particular perception data geometry feature based on the local map data and a corresponding road segment point based on the high definition map data.

Description

Technical Field

[0001] This disclosure relates to a perception data geometry calibration system for calibrating perception data geometry captured by one or more vehicles. Background Technology

[0002] Autonomous vehicles perform a variety of tasks, including but not limited to perception, localization, mapping, path planning, decision-making, and motion control. Autonomous vehicles rely on map data to perform many tasks, such as localization, mapping, and path planning. One example of a version of map data is based on perception data collected by the vehicle's perception sensors. A standalone autonomous vehicle may include many perception sensors, such as a front-facing camera module, radar, and lidar. The front-facing camera module can capture image data representing the environment surrounding the specific vehicle. This image data can be used to extract perception data geometry representing the positions of various objects within the environment, such as lanes, road signs, and traffic lights on a road.

[0003] It should be understood that inaccurate perception geometry extracted from image data captured by the front-facing camera module can lead to map generation problems. Specifically, for example, the spacing of lane lines extracted from image data captured by the front-facing camera module may be narrower or wider than the actual lane lines in reality. Some possible causes of inaccurate perception geometry data include, but are not limited to, inaccurate factory calibration of the front-facing camera module, changes in camera height due to events such as tire pressure or suspension variations, and the presence of road cambers to channel water to the road edges or drainage ditches. Inaccurate perception geometry data can lead to incorrect lane line placement in map data determined based on the perception data. Incorrect lane line placement can cause localization problems and lead to incorrect decisions by the control system of autonomous vehicles. Furthermore, if perception data is crowdsourced with perception data captured by other nearby vehicles, even if the position of one lane line is inaccurate, the aggregated lane line map error will accumulate across multiple lanes as part of a multi-lane highway.

[0004] One way to mitigate the aforementioned problem of inaccurate perception of geometry is to use a scaling ratio to adjust the width between lane lines captured by the front camera module. However, the scaling ratio cannot be used for multiple vehicles of different brands and models. In fact, even when only one vehicle is involved, the value of the scaling ratio may change over time.

[0005] Therefore, while maps used for autonomous vehicles have achieved their intended purpose, there is still a need in the art for an improved method for calibrating conversion ratios to improve the accuracy of perception data. Summary of the Invention

[0006] According to several aspects, a perception data geometry calibration system is disclosed that calibrates perception data geometry captured by one or more vehicles located within a predefined geographic area. The perception data geometry calibration system includes one or more central computers that wirelessly communicate with one or more vehicles and one or more communication networks for receiving high-resolution map data. The one or more central computers execute instructions to receive, via the communication networks, local map data representing a predefined geographic area and corresponding high-resolution map data from a specific vehicle, wherein the local map data includes multiple perception data geometric features defining objects within the environment surrounding the specific vehicle. The one or more central computers divide a vehicle trajectory determined based on perception data collected from the specific vehicle into multiple road segments. The one or more central computers determine a unique calibration conversion ratio for each road segment that is part of the vehicle trajectory based on one or more nonlinear optimization algorithms, wherein the one or more central computers iteratively compute the unique calibration conversion ratio to minimize a cost function between road segment points representing specific perception data geometric features based on the local map data and corresponding road segment points based on the high-resolution map data, and calibrate the multiple perception data geometric features included in the local map data based on the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory to create a calibrated local map.

[0007] On the other hand, one or more central computers execute instructions to evaluate the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory, in order to identify any outliers based on one or more outlier removal methods.

[0008] On the other hand, outlier removal methods include one or more of the following: Random Sample Consensus (RANSAC) algorithm, z-score algorithm, and density-based applied spatial clustering with noise (DBSCAN) algorithm.

[0009] On one hand, one or more central computers execute instructions to determine a unique calibration conversion ratio by finding the association between a road segment point and a corresponding road segment point, wherein the road segment point represents a specific perceptual data geometric feature within the local map data, and the corresponding road segment point also represents a specific perceptual data geometric feature of each road segment as part of the vehicle trajectory within the high-definition map data.

[0010] On the other hand, one or more central computers execute instructions to determine a unique calibration conversion ratio by calculating a cost function based on the Euclidean distance between each road segment point within the local map data and the corresponding road segment point based on high-definition map data of each road segment as part of the vehicle trajectory.

[0011] The cost function is determined based on the following:

[0012]

[0013] Where, p i ,q i Let p represent a pair of related points. i For HD map data map The point, q i For local map data map The point, p atti ,q atti This indicates the relationship between the two related points p. i ,q i The associated geometric features of the perceived data are attributes, where n represents the total number of associated point pairs, and cost_function(HD_map,local_map) represents the cost function.

[0014] On one hand, the geometric features of the perceived data include one or more of the following: lane lines, road signs, traffic lights, and location coordinates on the road.

[0015] On the other hand, the geometric features of the perceived data are lane lines, and the attributes include one or more of the following: line color and line type.

[0016] On the other hand, the length of each road segment is equal to the average accurate perception range of a particular vehicle.

[0017] In one aspect, a perception data geometry calibration system is disclosed, which calibrates perception data geometry captured by one or more vehicles located within a predefined geographic area. The perception data geometry calibration system includes: multiple perception sensors that capture collected perception data about the predefined geographic area; and one or more controllers in electronic communication with the multiple perception sensors, wherein the one or more controllers receive high-resolution map data for a specific vehicle portion via a communication network. The one or more controllers execute instructions to determine local map data based on the perception data collected by the multiple perception sensors, wherein the local map data includes multiple perception data geometric features defining objects within the environment surrounding the specific vehicle. The one or more controllers divide a vehicle trajectory determined based on the perception data collected from the specific vehicle into multiple road segments and determine a unique calibration conversion ratio for each road segment as part of the vehicle trajectory based on one or more nonlinear optimization algorithms, wherein the one or more controllers iteratively compute the unique calibration conversion ratio to minimize a cost function between road segment points representing specific perception data geometric features based on the local map data and corresponding road segment points based on the high-resolution map data. The one or more controllers calibrate the multiple perception data geometric features included in the local map data based on the unique calibration conversion ratio corresponding to each road segment as part of the vehicle trajectory to create a calibrated local map.

[0018] On the other hand, one or more controllers execute instructions to evaluate the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory, in order to identify any outliers based on one or more outlier removal methods.

[0019] In another aspect, a perception data geometry calibration system is disclosed, which calibrates the perception data geometry captured by two or more vehicles located within a predefined geographic area. The perception data geometry calibration system includes one or more central computers that wirelessly communicate with the two or more vehicles. The one or more central computers execute instructions to receive a first set of perception data representing a predefined geographic area captured by a first vehicle and a second set of perception data representing a predefined geographic area captured by a second vehicle. The one or more central computers create a first point cloud map representing the predefined geographic area based on the first set of perception data and a second point cloud map representing the predefined geographic area based on the second set of perception data, wherein the first point cloud map and the second point cloud map include multiple perception data geometric features defining objects within the environment surrounding the first and second vehicles. The one or more central computers align the first point cloud map and the second point cloud map relative to a world coordinate system based on Global Navigation Satellite System (GNSS) data collected from the corresponding vehicles. The one or more central computers divide the vehicle trajectories determined based on the perception data collected from the first and second vehicles into multiple road segments. One or more central computers determine a unique calibration conversion ratio for each road segment that is part of a vehicle trajectory based on one or more nonlinear optimization algorithms. The one or more central computers iteratively compute the unique calibration conversion ratio to minimize the cost function between point cloud data points representing specific perceptual data geometry features based on a first point cloud map and corresponding point cloud data points based on a second point cloud map. The multiple perceptual data geometry features included in the first point cloud map are calibrated based on the unique calibration conversion ratio corresponding to each road segment that is part of a vehicle trajectory to create a calibrated first point cloud map.

[0020] On the other hand, one or more central computers execute instructions to evaluate the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory, in order to identify any outliers based on one or more outlier removal methods.

[0021] On the other hand, outlier removal methods include one or more of the following: RANSAC algorithm, z-score algorithm, and DBSCAN algorithm.

[0022] On one hand, one or more central computers execute instructions to determine a unique calibration conversion ratio by finding the association between the point cloud data point and the corresponding point cloud data point, wherein the point cloud data point represents a specific perceptual data geometric feature within a first point cloud map, and the corresponding point cloud data point also represents a specific perceptual data geometric feature of each road segment as part of the vehicle trajectory within a second point cloud map.

[0023] On the other hand, one or more central computers execute instructions to calculate a cost function to determine a unique calibration conversion ratio by means of the Euclidean distance between each point cloud data point in a first point cloud map and the corresponding point cloud data point in a second point cloud map based on each road segment as part of the vehicle trajectory.

[0024] On another front, the cost function is determined based on the following:

[0025]

[0026] Where, p i ,q i Let p represent a pair of related points. i For the second point cloud map v2_local map The point, q i For the first point cloud map v1_local map The point, p atti ,q atti This indicates the relationship between the two points p. i ,q i The associated geometric features of the perceived data are attributes, where n represents the total number of associated point pairs, and cost_function(v1_local_map,v2_local_map) represents the cost function.

[0027] On the other hand, the geometric features of the perceived data include one or more of the following: lane lines, road signs, traffic lights, and location coordinates on the road.

[0028] On the other hand, the geometric features of the perceived data are lane lines, and the attributes include one or more of the following: line color and line type.

[0029] On the one hand, the length of each road segment is equal to the average accurate perception range of the first or second vehicle.

[0030] Further areas of application will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0031] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0032] Figure 1 This is a schematic diagram of a disclosed perception data geometry calibration system according to an exemplary embodiment, the perception data geometry calibration system including one or more central computers that communicate wirelessly with a plurality of vehicles and a wireless network for receiving high-definition map data;

[0033] Figure 2 This illustrates an exemplary embodiment. Figure 1 A block diagram illustrating one embodiment of the software architecture of one or more central computers;

[0034] Figure 3 It is a graph representing road segment points with specific perceptual data geometric features within an environment and corresponding road segment points based on high-definition map data, according to an exemplary embodiment.

[0035] Figure 4 This illustrates an exemplary embodiment. Figure 1 A block diagram illustrating yet another embodiment of the software architecture of one or more central computers; and

[0036] Figure 5 It is a map, according to an exemplary embodiment, representing road segment points of specific perceived data geometric features within the environment captured by a first vehicle and corresponding road segment points captured by a second vehicle. Detailed Implementation

[0037] The following description is merely exemplary in nature and is not intended to limit this disclosure, application, or use.

[0038] refer to Figure 1 An exemplary perception data geometry calibration system 10 is illustrated. The perception data geometry calibration system 10 includes one or more central computers 20 located in a back-end office 22, which wirelessly communicate with one or more controllers 30, each controller 30 corresponding to one of a plurality of vehicles 24. It should be understood that each of the plurality of vehicles 24 can be any type of vehicle, such as, but not limited to, a sedan, truck, SUV, van, or motorhome. Figure 1 As shown, multiple vehicles 24 are located within a predefined geographic area 32. The predefined geographic area 32 can represent any geographic region, such as a block, city, town, or state, such as Michigan or Ohio. In one embodiment, one or more central computers 20 can also obtain high-resolution (HD) map data representing the predefined geographic area 32 via one or more communication networks 28. map .

[0039] In such Figure 1In the non-limiting embodiment shown, each vehicle 24 includes a plurality of sensing sensors 34 that collect perception data about a predefined geographic area 32, wherein the sensing sensors 34 are in electronic communication with one or more controllers 30. The one or more controllers 30 of each vehicle 24 are in wireless communication with one or more central computers 20 and with one or more controllers 30 corresponding to one or more remaining vehicles 24. Figure 1 As shown, each vehicle 24's plurality of sensing sensors 34 includes one or more cameras 36 for collecting image data, an inertial measurement unit (IMU) 38, a global navigation satellite system (GNSS) 40, a radar 42, and a lidar 44. However, it should be understood that different or additional sensors may also be used. One or more cameras 36 collect image data representing a predefined geographic area 32. In a non-limiting embodiment, one or more cameras 36 may include a front-facing camera module.

[0040] Each vehicle 24 determines local map data representing a predefined geographic area 32 based on the perception data 34 collected by its respective perception sensors. map One or more controllers 30 of each vehicle 24 can transmit local map data representing a predefined geographic region 32 via communication network 28. map Send to one or more central computers 20.

[0041] It should be understood that the local map data representing the predefined geographic region 32 includes multiple perceptual data geometric features of objects within the environment surrounding a particular vehicle 24, wherein the perceptual data geometric features are determined based on image data captured by one or more cameras 36. Some examples of perceptual data geometric features include, but are not limited to, lane lines along roads, road signs, traffic lights, and location coordinates, such as two-dimensional or three-dimensional coordinates indicating latitude and longitude, such as x, y, z coordinates.

[0042] Figure 2 It is shown Figure 1 A block diagram illustrating one embodiment of the software architecture of one or more central computers 20. Figure 2 In the example shown, one or more central computers 20 include a segmentation module 50, a nonlinear optimization module 52, and an outlier removal module 54. (See reference) Figure 1 and Figure 2 One or more segmented modules 50 of the central computer 20 receive local map data representing a predefined geographic region 32 from a specific vehicle 24, which is part of a plurality of vehicles 24, via a communication network 28. map and HD map data mapIn one embodiment, alternatively, one or more central computers 20 may receive sensing data collected by various sensing sensors 34 from a specific vehicle 24, and then calculate local map data representing a predefined geographic region 32 based on the sensing data collected by the specific vehicle 24. map In one embodiment, one or more central computers 20 receive local map data from more than one vehicle 24. map Or the sensing data collected by the corresponding sensing sensor 34.

[0043] As described below, one or more central computers 20 calculate the calibrated conversion ratio α. In such cases... Figure 2 In the illustrated embodiment, the calibrated conversion ratio α is based on local map data. map and HD map data map The differences between the geometric features of the corresponding sensing data within the data, including high-definition map data (HD). map This is considered to be ground-based real data. It should be understood that the calibration conversion ratio α is a function of time, the position of a specific vehicle 24, the lateral distance between the specific perceived data geometry and the specific vehicle 24, the brand and model of the specific vehicle 24, road curvature, lane line color, lane line quality, and the estimated horizontal position error (EHPE). Once the calibration conversion ratio α is determined, one or more central computers 20 can calibrate the local map data based on the calibration conversion ratio. map The included geometric features of multiple sensory data. Then, the calibrated local map data... map It can be used in a variety of applications. For example, calibrating local map data. map It can be teleported back to a specific vehicle 24.

[0044] It should be understood that, although Figure 2 The software architecture of one or more central computers 20 is shown; one or more controllers 30 of the vehicle 24 may also include a similar architecture. In other words, it should be understood that, although Figure 2 One or more central computers 20 are shown that determine the calibration conversion ratio α. In another embodiment, one or more controllers 30 of one of the vehicles 24 may alternatively determine the calibration conversion ratio α. In this embodiment, one or more controllers 30 of the vehicle receive high-definition map data HD via a communication network 28. map .

[0045] refer to Figure 1 and Figure 2The segmentation module 50 of one or more central computers 20 divides the vehicle trajectory determined based on perception data collected from a specific vehicle 24 into multiple segments. Each of the multiple segments comprises the same length, wherein the length of each segment may be equal to the average accurate perception range of the specific vehicle 24. In a non-limiting embodiment, the length is approximately one hundred meters.

[0046] One or more central computer 20 nonlinear optimization modules 52 receive multiple road segments and high-resolution map data (HD) representing predefined geographic areas 32 from segmentation module 50 via one or more communication networks 28. map The nonlinear optimization module 52 determines a unique calibration conversion ratio α for each road segment as part of the vehicle trajectory based on one or more nonlinear optimization algorithms, wherein the nonlinear optimization module 52 of one or more central computers 20 iteratively calculates the unique calibration conversion ratio α to minimize the road segment point 80 ( Figure 3 (as shown) and corresponding road segment point 82 ( Figure 3 The cost function between ) and road segment point 80 represents the cost based on local map data. map Specific perceptual data geometric features within a predefined geographic area 32, corresponding to road segment points 82 ( Figure 3 Based on high-definition map data HD map An example of a nonlinear optimization algorithm that can be used is the Levenberg-Marquardt algorithm. For example... Figure 2 As shown, the nonlinear optimization module 52 of one or more central computers 20 includes a conversion ratio submodule 60, an association submodule 62, a cost function submodule 64, and a nonlinear optimization submodule 66.

[0047] refer to Figure 2 and Figure 3 The conversion ratio submodule 60 of the nonlinear optimization module 52 first selects a temporary conversion ratio α′ corresponding to a specific road segment that is part of the vehicle trajectory. In a non-limiting embodiment, the temporary conversion ratio α′ is a predetermined value ranging from 0.9 to 1.2; however, it should be understood that other values ​​of the temporary conversion ratio α′ may also be used. The association submodule 62 of the nonlinear optimization module 52 can then find the road segment point 80 ( Figure 3 The association between (as shown) and the corresponding road segment point 82, where road segment point 80 represents local map data. map Specific perceptual data geometric features within the area, corresponding to road segment point 82 in high-definition map data HD map The inner part also represents the specific geometric characteristics of the sensing data for that particular road segment.

[0048] For details, please refer to the following: Figure 3Draw a circle 84 around each road segment point 80 and its corresponding road segment point 82. It should be understood that each road segment may include more than one road segment point 80 and is based on high-resolution map data (HD). map The corresponding road segment point 82. Road segment point 80 and HD based on high-definition map data. map The association between the corresponding road segment points 82 can be determined based on a variety of methods, such as point cloud registration or feature matching.

[0049] Once the relationship between road segment point 80 and corresponding road segment point 82 for a specific road segment is determined, the cost function submodule 64 of the nonlinear optimization module 52 can be based on local map data. map Each road segment point within the area is matched with high-definition map data (HD) based on specific road segments. map The cost function is calculated based on the Euclidean distance between the corresponding road segment points 82. A unique calibration conversion ratio α corresponding to a specific road segment is selected to minimize the cost function. Specifically, in one embodiment, the cost function is determined based on Equation 1, which is:

[0050]

[0051] Where, p i ,q i Let p represent a pair of related points. i For HD map data map The point, q i For local map data map The point, p atti ,q atti This indicates the relationship between the two related points p. i ,q i The associated geometric features of the perceived data are attributes, where n represents the total number of associated point pairs. In an embodiment where the perceived data geometry is lane lines, attributes may include features such as line color and line type (e.g., solid line, dashed line, etc.). In another embodiment where the perceived data feature is a road sign, attributes may include color and type (e.g., speed limit, direction, etc.).

[0052] Once the cost function is calculated, the nonlinear optimization submodule 66 of the nonlinear optimization module 52 can calculate a unique calibration conversion ratio α corresponding to a specific road segment based on one or more nonlinear optimization algorithms to minimize the cost function. In response to determining that the cost function has been minimized, the nonlinear optimization submodule 66 of the nonlinear optimization module 52 can then transmit the unique calibration conversion ratio α corresponding to the specific road segment to the outlier removal module 54. However, in response to determining that the cost function has not been minimized, the nonlinear optimization submodule 66 of the nonlinear optimization module 52 can then transmit the unique calibration conversion ratio α corresponding to the specific road segment to the conversion ratio submodule 60 of the nonlinear optimization module 52. The nonlinear optimization module 52 can then iteratively calculate the unique calibration conversion ratio α until the cost function is minimized. Minimization can be achieved by applying a conversion ratio to local map feature points to reduce the offset between the perceptual data geometric feature points from the local map data and the corresponding geometric feature points from the high-resolution (ground reality) map data. In one embodiment, the conversion ratio submodule 60 of the nonlinear optimization module 52 can calculate the unique calibration conversion ratio α corresponding to the specific road segment based on Equation 2, which is:

[0053] α = argmin α (cost_function(HD map ,local map Equation 2

[0054] refer to Figure 2 Once the cost function for each road segment as part of the vehicle trajectory is minimized, the outlier removal module 54 of one or more central computers 20 can receive a unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory. The outlier removal module 54 can evaluate the unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory to identify any outliers based on one or more outlier removal methods. Some examples of outlier removal methods include, but are not limited to, the Random Sample Consensus (RANSAC) algorithm, the z-score algorithm, and the Density-Based Applied Spatial Clustering with Noise (DBSCAN) algorithm. Outliers are removed from the vehicle trajectory, and non-outliers can be used to update the local map data.

[0055] Once a unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory is determined, one or more central computers 20 can calibrate the local map data based on the unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory. map The included multiple perceptual data geometric features, and the corrected local map data. map Retransmitted back to the specific vehicle 24.

[0056] Return to reference Figure 1 It should be understood that, in some cases, high-resolution map data (HD) representing a predefined geographic region 32 is used. map It may not be available. Therefore, Figure 4 This is an illustration of an embodiment of one or more central computers 20 used to determine a unique calibration conversion ratio α based on differences between geometric features of corresponding perceived data located within local map data created based on perceived data corresponding to two or more vehicles 24 within a predefined geographic area 32. It should be understood that the two or more vehicles 24 are each located in different lanes along a road. As described above, although... Figure 4 The software architecture of one or more central computers 20 is shown, and one or more controllers 30 of vehicle 24 may also include a similar architecture.

[0057] refer to Figure 1 and Figure 4 The central computer 20 includes a point cloud module 120, an alignment module 122, a precise positioning module 124, a nonlinear optimization module 126, and an outlier removal module 128. The point cloud module 120 of one or more central computers 20 receives a first set of perception data representing a predefined geographic region 32 captured by perception sensors 34 of a first vehicle 24, which is part of a plurality of vehicles 24, and a second set of perception data representing a predefined geographic region 32 captured by perception sensors 34 of a second vehicle 24, which is also part of a plurality of vehicles 24, located in two separate lanes of a road.

[0058] One or more central computer 20 point cloud modules 120 create a first point cloud map v1_local representing a predefined geographic region 32 based on the first set of sensing data. map And represents the second point cloud map v2_local representing a predefined geographic region 32 based on the second set of sensing data. map It should be understood that the first point is cloud map v1_local. map Second point cloud map v2_local map All are based on arbitrary coordinate systems. First point cloud map v1_local map And the first point cloud map v1_local map This includes multiple perceptual data geometric features of objects within the environment surrounding the first vehicle 24 and the second vehicle 24.

[0059] One or more central computer 20 alignment modules 122 receive a first point cloud map v1_local from point cloud module 120. map Second point cloud map v2_localmap And based on GNSS40 ( Figure 1 The GNSS data collected from the corresponding vehicle 24 is aligned with the first point cloud map v1_local relative to the world coordinate system. map Second point cloud map v2_local map Specifically, the GNSS data collected by the first vehicle 24 is used to align the first point cloud map v1_local. map And the GNSS data collected by the second vehicle 24 is used to align the second point cloud map v2_local. map Alignment module 122 can align the first point cloud map v1_local relative to the world coordinate system based on any type of alignment algorithm that performs translation, scaling, and rotation to align the data. map Second point cloud map v2_local map For example, pose graph optimization and binding adjustment.

[0060] It should be understood that alignment performed based on GNSS data tends to be less accurate for relatively short distances (e.g., less than ten meters) measured between two point cloud data points, but tends to be more accurate for relatively long distances (e.g., about one kilometer or more) measured between two point cloud data points. Therefore, when the alignment module 122 aligns with the first point cloud map or the second point cloud map, a longer distance (e.g., one kilometer or more) will be used.

[0061] Then, one or more central computer 20 precision positioning modules 124 can receive the first point cloud map v1_local from the alignment module 122. map Second point cloud map v2_local map And the first point cloud map v1_local map Aligned with the perception data captured by the first vehicle 24 and the second point cloud map v2_local map Align with the perception data captured by the second vehicle 24 to form a first point cloud map v1_local map Second point cloud map v2_local map A portion of the point cloud data points are used to create precise localization. Specifically, it should be understood that the precise localization module 124 provides localization corresponding to the first point cloud map v1_local. map Second point cloud data map v2_local map Precise positioning between point cloud data points over relatively short distances (less than about ten meters). In other words, after the alignment module 122 aligns the point cloud map with relatively long but more precise distances, the precise positioning module 124 provides precise positioning over relatively short distances.

[0062] The nonlinear optimization module 126 determines a unique calibration conversion ratio α for each road segment that is part of the vehicle trajectory based on one or more nonlinear optimization algorithms. The nonlinear optimization module 126 of one or more central computers 20 iteratively calculates the unique calibration conversion ratio α to minimize the value based on the first point cloud map v1_local. map 180 point cloud data points Figure 5 (As shown) and based on the second point cloud map v2_local map The cost function between the corresponding point cloud data points 182.

[0063] The nonlinear optimization module 126 of one or more central computers 20 includes a segmentation submodule 130, a conversion ratio submodule 132, an association submodule 134, a cost function submodule 136, and a nonlinear optimization submodule 138. The segmentation submodule 130 of one or more central computers 20 divides the vehicle trajectory determined based on perception data collected from the first vehicle 24 and the second vehicle 24 into multiple road segments.

[0064] The conversion ratio submodule 132 of the nonlinear optimization module 126 first selects a temporary conversion ratio α′ corresponding to a specific road segment that is part of the vehicle trajectory. The association submodule 134 of the nonlinear optimization module 126 then finds the point cloud data points 180 (…). Figure 5 The association between (as shown) and the corresponding point cloud data point 182, where point cloud data point 180 represents the first point cloud map v1_local map The specific perceptual data geometric features within the area correspond to point cloud data point 182 in the second point cloud map v2_local. map The inner part also represents the specific geometric characteristics of the sensing data for that particular road segment.

[0065] It should be understood that, due to the alignment module 122 executing the first point cloud map v1_local based on GNSS data... map Second point cloud map v2_local map The alignment and the pairing performed by the precise positioning module 124 as the first point cloud map v1_local map Second point cloud map v2_local map Precise localization of a portion of the point cloud data points, the first point cloud map v1_local map Point cloud data points 180 and the second point cloud map v2_local map The corresponding point cloud data point 182 is already accurate. Therefore, a unique calibration conversion ratio α is determined so that it can be used as the first point cloud map v1_local. map A portion of the point cloud data points and used as the second point cloud map v2_local map A portion of the point cloud data points are more closely aligned.

[0066] Once the association between point cloud data point 180 and the corresponding point cloud data point 182 for a specific road segment is determined, the cost function submodule 136 of the nonlinear optimization module 126 can be based on the first point cloud map v1_local map Each point cloud data point 180 within the map is compared with a specific road segment based on the second point cloud map v2_local. map The cost function is calculated based on the Euclidean distance between the corresponding point cloud data points 182. A unique calibration conversion ratio α corresponding to a specific road segment is selected to minimize the cost function. Specifically, in one embodiment, the cost function is determined based on Equation 3, which is:

[0067]

[0068] Where, p i ,q i Let p represent a pair of related points. i For the second point cloud map v2_local map The point, q i For the first point cloud map v1_local map The point, p atti ,q atti This indicates the relationship between the two points p. i ,q i The properties of the geometric features of the associated perceptual data.

[0069] Once the cost function is calculated, the nonlinear optimization submodule 138 of the nonlinear optimization module 126 can calculate a unique calibration conversion ratio α corresponding to a specific road segment based on one or more nonlinear optimization algorithms to minimize the cost function. In response to determining that the cost function has been minimized, the nonlinear optimization submodule 138 of the nonlinear optimization module 126 can then transmit the unique calibration conversion ratio α corresponding to the specific road segment to the outlier removal module 128. However, in response to determining that the cost function has not been minimized, the conversion ratio submodule 132 of the nonlinear optimization module 126 can then send the unique calibration conversion ratio α corresponding to the specific road segment to the associated submodule 132 of the nonlinear optimization module 126. The nonlinear optimization module 126 can then iteratively calculate the unique calibration conversion ratio α until the cost function is minimized. In one embodiment, the nonlinear optimization submodule 138 of the nonlinear optimization module 126 can calculate the unique calibration conversion ratio α corresponding to the specific road segment based on Equation 4, which is:

[0070] α = argmin α (cost_function(v1_local_map,v2_local_map)) Equation 4

[0071] Once the cost function of each road segment as part of the vehicle trajectory is minimized, the outlier removal module 128 of one or more central computers 20 can receive a unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory. The outlier removal module 128 can evaluate the unique calibration conversion ratio α corresponding to each road segment as part of the vehicle trajectory to identify any outliers based on one or more outlier removal methods. Outliers are removed from the vehicle trajectory, and non-outliers can be used to update the local map data.

[0072] Once a unique calibration conversion ratio α is determined for each road segment that is part of the vehicle trajectory (i.e., a non-outlier), one or more central computers 20 can calculate the first point cloud map v1_local based on the unique calibration conversion ratio α corresponding to each road segment that is part of the vehicle trajectory. map The included multiple perceptual data geometric features, and the corrected first point cloud map v1_local map Retransmitted back to the first vehicle 24.

[0073] Referring generally to the accompanying drawings, the disclosed perception data geometry calibration system offers various technical effects and benefits. Specifically, the perception data geometry calibration system calibrates the perception data geometry captured by one or more vehicles to improve the accuracy of map data. Specifically, this disclosure provides a method for calibrating perception data geometry based on high-definition map data or alternatively based on perception data captured between two different vehicles located in different lanes along a road. In embodiments where perception data is captured between two vehicles, the perception data geometry calibration system performs alignment based on relatively long distances measured between two point cloud data points using GNSS data, and also performs precise positioning based on relatively short distances measured between the two point cloud data points using the perception data.

[0074] A central computer can refer to electronic circuitry, combinational logic circuitry, a field-programmable gate array (FPGA), a processor (shared, dedicated, or grouped) that executes code, or some or all of the above, such as a combination in a system-on-a-chip. Additionally, the controller can be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor can operate under the control of an operating system residing in memory. The operating system can manage computer resources so that computer program code embodied as one or more computer software applications (e.g., applications residing in memory) can have instructions that are executed by the processor. In an alternative embodiment, the processor can directly execute the application, in which case the operating system can be omitted.

[0075] The descriptions in this disclosure are merely exemplary in nature, and variations thereof that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A perception data geometry calibration system for calibrating perception data geometry captured by one or more vehicles located within a predefined geographic area, the perception data geometry calibration system comprising: One or more central computers, wirelessly communicating with the one or more vehicles and one or more communication networks, are used to receive high-definition map data, and the one or more central computers execute instructions to: The communication network receives local map data and corresponding high-resolution map data representing the predefined geographical area from a specific vehicle, wherein the local map data includes multiple perceptual data geometric features that define objects within the environment surrounding the specific vehicle. The vehicle trajectory, determined based on perception data collected from the specific vehicle, is divided into multiple road segments. A unique calibration conversion ratio is determined for each road segment that is part of the vehicle trajectory based on one or more nonlinear optimization algorithms, wherein the one or more central computers iteratively compute the unique calibration conversion ratio to minimize the cost function between road segment points representing specific perceptual data geometry features based on the local map data and corresponding road segment points based on the high-definition map data. and The geometric features of the multiple sense data included in the local map data are calibrated based on the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory, in order to create a corrected local map.

2. The sensing data geometric calibration system according to claim 1, wherein, The one or more central computers execute instructions to: The unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory is evaluated to identify any outliers based on one or more outlier removal methods.

3. The sensing data geometric calibration system according to claim 2, wherein, The outlier removal methods include one or more of the following: Random Sample Consensus (RANSAC) algorithm, z-... sco r e Algorithms and density-based applied spatial clustering (DBSCAN) algorithm with noise.

4. The sensing data geometric calibration system according to claim 1, wherein, The one or more central computers execute instructions to determine the unique calibration conversion ratio in the following manner: Find the association between the road segment point and the corresponding road segment point, wherein the road segment point represents a specific perceptual data geometric feature within the local map data, and the corresponding road segment point also represents the specific perceptual data geometric feature of each road segment that is part of the vehicle trajectory within the high-definition map data.

5. The sensing data geometric calibration system according to claim 4, wherein, The one or more central computers execute instructions to determine the unique calibration conversion ratio in the following manner: The cost function is calculated based on the Euclidean distance between each road segment point in the local map data and the corresponding road segment point in the high-definition map data based on each road segment as part of the vehicle trajectory.

6. The sensing data geometric calibration system according to claim 5, wherein, The cost function is determined based on the following: Where, p i q i Let p represent a pair of related points. i For HD map data map The point, q i For the local map data map The point, p atti q atti This indicates the relationship between the two related points p. i q i The associated geometric features of the perceived data are attributed to n, which represents the total number of associated point pairs, and cost_function(HD_map, local_map) represents the cost function.

7. The sensing data geometric calibration system according to claim 6, wherein, The geometric features of the perceived data include one or more of the following: lane lines, road signs, traffic lights, and location coordinates on the road.

8. The sensing data geometric calibration system according to claim 7, wherein, The geometric features of the perception data are lane lines, and the attributes include one or more of the following: line color and line type.

9. The sensing data geometric calibration system according to claim 1, wherein, The length of each road segment is equal to the average accurate perception range of the specific vehicle.

10. A perception data geometry calibration system for calibrating perception data geometry captured by one or more vehicles located within a predefined geographic area, the perception data geometry calibration system comprising: Multiple sensing sensors capture collected sensing data about the predefined geographic area; as well as One or more controllers, which electronically communicate with the plurality of sensing sensors, wherein the one or more controllers are a portion of a specific vehicle that receives high-definition map data via a communication network, and the one or more controllers execute instructions to: Local map data is determined based on the perception data collected by the plurality of perception sensors, wherein the local map data includes multiple perception data geometric features of objects defining the environment surrounding the particular vehicle. The vehicle trajectory, determined based on perception data collected from the specific vehicle, is divided into multiple road segments. A unique calibration conversion ratio is determined for each road segment that is part of the vehicle trajectory based on one or more nonlinear optimization algorithms, wherein the one or more controllers iteratively compute the unique calibration conversion ratio to minimize the cost function between road segment points representing specific perceptual data geometry features based on the local map data and corresponding road segment points based on the high-definition map data; and The geometric features of the multiple sense data included in the local map data are calibrated based on the unique calibration conversion ratio corresponding to each road segment that is part of the vehicle trajectory, in order to create a corrected local map.

Images