CARD QUALITY RATING SYSTEM

Abstract

A map quality evaluation system that evaluates an error associated with primary map data includes one or more central computers that execute instructions that determine the error associated with the primary map data, compare the error associated with the primary map data to a range of values ​​defined by one or more quality metrics, and, in response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metrics, maintain a template for selecting the primary map data. The one or more central computers, in response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metrics, evaluate the primary map data for real anomalies within one or more roads represented by the primary map data.

Description

INTRODUCTION

[0001] The present invention relates to a map quality evaluation system that evaluates an error associated with map data. The map quality evaluation system evaluates the error associated with the map data based on gold-source map data or, alternatively, based on a self-evaluation without the gold-source map data.

[0002] A card quality rating system according to the preamble of claim 1 is essentially disclosed in DE 11 2018 008 077 T5. Further prior art is also disclosed in the documents US 2022 / 0 161 817 A1, DE 10 2006 032 374 A1, and US 2020 / 0 250 439 A1.

[0003] An autonomous vehicle performs various tasks such as perception, positioning, mapping, path planning, decision-making, and motion control. Autonomous vehicles rely on map data for many of the tasks they perform, such as positioning, mapping, and path planning. It should be noted that different versions of map data representing the same geographic area can be generated, with each version of the map data being generated from different data sources.

[0004] An example version of map data is based on telemetry data. The telemetry data can be collected from numerous vehicles and combined based on various fusion algorithms to determine various map content types, such as Global Positioning System (GPS) trajectories and subsequently inferred lane geometries. However, it should be noted that errors such as lateral bias, white noise, and the like may be present when comparing the map content with content generated from ground truth data. This error may cause the map data to indicate an inaccurate position of the guidelines, which in turn can cause location determination problems and lead to erroneous decisions being made by the autonomous vehicle's control system.

[0005] Although maps for autonomous vehicles serve their intended purpose, there is a need in the field for an improved approach to assessing edge quality. SUMMARY

[0006] According to the invention, a map quality evaluation system which evaluates an error associated with primary map data is presented, which is characterized by the features of claim 1 or those of claim 6.

[0007] The map quality evaluation system includes one or more central computers in wireless communication with one or more communication networks for receiving the primary map data. The one or more central computers execute instructions for determining the error associated with the primary map data, wherein the primary map data represents a predefined virtual bounded area. The one or more central computers compare the error associated with the primary map data to a range of values ​​defined by one or more quality metric values. In response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values, the one or more central computers maintain a template for selecting the primary map data representing the predefined virtual bounded area.In response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values, the one or more central computers evaluate the primary map data for real anomalies within one or more roads represented by the primary map data. In response to determining that no real anomalies exist within the one or more roads represented by the primary map data, the one or more central computers implement, based on the one or more quality metric values, an updated template for selecting primary map data points that represent the predefined virtually bounded area.

[0008] In another aspect, the one or more central computers execute instructions to implement the updated template, the instructions implemented by: updating weights assigned to the primary map data points of the primary map data based on a corresponding level of accuracy.

[0009] In yet another aspect, a higher weighted value is assigned to the weights corresponding to the primary map data points containing a higher degree of accuracy based on the one or more quality metric values, and a lower weighted value is assigned to the weights corresponding to the primary map data points containing a lower degree of accuracy based on the one or more quality metric values.

[0010] In yet another aspect, the one or more central computers determine the absolute offset between the primary map data points and the gold source map data points based on a bounding box approach that generates a plurality of bounding boxes, each enclosing one of the plurality of road segments that are part of the road network.

[0011] According to another aspect, a distance between the cross-sectional ends of the plurality of road sections is measured, and the distance between the cross-sectional ends of each road section is dimensioned based on a targeting accuracy of the primary map data.

[0012] According to another aspect, the first set of guidelines is determined at the first timestamp and drawn based on the primary map data, and the second set of guidelines is determined at the second timestamp, wherein the second timestamp occurs after the first timestamp.

[0013] In another aspect, the one or more central computers execute instructions to: determine a spatial offset between a first set of guidelines and a second set of guidelines, wherein the spatial offset is measured between an end portion of the first set of guidelines and a starting position of the second set of guidelines, and wherein the spatial offset represents the error associated with the primary map data.

[0014] According to yet another aspect, the first set of guidelines is determined at a first location and the second set of guidelines is determined at a second location positioned directly adjacent to the first location.

[0015] In one aspect, the one or more central computers execute instructions to: determine a probability of user intervention representing, for a specific road segment located within the predefined virtually bounded area, a number of times user intervention is required during autonomous driving compared to a total number of passes by autonomous vehicles, wherein the probability of user intervention represents the error associated with the primary map data.

[0016] According to another aspect, the probability of user intervention is expressed as: Pi=UiNi, where P i represents the probability of user intervention, U i represents the number of required user interventions and N irepresents the total number of passages of autonomous vehicles for the specific road section.

[0017] According to yet another aspect, a method for evaluating the error associated with primary map data is disclosed. The method includes determining, by one or more central computers, the error associated with the primary map data, wherein the primary map data represents a predefined virtually bounded area, and wherein the one or more central computers are in wireless communication with one or more communication networks to receive the primary map data. The method includes comparing, by the one or more central computers, the error associated with the primary map data to a range of values ​​defined by one or more quality metric values.The method includes maintaining a template for selecting, by the one or more central computers, the primary map data representing the predefined virtually bounded area in response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values. The method includes evaluating, by the one or more central computers, the primary map data for real anomalies within one or more roads represented by the primary map data in response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values.The method includes implementing an updated template for selecting primary map data points representing the predefined virtually bounded area based on the one or more quality metric values ​​in response to determining that no real anomalies exist within the one or more roads represented by the primary map data.

[0018] According to one aspect, the method further includes implementing the updated template implemented by: updating weights assigned to the primary map data points of the primary map data based on a corresponding accuracy level.

[0019] In another aspect, the method further includes determining the error associated with the primary map data based on an absolute offset between primary map data points corresponding to the primary map data and gold source map data points corresponding to gold source map data.

[0020] In yet another aspect, the method further includes receiving road network data representing a road network for the predefined virtually bounded area, wherein the road network is a network graph modeling roads based on a plurality of road segments.

[0021] In one aspect, the method further includes determining the absolute offset between the primary map data points and the gold source map data points based on a bounding box approach that generates a plurality of bounding boxes each enclosing one of the plurality of road segments that are part of the road network.

[0022] According to another aspect, the method further includes aligning the primary map data points and the gold source map data points to each other, wherein the primary map data points and the gold source map data points are located at cross-sectional ends of each of the plurality of road segments, and determining the absolute offset between the primary map data points and the gold source map data for all cross-sectional ends of the road segments located within the predefined virtually bounded area.

[0023] Further areas of applicability will become apparent from the description provided herein. It is understood that the description and specific examples are for illustrative purposes only. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described here are for illustrative purposes only; they show: Fig. 1 is a schematic representation of the disclosed map quality rating system including one or more central computers receiving map data via one or more communication networks, according to an exemplary embodiment; Fig. 2 a block diagram showing the software architecture for the one or more Fig. 1 according to an exemplary embodiment; Fig. 3 is a schematic representation of a bounding box determined by the one or more central computers according to an exemplary embodiment; Fig. 4 is a schematic representation of road network data including a plurality of intersection road sections according to an exemplary embodiment; Fig. 5A is a schematic diagram illustrating a method for determining an error associated with primary map data based on timing inconsistencies according to an exemplary embodiment; Fig. 5B is a schematic diagram illustrating a procedure for determining the error associated with the primary map data based on spatial inconsistencies according to an exemplary embodiment; and Fig. 6 is a process flow diagram illustrating a method for evaluating the error associated with primary card data according to an exemplary embodiment. DETAILED DESCRIPTION

[0025] The following description is merely exemplary in nature.

[0026] In Fig. 1 illustrates an exemplary card quality assessment system 10 for assessing a defect associated with card data. The card quality assessment system 10 includes one or more central computers 20 located at a back-end office 22, the one or more central computers 20 in wireless communication with one or more communication networks 24. The one or more central computers 20 receive primary card data via the one or more communication networks 24. According to one embodiment, the one or more central computers 20 may also receive gold source card data via the one or more communication networks 24. However, it should be noted that the gold source card data may not be available according to some embodiments.

[0027] The primary map data and the gold source map data may both represent the same predefined, virtually geofenced area. The predefined, virtually geofenced area represents a real geographic area defined by a virtual outer boundary. It should be noted that the primary map data is based on one or more unique data sources, such as Global Positioning System (GPS) data, imagery collected by a vehicle's onboard camera, a vehicle telemetry source, aerial or satellite imagery, or data collected by survey vehicles. Some example versions of primary data include crowdsourced map data, telemetry-based map data, and aerial map data.The gold source map data represents ground-truth data or, alternatively, the most accurate and up-to-date version of map data representing the predefined, virtually bounded area currently available. According to one embodiment, the gold source map data is high-resolution map data.

[0028] According to the Fig. 1, the one or more central computers 20 are in wireless communication with one or more autonomous vehicles 26. Although autonomous vehicles are described, it should be appreciated that semi-autonomous vehicles equipped with an advanced driver assistance system (ADAS) may also be included. After evaluating the primary map data, the one or more central computers 20 may share a vehicle map generated based on the primary map data with the one or more autonomous vehicles 26. As explained below, the one or more central computers 20 evaluate the error associated with the primary map data based on an absolute offset between primary map data points and gold source map data points.Alternatively, according to another embodiment, the one or more central computers 20 determine the error associated with the primary map data based on a self-assessment without the gold source map data. According to yet another embodiment, the error associated with the primary map data is based on a probability of user intervention during autonomous driving. It should be noted that the error associated with the primary map data may represent any type of error associated with map data, such as a lateral bias or a perceptual bias. According to one embodiment, the primary map data is specifically generated for an autonomous driving system, such as, for example, an automated driving system (ADS) or ADAS.

[0029] Fig. 2 is a block diagram illustrating the software architecture of the one or more Fig. 1. According to the central computer 20 shown in Fig. 2, the one or more central computers 20 include a gold source evaluation module 30, a self-evaluation module 32, and a selection module 34. As explained below, the gold source evaluation module 30 compares the primary map data with the gold source map data to determine the error associated with the primary map data, while the self-evaluation module 32 determines the error when the gold source map data is not available. Based on both Fig. 1 as well as Fig. 2, the gold source evaluation module 30 of the one or more central computers 20 includes a bounding box evaluation sub-module 40 that compares the primary map data to the gold source map data based on a bounding box approach, and a cross-sectional evaluation sub-module 42 that compares the primary map data to the gold source map data based on a cross-sectional approach, both of which are further described below.

[0030] The one or more central computers 20 receive, as input from the one or more communication networks 24, road network data representing a road network of the predefined, virtually bounded area, where the road network is a network graph that models roads based on multiple road segments. In addition, the one or more central computers 20 receive, also from the one or more communication networks 24, the primary map data and the gold source map data, both of which represent the predefined, virtually bounded area. An example of road network data is OpenStreetMap (OSM), but it should be noted that other types of road network data may also be used.

[0031] The evaluation of the primary map data for the absolute offset between the primary map data and the gold source map data based on a bounding box approach will now be described. The bounding box evaluation sub-module 40 of the one or more central computers 20 divides the road network data into several road sections 52, which Fig. 3. Both based on Fig. 2 as well as Fig. 3, each of the plurality of road segments 52 includes the same length, where the length of each road segment 52 may be equal to an average range of accurate perception of a vehicle 26. According to one embodiment, the length is approximately fifty meters. The bounding box evaluation sub-module 40 of the one or more central computers 20 generates bounding boxes 64 corresponding to the primary map data and the gold source map data. Each bounding box 64 has a width 48 and a height 50 (i.e., a maximum height). Each bounding box 64 encloses a road segment 52, where the width 48 of each bounding box 64 is equal to the length of the road segment 52. The height 50 of the bounding box 64 may be greater than the width 48 of the bounding box 64.

[0032] Based on Fig. 3, the bounding boxes 64 enclose the road section 52. Each road section 52 contains at least two road section points (ie, the first road section point 66 and the second road section point 68) that define boundaries of the road section 52. In Fig. 3, the bounding box 64 includes a first linear boundary 70, a second linear boundary 72, a third linear boundary 74, and a fourth linear boundary 76. The first linear boundary 70 intersects the first road segment point 66. The second linear boundary 72 intersects the second road segment point 68. The first linear boundary 70 is parallel to the second linear boundary 72. The third linear boundary 74 is parallel to the fourth linear boundary 76. The distance 78 from the third linear boundary to the first road segment point 66 along the first direction y' is equal to the distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y'. The height 50 (i.e., the maximum height) of the bounding box extends from the third linear boundary 74 to the fourth linear boundary 76.The distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y' is half the maximum height of the first bounding box. The road segment 52 is parallel to a second direction x'. The second direction x' is orthogonal to the first direction y', where the first direction y' and the second direction x' can be defined as axes forming a vehicle coordinate system 82 for the bounding box 64.

[0033] As discussed above, the first road segment point 66 and the second road segment point 68 define the outermost ends (i.e., endpoints) of the road segment 52. The bounding box evaluation submodule 40 of the one or more central computers 20 uses the first road segment point 66 and the second road segment point 68 to generate the bounding box 64. In the vehicle coordinate system 82, the x'-axis (defined by the x'-direction) is parallel to the road segment 52 and the y'-axis (defined by the y'-direction) is perpendicular to the road segment 52. The angle θ is the direction of travel of a vehicle 26. In the vehicle coordinate system 82, the first road segment point 66 is at the coordinate (x1, y1) of the vehicle coordinate system 82 and the second road segment point 68 is at the coordinate (x2, y2).The distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y' is half the height 50 of the bounding box 64. The distance 80 from the second road segment point 68 to the fourth linear boundary 76 is constant for all bounding boxes 64. Furthermore, the distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y' can be represented by the letter "d." The bounding box defines four vertices (i.e., the first vertex Q1 at coordinate x1, y1+d), the second vertex Q2 at coordinate (x2, y2+d), the third vertex Q3 (x1, y1-d), and the fourth vertex Q4 (x2, y1-d).

[0034] Continue with Fig. 3 is the coordinate of the origin of the vehicle coordinate system 82 (m, n). The origin of the vehicle coordinate system 82 is shifted relative to the origin of the global coordinate system 84. The coordinates of the bounding boxes are as follows: the first corner point Q1 (x1+m, y1+d+n), the second corner point Q2 (x2+m, y2+d+n), the third corner point Q3 (x1+m, y1-d+n), and the fourth corner point Q4 (x2+m, y2-d+n). The vehicle coordinate system 82 is then rotated such that the x'-axis of the vehicle coordinate system 82 is aligned with the x-axis of the global coordinate system 84 and the y'-axis of the vehicle coordinate system 82 is aligned with the y-axis of the global coordinate system 84. In this rotation, the angle of rotation in the clockwise direction is θ (which is the direction of travel of the vehicle 26).After the rotation, the coordinates of the bounding box 64 in the global coordinate system xy are: the first vertex Q1 ((x1+m)·cosθ+(y1+d+n)·sinθ, (y1+d+n)·cosθ-(x1+m)·sinθ), the second vertex Q2 ((x2+m)·cosθ+(y2+d+n)·sinθ, (y2+d+n)·cosθ-(x2+m)·sinθ), the third vertex Q3 ((x1+m)·cosθ+(y1-d+n)·sinθ, (y1-d+n)·cosθ-(x1+m)·sinθ) and the fourth vertex Q4 ((x2+m)·cosθ+(y2-d+n)·sinθ, (y2-d+n)·cosθ-(x2+m)·sinθ).

[0035] The bounding box evaluation sub-module 40 of the one or more central computers 20 generates a primary map tile and a gold source map tile by filtering the map data within the bounding box 64 based on the primary map data and the gold source map data. The primary map tile is obtained by filtering the primary map data, and the gold source map tile is obtained by filtering the gold source map data. The bounding box evaluation sub-module 40 of the one or more central computers 20 generates the primary map tile and the gold source map tile for all bounding boxes 64 from the primary map data and the gold source map data, respectively. The filtered data for generating the map tiles (i.e., the primary map tile and the gold source map tile) may be part of the map data within the bounding box 64 but outside the guidelines.

[0036] The bounding box evaluation submodule 40 of the one or more central computers 20 performs point cloud registration to align the plurality of primary map data points in the primary map tile to the plurality of gold source map data points in the gold source map data tile using rotation and translation transformations to determine the absolute offsets between the primary map data points of the primary map data and the gold source map data points of the gold source map data. More specifically, an iterative closest point (ICP) process or algorithm may align the plurality of primary map data points in the primary map tile to the plurality of gold source map data points in the gold source map tile using rotation and translation transformations.In addition, a random initialization process can be used to align the primary map data points to the gold source map data points based on the point cloud registration.

[0037] The bounding box evaluation sub-module 40 of the one or more central computers 20 constructs a KD tree with the primary map data points using a KD algorithm, with the primary map data points being generated first as the routes. The center points of the routes are then obtained and referred to as the KD tree nodes. For each point of the gold source map data, the nearest point and the corresponding routes from the primary map data are determined by KD tree query. The offset from this point in the gold source map data to the routes from the first map is then calculated. The bounding box evaluation sub-module 40 of the one or more central computers 20 determines, based on the KD tree, the absolute offsets from each of the plurality of primary map data points to each of the corresponding gold source map data points.To this end, the bounding box evaluation submodule 40 of the one or more central computers 20 calculates the distance from one of the primary map data points of the primary map data to the corresponding gold source map data point of the gold source map data. This is repeated by all to determine all absolute offsets between the primary map data points and the gold source map data points. The bounding box evaluation submodule 40 of the one or more central computers 20 then determines the relative map error based on a histogram of all absolute offsets.

[0038] A procedure for determining the absolute offset between the primary map data points and the gold source map data points based on a cross-sectional approach is now described. Fig. 4 is a representation of the road network data of the predefined virtually bounded area, wherein the roads located within the predefined virtually bounded area are represented by a plurality of road segments 92, a plurality of nodes 94, and opposing road edges 98. The plurality of nodes 94 each represent a cross-sectional end 96 of one of the road segments 92. The opposing road edges 98 represent theoretical road edges and not the opposing topological road edges. It should be noted that the opposing road edges 98 contain a unique edge identifier (edge ​​ID), wherein each edge ID identifies a pair of opposing road edges 98 corresponding to a particular road segment 92.A distance D is measured between the cross-sectional ends 96 of the road sections 92, wherein the distance D between the cross-sectional ends 96 of each road section 92 is dimensioned based on a targeting accuracy of the primary map data. For example only, the distance D between the cross-sectional ends 96 of the road sections 92 is approximately one meter if the primary map data is generated for an autonomous driving system such as ADS or ADAS.

[0039] The cross-sectional evaluation sub-module 42 of the one or more central computers 20 first determines the absolute offset between the primary map data points 100 corresponding to the primary map data and the gold source map data points 102 corresponding to the gold source map data by aligning the primary map data points 100 and the gold source map data points 102 with each other. As in Fig. 4, the primary map data points 100 and the gold source map data points 102 are located at the cross-section ends 96 of the road sections 92. The cross-section evaluation submodule of the one or more central computers 20 aligns the primary map data points 100 and the gold source map data points 102 by executing one or more matching algorithms to create a topology. As shown in Fig. 4, the cross-section ends 96 of each road segment 92 intersect one or more primary map data points 100 and one or more corresponding gold source map data points 102. The cross-section evaluation sub-module 42 of the one or more central computers 20 then determines, for all cross-section ends 96 of the road segments 92 located within the predefined virtually bounded area, an absolute offset between the one or more primary map data points 100 and the one or more corresponding gold source map data points 102.The cross-section evaluation submodule 42 of the one or more central computers 20 then determines a sum of all absolute offsets for each road segment 92 within the predefined virtual bounded area and divides the sum of all of the absolute offsets by a total number of road segments 92 within the predefined virtual bounded area to determine the relative error.

[0040] As mentioned above, the self-assessment module 32 determines the map error when gold source map data is not available. Fig. 1 as well as Fig. 2, the self-assessment module 32 of the one or more central computers 20 includes a temporal assessment sub-module 44 that determines the error of the primary map data based on temporal inconsistencies, a spatial assessment sub-module 46 that determines the error of the primary map data based on spatial inconsistencies, and a user intervention sub-module 60 that determines the error of the primary map data based on user intervention during autonomous or semi-autonomous driving. Although Fig. 2 illustrates that the one or more central computers 20 determine the error based on temporal inconsistencies or spatial inconsistencies, it should be noted that the error may instead be determined locally by one or more controllers that are part of one of the autonomous vehicles 26 ( Fig. 1) can be determined.

[0041] The determination of the error based on temporal inconsistencies is now described. Fig. 5A is a schematic representation of a first set of guidelines 110 determined at a first timestamp T, drawn based on the primary map data. It also includes Fig. 5A, a second set of guidelines 112 determined at a second timestamp T+1, wherein the second timestamp T+1 occurs after the first timestamp T. Both the first set of guidelines 110 and the second set of guidelines 112 represent the same road.

[0042] Both based on Fig. 2 as well as Fig. 5A, the temporal evaluation sub-module 44 of the one or more central computers 20 determines a temporal offset 114 between the first set of guidelines 110 and the second set of guidelines 112, where the temporal offset 114 represents the error of the primary map data. The temporal offset 114 represents a perpendicular distance measured at the same primary map data point between the first timestamp T and the second timestamp T+1. The temporal evaluation sub-module 44 of the one or more central computers 20 then compares the temporal offset 114 to a temporal threshold, where the temporal threshold is selected based on a targeting accuracy level. It should be noted that the targeting accuracy level is determined based on the specific application of the primary map data.According to one embodiment, a notification is generated if the timing error associated with any of the primary card data exceeds the timing threshold.

[0043] According to one embodiment, the temporal evaluation sub-module 44 of the one or more central computers 20 selects the maximum time offset value from the primary map data points as the total time offset of the primary map data. More specifically, the total time offset of the primary map data is determined by Equation 1, which is as follows: IIT=maxi|A⇀i−B⇀i⋅A⇀i|A⇀i||, where II T represents the total temporal offset of the primary map data, i represents the total number of primary map data points defining guidelines 110, 112, A⇀i represents a point along the first set of guidelines 110 of the first time step T and B⇀i represents a point along the second set of guidelines 112 at the second timestamp T2.

[0044] The determination of the error based on spatial inconsistencies is now described. Fig. 5B is a schematic representation of a first set of guidelines 120 determined at a first location S drawn based on the primary map data. Fig. 5B also includes a second set of guidelines 122 based on the primary map data determined at a second location S+1, wherein the second location S+1 is positioned directly adjacent to the first location S. Note that the second location S+1 is positioned directly adjacent to the first location S in either the lateral or longitudinal direction.

[0045] Based on Fig. 2 and Fig. 5B, the spatial evaluation sub-module 46 of the one or more central computers 20 determines a spatial offset 124 between the first set of guidelines 120 and the second set of guidelines 122, where the spatial offset 124 is measured between an end portion 126 of the first set of guidelines 120 and a starting position 128 of the second set of guidelines 122, where the first set and the second set of guidelines 120, 122 overlap. The spatial offset 124 represents the error associated with the primary map data. The spatial evaluation sub-module 46 of the one or more central computers 20 then compares the spatial offset 124 to a spatial threshold, where the spatial threshold is selected based on the targeting accuracy level described above.According to one embodiment, a notification is generated if the spatial error associated with one of the primary map data points exceeds the spatial threshold.

[0046] According to one embodiment, the spatial evaluation sub-module 46 of the one or more central computers 20 determines the spatial offset 124 based on Equation 2, which is as follows: IIS=|AEnd⇀−BBeginning⇀⋅AEnd⇀|AEnd⇀||, where II S represents the overall spatial intent of the primary data points, AEnd⇀ represents a point at the end portion 126 of the first set of guidelines 120 and BBeginning⇀ represents a point at the initial position 128 of the second set of guidelines 122.

[0047] Based on Fig. 2, the user intervention submodule 60 of the one or more central computers 20 determines a probability of user intervention during autonomous driving, wherein the probability of user intervention represents the error associated with the primary map data. This is because the autonomous vehicle 26 ( Fig. 1) can be navigated with minimal or no user intervention if the error associated with the primary map data is relatively low. However, the incidence of a user intervening by performing one or more manual driving maneuvers to navigate the vehicle during autonomous driving increases as the error increases. User intervention represents one or more manual driving maneuvers performed by a user to compensate for the error associated with the primary map data. Some examples of map errors that may cause the user to perform one or more manual driving maneuvers include map curvature error, guideline inaccuracies, perception issues, missing signs, and incorrect sign locations.

[0048] According to one embodiment, the probability of user intervention represents the number of times user intervention is required during autonomous driving compared to the total number of passes by autonomous vehicles for a specific road section located within the predefined, virtually bounded area. More specifically, the probability of user intervention, according to one embodiment, is expressed in Equation 3 and is as follows: Pi=UiNi, where P i represents the probability of user intervention, U i represents the number of times user intervention is required and N i represents the total number of passages by autonomous vehicles for the specific road section.

[0049] Based on Fig. 2, the selection module 34 of the one or more central computers 20 receives the error associated with the primary map data from either the gold source evaluation module 30 or the self-evaluation module 32 and compares the error associated with the primary map data to one or more quality metric values. The quality metric values ​​are each based on a specific quality metric indicating the quality of the primary map data and based on the target accuracy level of the primary map data. As mentioned above, the target accuracy level is determined based on the specific application of the primary map data. For example only, according to one embodiment, the quality metric values ​​indicate an estimated horizontal position error (EHPE) of the primary map data, although it should be noted that other measures of quality, such as confidence values, may also be used.According to one embodiment, the quality metric values ​​may be defined by a range of values. For example, the range of quality metric values ​​may be expressed as a number in the range of 0 to 9, where 0 represents the highest level of accuracy and 9 represents the lowest level of accuracy. According to one embodiment, the one or more quality metric values ​​may be weighted based on importance, with primary map data points containing a higher level of accuracy being assigned a higher level of importance compared to primary map data points containing a lower level of accuracy.

[0050] The selection module 34 of the one or more central computers 20 compares the error associated with the primary map data with the one or more quality metric values. In response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values, the selection module 34 maintains a template for selecting the primary map data representing the predefined virtually bounded area based on the one or more quality metric values. The template for selecting the primary map data points assigns weights to the primary map data points, with a higher weight assigned to primary map data points containing a higher degree of accuracy and a lower weight assigned to primary map data points containing a lower degree of accuracy.The selection module 34 of the one or more central computers 20 can then generate a vehicle map based on the primary map data. According to embodiments, the vehicle map can be linked to the one or more autonomous vehicles 26 (. Fig. 1) be shared.

[0051] In response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values, the selection module 34 of the one or more central computers 20 then evaluates the primary map data for real-world anomalies within one or more roads defined by the primary map data. The real-world anomalies within the one or more roads may adversely impact the accuracy of the primary map data. Some examples of the real-world anomalies within the one or more roads include faded guidelines, repainted guidelines, construction activity, road closures, and traffic accidents.In response to detecting one or more real anomalies within the one or more roads represented by the primary map data, the selection module 34 of the one or more central computers 20 continues to monitor the primary map data until the real anomalies are no longer present and then collects a new set of primary map data for evaluation.

[0052] In response to determining that no real anomalies exist within the one or more roads represented by the primary map data, the selection module 34 of the one or more central computers 20 implements an updated template for selecting the primary map data points representing the predefined bounded areas based on the one or more quality metric values. The updated template is implemented by updating the weights assigned to the primary map data points of the primary map data based on a corresponding accuracy level.More specifically, the weights corresponding to the primary map data points containing a higher level of accuracy are assigned a higher weighted value based on the one or more quality metric values, while the weights corresponding to the primary map data points containing a lower level of accuracy are assigned a lower weighted value based on the one or more quality metric values.

[0053] According to one embodiment, the weights are assigned to the primary map data points based on a probability distribution to ensure that the primary map data points contain a mixture of different error levels. According to one embodiment, the updated template is determined based on an iterative process, wherein a selection criterion for determining the new weighted values ​​is refined during each iteration until the error associated with the primary map data is within the range defined by the one or more quality metric values.

[0054] Fig. 6 is a process flow diagram illustrating a method 600 for evaluating the error associated with the primary card data by the Fig. 1 shows the map quality rating system 10. Generally speaking, Fig.1-6, the method 600 may begin at block 602. At block 602, the one or more central computers 20 determine the error associated with the primary map data, where the primary map data represents the predefined virtually bounded area. As mentioned above, the one or more central computers 20 may determine the error associated with the primary map data based on a variety of different approaches. More specifically, in one embodiment, the error is an absolute offset between the primary map data points and the gold source map data points. In another embodiment, the one or more central computers 20 determine the error associated with the primary map data based on a self-assessment without the gold source map data.According to yet another embodiment, the error associated with the primary map data is based on a probability of user intervention during autonomous driving. Method 600 may then proceed to block 604.

[0055] At block 604, the selection module 34 of the one or more central computers 20 compares the error associated with the primary map data with the range of values ​​defined by the one or more quality metric values. The method 600 may then proceed to decision block 606.

[0056] At decision block 606, in response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values, the method 600 proceeds to block 608. At block 608, the selection module 34 of the one or more central computers 20 maintains the template for selecting the primary map data representing the predefined virtual bounded area. According to one embodiment, the one or more central computers 20 may generate a vehicle map based on the primary map data sent to the autonomous vehicle 26 located within the predefined virtual bounded area. The method 600 may then end.

[0057] Returning to decision block 606, in response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values, the method 600 proceeds to block 610. At block 610, the selection module 34 of the one or more central computers 20 evaluates the primary map data for real anomalies within the one or more roads represented by the primary map data. The method 600 may then proceed to decision block 612.

[0058] At decision block 612, in response to detecting one or more real anomalies within the one or more roads represented by the primary map data, the method 600 proceeds to block 614. At block 614, the selection module 34 of the one or more central computers 20 continues to monitor the primary map data until the real anomalies are no longer present and then collects a new set of primary map data for evaluation. The method 600 may then return to block 602.

[0059] Referring back to decision block 612, in response to determining that no real anomalies exist within the one or more roads represented by the primary map data, the method 600 proceeds to block 616. At block 616, the selection module 34 of the one or more central computers 20 implements the updated template for selecting the primary map data points representing the predefined virtually bounded area based on the one or more quality metric values. As noted above, the updated template is implemented by updating the weights assigned to the primary map data points of the primary map data based on a corresponding accuracy level. The method 600 may then end.

[0060] Generally speaking, based on the figures, the disclosed map quality evaluation system provides various technical effects and advantages. More specifically, the map quality evaluation system provides a method for evaluating the error associated with map data. It should be noted that the map quality evaluation system provides a variety of methods for determining the error associated with the map data. Specifically, the error can be determined based on gold-source map data or, alternatively, if the gold-source map data is unavailable, based on self-assessment.

[0061] The central computers may refer to, or be a part of, an electronic circuit, a combinational logic circuit, a field-programmable gate array (FPGA), a processor (shared, dedicated, or group) that executes code, or a combination of some or all of the above, such as in a system-on-chip. Additionally, the controllers may be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor may operate under the control of an operating system residing in memory. The operating system may manage computer resources such that computer program code embodied as one or more computer software applications, such as an application residing in memory, may have instructions that are executed by the processor.According to an alternative embodiment, the processor may execute the application directly, in which case the operating system may be omitted. legend

[0062] In the drawings, N stands for No and Y stands for Yes.

Claims

[1] A map quality evaluation system (10) that evaluates an error associated with primary map data, the map quality evaluation system (10) comprising: one or more central computers (20) in wireless communication with one or more communication networks (24) for receiving the primary card data, the one or more central computers (20) executing instructions to: Determining the error associated with the primary map data based on an absolute offset between primary map data points (100) corresponding to the primary map data and gold source map data points (102) corresponding to gold source map data, wherein the primary map data represents a predefined virtually bounded area; Comparing the error associated with the primary map data with a range of values ​​defined by one or more quality metric values, and Maintaining a template for selecting the primary map data representing the predefined virtually bounded area in response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values; characterized by , that the one or more central computers (20) further execute instructions to: Evaluating the primary map data for real anomalies within one or more roads represented by the primary map data in response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values; implementing an updated template for selecting primary map data points (100) representing the predefined virtually bounded area in response to determining that no real anomalies exist within the one or more roads represented by the primary map data based on the one or more quality metric values; Receiving road network data representing a road network for the predefined virtually bounded area, wherein the road network is a network graph that models roads based on a plurality of road segments (52); Aligning the primary map data points (100) and the gold source map data points (102) with each other, wherein the primary map data points (100) and the gold source map data points (102) are located at cross-sectional ends of each of the plurality of road sections (52); and Determining the absolute offset between the primary map data points (100) and the gold source map data for all cross-section ends of the road sections (52) located within the predefined virtually delimited area [2] The card quality rating system (10) of claim 1, wherein the one or more central computers (20) execute instructions for implementing the updated template, the instructions being implemented by: Updating weights assigned to the primary map data points (100) of the primary map data based on an appropriate level of accuracy. [3] The map quality evaluation system (10) of claim 2, wherein the weights corresponding to the primary map data points (100) containing a higher degree of accuracy based on the one or more quality metric values ​​are assigned a higher weighted value and the weights corresponding to the primary map data points (100) containing a lower degree of accuracy based on the one or more quality metric values ​​are assigned a lower weighted value. [4] The map quality evaluation system (10) of claim 1, wherein the one or more central computers (20) determine the absolute offset between the primary map data points (100) and the gold source map data points (102) based on a bounding box approach that generates a plurality of bounding boxes each enclosing one of the plurality of road segments (52) that are part of the road network. [5] The map quality evaluation system (10) according to claim 1, wherein a distance between the cross-sectional ends of the plurality of road sections (52) is measured, and wherein the distance between the cross-sectional ends of each road section (52) is dimensioned based on a targeting accuracy of the primary map data. [6] A map quality evaluation system (10) that evaluates an error associated with primary map data, the map quality evaluation system (10) comprising: one or more central computers (20) in wireless communication with one or more communication networks (24) for receiving the primary card data, the one or more central computers (20) executing instructions to: Determining the error associated with the primary map data, wherein the primary map data represents a predefined virtually bounded area; Comparing the error associated with the primary map data with a range of values ​​defined by one or more quality metric values, and Maintaining a template for selecting the primary map data representing the predefined virtually bounded area in response to determining that the error associated with the primary map data is within the range of values ​​defined by the one or more quality metric values; characterized by , that the one or more central computers (20) further execute instructions to: Evaluating the primary map data for real anomalies within one or more roads represented by the primary map data in response to determining that the error associated with the primary map data is outside the range defined by the one or more quality metric values; Implementing an updated template for selecting primary map data points (100) representing the predefined virtually bounded area in response to determining that no real anomalies exist within the one or more roads represented by the primary map data based on the one or more quality metric values; and Determining a temporal offset (114) between a first set of guidelines (120) and a second set of guidelines (122), wherein the temporal offset (114) represents the error associated with the primary map data and is a perpendicular distance measured at the same primary map data point (100) between a first timestamp and a second timestamp. [7] The map quality evaluation system (10) of claim 6, wherein the first set of guidelines (120) is determined at the first timestamp and is drawn based on the primary map data, and the second set of guidelines (122) is determined at the second timestamp, the second timestamp occurring after the first timestamp.

Images