Map quality evaluation system

Abstract

A map quality assessment system that assesses an error associated with primary map data, the map quality assessment system comprising one or more central computers that execute instructions to determine an error associated with the primary map data, an error associated with the primary map data is compared to a range of values defined by the one or more quality metric values, and in response to determining that the error associated with the primary map data falls within the range of values defined by the one or more quality metric values, a template for selecting the primary map data is retained. In response to determining that an error associated with the primary map data falls outside of a range defined by the one or more quality metric values, the one or more central computers evaluate the primary map data for real anomalies within the one or more roads represented by the primary map data.

Description

Technical Field

[0001] The present invention relates to a map quality assessment system for assessing errors associated with map data. The map quality assessment system assesses errors associated with map data based on golden source map data or alternatively based on self-assessment without golden source map data. Background Art

[0002] Autonomous vehicles perform a variety of tasks, such as, but not limited to, perception, localization, mapping, path planning, decision making, and motion control. Autonomous vehicles rely on map data for many of the tasks they perform, such as localization, mapping, and path planning. It should be understood that different versions of map data representing the same geographic area can be generated, with each version of the map data generated from a different data source.

[0003] One example of a map data version is based on telemetry data. Telemetry data can be collected from many vehicles and combined based on various aggregation algorithms to determine various types of map content, such as, for example, Global Positioning System (GPS) tracks and subsequently inferred lane geometry. However, it should be understood that when comparing map content with content generated based on ground truth data, there may be errors such as, for example, lateral offset errors, random noise, etc. This error may cause the map data to indicate inaccurate locations of lane lines, which in turn may cause positioning problems and may lead to inaccurate decisions made by the control system of the autonomous vehicle.

[0004] Therefore, while maps for autonomous vehicles achieve their intended purpose, there is a need in the art for improved methods for assessing map quality. Summary of the Invention

[0005] According to several aspects, a map quality assessment system is disclosed that assesses errors associated with primary map data. The map quality assessment system includes one or more central computers in wireless communication with one or more communication networks for receiving primary map data. The one or more central computers execute instructions to determine errors associated with the primary map data, wherein the primary map data represents a predefined geofenced area. The one or more central computers compare the errors associated with the primary map data to a range of values ​​defined by one or more quality metrics. In response to determining that the errors associated with the primary map data fall within the range of values ​​defined by the one or more quality metrics, the one or more central computers retain a template for selecting primary map data representing the predefined geofenced area. In response to determining that the errors associated with the primary map data fall outside the range defined by the one or more quality metrics, the one or more central computers evaluate the primary map data for realistic anomalies within one or more roads represented by the primary map data. In response to determining that realistic anomalies are not present within the one or more roads represented by the primary map data, the one or more central computers implement an updated template for selecting primary map data points representing the predefined geofenced area based on the one or more quality metrics.

[0006] In another aspect, the one or more central computers execute instructions to implement the updated template by updating weights assigned to primary map data points of the primary map data based on corresponding accuracy levels.

[0007] In yet another aspect, weights corresponding to primary map data points comprising higher levels of accuracy are assigned higher weighted values ​​based on the one or more quality metric values, and weights corresponding to primary map data points comprising lower levels of accuracy are assigned lower weighted values ​​based on the one or more quality metric values.

[0008] In one aspect, the one or more central computers execute instructions to determine an error associated with the master map data based on an absolute offset between a master map data point corresponding to the master map data and a golden source map data point corresponding to the golden source map data.

[0009] In another aspect, one or more central computers execute instructions to: receive road network data representing a road network of the predefined geo-fenced area, wherein the road network is a network graph modeling roads based on a plurality of road segments.

[0010] In yet another aspect, the one or more central computers determine the absolute offset between the master map data point and the golden source map data point based on a bounding box method that creates a plurality of bounding boxes, each of the plurality of bounding boxes enclosing one of a plurality of road segments that are part of the road network.

[0011] In one aspect, the one or more central computers execute instructions to: align the master map data point and the golden source map data point with each other, wherein the master map data point and the golden source map data point are located at a cross-sectional end of each of the plurality of road segments, and determine an absolute offset between the master map data point and the golden source map data point for all cross-sectional ends of the road segments located within the predefined geo-fenced area.

[0012] In another aspect, distances between cross-sectional ends of the plurality of road segments are measured, and a magnitude of the distance between the cross-sectional ends of each road segment is determined based on a target accuracy of the master map data.

[0013] In yet another aspect, one or more central computers execute instructions to: determine a time offset between a first set of lane markings and a second set of lane markings, wherein the time offset represents an error associated with the master map data and is a vertical distance measured at the same master map data point between the first time step and the second time step.

[0014] In one aspect, a first set of lane markings is determined at a first time step and drawn based on primary map data, and a second set of lane markings is determined at a second time step, wherein the second time step occurs after the first time step.

[0015] In another aspect, one or more central computers execute instructions to: determine a spatial offset between a first set of lane markings and a second set of lane markings, wherein the spatial offset is measured between an end portion of the first set of lane markings and a starting position of the second set of lane markings, and the spatial offset represents an error associated with the primary map data.

[0016] In yet another aspect, a first set of lane markings is determined at a first location, and a second set of lane markings is determined at a second location directly adjacent to the first location.

[0017] In one aspect, the one or more central computers execute instructions to: determine a likelihood of user intervention, the likelihood of user intervention representing a number of times user intervention is required during autonomous driving when compared to a total number of times the autonomous vehicle passes a particular road segment within a predefined geo-fenced area, wherein the likelihood of user intervention represents an error associated with the primary map data.

[0018] On the other hand, the possibility of user intervention is expressed as:

[0019]

[0020] Among them, P i Indicates the possibility of user intervention, U i Indicates the number of times user intervention is required, and N i Indicates the total number of times a specific road segment has been passed by an autonomous vehicle.

[0021] In yet another aspect, a method for evaluating errors associated with primary map data includes determining, by one or more central computers, errors associated with the primary map data, wherein the primary map data represents a predefined geofenced area, and the one or more central computers are in wireless communication with one or more communication networks for receiving the primary map data. The method includes comparing, by the one or more central computers, the errors associated with the primary map data to a range of values ​​defined by one or more quality metrics. In response to determining that the errors associated with the primary map data fall within the range of values ​​defined by the one or more quality metrics, the method includes retaining, by the one or more central computers, a template for selecting primary map data representing the predefined geofenced area. In response to determining that the errors associated with the primary map data fall outside the range defined by the one or more quality metrics, the method includes evaluating, by the one or more central computers, the primary map data for realistic anomalies within one or more roads represented by the primary map data. In response to determining that realistic anomalies are not present within the one or more roads represented by the primary map data, the method includes implementing an updated template for selecting primary map data points representing the predefined geofenced area based on the one or more quality metrics.

[0022] In one aspect, the method further includes implementing the updated template by updating weights assigned to primary map data points of the primary map data based on corresponding accuracy levels.

[0023] In another aspect, the method further includes determining an error associated with the primary map data based on an absolute offset between primary map data points corresponding to the primary map data and golden source map data points corresponding to the golden source map data.

[0024] In yet another aspect, the method further includes receiving road network data representing a road network of the predefined geo-fenced area, wherein the road network is a network graph modeling roads based on a plurality of road segments.

[0025] In one aspect, the method further includes determining an absolute offset between the primary map data point and the golden source map data point based on a bounding box method that creates a plurality of bounding boxes, each of the plurality of bounding boxes enclosing one of a plurality of road segments that are part of the road network.

[0026] In another aspect, the method further includes aligning the master map data point and the golden source map data point with each other, wherein the master map data point and the golden source map data point are located at a cross-sectional end of each of the plurality of road segments; and determining an absolute offset between the master map data point and the golden source map data point for all cross-sectional ends of the road segments within the predefined geo-fenced area.

[0027] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0029] Figure 1 is a schematic diagram of a disclosed map quality assessment system according to an exemplary embodiment, the map quality assessment system including one or more central computers receiving map data via one or more communication networks;

[0030] Figure 2 According to an exemplary embodiment Figure 1 A block diagram of the software architecture of one or more central computers is shown;

[0031] Figure 3 is a schematic diagram of a bounding box determined by one or more central computers according to an exemplary embodiment;

[0032] Figure 4 is a schematic diagram of road network data including a plurality of cross-sectional road segments according to an exemplary embodiment;

[0033] Figure 5A is a schematic diagram illustrating a method of determining an error associated with primary map data based on temporal inconsistency according to an exemplary embodiment;

[0034] Figure 5B is a schematic diagram illustrating a method of determining an error associated with primary map data based on spatial inconsistency according to an exemplary embodiment; and

[0035] Figure 6 is a process flow diagram illustrating a method for evaluating errors associated with primary map data according to an exemplary embodiment. DETAILED DESCRIPTION

[0036] The following description is merely exemplary in nature and is not intended to limit the present invention, application, or uses.

[0037] refer to Figure 1 , illustrates an exemplary map quality assessment system 10 for assessing errors associated with map data. The map quality assessment system 10 includes one or more central computers 20 located in a back-end office 22, wherein the one or more central computers 20 wirelessly communicate with one or more communication networks 24. The one or more central computers 20 obtain master map data via the one or more communication networks 24. In one embodiment, the one or more central computers 20 may also obtain golden source map data via the one or more communication networks 24. However, it should be understood that in some embodiments, golden source map data may not be available.

[0038] Both the master map data and the golden source map data represent the same predefined geo-fenced area. The predefined geo-fenced area represents a real-world geographic area bounded by a virtual perimeter. It should be understood that the master map data is based on one or more unique data sources, such as, for example, Global Positioning System (GPS) data, image data collected by a vehicle's onboard camera, a vehicle telemetry source, aerial or satellite imagery data, or data collected from a survey vehicle. Some examples of versions of the master map data include, but are not limited to, crowdsourced map data, telemetry-based map data, and aerial map data. The golden source map data represents ground truth data, or alternatively, the most accurate and up-to-date version of map data for the predefined geo-fenced area that is currently available. In one non-limiting embodiment, the golden source map data is high-definition map data.

[0039] In such Figure 1In the illustrated, non-limiting embodiment, one or more central computers 20 wirelessly communicate with one or more autonomous vehicles 26. While autonomous vehicles are described, it should be understood that semi-autonomous vehicles equipped with advanced driver assistance systems (ADAS) may also be included. After evaluating the master map data, the one or more central computers 20 may share a vehicle map generated based on the master map data with the one or more autonomous vehicles 26. As described below, the one or more central computers 20 evaluate errors associated with the master map data based on the absolute offset between master map data points and golden source map data points. Alternatively, in another embodiment, the one or more central computers 20 determine errors associated with the master map data based on a self-evaluation without golden source map data. In yet another embodiment, the errors associated with the master map data are based on the likelihood of user intervention during autonomous driving. It should be understood that the errors associated with the master map data may represent any type of errors associated with the map data, such as, but not limited to, lateral offset errors or perception errors. In one embodiment, the master map data is created specifically for an autonomous driving system, such as an automated driving system (ADS) or ADAS.

[0040] Figure 2 yes Figure 1 A block diagram of the software architecture of one or more central computers 20 is shown in FIG. Figure 2 In the illustrated example, the one or more central computers 20 include a golden source evaluation module 30, a self-evaluation module 32, and a selection module 34. As described below, the golden source evaluation module 30 compares the master map data with the golden source map data to determine errors associated with the master map data, while the self-evaluation module 32 determines errors when the golden source map data is not available. Figure 1 and Figure 2 The golden source evaluation module 30 of one or more central computers 20 includes a bounding box evaluation submodule 40 that compares the main map data with the golden source map data based on a bounding box method, and a cross-section evaluation submodule 42 that compares the main map data with the golden source map data based on a cross-section method. Both the bounding box evaluation submodule 40 and the cross-section evaluation submodule 42 are described below.

[0041] One or more central computers 20 receive as input road network data representing a road network within a predefined geofenced area from one or more communication networks 24, where the road network is a network graph that models roads based on a plurality of road segments. The one or more central computers 20 also receive master map data and golden source map data from the one or more communication networks 24, both representing the predefined geofenced area. One example of road network data is Open Street Map (OSM), however, it should be understood that other types of road network data may also be used.

[0042] The method of evaluating the absolute offset between the master map data and the golden source map data based on the bounding box method will now be described. The bounding box evaluation submodule 40 of one or more central computers 20 divides the road network data into a plurality of road segments 52, such as Figure 3 Reference Figure 2 and Figure 3 , each of the plurality of road segments 52 comprises the same length, wherein the length of each road segment 52 can be equal to the average accurate perception range of the vehicle 26. In one non-limiting embodiment, the length is approximately 50 meters. The bounding box evaluation submodule 40 of the one or more central computers 20 creates a bounding box 64 corresponding to the primary map data and the golden source map data. Each bounding box 64 has a width 48 and a height 50 (i.e., a maximum height). Each bounding box 64 encloses a road segment 52, and the width 48 of each bounding box 64 is equal to the length of the road segment 52. The height 50 of the bounding box 64 can be greater than the width 48 of the bounding box 64.

[0043] refer to Figure 3 , the bounding box 64 surrounds the road segment 52. Each road segment 52 includes at least two road segment points (i.e., a first road segment point 66 and a second road segment point 68) that define the boundary of the road segment 52. Figure 3In FIG. 6 , the bounding box 64 includes a first linear boundary 70, a second linear boundary 72, a third linear boundary 74, and a fourth linear boundary 76. The first linear boundary 70 intersects the first road segment point 66. The second linear boundary 72 intersects the second road segment point 68. The first linear boundary 70 is parallel to the second linear boundary 72. The third linear boundary 74 is parallel to the fourth linear boundary 76. A distance 78 from the third linear boundary to the first road segment point 66 along the first direction y′ is equal to a distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y′. The height 50 (i.e., the maximum height) of the bounding box extends from the third linear boundary 74 to the fourth linear boundary 76. The distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y′ is half the maximum height of the first bounding box. The road segment 52 is parallel to the second direction x′. The second direction x′ is orthogonal to the first direction y′. The first and second directions y′ and x′ can be defined as axes of a vehicle coordinate system 82 forming the bounding box 64.

[0044] As described above, the first and second road segment points 66 and 68 define the extreme ends (i.e., endpoints) of the road segment 52. The bounding box evaluation submodule 40 of the one or more central computers 20 uses the first and second road segment points 66 and 68 to create the bounding box 64. In the vehicle coordinate system 82, the x' axis (defined by the x' direction) is parallel to the road segment 52, and the y' axis (defined by the y' direction) is perpendicular to the road segment 52. This angle θ represents the heading of the vehicle 26. In the vehicle coordinate system 82, the first road segment point 66 is at coordinates (x1, y1) of the vehicle coordinate system 82, and the second road segment point 68 is at coordinates (x2, y2). The distance 80 from the second road segment point 68 to the fourth linear boundary 76 along the first direction y' is half the height 50 of the bounding box 64. The distance 80 from the second road segment point 68 to the fourth linear boundary 76 is constant for all bounding boxes 64. In addition, a distance 80 from the second segment point 68 to the fourth linear boundary 76 along the first direction y′ may be represented by the letter “d.” The bounding box defines four corner points (i.e., a first corner point Q1 at coordinates (x1, y1+d), a second corner point Q2 at coordinates (x2, y2+d), a third corner point Q3 (x1, y1−d), and a fourth corner point Q4 (x2, y1−d)).

[0045] Continue to refer Figure 3, the coordinates of the origin of the vehicle coordinate system 82 are (m, n). The origin of the vehicle coordinate system 82 is converted to the origin of the global coordinate system 84. At this time, the coordinates of the bounding box are as follows: the first corner point Q1 (x1+m, y1+d+n), the second corner point Q2 (x2+m, y2+d+n), the third corner point Q3 (x1+m, y1-d+n), and the fourth corner point Q4 (x2+m, y2-d+n). Then, the vehicle coordinate system 82 is rotated so that the x' axis of the vehicle coordinate system 82 is aligned with the x axis of the global coordinate system 84, and the y' axis of the vehicle coordinate system 82 is aligned with the y axis of the global coordinate system 84. In this rotation, the rotation angle in the clockwise direction is θ (which is the forward direction of the vehicle 26). After the rotation, in the global coordinate system xy, the coordinates of the bounding box 64 are: the first corner point Q1 ((x1+m)*cosθ+(y1+d+n)*sinθ, (y1+d+n)*cosθ-(x1+m)*sinθ)), the second corner point Q2 ((x2+m)*cosθ+(y2+d+n)*sinθ, (y2+d+n)*cosθ-(x2+m)*sinθ)), the third corner point Q3 ((x1+m)*cosθ+(y1-d+n)*sinθ, (y1-d+n)*cosθ-(x1+m)*sinθ)) and the fourth corner point Q4 ((x2+m)*cosθ+(y2-d+n)*sinθ, (y2-d+n)*cosθ-(x2+m)*sinθ)).

[0046] The bounding box evaluation submodules 40 of the one or more central computers 20 create a primary map mosaic and a golden source map mosaic based on the primary map data and the golden source map data, respectively, by filtering the map data within the bounding box 64. The primary map mosaic is obtained by filtering the primary map data, and the golden source map mosaic is obtained by filtering the golden source map data. For all bounding boxes 64, the bounding box evaluation submodules 40 of the one or more central computers 20 create a primary map mosaic and a golden source map mosaic from the primary map data and the golden source map data, respectively. The filtered data used to create the map mosaics (i.e., the primary map mosaic and the golden source map mosaic) may be part of the map data within the bounding box 64 but outside the lane lines.

[0047] The bounding box evaluation submodule 40 of the one or more central computers 20 performs point cloud registration to align a plurality of primary map data points in the primary map mosaic with a plurality of golden source map data points in the golden source map mosaic using rotation and translation transformations to determine the absolute offset between the primary map data points of the primary map data and the golden source map data points of the golden source map data. Specifically, an iterative closest point (ICP) process or algorithm can use rotation and translation transformations to align the plurality of primary map data points in the primary map mosaic with the plurality of golden source map data points in the golden source map mosaic. Additionally, based on the point cloud registration, a random initialization process can be used to align the primary map data points with the golden source map data points.

[0048] The bounding box evaluation submodule 40 of the one or more central computers 20 uses a KD algorithm to construct a KD tree with the primary map data points. The primary map point data is first created as a line segment. The center point of the line segment is then obtained and assigned as a KD tree node. For each point in the golden source map data, the nearest point is determined using a KD tree query and the corresponding line segment from the primary map data. The offset from that point on the golden source map data to the line segment from the first map is then calculated. Based on the KD tree, the bounding box evaluation submodule 40 of the one or more central computers 20 determines the absolute offset from each of the multiple primary map data points to each corresponding golden source map data point. To this end, the bounding box evaluation submodule 40 of the one or more central computers 20 calculates the distance from one of the primary map data points to the corresponding golden source map data point in the golden source map data. This operation is repeated for all points to determine all absolute offsets between the primary map data points and the golden source map data points. The bounding box evaluation submodule 40 of the one or more central computers 20 then determines the relative map error based on a histogram of all absolute offsets.

[0049] A method for determining the absolute offset between a master map data point and a golden source map data point based on a cross-sectional approach will now be described. Figure 4is an illustration of road network data for a predefined geofenced area, wherein roads within the predefined geofenced area are represented by a plurality of road segments 92, a plurality of nodes 94, and opposing road edges 98. Each of the plurality of nodes 94 represents a cross-sectional end 96 of one of the road segments 92. The opposing road edges 98 represent theoretical road edges, rather than opposing topological edges. It should be understood that the opposing road edges 98 include unique edge identifiers (IDs), wherein each edge ID identifies a pair of opposing road edges 98 corresponding to a particular road segment 92. A distance D is measured between the cross-sectional ends 96 of the road segments 92, wherein the distance D between the cross-sectional ends 96 of each road segment 92 is sized based on a target accuracy of the master map data. By way of example only, if the master map data is created for an autonomous driving system such as an ADS or ADAS, the distance D between the cross-sectional ends 96 of the road segments 92 is approximately one meter.

[0050] The cross-section evaluation submodule 42 of the one or more central computers 20 determines the absolute offset between the master map data point 100 corresponding to the master map data and the golden source map data point 102 corresponding to the golden source map data by first aligning the master map data point 100 and the golden source map data point 102 with each other. Figure 4 As shown, the master map data point 100 and the golden source map data point 102 are located at the cross-sectional end 96 of the road segment 92. The cross-sectional evaluation submodule of the one or more central computers 20 aligns the master map data point 100 and the golden source map data point 102 with each other by executing one or more map matching algorithms to perform topological analysis. Figure 4 As shown, the cross-sectional end 96 of each road segment 92 intersects one or more master map data points 100 and one or more corresponding golden source map data points 102. The cross-sectional evaluation submodule 42 of the one or more central computers 20 then determines the absolute offset between the one or more master map data points 100 and the one or more corresponding golden source map data points 102 for all cross-sectional ends 96 of the road segments 92 that are within the predefined geo-fenced area. The cross-sectional evaluation submodule 42 of the one or more central computers 20 then determines the sum of all absolute offsets for each road segment 92 within the predefined geo-fenced area and divides the sum of all absolute offsets by the total number of road segments 92 within the predefined geo-fenced area to determine a relative error.

[0051] As described above, when golden source map data is not available, the self-assessment module 32 determines map errors. Figure 1 and Figure 2The self-assessment module 32 of the one or more central computers 20 includes: a temporal assessment submodule 44 that determines errors in the master map data based on temporal inconsistencies; a spatial assessment submodule 46 that determines errors in the master map data based on spatial inconsistencies; and a user intervention submodule 60 that determines errors in the master map data based on user intervention during autonomous or semi-autonomous driving. It should be understood that although Figure 2 One or more central computers 20 are shown that determine errors based on temporal or spatial inconsistencies, but the errors may be determined by autonomous vehicles 26 ( Figure 1 ) is determined locally by one or more controllers that are part of one of the ).

[0052] Determining the error based on the temporal inconsistency will now be described. Figure 5A 1 is a schematic diagram of a first set of lane lines 110 determined at a first time step T, where these lane lines are drawn based on the main map data. Figure 5A Also included is a second set of lane markings 112 determined at a second time step T+1, where the second time step T+1 occurs after the first time step T. The first set of lane markings 110 and the second set of lane markings 112 represent the same road.

[0053] refer to Figure 2 and Figure 5A The time assessment submodule 44 of the one or more central computers 20 determines a time offset 114 between the first set of lane markings 110 and the second set of lane markings 112, where the time offset 114 represents an error in the master map data. The time offset 114 represents the vertical distance measured at the same master map data point between the first time step T and the second time step T+1. The time assessment submodule 44 of the one or more central computers 20 then compares the time offset 114 to a time threshold, where the time threshold is selected based on a target level of accuracy. It should be understood that the target level of accuracy is determined based on the specific application of the master map data. In one embodiment, if the time error associated with any master map data point exceeds the time threshold, a notification is generated.

[0054] In one embodiment, the time evaluation submodule 44 of the one or more central computers 20 selects the maximum time offset value from the master map data points as the total time offset of the master map data. Specifically, Equation 1 determines the total time offset of the master map data as follows:

[0055]

[0056] II T represents the total time offset of the main map data, i represents the total number of main map data points defining lane lines 110 and 112, represents a point along the first set of lane lines 110 at the first time step T, and represents a point along the second set of lane markings 112 at a second time step T2.

[0057] Determining the error based on spatial inconsistency will now be described. Figure 5B 1 is a schematic diagram of a first set of lane lines 120 determined at a first position S and drawn based on the main map data. Figure 5B Also included is a second set of lane lines 122 based on the main map data determined at a second position S+1, where the second position S+1 is directly adjacent to the first position S. It should be understood that the second position S+1 is directly adjacent to the first position S in the lateral direction or the longitudinal direction.

[0058] refer to Figure 2 and Figure 5B , the spatial assessment submodule 46 of the one or more central computers 20 determines a spatial offset 124 between the first set of lane markings 120 and the second set of lane markings 122, wherein the spatial offset 124 is measured between an end portion 126 of the first set of lane markings 120 and a start position 128 of the second set of lane markings 122, where the first set of lane markings 120 and the second set of lane markings 122 overlap. Spatial offset 124 represents an error associated with the primary map data. The spatial assessment submodule 46 of the one or more central computers 20 then compares the spatial offset 124 to a spatial threshold, wherein the spatial threshold is selected based on the target accuracy level described above. In one embodiment, if the spatial error associated with any primary map data point exceeds the spatial threshold, a notification is generated.

[0059] In one embodiment, the spatial assessment submodule 46 of the one or more central computers 20 determines the spatial offset 124 based on Equation 2, which is as follows:

[0060]

[0061] II s represents the total spatial offset of the main data points, represents a point at the end portion 126 of the first set of lane markings 120, and represents a point at the starting position 128 of the second set of lane markings 122 .

[0062] refer to Figure 2 , the user intervention submodule 60 of the one or more central computers 20 determines the likelihood of user intervention during autonomous driving, wherein the likelihood of user intervention represents the error associated with the master map data. This is because when the error associated with the master map data is relatively low, the autonomous driving vehicle 26 ( Figure 1) can be navigated with minimal or no user intervention. However, as errors increase, there are increasing instances where a user navigates the vehicle during automated driving by performing one or more manual driving maneuvers. User intervention refers to the user performing one or more manual driving maneuvers to accommodate errors associated with the primary map data. Some examples of map errors that may cause a user to perform one or more manual driving maneuvers include, but are not limited to, map curvature errors, lane line inaccuracies, perception issues, missed signs, and incorrect placement of signs.

[0063] In one embodiment, the likelihood of user intervention represents the number of times user intervention is required during autonomous driving, when compared to the total number of times the autonomous vehicle has passed a particular road segment within a predefined geo-fenced area. Specifically, in one embodiment, the likelihood of user intervention is represented in Equation 3 and is as follows:

[0064]

[0065] Among them, P i Indicates the possibility of user intervention, U i Indicates the number of times user intervention is required, and N i Indicates the total number of times a specific road segment has been passed by an autonomous vehicle.

[0066] refer to Figure 2 The selection module 34 of one or more central computers 20 receives the error associated with the primary map data from the golden source assessment module 30 or the self-assessment module 32 and compares the error associated with the primary map data to one or more quality metric values. Each of the quality metric values ​​is based on a specific quality metric that indicates the quality of the primary map data and a target accuracy level for the primary map data. As described above, the target accuracy level is determined based on the specific application of the primary map data. By way of example only, in one embodiment, the quality metric value indicates the estimated high-level position error (EHPE) of the primary map data; however, it should be understood that other quality metrics, such as a confidence value, may also be used. In one non-limiting embodiment, the quality metric value may be defined by a range of values. For example, the range of quality metric values ​​may be represented as a number ranging from 0 to 9, where 0 represents the highest level of accuracy and 9 represents the lowest level of accuracy. In one embodiment, the one or more quality metric values ​​may be weighted based on importance, where primary map data points having a higher level of accuracy are assigned a higher importance when compared to primary map data points having a lower level of accuracy.

[0067] The selection module 34 of the one or more central computers 20 compares the error associated with the primary map data to the one or more quality metrics. In response to determining that the error associated with the primary map data falls within a range of values ​​defined by the one or more quality metrics, the selection module 34 retains a template for selecting the primary map data representing the predefined geo-fenced area based on the one or more quality metrics. The template for selecting primary map data points assigns weights to primary map data points, wherein greater weights are assigned to primary map data points that include a higher level of accuracy, and lesser weights are assigned to primary map data points that include a lower level of accuracy. The selection module 34 of the one or more central computers 20 may then generate a vehicle map based on the primary map data. In an embodiment, the vehicle map may be generated by one or more autonomous vehicles 26 ( Figure 1 )shared.

[0068] In response to determining that errors associated with the primary map data fall outside the range defined by the one or more quality metrics, the selection module 34 of the one or more central computers 20 then evaluates the primary map data for realistic anomalies within the one or more roads represented by the primary map data. Realistic anomalies within the one or more roads may adversely affect the accuracy of the primary map data. Some examples of realistic anomalies within the one or more roads include, but are not limited to, faded lanes, repainted lane markings, construction activity, road closures, and traffic accidents. In response to detecting one or more realistic anomalies within the one or more roads represented by the primary map data, the selection module 34 of the one or more central computers 20 continues to monitor the primary map data until the realistic anomalies no longer exist, and then collects a new set of primary map data for evaluation.

[0069] In response to determining that no real-world anomalies exist within one or more roads represented by the primary map data, the selection module 34 of the one or more central computers 20 implements an updated template for selecting primary map data points representing the predefined geofenced area based on the one or more quality metrics. The updated template is implemented by updating the weights assigned to the primary map data points of the primary map data based on their corresponding accuracy levels. Specifically, a higher weighting value is assigned to weights corresponding to primary map data points that include a higher level of accuracy based on the one or more quality metrics, while a lower weighting value is assigned to weights corresponding to primary map data points that include a lower level of accuracy based on the one or more quality metrics.

[0070] In one embodiment, weights are assigned to the primary map data points based on a probability distribution to ensure that the primary map data points include a mixture of different error levels. In one embodiment, the updated template is determined based on an iterative process, wherein the selection criteria for determining the new weight values ​​are refined during each iteration until the error associated with the primary map data falls within a range defined by one or more quality metrics.

[0071] Figure 6 Is used to pass Figure 1 A process flow diagram of a method 600 for evaluating errors associated with primary map data using the map quality assessment system 10 is shown in FIG. Figures 1-6 Method 600 may begin at block 602. In block 602, one or more central computers 20 determine an error associated with primary map data, where the primary map data represents a predefined geofenced area. As described above, the one or more central computers 20 may determine the error associated with the primary map data based on a variety of different methods. Specifically, in one embodiment, the error is an absolute offset between a primary map data point and a golden source map data point. In another embodiment, the one or more central computers 20 determine the error associated with the primary map data based on a self-assessment without golden source map data. In yet another embodiment, the error associated with the primary map data is based on the likelihood of user intervention during autonomous driving. Method 600 may then proceed to block 604.

[0072] In block 604 , the selection module 34 of the one or more central computers 20 compares the error associated with the primary map data to a range of values ​​defined by the one or more quality metrics. The method 600 may then proceed to decision block 606 .

[0073] In response to determining at decision block 606 that the error associated with the primary map data falls within a range of values ​​defined by the one or more quality metrics, method 600 proceeds to block 608. At block 608, selection module 34 of one or more central computers 20 retains a template for selecting primary map data representing the predefined geofenced area. In one embodiment, one or more central computers 20 may generate a vehicle map based on the primary map data transmitted to the autonomous vehicle 26 within the predefined geofenced area. Method 600 may then terminate.

[0074] Returning to decision block 606, in response to determining that the error associated with the primary map data falls outside the range defined by the one or more quality metrics, method 600 proceeds to block 610. In block 610, the selection module 34 of the one or more central computers 20 evaluates the primary map data for realistic anomalies within the one or more roads represented by the primary map data. Method 600 may then proceed to decision block 612.

[0075] In response to detecting one or more real-world anomalies within one or more roads represented by the primary map data at decision block 612, method 600 proceeds to block 614. At block 614, selection module 34 of one or more central computers 20 continues to monitor the primary map data until no real-world anomalies are present, and then collects a new set of primary map data for evaluation. Method 600 may then return to block 602.

[0076] Returning to decision block 612 , in response to determining that no real anomalies exist within the one or more roads represented by the primary map data, method 600 proceeds to block 616 . At block 616 , selection module 34 of one or more central computers 20 implements an updated template for selecting primary map data points representing the predefined geofenced area based on one or more quality metrics. As described above, the updated template is implemented by updating the weights assigned to the primary map data points of the primary map data based on their corresponding accuracy levels. Method 600 may then terminate.

[0077] Referring generally to the accompanying drawings, the disclosed map quality assessment system has various technical effects and benefits. Specifically, the map quality assessment system provides a method for assessing errors associated with map data. It should be understood that the map quality assessment system provides multiple methods for determining errors associated with map data. In particular, errors can be determined based on golden source map data or, alternatively, based on self-assessment when golden source map data is unavailable.

[0078] The central computer may refer to an electronic circuit, a combinational logic circuit, a field programmable gate array (FPGA), a processor (shared, dedicated, or group) that executes code, or a combination of some or all of the above, or as part of them, such as in a system on a chip. In addition, the controller may be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor may operate under the control of an operating system residing in the memory. The operating system may manage computer resources so that computer program code embodied as one or more computer software applications (such as applications residing in the memory) may have instructions executed by the processor. In an alternative embodiment, the processor may execute the applications directly, in which case the operating system may be omitted.

[0079] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to be within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

Claims

1. A map quality assessment system for assessing errors associated with master map data, the map quality assessment system comprising: One or more central computers in wireless communication with one or more communication networks for receiving the master map data, the one or more central computers executing instructions to: determining an error associated with the master map data, wherein the master map data represents a predefined geo-fenced area; comparing an error associated with the primary map data to a range of values ​​defined by one or more quality metrics; responsive to determining that the error associated with the primary map data falls within a range of values ​​defined by the one or more quality metric values, retaining a template for selecting primary map data representing the predefined geofenced area; In response to determining that an error associated with the primary map data falls outside a range defined by the one or more quality metric values, evaluating the primary map data for realistic anomalies within one or more roads represented by the primary map data; and In response to determining that no real anomalies exist within the one or more roads represented by the primary map data, an updated template for selecting primary map data points representing the predefined geo-fenced area based on the one or more quality metric values ​​is implemented.

2. The map quality assessment system according to claim 1, wherein: The one or more central computers execute instructions to implement the updated template, and the updated template is implemented by: Weights assigned to primary map data points of the primary map data are updated based on corresponding accuracy levels.

3. The map quality assessment system according to claim 2, wherein: Weights corresponding to primary map data points comprising higher levels of accuracy are assigned higher weighted values ​​based on the one or more quality metric values, and weights corresponding to primary map data points comprising lower levels of accuracy are assigned lower weighted values ​​based on the one or more quality metric values.

4. The map quality assessment system according to claim 1, wherein: The one or more central computers execute instructions to: An error associated with the primary map data is determined based on an absolute offset between primary map data points corresponding to the primary map data and golden source map data points corresponding to golden source map data.

5. The map quality assessment system according to claim 4, wherein: The one or more central computers execute instructions to: Road network data representing a road network of the predefined geo-fenced area is received, wherein the road network is a network graph modeling roads based on a plurality of road segments.

6. The map quality assessment system according to claim 5, wherein: The one or more central computers determine an absolute offset between the master map data point and the golden source map data point based on a bounding box method that creates a plurality of bounding boxes, each of the plurality of bounding boxes enclosing one of a plurality of road segments that are part of the road network.

7. The map quality assessment system according to claim 5, wherein: The one or more central computers execute instructions to: aligning the primary map data point and the golden source map data point with one another, wherein the primary map data point and the golden source map data point are located at cross-sectional ends of each of the plurality of road segments; and An absolute offset between the master map data point and the golden source map data point is determined for all cross-sectional ends of a road segment that are within the predefined geo-fenced area.

8. The map quality assessment system according to claim 7, wherein: Distances between cross-sectional ends of the plurality of road segments are measured, and wherein a magnitude of the distance between the cross-sectional ends of each road segment is determined based on a target accuracy of the main map data.

9. The map quality assessment system according to claim 1, wherein: The one or more central computers execute instructions to: A time offset between a first set of lane markings and a second set of lane markings is determined, wherein the time offset represents an error associated with the primary map data and is a vertical distance measured at the same primary map data point between a first time step and a second time step.

10. The map quality assessment system according to claim 9, wherein: The first set of lane markings is determined at the first time step and drawn based on the main map data, and the second set of lane markings is determined at the second time step, wherein the second time step occurs after the first time step.

Images