Positioning evaluation method, device and equipment, medium and vehicle

Abstract

The invention discloses a positioning evaluation method, device and equipment, a medium and a vehicle, and the method comprises the steps: carrying out the matching of the geographic position information of each data frame with a corresponding geographic position truth value, and obtaining a successfully matched data frame; for each successfully matched data frame, calculating a positioning error corresponding to the data frame, and performing label classification on the driving information corresponding to the data frame to obtain a system-level label, a road-level label and a scene-level label corresponding to the data frame; for each system-level label, calculating a system-level positioning index based on the positioning errors of all the data frames corresponding to the system-level label, for each road-level label, calculating a road-level positioning index based on the positioning errors of all the data frames corresponding to the road-level label, and for each scene-level label, calculating a scene-level positioning index based on the positioning errors of all the data frames corresponding to the road-level label. And calculating a scene-level positioning index based on the positioning errors of all the data frames corresponding to the scene-level label. By applying the scheme of the invention, the evaluation effect and the evaluation efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and more specifically, to a positioning evaluation method, device, equipment, medium, and vehicle. Background Art

[0002] With the rapid development of positioning technology, the requirements for the positioning performance of the positioning algorithm of autonomous driving vehicles are getting higher and higher, because the accuracy of the positioning position will directly affect the user experience of using the map navigation service.

[0003] Therefore, in order to continuously improve the positioning effect of the positioning algorithm, it is necessary to evaluate the positioning effect of the positioning algorithm. If the positioning algorithm is directly evaluated on the terminal device, it will consume a lot of hardware resources of the terminal device, and since a large amount of positioning signal data cannot be obtained on a single terminal device, both the evaluation effect and the evaluation efficiency are not ideal. Summary of the Invention

[0004] This application provides a positioning evaluation method, device, equipment, medium, and vehicle, which improves the evaluation effect and evaluation efficiency. The specific technical solutions are as follows.

[0005] In a first aspect, this application provides a positioning evaluation method, including: [[ID=2Y]]

[0006] Obtain the positioning data stream to be measured of the autonomous driving vehicle. For each data frame in the positioning data stream to be measured, obtain the true geographical location corresponding to the data frame, and extract the geographical location information and driving information in the data frame;

[0007] Match the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matches;

[0008] For each data frame with a successful match, calculate the positioning error corresponding to the data frame, and classify the driving information corresponding to the data frame to obtain the system-level label, road-level label, and scene-level label corresponding to the data frame, where the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scene-level label represents the driving behavior of the autonomous driving vehicle;

[0009] For each system-level label, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label. For each road-level label, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label. For each scene-level label, calculate the scene-level positioning index based on the positioning errors of all data frames corresponding to the scene-level label.

[0010] Optionally, the step of matching the geographical location information of each data frame with the corresponding geographical location true value to obtain the data frames with successful matching includes:

[0011] Based on the nearest neighbor matching method, match the geographical location information of each data frame with the corresponding geographical location true value, mark the successfully matched geographical location information and the corresponding geographical location true value, and use the data frame where the successfully matched geographical location information is located as the data frame with successful matching.

[0012] Optionally, the step of calculating the positioning error corresponding to each data frame with successful matching includes:

[0013] For each data frame with successful matching, calculate the error between the geographical location information of the data frame and the corresponding geographical location true value as the positioning error corresponding to the data frame.

[0014] Optionally, the step of classifying the driving information corresponding to the data frame into the system-level label, road-level label, and scenario-level label corresponding to the data frame includes:

[0015] Generate the system-level label corresponding to the data frame from the part of the driving information corresponding to the data frame that matches the system-level regular expression, generate the road-level label corresponding to the data frame from the part that matches the road-level regular expression, and generate the scenario-level label corresponding to the data frame from the part that matches the scenario-level regular expression.

[0016] Optionally, the system-level positioning metrics include system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value. The step of calculating the system-level positioning metrics based on the positioning errors of all data frames corresponding to each system-level label includes:

[0017] For each system-level label, respectively determine the maximum positioning error as the system-level maximum value and the minimum positioning error as the system-level minimum value from the positioning errors of all data frames corresponding to the system-level label;

[0018] Based on the preset average value calculation formula, preset standard deviation calculation formula, and preset 3-sigma criterion, perform average value calculation, standard deviation calculation, and 3-sigma calculation on the positioning errors of all data frames corresponding to the system-level label to obtain the system-level average value, system-level standard deviation, and system-level 3-sigma value. Among them, the system-level 3-sigma value is the positioning error corresponding to the position of 3 standard deviations under the system-level label;

[0019] From the positioning errors of all data frames corresponding to the system-level tags, respectively determine a first quantity of positioning errors greater than a first preset error threshold and a second quantity of positioning errors greater than a second preset error threshold, where the first preset error threshold is less than the second preset error threshold;

[0020] Determine the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured;

[0021] Calculate the quotient of the total mileage and the first quantity as the system-level problem mileage value, where the system-level problem mileage value represents that there are a first quantity of positioning errors greater than the first preset error threshold within the total mileage;

[0022] Calculate the quotient of the total mileage and the second quantity as the system-level takeover mileage value, where the system-level takeover mileage value represents that there are a second quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0023] In a second aspect, the present application provides a positioning evaluation device, including:

[0024] An acquisition module, configured to acquire a positioning data stream to be measured of an autonomous vehicle, and for each data frame in the positioning data stream to be measured, acquire the true geographical location corresponding to the data frame, and extract the geographical location information and driving information in the data frame;

[0025] A matching module, which matches the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matches;

[0026] A label classification module, configured to, for each data frame with a successful match, calculate the positioning error corresponding to the data frame, and perform label classification on the driving information corresponding to the data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to the data frame, where the system-level label represents the overall road conditions where the autonomous vehicle is located, the road-level label represents the driving road where the autonomous vehicle is located, and the scenario-level label represents the driving behavior of the autonomous vehicle;

[0027] An index calculation module, configured to, for each system-level label, calculate a system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label, for each road-level label, calculate a road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label, and for each scenario-level label, calculate a scenario-level positioning index based on the positioning errors of all data frames corresponding to the scenario-level label.

[0028] Optionally, the matching module is specifically configured to:

[0029] Based on the nearest neighbor matching device, match the geographical location information of each data frame with the corresponding true geographical location value, mark the successfully matched geographical location information and the corresponding true geographical location value, and use the data frame where the successfully matched geographical location information is located as the successfully matched data frame.

[0030] Optionally, the label classification module is specifically configured to:

[0031] For each successfully matched data frame, calculate the error between the geographical location information of the data frame and the corresponding true geographical location value as the positioning error corresponding to the data frame.

[0032] Optionally, the label classification module is specifically configured to:

[0033] Generate the system-level label corresponding to the data frame for the part of the driving information corresponding to the data frame that matches the system-level regular expression, generate the road-level label corresponding to the data frame for the part that matches the road-level regular expression, and generate the scene-level label corresponding to the data frame for the part that matches the scene-level regular expression.

[0034] Optionally, the system-level positioning metrics include system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value. The metric calculation module includes:

[0035] The first calculation sub-module is used to, for each system-level label, respectively determine the maximum positioning error as the system-level maximum value and the minimum positioning error as the system-level minimum value from the positioning errors of all data frames corresponding to the system-level label;

[0036] The second calculation sub-module is used to perform average value calculation, standard deviation calculation, and 3-sigma calculation on the positioning errors of all data frames corresponding to the system-level label respectively based on a preset average value calculation formula, a preset standard deviation calculation formula, and a preset 3-sigma criterion to obtain the system-level average value, system-level standard deviation, and system-level 3-sigma value. Among them, the system-level 3-sigma value is the positioning error corresponding to the position of 3 standard deviations under the system-level label;

[0037] The quantity determination sub-module is used to respectively determine the first quantity of positioning errors greater than the first preset error threshold and the second quantity of positioning errors greater than the second preset error threshold from the positioning errors of all data frames corresponding to the system-level label, where the first preset error threshold is less than the second preset error threshold;

[0038] The total mileage determination sub-module is used to determine the total mileage traveled by the autonomous vehicle corresponding to the to-be-tested positioning data stream;

[0039] The system-level problem mileage value calculation sub-module is used to calculate the quotient of the total mileage and the first quantity as the system-level problem mileage value, where the system-level problem mileage value represents that there are a first quantity of positioning errors greater than the first preset error threshold within the total mileage;

[0040] The system-level takeover mileage value calculation sub-module is used to calculate the quotient of the total mileage and the second quantity as the system-level takeover mileage value, where the system-level takeover mileage value represents that there are a second quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0041] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the above first aspects is implemented.

[0042] In a fourth aspect, the present application provides an electronic device, and the electronic device includes:

[0043] One or more processors;

[0044] The processor is coupled to a storage device, and the storage device is used to store one or more programs,

[0045] When the one or more programs are executed by the one or more processors, the electronic device implements the method described in any one of the above first aspects.

[0046] In a fifth aspect, the present application provides a vehicle, and the vehicle includes the device described in any one of the above second aspects, or includes the electronic device described in the above fourth aspect.

[0047] As can be seen from the above, the positioning evaluation method, device, equipment, medium and vehicle provided by the embodiments of the present application can obtain the to-be-tested positioning data stream of an autonomous driving vehicle. For each data frame in the to-be-tested positioning data stream, obtain the true geographical location corresponding to the data frame, and extract the geographical location information and driving information in the data frame; match the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matches; for each data frame with a successful match, calculate the positioning error corresponding to the data frame, and classify the driving information corresponding to the data frame to obtain the system-level label, road-level label and scenario-level label corresponding to the data frame, where the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scenario-level label represents the driving behavior of the autonomous driving vehicle; for each system-level label, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label, for each road-level label, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label, and for each scenario-level label, calculate the scenario-level positioning index based on the positioning errors of all data frames corresponding to the scenario-level label. In the present application, by obtaining the to-be-tested positioning data stream of the autonomous driving vehicle, matching the geographical location information of each data frame with the corresponding true geographical location, further classifying the driving information of each successfully matched data frame, and calculating the positioning indexes of each level and label, the to-be-evaluated positioning algorithm is evaluated. There is no need to perform evaluation on the terminal device, which reduces resource consumption, and a large amount of to-be-tested positioning data stream support can be obtained, improving the evaluation efficiency. Moreover, since 3-level evaluations at the system level, road level and scenario level can be realized, more comprehensive and accurate evaluations can be achieved, greatly improving the evaluation effect.

[0048] The innovation points of the embodiments of the present application include:

[0049] 1. By obtaining the to-be-tested positioning data stream of the autonomous driving vehicle, matching the geographical location information of each data frame with the corresponding true geographical location, further classifying the driving information of each successfully matched data frame, and calculating the positioning indexes of each level and label, the to-be-evaluated positioning algorithm is evaluated. There is no need to perform evaluation on the terminal device, which reduces resource consumption, and a large amount of to-be-tested positioning data stream support can be obtained, improving the evaluation efficiency. Moreover, since 3-level evaluations at the system level, road level and scenario level can be realized, more comprehensive and accurate evaluations can be achieved, greatly improving the evaluation effect.

[0050] 2. When conducting system-level evaluations, multiple system-level positioning metrics can be obtained. The system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluations, thereby improving the evaluation effect.

[0051] 3. When conducting road-level evaluations, multiple road-level positioning metrics can be obtained. The road-level maximum value, road-level minimum value, road-level average value, road-level standard deviation, road-level 3-sigma value, road-level problem mileage value, and road-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluations, thereby improving the evaluation effect.

[0052] 4. When conducting scenario-level evaluations, multiple scenario-level positioning metrics can be obtained. The scenario-level maximum value, scenario-level minimum value, scenario-level average value, scenario-level standard deviation, scenario-level 3-sigma value, scenario-level problem mileage value, and scenario-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluations, thereby improving the evaluation effect.

[0053] 5. Since it is possible to implement three-level evaluations at the system level, road level, and scenario level, and multiple positioning metrics can be obtained when conducting evaluations at each level, more comprehensive and accurate positioning evaluations can be achieved, greatly improving the evaluation effect.

[0054] Of course, it is not necessary for any product or method implementing this application to simultaneously achieve all the above-mentioned advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is a schematic flowchart of a positioning evaluation method provided by an embodiment of this application;

[0057] Figure 2 It is a schematic structural diagram of a positioning evaluation device provided by an embodiment of this application;

[0058] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of this application;

[0059] Figure 4A structural schematic diagram of a vehicle provided by an embodiment of the present application. Detailed implementation manners

[0060] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0061] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0062] The embodiments of the present application disclose a positioning evaluation method, device, equipment, medium, and vehicle, which can improve the evaluation effect and evaluation efficiency. The embodiments of the present application will be described in detail below.

[0063] Figure 1 A flowchart of a positioning evaluation method provided by an embodiment of the present application. This method is applied to an electronic device. Exemplarily, the electronic device can be a cloud device. The method specifically includes the following steps.

[0064] S110: Obtain the to-be-tested positioning data stream of the autonomous vehicle. For each data frame in the to-be-tested positioning data stream, obtain the corresponding geographical location ground truth of the data frame, and extract the geographical location information and driving information in the data frame.

[0065] In order to evaluate the positioning effect of the to-be-tested positioning algorithm, it is necessary to obtain the to-be-tested positioning data stream of the autonomous vehicle. Among them, the to-be-tested positioning data stream is a data stream obtained by performing positioning calculations on the positioning signal data of the autonomous vehicle within a preset time period based on the to-be-tested positioning algorithm. The positioning signal data at least includes the geographical location information and driving information of the vehicle.

[0066] In order to determine whether the positioning effect of the to-be-tested positioning algorithm is accurate, it is necessary to compare with the geographical location ground truth. Therefore, after obtaining the to-be-tested positioning data stream, for each data frame in the to-be-tested positioning data stream, obtain the corresponding geographical location ground truth of the data frame, and extract the geographical location information and driving information in the data frame. Among them, the geographical location ground truth is obtained through GPS (Global Positioning System).

[0067] S120: Match the geographical location information of each data frame with the corresponding true geographical location value to obtain the data frames with successful matches.

[0068] After extracting the geographical location information and driving information in each data frame, match the geographical location information of each data frame with the corresponding true geographical location value to obtain the data frames with successful matches.

[0069] Among them, step S120 may include:

[0070] Match the geographical location information of each data frame with the corresponding true geographical location value based on the nearest neighbor matching method, mark the successfully matched geographical location information and the corresponding true geographical location value, and use the data frame where the successfully matched geographical location information is located as the data frame with a successful match.

[0071] S130: For each data frame with a successful match, calculate the positioning error corresponding to this data frame, and perform label classification on the driving information corresponding to this data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to this data frame. Among them, the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scenario-level label represents the driving behavior of the autonomous driving vehicle.

[0072] Even for the data frames with successful matches, there are still errors between the corresponding geographical location information and the true geographical location value. For each data frame with a successful match, it is necessary to calculate the positioning error corresponding to this data frame.

[0073] Among them, for each data frame with a successful match, calculating the positioning error corresponding to this data frame may include:

[0074] For each data frame with a successful match, calculate the error between the geographical location information of this data frame and the corresponding true geographical location value as the positioning error corresponding to this data frame.

[0075] Specifically, the method for calculating the error between the geographical location information of this data frame and the corresponding true geographical location value can be any error calculation method in the prior art, and the embodiments of the present application do not make any limitations in this regard.

[0076] In order to improve the accuracy of the evaluation results, three-level evaluations are proposed in the embodiments of the present application. The three-level evaluations correspond to three-level labels, namely the system-level label, road-level label, and scenario-level label. Among them, the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scenario-level label represents the driving behavior of the autonomous driving vehicle. Specifically, the overall road conditions may at least include how many roads there are in total. The driving behavior may at least include turning, going straight, going uphill, and going downhill.

[0077] Then, for each successfully matched data frame, classify the driving information corresponding to the data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to the data frame.

[0078] Specifically, classifying the driving information corresponding to the data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to the data frame may include:

[0079] Generate the system-level label corresponding to the data frame from the part of the driving information corresponding to the data frame that matches the system-level regular expression, generate the road-level label corresponding to the data frame from the part that matches the road-level regular expression, and generate the scenario-level label corresponding to the data frame from the part that matches the scenario-level regular expression.

[0080] Among them, the system-level regular expression, road-level regular expression, and scenario-level regular expression can be preset.

[0081] For example: Suppose the driving information is that autonomous vehicle A is driving on the Cth road in city B and is going uphill, and there are 100 roads in city B;

[0082] Then, classifying the driving information corresponding to the data frame to obtain the system-level label corresponding to the data frame is: 100 roads, the road-level label is: the Cth road, and the scenario-level label is: going uphill.

[0083] S140: For each system-level label, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label. For each road-level label, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label. For each scenario-level label, calculate the scenario-level positioning index based on the positioning errors of all data frames corresponding to the scenario-level label.

[0084] After obtaining the system-level label, road-level label, and scenario-level label corresponding to each successfully matched data frame, in order to evaluate the positioning effect of the positioning algorithm to be evaluated, calculate the positioning index for each level of label respectively. That is, for each system-level label, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label. For each road-level label, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label. For each scenario-level label, calculate the scenario-level positioning index based on the positioning errors of all data frames corresponding to the scenario-level label.

[0085] 1. Calculation of system-level positioning index:

[0086] System-level positioning metrics include system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value. For each system-level label, the steps to calculate system-level positioning metrics based on the positioning errors of all data frames corresponding to the system-level label are as follows:

[0087] For each system-level label, respectively determine the maximum positioning error as the system-level maximum value and the minimum positioning error as the system-level minimum value from the positioning errors of all data frames corresponding to the system-level label;

[0088] Based on a preset average formula, a preset standard deviation formula, and a preset 3-sigma criterion, perform average calculation, standard deviation calculation, and 3-sigma calculation on the positioning errors of all data frames corresponding to the system-level label to obtain the system-level average value, system-level standard deviation, and system-level 3-sigma value. Among them, the system-level 3-sigma value is the positioning error corresponding to the position of 3 standard deviations under the system-level label;

[0089] From the positioning errors of all data frames corresponding to the system-level label, respectively determine the first quantity of positioning errors greater than the first preset error threshold and the second quantity of positioning errors greater than the second preset error threshold. Among them, the first preset error threshold is less than the second preset error threshold;

[0090] Determine the total mileage of the autonomous vehicle corresponding to the positioning data stream to be measured;

[0091] Calculate the quotient of the total mileage and the first quantity as the system-level problem mileage value, where the system-level problem mileage value represents that there are the first quantity of positioning errors greater than the first preset error threshold within the total mileage;

[0092] Calculate the quotient of the total mileage and the second quantity as the system-level takeover mileage value, where the system-level takeover mileage value represents that there are the second quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0093] In order to more accurately and comprehensively evaluate the positioning effect of the positioning algorithm to be evaluated, in the embodiments of the present application, multiple positioning metrics for evaluation are proposed for each level of label. Among them, the system-level positioning metrics include system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value.

[0094] Then, for each system-level label, calculate each system-level positioning metric respectively.

[0095] Among them, the 3-sigma value calculated based on the preset 3-sigma criterion can be:

[0096] For each system-level tag, take the absolute value of the positioning errors of all data frames corresponding to the system-level tag, sort all the absolute positioning errors from smallest to largest, determine the total length of the data in the sorted queue, multiply the total length by 0.6826, 0.9544, and 0.9973 respectively to obtain three index positions, determine three positioning errors corresponding to the three index positions from the sorted queue, and take the largest positioning error among the three positioning errors as the system-level 3-sigma value.

[0097] Since the positioning data stream to be measured is the positioning data stream within a preset time period, therefore, the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured can be:

[0098] Determine that the total mileage traveled by the autonomous vehicle within the preset time period is the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured.

[0099] Then calculate the quotient of the total mileage and the first quantity as the system-level problem mileage value, where the system-level problem mileage value represents that there are a first quantity of positioning errors greater than the first preset error threshold within the total mileage, calculate the quotient of the total mileage and the second quantity as the system-level takeover mileage value, where the system-level takeover mileage value represents that there are a second quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0100] After calculating each system-level positioning index, each system-level positioning index can be displayed. The display method can be displayed as a table or displayed as a graph.

[0101] Through the displayed system-level positioning indexes, the user can intuitively understand the positioning effect of the positioning algorithm to be evaluated. Through the system-level maximum value, the user can know what the maximum positioning error is when the positioning algorithm to be evaluated is evaluated at the system level. Through the system-level minimum value, the user can know what the minimum positioning error is when the positioning algorithm to be evaluated is evaluated at the system level. Through the system-level average value, the user can know what the average positioning error is when the positioning algorithm to be evaluated is evaluated at the system level. Through the system-level standard deviation, the user can know what the standard deviation of the positioning error is when the positioning algorithm to be evaluated is evaluated at the system level. Through the system-level 3-sigma value, the user can know what the positioning errors corresponding to the positions of the three standard deviations are when the positioning algorithm to be evaluated is evaluated at the system level.

[0102] Through the system-level problem mileage value, the user can know how many times the positioning error exceeds the first preset error threshold within the total mileage when the positioning algorithm to be evaluated is evaluated at the system level. At this time, the first preset error threshold is set relatively small. If there are many positioning errors exceeding the first preset error threshold, it means that the positioning within this total mileage is not accurate enough, but at this time, the driver does not need to take over the autonomous driving yet.

[0103] By taking over the mileage value at the system level, it can be understood how many times the positioning error exceeds the second preset error threshold within the total mileage when evaluating the positioning algorithm to be evaluated at the system level. At this time, the second preset error threshold is set relatively large. If there are many positioning errors exceeding the second preset error threshold, it indicates that the positioning within this total mileage is very inaccurate. At this time, the driver needs to be notified to take over the automatic driving.

[0104] Therefore, when conducting system-level evaluation, multiple system-level positioning indicators can be obtained. The system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluation, thereby improving the evaluation effect.

[0105] The calculation methods for road-level positioning indicators and scenario-level positioning indicators are the same as those for system-level positioning indicators. Specifically, refer to the following:

[0106] 2. Calculation of road-level positioning indicators:

[0107] Road-level positioning indicators include road-level maximum value, road-level minimum value, road-level average value, road-level standard deviation, road-level 3-sigma value, road-level problem mileage value, and road-level takeover mileage value. For each road-level label, the steps for calculating road-level positioning indicators based on the positioning errors of all data frames corresponding to the road-level label include:

[0108] For each road-level label, respectively determine the maximum positioning error as the road-level maximum value and the minimum positioning error as the road-level minimum value from the positioning errors of all data frames corresponding to the road-level label;

[0109] Based on the preset average value calculation formula, preset standard deviation calculation formula, and preset 3-sigma criterion, respectively perform average value calculation, standard deviation calculation, and 3-sigma calculation on the positioning errors of all data frames corresponding to the road-level label to obtain the road-level average value, road-level standard deviation, and road-level 3-sigma value. Among them, the road-level 3-sigma value is the positioning error corresponding to the position of 3 standard deviations under the road-level label;

[0110] From the positioning errors of all data frames corresponding to the road-level label, respectively determine the third quantity of positioning errors greater than the first preset error threshold and the fourth quantity of positioning errors greater than the second preset error threshold, where the first preset error threshold is less than the second preset error threshold;

[0111] Determine the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured;

[0112] Calculate the quotient of the total mileage and the third quantity as the road-level problem mileage value, where the road-level problem mileage value indicates that there are a third quantity of positioning errors greater than the first preset error threshold within the total mileage;

[0113] Calculate the quotient of the total mileage and the fourth quantity as the road-level takeover mileage value, where the road-level takeover mileage value indicates that there are a fourth quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0114] In order to more accurately and comprehensively evaluate the positioning effect of the positioning algorithm to be evaluated, in the embodiments of the present application, a variety of positioning indicators for evaluation are proposed for each level of tags. Among them, the road-level positioning indicators include the road-level maximum value, the road-level minimum value, the road-level average value, the road-level standard deviation, the road-level 3-sigma value, the road-level problem mileage value, and the road-level takeover mileage value.

[0115] Then, for each road-level tag, calculate each road-level positioning indicator respectively.

[0116] Among them, the 3-sigma value calculated based on the preset 3-sigma criterion can be:

[0117] For each road-level tag, take the absolute value of the positioning errors of all data frames corresponding to the road-level tag, sort all the absolute value-taking positioning errors from smallest to largest, determine the total length of the data in the sorted queue, multiply the total length of the data by 0.6826, 0.9544, and 0.9973 respectively to obtain 3 index positions, determine 3 positioning errors corresponding to the 3 index positions from the sorted queue, and take the largest positioning error among the 3 positioning errors as the road-level 3-sigma value.

[0118] Since the positioning data stream to be measured is the positioning data stream within a preset time period, therefore, the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured can be:

[0119] Determine that the total mileage traveled by the autonomous vehicle within the preset time period is the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured.

[0120] Then calculate the quotient of the total mileage and the third quantity as the road-level problem mileage value, where the road-level problem mileage value indicates that there are a third quantity of positioning errors greater than the first preset error threshold within the total mileage, and calculate the quotient of the total mileage and the fourth quantity as the road-level takeover mileage value, where the road-level takeover mileage value indicates that there are a fourth quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0121] After calculating each road-level positioning indicator, each road-level positioning indicator can be displayed. The display method can be displayed as a table or displayed as a graph.

[0122] Through the displayed road-level positioning metrics, users can intuitively understand the positioning effect of the positioning algorithm to be evaluated. Through the road-level maximum value, they can know what the maximum positioning error is when the positioning algorithm to be evaluated is evaluated at the road level. Through the road-level minimum value, they can know what the minimum positioning error is when the positioning algorithm to be evaluated is evaluated at the road level. Through the road-level average value, they can know what the average positioning error is when the positioning algorithm to be evaluated is evaluated at the road level. Through the road-level standard deviation, they can know what the standard deviation of the positioning error is when the positioning algorithm to be evaluated is evaluated at the road level. Through the road-level 3-sigma value, they can know what the positioning error corresponding to the position of 3 standard deviations is when the positioning algorithm to be evaluated is evaluated at the road level.

[0123] Through the road-level problem mileage value, it can be known how many times the positioning error exceeds the first preset error threshold within the total mileage when the positioning algorithm to be evaluated is evaluated at the road level. At this time, the first preset error threshold is set relatively small. If there are many positioning errors exceeding the first preset error threshold, it means that the positioning within this total mileage is not accurate enough, but at this time, the driver does not need to take over the automatic driving yet.

[0124] Through the road-level takeover mileage value, it can be known how many times the positioning error exceeds the second preset error threshold within the total mileage when the positioning algorithm to be evaluated is evaluated at the road level. At this time, the second preset error threshold is set relatively large. If there are many positioning errors exceeding the second preset error threshold, it means that the positioning within this total mileage is very inaccurate, and at this time, the driver needs to be notified to take over the automatic driving.

[0125] Thus, when conducting road-level evaluation, multiple road-level positioning metrics can be obtained, and the road-level maximum value, road-level minimum value, road-level average value, road-level standard deviation, road-level 3-sigma value, road-level problem mileage value, and road-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluation, thereby improving the evaluation effect.

[0126] 3. Calculation of scenario-level positioning metrics:

[0127] The scenario-level positioning metrics include the scenario-level maximum value, scenario-level minimum value, scenario-level average value, scenario-level standard deviation, scenario-level 3-sigma value, scenario-level problem mileage value, and scenario-level takeover mileage value. For each scenario-level label, the steps for calculating the scenario-level positioning metrics based on the positioning errors of all data frames corresponding to the scenario-level label are as follows:

[0128] For each scenario-level label, respectively determine the maximum positioning error as the scenario-level maximum value and the minimum positioning error as the scenario-level minimum value from the positioning errors of all data frames corresponding to the scenario-level label;

[0129] Based on the preset average value calculation formula, preset standard deviation calculation formula, and preset 3-sigma criterion, calculate the average value, standard deviation, and 3-sigma value of the positioning errors of all data frames corresponding to the scene-level label respectively, where the scene-level 3-sigma value is the positioning error corresponding to the position of 3 standard deviations under the scene-level label;

[0130] From the positioning errors of all data frames corresponding to the scene-level label, determine the fifth quantity of positioning errors greater than the first preset error threshold and the sixth quantity of positioning errors greater than the second preset error threshold respectively, where the first preset error threshold is less than the second preset error threshold;

[0131] Determine the total mileage of the autonomous vehicle corresponding to the positioning data stream to be measured;

[0132] Calculate the quotient of the total mileage and the fifth quantity as the scene-level problem mileage value, where the scene-level problem mileage value represents that there are fifth quantity of positioning errors greater than the first preset error threshold within the total mileage;

[0133] Calculate the quotient of the total mileage and the sixth quantity as the scene-level takeover mileage value, where the scene-level takeover mileage value represents that there are sixth quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0134] In order to achieve a more accurate and comprehensive evaluation of the positioning effect of the positioning algorithm to be evaluated, in the embodiments of the present application, multiple positioning indicators for evaluation are proposed for each level of label. Among them, the scene-level positioning indicators include scene-level maximum value, scene-level minimum value, scene-level average value, scene-level standard deviation, scene-level 3-sigma value, scene-level problem mileage value, and scene-level takeover mileage value.

[0135] Then, for each scene-level label, calculate each scene-level positioning indicator respectively.

[0136] Among them, the 3-sigma value calculated based on the preset 3-sigma criterion can be:

[0137] For each scene-level label, take the absolute value of the positioning errors of all data frames corresponding to the scene-level label, sort all the absolute value-taking positioning errors from small to large, determine the total length of the data in the sorted queue, multiply the total length of the data by 0.6826, 0.9544, and 0.9973 respectively to obtain 3 index positions, determine the 3 positioning errors corresponding to the 3 index positions from the sorted queue, and take the maximum positioning error among the 3 positioning errors as the scene-level 3-sigma value.

[0138] Since the positioning data stream to be measured is the positioning data stream within a preset time period, therefore, the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured can be:

[0139] Determine that the total mileage traveled by the autonomous vehicle within the preset time period is the total mileage traveled by the autonomous vehicle corresponding to the positioning data stream to be measured.

[0140] Then calculate the quotient of the total mileage and the fifth quantity as the scenario-level problem mileage value. Among them, the scenario-level problem mileage value represents that there are a fifth quantity of positioning errors greater than the first preset error threshold within the total mileage. Calculate the quotient of the total mileage and the sixth quantity as the scenario-level takeover mileage value. Among them, the scenario-level takeover mileage value represents that there are a sixth quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0141] After calculating each scenario-level positioning index, each scenario-level positioning index can be displayed. The display method can be displayed as a table or displayed as a graph.

[0142] Through each scenario-level positioning index displayed, the user can intuitively understand the positioning effect of the positioning algorithm to be evaluated. Through the scenario-level maximum value, the user can know what the maximum positioning error is when the positioning algorithm to be evaluated is evaluated at the scenario level. Through the scenario-level minimum value, the user can know what the minimum positioning error is when the positioning algorithm to be evaluated is evaluated at the scenario level. Through the scenario-level average value, the user can know what the average positioning error is when the positioning algorithm to be evaluated is evaluated at the scenario level. Through the scenario-level standard deviation, the user can know what the standard deviation of the positioning error is when the positioning algorithm to be evaluated is evaluated at the scenario level. Through the scenario-level 3-sigma value, the user can know what the positioning error corresponding to the position of 3 standard deviations is when the positioning algorithm to be evaluated is evaluated at the scenario level. <

[0143] Through the scenario-level problem mileage value, the user can know how many times the positioning error exceeds the first preset error threshold within the total mileage when the positioning algorithm to be evaluated is evaluated at the scenario level. At this time, the first preset error threshold is set relatively small. If there are more positioning errors exceeding the first preset error threshold, it means that the positioning within this total mileage is not accurate enough, but at this time, the driver does not need to take over the autonomous driving yet.

[0144] Through the scenario-level takeover mileage value, the user can know how many times the positioning error exceeds the second preset error threshold within the total mileage when the positioning algorithm to be evaluated is evaluated at the scenario level. At this time, the second preset error threshold is set relatively large. If there are more positioning errors exceeding the second preset error threshold, it means that the positioning within this total mileage is very inaccurate, and at this time, the driver needs to be notified to take over the autonomous driving.

[0145] Thus, when performing scenario-level evaluation, multiple scenario-level positioning metrics can be obtained. The scenario-level maximum value, scenario-level minimum value, scenario-level average value, scenario-level standard deviation, scenario-level 3-sigma value, scenario-level problem mileage value, and scenario-level takeover mileage value cover most of the positioning evaluation requirements, enabling more comprehensive and accurate positioning evaluation, thereby improving the evaluation effect.

[0146] Moreover, since it is possible to implement three-level evaluations at the system level, road level, and scenario level, and multiple positioning metrics can be obtained during each level of evaluation, more comprehensive and accurate positioning evaluation can be achieved, greatly improving the evaluation effect.

[0147] As can be seen from the above, in this embodiment, the to-be-tested positioning data stream of the autonomous driving vehicle can be obtained. For each data frame in the to-be-tested positioning data stream, the corresponding ground truth of the geographical location is obtained, and the geographical location information and driving information in the data frame are extracted; the geographical location information of each data frame is matched with the corresponding ground truth of the geographical location to obtain the data frames with successful matches; for each data frame with a successful match, the positioning error corresponding to the data frame is calculated, and the driving information corresponding to the data frame is classified by label to obtain the system-level label, road-level label, and scenario-level label corresponding to the data frame. Among them, the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scenario-level label represents the driving behavior of the autonomous driving vehicle; for each system-level label, the system-level positioning metric is calculated based on the positioning errors of all data frames corresponding to the system-level label, for each road-level label, the road-level positioning metric is calculated based on the positioning errors of all data frames corresponding to the road-level label, and for each scenario-level label, the scenario-level positioning metric is calculated based on the positioning errors of all data frames corresponding to the scenario-level label. In this application, by obtaining the to-be-tested positioning data stream of the autonomous driving vehicle, matching the geographical location information of each data frame with the corresponding ground truth of the geographical location, further classifying the driving information of each data frame with a successful match by label, and calculating the positioning metrics of each level of label, the method is used to evaluate the to-be-tested positioning algorithm. There is no need to perform evaluation on the terminal device, reducing resource consumption, and a large amount of support from the to-be-tested positioning data stream can be obtained, improving the evaluation efficiency. Moreover, since three-level evaluations at the system level, road level, and scenario level can be implemented, more comprehensive and accurate evaluations can be achieved, greatly improving the evaluation effect.

[0148] Figure 2 It is a schematic structural diagram of a positioning evaluation device provided by an embodiment of the present application. Refer to Figure 2 An embodiment of the present application provides a positioning evaluation device, including:

[0149] An acquisition module 210 is configured to acquire a data stream of the positioning to be measured of an autonomous vehicle. For each data frame in the data stream of the positioning to be measured, the true geographical location corresponding to the data frame is acquired, and the geographical location information and driving information in the data frame are extracted.

[0150] A matching module 220 is configured to match the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matching.

[0151] A label classification module 230 is configured to, for each data frame with successful matching, calculate the positioning error corresponding to the data frame, perform label classification on the driving information corresponding to the data frame to obtain the system-level label, road-level label, and scene-level label corresponding to the data frame, where the system-level label represents the overall road conditions where the autonomous vehicle is located, the road-level label represents the driving road where the autonomous vehicle is located, and the scene-level label represents the driving behavior of the autonomous vehicle.

[0152] An index calculation module 240 is configured to, for each system-level label, calculate a system-level positioning index based on the positioning errors of all data frames corresponding to the system-level label, for each road-level label, calculate a road-level positioning index based on the positioning errors of all data frames corresponding to the road-level label, and for each scene-level label, calculate a scene-level positioning index based on the positioning errors of all data frames corresponding to the scene-level label.

[0153] The positioning evaluation device in this embodiment can obtain the to-be-tested positioning data stream of the autonomous vehicle. For each data frame in the to-be-tested positioning data stream, it obtains the true geographical location corresponding to this data frame, and extracts the geographical location information and driving information in this data frame; matches the geographical location information of each data frame with the corresponding true geographical location value to obtain the data frames with successful matches; for each data frame with a successful match, calculates the positioning error corresponding to this data frame, and classifies the driving information of this data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to this data frame. Among them, the system-level label represents the overall road conditions where the autonomous vehicle is located, the road-level label represents the driving road where the autonomous vehicle is located, and the scenario-level label represents the driving behavior of the autonomous vehicle; for each system-level label, calculates the system-level positioning index based on the positioning errors of all data frames corresponding to this system-level label, for each road-level label, calculates the road-level positioning index based on the positioning errors of all data frames corresponding to this road-level label, and for each scenario-level label, calculates the scenario-level positioning index based on the positioning errors of all data frames corresponding to this scenario-level label. In this application, by obtaining the to-be-tested positioning data stream of the autonomous vehicle, matching the geographical location information of each data frame with the corresponding true geographical location value, further classifying the driving information of each data frame with a successful match, and calculating the positioning indexes of each level of labels, the method realizes the evaluation of the to-be-tested positioning algorithm. There is no need to perform evaluation on the terminal device, reducing resource consumption, and it can obtain the support of a large amount of to-be-tested positioning data stream, improving the evaluation efficiency. And because it can achieve three-level evaluations at the system level, road level, and scenario level, it can achieve a more comprehensive and accurate evaluation, greatly improving the evaluation effect.

[0154] In one implementation manner, the matching module 220 can be specifically used for:

[0155] Based on the nearest neighbor matching device, match the geographical location information of each data frame with the corresponding true geographical location value, mark the successfully matched geographical location information and the corresponding true geographical location value, and use the data frame where the successfully matched geographical location information is located as the data frame with a successful match.

[0156] In one implementation manner, the label classification module 230 can be specifically used for:

[0157] For each data frame with a successful match, calculate the error between the geographical location information of this data frame and the corresponding true geographical location value as the positioning error corresponding to this data frame.

[0158] In one implementation manner, the label classification module 230 can be specifically used for:

[0159] Generate the system - level label corresponding to the data frame for the part of the driving information corresponding to the data frame that matches the system - level regular expression, the road - level label corresponding to the data frame for the part that matches the road - level regular expression, and the scene - level label corresponding to the data frame for the part that matches the scene - level regular expression.

[0160] In one implementation, the system - level positioning metrics include system - level maximum value, system - level minimum value, system - level average value, system - level standard deviation, system - level 3 - sigma value, system - level problem mileage value, and system - level takeover mileage value. The metric calculation module 240 may include:

[0161] The first calculation sub - module is used to determine, for each system - level label, the maximum positioning error as the system - level maximum value and the minimum positioning error as the system - level minimum value respectively from the positioning errors of all data frames corresponding to the system - level label.

[0162] The second calculation sub - module is used to perform average value calculation, standard deviation calculation, and 3 - sigma calculation on the positioning errors of all data frames corresponding to the system - level label respectively based on a preset average value calculation formula, a preset standard deviation calculation formula, and a preset 3 - sigma criterion to obtain the system - level average value, system - level standard deviation, and system - level 3 - sigma value. Among them, the system - level 3 - sigma value is the positioning error corresponding to the position of 3 standard deviations under the system - level label.

[0163] The quantity determination sub - module is used to determine, from the positioning errors of all data frames corresponding to the system - level label, the first quantity of positioning errors greater than the first preset error threshold and the second quantity of positioning errors greater than the second preset error threshold respectively. Among them, the first preset error threshold is less than the second preset error threshold.

[0164] The total mileage determination sub - module is used to determine the total mileage of the autonomous vehicle corresponding to the positioning data stream to be measured during driving.

[0165] The system - level problem mileage value calculation sub - module is used to calculate the quotient of the total mileage and the first quantity as the system - level problem mileage value. Among them, the system - level problem mileage value represents that there are the first quantity of positioning errors greater than the first preset error threshold within the total mileage.

[0166] The system - level takeover mileage value calculation sub - module is used to calculate the quotient of the total mileage and the second quantity as the system - level takeover mileage value. Among them, the system - level takeover mileage value represents that there are the second quantity of positioning errors greater than the second preset error threshold within the total mileage.

[0167] The above device embodiments correspond to the method embodiments and have the same technical effects as the method embodiments. For specific descriptions, please refer to the method embodiments. The device embodiments are obtained based on the method embodiments. For specific descriptions, please refer to the method embodiment section and will not be elaborated here.

[0168] Figure 3 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 3 shown, the electronic device includes:

[0169] One or more processors 310;

[0170] The processor 310 is coupled to the storage device 320, and the storage device 320 is used to store one or more programs;

[0171] When the one or more programs are executed by the one or more processors 310, the electronic device implements the positioning evaluation method provided in any embodiment of the present application.

[0172] Based on the above embodiments, another embodiment of the present application provides a vehicle, which includes the positioning evaluation device provided in any embodiment of the present application, or includes the electronic device provided in any embodiment of the present application.

[0173] Figure 4 The following is a schematic diagram of a vehicle provided by an embodiment of the present application. As Figure 4 shown, the vehicle includes a speed sensor 41, an ECU (Electronic Control Unit) 42, a GPS (Global Positioning System) positioning device 43, and a T-Box (Telematics Box) 44. Among them, the speed sensor 41 is used to measure the vehicle speed and use the vehicle speed as the empirical speed for model training; the GPS positioning device 43 is used to obtain the current geographical location of the vehicle; the T-Box 44 can be used as a gateway to communicate with the server; the ECU 42 can execute the above positioning evaluation method.

[0174] In addition, the vehicle may further include: a V2X (Vehicle-to-Everything) module 45, a radar 46, and a camera 47. The V2X module 45 is used to communicate with other vehicles, roadside devices, etc.; the radar 46 or the camera 47 is used to sense the road environment information in the front and / or other directions to obtain the original point cloud data; the radar 46 and / or the camera 47 can be configured at the front and / or the rear of the vehicle body.

[0175] Based on the above method embodiments, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the positioning evaluation method provided in any embodiment of the present application.

[0176] Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of one embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present application.

[0177] Those of ordinary skill in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be correspondingly changed and located in one or more devices different from this embodiment. The modules in the above embodiments can be combined into one module, or further split into multiple sub-modules.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A positioning evaluation method, characterized in that, Including: Obtain the positioning data stream to be measured of an autonomous driving vehicle. For each data frame in the positioning data stream to be measured, obtain the true geographical location corresponding to this data frame, and extract the geographical location information and driving information in this data frame; Match the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matches; For each data frame with a successful match, calculate the positioning error corresponding to this data frame, and perform label classification on the driving information corresponding to this data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to this data frame. Among them, the system-level label represents the overall road conditions where the autonomous driving vehicle is located, the road-level label represents the driving road where the autonomous driving vehicle is located, and the scenario-level label represents the driving behavior of the autonomous driving vehicle; For each system-level label, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to this system-level label. For each road-level label, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to this road-level label. For each scenario-level label, calculate the scenario-level positioning index based on the positioning errors of all data frames corresponding to this scenario-level label.

2. The method according to claim 1, characterized in that, The step of matching the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matches includes: Match the geographical location information of each data frame with the corresponding true geographical location based on the nearest neighbor matching method, mark the successfully matched geographical location information and the corresponding true geographical location, and use the data frame where the successfully matched geographical location information is located as the data frame with a successful match.

3. The method according to claim 1, characterized in that, The step of calculating the positioning error corresponding to each data frame with a successful match includes: For each data frame with a successful match, calculate the error between the geographical location information of this data frame and the corresponding true geographical location as the positioning error corresponding to this data frame.

4. The method according to claim 1, characterized in that The step of performing label classification on the driving information corresponding to this data frame to obtain the system-level label, road-level label, and scenario-level label corresponding to this data frame includes: Generate the system-level label corresponding to this data frame from the part of the driving information corresponding to this data frame that matches the system-level regular expression, generate the road-level label corresponding to this data frame from the part that matches the road-level regular expression, and generate the scenario-level label corresponding to this data frame from the part that matches the scenario-level regular expression.

5. The method according to claim 1, characterized in that The system-level positioning index includes system-level maximum value, system-level minimum value, system-level average value, system-level standard deviation, system-level 3-sigma value, system-level problem mileage value, and system-level takeover mileage value. The step of calculating the system-level positioning index based on the positioning errors of all data frames corresponding to each system-level label includes: For each system-level label, respectively determine the maximum positioning error as the system-level maximum value and the minimum positioning error as the system-level minimum value from the positioning errors of all data frames corresponding to this system-level label; Based on a preset average value calculation formula, a preset standard deviation calculation formula, and a preset 3-sigma criterion, calculate the average value, standard deviation, and 3-sigma value of the positioning errors of all data frames corresponding to the system-level tag respectively, to obtain the system-level average value, system-level standard deviation, and system-level 3-sigma value. Among them, the system-level 3-sigma value is the positioning error corresponding to the position where 3 standard deviations are located under the system-level tag; From the positioning errors of all data frames corresponding to the system-level tag, respectively determine the first quantity of positioning errors greater than the first preset error threshold and the second quantity of positioning errors greater than the second preset error threshold. Among them, the first preset error threshold is less than the second preset error threshold; Determine the total mileage of the autonomous vehicle corresponding to the to-be-tested positioning data stream; Calculate the quotient of the total mileage and the first quantity as the system-level problem mileage value, where the system-level problem mileage value represents that there are a first quantity of positioning errors greater than the first preset error threshold within the total mileage; Calculate the quotient of the total mileage and the second quantity as the system-level takeover mileage value, where the system-level takeover mileage value represents that there are a second quantity of positioning errors greater than the second preset error threshold within the total mileage.

6. A positioning evaluation device, characterized in that, Including: An acquisition module, configured to acquire the to-be-tested positioning data stream of the autonomous vehicle. For each data frame in the to-be-tested positioning data stream, acquire the true geographical location corresponding to the data frame, and extract the geographical location information and driving information in the data frame; A matching module, which matches the geographical location information of each data frame with the corresponding true geographical location to obtain the data frames with successful matching; A tag classification module, configured to, for each data frame with successful matching, calculate the positioning error corresponding to the data frame, and perform tag classification on the driving information corresponding to the data frame to obtain the system-level tag, road-level tag, and scenario-level tag corresponding to the data frame. Among them, the system-level tag represents the overall road condition where the autonomous vehicle is located, the road-level tag represents the driving road where the autonomous vehicle is located, and the scenario-level tag represents the driving behavior of the autonomous vehicle; An index calculation module, configured to, for each system-level tag, calculate the system-level positioning index based on the positioning errors of all data frames corresponding to the system-level tag, for each road-level tag, calculate the road-level positioning index based on the positioning errors of all data frames corresponding to the road-level tag, and for each scenario-level tag, calculate the scenario-level positioning index based on the positioning errors of all data frames corresponding to the scenario-level tag.

7. The device according to claim 6, characterized in that, The matching module is specifically configured to: Match the geographical location information of each data frame with the corresponding true geographical location based on the nearest neighbor matching device, mark the successfully matched geographical location information and the corresponding true geographical location, and use the data frame where the successfully matched geographical location information is located as the data frame with successful matching.

8. The device according to claim 6, characterized in that, The tag classification module is specifically configured to: For each data frame with successful matching, calculate the error between the geographical location information of the data frame and the corresponding true geographical location as the positioning error corresponding to the data frame.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the method described in any one of claims 1-5.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; The processor is coupled to a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the electronic device implements the method described in any one of claims 1-5.

Images