Calculation path evaluation method, device, equipment, medium and program product

Abstract

The invention provides a path calculation evaluation method, device and equipment, a medium and a program product, and relates to the field of navigation, and the method comprises the following steps: according to the path similarity between a first path calculation result of a to-be-evaluated path calculation service and second path calculation results of a plurality of preset path calculation services, calculating the path calculation result of the to-be-evaluated path calculation service; clustering each route in the first route calculation result and the second route calculation result to obtain a plurality of route clustering results; determining a weighted score of each route clustering result according to a preset weight coefficient corresponding to each route calculation service and a priority recommendation value of each route carried in each route calculation result; and according to a weighted scoring result, obtaining a first evaluation result of the to-be-evaluated road calculation service about the first road calculation result, the first evaluation result being used for indicating the road calculation quality of the to-be-evaluated service. According to the method, the path calculation evaluation efficiency and precision can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of navigation technology, and in particular to a route calculation evaluation method, device, equipment, medium and program product. Background Art

[0002] In the field of navigation, path calculation (or route calculation) is one of the core functions of the navigation system. Route calculation involves path planning from the starting point to the end point. With the development of intelligent transportation systems and the diversification of user needs, path calculation technology continues to evolve, and evaluating its performance and accuracy has become particularly important.

[0003] In the related art, the navigation route of the system is compared with the reference navigation route, and the quality of the navigation route of the system is evaluated based on the comparison results of indicators such as the route length between the road route and the reference navigation route, thereby achieving the route calculation quality of the route calculation service of the navigation route provider.

[0004] However, the evaluation method based on reference navigation routes is overly dependent on the quality of the reference navigation routes. If the quality of the selected navigation routes is too low, the evaluation accuracy will be greatly affected. In addition, since the route calculation service usually needs to evaluate a large number of planned routes to improve the evaluation accuracy, the one-to-one comparison method mentioned above consumes a lot of time, resulting in extremely low evaluation efficiency of the route calculation service. Summary of the invention

[0005] The present application provides a path calculation evaluation method, device, equipment, medium and program product to solve at least one of the above technical problems.

[0006] According to one aspect of the present application, a path calculation evaluation method is provided, comprising:

[0007] According to the first route calculation result of the route calculation service to be evaluated and the route similarity of each route in the second route calculation results of multiple preset route calculation services, clustering each route in the first route calculation result and the second route calculation result to obtain multiple route clustering results;

[0008] Determine the weighted score of each route clustering result according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result;

[0009] According to the weighted scoring result, a first evaluation result of the path calculation service to be evaluated with respect to the first path calculation result is obtained, where the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

[0010] In an optional implementation manner, the first path calculation result and the second path calculation result are obtained by:

[0011] In response to a trigger start for the evaluation task, a pre-selected path calculation sample set from a starting point of the evaluation location to an end point of the evaluation location is obtained; wherein the path calculation sample set is obtained by extracting multi-source navigation data;

[0012] Calling the to-be-evaluated path calculation service to perform path planning for the path calculation sample set to obtain the first path calculation result; and

[0013] Multiple preset path calculation services are called to perform path planning for the path calculation sample set respectively to obtain the second path calculation result.

[0014] In an optional implementation manner, obtaining, according to the weighted scoring result, a first evaluation result of the path calculation service to be evaluated regarding the first path calculation result includes:

[0015] According to the weighted scoring result, the plurality of route clustering results are sorted according to the score to obtain a reasonable route sorting result;

[0016] A first evaluation result of the first route calculation result on the route is determined according to data information of each route in the first route calculation result falling into the reasonable route ranking result.

[0017] In an optional implementation manner, determining the first evaluation result of the first route calculation result on the route according to the data information that each route in the first route calculation result falls into the reasonable route sorting result includes at least one of the following:

[0018] Determining a rationality rate of the first route calculation result according to the number of routes in the first route calculation result that fall into the preset ranking of the reasonable route ranking result and the total number of routes in the first route calculation result;

[0019] Determine the optimal rate of the first route calculation result according to the number of routes with the highest priority recommendation value in the first route calculation result that fall into the most forward-ranked route cluster result in the reasonable route ranking result and the total number of routes with the highest priority recommendation value in the first route calculation result;

[0020] Determining the diversity rate of the first route calculation result according to the number of routes in the first route calculation result that fall into different route clustering results in the reasonable route sorting result and the total number of routes in the first route calculation result;

[0021] The abnormality rate of the first route calculation result is determined based on at least one of the number of routes in the first route calculation result whose route distance is greater than the distance threshold of the shortest route in the reasonable route sorting result, the number of routes in the first route calculation result whose route congestion distance reaches a preset proportion of the total route length, and the number of routes in the first route calculation result whose route traffic lights reach a preset number, and the total number of routes in the first route calculation result. In an optional implementation, the method for determining the weight coefficient includes at least one of the following:

[0022] In response to a setting operation on a weight setting area in the evaluation task, obtaining a weight coefficient corresponding to each path calculation service;

[0023] A weight coefficient of each path calculation service is determined according to at least one of the confidence level, user usage and user score of each path calculation service.

[0024] In an optional implementation, the weighted score of each route clustering result is determined according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result to obtain the weighted score result, including:

[0025] For each path calculation result, weight each line in the path calculation result according to the weight coefficient of the corresponding path calculation service to obtain a weighted value;

[0026] Determine the score of each route according to the product of the weighted value and the priority recommendation value of each route; wherein the priority recommendation value is determined according to the priority recommendation rule of the corresponding route calculation service, and the priority recommendation rule includes at least one of the shortest route distance rule, the minimum route time rule and the user preference rule;

[0027] For each route clustering result, a weighted score of each route clustering result is obtained according to the sum of the scores of all routes in the route clustering result.

[0028] In an optional implementation manner, the method for calculating the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services includes:

[0029] According to the pre-divided grid size, the map data is gridded to obtain a geographic grid;

[0030] For each route in the first route calculation result and the second route calculation result, obtaining the number of grids whose shape points of each route fall in the same geographical network;

[0031] The similarity between each route is calculated according to the number of grids to obtain the route similarity.

[0032] In an optional implementation, the method further includes:

[0033] According to the weighted scoring result, obtaining second evaluation results of the plurality of preset path calculation services on respective second path calculation results, wherein the second evaluation results are used to indicate the path calculation quality of each preset path calculation service;

[0034] In response to a triggering operation on a comparison result between the path calculation services, comparison information between the first evaluation result and the second evaluation result is displayed.

[0035] According to a second aspect of the present application, a path calculation evaluation device is provided, including:

[0036] a clustering module configured to cluster the routes in the first route calculation result and the second route calculation result of the route calculation service to be evaluated according to the route similarity of each route in the second route calculation results of the plurality of preset route calculation services, so as to obtain a plurality of route clustering results;

[0037] A weighted scoring module is configured to determine a weighted score for each route clustering result based on a preset weight coefficient corresponding to each route calculation service and a priority recommendation value of each route carried in each route calculation result;

[0038] The evaluation result acquisition module is configured to acquire a first evaluation result of the path calculation service to be evaluated on the first path calculation result according to the weighted scoring result, wherein the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

[0039] In an optional implementation manner, a path calculation acquisition module for acquiring the first path calculation result and the second path calculation result is further included, which includes:

[0040] A sample set acquisition unit, which is configured to acquire a path calculation sample set from a pre-selected evaluation location starting point to an evaluation location end point in response to a trigger start for the evaluation task; wherein the path calculation sample set is obtained by extracting multi-source navigation data;

[0041] A first path calculation unit is configured to call the path calculation service to be evaluated to perform path planning for the path calculation sample set to obtain the first path calculation result; and

[0042] The second path calculation unit is configured to call a plurality of preset path calculation services to perform path planning for the path calculation sample set respectively to obtain the second path calculation result.

[0043] In an optional implementation manner, the evaluation result acquisition module includes:

[0044] A reasonable route sorting unit, which is configured to sort the plurality of route clustering results according to the weighted scoring result and the size of the scores to obtain a reasonable route sorting result;

[0045] An evaluation unit is configured to determine a first evaluation result of the first route calculation result on a route according to data information of each route in the first route calculation result falling into the reasonable route sorting result.

[0046] In an optional implementation, the evaluation unit includes at least one of the following:

[0047] a rationality rate acquisition unit, configured to determine the rationality rate of the first route calculation result according to the number of routes in the first route calculation result that fall into a preset ranking in the rational route ranking result and the total number of routes in the first route calculation result;

[0048] an optimal rate acquisition unit, configured to determine the optimal rate of the first route calculation result according to the number of routes with the highest priority recommendation value in the first route calculation result that fall into the most forward-ranked route cluster result in the reasonable route ranking result and the total number of routes with the highest priority recommendation value in the first route calculation result;

[0049] a diversity rate acquisition unit, configured to determine the diversity rate of the first route calculation result according to the number of routes in the first route calculation result that fall into different route clustering results in the reasonable route sorting result and the total number of routes in the first route calculation result;

[0050] The abnormality rate acquisition unit is configured to determine the abnormality rate of the first route calculation result based on at least one of the number of routes in the first route calculation result whose route distance is greater than the distance threshold of the shortest route in the reasonable route sorting result, the number of routes in the first route calculation result whose route congestion distance reaches a preset proportion of the total route length, and the number of routes in the first route calculation result whose route traffic lights reach a preset number, and the total number of routes in the first route calculation result. In an optional embodiment, it also includes a weight determination module for determining a weight coefficient, which includes at least one of the following:

[0051] A first weight acquisition unit, configured to acquire a weight coefficient corresponding to each path calculation service in response to a setting operation on a weight setting area in the evaluation task;

[0052] The second weight acquisition unit is configured to determine a weight coefficient of each path calculation service according to at least one of the confidence level, user usage and user score of each path calculation service.

[0053] In an optional implementation, the weighted scoring module includes:

[0054] A weighting unit, which is configured to weight each line in each path calculation result according to a weight coefficient of a corresponding path calculation service to obtain a weighted value;

[0055] a score determination unit, configured to determine the score of each route according to the product of the weighted value and the priority recommendation value of each route; wherein the priority recommendation value is determined according to a priority recommendation rule of the corresponding route calculation service, and the priority recommendation rule includes at least one of a shortest route distance rule, a minimum route time rule, and a user preference rule;

[0056] The scoring unit is configured to obtain a weighted score for each route clustering result according to the sum of scores of all routes in the route clustering result.

[0057] In an optional implementation, the device further includes a similarity calculation module for calculating the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services, which includes:

[0058] A gridding unit, which is configured to grid the map data including the starting point of the evaluation location and the end point of the evaluation location according to the pre-divided grid size to obtain a geographic grid;

[0059] A grid number acquisition unit, configured to acquire, for each route in the first route calculation result and the second route calculation result, the number of grids in which the shape points of each route fall in the same geographical network;

[0060] The similarity calculation unit is configured to calculate the similarity between each route according to the number of grids to obtain the route similarity.

[0061] In an optional embodiment, the device further includes a display module;

[0062] The evaluation result acquisition unit is further configured to acquire second evaluation results of the plurality of preset path calculation services on respective second path calculation results according to the weighted scoring result, wherein the second evaluation results are used to indicate the path calculation quality of each preset path calculation service;

[0063] The display module is configured to display the comparison information between the first evaluation result and the second evaluation result in response to a trigger operation on the comparison result between the path calculation services.

[0064] According to the third aspect of the present application, an electronic device is provided, comprising: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the path calculation evaluation method provided in any one of the first aspects above.

[0065] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the path calculation evaluation method provided in any one of the first aspects above.

[0066] According to a fifth aspect of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the path calculation and evaluation method provided in any one of the first aspects above.

[0067] The path calculation evaluation method, device, equipment, medium and program product provided in the present application cluster the routes in the first path calculation result and the second path calculation result of the path calculation service to be evaluated according to the route similarity between the first path calculation result and the second path calculation results of multiple preset path calculation services to obtain multiple route clustering results, determine the weighted score of each route clustering result according to the preset weight coefficient corresponding to each path calculation service and the priority recommendation value of each route carried in each path calculation result, obtain the first evaluation result of the path calculation service to be evaluated regarding the first path calculation result according to the weighted score result, and the first evaluation result is used to indicate the path calculation quality of the service to be evaluated. In this process, by clustering the route similarities of the route calculation results of all parties, when the number of planned routes is large, the process of analyzing a single piece of data one by one can be reduced, thereby improving the overall evaluation efficiency. At the same time, the route is scored using the multi-dimensional weight coefficients of the route calculation service provider and the priority recommendation value of each route, and the route calculation service to be evaluated is evaluated based on the scoring results, which effectively integrates the data information of all parties, reduces the excessive reliance on the quality of a single reference route, and effectively improves the scoring accuracy of the route to be evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] Figure 1a A possible scenario diagram provided for an embodiment of the present application;

[0070] Figure 1b A schematic diagram of another possible scenario provided for an embodiment of the present application;

[0071] Figure 2 A flowchart of a path calculation evaluation method provided in an embodiment of the present application;

[0072] Figure 3 A flowchart of another path calculation evaluation method provided in an embodiment of the present application;

[0073] Figure 4A flowchart of another path calculation evaluation method provided in an embodiment of the present application;

[0074] Figure 5a This is an interface diagram of a path calculation sample set in an embodiment of the present application;

[0075] Figure 5b This is one of the interface diagrams for uploading a path calculation sample set in an embodiment of the present application;

[0076] Figure 5c This is the second interface diagram for uploading a path calculation sample set in an embodiment of the present application;

[0077] Figure 6 This is an interface diagram of the evaluation task in the embodiment of this application;

[0078] Figure 7a This is one of the interface diagrams of the route clustering results in the embodiment of the present application;

[0079] Figure 7b This is the second interface diagram of the route clustering result in the embodiment of the present application;

[0080] Figure 8 A schematic diagram of the structure of a path calculation evaluation device provided in an embodiment of the present application;

[0081] Figure 9a This is one of the interface diagrams for comparing the first evaluation result and the second evaluation result in the embodiment of the present application;

[0082] Figure 9b This is the second interface diagram of the comparison information between the first evaluation result and the second evaluation result in the embodiment of the present application;

[0083] Fig.9c This is the third interface diagram of the comparison information between the first evaluation result and the second evaluation result in the embodiment of the present application;

[0084] Figure 9d This is the fourth interface diagram of the comparison information between the first evaluation result and the second evaluation result in the embodiment of the present application;

[0085] Fig.10 A schematic diagram of the structure of a path calculation evaluation device provided in an embodiment of the present application;

[0086] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0087] Fig.12 It is a block diagram of a server provided by an exemplary embodiment of the present application.

[0088] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0089] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0090] The following is an explanation of the embodiments of the present application in conjunction with the application scenario. The route calculation and evaluation method provided in the embodiments of the present application can be applied to the application scenario of intelligent driving. For example, the execution subject of the method provided in the embodiments of the present application can be a server, specifically, it can be a server of a navigation service provider, or it can be a server of a route calculation and evaluation party, and this embodiment does not specifically limit this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, and cloud computing, and this embodiment does not specifically limit this.

[0091] Figure 1a A schematic diagram of a scenario of a path calculation evaluation method provided in an embodiment of the present application, such as Figure 1aAs shown, it includes a path calculation evaluation server 110, a first path calculation server 120, and multiple second path calculation servers 130, wherein the path calculation service provided by the first path calculation server 120 can be a path calculation service to be evaluated, and the path calculation service provided by the second path calculation server 130 can be a path calculation service to be evaluated, or a path calculation service that has been evaluated, and this embodiment does not specifically limit this. Optionally, the server 110 can call the path calculation results (i.e., the first path calculation result and the second path calculation result) generated by the first path calculation server 120 and the second path calculation server 130 for the same path calculation sample set, and cluster the first path calculation result and the second path calculation result by lines, and at the same time, combine the weight coefficient of the path calculation service and the weighted scoring method of the priority recommendation value of each line to realize the path calculation evaluation result of the path calculation service to be measured, thereby realizing the path calculation quality evaluation of the path calculation service to be measured, and the evaluation result can be sent back to the corresponding path calculation server to provide data support for the development and improvement of the path calculation service. It is understandable that the above-mentioned path calculation server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, and cloud computing, and this embodiment does not specifically limit this. In some application scenarios, the server 110 can undertake the computing work alone, that is, there is no need to obtain the first path calculation result and the second path calculation result transmitted by the first path calculation server 120 and the second path calculation server, and perform the path calculation evaluation process alone. Exemplarily, the first path calculation result of the service to be evaluated and the second path calculation result of the preset path calculation service can be stored in the server 110, and the server 110 executes the path calculation evaluation process of the service to be evaluated based on the stored first path calculation result and the second path calculation result.

[0092] In another application scenario, the execution subject of the method provided in this embodiment may also be a terminal device, such as a terminal device of the road calculation evaluation party, and the terminal device may include but is not limited to a computer, a smart phone, a tablet computer, an e-book reader, a portable computer, a car computer, a wearable device, a desktop computer, etc. Figure 1b Another possible application scenario provided by this embodiment is Figure 1a The application scenarios shown are different in that, through the terminal device 140, the first path calculation server 120 and the plurality of second path calculation servers 130 are electrically connected, the terminal device 140 can receive the user's evaluation instruction, create or trigger the evaluation task, and call the first path calculation service and the second path calculation service to perform path calculation to obtain the first path calculation result and the second path calculation result, and execute the path calculation evaluation process of the service to be evaluated according to the first path calculation result and the second path calculation result. This embodiment does not specifically limit the application scenario of the path calculation evaluation method.

[0093] Instead of performing route quality scoring by comparing navigation routes with reference navigation routes one by one in the related art, the embodiment of the present application provides a technical solution by clustering the routes in the first route calculation result and the second route calculation result of the route calculation service to be evaluated according to the route similarity between the first route calculation result and the second route calculation results of multiple preset route calculation services, to obtain multiple route clustering results, and determining the weighted score of each route clustering result according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result, and obtaining the first evaluation result of the route calculation service to be evaluated regarding the first route calculation result according to the weighted scoring result, and the first evaluation result is used to indicate the route calculation quality of the service to be evaluated. In this process, by clustering the route similarities of the route calculation results of all parties, when the number of planned routes is large, the process of analyzing a single piece of data one by one can be reduced, thereby improving the overall evaluation efficiency. At the same time, the route is scored using the multi-dimensional weight coefficients of the route calculation service provider and the priority recommendation value of each route, and the scoring result is used to evaluate the route calculation service to be evaluated, which effectively integrates the data information of all parties, reduces the excessive dependence on the quality of the reference route, and effectively improves the scoring accuracy of the route to be evaluated.

[0094] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above technical problems in conjunction with the accompanying drawings and specific embodiments. It should be noted that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0096] Figure 2 A schematic diagram of a flow chart of a path calculation evaluation method provided in an embodiment of the present application, such as Figure 2 As shown, the execution subject may be the server 110, and the method may include steps S201-S203:

[0097] Step S201: cluster the routes in a first route calculation result of a route calculation service to be evaluated and second route calculation results of a plurality of preset route calculation services according to the route similarities of the routes in the first route calculation result and the second route calculation results to obtain a plurality of route clustering results.

[0098] In this embodiment, the first route calculation result may be a path planning result based on the evaluation location starting point and the evaluation location end point of the route calculation service to be evaluated (the evaluation location starting point may be any geographical location, and the evaluation location end point may be any geographical location other than the starting point), and the second route calculation result may be a path planning result of each preset route calculation service for the evaluation location starting point and the evaluation location end point, wherein the evaluation location calculating service to be evaluated may be a route calculation service that has not yet been launched or has not yet been evaluated, and the preset route calculation service may be an existing route calculation service on the market that has been evaluated or not.

[0099] Exemplarily, the first route calculation result and each second route calculation result may include one or more routes. Exemplarily, it is assumed that there are three preset route calculation service providers added for comparison, such as A, B, and C, and there are two route calculation services to be evaluated, N and NT. The route calculation results generated by each route calculation service may include multiple routes (assuming that 3 routes are generated, this is just an example) A1, A2, A3, B1, ..., N2, N3, NT1, NT2, NT3 (a total of 15 routes). After obtaining the route calculation results, these routes can be clustered according to route similarity. By clustering the route calculation results, when the number of planned paths is large, the process of analyzing a single piece of data one by one can be reduced, thereby improving the overall evaluation efficiency.

[0100] In an optional implementation, the clustering accuracy can be improved by dividing the geographic grids and calculating the route similarity by having the shape points of each route fall into the same number of grids. Specifically, the method for calculating the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services in the above step S201 can include the following steps: gridding the map data according to the pre-divided grid size to obtain the geographic grid; for each route in the first route calculation result and the second route calculation result, obtaining the number of grids in which the shape points of each route fall in the same geographic network; and calculating the similarity between each route according to the number of grids to obtain the route similarity.

[0101] Exemplarily, the grid size can be defined according to actual needs or empirical values, for example, set to 50m*50m, which can effectively ensure the accuracy of route result aggregation. Using this grid size, the entire map area is divided into multiple small grids, each of which can be represented by the coordinates of its lower left corner or a unique identifier to distinguish different grids. Then, for each route, its shape points (which can be a GPS coordinate sequence) are obtained and the shape points are assigned to the corresponding grids. Specifically, each shape point is traversed and its grid is determined according to its coordinates. For each route, the number of its shape points falling into each grid is counted, and then the route similarity is calculated for each route and the grid coverage between other routes.

[0102] For example, for route 1 and route 2, the number of grids that the two routes pass through together is calculated. For example, the Jaccard similarity coefficient can be used to calculate the route similarity: J(A,B) = |A∪B| / |A∩B|, where J(A,B) is the route similarity between route 1 and route 2, A and B are the grid sets covered by the shape points of route 1 and route 2, respectively. |A∪B| is the number of grids covered by the shape points of route 1 and route 2, and |A∩B| is the total number of grids covered by the two routes.

[0103] After calculating the similarity of the routes, clustering can be performed using a similarity threshold (for example, clustering is performed as one group when the similarity reaches 0.8). For each route calculation result corresponding to the route, that is, 15 routes (these 15 routes are corresponding planned routes, which are reflected in a larger number of actual routes under different traffic environments, time conditions, etc.) are clustered into the following 5 groups (G1-G5), as shown in Table 1 below:

[0104] Table 1

[0105] Route aggregation results route G1 A1, B1, C1, N1, NT1 G2 A2, B2, C2, N3, NT2 G3 N2, B3, NT3 G4 A3 G5 C3

[0106] By calculating the route similarity by dividing the grids as described above, the complex geographical calculations are simplified, and the calculation accuracy and efficiency of the route similarity are effectively improved by calculating the number of route shape point grids.

[0107] In some embodiments, in addition to the above-mentioned method of calculating route similarity through grid division, route similarity can also be calculated through other methods, such as similarity based on topological structure, by representing the route as a graph structure and comparing the similarity of nodes and edges to evaluate route similarity; or, using a deep learning model to extract route features and calculate route similarity in an embedded space, etc.

[0108] Step S202: Determine a weighted score for each route clustering result according to a preset weight coefficient corresponding to each route calculation service and a priority recommendation value of each route carried in each route calculation result.

[0109] In this embodiment, considering the differences in performance and influence of different route calculation services in actual applications, a corresponding weight coefficient is configured for each route calculation service. The weight coefficient can be calculated by the system or set by the evaluator according to the actual situation. The calculated scores of each group of clustering results can be made more in line with actual needs, thereby effectively improving the accuracy of clustering scores. The priority recommendation value of each route, that is, the route priority weight, is directly reflected in the route that is recommended to the user in the route calculation result. The higher the route score, for example, the shorter the route distance and the shorter the time, the higher the priority recommendation value.

[0110] In an example, the weight coefficient may be determined by determining the weight coefficient of each route calculation service according to at least one of the confidence level, user usage, and user rating of each route calculation service.

[0111] Among them, the confidence of the route calculation service can be determined based on the qualifications of the route calculation service (such as whether it has been launched), industry recognition (such as the equal sharing of route calculation services between route calculation service providers), user usage such as the number of daily active users (Daily Active Users, DAU) and user ratings, i.e., the user's satisfaction evaluation of the route based on their experience. In the early stage of route calculation evaluation, the route to be evaluated is the main object to be evaluated. In order to affect the evaluation accuracy, the weight coefficients of the N and NT routes can be set to 0. In some embodiments, if the user recognition of the route to be evaluated (such as the user's satisfaction score reaches 80%) and the daily usage volume can be close to other preset route calculation services, the weight coefficient of the route to be evaluated can also be increased accordingly, for example, the weight coefficient of the N community route is increased, and only the route calculation quality of the NT route (assuming that the NT route is the baseline route calculation and the N community route is the customized route calculation, and the customized route calculation is of higher quality than the baseline route calculation) is evaluated.

[0112] In another example, the weight coefficients may be determined by obtaining the weight coefficients corresponding to each path calculation service in response to a setting operation in a weight setting area in the evaluation task.

[0113] Specifically, during the evaluation process of the path calculation, an evaluation task may be created in the server, and in the corresponding weight setting area of ​​the evaluation task, a weight coefficient may be selected or input or selected for each path calculation service according to actual experience. For example, the server may receive a setting operation for the weight setting area in the evaluation task through an API or a user interface, or may obtain the weight coefficient corresponding to each path calculation service through an HTTP request or other communication protocols.

[0114] It can be understood that in response to the conditions or states on which the executed operations depend, when the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0115] In some examples, the weight coefficient of each path calculation service may be determined comprehensively by combining the user set value and the system calculated value. This embodiment does not specifically limit the method for determining the weight coefficient corresponding to each path calculation service.

[0116] Next, the process of determining the weighted score of each route clustering result according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result to obtain the weighted score result in step S202 is further introduced. The weighted score calculation process can be adopted as follows: for each route calculation result, weight each route in the route calculation result according to the weight coefficient of the corresponding route calculation service to obtain a weighted value; determine the score of each route according to the product of the weighted value and the priority recommendation value of each route; wherein the priority recommendation value is determined according to the priority recommendation rule of the corresponding route calculation service, and the priority recommendation rule includes at least one of the shortest route distance rule, the minimum route time rule and the user preference rule; for each route clustering result, obtain the weighted score of each route clustering result according to the sum of the scores of all routes in the route clustering result.

[0117] In this embodiment, the priority recommendation value is expressed as the route that is recommended to the user in the route calculation result after the route calculation service performs route calculation, and the higher the route score is. In actual application, the route recommended to the user in priority is determined in combination with the priority recommendation rule, for example, the route with the shortest distance in the route calculation result is recommended (shortest route distance rule), or the route with the shortest time (minimum route time rule), and based on user preference information such as the user likes to go in a straight line and does not like to turn, the route with the least number of turns is recommended (user preference rule), or any combination of the above-mentioned multiple rules is recommended to obtain the optimal route that meets the above-mentioned rules at the same time, and the combination method is not particularly limited in this embodiment.

[0118] In some embodiments, the priority recommendation value of the route can also be determined separately or comprehensively based on factors such as traffic conditions, road type and quality, and safety. That is, the priority recommendation rules can be adaptively determined or adjusted based on actual applications or prior data, and this embodiment does not specifically limit this.

[0119] For example, taking the weight coefficient of A company as 12, the weight coefficient of B company as 11, the weight coefficient of C company as 10, the weight coefficients of N company and NT company as 0, and the priority recommendation values ​​of each route as 6, 2, and 1 respectively, the scores of each route in each route calculation service are shown in Table 2 below:

[0120] Table 2

[0121] Route 1(6) Route 2(2) Route 3(1) Company A(12) 12*6＝72 24 12 Company B (11) 66 22 11 Company C (10) 60 20 10 N Society(0) 0 0 0 NT(0) 0 0 0

[0122] The weighted score is calculated based on the route clustering results, that is, the process of voting on the rationality of the route based on the score. The final rationality score of the route is the sum of the scores of each group of routes. As shown in Table 3 below:

[0123] Table 3

[0124] Route aggregation results route Score G1 A1, B1, C1, N1, NT1 72+66+60+0+0＝198 G2 A2, B2, C2, N3, NT2 24+22+20+0+0＝66 G3 N2, B3, NT3 0+11+0＝11 G4 A3 12 G5 C3 10

[0125] According to the above scoring results, the G1 route can be judged as the most reasonable route set. Through the above cluster scoring calculation process, route aggregation more effectively integrates data from all parties and uses the weighted voting method with multi-dimensional weight coefficients to score the route. The combination of the two can scientifically determine the reasonable route and effectively improve the scoring accuracy.

[0126] Step S203: Obtain a first evaluation result of the path calculation service to be evaluated with respect to the first path calculation result according to the weighted scoring result, wherein the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

[0127] Exemplarily, the weighted scoring result includes the score of each cluster, and the evaluation result of the service to be evaluated can be obtained by comparing which cluster each route in the first route calculation result is located in and which scoring section it is in. For example, if each route in the first route calculation result is located in the cluster with the highest score to the third highest score, it is considered that the route calculation quality meets the route calculation quality condition, otherwise it does not meet the need for route calculation optimization. In some embodiments, in order to further improve the evaluation accuracy of the route calculation service to be evaluated, a comprehensive judgment can also be made by calculating the reasonable rate, optimal rate, and diversity rate of the route calculation results. For details, please refer to the embodiments below, which will not be described in detail here.

[0128] Through the above technical solution, the route similarity of the route calculation results of each party is used for clustering. When the number of planned routes is large, the process of analyzing a single piece of data one by one can be reduced, thereby improving the overall evaluation efficiency. At the same time, the route is scored by using the multi-dimensional weight coefficients of the route calculation service provider and the priority recommendation value of each route, and the route calculation service to be evaluated is evaluated using the scoring result, which effectively integrates the data information of all parties, reduces the excessive dependence on the quality of the reference route, and effectively improves the scoring accuracy of the route calculation to be evaluated.

[0129] Figure 3 This is a flow chart of another method for evaluating route calculation provided by an embodiment of the present application. Based on the above embodiment, this embodiment uses weighted scoring results to perform grouping and sorting, and uses the sorting results to perform route evaluation to further improve the evaluation accuracy. Specifically, in addition to the above steps S201-S203, this embodiment further divides the above step S203 into the following steps S2031-S2032 for obtaining the first evaluation result of the route calculation service to be evaluated on the first route calculation result according to the weighted scoring result:

[0130] Step S2031: According to the weighted scoring result, the plurality of route clustering results are sorted according to the score to obtain a reasonable route sorting result.

[0131] As shown in Table 3 above, the sorting results are G1>G2>G3>G4>G5. Using the preset number of reasonable route sortings, such as the top three sortings of the weighted scoring results, the preset sorting of the reasonable route sorting results is G1, G2, and G3. Among them, the G1 route can be judged as the most reasonable route, that is, the true value of the route of the current sample. The closer the first route calculation result is to the true value of the route, the higher its rationality is, and it better reflects the route plan selected and recognized by a large number of users.

[0132] Step S2032: Determine a first evaluation result of the first route calculation result on the route according to the data information of each route in the first route calculation result falling into the reasonable route sorting result.

[0133] Since there are different routes in the route calculation results, they may fall into different route clustering results respectively. In order to further improve the accuracy of route calculation evaluation, this embodiment calculates the data information of each route in the route calculation result falling into the reasonable route sorting result to determine the final route calculation evaluation result. The data information may be the number of routes or other data information.

[0134] In an optional implementation, the route calculation evaluation can be performed by calculating multiple ratio data such as the reasonable rate, optimal rate, diversity rate and abnormal rate of the route to further improve the accuracy of the route calculation evaluation. Specifically, the above step S2032 determines the first evaluation result of the first route calculation result on the route according to the data information of each route in the first route calculation result falling into the reasonable route sorting result, and can be performed in the following manner:

[0135] In one example, the rationality rate of the first calculation result can be determined according to the number of routes in the first calculation result that fall into the preset ranking of the reasonable route ranking result and the total number of routes in the first calculation result, and the first evaluation result can be determined according to the rationality rate. As mentioned above, those skilled in the art can determine the preset ranking in combination with the empirical value, for example, select the first three groups in the five groups as the reasonable route ranking, and calculate the rationality rate of the first calculation result by calculating the number of routes in G1, G2, and G3 in the first calculation result, and the total number of routes. Specifically, taking the first calculation result NT as an example, the routes NT falls into G1, G2, and G3 are NT1, NT2, and NT3. The routes falling into the first three groups are considered reasonable routes, and the rest are unreasonable routes. The examples in Table 3 are all reasonable routes. This is only an example. In the actual evaluation process, the generated calculation routes can reach thousands or even tens of thousands. By calculating the rationality rate, the evaluation accuracy can be effectively improved. Among them, the higher the rationality rate, the higher the calculation quality represented by the evaluation result.

[0136] In another example, the optimal rate of the first route calculation result can be determined based on the number of routes with the highest priority recommendation value in the first route calculation result that fall into the top-ranked route clustering result in the reasonable route sorting result, and the total number of routes with the highest priority recommendation value in the first route calculation result.

[0137] In this example, after the reasonable routes are sorted based on weighted voting, the first group of reasonable routes, i.e., the routes of G1, are regarded as the optimal route combination, and the one with the highest priority recommendation value, i.e., the first solution introduced by the route calculation result, is regarded as the route with the highest proportion of user-initiated navigation. This embodiment evaluates the route calculation results by paying attention to whether the first solution is the optimal route, so that the evaluation results can meet the actual user needs, wherein the higher the optimal rate, the higher the route calculation quality represented by the evaluation results. Specifically, by calculating the number of the first route in the first group / the total number of the first routes in the route calculation results, for example, the route calculation service calculates different OD (Origin-Destination) paths, and under each OD, the preferred solution in the route calculation result falls into the number of weighted scores in the first group in the corresponding OD route clustering result, i.e., the number of second routes.

[0138] In another example, the diversity of the first route calculation result may be determined according to the number of routes in the first route calculation result that fall into different route clustering results in the reasonable route sorting result and the total number of routes in the first route calculation result.

[0139] It can be understood that the present embodiment falls into the different route aggregation results in the reasonable route sorting results, that is, the route combinations that fall into different route clustering results in the first route calculation result, rather than the route combinations in the same clustering result. In Table 3 above, NT1, NT2, and NT3 are respectively located in G1, G2, and G3, and are all grouped together with the routes in other route calculation services, and they all fall into different route clustering results. In other words, after voting group sorting, the routes assigned to the same group represent a high degree of repetition, so the number of groups = the diversity number, the higher the diversity rate, the higher the route calculation quality represented by the evaluation result, and the credibility of the route calculation result is also higher when evaluating the route calculation result based on the diversity rate.

[0140] In another example, the abnormality rate of the first route calculation result is determined based on at least one of the number of routes in the first route calculation result whose route distance is greater than the distance threshold of the shortest route in the reasonable route sorting result, the number of routes in the first route calculation result whose route congestion distance reaches a preset proportion of the total route length, and the number of routes in the first route calculation result whose number of traffic lights reaches a preset number, and the total number of routes in the first route calculation result.

[0141] In an optional method of this example, the abnormality rate of the first route calculation result can be determined based on the number of routes in the first route calculation result whose route distance is greater than the distance threshold of the shortest route in the reasonable route sorting result and the total number of routes in the first route calculation result. Among them, those skilled in the art can adaptively set the distance threshold in combination with the empirical value, and this embodiment does not specifically limit this. For example, a route that is 1.5 times longer than the shortest route is considered to be an abnormal route. The higher the abnormality rate, the lower the route calculation quality represented by the evaluation result. It should be noted that in addition to using the above-mentioned number of routes (i.e., based on the fact that the route is 1.5 times longer than the shortest route, it is considered to be an abnormal route that is detoured overall), the abnormality rate can also be calculated using the route congestion distance and the number of route traffic lights separately. Among them, the calculation method of using the route congestion distance and the number of route traffic lights to calculate the abnormality rate is similar to the calculation method of using the distance threshold of the shortest route mentioned above, and the relevant description will not be repeated. Alternatively, the abnormality rate can also be calculated by combining any combination of the distance threshold of the shortest route, the route congestion distance, and the number of traffic lights on the route. For example, a route with a congestion distance exceeding 25% of the overall route length or greater than 10 km is considered to be non-congested, a congested route is considered to be an abnormal route, and a route with the number of traffic lights reaching 2 lights / km is considered to have many traffic lights and an abnormal route. The calculation accuracy of the abnormality rate combined with other dimensions is higher. For example, the calculation method of the abnormality rate is: y=ax1+bx2+cx3, y represents the abnormality rate of the first route calculation result, a, b, c represent the weights corresponding to the distance threshold of the shortest route, the route congestion distance, and the number of route signal lights (which can be adjusted by the user according to the actual application adaptability, or determined according to the empirical value), x1, x2, and x3 are respectively the number of routes that exceed the distance threshold of the shortest route distance, the number of routes that reach a preset proportion of the total route length, and the number of routes where the number of signal lights on the route reaches a preset number. The comparison of the above shortest routes can be statistically analyzed by speed range, distance mileage, time consumption, departure time, etc.

[0142] In some examples, the rationality rate, optimality rate, diversity rate and abnormality rate mentioned above can also be used as multi-dimensional indicators, and any combination of the multi-dimensional indicators can be used to jointly calculate the evaluation results of the path calculation, and corresponding weights can be assigned to each indicator to obtain a comprehensive evaluation result, thereby further improving the evaluation accuracy.

[0143] Figure 4 A flow chart of another path calculation evaluation method provided in an embodiment of the present application. On the basis of the above embodiment, this embodiment can also realize real-time and personalized path calculation evaluation through evaluation interaction. Specifically, in addition to the above steps S201-S203, this embodiment also includes the steps of obtaining the first path calculation result and the second path calculation result, including step S401 and step S402.

[0144] Step S401: in response to a trigger start for an evaluation task, a path calculation sample set from a pre-selected evaluation location start point to an evaluation location end point is obtained; wherein the path calculation sample set is obtained by extracting multi-source navigation data.

[0145] Exemplarily, in order to effectively improve the accuracy of the route calculation evaluation, a large number of route calculation sample sets can be collected and processed in the early stage. Specifically, real multi-source logs will be automatically recovered during the operation of the navigation service. These logs mainly include popular POIs, cooperative source logs, user feedback, daily route calculation logs, car factory user logs, test cases, etc., that is, multi-source navigation data. In each evaluation, the corresponding route calculation set can be extracted according to the corresponding evaluation task (such as the starting and ending OD of different evaluation locations) (such as random extraction by the system, or extraction by the user). The main process of sample processing includes stratified sampling and extracting OD according to city, mileage, peak and off-peak time periods as the input of the evaluation task. Among them, the route calculation sample set is as follows: Figure 5a In some scenarios, users can also manually filter out the path calculation sample set and upload it to a server (such as a server corresponding to a path calculation evaluation platform, which can be an evaluation website, software program, etc.). The upload page is as follows: Figure 5b and Figure 5c shown.

[0146] For further example, after the path calculation evaluation platform obtains the original sample of OD data, the user can create an evaluation task. When creating an automatic evaluation task, the user can set the sorting weight (i.e., weight coefficient) and grid size of each path calculation service, and select the corresponding path calculation sample set (such as the path calculation sample set of one or more ODs) in the evaluation task. The server obtains the path calculation sample set of one or more ODs and enters the subsequent evaluation process, wherein the evaluation task is such as Figure 6 As shown, it can be understood that the evaluation task can be to perform path calculation evaluation on one target OD sample set, or to perform path calculation evaluation on multiple target ODs, which is not particularly limited in this embodiment.

[0147] Step S402, calling the to-be-evaluated path calculation service to perform path planning for the path calculation sample set to obtain a first path calculation result; and step S403, calling multiple preset path calculation services to perform path planning for the path calculation sample set respectively to obtain a second path calculation result.

[0148] Next, the evaluation task can obtain the corresponding route calculation results by calling the route calculation service API to be evaluated and other preset route calculation service APIs to obtain each sample in the route calculation sample set for route calculation (for example, a route calculation request can be initiated to the route calculation services of each party to obtain the route calculation results corresponding to each party), and the route calculation results include multiple routes (samples) that have been calculated. During the evaluation process, the route calculation services of each party call each route calculation service API (Application Programming Interface) for the same OD sample set (same time point and same route preference settings) to ensure that the routes calculated by the route calculation services of each party are obtained at the same time when the road traffic conditions are consistent, so that the route samples are horizontally comparable.

[0149] Furthermore, in the process of clustering routes by the route calculation services of each party, clustering display can be performed. Taking G1, G2, G3, and G4 as an example, G1 (A1, B1, C1, N1, NT1) and G2 (A2, B2, C2, N3, NT3) are clustered as shown below: Figure 7a As shown, G3 (N2, B3, NT3), G4 (A3) Figure 7b shown.

[0150] Through the above technical solution, user feedback and system response can be obtained in real time during the path calculation evaluation process, and parameters and strategies can be dynamically adjusted during the evaluation process to realize personalized evaluation functions, improve the interactivity and adaptability of the evaluation process, and effectively meet the actual path calculation evaluation needs of actual users.

[0151] Figure 8 This is a flowchart of another method for evaluating the path calculation provided by an embodiment of the present application. Based on the above embodiment, in addition to evaluating the path calculation quality of the path calculation service to be evaluated, this embodiment can also evaluate other path calculation services at the same time and compare multiple path calculation services, so that users can perform comparative analysis to efficiently locate the shortcomings of subsequent path calculations that need to be improved and optimized. Specifically, in addition to the above steps S201-S203, the method provided by this embodiment can also include the following steps S801 and S802:

[0152] Step S801: Obtain second evaluation results of respective second path calculation results of multiple preset path calculation services according to the weighted scoring result, wherein the second evaluation results are used to indicate the path calculation quality of each preset path calculation service.

[0153] It should be noted that the evaluation process for the second evaluation result of each preset path calculation service is similar to the evaluation process for the algorithm service to be evaluated, and will not be described in detail here.

[0154] Step S802: In response to a triggering operation on a comparison result between path calculation services, display comparison information between the first evaluation result and the second evaluation result.

[0155] Exemplarily, the server may receive a trigger operation through an API or a user interface, and display the comparison information through an evaluation page of the client. For example, a control (such as a button) may be included on the display page for comparing the results of the route calculation services. After the evaluation of the multi-party route calculation service, the user clicks on the control to display the comparison information between the evaluation results, such as the reasonable rate, the optimal rate, the diversity rate, and the abnormal rate. The display page is as follows: Figure 9a to Figure 9d In this way, users can more intuitively analyze the advantages and improvements of the path calculation service to be evaluated, providing strong data support for the subsequent optimization of the path calculation service.

[0156] In summary, the technical solution provided by this embodiment can effectively solve the problem of obtaining relatively objective and reasonable optimal route results to support the subsequent route quality evaluation process in the absence of an objective standard optimal route through weighted voting for each route calculation service, and the automated calculation and statistical method of navigation route satisfaction indicators based on route aggregation, weighted voting, and a large number of real data samples can improve the efficiency of the route evaluation process, reduce costs, and make the evaluation results more objective and reliable. Specifically, since the rationality of the route calculation results is difficult to objectively define, this embodiment determines a reasonable route set by weighted voting for the route calculation services of all parties, which can reflect the solutions selected and recognized by a large number of users, and combines multi-dimensional evaluation to make the evaluation more accurate. In addition, route aggregation can effectively integrate information, and the weighted voting method scores routes based on multiple factors. The combination of the two can scientifically determine reasonable routes, avoiding the subjectivity of traditional manual comparisons. A large number of real data samples reflect the real choices and behaviors of users, making calculations and statistics more in line with actual conditions. Based on this, the automated path calculation evaluation process provided in this embodiment does not require manual testing and comparison one by one, which greatly improves efficiency and reduces costs. In addition, the evaluation process based on a large number of real data samples makes the evaluation results more objective and reliable.

[0157] Fig.10 A schematic diagram of the structure of a path calculation evaluation device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the device 1000 includes a clustering module 1001, a weighted scoring module 1002, and an evaluation result acquisition module 1003, wherein:

[0158] A clustering module 1001 is configured to cluster the routes in the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services according to the route similarities of the routes in the first route calculation result and the second route calculation results to obtain multiple route clustering results;

[0159] A weighted scoring module 1002 is configured to determine a weighted score for each route clustering result based on a preset weight coefficient corresponding to each route calculation service and a priority recommendation value of each route carried in each route calculation result;

[0160] The evaluation result acquisition module 1003 is configured to acquire a first evaluation result of the path calculation service to be evaluated with respect to the first path calculation result according to the weighted scoring result, wherein the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

[0161] In an optional implementation manner, a path calculation acquisition module for acquiring the first path calculation result and the second path calculation result is further included, which includes:

[0162] A sample set acquisition unit, which is configured to acquire a path calculation sample set from a pre-selected evaluation location starting point to an evaluation location end point in response to a trigger start for the evaluation task; wherein the path calculation sample set is obtained by extracting multi-source navigation data;

[0163] A first path calculation unit is configured to call the path calculation service to be evaluated to perform path planning for the path calculation sample set to obtain the first path calculation result; and

[0164] The second path calculation unit is configured to call a plurality of preset path calculation services to perform path planning for the path calculation sample set respectively to obtain the second path calculation result.

[0165] In an optional implementation manner, the evaluation result acquisition module 1003 includes:

[0166] A reasonable route sorting unit, which is configured to sort the plurality of route clustering results according to the weighted scoring result and the size of the scores to obtain a reasonable route sorting result;

[0167] An evaluation unit is configured to determine a first evaluation result of the first route calculation result on a route according to data information of each route in the first route calculation result falling into the reasonable route sorting result.

[0168] In an optional implementation, the evaluation unit includes at least one of the following:

[0169] a rationality rate acquisition unit, configured to determine the rationality rate of the first route calculation result according to the number of routes in the first route calculation result that fall into a preset ranking in the rational route ranking result and the total number of routes in the first route calculation result;

[0170] an optimal rate acquisition unit, configured to determine the optimal rate of the first route calculation result according to the number of routes with the highest priority recommendation value in the first route calculation result that fall into the most forward-ranked route cluster result in the reasonable route ranking result and the total number of routes with the highest priority recommendation value in the first route calculation result;

[0171] a diversity rate acquisition unit, configured to determine the diversity rate of the first route calculation result according to the number of routes in the first route calculation result that fall into different route clustering results in the reasonable route sorting result and the total number of routes in the first route calculation result;

[0172] An abnormality rate acquisition unit is configured to determine the abnormality rate of the first route calculation result based on at least one of the number of routes in the first route calculation result whose route distance is greater than a distance threshold of the shortest route in the reasonable route sorting result, the number of routes in the first route calculation result whose route congestion distance reaches a preset proportion of the total route length, and the number of routes in the first route calculation result whose number of traffic lights reaches a preset number, and the total number of routes in the first route calculation result.

[0173] In an optional implementation, a weight determination module for determining a weight coefficient is further included, which includes at least one of the following:

[0174] A first weight acquisition unit, configured to acquire a weight coefficient corresponding to each path calculation service in response to a setting operation on a weight setting area in the evaluation task;

[0175] The second weight acquisition unit is configured to determine a weight coefficient of each path calculation service according to at least one of the confidence level, user usage and user score of each path calculation service.

[0176] In an optional implementation, the weighted scoring module 1002 includes:

[0177] A weighting unit, which is configured to weight each line in each path calculation result according to a weight coefficient of a corresponding path calculation service to obtain a weighted value;

[0178] a score determination unit, configured to determine the score of each route according to the product of the weighted value and the priority recommendation value of each route; wherein the priority recommendation value is determined according to a priority recommendation rule of the corresponding route calculation service, and the priority recommendation rule includes at least one of a shortest route distance rule, a minimum route time rule, and a user preference rule;

[0179] The scoring unit is configured to obtain a weighted score for each route clustering result according to the sum of scores of all routes in the route clustering result.

[0180] In an optional implementation, the device further includes a similarity calculation module for calculating the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services, which includes:

[0181] A gridding unit, which is configured to grid the map data according to a pre-divided grid size to obtain a geographic grid;

[0182] A grid number acquisition unit, configured to acquire, for each route in the first route calculation result and the second route calculation result, the number of grids in which the shape points of each route fall in the same geographical network;

[0183] The similarity calculation unit is configured to calculate the similarity between each route according to the number of grids to obtain the route similarity.

[0184] In an optional embodiment, the device further includes a display module;

[0185] The evaluation result acquisition unit is further configured to acquire second evaluation results of the plurality of preset path calculation services on respective second path calculation results according to the weighted scoring result, wherein the second evaluation results are used to indicate the path calculation quality of each preset path calculation service;

[0186] The display module is configured to display the comparison information between the first evaluation result and the second evaluation result in response to a trigger operation on the comparison result between the path calculation services.

[0187] Fig.11 is a schematic diagram of the structure of an electronic device provided in accordance with an embodiment of the present application, such as Fig.11 As shown, the device includes: a processor 1101, and a memory 1102 communicatively connected to the processor 1101;

[0188] The memory 1102 stores computer-executable instructions;

[0189] The processor 1101 executes the computer-executable instructions stored in the memory 1102 to implement the path calculation evaluation method. The memory 1102 and the processor 1101 are connected via a bus 1103 .

[0190] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0191] The embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the path calculation evaluation method provided in the above method embodiment.

[0192] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0193] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0194] The embodiment of the present application also provides a computer program product including a computer program, which, when executed by a processor, implements the path calculation and evaluation method provided in the above method embodiment.

[0195] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0196] The embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory to perform the path calculation evaluation method provided in the above method embodiment.

[0197] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the method embodiments of the present application, and no further elaboration is made here.

[0198] Fig.12 FIG. 1 is a schematic block diagram of an electronic device 1200 shown in an exemplary embodiment of the present application, and the electronic device may be a server. Fig.12 As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0199] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206 for connecting to external devices or performing network communications; an output unit 1207, such as a network interface or a USB interface, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0200] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the calculation evaluation method and processing described above. For example, in some embodiments, the calculation evaluation method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the calculation evaluation method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to execute the path calculation evaluation method in any other appropriate manner (for example, by means of firmware).

[0201] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0202] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0203] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0204] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0205] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A path calculation evaluation method, characterized in that: include: Clustering the routes in the first route calculation result and the second route calculation result according to the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of the plurality of preset route calculation services to obtain a plurality of route clustering results; Determine the weighted score of each route clustering result according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result; According to the weighted scoring result, a first evaluation result of the path calculation service to be evaluated with respect to the first path calculation result is obtained, where the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

2. The method according to claim 1, characterized in that The method for obtaining the first path calculation result and the second path calculation result includes: In response to a trigger start for the evaluation task, a pre-selected set of route calculation samples from a starting point of the evaluation location to an end point of the evaluation location is obtained; wherein the set of route calculation samples is obtained by extracting multi-source navigation data; Calling the to-be-evaluated path calculation service to perform path planning for the path calculation sample set to obtain the first path calculation result; and Multiple preset path calculation services are called to perform path planning for the path calculation sample set respectively to obtain the second path calculation result.

3. The method according to claim 1 or 2, characterized in that: The obtaining, according to the weighted scoring result, a first evaluation result of the path calculation service to be evaluated regarding the first path calculation result includes: According to the weighted scoring result, the plurality of route clustering results are sorted according to the score to obtain a reasonable route sorting result; A first evaluation result of the first route calculation result on the route is determined according to data information of each route in the first route calculation result falling into the reasonable route ranking result.

4. The method according to claim 3, characterized in that Determining a first evaluation result of the first route calculation result on the route according to data information of each route in the first route calculation result falling into the reasonable route sorting result includes at least one of the following: Determining a rationality rate of the first route calculation result according to the number of routes in the first route calculation result that fall into the preset ranking of the reasonable route ranking result and the total number of routes in the first route calculation result; Determine the optimal rate of the first route calculation result according to the number of routes with the highest priority recommendation value in the first route calculation result that fall into the most forward-ranked route cluster result in the reasonable route ranking result and the total number of routes with the highest priority recommendation value in the first route calculation result; Determining the diversity rate of the first route calculation result according to the number of routes in the first route calculation result that fall into different route clustering results in the reasonable route sorting result and the total number of routes in the first route calculation result; The abnormality rate of the first route calculation result is determined based on at least one of the number of routes in the first route calculation result whose route distance is greater than the distance threshold of the shortest route in the reasonable route sorting result, the number of routes in the first route calculation result whose route congestion distance reaches a preset proportion of the total route length, and the number of routes in the first route calculation result whose number of traffic lights reaches a preset number, and the total number of routes in the first route calculation result.

5. The method according to claim 1 or 2, characterized in that: The weight coefficient is determined in at least one of the following ways: In response to a setting operation on a weight setting area in the evaluation task, obtaining a weight coefficient corresponding to each path calculation service; A weight coefficient of each path calculation service is determined according to at least one of the confidence level, user usage and user score of each path calculation service.

6. The method according to any one of claims 1 to 5, characterized in that: The weighted score of each route clustering result is determined according to the preset weight coefficient corresponding to each route calculation service and the priority recommendation value of each route carried in each route calculation result to obtain the weighted score result, including: For each path calculation result, weight each line in the path calculation result according to the weight coefficient of the corresponding path calculation service to obtain a weighted value; Determine the score of each route according to the product of the weighted value and the priority recommendation value of each route; wherein the priority recommendation value is determined according to the priority recommendation rule of the corresponding route calculation service, and the priority recommendation rule includes at least one of the shortest route distance rule, the minimum route time rule and the user preference rule; For each route clustering result, a weighted score of each route clustering result is obtained according to the sum of the scores of all routes in the route clustering result.

7. The method according to any one of claims 1 to 5, characterized in that: The method for calculating the route similarity between the first route calculation result of the route calculation service to be evaluated and the second route calculation results of multiple preset route calculation services includes: According to the pre-divided grid size, the map data including the starting point and the end point of the evaluation location are gridded to obtain a geographic grid; For each route in the first route calculation result and the second route calculation result, obtaining the number of grids whose shape points of each route fall in the same geographical network; The similarity between each route is calculated according to the number of grids to obtain the route similarity.

8. The method according to any one of claims 1 to 5, characterized in that: Also includes: According to the weighted scoring result, obtaining second evaluation results of the plurality of preset path calculation services on respective second path calculation results, wherein the second evaluation results are used to indicate the path calculation quality of each preset path calculation service; In response to a triggering operation on a comparison result between the path calculation services, comparison information between the first evaluation result and the second evaluation result is displayed.

9. A path calculation evaluation device, characterized in that: include: a clustering module configured to cluster the routes in the first route calculation result and the second route calculation result of the route calculation service to be evaluated according to the route similarity of each route in the second route calculation results of the plurality of preset route calculation services, so as to obtain a plurality of route clustering results; A weighted scoring module is configured to determine a weighted score for each route clustering result based on a preset weight coefficient corresponding to each route calculation service and a priority recommendation value of each route carried in each route calculation result; The evaluation result acquisition module is configured to acquire a first evaluation result of the path calculation service to be evaluated on the first path calculation result according to the weighted scoring result, wherein the first evaluation result is used to indicate the path calculation quality of the service to be evaluated.

10. An electronic device, characterized in that: The electronic device comprises: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the path calculation evaluation method according to any one of claims 1 to 8.

11. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by the processor, they are used to implement the path calculation and evaluation method according to any one of claims 1 to 8; and / or, The computer program product comprises a computer program, and when the computer program is executed by a processor, the path calculation and evaluation method according to any one of claims 1 to 8 is implemented.

Images