Evaluation Method, Device and Computer Storage Medium for Obstacle Detection Model

Abstract

The present application provides a method, device, and computer storage medium for evaluating an obstacle detection model. The method includes: obtaining point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data; obtaining a target distance between the vehicle and a first obstacle according to the annotation data; obtaining a target occlusion index of a second obstacle occluding the first obstacle according to the annotation data and the point cloud data, where the second obstacle is located between the vehicle and the first obstacle; obtaining a priority of the first obstacle according to the target distance and the target occlusion index; and evaluating the obstacle detection model according to the priority of the first obstacle. This embodiment makes the evaluation metrics of the model more suitable for the actual application scenario, and at the same time makes the evaluation results of the obstacle detection model more reasonable and accurate.

Description

Technical Field

[0001] This application relates to the field of driverless technology, and in particular, to a method, device, and computer storage medium for evaluating an obstacle detection model. Background Art

[0002] With the development of computer and artificial intelligence technologies, in order to change the way of human travel and logistics, driverless technology has developed rapidly. In driverless technology, the system senses the surrounding environment of the vehicle through sensors and identifies obstacles based on the sensing results to achieve functions such as path planning.

[0003] Common obstacle detection methods mainly use deep learning technology to divide samples into a training set and a test set, train an obstacle detection model based on the training set, and obtain a trained obstacle detection model. In order to evaluate the performance of the trained obstacle detection model, the obstacle detection model is tested through the test set.

[0004] However, currently, when testing the obstacle detection model through the test set, the evaluation indicators cannot fit the actual application scenario, and the model evaluation results are not reasonable and accurate enough. Summary of the Invention

[0005] This application provides a method, device, and computer storage medium for evaluating an obstacle detection model, so that the evaluation indicators of the obstacle detection model are more in line with the actual application scenario, and the model evaluation effect is more reasonable and accurate.

[0006] In a first aspect, this application provides a method for evaluating an obstacle detection model, including:

[0007] Obtain point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data;

[0008] According to the annotation data, obtain the target distance between the vehicle and the first obstacle;

[0009] According to the annotation data and the point cloud data, obtain the target occlusion index of the second obstacle occluding the first obstacle, where the second obstacle is located between the vehicle and the first obstacle;

[0010] According to the target distance and the target occlusion index, obtain the priority of the first obstacle;

[0011] Evaluate the obstacle detection model according to the priority of the first obstacle.

[0012] Optionally, the obtaining the target distance between the vehicle and the first obstacle according to the annotation data includes:

[0013] Determine the coordinates of the first obstacle according to the labeled data;

[0014] According to the coordinates of the first obstacle and the coordinates of the vehicle, obtain the Euclidean distance between the coordinates of the first obstacle and the coordinates of the vehicle, and use the Euclidean distance as the target distance.

[0015] Optionally, the obtaining the target occlusion index of the second obstacle occluding the first obstacle according to the labeled data and the point cloud data includes:

[0016] Obtain the first occlusion index of the second obstacle occluding the first obstacle according to the labeled data;

[0017] Obtain the second occlusion index of the second obstacle occluding the first obstacle according to the point cloud data;

[0018] Obtain the target occlusion index of the second obstacle occluding the first obstacle according to the first occlusion index and the second occlusion index.

[0019] Optionally, the obtaining the first occlusion index of the second obstacle occluding the first obstacle according to the labeled data includes:

[0020] Determine the endpoint coordinates of the first obstacle and the endpoint coordinates of the second obstacle according to the labeled data;

[0021] Obtain the first viewing angle between the first obstacle and the vehicle according to the endpoint coordinates of the first obstacle and the coordinates of the vehicle;

[0022] Obtain the first occlusion angle of the second obstacle occluding the first obstacle according to the endpoint coordinates of the second obstacle and the coordinates of the vehicle;

[0023] Obtain the first occlusion index according to the first viewing angle and the first occlusion angle.

[0024] Optionally, the obtaining the second occlusion index of the second obstacle occluding the first obstacle according to the point cloud data includes:

[0025] Perform rasterization processing on the point cloud data to obtain rasterized point cloud data;

[0026] Traverse each grid and determine whether there is a part of the second obstacle in each grid. If so, obtain the second occlusion angle of the grid occluding the first obstacle;

[0027] Determine the third occlusion angle according to the second occlusion angles respectively corresponding to at least one grid;

[0028] Obtain the second occlusion index according to the first observation perspective and the third occlusion perspective.

[0029] Optionally, the obtaining the priority of the first obstacle according to the target distance and the target occlusion index includes:

[0030] Obtain a priority index according to the target distance and the target occlusion index;

[0031] Determine the priority of the first obstacle according to the priority index, where the priority index is inversely proportional to the priority of the first obstacle.

[0032] Optionally, the evaluating the obstacle detection model according to the priority of the first obstacle includes:

[0033] Input the point cloud data into the obstacle detection model to obtain the detection result output by the obstacle detection model;

[0034] Evaluate the obstacle detection model according to the detection result and the priority of the first obstacle.

[0035] Optionally, after obtaining the priority of the first obstacle according to the target distance and the target occlusion index, the method further includes:

[0036] Display the three-dimensional graph of the point cloud data and the priority of the first obstacle on the display interface;

[0037] In response to a user's trigger operation, adjust the priority of the first obstacle.

[0038] In a second aspect, the present application provides an evaluation device for an obstacle detection model, including:

[0039] A first acquisition module, configured to acquire point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data;

[0040] A second acquisition module, configured to acquire a target distance between the vehicle and a first obstacle according to the annotation data; and acquire a target occlusion index of a second obstacle occluding the first obstacle according to the annotation data and the point cloud data, where the second obstacle is located between the vehicle and the first obstacle;

[0041] A third acquisition module, configured to acquire the priority of the first obstacle according to the target distance and the target occlusion index;

[0042] An evaluation module, configured to evaluate the obstacle detection model according to the priority of the first obstacle.

[0043] Optionally, the second acquisition module is specifically configured to determine the coordinates of the first obstacle according to the labeled data; obtain the Euclidean distance between the coordinates of the first obstacle and the coordinates of the vehicle according to the coordinates of the first obstacle and the coordinates of the vehicle, and use the Euclidean distance as the target distance.

[0044] Optionally, the second acquisition module is specifically configured to obtain a first occlusion index of the second obstacle occluding the first obstacle according to the labeled data; obtain a second occlusion index of the second obstacle occluding the first obstacle according to the point cloud data; and obtain a target occlusion index of the second obstacle occluding the first obstacle according to the first occlusion index and the second occlusion index.

[0045] Optionally, the second acquisition module is specifically configured to determine the endpoint coordinates of the first obstacle and the endpoint coordinates of the second obstacle according to the labeled data; obtain a first viewing angle between the first obstacle and the vehicle according to the endpoint coordinates of the first obstacle and the coordinates of the vehicle; obtain a first occlusion angle of the second obstacle occluding the first obstacle according to the endpoint coordinates of the second obstacle and the coordinates of the vehicle; and obtain a first occlusion index according to the first viewing angle and the first occlusion angle.

[0046] Optionally, the second acquisition module is specifically configured to perform rasterization processing on the point cloud data to obtain rasterized point cloud data; traverse each grid to determine whether there is a part of the second obstacle in each grid, and if so, obtain a second occlusion angle of the grid occluding the first obstacle; determine a third occlusion angle according to the second occlusion angles corresponding to at least one grid; and obtain a second occlusion index according to the first viewing angle and the third occlusion angle.

[0047] Optionally, the third acquisition module is specifically configured to obtain a priority index according to the target distance and the target occlusion index; and determine the priority of the first obstacle according to the priority index, where the priority index is inversely proportional to the priority of the first obstacle.

[0048] Optionally, the evaluation module is specifically configured to input the point cloud data into an obstacle detection model to obtain a detection result output by the obstacle detection model; and evaluate the obstacle detection model according to the detection result and the priority of the first obstacle.

[0049] The evaluation device of the obstacle detection model further includes:

[0050] A display module, configured to display a three-dimensional graph of the point cloud data and the priority of the first obstacle on a display interface.

[0051] An optimization module, configured to adjust the priority of the first obstacle in response to a trigger operation by a user.

[0052] In a third aspect, the present application provides an evaluation device for an obstacle detection model, including:

[0053] A memory;

[0054] A processor;

[0055] Wherein, the memory stores computer-executable instructions;

[0056] The processor executes the computer-executable instructions stored in the memory to implement the evaluation method of the obstacle detection model as described in the first aspect and various possible implementation manners of the first aspect above.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the evaluation method of the obstacle detection model as described in the first aspect and various possible implementation manners of the first aspect above.

[0058] The evaluation method of the obstacle detection model provided in this embodiment obtains point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data; according to the annotation data, obtains the target distance between the vehicle and the first obstacle; according to the annotation data and the point cloud data, obtains the target occlusion index of the second obstacle occluding the first obstacle, where the second obstacle is located between the vehicle and the first obstacle; according to the target distance and the target occlusion index, obtains the priority of the first obstacle; and evaluates the obstacle detection model according to the priority of the first obstacle. The evaluation method of the obstacle detection model provided in this embodiment adds a new evaluation index of obstacle priority on the basis of the prior art, making the evaluation index of the model more in line with the actual application scenario and at the same time making the evaluation result of the obstacle detection model more reasonable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0060] Figure 1 It is a schematic diagram of the scenario of the evaluation method of the obstacle detection model provided by the present application;

[0061] Figure 2 It is the flow of the evaluation method of the obstacle detection model provided by the present application Figure 1 ;

[0062] Figure 3 It is the flow of the evaluation method of the obstacle detection model provided by the present application Figure 2 ;

[0063] Figure 4 Scenario schematic diagram for obtaining the first occlusion index based on the labeled data provided by this application;

[0064] Figure 5 Scenario schematic diagram for obtaining the second occlusion index based on the point cloud data provided by this application;

[0065] Figure 6 Structural schematic diagram of the evaluation device for the obstacle detection model provided by this application;

[0066] Figure 7 Structural schematic diagram of the evaluation device for the obstacle detection model provided by this application.

[0067] Through the above-mentioned drawings, the specific embodiments of this application have been shown, and there will be more detailed descriptions in the following text. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the protection scope of this application.

[0069] Driverless refers to the technology that enables a vehicle to drive normally without human control. This technology is becoming increasingly mature, and since driverless vehicles can save more human resources, this technology is being taken more and more seriously.

[0070] In driverless technology, the detection of obstacles by driverless vehicles is particularly important. Currently, an obstacle detection model is mainly used to determine obstacles. During the training process of the obstacle detection model, sample data needs to be obtained first. When obtaining the sample data, relevant data can be obtained in an obstacle scenario. Figure 1 Scenario schematic diagram of the evaluation method for the obstacle detection model provided by the embodiments of this application. As Figure 1 shown, there are obstacles such as pedestrians and vehicles on the road. During the driving process of the driverless vehicle, the point cloud data in front of the vehicle during driving is obtained and stored. Then, the obstacles in the point cloud data are labeled through manual annotation to obtain sample data.

[0071] However, when evaluating the obstacle detection model using the test set in the sample data, the evaluation metrics do not conform to the actual application scenario, and the model evaluation effect is not reasonable and accurate enough. In the actual scenario, different obstacles have priorities. For example, obstacles that are closer to the vehicle and have less occlusion are more important to the vehicle; in contrast, obstacles that are farther from the vehicle and have more occlusion should be given lower priorities. Therefore, a model that can more accurately detect obstacles with higher priorities should be given higher evaluation metrics by the evaluation system.

[0072] The method provided in this application evaluates the priority of obstacles based on two metrics: the distance between the obstacle and the vehicle and the occlusion condition of the obstacle. The smaller the distance between the obstacle and the vehicle and the less the occlusion, the higher the priority of the obstacle. When evaluating the obstacle detection model, evaluating in combination with the priority of the obstacle can make the evaluation metrics of the model more conform to the actual application scenario and at the same time make the evaluation results of the obstacle detection model more reasonable and accurate.

[0073] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0074] Figure 2 is the flow of the evaluation method for the obstacle detection model shown in the embodiments of this application Figure 1 As Figure 2 shown, the evaluation method for the obstacle detection model provided in this embodiment includes:

[0075] S101: Obtain point cloud data and the annotation data of the point cloud data, where the annotation data is used to indicate the obstacles in the point cloud data.

[0076] Among them, point cloud data refers to a set of vectors in a three-dimensional coordinate system. The annotation data of the point cloud data refers to the data annotated by the user for the point cloud data, which can be used to indicate the obstacles in the point cloud data.

[0077] In this step, obtaining the point cloud data and the annotation data of the point cloud data can be to obtain the pre-stored point cloud data and the annotation data of the point cloud data. The pre-stored point cloud data and the annotation data in the point cloud data can be obtained through Figure 1 the scenario of the embodiment and then stored in the database.

[0078] S102: According to the annotation data, obtain the target distance between the vehicle and the first obstacle.

[0079] In this step, the position coordinates of the first obstacle can be determined based on the labeled data, and then the target distance between the vehicle and the first obstacle can be obtained according to the position coordinates of the first obstacle and the position coordinates of the vehicle. The embodiments of the present application do not particularly limit the implementation manner of obtaining the target distance.

[0080] Exemplarily, a possible implementation manner is given here. According to the labeled data, the centroid coordinates of the first obstacle are determined, and according to the centroid coordinates and the coordinates of the vehicle, the Euclidean distance between the centroid coordinates and the coordinates of the vehicle is obtained, and the Euclidean distance is used as the target distance.

[0081] Specifically, the labeled data has point cloud data constituting the first obstacle. Based on the point cloud data of the first obstacle, the centroid coordinates of the first obstacle can be determined. The coordinates of the vehicle can be the coordinates of a preset position in the vehicle. Those skilled in the art can understand that the centroid coordinates and the coordinates of the vehicle are coordinates in the same coordinate system.

[0082] S103: According to the labeled data and the point cloud data, obtain a target occlusion index of the second obstacle occluding the first obstacle, where the second obstacle is located between the vehicle and the first obstacle.

[0083] Among them, the first obstacle can be a target object, and the second obstacle is other obstacles located between the first obstacle and the vehicle. From the perspective of the vehicle, the second obstacle occludes part or all of the first obstacle. According to the labeled data and the point cloud data, it can be known how much the second obstacle occludes the first obstacle, so as to obtain the target occlusion index of the second obstacle occluding the first obstacle.

[0084] Combined with Figure 1 the scenario in, pedestrian 2 is located between vehicle 1 and vehicle 3. Vehicle 3 can be used as the first obstacle, and pedestrian 2 can be used as the second obstacle. From the perspective of vehicle 1, pedestrian 2 occludes part of vehicle 3. Therefore, according to the labeled data and the point cloud data, the target occlusion index of pedestrian 2 occluding vehicle 3 can be obtained.

[0085] Exemplarily, a possible implementation manner is given here. In the above implementation manner, the first occlusion index of the second obstacle occluding the first obstacle can be obtained according to the labeled data first, and then the second occlusion index of the second obstacle occluding the first obstacle can be obtained according to the point cloud data, and the target occlusion index of the first obstacle is obtained according to the above first occlusion index and the second occlusion index. The present application does not particularly limit the order of obtaining the first occlusion index and the second occlusion index.

[0086] Those skilled in the art can understand that the first occlusion index is obtained through the labeled data. However, in order to more accurately obtain the occlusion situation of the second obstacle to the first obstacle, the occlusion situation can also be obtained through the point cloud data, and the target occlusion index is jointly determined based on the two, making the target occlusion index more accurate.

[0087] S104: Obtain the priority of the first obstacle according to the target distance and the target occlusion index.

[0088] Among them, the priority of the obstacle can reflect the degree of influence of the obstacle on the vehicle. For example, an obstacle that is closer to the vehicle and has less occlusion is more important to the vehicle and should be given a higher priority; in contrast, an obstacle that is farther from the vehicle and has more occlusion has little influence on the vehicle and should be given a lower priority. Therefore, after obtaining the target distance and the target occlusion index, the priority of the first obstacle can be obtained according to the above target distance and target occlusion index. The embodiments of the present application do not make special limitations on the implementation manner of obtaining the priority of the first obstacle.

[0089] Exemplarily, a possible implementation manner is given here. According to the target distance and the target occlusion index, obtain a priority index; according to the priority index, determine the priority of the first obstacle, where the priority index is inversely proportional to the priority of the first obstacle.

[0090] Exemplarily, a possible implementation manner of obtaining the priority index is given here. After obtaining the target distance and the target occlusion index, the priority index can be calculated through Formula 1. Formula (1) is as follows:

[0091]

[0092] Among them, P i is the priority index, D i is the target distance, B i is the target occlusion index, and α and β are given coefficients.

[0093] S105: Evaluate the obstacle detection model according to the priority of the first obstacle.

[0094] Among them, when evaluating the obstacle detection model, the priority of the obstacle can be introduced. The embodiments of the present application do not make special limitations on the implementation manner of evaluating the model according to the priority.

[0095] Exemplarily, a possible implementation manner is given here. Input the point cloud data into the obstacle detection model to obtain the detection result output by the obstacle detection model; evaluate the obstacle detection model according to the detection result and the priority of the first obstacle.

[0096] The existing process for evaluating an obstacle detection model is as follows: Input the point cloud data in the test set into the obstacle detection model, and the obstacle detection model outputs the detection results. The detection results include the positions and sizes of each detected obstacle. Based on the positions and sizes of all the obstacles detected in the detection results and the positions and sizes of the obstacles corresponding to the annotation data in the test set, determine the evaluation metrics for each obstacle in the test set. For example, the evaluation metric can be the overlap degree between the detected obstacle and the obstacle in the annotation data, and evaluate the obstacle detection model according to the above evaluation metric.

[0097] This application takes the priority of the obstacle as an evaluation metric, and evaluates the obstacle detection model based on the priority of the obstacle and the detection results output by the obstacle detection model.

[0098] For example: Input the point cloud data in the test set into the obstacle detection model, the obstacle detection model outputs the detection results, and obtain the evaluation metrics for each obstacle according to the detection results and the annotation data. Then jointly evaluate the obstacle detection model according to the evaluation metrics of each obstacle and the priority of each obstacle.

[0099] Those skilled in the art can understand that this application combines the priority of the obstacle with the evaluation metrics of the traditional obstacle detection model to jointly evaluate the obstacle detection model.

[0100] Since the priority of the obstacle can reflect the degree of influence of the obstacle on the vehicle, introducing the priority to evaluate the obstacle detection model can make the evaluation metrics closer to the actual application scenario, and at the same time make the model evaluation results more reasonable and accurate.

[0101] Exemplarily, after the above S104, it is also possible to display the three-dimensional graph of the point cloud data and the priority of the first obstacle on the display interface; in response to the user's trigger operation, adjust the priority of the first obstacle.

[0102] After obtaining the priority of the first obstacle, the three-dimensional graph of the point cloud data and the priority of the first obstacle can be displayed on the user interface, so that the user can intuitively and efficiently browse the point cloud data and visually observe the priority classification results of each obstacle.

[0103] At the same time, the user can also click on the priority result of the first obstacle to manually increase or decrease the priority of the first obstacle, realize the optimization of the priority result of the first obstacle, and make the evaluation result of the obstacle detection model more accurate.

[0104] The evaluation method of the obstacle detection model provided in this embodiment obtains point cloud data and the annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data; obtains the target distance between the vehicle and the first obstacle according to the annotation data; obtains the target occlusion index of the second obstacle occluding the first obstacle according to the annotation data and the point cloud data, where the second obstacle is located between the vehicle and the first obstacle; obtains the priority of the first obstacle according to the target distance and the target occlusion index; and evaluates the obstacle detection model according to the priority of the first obstacle. The evaluation method of the obstacle detection model provided in this embodiment adds a new evaluation index for the obstacle priority on the basis of the prior art, making the evaluation index of the model more in line with the actual application scenario and at the same time making the evaluation result of the obstacle detection model more reasonable and accurate.

[0105] Figure 3 is the flow chart of the evaluation method of the obstacle detection model shown in the embodiments of the present application Figure 2 , Figure 4 is a schematic diagram of the scenario for obtaining the first occlusion index based on the annotation data, Figure 5 is a schematic diagram of the scenario for obtaining the second occlusion index based on the point cloud data. Combining Figures 3 to 5 , in this embodiment, on the basis of the above embodiment, the implementation manner of obtaining the target occlusion index is described in detail.

[0106] S201: Determine the end point coordinates of the first obstacle and the end point coordinates of the second obstacle according to the annotation data.

[0107] S202: Obtain the first viewing angle between the first obstacle and the vehicle according to the end point coordinates of the first obstacle and the coordinates of the vehicle.

[0108] Combining Figure 4 in the scenario, the target object is the first obstacle and the occluder is the second obstacle. According to the annotation data, the end point coordinates of the target object and the occluder can be obtained. According to the end point coordinates of the target object and the coordinates of the vehicle, the first viewing angle γ between the target object and the vehicle can be obtained i .

[0109] S203: Obtain the first occlusion angle of the second obstacle occluding the first obstacle according to the end point coordinates of the second obstacle and the coordinates of the vehicle.

[0110] S204: Obtain the first occlusion index according to the first viewing angle and the first occlusion angle.

[0111] Combining Figure 4In the scenario, according to the endpoint coordinates of the occluder and the coordinates of the vehicle, the first occlusion viewing angle between the occluder and the vehicle can be obtained. According to the above-mentioned first observation viewing angle γ i and the first occlusion viewing angle obtain the first occlusion index of the occluder occluding the target object The embodiments of the present application do not particularly limit the implementation manner of determining the first occlusion index

[0112] Exemplarily, a possible implementation manner is given here. In the above implementation manner, the first occlusion index can be the ratio between the first occlusion viewing angle and the first observation viewing angle γ i For example, the first occlusion index can be obtained by using formula (2) Formula (2) is as follows:

[0113]

[0114] Exemplarily, when there are multiple occluders between the vehicle and the target object, that is, when the second obstacle is multiple obstacles, the first occlusion index can be calculated by using formula (3). Formula (3) is as follows:

[0115]

[0116] where n is the total number of second obstacles between the vehicle and the first obstacle.

[0117] S205: Perform rasterization processing on the point cloud data to obtain rasterized point cloud data.

[0118] S206: Traverse each grid and determine whether there are partial second obstacles in each grid. If so, obtain the second occlusion viewing angle of the grid occluding the first obstacle.

[0119] Combined Figure 5 In view of this, after rasterization processing, some grids will include obstacles. Traverse all grids to determine the grids with partial second obstacles, such as Figure 5 the grids shown in j to obtain the second occlusion viewing angle θ of the grid with partial second obstacles occluding the first obstacle.

[0120] S207: Determine the third occlusion viewing angle according to the second occlusion viewing angles corresponding to at least one grid.

[0121] S208: Obtain the second occlusion index according to the first observation viewing angle and the third occlusion viewing angle. ​

[0122] Among them, different grids correspond to different second occlusion perspectives. After determining the second occlusion perspectives corresponding to all the grids, a third occlusion perspective is determined. Third occlusion perspective is the union of the second occlusion perspectives corresponding to all the grids with partial second obstacles above. The embodiments of the present application do not particularly limit the implementation manner of determining the third occlusion index and do not make any particular restrictions on the implementation manner of determining the third occlusion index.

[0123] Exemplarily, a possible implementation manner is given here. In the above implementation manner, the third occlusion perspective can be calculated by formula (4) Formula (4) is as follows:

[0124]

[0125] where n is the total number of grids with second obstacles in the grid map.

[0126] According to the above first observation perspective γ i and the third occlusion perspective obtain the second occlusion index of the occluder occluding the target object The embodiments of the present application do not particularly limit the implementation manner of determining the second occlusion index and do not make any particular restrictions on the implementation manner of determining the second occlusion index.

[0127] Exemplarily, a possible implementation manner is given here. The second occlusion index can be the ratio between the third occlusion perspective and the first observation perspective γ i For example: The second occlusion index can be calculated by formula (5) Formula (5) is as follows:

[0128]

[0129] S209: According to the first occlusion index and the second occlusion index, obtain the target occlusion index of the second obstacle occluding the first obstacle.

[0130] Among them, after obtaining the first occlusion index and the second occlusion index it is possible to obtain the target occlusion index B of the second obstacle occluding the first obstacle according to the above first occlusion index and the second occlusion index The target occlusion index B i is the union of the first occlusion index i and the second occlusion index and the second occlusion index The embodiments of the present application do not particularly limit the determination of the target occlusion index Bi There is no special limitation on the implementation method.

[0131] Exemplarily, a possible implementation method is given here. The target occlusion index B i can be calculated using formula (6). Formula (6) is as follows:

[0132]

[0133] Since the labeled data generally only labels obstacles of specific categories, if only the labeled data is considered, the occlusion situation of objects outside the categories to the obstacles will be missed. Therefore, in this application, the occlusion index is not only calculated based on the specific category obstacles labeled in the labeled data, but also the occlusion information is extracted from the original point cloud data, and the occlusion index of the first obstacle occluded by the unlabeled obstacles is calculated according to the extracted occlusion information; thus, the occlusion index of all obstacles to the first obstacle can be calculated more accurately.

[0134] The evaluation method of the obstacle detection model provided in this embodiment determines the occlusion situation of the obstacle jointly according to the labeled data and the point cloud data, avoiding the occurrence of the existing situation of missing the occlusion of the target obstacle by obstacles outside the category, and making the obtained target occlusion index more accurate. At the same time, the priority of the obstacle is evaluated based on two indicators: the distance between the obstacle and the vehicle and the occlusion situation of the obstacle. The smaller the distance between the obstacle and the vehicle and the less the occlusion, the higher the priority of the obstacle. When evaluating the obstacle detection model, evaluating in combination with the priority of the obstacle can make the evaluation indicators of the model more in line with the actual application scenario, and at the same time make the evaluation result of the obstacle detection model more reasonable and accurate.

[0135] Figure 6 It is a schematic structural diagram of the evaluation device for the obstacle detection model provided by this application. As Figure 6 shown, this application provides an evaluation device for an obstacle detection model. The evaluation device 300 for the obstacle detection model includes:

[0136] A first acquisition module 301, configured to acquire point cloud data and labeled data of the point cloud data, where the labeled data is used to indicate obstacles in the point cloud data;

[0137] A second acquisition module 302, configured to acquire the target distance between the vehicle and the first obstacle according to the labeled data; and acquire the target occlusion index of the second obstacle occluding the first obstacle according to the labeled data and the point cloud data, where the second obstacle is located between the vehicle and the first obstacle;

[0138] A third acquisition module 303, configured to acquire the priority of the first obstacle according to the target distance and the target occlusion index;

[0139] An evaluation module 304 is configured to evaluate the obstacle detection model according to the priority of the first obstacle.

[0140] Optionally, the second acquisition module 302 is specifically configured to determine the coordinates of the first obstacle according to the labeled data; obtain the Euclidean distance between the coordinates of the first obstacle and the coordinates of the vehicle according to the coordinates of the first obstacle and the coordinates of the vehicle, and use the Euclidean distance as the target distance.

[0141] Optionally, the second acquisition module 302 is specifically configured to obtain a first occlusion index of the second obstacle occluding the first obstacle according to the labeled data; obtain a second occlusion index of the second obstacle occluding the first obstacle according to the point cloud data; and obtain a target occlusion index of the second obstacle occluding the first obstacle according to the first occlusion index and the second occlusion index.

[0142] Optionally, the second acquisition module 302 is specifically configured to determine the endpoint coordinates of the first obstacle and the endpoint coordinates of the second obstacle according to the labeled data; obtain a first viewing angle between the first obstacle and the vehicle according to the endpoint coordinates of the first obstacle and the coordinates of the vehicle; obtain a first occlusion angle of the second obstacle occluding the first obstacle according to the endpoint coordinates of the second obstacle and the coordinates of the vehicle; and obtain the first occlusion index according to the first viewing angle and the first occlusion angle.

[0143] Optionally, the second acquisition module 302 is specifically configured to perform rasterization processing on the point cloud data to obtain rasterized point cloud data; traverse each grid to determine whether there is a part of the second obstacle in each grid, and if so, obtain a second occlusion angle of the grid occluding the first obstacle; determine a third occlusion angle according to the second occlusion angles corresponding to at least one grid; and obtain the second occlusion index according to the first viewing angle and the third occlusion angle.

[0144] Optionally, the third acquisition module 303 is specifically configured to obtain a priority index according to the target distance and the target occlusion index; and determine the priority of the first obstacle according to the priority index, where the priority index is inversely proportional to the priority of the first obstacle.

[0145] Optionally, the evaluation module 304 is specifically configured to input the point cloud data into the obstacle detection model to obtain a detection result output by the obstacle detection model; and evaluate the obstacle detection model according to the detection result and the priority of the first obstacle.

[0146] The evaluation device 300 of the obstacle detection model further includes:

[0147] A display module 305 is configured to display a three-dimensional graph of the point cloud data and the priority of the first obstacle on a display interface.

[0148] The optimization module 306 is configured to adjust the priority of the first obstacle in response to a triggering operation by the user.

[0149] Figure 7 Schematic structural diagram of an evaluation device for an obstacle detection model provided by this application. As Figure 7 shown, this application provides an evaluation device for an obstacle detection model. The evaluation device 400 for the obstacle detection model includes: a receiver 401, a transmitter 402, a processor 403, and a memory 404.

[0150] The receiver 401 is configured to receive instructions and data;

[0151] The transmitter 402 is configured to send instructions and data;

[0152] The memory 404 is configured to store computer-executable instructions;

[0153] The processor 403 is configured to execute the computer-executable instructions stored in the memory 404 to implement each step performed by the evaluation method of the obstacle detection model in the above embodiments. For specific details, reference can be made to the relevant descriptions in the embodiments of the foregoing obstacle data processing method.

[0154] Optionally, the above-mentioned memory 503 can be either independent or integrated with the processor 504.

[0155] When the memory 503 is independently provided, the electronic device further includes a bus for connecting the memory 503 and the processor 504.

[0156] This application also provides a computer-readable storage medium storing computer-executable instructions. When the processor executes the computer-executable instructions, the evaluation method of the obstacle detection model performed by the evaluation device for the obstacle detection model as described above is implemented.

[0157] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations. In the hardware implementation, the division between the functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be executed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that a communication medium typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0158] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0159] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An evaluation method for an obstacle detection model, characterized in that Including: Obtaining point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data; Obtaining a target distance between the vehicle and a first obstacle according to the annotation data; Obtaining a target occlusion index of a second obstacle occluding the first obstacle according to the annotation data and the point cloud data, where the second obstacle is located between the vehicle and the first obstacle; Obtaining a priority of the first obstacle according to the target distance and the target occlusion index; Evaluating an obstacle detection model according to the priority of the first obstacle; Among them, the obtaining a target occlusion index of a second obstacle occluding the first obstacle according to the annotation data and the point cloud data includes: Determining endpoint coordinates of the first obstacle and endpoint coordinates of the second obstacle according to the annotation data; Obtaining a first observation angle between the first obstacle and the vehicle according to the endpoint coordinates of the first obstacle and the coordinates of the vehicle; Obtaining a first occlusion angle of the second obstacle occluding the first obstacle according to the endpoint coordinates of the second obstacle and the coordinates of the vehicle; Obtaining the first occlusion index according to the first observation angle and the first occlusion angle; Performing rasterization processing on the point cloud data to obtain rasterized point cloud data; Traversing each grid, determining whether there is a part of the second obstacle in each grid, and if so, obtaining a second occlusion angle of the grid occluding the first obstacle; Determining a third occlusion angle according to the second occlusion angles corresponding to at least one grid; Obtaining the second occlusion index according to the first observation angle and the third occlusion angle; Obtaining a target occlusion index of the second obstacle occluding the first obstacle according to the first occlusion index and the second occlusion index.

2. The method according to claim 1, wherein The obtaining a target distance between the vehicle and a first obstacle according to the annotation data includes: Determining the coordinates of the first obstacle according to the annotation data; Obtaining an Euclidean distance between the coordinates of the first obstacle and the coordinates of the vehicle according to the coordinates of the first obstacle and the coordinates of the vehicle, and using the Euclidean distance as the target distance.

3. The method according to claim 1, characterized in that The obtaining a priority of the first obstacle according to the target distance and the target occlusion index includes: Obtaining a priority index according to the target distance and the target occlusion index; Determining the priority of the first obstacle according to the priority index, where the priority index is inversely proportional to the priority of the first obstacle.

4. The method according to claim 1, wherein The evaluating an obstacle detection model according to the priority of the first obstacle includes: Inputting the point cloud data into the obstacle detection model to obtain a detection result output by the obstacle detection model; Evaluating the obstacle detection model according to the detection result and the priority of the first obstacle.

5. The method according to claim 1, characterized in that, After the obtaining a priority of the first obstacle according to the target distance and the target occlusion index, the method further includes: Display the three-dimensional graph of the point cloud data and the priority of the first obstacle on the display interface; In response to a trigger operation by the user, adjust the priority of the first obstacle.

6. An evaluation device for an obstacle detection model, characterized in that, It includes: A first acquisition module, configured to acquire point cloud data and annotation data of the point cloud data, where the annotation data is used to indicate obstacles in the point cloud data; A second acquisition module, configured to acquire a target distance between the vehicle and the first obstacle according to the annotation data; According to the annotation data and the point cloud data, acquire a target occlusion index of the second obstacle occluding the first obstacle, where the second obstacle is located between the vehicle and the first obstacle; A third acquisition module, configured to acquire the priority of the first obstacle according to the target distance and the target occlusion index; An evaluation module, configured to evaluate an obstacle detection model according to the priority of the first obstacle; Wherein, when acquiring the target occlusion index of the second obstacle occluding the first obstacle according to the annotation data and the point cloud data, the second acquisition module specifically is configured to: Determine the endpoint coordinates of the first obstacle and the endpoint coordinates of the second obstacle according to the annotation data; Acquire a first viewing angle between the first obstacle and the vehicle according to the endpoint coordinates of the first obstacle and the coordinates of the vehicle; Acquire a first occlusion angle of the second obstacle occluding the first obstacle according to the endpoint coordinates of the second obstacle and the coordinates of the vehicle; Acquire the first occlusion index according to the first viewing angle and the first occlusion angle; Perform rasterization processing on the point cloud data to obtain rasterized point cloud data; Traverse each grid, and determine whether there is a part of the second obstacle in each grid. If so, acquire a second occlusion angle of the grid occluding the first obstacle; Determine a third occlusion angle according to the second occlusion angles corresponding to at least one grid; Acquire the second occlusion index according to the first viewing angle and the third occlusion angle; Acquire the target occlusion index of the second obstacle occluding the first obstacle according to the first occlusion index and the second occlusion index.

7. An evaluation device for an obstacle detection model, characterized in that, It includes: A memory; A processor; Wherein, the memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method for evaluating an obstacle detection model according to any one of claims 1-5.

8. A computer storage medium, characterized in that, Computer execution instructions are stored in the computer storage medium, and when the computer execution instructions are executed by the processor, they are used to implement the method for evaluating an obstacle detection model according to any one of claims 1 to 5.

Images