Method, apparatus, and device for rule-based annotation of point cloud data

Abstract

This application belongs to the technical field of data annotation, and in particular, relates to a method, device, and equipment for annotating point cloud data based on rules. The method includes: obtaining annotation data; determining whether there is original data for the annotation data, where the original data includes existing annotation information; if there is no original data, identifying the data type in the annotation data and annotating the annotation data in the form of an annotation box; selecting the annotation box as the father box, obtaining a prediction result according to a preset rule, and annotating the son box according to the prediction result, and outputting it as the task data annotation result, where the prediction result is that other annotation boxes that may have a father-son relationship are automatically associated with the father box by the preset rule; if there is original data, classifying the annotation data according to the original relationship of the annotation boxes in the annotation information in the task data; establishing a bipartite graph for matching according to the classification result; and outputting the annotation result according to the bipartite graph matching result, which can improve the annotation speed and the annotation accuracy.

Description

Technical Field

[0001] This application relates to the technical field of data annotation, and in particular, to a method, device, and equipment for annotating point cloud data based on rules. Background Art

[0002] Data annotation is to use automated tools to capture and collect data from the Internet, including text, pictures, voices, etc., and then organize and annotate the captured data.

[0003] Data annotation is a basic task in the field of artificial intelligence. A large number of data annotation specialists are required to engage in relevant work to meet the needs of artificial intelligence training data.

[0004] With the increase in the amount of annotated data and annotation scenarios, it is inevitable that the quality of the annotated data is not good. However, due to the huge amount of data, the cost of re-annotation is too high. To solve this problem, for some annotation content that needs to improve data quality, annotators can perform fine annotation on this part of the content. However, due to the particularity of the scenario, the annotation speed is not high. Summary of the Invention

[0005] Therefore, the embodiments of this application provide a method, device, and equipment for annotating point cloud data based on rules, which can solve the technical problems of low existing annotation accuracy and slow annotation speed. The specific technical solution content is as follows:

[0006] In a first aspect, the embodiments of this application provide a method for annotating point cloud data based on rules, and the method includes:

[0007] Obtain annotation data;

[0008] Determine whether there is original data in the annotation data, and the original data includes existing annotation information;

[0009] If there is no original data, identify the data type in the annotation data, and annotate the annotation data in the form of an annotation box;

[0010] Select the annotation box as the father box, obtain a prediction result according to a preset rule, and annotate the son box according to the prediction result, and output it as the annotation result of the task data. The prediction result is that other annotation boxes that may have a father-son relationship are automatically associated with the father box by the preset rule;

[0011] If there is original data, classify the annotation data according to the original relationship of the annotation boxes in the annotation information in the task data;

[0012] Establish a bipartite graph for matching according to the classification result;

[0013] Output the annotation result according to the bipartite graph matching result.

[0014] By adopting the above technical solution, the annotation data to be annotated is annotated in the form of an annotation box, which is convenient for quickly displaying the annotation content; according to the preset rules, when annotating the father box, the son box can be automatically associated according to the rules, improving the annotation accuracy; and after the first annotation, a second annotation is carried out to further improve the annotation accuracy. The second annotation uses a bipartite graph to achieve fast and accurate matching and improve the annotation accuracy.

[0015] Preferably, the father box is the position data of a certain vehicle, and the prediction result is obtained according to the preset rules:

[0016] The scoring index is carried out according to the relative position relationship of each object to be annotated set in advance, and the prediction result is obtained from each scoring index and the relative position relationship of each object to be annotated.

[0017] Preferably, obtaining the scoring index according to the relative position relationship of each object to be annotated set in advance, and obtaining the prediction result from each scoring index and the relative position relationship of each object to be annotated includes:

[0018] According to the fixed predetermined distance relationship with the father box, the predicted data of the attachment distance is obtained, the type of the attachment is judged, and the scoring index is obtained;

[0019] The prediction result is obtained according to the scoring index.

[0020] Preferably, obtaining the scoring index according to the relative position relationship of each object to be annotated set in advance, and obtaining the prediction result from each scoring index and the relative position relationship of each object to be annotated includes:

[0021] According to the distance from the relative center point of the attachment, the predicted data of the parallel vehicle is obtained, and the scoring index is obtained;

[0022] The prediction result is obtained according to the scoring index.

[0023] Preferably, obtaining the scoring index according to the relative position relationship of each object to be annotated set in advance, and obtaining the prediction result from each scoring index and the relative position relationship of each object to be annotated includes:

[0024] According to the relationship between the distance between the father box and the parallel vehicle, the rotation angle of the father box relative to the world coordinate system, and the rotation angle of the parallel vehicle relative to the father box coordinate system at present, the predicted data of the attribution of the attachment is obtained, and the scoring index is obtained;

[0025] The prediction result is obtained according to the scoring index.

[0026] Preferably, obtaining the scoring index according to the relative position relationship of each object to be annotated set in advance, and obtaining the prediction result from each scoring index and the relative position relationship of each object to be annotated includes:

[0027] According to the type of appendages and the number of the corresponding appendage types, the prediction data of the number of appendages is obtained to obtain the scoring index;

[0028] The prediction results are obtained based on the scoring indicators.

[0029] Preferably, obtaining a scoring index according to the preset relative position relationship of each object to be labeled, and obtaining a prediction result according to each scoring index and the relative position relationship of each object to be labeled includes:

[0030] According to the speed of the vehicle in the parent frame, the movement distance and direction of the corresponding relative position of the appendage are given, the prediction data of the movement distance and direction are generated, and the scoring index is obtained;

[0031] The prediction results are obtained based on the scoring indicators.

[0032] Preferably, obtaining a scoring index according to the preset relative position relationship of each object to be labeled, and obtaining a prediction result according to each scoring index and the relative position relationship of each object to be labeled includes:

[0033] Based on the fixed predetermined distance relationship with the father frame, the predicted data of the distance of the appendage is obtained, the type of the appendage is determined, and the scoring index is obtained;

[0034] The prediction data of the parallel vehicles are obtained according to the relative center point distances with the appendages, and a scoring index is obtained;

[0035] According to the relationship between the distance between the parent frame and the parallel vehicle, the rotation angle of the parent frame relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the parent frame coordinate system, the predicted data of the ownership of the attachment is obtained, and the scoring index is obtained;

[0036] According to the type of appendages and the number of the corresponding appendage types, the prediction data of the number of appendages is obtained to obtain the scoring index;

[0037] According to the speed of the parent frame vehicle, the movement distance and direction of the corresponding relative position of the appendage are given, the prediction data of the movement distance and direction are generated, and the scoring index is obtained.

[0038] The prediction result is obtained based on the product of various scoring indicators.

[0039] Preferably, the method further comprises:

[0040] Get the trigger signal and determine whether the trigger signal belongs to the child frame or the father frame;

[0041] If the trigger signal belongs to the child frame, the parent-child relationship between the child frame and the father frame is terminated;

[0042] If the trigger signal belongs to the father box, the father-son relationship between the father box and all its son boxes is terminated.

[0043] Preferably, the method further includes:

[0044] Set an ID for the father box, set the ID of the father box in the son box, and establish the father-son relationship between the father box and the son box.

[0045] In a second aspect, an embodiment of the present application provides a device for rule-based annotation of point cloud data, the device including:

[0046] A data acquisition module for acquiring annotation data;

[0047] A judgment module for judging whether there is original data for the annotation data, where the original data includes existing annotation information;

[0048] An annotation module for, if there is no original data, identifying the data type in the annotation data and annotating the annotation data in the form of an annotation box;

[0049] A first output module for selecting the annotation box as the father box, obtaining a prediction result according to a preset rule, and annotating the son box according to the prediction result, and outputting it as an annotation result, where the prediction result is that other annotation boxes that may have a father-son relationship are automatically associated with the father box by the preset rule;

[0050] A classification module for, if there is original data, classifying the annotation data according to the original relationship of the annotation boxes in the annotation information;

[0051] A bipartite graph matching module for establishing a bipartite graph for matching according to the classification result;

[0052] A second output module for outputting an annotation result according to the bipartite graph matching result.

[0053] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for rule-based annotation of point cloud data described in any one of the foregoing are implemented.

[0054] In summary, compared with the prior art, the beneficial effects brought by the technical solution provided by the embodiment of the present application at least include:

[0055] 1. For the labeled data without original data, it is labeled in the form of a labeled box to facilitate the quick display of the labeled content. According to the preset rules, when labeling the father box, the son box can be automatically associated according to the rules, improving the labeling accuracy. For the labeled data with original data, secondary labeling is performed to further improve the labeling accuracy. The secondary labeling uses a bipartite graph to achieve fast and accurate matching and improve the labeling precision.

[0056] 2. Point cloud data is generally real-time data. When performing data labeling, in the real-time data, the relative position relationship of each vehicle can be quickly obtained. By comparing the data of the front and rear frames, it is easy to know the position relationship between the vehicle and its accessories, and thus it can be known whether there is a parent-child relationship between the vehicle and the accessories. Therefore, when labeling point cloud data, only one father box needs to be labeled, and the son box associated with the father box can be directly obtained by the preset rules, thus achieving fast labeling. Moreover, when performing labeling, the parent-child relationship in the labeled data is determined in advance, so the data in the labeled data whose parent-child relationship is not confirmed will be reduced, facilitating subsequent fast labeling and shortening the data labeling time.

[0057] 3. In point cloud data, the easiest relationship to find is the relative position relationship between vehicles. As long as the labeled box is determined, the relative position relationship between vehicles can be obtained, and whether there is a parent-child relationship is confirmed through the relative position relationship between vehicles. The calculation is simpler and easier to implement. Brief Description of the Drawings

[0058] Figure 1 It is a schematic flowchart of a method for labeling point cloud data based on rules provided by one embodiment of the present application. Detailed Embodiment

[0059] This specific embodiment is only an explanation of the present application and does not limit the present application. After reading this specification, those skilled in the art can make modifications to this embodiment without creative contributions as needed, but as long as it is within the scope of the claims of the present application, it is protected by the patent law.

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0061] In this application, terms such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions. It should be understood that there is no logical or chronological dependence between "first", "second", and "nth", nor are the quantity and execution order limited.

[0062] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0063] Refer to Figure 1 , in an embodiment of the present application, a method for rule-based annotation of point cloud data is provided. The main steps of the method are described as follows:

[0064] S1: Obtain annotation data;

[0065] S2: Determine whether there is original data in the annotation data. The original data includes existing annotation information;

[0066] S3: If there is no original data, identify the data type in the annotation data and annotate the annotation data in the form of an annotation box;

[0067] S4: Select the annotation box as the father box, obtain the prediction result according to the preset rule, and annotate the son box according to the prediction result. The output is the annotation result of the task data. The prediction result is that other annotation boxes that may have a father-son relationship are automatically associated with the father box by the preset rule;

[0068] S5: If there is original data, classify the annotation data according to the original relationship of the annotation boxes in the annotation information;

[0069] S6: Establish a bipartite graph for matching according to the classification result by a predetermined rule;

[0070] S7: Output the annotation result according to the bipartite graph matching result.

[0071] Specifically, in this embodiment, the annotation data is obtained, and the annotation data is point cloud data.

[0072] Identify the data type of the annotation data, that is, identify that the annotation data is point cloud data.

[0073] The father-son relationship is to confirm the relationship between various accessory items on the vehicle. The father box is usually used to annotate the vehicle body, and the son box is usually used to annotate vehicle accessories such as rearview mirrors, tires, and headlights. The father-son relationship is the relationship between the vehicle body and the rearview mirrors, tires, headlights, etc. provided on the vehicle body, which is a subordinate relationship.

[0074] Label the father box on the labeled data. The preset rule is the relationship between the father box and the son box. In this embodiment, it is assumed that the labeled data is point cloud data, and the preset rule is that the son box having a fixed distance relationship with the father box has a father-son relationship with this father box; in other embodiments, other methods can be adopted for the preset rule to confirm the father-son relationship, such as the son box having an equation calculation relationship with the father box has a father-son relationship with this father box, the son box having a relative position relationship with the father box has a father-son relationship with this father box, the son box having a relative size and relative position relationship with the father box has a father-son relationship with this father box, etc., which will not be elaborated here.

[0075] The prediction result is that other labeled boxes that may have a father-son relationship are automatically associated with the father box by the preset rule. It can be a box pre-labeled as a son box or a box pre-labeled as a father box. In this embodiment, it is a box pre-labeled as a son box.

[0076] Furthermore, if the same son box has a father-son relationship with multiple father boxes, the actual father-son relationship can be determined by the criteria of the preset rule. For example, when the preset rule is that the son box having a relative position relationship with the father box has a father-son relationship with this father box, and in the case where the labeled data is point cloud data, the vehicle corresponding to the father box A is moving, and the vehicle corresponding to the father box B is moving forward in parallel with the vehicle corresponding to the father box A. In this case, the criterion of the preset rule is that the relative position between the son box and the father box is stationary. The rearview mirror (son box) is stationary relative to the father box A, and the rearview mirror has a relative displacement with the father box B. Then it is determined that the rearview mirror is an accessory of the father box A, that is, the son box corresponding to this rearview mirror has a father-son relationship with the father box A.

[0077] Specifically, the original data is the labeled information that already exists in the labeled data, such as there already being labeled boxes, the IDs of the labeled boxes, etc. in the labeled data; the labeled data without original data is output as labeled data after the first father box and son box labeling and establishing the relationship between the father box and the son box; for the labeled data with original data, in this embodiment, when labeling the labeled data, the son boxes that may have a father-son relationship are automatically associated with the father box by the pre-set rules, realizing fast labeling and reducing labeling errors.

[0078] Perform secondary labeling on the labeled data with original data, and classify the labeled boxes that already exist in the task data. In this embodiment, the existing labeled boxes are classified as identifying father boxes and son boxes, and the father boxes and son boxes having a father-son relationship are divided into attribution relationships. According to the preset regulations, calculate again whether there is a father-son relationship between the father box and the son box, establish a bipartite graph for matching according to the classification results, and label the matched father boxes and son boxes on the task data to update the father-son relationship. In this embodiment, through bipartite graph matching, the accuracy of labeling is improved.

[0079] Bipartite graph maximum weight matching algorithm: After defining the degree of the parent-child relationship between two boxes, an adjacency matrix of the relationship degree corresponding to each son box and father box can be obtained. Next, the problem is transformed into finding the matching between the two to maximize the sum of the similarities of all connections. Generally, the KM algorithm is used to solve this problem. Of course, the minimum cost maximum flow algorithm can also be used to run, which will not be elaborated here. Since there are no requirements for the labeled data in terms of time and space, a general algorithm can be used.

[0080] Further, in another embodiment, a father box is selected, and the prediction result is obtained according to a preset rule, and the son boxes of the father box are labeled.

[0081] The point cloud data can be labeled in real time. In this embodiment, when labeling the point cloud data, only the father box needs to be recognized, and the son box can be predicted in real time by the preset rule. Because in real-time data, when the vehicle labeled by the father box is in a moving state in real-time data, generally, it is rare for the relative positions of more than two vehicles to be stationary during driving. However, the accessories of the vehicle are relatively stationary with the vehicle. Therefore, in the point cloud data labeling, only the father box needs to be selected, and the son box with a parent-child relationship can be automatically associated according to the preset rule, and the labeling accuracy is relatively high and the situation of missing labels is not likely to occur.

[0082] Further, in another embodiment, the labeled data is point cloud data, and the father box is the position data of a certain vehicle, that is, the father box labels the position of the vehicle body of a certain vehicle.

[0083] The preset rule is: Obtain the scoring index according to the relative position relationship of each object to be labeled set in advance, and obtain the prediction result from each scoring index and the relative position relationship of each object to be labeled.

[0084] Specifically, for a vehicle, the relative position relationships of each object to be labeled set in advance can be: the fixed position relationship of the rearview mirror relative to the vehicle body, the fixed position relationship of the tire relative to the vehicle body, the fixed position relationship of the headlight relative to the vehicle body, the relative position relationship between the vehicle body and the parallel vehicle, etc. In this embodiment, the way to obtain the scoring index can be:

[0085] 1. Predetermine the fixed positional relationships between multiple vehicle bodies and various accessories. Calculate the scoring index based on the difference between the relative positional relationship of the vehicle body and the accessory and the fixed positional relationship, that is, scoring index = full score value - |distance value between the center of the father box and the center of the son box - numerical value of the fixed positional relationship|. For example, the fixed positional relationship between the center of the vehicle body and the center of the rearview mirror is 60 cm, and the error range is 60 ± 10 cm. The fixed position between the center of vehicle body A (father box A) and the center of the rearview mirror (son box) is 61 cm, and the fixed position between the center of vehicle body B (father box B) and the center of the rearview mirror (son box) is 58 cm. When it is 60 cm, the full score is 20 points. The scoring index for the distance between the center of vehicle body A and the center of the rearview mirror is calculated by this formula: scoring index = 20 - |61 - 60| = 19. The scoring index for the distance between the center of vehicle body B and the center of the rearview mirror is calculated by this formula: scoring index = 20 - |58 - 60| = 18. Since 19 > 18, vehicle body A has a father-son relationship with this rearview mirror. This kind of comparison is often used for stationary vehicles, and of course, it is also applicable to moving vehicles.

[0086] 2. In another way of obtaining the scoring index, when the vehicle is moving, the father box and the son box that can calculate the scoring index with the same numerical value have a father-son relationship, without the need for comparison.

[0087] Furthermore, in another embodiment, the scoring index is carried out according to the preset relative positional relationships of various objects to be marked. The prediction results obtained from the scoring indexes and the relative positional relationships of various objects to be marked include:

[0088] Based on the fixed predetermined distance relationship with the father box, obtain the predicted data of the accessory distance, judge the type of the accessory, obtain the scoring index, and obtain the prediction result according to the scoring index.

[0089] Furthermore, in another embodiment, the scoring index is carried out according to the preset relative positional relationships of various objects to be marked. The prediction results obtained from the scoring indexes and the relative positional relationships of various objects to be marked include:

[0090] Based on the distance from the relative center point to the accessory, obtain the predicted data of the parallel vehicle, obtain the scoring index, and obtain the prediction result according to the scoring index.

[0091] Furthermore, in another embodiment, the scoring index is carried out according to the preset relative positional relationships of various objects to be marked. The prediction results obtained from the scoring indexes and the relative positional relationships of various objects to be marked include:

[0092] Based on the relationship between the distance of the father frame from the parallel vehicle, the rotation angle of the father frame relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the father frame coordinate system, predictive data on the attribution of the appendage is obtained, and a scoring metric is acquired. Based on the scoring metric, a prediction result is obtained.

[0093] Further, in another embodiment, a scoring metric is based on the pre-set relative position relationships of the various objects to be labeled. The prediction result obtained from the various scoring metrics and the relative position relationships of the various objects to be labeled includes:

[0094] Based on the appendage type and the quantity corresponding to that appendage type, a judgment is made to obtain predictive data on the appendage quantity, and a scoring metric is acquired. Based on the scoring metric, a prediction result is obtained.

[0095] Further, in another embodiment, a scoring metric is based on the pre-set relative position relationships of the various objects to be labeled. The prediction result obtained from the various scoring metrics and the relative position relationships of the various objects to be labeled includes:

[0096] Based on the speed of the movement of the father frame vehicle, the movement distance and direction of the corresponding appendage relative position are given, predictive data on the movement distance and direction are generated, and a scoring metric is acquired. Based on the scoring metric, a prediction result is obtained.

[0097] Further, in another embodiment, a scoring metric is based on the pre-set relative position relationships of the various objects to be labeled. The prediction result obtained from the various scoring metrics and the relative position relationships of the various objects to be labeled includes:

[0098] 1. Based on the fixed predetermined distance relationship with the father frame, predictive data on the appendage distance is obtained, the appendage type is judged, and a scoring metric is acquired;

[0099] 2. Based on the distance from the relative center point of the appendage, predictive data on the parallel vehicle is obtained, and a scoring metric is acquired;

[0100] 3. Based on the relationship between the distance of the father frame from the parallel vehicle, the rotation angle of the father frame relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the father frame coordinate system, predictive data on the attribution of the appendage is obtained, and a scoring metric is acquired;

[0101] 4. Based on the appendage type and the quantity corresponding to that appendage type, a judgment is made to obtain predictive data on the appendage quantity, and a scoring metric is acquired;

[0102] 5. Based on the speed of the movement of the father frame vehicle, the movement distance and direction of the corresponding appendage relative position are given, predictive data on the movement distance and direction are generated, and a scoring metric is acquired;

[0103] 6. Obtain the prediction result based on the product of each scoring index.

[0104] Specifically, in this embodiment, one or several of 1-5 can be selected according to actual needs. And each time a judgment is made, peer judgment is not allowed, and the scoring indexes are judged according to the priority order, and the priorities of each item can be swapped.

[0105] An example of obtaining the scoring indexes in this embodiment is as follows:

[0106] There are several vehicles on the road. For the rearview mirror A:

[0107] 1. Generally, the rearview mirror (appendage) is at the position of 0.6-0.8 of the vehicle body. Determine the midpoint of vehicle A (midpoint of the father frame) at the position of 0.64 (plus or minus 0.2) from the midpoint of the rearview mirror (midpoint of the son frame) according to the comparison of the front and rear frames. Give a score of 1.4 to the vehicle within the error range, and give a score of 0.5 to the non-conforming one; the distance between the vehicle and its appendage is a fixed value, that is, when judging the appendage relationship between the appendage and the vehicle, several distance ranges adapted to different appendages are preset as prediction data. For example, the rearview mirror is in the range of 0.6-0.8 of the vehicle body, and the wheel is in the range of 0.9-1.0 of the vehicle body, etc. If it is detected that the appendage is within the range of 0.6-0.8 of vehicle A, the score is 1.4, and this appendage is classified as a rearview mirror that may be vehicle A. If it does not conform, that is, the appendage outside the possible range of any appendage of the current vehicle A gets a score of 0.5.

[0108] 2. Calculate the distance between the midpoint of each current vehicle and the midpoint of the rearview mirror A as the prediction data. The vehicle A with the smallest distance between the midpoint and the midpoint of the rearview mirror A gets a score of 1.0, and other vehicles get a score of 0.5.

[0109] 3. Since the scene is going straight without turning, all vehicles are given 1.0; if in the coordinate system of turning or cornering, calculate the angle between the rearview mirror and the same coordinate axis of the vehicle coordinate system with the midpoint of the vehicle as the coordinate origin in the established coordinate system as the prediction data. If the angle is within the preset angle range, the score is 1.0, and if the angle is not within the preset angle range, the score is 0.5; in other embodiments, the vector sum and turning angle before and after the vehicle movement can also be calculated using the PCD result received when the vehicle is driving. According to the vector sum and turning angle, it can be judged whether it is turning or going straight, and the turning angle, so as to obtain the scoring index.

[0110] 4. The predicted data of the appendage corresponding to the detected type near the vehicle is no more than 2. If there are less than 2 appendages with a father-son relationship of this type near vehicle A, then vehicle A gets a score of 1.3; if the number of appendages of the same type with a father-son relationship that have been judged for vehicle A exceeds 2, then the score is 0.5.

[0111] 5. The motion trajectory of the vehicle and the rearview mirror A is predicted based on the relative position relationship between the vehicle speed and the rearview mirror A as the predicted data. When the predicted data of the vehicle and the rearview mirror are relatively stationary, the vehicle A and the rearview mirror A are relatively stationary so a score of 1.0 is given. If not, a score of 0.5 is given. If both vehicles are in a moving scene, the movement speed of the vehicle is detected to give the movement distance and direction of the corresponding relative position of the accessory, and the predicted data of the movement distance and direction are generated. If the preset data is within the preset relationship between the accessory and its vehicle, a score of 1.0 is given. If not, a score of 0.5 is given.

[0112] 6. The final scoring index of the parent-child relationship between vehicle A and rearview mirror A is: F=1.4*1.0*1.0*1.3*1.0=1.82.

[0113] When any one, two or more of the above 1-5 rules are used to obtain the corresponding scoring index, the scoring index is calculated by scoring. If one of the rules is used to obtain the corresponding scoring index, and there is only one scoring item, then the scoring item can be used as the scoring index. In other implementations, there may be other calculation methods, which will not be described in detail here.

[0114] Optionally, in another embodiment, the method further comprises:

[0115] S8: Setting an ID for the father frame, setting the ID of the father frame in the son frame, and establishing a parent-child relationship between the father frame and the son frame. The parent-child relationship between the father frame and the son frame is determined by the matching result, the father frame is marked with a determined ID, and the ID of the father frame is copied to the son frame with the parent-child relationship, thereby generating a parent-child relationship between the father frame and the son frame.

[0116] Optionally, in another embodiment, the method further comprises:

[0117] S9: Obtain a trigger signal and determine whether the trigger signal belongs to the child frame or the father frame;

[0118] S10: if the trigger signal belongs to the child frame, the parent-child relationship between the child frame and the father frame is terminated;

[0119] S11: If the trigger signal belongs to the father frame, the parent-child relationship between the father frame and all the child frames belonging to it is terminated.

[0120] The trigger signal can be generated by the user clicking the mouse, or by moving the mouse to the corresponding annotation box. This will not be elaborated here. In this embodiment, an example is given where the user moves the mouse to the corresponding annotation box and clicks the right mouse button; according to the given ID, it is expressed whether it is a pair of parent and child boxes. After the data annotation is completed, the staff can use a computer device, such as a computer, to move the mouse to an incorrect connection box on the annotated data. Whether the annotation box is a father box or a son box is fine. Right-click the mouse to cancel the parent-child relationship between the two boxes. Left-click to select a father box on the annotated data, and then the son boxes that have not established a parent-child relationship near the just-selected father box will be switched on the annotated data. Then left-click to select a son box, and a specific same ID can be given to the two boxes to represent the parent-child relationship between the two boxes.

[0121] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0122] In an embodiment of the present application, a device for rule-based annotation of point cloud data is provided, and the device for rule-based annotation of point cloud data corresponds one-to-one with the method for rule-based annotation of point cloud data in the above embodiment. The device for rule-based annotation of point cloud data includes:

[0123] A judgment module, configured to judge whether there is original data in the annotated data, and the original data includes existing annotation information;

[0124] An annotation module, configured to, if there is no original data, identify the data type in the annotated data and annotate the annotated data in the form of an annotation box;

[0125] A first output module, configured to select an annotation box as the father box, obtain a prediction result according to a preset rule, and annotate the son box according to the prediction result, and output it as an annotation result. The prediction result is that other annotation boxes that may have a parent-child relationship are automatically associated with the father box by the preset rule;

[0126] A classification module, configured to, if there is original data, classify the annotated data according to the original relationship of the annotation boxes in the annotation information;

[0127] A bipartite graph matching module, configured to establish a bipartite graph for matching according to the classification result;

[0128] A second output module, configured to output an annotation result according to the bipartite graph matching result.

[0129] Further, in another embodiment, the father box is the position data of a certain vehicle, and the prediction result is obtained according to the preset rule:

[0130] Obtain a scoring index based on the relative position relationships of various objects to be labeled preset, and obtain a prediction result from each scoring index and the relative position relationships of various objects to be labeled.

[0131] Further, in another embodiment, the obtaining a scoring index based on the relative position relationships of various objects to be labeled preset, and obtaining a prediction result from each scoring index and the relative position relationships of various objects to be labeled includes:

[0132] Obtain predicted data on the distance of the appendage according to the fixed predetermined distance relationship with the father frame, determine the type of the appendage, and obtain a scoring index;

[0133] Obtain a prediction result according to the scoring index.

[0134] Further, in another embodiment, the obtaining a scoring index based on the relative position relationships of various objects to be labeled preset, and obtaining a prediction result from each scoring index and the relative position relationships of various objects to be labeled includes:

[0135] Obtain predicted data on the side-by-side vehicle according to the distance from the father frame to the side-by-side vehicle, the rotation angle of the father frame relative to the world coordinate system, and the rotation angle of the side-by-side vehicle relative to the father frame coordinate system currently, and obtain a scoring index;

[0136] Obtain a prediction result according to the scoring index.

[0137] Further, in another embodiment, the obtaining a scoring index based on the relative position relationships of various objects to be labeled preset, and obtaining a prediction result from each scoring index and the relative position relationships of various objects to be labeled includes:

[0138] Obtain predicted data on the attribution of the appendage according to the relationship between the distance from the father frame to the side-by-side vehicle, the rotation angle of the father frame relative to the world coordinate system, and the rotation angle of the side-by-side vehicle relative to the father frame coordinate system currently, and obtain a scoring index;

[0139] Obtain a prediction result according to the scoring index.

[0140] Further, in another embodiment, the obtaining a scoring index based on the relative position relationships of various objects to be labeled preset, and obtaining a prediction result from each scoring index and the relative position relationships of various objects to be labeled includes:

[0141] Judge according to the appendage type and the quantity corresponding to the appendage type to obtain predicted data on the appendage quantity, and obtain a scoring index;

[0142] Obtain a prediction result according to the scoring index.

[0143] Furthermore, in another embodiment, obtaining a scoring index according to the preset relative position relationship of each object to be labeled, and obtaining a prediction result according to each scoring index and the relative position relationship of each object to be labeled includes:

[0144] According to the speed of the vehicle in the parent frame, the movement distance and direction of the corresponding relative position of the appendage are given, the prediction data of the movement distance and direction are generated, and the scoring index is obtained;

[0145] The prediction results are obtained based on the scoring indicators.

[0146] Furthermore, in another embodiment, obtaining a scoring index according to the preset relative position relationship of each object to be labeled, and obtaining a prediction result according to each scoring index and the relative position relationship of each object to be labeled includes:

[0147] Based on the fixed predetermined distance relationship with the father frame, the predicted data of the distance of the appendage is obtained, the type of the appendage is determined, and the scoring index is obtained;

[0148] The prediction data of the parallel vehicles are obtained according to the relative center point distances with the appendages, and a scoring index is obtained;

[0149] According to the relationship between the distance between the parent frame and the parallel vehicle, the rotation angle of the parent frame relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the parent frame coordinate system, the predicted data of the ownership of the attachment is obtained, and the scoring index is obtained;

[0150] According to the type of appendages and the number of the corresponding appendage types, the prediction data of the number of appendages is obtained to obtain the scoring index;

[0151] According to the speed of the parent frame vehicle, the movement distance and direction of the corresponding relative position of the appendage are given, the prediction data of the movement distance and direction are generated, and the scoring index is obtained.

[0152] The prediction result is obtained based on the product of various scoring indicators.

[0153] Furthermore, in another embodiment, the method further comprises:

[0154] Get the trigger signal and determine whether the trigger signal belongs to the child frame or the father frame;

[0155] If the trigger signal belongs to the child frame, the parent-child relationship between the child frame and the father frame is terminated;

[0156] If the trigger signal belongs to the father frame, the parent-child relationship between the father frame and all the child frames belonging to it is released.

[0157] Furthermore, in another embodiment, the method further comprises:

[0158] Set an ID for the father box, set the ID of the father box in the son box, and establish the parent-child relationship between the father box and the son box.

[0159] Each module of the above device for rule-based annotation of point cloud data can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0160] In an embodiment of the present application, a computer device is provided. The computer device can be a server. The computer device includes a processor, a memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Volatile or non-volatile storage devices include, but are not limited to: magnetic disks, optical disks, EEPROM (Electrically-Erasable Programmable Read Only Memory), EPROM (Erasable Programmable Read Only Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), magnetic memories, flash memories, PROM (Programmable Read-Only Memory). The memory of the computer device provides an environment for the operation of the operating system and computer programs stored therein. The network interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the method steps of the method for rule-based annotation of point cloud data described in the above embodiment.

[0161] In an embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program is executed by the processor, it implements the method steps of the method for rule-based annotation of point cloud data described in the above embodiment. The computer-readable storage medium includes ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic disks, floppy disks, etc.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be assigned to different functional units and modules according to needs, that is, the internal structure of the device described in this application is divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for rule-based annotation of point cloud data, characterized in that, The method includes: obtaining labeled data; judging whether there is original data in the labeled data, where the original data includes existing labeled information; if there is no original data, identifying the data type in the labeled data and labeling the labeled data in the form of a labeled box; selecting the labeled box as the father box, obtaining a prediction result according to a preset rule, and labeling the son box according to the prediction result, and outputting it as a labeling result, where the prediction result is that other labeled boxes that may have a father-son relationship are automatically associated with the father box by the preset rule; if there is original data, classifying the labeled data according to the original relationship of the labeled boxes in the labeled information; establishing a bipartite graph for matching according to the classification result; outputting a labeling result according to the bipartite graph matching result; the father box is the position data of a certain vehicle, and a prediction result is obtained according to a preset rule: obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled.

2. The method for rule-based annotation of point cloud data according to claim 1, wherein, The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: obtaining predicted data on the distance of the accessory according to the fixed predetermined distance relationship with the father box, judging the type of the accessory, and obtaining a scoring index; obtaining a prediction result according to the scoring index.

3. The method for rule-based annotation of point cloud data according to claim 1, wherein The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: obtaining predicted data on the parallel vehicle according to the distance from the relative center point of the accessory, and obtaining a scoring index; obtaining a prediction result according to the scoring index.

4. The method for rule-based annotation of point cloud data according to claim 1, wherein The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: obtaining predicted data on the attribution of the accessory according to the relationship between the distance between the father box and the parallel vehicle, the rotation angle of the father box relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the father box coordinate system, and obtaining a scoring index; obtaining a prediction result according to the scoring index.

5. The method for rule-based annotation of point cloud data according to claim 1, wherein The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: judging according to the accessory type and the quantity corresponding to the accessory type to obtain predicted data on the quantity of the accessory, and obtaining a scoring index; obtaining a prediction result according to the scoring index.

6. The method for rule-based annotation of point cloud data according to claim 1, wherein The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: giving the moving distance and direction of the relative position of the corresponding accessory according to the moving speed of the vehicle of the father box, generating predicted data on the moving distance and direction, and obtaining a scoring index; obtaining a prediction result according to the scoring index.

7. The method for rule-based annotation of point cloud data according to claim 1, characterized in that, The obtaining a scoring index according to the relative position relationship of each object to be labeled set in advance, and obtaining a prediction result from each scoring index and the relative position relationship of each object to be labeled includes: Predictive data of the attachment distance is obtained according to the fixed predetermined distance relationship with the father box, the type of the attachment is judged, and a scoring index is obtained; Predictive data of the parallel vehicle is obtained according to the distance from the relative center point of the attachment, and a scoring index is obtained; Predictive data of the attribution of the attachment is obtained according to the relationship between the distance between the father box and the parallel vehicle, the rotation angle of the father box relative to the world coordinate system, and the current rotation angle of the parallel vehicle relative to the father box coordinate system, and a scoring index is obtained; Judgment is made according to the type of the attachment and the quantity corresponding to the type of the attachment to obtain predictive data of the attachment quantity, and a scoring index is obtained; The moving distance and direction of the relative position of the corresponding attachment are given according to the moving speed of the father box vehicle, predictive data of the moving distance and direction are generated, and a scoring index is obtained; The prediction result is obtained according to the product of the scoring indexes.

8. The method for rule-based annotation of point cloud data according to claim 3, wherein The method further includes: Obtain a trigger signal and judge whether the trigger signal belongs to the son box or the father box; If the trigger signal belongs to the son box, the father-son relationship between the son box and the father box is released; If the trigger signal belongs to the father box, the father-son relationship between the father box and all its subordinate son boxes is released.

9. The method for rule-based annotation of point cloud data according to claim 3, wherein The method further includes: Set an ID for the father box, set the ID of the father box in the son box, and establish the father-son relationship between the father box and the son box.

10. An apparatus for rule-based annotation of point cloud data, characterized in that, The device includes: A data acquisition module for acquiring annotation data; A judgment module for judging whether there is original data in the annotation data, and the original data includes existing annotation information; An annotation module for identifying the data type in the annotation data and annotating the annotation data in the form of an annotation box if there is no original data; A first output module for selecting the annotation box as the father box, the father box being the position data of a vehicle, obtaining a scoring index according to the relative position relationship of each object to be annotated preset, obtaining a prediction result from each scoring index and the relative position relationship of each object to be annotated, and annotating the son box according to the prediction result, and outputting as an annotation result, the prediction result being that other annotation boxes that may have a father-son relationship are automatically associated with the father box by a preset rule; A classification module for classifying the annotation data according to the original relationship of the annotation boxes in the annotation information if there is original data; A bipartite graph matching module for establishing a bipartite graph for matching according to the classification result; A second output module for outputting an annotation result according to the bipartite graph matching result.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for rule-based annotation of point cloud data according to any one of claims 1 to 10 are implemented.

Images