Data detection methods, devices and media
By comparing the annotation results of manual quality inspection and point cloud annotation models, point cloud pairs with inconsistent semantic labels are identified and processed, solving the problem of incomplete coverage of manual quality inspection and improving the data quality and security of autonomous driving models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAOMI EV TECH CO LTD
- Filing Date
- 2023-08-24
- Publication Date
- 2026-06-30
AI Technical Summary
In existing technologies, manual quality inspection cannot cover all labeled data and there is inconsistent understanding of labeling rules, resulting in low quality of model training data and increasing the risk of false detections on autonomous driving roads.
By comparing the annotation results of manual quality inspection and point cloud annotation models, point cloud pairs with the same location but different semantic labels are identified. When the number reaches a threshold, difference information is sent for secondary annotation, and the updated point cloud annotation model is used to improve data quality.
This improved the quality of labeled data, reduced the risk of false detections during model training, and enhanced the safety and accuracy of autonomous driving.
Smart Images
Figure CN116824186B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of autonomous driving, and in particular to a data detection method, apparatus and medium. Background Technology
[0002] In related technologies, labeled sample data is typically used for model training. Generally, labeled data is first manually quality-checked. However, manual quality checking does not cover all labeled sample data, and quality checkers do not have a completely consistent understanding of the labeling rules. Missing or incorrect labels in ambiguous areas such as boundaries are not easy to detect. Therefore, if a model trained with a large number of incorrectly labeled sample data is used to achieve semantic segmentation of real road surfaces, it will reduce the semantic segmentation effect of the model, thereby increasing the risk of road surface misdetection and causing erroneous operations such as intermittent braking, which will affect autonomous driving.
[0003] Therefore, improving the quality of sample data used to train models is a pressing technical problem that needs to be solved. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides a data detection method, apparatus and medium.
[0005] According to a first aspect of the present disclosure, a data detection method is provided, comprising:
[0006] The first annotation result and the second annotation result of the point cloud data corresponding to the autonomous driving image are obtained. The first annotation result is obtained by manually quality checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data.
[0007] A target point cloud pair is determined, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud.
[0008] If the number of target point cloud pairs is determined to be greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal, and the difference information is used to indicate the position of the second point cloud in the second annotation result.
[0009] Optionally, the method further includes:
[0010] Identify interfering target point cloud pairs among all the stated target point cloud pairs;
[0011] The interfering target point cloud pairs are filtered out from all the target point cloud pairs;
[0012] The step of sending the difference information and the second annotation result to the target terminal when the number of target point cloud pairs is determined to be greater than a first preset number threshold includes:
[0013] If the number of remaining target point cloud pairs after filtering out the interfering target point cloud pairs is greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal.
[0014] Optionally, determining the interfering target point cloud pairs among all the target point cloud pairs includes:
[0015] All target point clouds are clustered to obtain clustering results. The target point clouds are either the first point cloud or the second point cloud. The clustering results include at least one cluster of point clouds, and the distance between the target point clouds in each cluster is less than a preset distance threshold.
[0016] Determine the target cluster in the clustering result, wherein the number of target point clouds in the target cluster is less than a second preset number threshold;
[0017] The target point cloud pairs corresponding to the target point clouds in the target cluster are identified as interfering target point cloud pairs.
[0018] Optionally, the first preset quantity threshold is determined in the following way:
[0019] Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold;
[0020] For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label.
[0021] An intermediate value is determined based on the total number of point clouds and the number of the first sample autonomous driving images;
[0022] The first preset quantity threshold is determined based on the median value and the preset percentage.
[0023] Optionally, the target semantic label includes a sidewalk label and / or a curb label.
[0024] Optionally, the point cloud annotation model is obtained in the following way:
[0025] Obtain a training sample dataset, which includes sample point cloud data corresponding to multiple second sample autonomous driving images and sample annotation results corresponding to each sample point cloud data;
[0026] Based on the training sample dataset, the initial model is trained to obtain a point cloud annotation model.
[0027] Optionally, the method further includes:
[0028] Obtain an updated sample dataset, which includes multiple third annotation results and point cloud data corresponding to each of the third annotation results. The third annotation results are annotation results obtained by manually quality checking the second annotation results based on the difference information.
[0029] Based on the updated sample dataset, the point cloud annotation model is updated to obtain the updated point cloud annotation model. The updated point cloud annotation model is then used to predict the newly acquired point cloud data to obtain the corresponding annotation results.
[0030] According to a second aspect of the present disclosure, a data detection apparatus is provided, comprising:
[0031] The first acquisition module is configured to acquire a first annotation result and a second annotation result of the point cloud data corresponding to the autonomous driving image. The first annotation result is obtained by manually quality-checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data.
[0032] The first determining module is configured to determine a target point cloud pair, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud.
[0033] The sending module is configured to send the difference information and the second annotation result to the target terminal when it is determined that the number of target point cloud pairs is greater than a first preset number threshold. The difference information is used to indicate the position of the second point cloud in the second annotation result.
[0034] Optionally, the device further includes:
[0035] The second determining module is configured to determine interfering target point cloud pairs among all the target point cloud pairs;
[0036] The filtering module is configured to filter out the interfering target point cloud pairs from all the target point cloud pairs;
[0037] The sending module is specifically configured to send the difference information and the second annotation result to the target terminal when the number of remaining target point cloud pairs after filtering out the interfering target point cloud pairs is greater than a first preset number threshold.
[0038] Optionally, the second determining module is specifically configured as follows:
[0039] All target point clouds are clustered to obtain clustering results. The target point clouds are either the first point cloud or the second point cloud. The clustering results include at least one cluster of point clouds, and the distance between the target point clouds in each cluster is less than a preset distance threshold.
[0040] Determine the target cluster in the clustering result, wherein the number of target point clouds in the target cluster is less than a second preset number threshold;
[0041] The target point cloud pairs corresponding to the target point clouds in the target cluster are identified as interfering target point cloud pairs.
[0042] Optionally, the first preset quantity threshold is determined in the following way:
[0043] Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold;
[0044] For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label.
[0045] An intermediate value is determined based on the total number of point clouds and the number of the first sample autonomous driving images;
[0046] The first preset quantity threshold is determined based on the median value and the preset percentage.
[0047] Optionally, the target semantic label includes a sidewalk label and / or a curb label.
[0048] Optionally, the point cloud annotation model is obtained in the following way:
[0049] Obtain a training sample dataset, which includes sample point cloud data corresponding to multiple second sample autonomous driving images and sample annotation results corresponding to each sample point cloud data;
[0050] Based on the training sample dataset, the initial model is trained to obtain a point cloud annotation model.
[0051] Optionally, the device further includes:
[0052] The second acquisition module is configured to acquire an updated sample dataset, which includes multiple third annotation results and point cloud data corresponding to each of the third annotation results. The third annotation results are annotation results obtained by manually quality checking the second annotation results based on the difference information.
[0053] The update module is configured to update the point cloud annotation model based on the updated sample dataset to obtain an updated point cloud annotation model, and then use the updated point cloud annotation model to predict the newly acquired point cloud data to obtain the corresponding annotation results.
[0054] According to a third aspect of the present disclosure, a data detection apparatus is provided, comprising:
[0055] processor;
[0056] Memory used to store processor-executable instructions;
[0057] The processor is configured as follows:
[0058] The first annotation result and the second annotation result of the point cloud data corresponding to the autonomous driving image are obtained. The first annotation result is obtained by manually quality checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data.
[0059] A target point cloud pair is determined, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud.
[0060] If the number of target point cloud pairs is determined to be greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal, and the difference information is used to indicate the position of the second point cloud in the second annotation result.
[0061] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the data detection method provided in the first aspect of the present disclosure.
[0062] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: by comparing the semantic labels of point clouds at the same position in the first annotation result and the second annotation result, target point cloud pairs with the same point cloud position but different semantic labels are obtained. When the number of target point cloud pairs is greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal. The difference information is used to indicate the position of the second point cloud in the second annotation result. In this way, during the process of manual secondary annotation of the second annotation result based on the difference information, the manual can pay more attention to the inconsistent point clouds in the first annotation result after quality inspection and the second annotation result obtained by model inference through the difference information in the target terminal, thereby improving the quality of the annotation data and thus helping to improve the performance of the model trained based on the annotation data.
[0063] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0064] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0065] Figure 1 This is a flowchart illustrating a data detection method according to an exemplary embodiment.
[0066] Figure 2 This is a flowchart illustrating a clustering target point cloud according to an exemplary embodiment.
[0067] Figure 3 This is a block diagram illustrating a data detection device according to an exemplary embodiment.
[0068] Figure 4 This is another block diagram illustrating a data detection device according to an exemplary embodiment.
[0069] Figure 5 This is another block diagram illustrating a data detection device according to an exemplary embodiment. Detailed Implementation
[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0071] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.
[0072] In related technologies, manual quality inspection is used to reduce mislabeling of point cloud data acquired by LiDAR in order to reduce mislabeling. However, manual quality inspection does not cover all labeled data, and the understanding of labeling rules among labelers is not entirely consistent, making it easy to overlook omissions or errors in ambiguous areas such as boundaries. Furthermore, when using multi-frame stitching for labeling, the stitching position may be inaccurate, or moving objects may shift upwards during stitching, leading to inherent quality problems in the submitted labeling data. Even manual quality inspection of this submitted data cannot resolve issues caused by inaccurate stitching positions or upward shifting of moving objects.
[0073] If a model trained on a large number of mislabeled sample data is used to achieve semantic segmentation of the road surface, the semantic segmentation effect of the model will be reduced, thereby increasing the risk of false detection of the road surface and causing erroneous operations such as intermittent braking, which will affect autonomous driving.
[0074] In view of this, the present disclosure provides a data detection method, apparatus and medium, which uses the inference results of the point cloud annotation model to automatically inspect the point cloud data, intuitively reflects the data that is prone to mislabeling or omission, and uses a secondary single-frame annotation method to re-annotate the labeled data, thereby improving the quality of the labeled data.
[0075] Figure 1 This is a flowchart illustrating a data detection method according to an exemplary embodiment, such as... Figure 1 As shown, the data detection method can be used in the vehicle's onboard terminal and includes the following steps.
[0076] In step S11, the first and second annotation results of the point cloud data corresponding to the autonomous driving image are obtained.
[0077] Among them, autonomous driving images are used to describe road scenes. Semantic segmentation of road elements in the road scene is performed, that is, semantic labeling, which can be used for autonomous driving.
[0078] Point cloud data corresponding to autonomous driving images is used to describe road scenes. Point cloud data corresponding to autonomous driving images can be acquired by LiDAR equipment. By performing semantic segmentation on the point cloud data corresponding to autonomous driving images, regions of different road elements in the road scene can be obtained. These road elements can be sidewalks, carriageways, curbs, and others. Carriageways can also be called road surfaces. Sidewalks, carriageways, and curbs are important road elements affecting autonomous driving. Other elements can include trees, etc.
[0079] The first annotation result is obtained by manually quality-checking the second annotation result. The second annotation result is predicted from the point cloud data using a pre-trained point cloud annotation model. Both the first and second annotation results are used to reflect the semantic labels of each point in the point cloud data. Here, the semantic labels are used to describe road elements, and the semantic labels corresponding to road elements can be sidewalk labels, carriageway labels, curb labels, or other labels, etc.
[0080] In some examples, bounding boxes can be used to label point cloud data. Since there are many consecutive points with the same semantic label in point cloud data, bounding boxes of the same color can be used to label consecutive points with the same semantic label, and the color of the bounding boxes used for different semantic labels can be different, thereby improving the readability of the labeled data.
[0081] In step S12, a target point cloud pair is determined. The first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair. The point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud.
[0082] As mentioned above, the first annotation result was obtained by manually quality-checking the second annotation result. Therefore, the first and second annotation results are data from the same point cloud data. For each point cloud in the first annotation result, there is a point cloud at the same location in the second annotation result. The difference is that the semantic labels of the point clouds at the same location in the first and second annotation results may be the same or different.
[0083] In step S13, if the number of target point cloud pairs is determined to be greater than the first preset number threshold, the difference information and the second annotation result are sent to the target terminal. The difference information is used to indicate the position of the second point cloud in the second annotation result.
[0084] It is worth noting that if the number of target point cloud pairs exceeds the first preset threshold, it indicates that there is a significant difference between the inference results of the point cloud annotation model and the results after manual quality inspection. This suggests that there may be errors in the annotation data, meaning that the data after manual quality inspection may contain mislabeled or missing labels. This situation may be caused by the fact that manual quality inspection does not cover all labeled sample data, and that the quality inspectors may not have a completely consistent understanding of the annotation rules. Therefore, it is necessary to verify the annotation data from manual quality inspection.
[0085] Conversely, if the number of target point cloud pairs is less than or equal to the first preset threshold, it is not necessary to send difference information and second annotation results. That is, there is no need to manually re-inspect the second annotation results, thus reducing labor costs and the resources occupied by sending information.
[0086] In one possible manner, the first preset quantity threshold can be determined by: acquiring M first sample autonomous driving images; for each first sample autonomous driving image, determining the number of point clouds in the point cloud data corresponding to that first sample autonomous driving image that belong to the target semantic label; determining an intermediate value based on the determined number of all point clouds and the number of first sample autonomous driving images; and determining the first preset quantity threshold based on the intermediate value and a preset proportion.
[0087] Where M is greater than or equal to the third preset quantity threshold, the third preset quantity threshold is a positive integer, and the third preset quantity threshold can be, for example, 100000.
[0088] The target semantic label includes a sidewalk label and / or a curb label. That is, the target semantic label may include a sidewalk label and a curb label; the target semantic label may include a sidewalk label; the target semantic label includes a curb label.
[0089] It's worth noting that in the point cloud data corresponding to autonomous driving images, the number of point clouds corresponding to sidewalks and curbs is relatively small, and there are even cases where point clouds corresponding to sidewalks and curbs do not exist in the autonomous driving images. Therefore, due to the small number of point clouds corresponding to sidewalks and curbs, the learning effect of the point cloud annotation model may be poor. That is, the point cloud annotation model is prone to mislabeling the semantic labels of point clouds corresponding to sidewalks and curbs, resulting in a high probability of inconsistency between the inference results of the point cloud annotation model and the annotation results that have undergone manual quality inspection. On the other hand, point clouds with other semantic labels are more numerous, and the point cloud annotation model performs better for point clouds with these semantic labels. Therefore, the average value of point clouds corresponding to sidewalks and curbs in a large number of point cloud data corresponding to autonomous driving images can be statistically analyzed as an intermediate value. Based on this, considering that a sufficiently small number of mislabeled point clouds will not affect the subsequent annotation results of the point cloud annotation model used by the vehicle, and that the probability of all point clouds corresponding to sidewalks and curbs being mislabeled is low, the product of this average value and the preset proportion can be used as the first preset quantity threshold, thereby rationally setting the first preset quantity threshold. As an example, the default percentage could be 20%.
[0090] The above scheme compares the semantic labels of point clouds at the same location in the first and second annotation results to obtain target point cloud pairs with the same point cloud location but different semantic labels. When the number of target point cloud pairs exceeds a first preset threshold, the difference information and the second annotation result are sent to the target terminal. The difference information is used to indicate the position of the second point cloud in the second annotation result. In this way, during the process of manual secondary annotation of the second annotation result based on the difference information, the manual person can pay more attention to the inconsistent point clouds in the first annotation result after quality inspection and the second annotation result obtained by model inference through the difference information in the target terminal, thereby improving the quality of the annotated data and thus improving the performance of the model trained based on the annotated data.
[0091] In some embodiments, the method further includes: identifying interfering target point cloud pairs among all target point cloud pairs; and filtering out the interfering target point cloud pairs from all target point cloud pairs. In this case, the step of sending the difference information and the second annotation result to the target terminal when the number of target point cloud pairs is greater than a first preset number threshold can be implemented as follows: when the number of remaining target point cloud pairs after filtering out the interfering target point cloud pairs is greater than the first preset number threshold, the difference information and the second annotation result are sent to the target terminal.
[0092] It's worth noting that, taking the first point cloud A in target point cloud pair A as an example, if the total number of first point clouds belonging to other target point cloud pairs within a preset range of the first point cloud A does not exceed a certain threshold, then the first point cloud A can be called a sparse point cloud. In practical autonomous driving applications, incorrect labeling of sparse point clouds does not affect autonomous driving. Therefore, the target point cloud pair corresponding to a sparse point cloud can be called an interfering target point cloud pair. Conversely, if the total number of first point clouds belonging to other target point cloud pairs within a preset range of the first point cloud A exceeds a certain threshold, it indicates the presence of relatively dense and inconsistent point clouds. Only when this situation occurs can incorrect labeling occur because manual quality inspection may not cover all labeled sample data, and quality inspectors may not have a completely consistent understanding of the labeling rules.
[0093] Therefore, it is necessary to filter out interfering target point cloud pairs from all target point cloud pairs. Then, based on the relationship between the number of remaining target point cloud pairs after filtering out interfering target point cloud pairs and the first preset number threshold, it is determined whether to send the difference information and the second annotation result to the target terminal, thereby improving the ability to judge whether the first annotation result belongs to erroneous annotation data.
[0094] In some embodiments, the step of determining the interfering target point cloud pairs among all target point cloud pairs described above can be implemented in the following manner: clustering all target point clouds to obtain clustering results; determining the target clusters in the clustering results, wherein the number of target point clouds in the target clusters is less than a second preset number threshold; and determining the target point cloud pairs corresponding to the target point clouds in the target clusters as interfering target point cloud pairs.
[0095] The target point cloud can be either a first point cloud or a second point cloud. That is, all first point clouds can be clustered to obtain the corresponding clustering results, and all second point clouds can be clustered to obtain the corresponding clustering results.
[0096] The clustering results include at least one cluster of point clouds, where the distance between the target point clouds in each cluster is less than a preset distance threshold; that is, the target point clouds are clustered according to distance. For example, refer to... Figure 2 The flowchart illustrates a clustering process for target point clouds. Target point cloud pairs are determined based on first and second annotation results, thus obtaining the target point cloud. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to cluster the target point cloud. DBSCAN defines a cluster as the largest set of density-connected points. By grouping regions with sufficiently high density into clusters, target point clouds whose distances to each other are less than a preset distance threshold can be grouped into the same cluster. Based on the clustering results, clusters with a number of target point clouds less than a second preset threshold are identified and designated as target clusters.
[0097] Using the above method, target clusters are obtained through clustering. The point cloud in the target cluster can be understood as the sparse point cloud mentioned above. Based on the target point cloud in the target cluster, the corresponding interference target point cloud pairs can be found.
[0098] In some embodiments, the point cloud annotation model is trained by: obtaining a training sample dataset, which includes sample point cloud data corresponding to multiple second sample autonomous driving images and sample annotation results corresponding to each sample point cloud data; and training the initial model based on the training sample dataset to obtain the point cloud annotation model.
[0099] The initial model here could be, for example, an FCN (Fully Convolutional Network).
[0100] The process involves iteratively training the initial model using the training sample dataset until a stopping condition is met. The initial model that meets this stopping condition is the point cloud annotation model. The stopping condition could be, for example, reaching a preset number of iterations.
[0101] In some embodiments, the method further includes: acquiring an updated sample dataset, the updated sample dataset including multiple third annotation results and point cloud data corresponding to each third annotation result, the third annotation results being annotation results obtained by manual quality inspection of the second annotation results based on difference information; updating the point cloud annotation model according to the updated sample dataset to obtain an updated point cloud annotation model; and using the updated point cloud annotation model to predict the newly acquired point cloud data to obtain the corresponding annotation results.
[0102] It is understandable that since the third annotation result is obtained by manually checking the second annotation result based on the difference information, the quality of the third annotation result is higher than that of the second annotation result in terms of data quality. Therefore, the point cloud annotation model can be updated using the third annotation result with higher data quality, thereby obtaining a model with better semantic segmentation effect. In this way, using the updated point cloud annotation model to predict newly acquired point cloud data can yield more ideal annotation results. Thus, when comparing the first and second annotation results in the next time, it is more accurate to filter out mislabeled data in the point cloud data, forming positive feedback.
[0103] Furthermore, in autonomous driving scenarios, when using the updated point cloud annotation model to perform semantic segmentation on the acquired autonomous driving images, the segmentation results are more consistent with the ideal annotation results, reducing erroneous operations such as point braking, thereby improving the autonomous driving experience.
[0104] Figure 3 This is a block diagram illustrating a data detection device according to an exemplary embodiment. (Refer to...) Figure 3 The device 300 includes a first acquisition module 301, a first determination module 302, and a transmission module 303.
[0105] The first acquisition module 301 is configured to acquire a first annotation result and a second annotation result of the point cloud data corresponding to the autonomous driving image. The first annotation result is obtained by manually inspecting the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data.
[0106] The first determining module 302 is configured to determine a target point cloud pair, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as the point cloud position of the second point cloud, and the semantic label of the first point cloud is different from the semantic label of the second point cloud.
[0107] The sending module 303 is configured to send the difference information and the second annotation result to the target terminal when it is determined that the number of target point cloud pairs is greater than a first preset number threshold. The difference information is used to indicate the position of the second point cloud in the second annotation result.
[0108] Optionally, the device 300 further includes:
[0109] The second determining module is configured to determine interfering target point cloud pairs among all the target point cloud pairs;
[0110] The filtering module is configured to filter out the interfering target point cloud pairs from all the target point cloud pairs;
[0111] The sending module 303 is specifically configured to send the difference information and the second annotation result to the target terminal when the number of remaining target point cloud pairs after filtering out the interfering target point cloud pairs is greater than a first preset number threshold.
[0112] Optionally, the second determining module is specifically configured as follows:
[0113] All target point clouds are clustered to obtain clustering results. The target point clouds are either the first point cloud or the second point cloud. The clustering results include at least one cluster of point clouds, and the distance between the target point clouds in each cluster is less than a preset distance threshold.
[0114] Determine the target cluster in the clustering result, wherein the number of target point clouds in the target cluster is less than a second preset number threshold;
[0115] The target point cloud pairs corresponding to the target point clouds in the target cluster are identified as interfering target point cloud pairs.
[0116] Optionally, the first preset quantity threshold is determined in the following way:
[0117] Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold;
[0118] For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label.
[0119] An intermediate value is determined based on the total number of point clouds and the number of the first sample autonomous driving images;
[0120] The first preset quantity threshold is determined based on the median value and the preset percentage.
[0121] Optionally, the target semantic label includes a sidewalk label and / or a curb label.
[0122] Optionally, the point cloud annotation model is obtained in the following way:
[0123] Obtain a training sample dataset, which includes sample point cloud data corresponding to multiple second sample autonomous driving images and sample annotation results corresponding to each sample point cloud data;
[0124] Based on the training sample dataset, the initial model is trained to obtain a point cloud annotation model.
[0125] Optionally, the device 300 further includes:
[0126] The second acquisition module is configured to acquire an updated sample dataset, which includes multiple third annotation results and point cloud data corresponding to each of the third annotation results. The third annotation results are annotation results obtained by manually quality checking the second annotation results based on the difference information.
[0127] The update module is configured to update the point cloud annotation model based on the updated sample dataset to obtain an updated point cloud annotation model, and then use the updated point cloud annotation model to predict the newly acquired point cloud data to obtain the corresponding annotation results.
[0128] Regarding the apparatus 300 in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the data detection method, and will not be elaborated here.
[0129] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the data detection method provided in this disclosure.
[0130] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the data detection method provided in the first aspect of this disclosure.
[0131] This disclosure also provides a data detection device, including:
[0132] processor;
[0133] Memory used to store processor-executable instructions;
[0134] The processor is configured to: acquire a first annotation result and a second annotation result of point cloud data corresponding to an autonomous driving image, wherein the first annotation result is obtained by manual quality inspection of the second annotation result, and the second annotation result is obtained by predicting the point cloud data using a pre-trained point cloud annotation model, and both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data; determine a target point cloud pair, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, wherein the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic labels of the first point cloud and the second point cloud are different; and, if the number of target point cloud pairs is greater than a first preset number threshold, send difference information and the second annotation result to a target terminal, wherein the difference information is used to indicate the position of the second point cloud in the second annotation result.
[0135] Figure 4 This is another block diagram illustrating a data detection device according to an exemplary embodiment. For example, device 400 may be a vehicle, a computer, etc.
[0136] Reference Figure 4 The device 400 may include one or more of the following components: a first processing component 402, a first memory 404, a first power supply component 406, a multimedia component 408, an audio component 410, a first input / output interface 412, a sensor component 414, and a communication component 416.
[0137] The first processing component 402 typically controls the overall operation of the device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. The first processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the data detection method described above. Furthermore, the first processing component 402 may include one or more modules to facilitate interaction between the first processing component 402 and other components. For example, the first processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the first processing component 402.
[0138] The first memory 404 is configured to store various types of data to support the operation of the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phonebook data, messages, pictures, videos, etc. The first memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0139] The first power supply component 406 provides power to the various components of the device 400. The first power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 400.
[0140] Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0141] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.
[0142] The first input / output interface 412 provides an interface between the first processing component 402 and the peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.
[0143] Sensor assembly 414 includes one or more sensors for providing status assessments of various aspects of device 400. For example, sensor assembly 414 may detect the on / off state of device 400, the relative positioning of components such as the display and keypad of device 400, changes in the position of device 400 or a component of device 400, the presence or absence of user contact with device 400, the orientation or acceleration / deceleration of device 400, and temperature changes of device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0144] Communication component 416 is configured to facilitate wired or wireless communication between device 400 and other devices. Device 400 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0145] In an exemplary embodiment, the apparatus 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data detection method described above.
[0146] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 404 including instructions that can be executed by the processor 420 of the device 400 to complete the data detection method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0147] Figure 5 This is another block diagram illustrating a data detection apparatus according to an exemplary embodiment. For example, apparatus 500 may be provided as a server. (Refer to...) Figure 5 The apparatus 500 includes a second processing component 522, which further includes one or more processors, and memory resources represented by a second memory 532 for storing instructions, such as application programs, that can be executed by the second processing component 522. The application programs stored in the second memory 532 may include one or more modules, each corresponding to a set of instructions. Furthermore, the second processing component 522 is configured to execute instructions to perform the aforementioned data detection method.
[0148] The device 500 may also include a second power supply component 526 configured to perform power management of the device 500, a wired or wireless network interface 550 configured to connect the device 500 to a network, and a second input / output interface 558. The device 500 can operate on an operating system, such as Windows Server, stored in a second memory 532. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.
[0149] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0150] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A data detection method, characterized in that, include: The first annotation result and the second annotation result of the point cloud data corresponding to the autonomous driving image are obtained. The first annotation result is obtained by manually quality checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data. A target point cloud pair is determined, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud. If the number of target point cloud pairs is determined to be greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal, wherein the difference information is used to indicate the position of the second point cloud in the second annotation result; The first preset quantity threshold is determined in the following way: Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold; For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label. Based on the determined number of all point clouds and the number of the first sample autonomous driving images, calculate the mean of the point cloud of the target semantic label and determine the median value. The first preset quantity threshold is determined based on the median value and the preset percentage.
2. The method according to claim 1, characterized in that, The method further includes: Identify interfering target point cloud pairs among all the stated target point cloud pairs; The interfering target point cloud pairs are filtered out from all the target point cloud pairs; The step of sending the difference information and the second annotation result to the target terminal when the number of target point cloud pairs is determined to be greater than a first preset number threshold includes: If the number of remaining target point cloud pairs after filtering out the interfering target point cloud pairs is greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal.
3. The method according to claim 2, characterized in that, The step of determining the interfering target point cloud pairs among all the target point cloud pairs includes: All target point clouds are clustered to obtain clustering results. The target point clouds are either the first point cloud or the second point cloud. The clustering results include at least one cluster of point clouds, and the distance between the target point clouds in each cluster is less than a preset distance threshold. Determine the target cluster in the clustering result, wherein the number of target point clouds in the target cluster is less than a second preset number threshold; The target point cloud pairs corresponding to the target point clouds in the target cluster are identified as interfering target point cloud pairs.
4. The method according to claim 1, characterized in that, The target semantic labels include sidewalk labels and / or curb labels.
5. The method according to claim 1, characterized in that, The point cloud annotation model was obtained through the following method: Obtain a training sample dataset, which includes sample point cloud data corresponding to multiple second sample autonomous driving images and sample annotation results corresponding to each sample point cloud data; Based on the training sample dataset, the initial model is trained to obtain a point cloud annotation model.
6. The method according to claim 5, characterized in that, The method further includes: Obtain an updated sample dataset, which includes multiple third annotation results and point cloud data corresponding to each of the third annotation results. The third annotation results are annotation results obtained by manually quality checking the second annotation results based on the difference information. Based on the updated sample dataset, the point cloud annotation model is updated to obtain the updated point cloud annotation model. The updated point cloud annotation model is then used to predict the newly acquired point cloud data to obtain the corresponding annotation results.
7. A data detection device, characterized in that, include: The first acquisition module is configured to acquire a first annotation result and a second annotation result of the point cloud data corresponding to the autonomous driving image. The first annotation result is obtained by manually quality-checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data. The first determining module is configured to determine a target point cloud pair, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud. The sending module is configured to send the difference information and the second annotation result to the target terminal when it is determined that the number of target point cloud pairs is greater than a first preset number threshold. The difference information is used to indicate the position of the second point cloud in the second annotation result. The first preset number threshold is determined in the following way: Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold; For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label. Based on the determined number of all point clouds and the number of the first sample autonomous driving images, calculate the mean of the point cloud of the target semantic label and determine the median value. The first preset quantity threshold is determined based on the median value and the preset percentage.
8. A data detection device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: The first annotation result and the second annotation result of the point cloud data corresponding to the autonomous driving image are obtained. The first annotation result is obtained by manually quality checking the second annotation result. The second annotation result is obtained by predicting the point cloud data through a pre-trained point cloud annotation model. Both the first annotation result and the second annotation result are used to reflect the semantic labels of each point cloud in the point cloud data. A target point cloud pair is determined, wherein the first point cloud in the first annotation result and the second point cloud in the second annotation result constitute the target point cloud pair, the point cloud position of the first point cloud is the same as that of the second point cloud, and the semantic label of the first point cloud is different from that of the second point cloud. If the number of target point cloud pairs is determined to be greater than a first preset number threshold, the difference information and the second annotation result are sent to the target terminal. The difference information is used to indicate the position of the second point cloud in the second annotation result. The first preset number threshold is determined in the following way: Acquire M first sample autonomous driving images, where M is greater than or equal to a third preset quantity threshold; For each of the first sample autonomous driving images, determine the number of point clouds in the point cloud data corresponding to the first sample autonomous driving image that belong to the target semantic label. Based on the determined number of all point clouds and the number of the first sample autonomous driving images, calculate the mean of the point cloud of the target semantic label and determine the median value. The first preset quantity threshold is determined based on the median value and the preset percentage.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 6.