Four-dimensional lane line marking method and device, electronic equipment and storage medium

Abstract

The invention provides a four-dimensional lane line marking method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence such as computer vision, deep learning and automatic driving. The method comprises the following steps: acquiring four-dimensional point cloud data to be labeled; obtaining a two-dimensional top view corresponding to the four-dimensional point cloud data; obtaining a two-dimensional lane line marking result of the two-dimensional top view; and generating a four-dimensional lane line marking result of the four-dimensional point cloud data according to the two-dimensional lane line marking result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to a four-dimensional lane line annotation method, apparatus, electronic device, and storage medium in the fields of computer vision, deep learning, and autonomous driving, etc. Background Art

[0002] With the development of technology, autonomous driving vehicles have been more and more widely used. In order to improve the path planning ability of autonomous driving vehicles, four-dimensional (4D) point cloud data can be annotated with 4D lane lines, and the autonomous driving perception model can be trained according to the annotation results, so as to use the trained autonomous driving perception model to sense the road structure and then perform path planning, etc. Summary of the Invention

[0003] The present disclosure provides a four-dimensional lane line annotation method, apparatus, electronic device, and storage medium.

[0004] A four-dimensional lane line annotation method includes:

[0005] Obtaining four-dimensional point cloud data to be annotated;

[0006] Obtaining a two-dimensional top view corresponding to the four-dimensional point cloud data;

[0007] Obtaining a two-dimensional lane line annotation result of the two-dimensional top view;

[0008] Generating a four-dimensional lane line annotation result of the four-dimensional point cloud data according to the two-dimensional lane line annotation result.

[0009] A four-dimensional lane line annotation apparatus includes: a first obtaining module, a second obtaining module, a third obtaining module, and a result generating module;

[0010] The first obtaining module is configured to obtain four-dimensional point cloud data to be annotated;

[0011] The second obtaining module is configured to obtain a two-dimensional top view corresponding to the four-dimensional point cloud data;

[0012] The third obtaining module is configured to obtain a two-dimensional lane line annotation result of the two-dimensional top view;

[0013] The result generating module is configured to generate a four-dimensional lane line annotation result of the four-dimensional point cloud data according to the two-dimensional lane line annotation result.

[0014] An electronic device includes:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above.

[0018] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method as described above.

[0019] A computer program product comprising a computer program / instructions which, when executed by a processor, implement the method as described above.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0022] Figure 1 is a flowchart of an embodiment of the 4D lane line annotation method described in the present disclosure;

[0023] Figure 2 is a flowchart of a first embodiment of the method for generating a 2D top view described in the present disclosure;

[0024] Figure 3 is a flowchart of a second embodiment of the method for generating a 2D top view described in the present disclosure;

[0025] Figure 4 is a schematic diagram of the annotation and quality inspection process described in the present disclosure;

[0026] Figure 5 is a schematic diagram of the composition structure of Embodiment 500 of the 4D lane line annotation device described in the present disclosure;

[0027] Figure 6 Shows a schematic block diagram of an electronic device 600 that can be used to implement the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following describes exemplary embodiments of the present disclosure in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist in understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] In addition, it should be understood that the term "and / or" in this text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this text generally indicates that the associated objects before and after are in an "or" relationship.

[0030] Figure 1 This is a flowchart of an embodiment of the 4D lane line annotation method described in this disclosure. As Figure 1 shown, it includes the following specific implementation manners.

[0031] In step 101, obtain the 4D point cloud data to be annotated.

[0032] In step 102, obtain the two-dimensional (2D) top view corresponding to the 4D point cloud data.

[0033] In step 103, obtain the 2D lane line annotation result of the 2D top view.

[0034] In step 104, generate the 4D lane line annotation result of the 4D point cloud data according to the 2D lane line annotation result.

[0035] In the traditional method, usually, annotators directly perform 4D lane line annotation on the 4D point cloud data based on annotation tools, which has a high annotation difficulty and low efficiency.

[0036] However, by adopting the solution described in the above method embodiment, for the 4D point cloud data to be annotated, its corresponding 2D top view can be obtained first, and then the 2D lane line annotation result of the 2D top view can be obtained. For example, the manually annotated 2D lane line annotation result can be obtained. Furthermore, the required 4D lane line annotation result can be generated according to the 2D lane line annotation result. Compared with directly annotating the 4D point cloud data, annotating the 2D top view is simpler and more convenient, thus reducing the annotation difficulty and simplifying the annotation process, and further reducing the annotation time and improving the annotation efficiency, etc.

[0037] Point cloud data refers to the data collected by a collection vehicle when driving on the road using a lidar, etc. Each point in it has three-dimensional (xyz) coordinates, where x is the horizontal axis coordinate, y is the vertical axis coordinate, and z is the vertical axis coordinate. The vertical axis coordinate can reflect the height of the point. On this basis, 4D point cloud data can be obtained by processing such as splicing the collection results at different time points. Each point in the 4D point cloud data can include, in addition to the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate, a time (t) coordinate.

[0038] For the 4D point cloud data to be annotated, its corresponding 2D top view can be obtained first. For example, the following method 1 or method 2 can be used to obtain the 2D top view.

[0039] 1) Method 1

[0040] In some embodiments of the present disclosure, each point in the 4D point cloud data can be clustered to obtain different point clusters and their corresponding categories respectively, and the points in the point cluster corresponding to the ground point category can be determined as ground points, and then a 2D top view can be generated based on the ground points.

[0041] For example, the patchwork++ algorithm can be extended to cluster each point in the 4D point cloud data, so as to obtain different point clusters and their corresponding categories respectively. Suppose a total of 3 point clusters are obtained. For the sake of description, they are respectively called point cluster A, point cluster B, and point cluster C. And suppose the category corresponding to point cluster B is the ground point category, then the points in point cluster B can be determined as ground points, and a 2D top view can be generated based on each ground point, such as stitching to generate a 2D top view.

[0042] By adopting the above method, the ground points in the 4D point cloud data can be simply and conveniently determined through clustering, thereby improving the generation efficiency of the 2D top view, etc.

[0043] In addition, in some embodiments of the present disclosure, in addition to directly generating a 2D top view based on the ground points, the adjacent points of the ground points can also be determined from the 4D point cloud data, and then a 2D top view can be generated based on the ground points and the adjacent points.

[0044] That is to say, in addition to the ground points, the adjacent points of the ground points can be selected from the 4D point cloud data, and the adjacent points can be combined to generate a 2D top view, so as to avoid problems such as the road edge being blocked by miscellaneous points as much as possible, thereby improving the generation quality of the 2D top view, and correspondingly improving the accuracy of the subsequent lane line annotation results, etc.

[0045] In some embodiments of the present disclosure, the ground plane height corresponding to the 4D point cloud data can be obtained, and the relative height between each point in the 4D point cloud data and the ground plane height can be determined respectively, and then the points whose relative height is within a predetermined height range and does not belong to the ground points can be determined as the adjacent points.

[0046] The specific value of the predetermined height range can be determined according to actual needs, such as -1 meter to 1 meter. There is no limitation on how to obtain the ground plane height, and it can also be determined according to actual needs. For example, the ground plane height can be determined based on the weighted average height of each point in the 4D point cloud data, etc. The points in the 4D point cloud data whose relative height is within the predetermined height range and does not belong to the ground points can be determined as the required adjacent points, and then a 2D top view can be generated based on the ground points and the adjacent points.

[0047] Combined with the above introduction, Figure 2Flowchart of the first method embodiment for generating a 2D top view according to the present disclosure. As Figure 2 shown, it includes the following specific implementation manners.

[0048] In step 201, each point in the 4D point cloud data is clustered to obtain different point clusters and their corresponding categories respectively.

[0049] In step 202, the points in the point cluster with the corresponding category of ground point category are determined as ground points.

[0050] In step 203, neighboring points are determined from the 4D point cloud data.

[0051] For example, the ground plane height corresponding to the 4D point cloud data can be obtained, and the relative height between each point in the 4D point cloud data and the ground plane height can be determined respectively. Furthermore, the points with the relative height within a predetermined height range and not belonging to the ground points can be determined as neighboring points.

[0052] In step 204, a 2D top view is generated according to the determined ground points and neighboring points.

[0053] 2) Method 2

[0054] In some embodiments of the present disclosure, the ground points in the 4D point cloud data can be determined according to the target points selected from the 4D point cloud data, and a 2D top view can be generated according to the ground points.

[0055] The target points may include: M target points selected according to a predetermined rule, where M is a positive integer greater than 1, and the specific values of the predetermined rule and M can be determined according to actual needs.

[0056] The target points can be determined by manual selection. For example, assuming that the 4D point cloud data corresponds to a length of 1000 meters in total, then a target point can be selected every 10 meters.

[0057] In some embodiments of the present disclosure, for each target point, the following processing can be performed respectively: the points with a planar distance less than the first threshold from the target point are determined from the 4D point cloud data as candidate points, and a first point set is formed by using the candidate points. The ground point corresponding to the target point is determined from the first point set, and then the ground points corresponding to each target point are determined as the ground points in the 4D point cloud data. The specific value of the first threshold can be determined according to actual needs.

[0058] Suppose there are a total of 100 target points, namely target point 1 to target point 100. Taking target point 1 as an example, points with a plane distance less than the first threshold from target point 1 can be determined from the 4D point cloud data as candidate points. If the first threshold is regarded as the plane radius, it means determining candidate points with a plane distance less than the plane radius from target point 1 in the 4D point cloud data. Accordingly, each candidate point is a point within the same cylindrical space, and the bottom radius of the cylindrical space is the plane radius. Then, the first point set can be composed of each candidate point, and the ground point can be determined from each candidate point in the first point set as the ground point corresponding to target point 1. For target points 2 to 100, the corresponding ground points can be determined respectively in the same way as above. Further, the ground points corresponding to these 100 target points can be used as the ground points determined from the 4D point cloud data.

[0059] In the above processing method, a local ground segmentation strategy is adopted, and the corresponding ground points can be determined respectively for different target points. Then, the ground points corresponding to each target point can be determined as the ground points in the 4D point cloud data, so as to meet the usage requirements of different scenarios, etc.

[0060] In some embodiments of the present disclosure, the method for determining the ground point corresponding to any target point from the first point set may include: according to the value range of the vertical axis coordinates, adding each candidate point in the first point set to the corresponding bin respectively. Different bins correspond to their respective value ranges, and there is no overlap between the value ranges corresponding to any two bins. Determine the target bin from each bin, and determine the ground point corresponding to the target point from the candidate points included in the target bin.

[0061] The number of bins and the specific value range corresponding to each bin can be determined according to actual needs.

[0062] For example, the maximum value and the minimum value of the vertical axis coordinates of each candidate point in the first point set can be obtained, and the difference between the maximum value and the minimum value can be obtained. Then, the value range corresponding to each bin can be determined according to the preset bin height (height_bin_size). For example, if the difference is 80 and the bin height is 20, then 4 bins can be set, and the corresponding value ranges are 0 to 20, 21 to 40, 41 to 60, and 61 to 80 respectively. It should be noted that the content here is only for illustrative purposes and does not limit the technical solution of the present disclosure. In practical applications, the bin heights of different bins can be the same or different. Then, according to the value range of the vertical axis coordinates of each candidate point in the first point set, each candidate point can be added to the corresponding bin respectively. Further, one bin can be selected from the 4 bins as the target bin, and the ground point corresponding to the target point can be determined from the candidate points included in the target bin.

[0063] It can be seen that by adopting the above processing method, the rough screening (i.e., primary screening) of the candidate points in the first point set can be realized by means of bucketing. Then, the final required ground points can be further screened out from the rough screening results (i.e., the candidate points in the target bucket). Through the hierarchical screening method, the processing efficiency is improved, etc.

[0064] In some embodiments of the present disclosure, when determining the target bucket from each bucket, the following method a) or method b) can be adopted.

[0065] Method a): Count the number of candidate points included in each bucket respectively, and determine the bucket with the largest number of candidate points as the target bucket.

[0066] Method b): Sort each bucket in ascending order according to the corresponding values, and each bucket can be traversed in the sorting order from front to back. Among them, for each traversed bucket, the following processing is performed respectively: determine the traversed bucket as the bucket to be processed, and in response to determining that the termination condition is met according to the number of candidate points included in the bucket to be processed, determine the bucket to be processed as the target bucket and end the traversal. Otherwise, continue to traverse the next bucket.

[0067] Specifically, which of the above methods is adopted to determine the target bucket can be determined according to actual needs, which is very flexible and convenient. For example, if it is determined that the 4D point cloud data only includes normal points, then method a) can be adopted. If it is determined that the 4D point cloud data also includes abnormal points, such as underground reflection points, etc., then method b) can be adopted.

[0068] Suppose there are a total of 4 buckets. If method a) is adopted, then the number of candidate points included in the 4 buckets can be obtained respectively. Then, the bucket with the largest number of candidate points can be selected and determined as the target bucket. By selecting the bucket with the largest number of candidate points, the omission of ground points can be reduced, etc. If method b) is adopted, assuming that after sorting, the buckets are bucket 1, bucket 2, bucket 3, and bucket 4 in order from front to back, then bucket 1, bucket 2, bucket 3, and bucket 4 can be traversed in turn. Suppose that when traversing bucket 3, it is determined that the termination condition is met, then the traversal can be ended, and bucket 3 can be determined as the target bucket. Otherwise, bucket 4 can be continued to be traversed.

[0069] In some embodiments of the present disclosure, determining that the termination condition is met may include: obtaining a first ratio, where the first ratio is the ratio of the number of candidate points included in the bucket to be processed to the number of candidate points included in the first point set. In response to determining that the first ratio is greater than a second threshold, it may be determined that the termination condition is met. Alternatively, obtain a second ratio, where the second ratio is the ratio of the number of candidate points included in the bucket to be processed to the number of candidate points included in the current largest bucket, and the current largest bucket is the bucket with the largest number of candidate points among the traversed buckets. In response to determining that the second ratio is less than a third threshold, it may be determined that the termination condition is met. The specific values of the second threshold and the third threshold can both be determined according to actual needs.

[0070] Taking the bucket to be processed (i.e., the traversed bucket) as the above-mentioned bucket 3 as an example, the ratio of the number of candidate points included in bucket 3 to the number of candidate points included in the first point set can be obtained as the first ratio. If the first ratio is greater than the second threshold, it can be determined that the termination condition is met. Alternatively, the current largest bucket can be selected from the traversed buckets (i.e., bucket 1 and bucket 2), that is, the bucket with the largest number of candidate points among bucket 1 and bucket 2, and the ratio of the number of candidate points included in bucket 3 to the number of candidate points included in the current largest bucket can be obtained as the second ratio. If the second ratio is less than the third threshold, it means that the window (a bucket can be regarded as a window) encounters a peak, and it can also be determined that the termination condition is met. In addition, if the bucket to be processed does not meet the above two situations, but there is no next bucket after the bucket to be processed, then it can also be considered that the termination condition is met.

[0071] By adopting the above processing method, the traversal of each bucket can be terminated in advance when the conditions are met, thereby simplifying the processing flow and improving the processing efficiency, etc.

[0072] For any target point, after determining the target bucket, the ground point corresponding to the target point can also be determined from the candidate points included in the target bucket.

[0073] In some embodiments of the present disclosure, candidate points included in a target bin can be used to form a second point set. Then, N iterative processes can be performed in sequence, where N is a positive integer greater than 1. Each iterative process can respectively include: randomly extracting L candidate points from the second point set, where L is a positive integer greater than 1 and less than the number of candidate points included in the second point set; fitting a linear plane model using the L candidate points, and respectively obtaining the distances between each candidate point in the second point set and the linear plane model; determining candidate points with distances meeting the requirements as ground points, and comparing the linear plane model with a stored reference model. In response to determining that the linear plane model is better than the reference model, updating the reference model to the linear plane model; otherwise, keeping the reference model unchanged; in response to determining that all N iterative processes have been completed, determining the ground points determined when the reference model was updated most recently as the ground points corresponding to the target point. The specific values of N and L can be determined according to actual needs.

[0074] Assume that the value of N is 100. Taking the 10th iterative process as an example, L candidate points can be randomly extracted from the second point set. Then, a linear plane model, such as ax + by + cz + d = 0, can be fitted according to the L candidate points, that is, the model parameters a, b, c, and d are fitted. Furthermore, the distances between each candidate point in the second point set and the linear plane model can be respectively obtained, and candidate points with distances meeting the requirements are determined as ground points. For example, candidate points with distances less than a fourth threshold are determined as ground points. Further, the linear plane model can be compared with a stored reference model (a linear plane model generated during a previous iterative process). If it is determined that the linear plane model is better than the reference model, the reference model can be updated to the linear plane model; otherwise, the original reference model can be kept unchanged. There is no limitation on how to evaluate the superiority of the two models. For example, a comprehensive evaluation parameter can be calculated based on the distances between each candidate point in the second point set and the linear plane model as the comprehensive evaluation parameter corresponding to the linear plane model, and this comprehensive evaluation parameter can be compared with the comprehensive evaluation parameter corresponding to the reference model calculated in a similar manner. If the value of the comprehensive evaluation parameter corresponding to the linear plane model is smaller, then it can be determined that the linear plane model is better than the reference model; otherwise, it can be determined that the reference model is better than the linear plane model. After all 100 iterative processes have been completed, the ground points determined when the reference model was updated most recently can be determined as the ground points corresponding to the target point. For example, assume that the reference model was updated during the 60th iterative process and has not been updated since then. Then, the ground points determined during the 60th iterative process can be used as the ground points corresponding to the target point.

[0075] That is, the Random Sample Consensus (RANSAC) algorithm can be used to determine the ground points corresponding to the target point. Multiple iterative processes are performed throughout the process, thereby improving the accuracy of the processing results, etc.

[0076] In addition, in some embodiments of the present disclosure, in addition to directly generating a 2D top view based on ground points, adjacent points of the ground points can also be determined from the 4D point cloud data, and then a 2D top view can be generated based on the ground points and the adjacent points.

[0077] That is to say, in addition to ground points, adjacent points of the ground points can be selected from the 4D point cloud data, and the adjacent points can be combined to generate a 2D top view, thus avoiding problems such as road curbs being blocked by miscellaneous points as much as possible, and then improving the generation quality of the 2D top view, and correspondingly improving the accuracy of the subsequent lane line annotation results, etc.

[0078] In some embodiments of the present disclosure, the ground plane height corresponding to the 4D point cloud data can be obtained, and the relative height between each point in the 4D point cloud data and the ground plane height can be determined respectively, and then the points whose relative height is within a predetermined height range and does not belong to the ground points can be determined as the adjacent points.

[0079] Combined with the above introduction, Figure 3 is a flowchart of the second method embodiment for generating a 2D top view according to the present disclosure. As Figure 3 shown, it includes the following specific implementation manners.

[0080] In step 301, M target points selected from the 4D point cloud data are determined.

[0081] After that, for each target point, the processing can be performed respectively in the manner shown in steps 302 to 307.

[0082] In step 302, points whose planar distance from the target point in the 4D point cloud data is less than a first threshold are determined as candidate points, and a first point set is formed by using each candidate point.

[0083] In step 303, according to the value range to which the vertical axis coordinate belongs, each candidate point in the first point set is added to the corresponding bin respectively. Different bins respectively correspond to their own value ranges, and there is no overlap between the value ranges corresponding to any two bins.

[0084] In step 304, a target bin is determined from each bin.

[0085] For example, the number of candidate points included in each bin can be obtained respectively, and the bin with the largest number of candidate points can be determined as the target bin. Alternatively, the bins can be sorted in ascending order according to the corresponding values, and each bin can be traversed in the sorting order from the front to the back. Among them, for each traversed bin, the following processing can be performed respectively: the traversed bin is determined as the bin to be processed. In response to determining that the termination condition is met according to the number of candidate points included in the bin to be processed, the bin to be processed is determined as the target bin and the traversal ends. Otherwise, continue to traverse the next bin.

[0086] In step 305, the candidate points included in the target bin are used to form a second point set.

[0087] In step 306, N iterative processes are performed in sequence. Each iterative process respectively includes: randomly extracting L candidate points from the second point set, fitting a linear plane model with the L candidate points, respectively obtaining the distances between the candidate points in the second point set and the linear plane model, determining the candidate points with distances meeting the requirements as ground points, comparing the linear plane model with the stored reference model. In response to determining that the linear plane model is better than the reference model, updating the reference model to the linear plane model. Otherwise, keeping the reference model unchanged.

[0088] In step 307, in response to determining that all N iterative processes have been executed, the ground points determined when the reference model was updated most recently are determined as the ground points corresponding to the target points.

[0089] In step 308, the ground points corresponding to each target point are used to form a third point set.

[0090] In step 309, adjacent points are determined from the 4D point cloud data.

[0091] For example, the ground plane height corresponding to the 4D point cloud data can be obtained, and the relative height between each point in the 4D point cloud data and the ground plane height can be determined respectively. Furthermore, the points with relative heights within a predetermined height range and not belonging to the ground points in the third point set can be determined as adjacent points.

[0092] In step 310, a 2D top view is generated based on the ground points and adjacent points in the third point set.

[0093] Generating a 2D top view corresponding to the 4D point cloud data is equivalent to performing a 2D flattening operation on the 4D point cloud data. Further, the 2D lane line annotation result of the 2D top view can also be obtained.

[0094] In some embodiments of the present disclosure, the 2D lane line annotation result may include: the annotation result obtained after the annotator annotates the 2D top view based on the annotation tool.

[0095] The annotator can view the 2D top view in the annotation tool and use various functions provided by the annotation tool to perform 2D lane line annotation on the 2D top view. Specifically, what to annotate can be determined according to actual needs. For example, the line shape of the lane line and the attributes of the lane line (such as dashed or solid lines) can be annotated, and traffic elements such as stop lines, zebra crossings, left turn signs, straight-ahead signs, and right turn signs on the road surface can be annotated. The annotator only needs to ensure that the lane line plane fits, and there is no need to use the functions related to height adjustment.

[0096] Since only 2D flattening annotation is required, the annotation process of the annotator is simplified, the annotation difficulty is reduced, and the annotation efficiency is improved, etc.

[0097] After obtaining the 2D lane line annotation result, the 4D lane line annotation result of the 4D point cloud data can be further generated according to the 2D lane line annotation result.

[0098] In some embodiments of the present disclosure, the method for generating the 4D lane line annotation result according to the 2D lane line annotation result may include: respectively obtaining the vertical axis coordinates of each point in the 2D lane line annotation result, and generating the 4D lane line annotation result according to the vertical axis coordinates.

[0099] Since the 4D point cloud data is converted into a 2D top view, correspondingly, the obtained is the 2D lane line annotation result. Therefore, in order to obtain the 4D lane line annotation result, the 2D lane line annotation result needs to be converted into the 4D space. The points in the 4D space include the horizontal axis coordinate, the vertical axis coordinate, the vertical axis coordinate, and the time coordinate. For each point in the 2D lane line annotation result, when it is converted into the 4D space, the other coordinates can be determined by existing methods, and the vertical axis coordinate can be determined by the method described in the present disclosure.

[0100] In some embodiments of the present disclosure, the following method a) or method b) can be used to determine the vertical axis coordinates of each point in the 2D lane line annotation result.

[0101] Method a): The ground fitting equation can be determined according to the 3D coordinates of each ground point in the 4D point cloud data. The 3D coordinates include the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate. According to the 2D coordinates of each point in the 2D lane line annotation result in the 2D top view and the ground fitting equation, the vertical axis coordinates of each point in the 2D lane line annotation result are determined. The 2D coordinates include the horizontal axis coordinate and the vertical axis coordinate.

[0102] The ground fitting equation refers to the plane equation about x, y, and z, and the equation parameters can be obtained by fitting. For example, according to the 3D coordinates of each ground point in the 2D top view in the 4D point cloud data, the plane normal vector and the centroid can be determined by the principal component analysis (PCA) algorithm. Then, the ground fitting equation can be determined based on the plane normal vector and the centroid. After that, for each point in the 2D lane line annotation result, the 2D coordinates of each point can be input into the ground fitting equation respectively, so as to obtain the output vertical axis coordinates.

[0103] Method b): For each point in the 2D lane line annotation result, the following processing can be performed respectively: taking the point as the point to be processed, determining K nearest neighbor ground points of the point to be processed from the 2D top view through the nearest neighbor algorithm, where K is a positive integer greater than 1, and determining the vertical axis coordinate of the point to be processed according to the vertical axis coordinates of the K nearest neighbor ground points in the 4D point cloud data.

[0104] For example, the K-dimensional tree (KD-Tree, K-Dimensional Tree) + max heap algorithm can be used to determine the K nearest neighbor points of the point to be processed, and these nearest neighbor points need to be ground points. The specific value of K can be determined according to actual needs.

[0105] In addition, in some embodiments of the present disclosure, according to the value range to which the vertical axis coordinates in the 4D point cloud data belong, the K nearest neighbor ground points can be added to the corresponding buckets respectively. Different buckets correspond to their respective value ranges, and there is no overlap between the value ranges corresponding to any two buckets. Then, the buckets can be sorted in ascending order according to the corresponding values, and the buckets can be traversed in order from front to back. Among them, for each bucket traversed, in response to determining that the number of ground points included in the traversed bucket is greater than the sixth threshold, the vertical axis coordinate with the smallest value can be selected from the vertical axis coordinates of the ground points included in the traversed bucket, and the selected vertical axis coordinate can be determined as the vertical axis coordinate of the point to be processed and the traversal can be ended. In response to determining that the number of ground points included in the traversed bucket is less than or equal to the sixth threshold, the next bucket can be continued to be traversed. The specific value of the sixth threshold can be determined according to actual needs.

[0106] The number of buckets and the specific value ranges corresponding to each bucket can be determined according to actual needs. Suppose there are 4 buckets in total, and after sorting, they are bucket 1, bucket 2, bucket 3, and bucket 4 in order from front to back. Suppose when traversing to bucket 3, it is found that the number of ground points included in bucket 3 is greater than the sixth threshold, then the traversal can be ended. In addition, suppose there are 5 ground points included in bucket 3, then the vertical axis coordinate with the smallest value can be selected from the vertical axis coordinates of these 5 ground points, that is, the lowest point in bucket 3 can be selected, and then the vertical axis coordinate with the smallest value can be determined as the vertical axis coordinate of the point to be processed.

[0107] In practical applications, whether to specifically adopt method a) or method b) can be determined according to actual needs, which is very flexible and convenient. For example, if it is determined that the standard deviation of the vertical axis coordinates of each ground point in the 4D point cloud data meets the requirements, method a) can be adopted; otherwise, method b) can be adopted, thereby improving the pertinence of processing and further improving the accuracy of the processing results, etc.

[0108] In some embodiments of the present disclosure, after obtaining the 4D lane line annotation result, a quality inspector can also perform quality inspection on the 4D lane line annotation result. The number of quality inspection times can be one or multiple. For example, an auditor can perform an audit on the 4D lane line annotation result once, and then an acceptor can perform an acceptance on the 4D lane line annotation result once. Both the audit and the acceptance can be regarded as quality inspections on the 4D lane line annotation result.

[0109] Correspondingly, in some embodiments of the present disclosure, in response to determining that the quality inspection performed by the quality inspector on the 4D lane line annotation result fails, and determining that the annotator has modified the 4D lane line annotation result based on the problems detected in the quality inspection, the 4D lane line annotation result can be updated according to the modification.

[0110] In some embodiments of the present disclosure, the quality inspection may include: the quality inspector checks whether the ground fitting degree of each lane line meets the requirements from P different perspectives respectively, where P is a positive integer greater than 1; the failure of the quality inspection may include: there are problems with at least one lane line, and the problems include: the ground fitting degree does not meet the requirements from at least one perspective, and the ground fitting degree not meeting the requirements may mean that the ground fitting degree is less than the fifth threshold. The specific values of the fifth threshold and P can both be determined according to actual needs. In addition, which perspectives the P different perspectives are can also be determined according to actual needs.

[0111] Combined with the above introduction, Figure 4 is a schematic diagram of the annotation and quality inspection process described in the present disclosure. As Figure 4As shown, assuming that the quality inspection personnel include review personnel and acceptance personnel, after obtaining the 2D top view corresponding to the 4D point cloud data, the labeling personnel can perform the first-round labeling in the 2D space. The first-round labeling refers to performing 2D lane line labeling on the 2D top view based on the labeling tool. After that, a spatial transformation can be performed (transforming the 2D lane line labeling result into the 4D space), and then the 4D lane line labeling result of the 4D point cloud data can be generated according to the 2D lane line labeling result. Furthermore, the 4D point cloud data carrying the 4D lane line labeling result can be submitted to the review personnel for review. Correspondingly, the review personnel can review the 4D lane line labeling result, such as switching different perspectives to check whether the ground fitting degree of each lane line meets the requirements. If problems are found, it can be sent back to the labeling personnel for labeling repair, that is, the labeling personnel modify the 4D lane line labeling result according to the problems detected during quality inspection. For example, the height adjustment related function of the labeling tool can be used to modify the problem of the lane line being in contact with the ground. Then, the 4D point cloud data carrying the modified 4D lane line labeling result can be submitted to the review personnel for re-review. If the review is passed, it can be further submitted to the acceptance personnel for acceptance. Correspondingly, the acceptance personnel can accept the 4D lane line labeling result, such as switching different perspectives to check whether the ground fitting degree of each lane line meets the requirements. If problems are found, it can be sent back to the review personnel, and the review personnel can further send it back to the labeling personnel for labeling repair. Correspondingly, the labeling personnel can modify the 4D lane line labeling result according to the problems detected during quality inspection. If both the review and acceptance are passed, the quality inspection process for the 4D point cloud data can be ended.

[0112] By performing quality inspection, it can be ensured that the finally obtained 4D lane line labeling result not only meets the plane fitting requirements but also meets the ground fitting degree requirements, etc., thereby further improving the accuracy of the labeling result.

[0113] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present disclosure. In addition, for the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions in other embodiments.

[0114] The above is the introduction of the method embodiments. The following further illustrates the solution of the present disclosure through device embodiments.

[0115] Figure 5 This is a schematic structural diagram of the composition of the 4D lane line labeling device embodiment 500 described in the present disclosure. As Figure 5As shown in the figure, it includes: a first acquisition module 501, a second acquisition module 502, a third acquisition module 503, and a result generation module 504.

[0116] The first acquisition module 501 is configured to acquire 4D point cloud data to be labeled.

[0117] The second acquisition module 502 is configured to acquire a 2D top view corresponding to the 4D point cloud data.

[0118] The third acquisition module 503 is configured to acquire a 2D lane line annotation result of the 2D top view.

[0119] The result generation module 504 is configured to generate a 4D lane line annotation result of the 4D point cloud data according to the 2D lane line annotation result.

[0120] Adopting the solution described in the above device embodiment, for the 4D point cloud data to be labeled, its corresponding 2D top view can be first acquired, and then the 2D lane line annotation result of the 2D top view can be acquired. For example, the manually annotated 2D lane line annotation result can be acquired. Furthermore, the required 4D lane line annotation result can be generated according to the 2D lane line annotation result. Compared with directly annotating the 4D point cloud data, annotating the 2D top view is simpler and more convenient, thus reducing the annotation difficulty and simplifying the annotation process, and further reducing the annotation time and improving the annotation efficiency, etc.

[0121] In some embodiments of the present disclosure, for the 4D point cloud data to be labeled, the second acquisition module 502 can acquire its corresponding 2D top view, that is, the points in the 4D point cloud data can be clustered to obtain different point clusters and their corresponding categories respectively, and the points in the point cluster corresponding to the ground point category can be determined as ground points. Furthermore, the 2D top view can be generated according to the ground points, or the ground points in the 4D point cloud data can be determined according to the target points selected from the 4D point cloud data, and the 2D top view can be generated according to the ground points.

[0122] Among them, the method of determining ground points according to the clustering process can refer to the relevant description in Method 1 of the method embodiment. For example, the patchwork++ algorithm can be used to cluster the points in the 4D point cloud data, so as to obtain different point clusters and the categories corresponding to each point cluster respectively. Furthermore, the points in the point cluster corresponding to the ground point category can be determined as ground points.

[0123] The method for determining the ground point based on the target point may refer to the relevant description in Method 2 described in the method embodiments. In some embodiments of the present disclosure, the target point may include: M target points selected according to a predetermined rule, where M is a positive integer greater than 1. For each target point, the second acquisition module 502 may respectively perform the following processing: determine points in the 4D point cloud data whose planar distance from the target point is less than a first threshold as candidate points, and use the candidate points to form a first point set, and determine the ground point corresponding to the target point from the first point set, and then the ground points corresponding to each target point may be determined as the ground points in the 4D point cloud data.

[0124] In some embodiments of the present disclosure, the method for the second acquisition module 502 to determine the ground point corresponding to any target point from the first point set may include: adding the candidate points in the first point set to the corresponding bins according to the value range of the vertical axis coordinates, different bins respectively correspond to their own value ranges, and there is no overlap between the value ranges corresponding to any two bins, determine the target bin from each bin, and determine the ground point corresponding to the target point from the candidate points included in the target bin.

[0125] In some embodiments of the present disclosure, when the second acquisition module 502 determines the target bin from each bin, it may respectively count the number of candidate points included in each bin, and then determine the bin with the largest number of candidate points as the target bin, or sort each bin in ascending order according to the corresponding values, and sequentially traverse each bin in the front-to-back sorting order. Among them, for each bin traversed, the following processing may be respectively performed: determine the traversed bin as the bin to be processed, and in response to determining that the termination condition is met according to the number of candidate points included in the bin to be processed, determine the bin to be processed as the target bin and end the traversal, otherwise, continue to traverse the next bin.

[0126] In some embodiments of the present disclosure, the second acquisition module 502 determining that the termination condition is met may include: obtaining a first ratio, where the first ratio is the ratio of the number of candidate points included in the bin to be processed to the number of candidate points included in the first point set, and in response to determining that the first ratio is greater than a second threshold, it may be determined that the termination condition is met, or obtaining a second ratio, where the second ratio is the ratio of the number of candidate points included in the bin to be processed to the number of candidate points included in the currently largest bin, and the currently largest bin is the bin with the largest number of candidate points among the bins that have been traversed, and in response to determining that the second ratio is less than a third threshold, it may be determined that the termination condition is met.

[0127] In some embodiments of the present disclosure, the manner in which the second acquisition module 502 determines the ground point corresponding to the target point from the candidate points in the target bin may include: using the candidate points included in the target bin to form a second point set, and then performing N iterative processes in sequence, where N is a positive integer greater than 1. Each iterative process may respectively include: randomly extracting L candidate points from the second point set, where L is a positive integer greater than 1 and less than the number of candidate points included in the second point set, fitting a linear plane model using the L candidate points, respectively obtaining the distances between each candidate point in the second point set and the linear plane model, determining the candidate points with distances meeting the requirements as ground points, and comparing the linear plane model with the stored reference model. In response to determining that the linear plane model is better than the reference model, updating the reference model to the linear plane model; otherwise, keeping the reference model unchanged; in response to determining that all N iterative processes have been completed, determining the ground point determined when the reference model was last updated as the ground point corresponding to the target point.

[0128] Regardless of the manner in which the ground points are determined, in some embodiments of the present disclosure, in addition to directly generating a 2D top view based on the determined ground points, the second acquisition module 502 may also determine the adjacent points of the ground points from the 4D point cloud data, and then generate a 2D top view based on the ground points and the adjacent points.

[0129] In some embodiments of the present disclosure, the second acquisition module 502 may obtain the ground plane height corresponding to the 4D point cloud data, and may respectively determine the relative height between each point in the 4D point cloud data and the ground plane height, and then may determine the points whose relative height is within a predetermined height range and does not belong to the ground points as the adjacent points.

[0130] Generating a 2D top view corresponding to the 4D point cloud data is equivalent to performing a 2D flattening operation on the 4D point cloud data. Further, the third acquisition module 503 may also obtain the 2D lane line annotation result of the 2D top view.

[0131] In some embodiments of the present disclosure, the 2D lane line annotation result may include: the annotation result obtained after the annotator annotates the 2D top view based on the annotation tool.

[0132] Since the 4D point cloud data is converted into a 2D top view, and correspondingly, a 2D lane line annotation result is obtained, in order to obtain a 4D lane line annotation result, the result generation module 504 also needs to convert the 2D lane line annotation result into the 4D space, that is, generate a 4D lane line annotation result of the 4D point cloud data according to the 2D lane line annotation result.

[0133] In some embodiments of the present disclosure, the manner in which the result generation module 504 generates a 4D lane line annotation result based on the 2D lane line annotation result may include: respectively obtaining the vertical axis coordinates of each point in the 2D lane line annotation result, and generating a 4D lane line annotation result based on the vertical axis coordinates.

[0134] In some embodiments of the present disclosure, the manner in which the result generation module 504 respectively obtains the vertical axis coordinates of each point in the 2D lane line annotation result may include: determining a ground fitting equation according to the 3D coordinates of each ground point in the 4D point cloud data, where the 3D coordinates include the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate, and determining the vertical axis coordinates of each point in the 2D lane line annotation result according to the 2D coordinates of each point in the 2D top view of the 2D lane line annotation result and the ground fitting equation, where the 2D coordinates include the horizontal axis coordinate and the vertical axis coordinate, or, for each point in the 2D lane line annotation result, respectively performing the following processing: taking the point as a point to be processed, determining K nearest neighbor ground points of the point to be processed in the 2D top view through the nearest neighbor algorithm, where K is a positive integer greater than 1, and determining the vertical axis coordinate of the point to be processed according to the vertical axis coordinates of the K nearest neighbor ground points in the 4D point cloud data.

[0135] In some embodiments of the present disclosure, the result generation module 504 may add the K nearest neighbor ground points to the corresponding bins respectively according to the value range to which the vertical axis coordinates in the 4D point cloud data belong. Different bins respectively correspond to their own value ranges, and there is no overlap between the value ranges corresponding to any two bins. Then, the bins may be sorted in ascending order according to the corresponding values, and the bins may be traversed in sequence according to the sorting order from front to back. Among them, for each bin traversed, in response to determining that the number of ground points included in the traversed bin is greater than the sixth threshold, the vertical axis coordinate with the smallest value may be selected from the vertical axis coordinates of each ground point included in the traversed bin, and the selected vertical axis coordinate may be determined as the vertical axis coordinate of the point to be processed and the traversal may be ended. In response to determining that the number of ground points included in the traversed bin is less than or equal to the sixth threshold, the next bin may be continued to be traversed.

[0136] After obtaining the 4D lane line annotation result, the quality inspection of the 4D lane line annotation result may also be performed by a quality inspector, and the number of quality inspections may be one or multiple. As Figure 4 shown, the quality inspection may include review and acceptance, that is, the 4D lane line annotation result may be reviewed by a reviewer, and the 4D lane line annotation result may be accepted by an acceptor. Both review and acceptance can be regarded as quality inspections of the 4D lane line annotation result. If both review and acceptance are passed, the quality inspection process of the 4D point cloud data may be ended.

[0137] Accordingly, in some embodiments of the present disclosure, when the result generation module 504 determines that the quality inspection of the 4D lane line annotation result by the quality inspector fails and determines that the annotator has modified the 4D lane line annotation result based on the problems detected in the quality inspection, the 4D lane line annotation result can be updated according to the modification.

[0138] In some embodiments of the present disclosure, the quality inspection may include: the quality inspector checks whether the ground fitting degree of each lane line meets the requirements from P different perspectives respectively, where P is a positive integer greater than 1; the failure of the quality inspection may include: there are problems with at least one lane line, and the problems include: the ground fitting degree does not meet the requirements in at least one perspective.

[0139] The solution of the present disclosure can be applied to the field of artificial intelligence, especially in the fields of computer vision, deep learning, and autonomous driving. Artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0140] In addition, the 4D point cloud data, 2D top view, and annotation results in the embodiments of the present disclosure are not for a specific user and do not reflect the personal information of a specific user. In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0141] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0142] Figure 6 The schematic block diagram of an electronic device 600 that can be used to implement the embodiments of the present disclosure is shown. The electronic device 600 can execute the 4D lane line annotation method of the present disclosure, that is, it can obtain the 4D point cloud data to be annotated, and can obtain the 2D top view corresponding to the 4D point cloud data. After that, it can obtain the 2D lane line annotation result of the 2D top view, and further can generate the 4D lane line annotation result of the 4D point cloud data according to the 2D lane line annotation result, etc. The specific implementation of the 4D lane line annotation method can refer to Figures 1 to 4 the relevant descriptions in the embodiments shown.

[0143] The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0144] As Figure 6 shown, the electronic device 600 includes a computing unit 601, which may perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 may also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0145] A plurality of components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as, for example, a keyboard, a mouse, etc.; an output unit 607, such as, for example, various types of displays, speakers, etc.; a storage unit 608, such as, for example, a magnetic disk, an optical disk, etc.; and a communication unit 609, such as, for example, a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0146] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphic processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the methods described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the methods described in this disclosure can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the methods described in this disclosure by any other suitable means (e.g., by means of firmware).

[0147] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0149] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0151] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0152] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0153] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0154] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A four-dimensional lane line annotation method, comprising: Obtaining four-dimensional point cloud data to be annotated; Obtaining a two-dimensional top view corresponding to the four-dimensional point cloud data; Obtaining a two-dimensional lane line annotation result of the two-dimensional top view; Generating a four-dimensional lane line annotation result of the four-dimensional point cloud data according to the two-dimensional lane line annotation result.

2. The method according to claim 1, wherein The obtaining of the two-dimensional top view corresponding to the four-dimensional point cloud data includes: Performing clustering processing on each point in the four-dimensional point cloud data to obtain different point clusters and their corresponding categories respectively, determining the points in the point cluster of the ground point category as ground points, and generating the two-dimensional top view according to the ground points; Alternatively, determining the ground points in the four-dimensional point cloud data according to the target points selected from the four-dimensional point cloud data, and generating the two-dimensional top view according to the ground points.

3. The method according to claim 2, wherein, The target points include: M target points selected according to a predetermined rule, where M is a positive integer greater than 1; The determining of the ground points in the four-dimensional point cloud data includes: For each target point, respectively performing the following processing: determining the points in the four-dimensional point cloud data whose planar distance from the target point is less than a first threshold as candidate points, and using the candidate points to form a first point set, and determining the ground points corresponding to the target point from the first point set; Determining the ground points corresponding to each target point as the ground points in the four-dimensional point cloud data.

4. The method according to claim 3, wherein The determining of the ground points corresponding to the target point from the first point set includes: According to the value range to which the vertical axis coordinates belong, respectively adding the candidate points in the first point set to the corresponding bins, different bins respectively corresponding to their own value ranges, and the value ranges corresponding to any two bins do not overlap; Determining a target bin from each bin, and determining the ground points corresponding to the target point from the candidate points included in the target bin.

5. The method according to claim 4, wherein The determining of the target bin from each bin includes: Respectively obtaining the number of candidate points included in each bin, and determining the bin with the largest number of candidate points as the target bin; Alternatively, sorting the bins in ascending order according to the corresponding values, and sequentially traversing each bin in the front-to-back sorting order. Among them, for each bin traversed each time, respectively performing the following processing: determining the traversed bin as the bin to be processed, in response to determining that the termination condition is met according to the number of candidate points included in the bin to be processed, determining the bin to be processed as the target bin and ending the traversal, and in response to determining that the termination condition is not met, continuing to traverse the next bin.

6. The method according to claim 5, wherein, The determining of the termination condition being met according to the number of candidate points included in the bin to be processed includes: Obtaining a first ratio, where the first ratio is the ratio of the number of candidate points included in the bin to be processed to the number of candidate points included in the first point set, and in response to determining that the first ratio is greater than a second threshold, determining that the termination condition is met; Alternatively, obtain a second ratio, where the second ratio is the ratio of the number of candidate points included in the bucket to be processed to the number of candidate points included in the current maximum bucket, and the current maximum bucket is the bucket with the largest number of candidate points among the buckets that have been traversed. In response to determining that the second ratio is less than a third threshold, determine that the termination condition is met.

7. The method according to claim 4, wherein Determining the ground point corresponding to the target point from the candidate points included in the target bucket includes: Using the candidate points included in the target bucket to form a second point set; Perform N iterative processes in sequence, where N is a positive integer greater than 1. Each iterative process respectively includes: randomly extracting L candidate points from the second point set, where L is a positive integer greater than 1 and less than the number of candidate points included in the second point set; fitting a linear plane model using the L candidate points; respectively obtaining the distances between each candidate point in the second point set and the linear plane model, and determining the candidate points with distances meeting the requirements as ground points; comparing the linear plane model with the stored reference model. In response to determining that the linear plane model is better than the reference model, update the reference model to the linear plane model. In response to determining that the reference model is better than the linear plane model, keep the reference model unchanged; In response to determining that all N iterative processes have been executed, determine the ground point determined when the reference model was updated most recently as the ground point corresponding to the target point.

8. The method according to claim 2, wherein Generating the two-dimensional top view according to the ground points includes: Directly generating the two-dimensional top view according to the ground points; Alternatively, determine the neighboring points of the ground points from the four-dimensional point cloud data, and generate the two-dimensional top view according to the ground points and the neighboring points.

9. The method according to claim 8, wherein, Determining the neighboring points of the ground points from the four-dimensional point cloud data includes: Obtain the ground plane height corresponding to the four-dimensional point cloud data; Respectively determine the relative heights between each point in the four-dimensional point cloud data and the ground plane height; Determine the points with relative heights within a predetermined height range and not belonging to the ground points as the neighboring points.

10. The method according to claim 2, wherein Generating the four-dimensional lane line annotation result of the four-dimensional point cloud data according to the two-dimensional lane line annotation result includes: Respectively obtain the vertical axis coordinates of each point in the two-dimensional lane line annotation result; Generate the four-dimensional lane line annotation result according to the vertical axis coordinates.

11. The method according to claim 10, wherein, Respectively obtaining the vertical axis coordinates of each point in the two-dimensional lane line annotation result includes: According to the three-dimensional coordinates of each ground point in the four-dimensional point cloud data, determine a ground fitting equation, where the three-dimensional coordinates include the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate. According to the two-dimensional coordinates of each point in the two-dimensional lane line annotation result in the two-dimensional top view and the ground fitting equation, determine the vertical axis coordinates of each point in the two-dimensional lane line annotation result; Alternatively, for each point in the two-dimensional lane line annotation result, the following processing is performed respectively: taking this point as a point to be processed, determining K nearest neighbor ground points of the point to be processed from the two-dimensional top view through the nearest neighbor algorithm, where K is a positive integer greater than 1, and determining the vertical axis coordinate of the point to be processed according to the vertical axis coordinates of the K nearest neighbor ground points in the four-dimensional point cloud data.

12. The method according to claim 11, wherein, The determining the vertical axis coordinate of the point to be processed according to the vertical axis coordinates of the K nearest neighbor ground points in the four-dimensional point cloud data includes: According to the value range to which the vertical axis coordinates in the four-dimensional point cloud data belong, adding the K nearest neighbor ground points to the corresponding bins respectively, different bins corresponding to their respective value ranges, and there is no overlap between the value ranges corresponding to any two bins; Sorting the bins in ascending order according to the corresponding values, and traversing the bins in order from front to back. Among them, for each bin traversed, in response to determining that the number of ground points included in the traversed bin is greater than the sixth threshold, selecting the vertical axis coordinate with the smallest value from the vertical axis coordinates of the ground points included in the traversed bin, determining the selected vertical axis coordinate as the vertical axis coordinate of the point to be processed and ending the traversal, and in response to determining that the number of ground points included in the traversed bin is less than or equal to the sixth threshold, continuing to traverse the next bin.

13. The method according to claim 1, wherein, The two-dimensional lane line annotation result includes: the annotation result obtained by the annotator based on the annotation tool for the two-dimensional top view.

14. The method according to claim 13, further comprising: In response to determining that the quality inspection of the four-dimensional lane line annotation result by the quality inspector fails, and determining that the annotator has modified the four-dimensional lane line annotation result based on the problems detected in the quality inspection, updating the four-dimensional lane line annotation result according to the modification.

15. The method according to claim 14, wherein, The quality inspection includes: the quality inspector checking whether the ground fitting degree of each lane line meets the requirements from P different perspectives, where P is a positive integer greater than 1; The quality inspection fails includes: there are problems with at least one lane line, and the problems include: the ground fitting degree does not meet the requirements in at least one perspective.

16. A four-dimensional lane line marking device, comprising: A first acquisition module, a second acquisition module, a third acquisition module, and a result generation module; The first acquisition module is used to acquire the four-dimensional point cloud data to be annotated; The second acquisition module is used to acquire the two-dimensional top view corresponding to the four-dimensional point cloud data; The third acquisition module is used to acquire the two-dimensional lane line annotation result of the two-dimensional top view; The result generation module is used to generate the four-dimensional lane line annotation result of the four-dimensional point cloud data according to the two-dimensional lane line annotation result.

17. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-15.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-15.

19. A computer program product comprising a computer program / instructions which, when executed by a processor, implement the method according to any one of claims 1-15.

Images