Automatic truth value generation method and system for long-distance BEV map perception
By introducing an automated labeling process into the BEV perception system, using radar odometer and 2D-3D information correlation technology, the problem of lane centerline and topological relationship labeling in autonomous heavy trucks is solved, reducing costs and time, and improving autonomous driving performance.
Patent Information
- Application Number
- CN202411960566.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-13
AI Technical Summary
The existing BEV sensing system is difficult to effectively mark the lane center line and topological relationship in autonomous heavy trucks, resulting in limited improvement in autonomous driving performance and high labeling costs.
By introducing the automated labeling process of BEV maps, a local point cloud map based on radar odometer is built, lane line map information is obtained, 2D-3D information association and matching optimization is carried out, continuous lane lines are obtained, and labeling quality is automated.
It greatly reduces the cost and time of BEV map perceptual data production, effectively solves the problem of lane centerline and topological relationship labeling, and provides the help of perception algorithms for the development of autonomous driving heavy-duty cards.
Smart Images
Figure CN119991979A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to an automatic truth value generation method and system for long-distance BEV map perception. Background Art
[0002] The perception system based on BEV has gradually become the mainstream solution used in autonomous driving systems. It uses multiple sensors installed on the vehicle body as input and directly outputs the position and posture of lane lines in 3D space. However, since its output is directly in 3D space, annotators cannot directly draw lines on 2D images to complete the corresponding tasks, so the annotation cost rises sharply, making map annotation difficult and costly from the perspective of BEV.
[0003] Existing automatic labeling of BEV perception tasks is mostly concentrated in lane line perception tasks and obstacle perception tasks, but there is little research on the task of map perception. Map perception provides more information than lane line perception, such as lane center lines, topological relationships between lanes, etc., and this information plays a vital role in improving the performance of autonomous driving; at the same time, existing automatic labeling technologies are mostly developed based on passenger cars, and these technologies are not well suited for autonomous driving of heavy trucks.
[0004] The main reasons for the poor effect of existing BEV automatic labeling are: since the autonomous heavy trucks have higher requirements for the perception distance, the lane line perception task often requires an effective distance of more than 90 meters to meet the requirements; in order to ensure driving comfort, the suspension between the front and chassis of heavy trucks is relatively soft, and high-frequency vibrations will occur continuously during driving. The existence of high-frequency vibrations seriously affects the positional relationship between multiple sensors, and the fusion of multiple sensors is required to be higher. Therefore, a solution is proposed to solve the problems existing in the existing technology. Summary of the invention
[0005] In view of the above problems, the first purpose of the present invention is to provide an automated truth value generation method for long-distance BEV map perception. By introducing the automated annotation process of BEV maps, the cost of BEV map perception data production is greatly reduced, and the data production speed is greatly improved, providing support for BEV map training. It effectively solves the problem that the original automatic annotation technology cannot annotate lane centerlines and topological relationships, and is not applicable due to the perception distance and the particularity of the vehicle itself for autonomous driving heavy trucks, providing support for the development of perception algorithms for autonomous driving heavy trucks.
[0006] A second object of the present invention is to provide an automated truth value generation method system for long-range BEV map perception.
[0007] To achieve the first objective, the first technical solution of the present invention is: an automatic truth value generation method for long-distance BEV map perception, comprising:
[0008] Step S01: constructing a local point cloud map based on radar odometer;
[0009] Step S02: Acquire lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data;
[0010] Step S03: performing 2D-3D information association and matching optimization on the 3D lane line map data, and transforming the 3D lane line map data into optimized lane line coordinate information;
[0011] Step S04: post-processing the optimized lane line coordinate information to obtain continuous lane lines;
[0012] Step S05: Performing automated quality inspection on the continuous lane marking quality and outputting the BEV marking result.
[0013] Preferably, step S01 includes, based on the pose transformation relationship between adjacent frames in the point cloud registration data, selecting any frame, converting frames within a specified distance near the frame to the coordinate system of the frame through pose transformation, and forming a local point cloud map of the frame.
[0014] Preferably, step S02 includes: transforming the lane line map information into a coordinate system corresponding to the local point cloud map, then identifying points on the lane line on the local point cloud map, and calculating the average height of points within a specified range corresponding to the points on the lane line as the height coordinates of the points on the lane line.
[0015] Preferably, step S02 further includes: fitting the points on the lane line to obtain a fitting equation, and using the fitting equation to calculate the height coordinates of the points on the lane line.
[0016] Preferably, step S03 includes: according to the 3D lane line map data, projecting the 3D point into the 2D image coordinate system to obtain the corresponding 2D point, calculating the projectable associated distance, projecting all 3D points within the projectable associated distance, finding the nearest neighbor to construct the optimization target, and obtaining the optimized lane line coordinate information.
[0017] Preferably, the projectable associated distance is calculated in the following manner: all 2D points on the lane line within a certain distance are selected for confirmation, and the corresponding maximum distance within the distance that makes all 2D points on the same lane line is the projectable associated distance.
[0018] Preferably, the optimization goal is:
[0019]
[0020] in, is the nearest neighbor found in the segmentation result after the i-th point is projected onto the image, π is the projection function, R and T are the rotation and translation transformations of the vehicle system and the camera system, respectively. A 3D point on a map.
[0021] Preferably, step S05 includes setting sampling points for each of the continuous lane lines to obtain sampling point coordinates, adding a disturbance of half the lane line width to the sampling point coordinates in the Y direction to obtain disturbed coordinates, respectively calculating the sampling point coordinates and the 2D point information corresponding to the projection of the disturbed coordinates, calculating the difference between the two points as the allowable error, and if all points on the continuous lane line are within the allowable error, it is determined that the quality inspection is qualified.
[0022] To achieve the second objective, the second technical solution of the present invention is: an automatic truth value generation system for long-distance BEV map perception, comprising:
[0023] Local point cloud map construction module, used to construct local point cloud maps based on radar odometer;
[0024] A lane line height assignment module is used to obtain lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data;
[0025] A 2D-3D information association and matching optimization module, used to perform 2D-3D information association and matching optimization on 3D lane line map data, and transform the 3D lane line map data into optimized lane line coordinate information;
[0026] A post-processing module, used for post-processing the optimized lane line coordinate information to obtain continuous lane lines;
[0027] The automatic marking quality inspection module is used to perform automatic marking quality inspection on the continuous lane lines.
[0028] Preferably, it also includes an output module for outputting the BEV annotation results.
[0029] Beneficial effects of the above technical solution:
[0030] The automated truth value generation method and system for long-distance BEV map perception provided by the present invention solve the problems existing in the prior art, including that the task of lane line perception often requires an effective distance of more than 90 meters to meet the requirements due to the higher requirements for the perception distance of autonomous driving heavy trucks; in order to ensure driving comfort, the suspension between the front and chassis of heavy trucks is relatively soft, and high-frequency vibrations will continue to occur during driving. The presence of high-frequency vibrations seriously affects the positional relationship between multiple sensors, and the fusion of multiple sensors has higher requirements. By introducing the automated annotation process of BEV maps, the present invention greatly reduces the cost of the BEV map perception data production process and greatly improves the data production speed, providing assistance for BEV map training. It effectively solves the problem that the original automatic annotation technology cannot annotate the center line and topological relationship of the lane, and is not applicable due to the perception distance and the particularity of the autonomous driving heavy trucks themselves, providing assistance for the development of perception algorithms for autonomous driving heavy trucks. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0032] Figure 1 A flow chart of an automated truth value generation method for long-distance BEV map perception provided by one embodiment of the present invention;
[0033] Figure 2 A local point cloud map provided for an embodiment of the present invention;
[0034] Figure 3 The labeling results and automated quality inspection results provided by an embodiment of the present invention;
[0035] Figure 4 A flow chart of an automated truth generation system for long-range BEV map perception provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0036] The following is a further detailed description of the implementation methods of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0037] The terms "first", "second", etc. (if any) in the specification and claims are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0038] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0039] Embodiment 1
[0040] In the first embodiment of the present invention, an automatic truth value generation method for long-distance BEV map perception is provided, as shown in the flowchart. Figure 1 As shown, the method specifically includes the following steps: Step S01: constructing a local point cloud map based on radar odometer;
[0041] Step S02: Acquire lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data;
[0042] Step S03: performing 2D-3D information association and matching optimization on the 3D lane line map data, and transforming the 3D lane line map data into optimized lane line coordinate information;
[0043] Step S04: post-processing the optimized lane line coordinate information to obtain continuous lane lines;
[0044] Step S05: Performing automated quality inspection on the continuous lane marking quality and outputting the BEV marking result.
[0045] In this embodiment, preferably, step S01: constructing a local point cloud map based on a radar odometer specifically includes: first constructing a local map based on a single-frame radar point cloud. In this method, a radar odometer based on an ICP algorithm is used to complete the corresponding task. First, the pose transformation relationship between adjacent frames in the data is obtained based on ICP (point cloud registration), and then for the i-th frame, frames within a certain distance are selected, and the point clouds of these frames are uniformly transformed to the coordinate system of the i-th frame through pose transformation to form a local point cloud map of the i-th frame, such as Figure 2 As shown in the local point cloud map.
[0046] Step S02: Since the lane map often does not contain height information, or even if it contains height information, the height error in the vehicle coordinates may be large due to the inaccurate measurement of the GPS. Therefore, this method uses the local point cloud map to assign the lane height. The specific method is to extract all the map information in the ROI of interest to the vehicle and transform it to the local point cloud map. Figure 1 For each point on the lane line, select all points in the local point cloud map around its two-dimensional coordinates and calculate their average height as the coordinate of the lane line. For some points on the lane line that do not have enough laser points in the local point cloud map to calculate the height, fit a two-dimensional equation z = ax to the points on the entire lane line. 2 +bx+cy 2 +dy+e, and calculate the height of the corresponding point based on this equation.
[0047] Step S03: 2D-3D information association and matching optimization are performed on the 3D lane map data, and the 3D lane map data is transformed into optimized lane coordinate information; specifically, 2D-3D information association and matching optimization are performed. After the lane points with 3D coordinates in the vehicle coordinate system are obtained, they are converted to the perspective of each camera for further mechanical processing. The necessity of this step is that the camera and radar cannot be considered to be always in a rigid body fixed relationship in the engineering, and the errors that may exist in each step also need to be considered, so further association and optimization are required. The specific process is as follows:
[0048] a) Project the 3D point to the image coordinate system through the camera's internal and external parameter matrix, and record the coordinates of the i-th lane line point in the 3D perspective as Then its projection on the 2D image is:
[0049] b) Calculate the projectable association distance. In scenarios such as large curvature ramps, the left and right lane line segmentation results of the same lane from a 2D perspective often intersect in the distance, which will affect the accuracy of the association. Therefore, it is necessary to calculate the effective projection distance. Use the 2D projection of each lane line generated in step a) to find all points on the image whose distance is less than a certain threshold. If there are points on different lane lines from the point, the distance is considered to be unassociated. After all lane lines are calculated, the minimum value is the farthest distance that can be associated.
[0050] c) After association, find the projection point of the lane line point on 2D based on the nearest neighbor search Nearest neighbor on lane segmentation result Then we can construct the optimization goal:
[0051]
[0052] In the formula represents the nearest neighbor found in the segmentation result after the i-th point is projected onto the image, π represents the projection function, RT represents the rotation and translation transformation of the vehicle system and the camera system respectively, Represents a 3D point under a map.
[0053] Step S04: Post-process the optimized lane line coordinate information to obtain continuous lane lines. Since the map information is different from the information used for perception, some areas need to be post-processed, mainly including: connecting lane lines with the same attributes that are interrupted for various reasons in the map; upsampling or downsampling points on the lane lines based on demand.
[0054] Step S05: Automatic quality inspection of annotation quality. In order to minimize the role of manual work in annotation, the algorithm will automatically inspect the annotation results. The quality inspection method is as follows:
[0055] a) Place a sampling point at a fixed distance (e.g. 20m, 50m, 100m) between each lane line and the vehicle body, and let the coordinate of the point be P i (x i ,y i ,z i ), then add a disturbance of half the lane width in the y direction to obtain the coordinate P' i (x i ,y i +θ,z i ), calculate their projections on the image respectively, and get the error on the image x. It can be approximately considered that if the point belongs to the segmentation result of the image, the error between it and the lane center should be less than this value.
[0056] b) Find the nearest neighbor of the point on the segmentation result. If the x-direction error is less than this value, the point is considered to have passed the inspection. The projection of the 3D point on the image falls within the lane line area.
[0057] c) By calculating all points according to the above logic, we can determine whether the automatic labeling of each lane is accurate.
[0058] Figure 3 The automatic labeling results and quality inspection results are shown. The automatic labeling results and quality inspection results provided by the present invention can provide labeling information such as lane centerline and lane line, and at the same time give the inspection results of the automatic labeling by manual quality inspectors, such as Figure 3 The front view and the rear view cameras are completely matched, but not completely matched. Manual quality inspectors can selectively check and correct the annotation results, greatly improving work efficiency.
[0059] like Figure 4 As shown in the flow chart of the automatic truth value generation system for long-distance BEV map perception, the present invention also provides an automatic truth value generation system for long-distance BEV map perception. The system includes: a local point cloud map construction module, which is used to construct a local point cloud map based on a radar odometer. A lane line height assignment module, which is used to obtain lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data. A 2D-3D information association and matching optimization module, which is used to perform 2D-3D information association and matching optimization on 3D lane line map data, and transform the 3D lane line map data into optimized lane line coordinate information. A post-processing module, which is used to post-process the optimized lane line coordinate information to obtain continuous lane lines. An automated annotation quality inspection module, which is used to perform automated annotation quality inspection on the continuous lane lines. It also includes an output module for outputting BEV annotation results.
[0060] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.
Claims
1. An automated truth generation method for long-range BEV map perception, characterized in that: include: Step S01: constructing a local point cloud map based on radar odometer; Step S02: Acquire lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data; Step S03: performing 2D-3D information association and matching optimization on the 3D lane line map data, and transforming the 3D lane line map data into optimized lane line coordinate information; Step S04: post-processing the optimized lane line coordinate information to obtain continuous lane lines; Step S05: Performing automated quality inspection on the continuous lane marking quality and outputting the BEV marking result.
2. The method for automatic truth value generation for long-distance BEV map perception according to claim 1, characterized in that: Step S01 includes, based on the pose transformation relationship between adjacent frames in the point cloud registration data, selecting any frame, converting frames within a specified distance near the frame to the coordinate system of the frame through pose transformation, and forming a local point cloud map of the frame.
3. The method for automatic truth value generation for long-distance BEV map perception according to claim 1, characterized in that: Step S02 includes: transforming the lane line map information into a coordinate system corresponding to the local point cloud map, then identifying points on the lane line on the local point cloud map, and calculating the average height of points within a specified range corresponding to the points on the lane line as the height coordinates of the points on the lane line.
4. The method for automatic truth value generation for long-distance BEV map perception according to claim 3, characterized in that: Step S02 also includes: fitting the points on the lane line to obtain a fitting equation, and using the fitting equation to calculate the height coordinates of the points on the lane line.
5. The method for automatic truth value generation for long-distance BEV map perception according to claim 1, characterized in that: Step S03 includes: according to the 3D lane line map data, projecting the 3D point into the 2D image coordinate system to obtain the corresponding 2D point, calculating the projectable associated distance, projecting all 3D points within the projectable associated distance, finding the nearest neighbor to construct the optimization target, and obtaining the optimized lane line coordinate information.
6. The method for automatic truth value generation for long-distance BEV map perception according to claim 5, characterized in that: The projectable associated distance is calculated in the following manner: all 2D points on the lane line within a certain distance are selected for confirmation, and the corresponding maximum distance within the distance that makes all 2D points on the same lane line is the projectable associated distance.
7. The method for automatic truth value generation for long-distance BEV map perception according to claim 5, characterized in that: The optimization goal is: in, is the nearest neighbor found in the segmentation result after the i-th point is projected onto the image, π is the projection function, R and T are the rotation and translation transformations of the vehicle system and the camera system, respectively. A 3D point on a map.
8. The method for automatic truth value generation for long-distance BEV map perception according to claim 1, characterized in that: Step S05 includes setting sampling points for each of the continuous lane lines to obtain sampling point coordinates, adding a disturbance of half the lane line width to the sampling point coordinates in the Y direction to obtain disturbed coordinates, respectively calculating the sampling point coordinates and the 2D point information corresponding to the projection of the disturbed coordinates, and calculating the difference between the two points as the allowable error. If all points on the continuous lane line are within the allowable error, it is determined to be qualified for quality inspection.
9. An automated truth value generation system for long-range BEV map perception, characterized in that: include: Local point cloud map construction module, used to construct local point cloud maps based on radar odometer; A lane line height assignment module is used to obtain lane line map information, assign lane line heights based on the local point cloud map, and obtain 3D lane line map data; A 2D-3D information association and matching optimization module, used to perform 2D-3D information association and matching optimization on 3D lane line map data, and transform the 3D lane line map data into optimized lane line coordinate information; A post-processing module, used for post-processing the optimized lane line coordinate information to obtain continuous lane lines; The automatic marking quality inspection module is used to perform automatic marking quality inspection on the continuous lane lines.
10. The automated truth value generation system for long-distance BEV map perception according to claim 9, characterized in that: It also includes an output module for outputting BEV annotation results.