High-precision map-based ground sign image semantic segmentation automatic labeling method and system
By utilizing high-precision maps and vehicle attitude data for automatic semantic segmentation of ground markings, the problem of time-consuming and labor-intensive manual annotation has been solved, achieving efficient and high-quality data annotation and improving the accuracy and efficiency of annotation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING SENIOR SMART DRIVING TECHNOLOGY CO LTD
- Filing Date
- 2022-07-21
- Publication Date
- 2026-06-30
AI Technical Summary
In existing technologies, semantic segmentation of ground markings such as lane lines requires a large amount of manual annotation, which results in high human resource consumption and inconsistent annotation quality, affecting the algorithm's iteration speed and performance.
By utilizing high-precision maps and vehicle attitude data, and through extrinsic parameters obtained via RTK and intrinsic parameters obtained via cameras, ground marker elements are automatically collected and projected into images for semantic segmentation, thereby achieving automatic annotation.
It saves human resources and time costs, achieves efficient and high-quality data annotation, and improves the accuracy and efficiency of annotation.
Smart Images
Figure CN115205529B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of map semantic annotation technology, and in particular to an automatic annotation method and system for semantic segmentation of ground marker images based on high-precision maps. Background Technology
[0002] Accurate semantic segmentation and mapping of road markings such as lane lines are crucial for visual localization. However, accurate and stable semantic segmentation models for road markings rely on a large amount of precise labeled data.
[0003] Currently, this can only be achieved through manual data collection, manual data cleaning, manual data annotation, and manual data quality inspection, which requires a lot of human resources. Moreover, the annotation quality is often inconsistent, which directly affects the iteration speed and performance of the algorithm. Summary of the Invention
[0004] To address the aforementioned technical issues, this application provides an automatic annotation method and system for semantic segmentation of ground marker images based on high-precision maps. By leveraging existing high-precision maps and utilizing calibrated extrinsic parameters, this method rapidly and automatically collects and annotates large amounts of ground marker data, meeting algorithm requirements and saving significant human resources and time costs.
[0005] The first aspect of this application provides an automatic annotation method for semantic segmentation of ground marker images based on high-precision maps, which may include:
[0006] S1: Obtain the ground marker elements from the high-precision map and convert the entire ground marker element into the vehicle coordinate system to form a map containing only map elements;
[0007] S2: Plan the area to be labeled, obtain the data of the vehicles in the area and their corresponding attitude areas, and form a vehicle attitude data group;
[0008] S3: Based on the location of the vehicle attitude data group, the ground marker elements are projected into the image according to the external parameters of the camera on the vehicle body and the internal parameters of the camera.
[0009] Furthermore, in step S1, the ground marking elements include lane lines, ground arrow markings, stop lines, and ground text markings.
[0010] Furthermore, in step S1, the data of the ground marker element includes region data and category data; the region data is displayed in coordinate form, and the category data is displayed according to its own definition.
[0011] Furthermore, in step S2, the area to be labeled is planned in the vehicle coordinate system.
[0012] Furthermore, in step S2, the vehicle attitude data set includes the vehicle, vehicle attitude, and timestamp in the vehicle coordinate system.
[0013] Furthermore, in step S3, based on the area where the vehicle attitude data group is located, a preset range area outside it is used as the map projection object. The corresponding area is extracted from the map containing map elements, and after data fusion with the vehicle attitude data group, it is projected onto the image to form output image data.
[0014] Furthermore, the outer preset range area is based on the set of areas where the vehicle posture data group is located, and it spreads outward from the edge of the area. The spread length in front and behind the vehicle body is greater than the spread length in the left and right sides of the vehicle body, forming a map projection object.
[0015] The second aspect of this application provides an automatic annotation system for semantic segmentation of ground marker images based on high-precision maps, including:
[0016] The vehicle system module is used to acquire ground marker elements from high-precision maps and convert the entire ground marker element into the vehicle coordinate system;
[0017] The attitude acquisition module plans the area to be labeled in the coordinate system formed by the vehicle system module, acquires the vehicle and its corresponding attitude area data in the area, and forms a vehicle attitude data group.
[0018] The image projection module, based on the attitude acquisition module, projects ground marker elements into the image and outputs them according to the external parameters of the camera vehicle and the internal parameters of the camera.
[0019] Furthermore, the ground marking elements in the output content of the vehicle system module include at least lane lines, ground arrow markings, stop lines, and ground text markings.
[0020] Furthermore, the image projection module takes the area involved in the attitude acquisition module as a basis, expands it to a preset range to form a larger area as the projection object range, extracts the corresponding area in the vehicle system module, performs data fusion with the attitude acquisition module, and projects it onto the image to form output image data.
[0021] In this embodiment, high-precision maps are used for automatic semantic segmentation and annotation of ground-labeled images. By using vehicle pose, camera-to-vehicle extrinsic parameters, and camera intrinsic parameters obtained by RTK, high-precision maps are used for fast, accurate, and automatic semantic segmentation data annotation. Compared with the current methods of manual data collection, manual data cleaning, manual data annotation, and manual data quality inspection, a lot of human resources and time costs can be saved, achieving efficient and high-quality data annotation. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart provided in an embodiment of this application;
[0024] Figure 2 This is an example of lane line ground markings and corresponding images in a high-precision map provided in this application embodiment;
[0025] Figure 3 This is an example of the final annotation result obtained by projecting elements from a high-precision map into an image, as provided in the embodiments of this application. Detailed Implementation
[0026] To make the purpose, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.
[0028] In the description of this application, it should be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0029] The automatic annotation system for semantic segmentation of ground marker images based on high-precision maps proposed in this application mainly consists of three parts: a vehicle system module, an attitude acquisition module, and an image projection module.
[0030] The vehicle system module is used to acquire ground marker elements from high-precision maps and convert the entire ground marker element into the vehicle coordinate system. The ground marker elements here include at least lane lines, ground arrows, stop lines, and ground text markings. Of course, other ground marker elements can be added, depending on the actual requirements.
[0031] The attitude acquisition module plans the area to be labeled in the coordinate system formed by the vehicle system module, acquires the data of the vehicles in the area and their corresponding attitude areas, and forms a vehicle attitude data group. Since the entire map data is in a dynamic state, and for the vehicles to be detected, it is not necessary to dynamically label the entire map, so only the vehicles need to be automatically labeled in a small area, which can greatly reduce the computational load of the system and thus improve accuracy.
[0032] As an image output module, the image projection module takes the area involved in the attitude acquisition module as a basis, expands it to a preset range to form a larger area as the projection object range, extracts the corresponding area in the vehicle system module, and projects it onto the image after data fusion with the attitude acquisition module to form output image data.
[0033] The operation of the entire system, along with the automatic annotation method, will be explained as follows:
[0034] S1: Obtain the ground marker elements from the high-precision map and convert the entire ground marker element into the vehicle coordinate system to form a map containing only map elements. The ground marker elements include lane lines, ground arrow marks, stop lines, and ground text marks.
[0035] The data for ground marker elements includes regional data and category data; the regional data is displayed in coordinate form, and the category data is displayed according to its own definition.
[0036] As a specific implementation, the coordinates of the ground marker elements in the high-precision map [X] are used to define the area coordinates. M ,Y M Z M Transform from map coordinate system to vehicle coordinate system [X] V ,Y V Z V ];
[0037] Where T VM This represents the rotational displacement matrix from the map to the vehicle body, which is obtained from the vehicle's pose information in the high-precision map. This pose information can generally be obtained through RTK in the vehicle.
[0038] S2: Plan the area to be labeled in the vehicle coordinate system, obtain the vehicle in the area and the data of the corresponding attitude area, and form a vehicle attitude data group, which includes the vehicle, vehicle attitude and timestamp in the vehicle coordinate system.
[0039] S3: Based on the location of the vehicle attitude data set, the area outside the vehicle attitude data set is used as the map projection object. The corresponding area is extracted from the map containing map elements, and after data fusion with the vehicle attitude data set, it is projected onto the image to form the output image data.
[0040] As a specific implementation, the outwardly expanded preset range area is based on the set of regions where the vehicle attitude data group is located, and expands outward from the edge of the region. The expansion length in front of and behind the vehicle is greater than the expansion length in the left and right sides of the vehicle, forming a map projection object. For example, after the map element region coordinates are converted to vehicle coordinates, only the map element coordinates within a certain area around the vehicle (e.g., 100 meters in front of the vehicle and 50 meters to the left and right) are selected, and these map element region coordinates are projected into the image.
[0041]
[0042] Where u and v represent the pixel coordinates of map element points projected onto the image, and T CV Let K be the extrinsic parameter matrix from the vehicle body to the camera obtained from offline calibration, K be the intrinsic parameters of the camera obtained from offline calibration, and λ be: λ = r 31 *X V +r 32 *Y V +r 33 *Z V +t z .
[0043] Simultaneously, the map element categories and regions in the projected image are automatically labeled for image semantic segmentation to obtain a labeled image. Specifically, the corresponding coordinates of the same map element are projected into the image to form a closed region, which is then used to obtain the segmentation and labeling result of the map element, and the map element is assigned a category. Figure 3 The image below shows an example of the final annotation result obtained by projecting lane line elements from a high-precision map onto an image.
[0044] The aforementioned annotation prevention can be achieved by using high-precision maps and calibrated extrinsic parameters to quickly and automatically collect and annotate large amounts of ground marker data, meeting algorithm requirements and saving significant human resources and time costs.
[0045] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solutions of the present invention, and these equivalent transformations are all protected by the present invention.
Claims
1. A high-precision map-based ground sign image semantic segmentation automatic labeling method, characterized in that, The method includes: S1: Obtain the ground marker elements from the high-precision map and convert the entire ground marker element into the vehicle coordinate system to form a map containing only map elements; S2: Plan the area to be labeled, obtain the data of the vehicles in the area and their corresponding attitude areas, and form a vehicle attitude data group; S3: Based on the location of the vehicle attitude data group, the ground marker elements are projected into the image according to the external parameters of the camera on the vehicle body and the internal parameters of the camera; In step S2, the vehicle attitude data set includes the vehicle, vehicle attitude, and timestamp in the vehicle coordinate system. In step S3, based on the area where the vehicle attitude data group is located, a preset range area outside it is used as the map projection object. The corresponding area is extracted from the map containing map elements, and after data fusion with the vehicle attitude data group, it is projected onto the image to form output image data. The pre-defined outward expansion range is based on the set of regions where the vehicle attitude data group is located, and it expands outward from the edge of the region. The expansion length in front of and behind the vehicle body is greater than the expansion length in the left and right sides of the vehicle body, forming a map projection object.
2. The high-definition map-based ground sign image semantic segmentation automatic labeling method according to claim 1, characterized in that, In step S1, the ground marking elements include lane lines, ground arrow markings, stop lines, and ground text markings.
3. The high-definition map-based ground sign image semantic segmentation automatic labeling method according to claim 1, characterized in that, In step S1, the data of the ground marker element includes regional data and category data; the regional data is displayed in coordinate form, and the category data is displayed according to its own definition.
4. The automatic annotation method for semantic segmentation of ground marker images based on high-precision maps according to claim 1, characterized in that, In step S2, the area to be labeled is planned in the vehicle coordinate system.
5. An automatic annotation system for semantic segmentation of ground marker images based on high-precision maps, characterized in that, The system includes: The vehicle system module is used to acquire ground marker elements from high-precision maps and convert the entire ground marker element into the vehicle coordinate system; The attitude acquisition module plans the area to be labeled in the coordinate system formed by the vehicle system module, acquires the vehicle and its corresponding attitude area data in the area, and forms a vehicle attitude data group. The image projection module, based on the attitude acquisition module, projects ground marker elements into the image and outputs them according to the external parameters of the camera vehicle and the internal parameters of the camera. The vehicle attitude data set includes the vehicle, vehicle attitude, and timestamp in the vehicle coordinate system. Based on the area where the vehicle attitude data set is located, a preset range outside the data set is used as the map projection object. The corresponding area is extracted from the map containing map elements, and after data fusion with the vehicle attitude data set, it is projected onto the image to form the output image data. The pre-defined outward expansion range is based on the set of regions where the vehicle attitude data group is located, and it expands outward from the edge of the region. The expansion length in front of and behind the vehicle body is greater than the expansion length in the left and right sides of the vehicle body, forming a map projection object.
6. The automatic annotation system for semantic segmentation of ground marker images based on high-precision maps according to claim 5, characterized in that, The ground marking elements in the output content of the vehicle system module include at least lane lines, ground arrow marks, stop lines, and ground text marks.