Roadside data labeling method and related system and storage medium

Abstract

The embodiment of the present application provides a method for annotating curb data and a related system and storage medium, the method comprising: obtaining a vehicle body posture data sequence of the vehicle and a laser point cloud data sequence containing the surrounding environment information of the vehicle, and obtaining a laser high-precision map according to the vehicle body posture data sequence and the laser point cloud data sequence; obtaining a curb feature point set according to the laser high-precision map and the laser point cloud data sequence; processing each feature point in the curb feature point set to obtain a curb map containing multiple curb instances; obtaining curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence. By adopting this method, the efficiency of curb data generation can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a roadside data labeling method and related systems and storage media. Background Art

[0002] For unmanned driving platforms and advanced driving assistance systems (ADAS), the environmental perception system is the interactive port between the intelligent vehicle platform and the surrounding traffic scenes, and is also the front-end input of the intelligent vehicle platform's motion decision-making and planning control system. The performance of the environmental perception system directly determines the stability of the unmanned driving platform and ADAS in executing driving tasks. In recent years, taking deep convolutional neural networks (CNN) as an example, data-driven perception algorithms in environmental perception systems have been widely used in the field of unmanned driving. At present, most data-driven perception algorithms require pre-annotated true value data to supervise the model training process. The number of annotations of the true value data, the quality of the annotations, and the diversity of the annotation scenes also directly determine the prediction performance and generalization ability of the model.

[0003] At present, for the annotated data in autonomous driving technology, such as the 2D or 3D true bounding box of the target, the semantic annotation at the pixel level of image data and the point level in the laser point cloud, and the annotation of road topological structures such as curbs and lane lines, data annotators need to spend a lot of time manually annotating each target or pixel one by one, which is inefficient and costly. In addition, for the annotated data, it still takes a lot of time to review and control the annotation quality, which indirectly increases the workload of annotated data generation. Taking the manual annotation of semantic segmentation of laser point cloud as an example, it takes about 3-4 hours to annotate each frame of data on average, while the working frequency of laser radar is generally 10-20Hz, that is, at least 10 frames of data are generated per second. The efficiency of manual annotation is far lower than the frequency of data generation.

[0004] Among them, the curb pixel area in the sample image and the pixel area where the curb is visible in the curb pixel area are manually determined; and the curb confidence of the sample image is annotated according to the ratio information of the pixel area where the curb is visible to the curb pixel area in the curb pixel area; finally, the curb detection model is trained based on the manually annotated curb data and the sample image data.

[0005] The method for generating curb data in the above road structure basically relies on manual annotation, which is inefficient and difficult to ensure the quality of data annotation. It often takes a lot of time to review the quality of data annotation. Moreover, this method generates curb annotation data in the image, and in actual application, it needs to use inverse perspective transformation to transform it into the vehicle body coordinate system. The transformation process will introduce projection errors, resulting in large deviations in curb data at a distance. Summary of the invention

[0006] The present application discloses a roadside data annotation method and related system and storage medium, which can improve the efficiency of roadside data generation.

[0007] In a first aspect, an embodiment of the present application provides a method for labeling curb data, comprising: obtaining a vehicle body posture data sequence of a vehicle and a laser point cloud data sequence containing information about the surrounding environment of the vehicle, and obtaining a laser high-precision map based on the vehicle body posture data sequence and the laser point cloud data sequence; obtaining a curb feature point set based on the laser high-precision map and the laser point cloud data sequence; processing each feature point in the curb feature point set to obtain a curb map containing multiple curb instances; and obtaining curb feature labeling data corresponding to each frame of laser point cloud data in the laser point cloud data sequence based on the curb map, the laser point cloud data sequence and the vehicle body posture data sequence.

[0008] In the embodiment of the present application, a laser high-precision map is obtained based on a vehicle posture data sequence and a laser point cloud data sequence, and a roadside feature point set is obtained according to the laser high-precision map and the laser point cloud data sequence; each feature point in the roadside feature point set is then processed to obtain a roadside map containing multiple roadside instances; and roadside feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence is obtained according to the roadside map, the laser point cloud data sequence and the vehicle posture data sequence. With this method, since the roadside map is a necessary condition for the automatic generation of a single frame of roadside data, the roadside data in the roadside map has better consistency in the world coordinate system than the roadside data generated frame by frame, and the roadside map data obtained using multiple frames of data has better data quality in the obstacle occlusion area, which improves the confidence and accuracy of the roadside annotation data; the roadside annotation data is obtained based on the roadside map and each frame of laser point cloud data. Compared with the existing manual annotation method, this solution effectively improves the data annotation efficiency while ensuring the accuracy of the roadside data, and at the same time improves the prediction performance of the model.

[0009] As an optional implementation method, the method of obtaining a curb feature point set based on the laser high-precision map and the laser point cloud data sequence includes: processing the laser point cloud data sequence to obtain laser point cloud semantic information; obtaining a road semantic map based on the laser point cloud semantic information and the laser high-precision map; and extracting curb feature points from the road semantic map to obtain a curb feature point set in the road semantic map.

[0010] This embodiment obtains a set of curb feature points based on the laser SLAM method and the laser semantic segmentation network. By using multiple frames of laser point cloud data to build a map, it can effectively avoid the impact of semantic segmentation errors in single-frame laser point cloud data, thereby improving the efficiency of obtaining the curb feature point set in the road semantic map.

[0011] Furthermore, the road semantic map includes a first type of point cloud and a second type of point cloud, and the roadside feature point extraction of the road semantic map to obtain a roadside feature point set in the road semantic map includes: mapping the road semantic map to a two-dimensional grid map to obtain a mapped road semantic map; obtaining a candidate grid from the mapped road semantic map, the candidate grid being a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold; and obtaining a roadside feature point set based on the first type of point cloud and the second type of point cloud in the candidate grid.

[0012] The point cloud in the road semantic map can be divided into two categories: the point cloud belonging to the road surface classification is the first category of point cloud, and the point cloud classification outside the curb is the second category of point cloud. The point cloud classification outside the curb can be, for example, a sidewalk, a green plant, and the like.

[0013] By adopting this method, a candidate grid containing two types of point clouds is obtained to improve the efficiency and accuracy of determining the curb feature point set; on the other hand, based on the characteristics of the curb in the road, feature points corresponding to some tree branches and lamp poles extending above the road surface in actual road scenes are excluded, thereby improving the reliability of determining the curb feature point set.

[0014] Further, the processing of each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes: iteratively searching feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein, for the pth submap in the K submaps, steps S1-S6 are executed:

[0015] S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold;

[0016] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold;

[0017] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set;

[0018] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0019] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is all 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0020] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers;

[0021] A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

[0022] This embodiment automatically annotates the roadside instances based on the roadside feature point set in the road semantic map, and obtains a roadside map containing the roadside instances. Compared with the current manual single-frame frame-by-frame annotation, this solution greatly improves the annotation efficiency; and because the roadside feature point set in the road semantic map is directly used to generate the roadside data, there is no dotting error in the manual annotation process, which further improves the reliability of the annotation.

[0023] As another optional implementation method, the method of obtaining a roadside feature point set based on the laser high-precision map and the laser point cloud data sequence includes: performing semantic feature extraction on each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence; and mapping the roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

[0024] Furthermore, the processing of each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes:

[0025] S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold;

[0026] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold;

[0027] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set;

[0028] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0029] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, wherein the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0030] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers;

[0031] A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

[0032] This embodiment automatically annotates the roadside instances based on the roadside feature point set to obtain a roadside map containing the roadside instances. Compared with the current manual single-frame frame-by-frame annotation, this solution greatly improves the annotation efficiency; and because the roadside feature point set is directly used to generate the roadside data, there is no dotting error in the manual annotation process, which further improves the reliability of the annotation.

[0033] As an optional implementation, the method of obtaining curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence includes: obtaining multiple reference curb maps according to multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, the multiple reference curb maps are obtained by mapping the curb maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference curb maps correspond one-to-one to the multiple frames of laser point cloud data; obtaining multiple preset areas from the multiple reference curb maps, the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference curb maps; determining a score for each feature point in the multiple preset areas according to the laser point cloud data sequence and the feature points in the multiple preset areas; obtaining curb feature annotation data corresponding to each frame of laser point cloud data according to the score of each feature point in the multiple preset areas, and the curb feature annotation data is obtained by processing the feature points whose scores are not less than a sixth threshold.

[0034] In this embodiment, the roadside map and the single-frame laser point cloud data are combined to obtain the score of each feature point, and then the roadside annotation data in a specific data form that matches the single-frame original laser point cloud data is screened out. This method has high annotation efficiency and has stronger global consistency compared to manual frame-by-frame annotation.

[0035] In a second aspect, an embodiment of the present application provides a roadside data labeling device, including:

[0036] An acquisition module, used to acquire a body posture data sequence of the vehicle and a laser point cloud data sequence containing the surrounding environment information of the vehicle;

[0037] Processing modules for:

[0038] Obtaining a laser high-precision map according to the vehicle body posture data sequence and the laser point cloud data sequence;

[0039] Obtain a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence;

[0040] Processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances;

[0041] According to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence, the curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence is obtained.

[0042] As an optional implementation method, the processing module is used to: process the laser point cloud data sequence to obtain laser point cloud semantic information; obtain a road semantic map based on the laser point cloud semantic information and the laser high-precision map; extract curb feature points from the road semantic map to obtain a curb feature point set in the road semantic map.

[0043] As an optional implementation method, the road semantic map includes a first type of point cloud and a second type of point cloud, and the processing module is further used to: map the road semantic map into a two-dimensional grid map to obtain a mapped road semantic map; obtain a candidate grid from the mapped road semantic map, the candidate grid is a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold; obtain a roadside feature point set based on the first type of point cloud and the second type of point cloud in the candidate grid.

[0044] As an optional implementation, the processing module is further used to: perform iterative search processing on feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein, for the pth submap of the K submaps, steps S1-S6 are executed:

[0045] S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold;

[0046] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold;

[0047] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set;

[0048] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0049] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is all 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0050] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers;

[0051] A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

[0052] As another optional implementation method, the processing module is used to: perform semantic feature extraction on each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence; map the roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

[0053] As an optional implementation, the processing module is further used to:

[0054] S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold;

[0055] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold;

[0056] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set;

[0057] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0058] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, wherein the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0059] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers;

[0060] A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

[0061] As an optional implementation method, the processing module is also used to: obtain multiple reference curb maps based on multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, the multiple reference curb maps are obtained by mapping the curb maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference curb maps correspond one-to-one to the multiple frames of laser point cloud data; obtain multiple preset areas from the multiple reference curb maps, the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference curb maps; determine the score of each feature point in the multiple preset areas based on the laser point cloud data sequence and the feature points in the multiple preset areas; obtain curb feature annotation data corresponding to each frame of laser point cloud data based on the score of each feature point in the multiple preset areas, and the curb feature annotation data is obtained by processing the feature points whose scores are not less than a sixth threshold.

[0062] In a third aspect, the present application provides a roadside data labeling device, comprising a processor and a memory; wherein the memory is used to store program code, and the processor is used to call the program code to execute a method provided in any possible implementation manner of the first aspect.

[0063] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a method provided in any possible implementation manner of the first aspect.

[0064] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute a method provided in any possible implementation of the first aspect.

[0065] In a sixth aspect, the present application provides a chip system, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes a method provided in any possible implementation manner of the first aspect.

[0066] In a seventh aspect, the present application provides an intelligent driving vehicle, comprising a travel system, a sensing system, a control system, and a computer system, wherein the computer system is used to execute a method provided in any possible implementation manner of the first aspect.

[0067] It can be understood that the device described in the second aspect, the device described in the third aspect, the computer-readable storage medium described in the fourth aspect, or the computer program product described in the fifth aspect, the chip system provided in the sixth aspect, and the intelligent driving vehicle provided in the seventh aspect are all used to execute any method provided in the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The following is an introduction to the drawings used in the embodiments of the present application.

[0069] Figure 1a It is a schematic diagram of the architecture of a roadside data annotation system provided in an embodiment of the present application;

[0070] Figure 1b This is a schematic diagram of a roadside data annotation application provided in an embodiment of the present application;

[0071] Figure 2 It is a flowchart of a method for marking roadside data provided by an embodiment of the present application;

[0072] Figure 3a It is a top view of laser point cloud data provided by an embodiment of the present application;

[0073] Figure 3b is a schematic diagram of a laser high-precision map provided in an embodiment of the present application;

[0074] Figure 3c is a schematic diagram of a laser point cloud semantic map provided in an embodiment of the present application;

[0075] Figure 3d is a schematic diagram of a road semantic map provided in an embodiment of the present application;

[0076] Figure 3e is a schematic diagram of a roadside feature point set provided in an embodiment of the present application;

[0077] Figure 4 is a schematic diagram of a method for obtaining a roadside map provided in an embodiment of the present application;

[0078] Figure 5a is a schematic diagram of a roadside feature point set provided in an embodiment of the present application;

[0079] Figure 5b is a schematic diagram of a growth iteration direction provided in an embodiment of the present application;

[0080] Figure 5c It is a schematic diagram of obtaining multiple curb instances provided by an embodiment of the present application;

[0081] Figure 5d is a schematic diagram of a sub-map merging provided in an embodiment of the present application;

[0082] Figure 6a is a schematic diagram of determining neighborhood feature points provided by an embodiment of the present application;

[0083] Figure 6b is a schematic diagram of determining different iterative growth directions provided in an embodiment of the present application;

[0084] Figure 7 It is a schematic diagram of marking roadside data based on a roadside map provided in an embodiment of the present application;

[0085] Figure 8 It is a structural schematic diagram of a roadside data labeling device provided in an embodiment of the present application;

[0086] Fig. 9 It is a structural schematic diagram of another roadside data labeling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method part of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0088] Reference Figure 1a As shown, it is a schematic diagram of the architecture of a roadside data annotation system provided by an embodiment of the present application. Figure 1a As shown, the system includes a laser radar, an integrated navigation system, and a curb data labeling device. Among them, the laser radar is used to obtain laser point cloud data of the vehicle's surrounding environment; the integrated navigation system may include a global positioning system (GPS) and an inertial measurement unit (IMU), and the integrated navigation system is used to obtain a vehicle body posture data sequence; the curb data labeling device is used to perform curb data labeling processing. Specifically, the laser radar sends the acquired point cloud data of the vehicle's surrounding environment to the curb data labeling device, and the global positioning system and the inertial measurement unit both send the acquired posture data to the curb data labeling device, so that the curb data labeling device can label the curb data.

[0089] As an optional implementation, the above-mentioned curb data annotation device may include a laser radar driving module, an integrated navigation system driving module, a curb map generation module, and a curb data annotation module. Among them, the laser radar driving module is used to convert the data packet sent by the laser radar into three-dimensional laser point cloud data; the integrated navigation system driving module is used to fuse the data obtained by the global positioning system and the data obtained by the inertial measurement unit and output continuous and smooth vehicle posture data; the curb map generation module is used to generate curb map data containing multiple curb instances; the curb data annotation module is used to generate curb annotation data of each frame of multiple data types.

[0090] The above embodiments are described using laser radar and a combined navigation system as examples, wherein the system can also be based on other sensors to obtain point cloud data of the vehicle's surrounding environment and vehicle posture data, etc., and this solution does not make specific limitations on this.

[0091] Optionally, the above-mentioned curb data labeling device can be a server, which can be a virtual server, a physical server, etc. It can also be other devices, and this solution does not make specific limitations on this.

[0092] The roadside data annotation system of the embodiment of the present application can be applied to the environment perception module of the autonomous driving platform and the advanced driver assistance system. Figure 1b As shown in the figure, the intelligent vehicle needs to fully perceive the surrounding environment of the vehicle, such as the road boundary information around the vehicle. In the current data-driven deep neural network algorithm in the industry, the prediction ability of the model depends largely on the quantity and quality of the labeled data. To this end, this solution provides a curb data annotation method for the laser curb data annotation (data generation) part. The automated curb data annotation method provided by this solution can improve the efficiency of curb data generation and indirectly improve the prediction performance of the model.

[0093] Reference Figure 2 FIG. 2 is a flow chart of a method for labeling roadside data provided by an embodiment of the present application. The method includes steps 201-204, which are as follows:

[0094] 201. Acquire a vehicle body posture data sequence of the vehicle and a laser point cloud data sequence containing information about the surrounding environment of the vehicle, and obtain a laser high-precision map according to the vehicle body posture data sequence and the laser point cloud data sequence;

[0095] The above vehicle body posture data sequence is based on multiple postures of vehicle body posture data obtained by continuous movement of the vehicle.

[0096] The vehicle body posture data sequence includes multiple frames of vehicle body posture data, which may include the position information and posture information of the vehicle.

[0097] The laser point cloud data sequence is laser point cloud data based on the surrounding environment information of the vehicle obtained by the vehicle continuously moving. That is, the laser point cloud data sequence includes multiple frames of laser point cloud data.

[0098] The above-mentioned high-precision laser map may be a map with centimeter-level accuracy, wherein the coordinate system corresponding to the map may be a world coordinate system.

[0099] As an optional implementation method, a laser high-precision map can be obtained by using a laser point cloud data sequence and a vehicle posture data sequence through a laser simultaneous localization and mapping (SLAM) method or a laser odometer method.

[0100] Among them, the laser SLAM method is suitable for data sequences containing multiple repeated sections (multiple loops), which can be understood as repeated passing through a certain section. In such scenarios, loop detection can ensure the consistency of map data and roadside data. The laser odometer method is suitable for scenarios without repeated sections such as highways (no loops).

[0101] During the map generation process, the frame number frame_id of the laser point used as the map point and the number point_id of the point in this frame are recorded. The laser high-precision map can be recorded as Where N is the number of points in the map, (x, y, z) is the three-dimensional spatial coordinates of the point cloud in the world coordinate system, and intensity is the reflection intensity of the corresponding laser point.

[0102] 202. Obtain a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence;

[0103] The roadside can be understood as the boundary between the road surface and other non-road elements, such as curbs, walls, green belts and other things on the road boundary.

[0104] As an optional implementation method (Example 1), the above-mentioned method of obtaining a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence includes steps 2021-2023, which are specifically as follows:

[0105] 2021. Processing the laser point cloud data sequence to obtain laser point cloud semantic information;

[0106] The semantic information of the laser point cloud can be understood as the category information to which each point in the laser point cloud belongs, for example, pedestrian, ground, etc.

[0107] Specifically, the laser point cloud data sequence is input into the laser semantic segmentation network for semantic segmentation processing, and then the laser point cloud semantic information can be obtained.

[0108] Of course, other semantic segmentation processing may also be used, such as semantic segmentation that only distinguishes between ground and non-ground point clouds, and this solution does not make specific limitations on this.

[0109] 2022. Obtain a road semantic map according to the laser point cloud semantic information and the laser high-precision map;

[0110] As an optional implementation method, the laser point cloud semantic information is mapped to the laser high-precision map through frame_id and point_id to obtain a point cloud semantic map, which can be recorded as Among them, label is the classification information of the point cloud; then, semantic map points related to road structure information are extracted from the point cloud semantic map to obtain a road semantic map.

[0111] The semantic map points related to the road structure information may be, for example, road surfaces, sidewalks, roadside vegetation, etc.

[0112] 2023. Extract curb feature points from the road semantic map to obtain a curb feature point set in the road semantic map.

[0113] The point cloud in the road semantic map can be divided into two categories: the point cloud belonging to the road surface classification is the first category of point cloud, and the point cloud classification outside the curb is the second category of point cloud. The point cloud classification outside the curb can be, for example, a sidewalk, green plants, etc. The curb point cloud is located at the junction of the first category of point cloud and the second category of point cloud.

[0114] As an optional implementation manner, the extracting roadside feature points from the road semantic map to obtain a roadside feature point set in the road semantic map includes:

[0115] Mapping the road semantic map into a two-dimensional grid map to obtain a mapped road semantic map;

[0116] Acquire a candidate grid from the mapped road semantic map, wherein the candidate grid is a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold;

[0117] A set of roadside feature points is obtained according to the first type of point cloud and the second type of point cloud in the candidate grid.

[0118] Specifically, the road semantic map is mapped into a two-dimensional grid map in the xy plane, and the roadside feature points are determined based on the following aspects:

[0119] For each grid in the two-dimensional grid image, if both the first type of point cloud and the second type of point cloud are located in the grid, the grid is marked as a candidate grid.

[0120] As an optional implementation method, considering that some tree branches will extend above the road surface in actual road scenes, if the height difference between the first and second point clouds in the candidate grid in the original road semantic map is greater than a certain height threshold, the candidate grid is removed, and the candidate grids in which the height difference between the first and second point clouds is less than a certain height threshold are retained.

[0121] Among them, the first type of point cloud in the filtered candidate grid can be marked as a curb feature point; or the second type of point cloud in the filtered candidate grid can be marked as a curb feature point; or the curb feature point can be obtained based on the intersection of the first type of point cloud and the second type of point cloud, etc.

[0122] By traversing the point cloud in the road semantic map, a set of roadside feature points is obtained.

[0123] like Figure 3a , Figure 3b , Figure 3c , Figure 3d as well as Figure 3e As shown, through the top view based on laser point cloud data ( Figure 3a ) is combined with the vehicle posture data sequence to obtain Figure 3b The laser point cloud data sequence is processed by semantic segmentation to obtain the laser point cloud semantic information, and the laser point cloud semantic information is mapped to the laser high-precision map to obtain Figure 3c The semantic map of the laser point cloud is extracted from the semantic map of the laser point cloud, and the semantic map points related to the road structure information are further obtained. Figure 3d The road semantic map shown in the figure is then used to obtain the road feature point set based on the two-dimensional grid map, as shown in Figure 3e As shown. Among them, region B corresponds to the above candidate grid, which has both types of point clouds, region A corresponds to the first type of point cloud, region C corresponds to the second type of point cloud, or region A corresponds to the second type of point cloud, region C corresponds to the first type of point cloud, etc.

[0124] The above embodiment is described by taking the extraction of the roadside feature point set based on the entire high-precision map as an example. Among them, the roadside feature point set can also be obtained based on the semantic features of the roadside of a single frame.

[0125] As another optional implementation (Example 2), the roadside feature point set obtained according to the laser high-precision map and the laser point cloud data sequence includes 202A-202B, which is as follows:

[0126] 202A, extracting semantic features from each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence;

[0127] 202B. Map the roadside feature point set of each frame of laser point cloud data in the laser point cloud data sequence to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

[0128] Among them, the single-frame roadside feature point set can be extracted based on the spatial geometric features of the point cloud, specifically in the form of combining the laser single scanning layer features and the features between adjacent scanning layers. A single scanning layer refers to a laser scanning line beam in a multi-line laser radar, and an adjacent scanning layer refers to multiple adjacent laser scanning line beams.

[0129] The single scan layer features mainly include slope features, number features and reflection intensity features.

[0130] Among them, the slope feature mainly takes into account the consistency between the spatial distribution of the curb and the direction of vehicle movement, that is, in the vehicle coordinate system, the slope of the line connecting the curb feature point and its neighborhood point is within a certain angle threshold range.

[0131] The number feature mainly takes into account the continuity of the spatial distribution of the roadside, and the number of neighborhood points of the roadside feature point is greater than a certain threshold.

[0132] The reflection intensity feature is designed to filter out noise points with low reflection intensity such as rain.

[0133] The neighborhood points of the above-mentioned curb feature point can be understood as points within the circular area that can be obtained by drawing a circle with a certain radius length with the curb feature point as the center, that is, the neighborhood points of the curb feature point. Of course, other methods can also be used to determine the neighborhood points, and this solution does not make specific limitations on this.

[0134] The features between adjacent scanning layers mainly take into account that the curb is generally a cross-section perpendicular to the road surface. The distribution of the laser point cloud on the curb section is denser than that of the road surface points. Therefore, the distance between the neighboring points between adjacent layers in the top view will be within a certain distance threshold; the number feature between layers, that is, the number of neighboring points between the curb feature points and their scanning layers, can also evaluate the continuity of the road curb distribution.

[0135] Based on the above laser single scan layer features and adjacent scan layer features, the roadside feature point set of each frame of laser point cloud data is determined. The data frame number frame_id and the point number point_id in the frame of the obtained single frame roadside feature point are recorded, and the feature points are mapped to the high-precision map through the vehicle posture information sequence to obtain the roadside feature point set corresponding to the coordinate system of the laser high-precision map, that is, the roadside point cloud feature map.

[0136] 203. Process each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances;

[0137] A curb instance can be understood as a curb that is continuous in space.

[0138] As an optional implementation manner (corresponding to the above-mentioned embodiment 1), processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes:

[0139] Iteratively search the feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein for the pth submap in the K submaps, execute steps S1-S6:

[0140] S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold;

[0141] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold;

[0142] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set;

[0143] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0144] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is all 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0145] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers;

[0146] A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

[0147] The above feature point T(i,j) represents the jth feature point in the i-th curb instance.

[0148] Specifically, taking a 64-line laser radar as an example, it generates about 1.2-1.3 million data points per second. The data points of a high-precision map are generally in the tens of millions. Considering the computing performance limit of the system, the road semantic map is generally divided into blocks and batches by region when processing map data. Therefore, the road semantic map is divided into multiple sub-maps, such as Figure 4 shown.

[0149] The segmentation may be uniform segmentation or arbitrary segmentation, etc. Of course, segmentation may not be performed, and this solution does not make any specific limitation on this.

[0150] Then, an iterative search process is performed on the feature points in the roadside feature point set of each sub-map to obtain feature point sets of multiple roadside instances.

[0151] The above-mentioned iterative search process may include:

[0152] 1) Create a K-dimensional (KD) tree for each sub-map, such as Figure 5a As shown;

[0153] 2) Randomly extract a feature point of a roadside in each sub-map, and use the nearest neighbor search algorithm (k-nearest neighbor, KNN) to search for feature points within a certain neighborhood of the feature point. If the number of feature points in its neighborhood is greater than a certain threshold, then this feature point is used as the initial feature point;

[0154] For example, Figure 6a As shown in FIG. 1 , the points in the circular area obtained by drawing a circle with a certain radius with feature point A as the center are the feature points (neighborhood feature points) in the neighborhood of feature point A. Since point D is not in the area, point D is not a neighborhood feature point of feature point A.

[0155] The initial feature point may also be determined based on other methods, and this solution does not make any specific limitation on this.

[0156] 3) Classifying the neighborhood feature points according to the angle between each neighborhood feature point searched in 2) and the line connecting the initial feature point, wherein the feature points with the same growth iteration direction are classified into one category;

[0157] Among them, for linearly distributed targets such as roadsides, each initial feature point generally has two growth iteration directions, such as Figure 5b The growth iteration direction can be understood as the distribution direction of the road edge in space.

[0158] Specifically, the neighborhood feature points are classified, wherein the angle formed by the line connecting any two feature points in the same type of point set and the feature point T(i, j) is not greater than a certain threshold. The angle can be calculated based on the vector angle. Figure 6b As shown, point A is connected with points B, E, F, and G respectively. in, and The angle between and The angle between and The angle between and The angle between and The angles between points A and C are all less than a certain threshold, and the line connecting point A and point C in, and The angle between and The angle between and The angle between and The angles between them are all greater than a certain threshold, so point B, point E, point F, and point G are regarded as a set of neighborhood points corresponding to point A (such as Figure 6a The point set 1 in the point set, the point set where point C is located is regarded as another type of point set (such as Figure 6a The point set in 2).

[0159] Among them, if the initial feature point is selected at the edge of this submap, all neighborhood feature points are in the same orientation relative to the initial feature point, and there is only one growth iteration direction.

[0160] 4) Select the feature point farthest from the initial feature point in each iteration direction from the neighborhood feature points as the new iteration search point, use KNN to search for feature points within a certain range of the neighborhood of the iteration search point again, and select the feature points whose connection direction between the iteration search point and its neighborhood feature points is consistent with the current iteration direction as the feature points grown in this round of iteration.

[0161] That is to say, the neighborhood feature point set of the new iterative search point does not overlap with the neighborhood feature point set of the initial feature point.

[0162] like Figure 6a As shown, the feature point B farthest from the initial feature point in one iteration direction is used as the new iteration search point corresponding to the iteration direction, and the feature point C farthest from the initial feature point in another iteration direction is used as the new iteration search point corresponding to the iteration direction.

[0163] 5) Repeat the iterative growth process in each iterative direction in 4) until the number of feature points grown in each iterative direction is 0, thereby obtaining a feature point set for the curb instance.

[0164] 6) Repeat 2) to 5) until all the roadside feature points in the sub-map are traversed, and finally output a roadside sub-map containing multiple roadside instances, such as Figure 5c shown.

[0165] By executing the above steps on all submaps, multiple curb submaps can be obtained. Since the curbs in multiple curb submaps may be divided by the submap boundary, the curb instances can be renumbered during the submap merging process, and the instances divided by the submap boundary can be merged into the same instance.

[0166] like Figure 5d As shown, the roadside 1 of sub-map 1 and the sub-map Figure 2 The roadside 1 of submap 1 is merged into one roadside instance, and the roadside 3 of submap 1 and the submap Figure 2 The Curb2 instances are merged into one Curb instance.

[0167] By renumbering all roadside instances as a whole, all roadside sub-maps are merged to obtain a roadside map containing multiple roadside instances.

[0168] This embodiment automatically annotates the roadside instances based on the roadside feature point set in the road semantic map, and obtains a roadside map containing the roadside instances. Compared with the current manual single-frame frame-by-frame annotation, this solution greatly improves the annotation efficiency; and because the roadside feature point set in the road semantic map is directly used to directly generate the roadside data, there is no dotting error in the manual annotation process, which further improves the reliability of the annotation.

[0169] The above embodiment is described by taking the example of obtaining a roadside map including multiple roadside instances by segmentation based on a road semantic map.

[0170] As another optional implementation manner (corresponding to the above-mentioned embodiment 2), processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes:

[0171] S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold;

[0172] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold;

[0173] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set;

[0174] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0175] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, wherein the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0176] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers;

[0177] A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

[0178] The difference from the above embodiment is that this embodiment processes the roadside feature point set mapped in the laser high-precision map to obtain a roadside map.

[0179] This embodiment automatically annotates the roadside instances based on the roadside feature point set to obtain a roadside map containing the roadside instances. Compared with the current manual single-frame frame-by-frame annotation, this solution greatly improves the annotation efficiency; and because the roadside feature point set is directly used to generate the roadside data, there is no dotting error in the manual annotation process, which further improves the reliability of the annotation.

[0180] Among them, the above specific implementation can refer to the introduction in the aforementioned embodiment, which will not be repeated here.

[0181] 204. Obtain curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence.

[0182] As an optional implementation, obtaining the curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence includes:

[0183] According to the multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, a plurality of reference roadside maps are obtained by mapping the roadside maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference roadside maps correspond one-to-one to the multiple frames of laser point cloud data;

[0184] Acquire multiple preset areas from the multiple reference roadside maps, where the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference roadside maps;

[0185] Determine a score for each feature point in the plurality of preset areas according to the laser point cloud data sequence and the feature points in the plurality of preset areas;

[0186] The roadside feature annotation data corresponding to each frame of laser point cloud data is obtained according to the score of each feature point in the multiple preset areas, and the roadside feature annotation data is obtained by processing the feature points with scores not less than a sixth threshold.

[0187] Specifically, since the coordinate system of the roadside map is the world coordinate system, and the coordinate system of the laser point cloud data is the local laser coordinate system, the roadside map and the laser point cloud data sequence are spatially aligned with the vehicle body posture data sequence. curb , the vehicle posture corresponding to the laser point cloud data is [R|T] t , the parameter matrix between the laser coordinate system and the vehicle coordinate system is Then the global roadside map is mapped to the current laser coordinate system and can be expressed as:

[0188]

[0189] Among them, M curb_trans It is the roadside map mapped to the laser coordinate system.

[0190] That is to say, the roadside maps are mapped to the coordinate system of each frame of laser point cloud data, so as to obtain a plurality of the above-mentioned reference roadside maps.

[0191] Then, based on the coarse-grained extraction and the fine-grained extraction, a local roadside map is extracted for each of the above reference roadside maps.

[0192] Specifically, coarse-grained extraction is achieved in the following way: select a rectangular area in the current coordinate system, such as an area with a range of (-40m, 70m) in the x direction and a range of (-30m, 30m) in the y direction, and retain the roadside map within the rectangular area by filtering out the roadside map outside the rectangular area.

[0193] Then, the roadside map within the retained rectangular area is finely extracted. For example, the laser road surface point cloud can be obtained based on the laser semantic segmentation network or the laser road surface segmentation algorithm; a KD tree is constructed for the laser road surface point cloud, and the KNN is used to search the road surface point cloud of the nearby neighborhood for any point P in the roadside map within the rectangular area, and the number of road surface point clouds obtained by the search is recorded as Neighbor P , the scoring function is determined based on the spatial distribution of the laser road surface point cloud and the roadside map and the global distance map. The scoring function can be expressed as:

[0194] Score(P)=t*Neigh P +DisMap(P);

[0195] Among them, DisMap(P) represents the energy value of point P in the global distance map. The closer to the origin of the laser coordinate system, the higher the energy value, and vice versa. t can be a preset positive number, etc.

[0196] By calculating the scores of the feature points in the roadside map within the rectangular area, the feature points with scores greater than a certain threshold are retained, and the roadside feature annotation data corresponding to each frame of laser point cloud data can be obtained based on the feature points with scores greater than a certain threshold.

[0197] like Figure 7 As shown, as an optional implementation method, after obtaining feature points with scores greater than a certain threshold, the screened roadside map points can be evenly sampled to obtain roadside feature point annotation data.

[0198] As another optional implementation, the curb map points with scores greater than a certain threshold are mapped to a two-dimensional grid map of the xy plane of the current coordinate system to obtain curb occupancy grid annotation data.

[0199] As another optional implementation method, according to feature points with scores greater than a certain threshold, laser point cloud data in the roadside grid annotation data is extracted to obtain the roadside original point cloud semantic annotation data.

[0200] The above description only uses three types of curb feature annotation data as examples. It can also be other data, and this solution does not make specific limitations on this.

[0201] In this embodiment, the roadside map and the single-frame laser point cloud data are combined to obtain the score of each feature point, and then the roadside annotation data in a specific data form that matches the single-frame original laser point cloud data is screened out. This method has high annotation efficiency and has stronger global consistency compared to manual frame-by-frame annotation.

[0202] It should be noted that the thresholds in the embodiments of the present application can be set arbitrarily and this solution does not make any specific limitations.

[0203] In the embodiment of the present application, a laser high-precision map is obtained based on a vehicle posture data sequence and a laser point cloud data sequence, and a roadside feature point set is obtained according to the laser high-precision map and the laser point cloud data sequence; each feature point in the roadside feature point set is then processed to obtain a roadside map containing multiple roadside instances; and roadside feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence is obtained according to the roadside map, the laser point cloud data sequence and the vehicle posture data sequence. With this method, since the roadside map is a necessary condition for the automatic generation of a single frame of roadside data, the roadside data in the roadside map has better consistency in the world coordinate system than the roadside data generated frame by frame, and the roadside map data obtained using multiple frames of data has better data quality in the obstacle occlusion area, which improves the confidence and accuracy of the roadside annotation data; the roadside annotation data is obtained based on the roadside map and each frame of laser point cloud data. Compared with the existing manual annotation method, this solution effectively improves the data annotation efficiency while ensuring the accuracy of the roadside data, and at the same time improves the prediction performance of the model.

[0204] Reference Figure 8 As shown, a roadside data labeling device provided in an embodiment of the present application includes an acquisition module 801 and a processing module 802, which are specifically as follows:

[0205] An acquisition module 801 is used to acquire a body posture data sequence of the vehicle and a laser point cloud data sequence containing the surrounding environment information of the vehicle;

[0206] The processing module 802 is used to:

[0207] Obtaining a laser high-precision map according to the vehicle body posture data sequence and the laser point cloud data sequence;

[0208] Obtain a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence;

[0209] Processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances;

[0210] According to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence, the curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence is obtained.

[0211] As an optional implementation method, the processing module 802 is used to: perform semantic segmentation processing on the laser point cloud data sequence to obtain laser point cloud semantic information; obtain a road semantic map based on the laser point cloud semantic information and the laser high-precision map; extract curb feature points on the road semantic map to obtain a curb feature point set in the road semantic map.

[0212] As an optional implementation method, the road semantic map includes a first type of point cloud and a second type of point cloud, and the processing module 802 is also used to: map the road semantic map into a two-dimensional grid map to obtain a mapped road semantic map; obtain a candidate grid from the mapped road semantic map, the candidate grid is a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold; obtain a roadside feature point set based on the first type of point cloud and the second type of point cloud in the candidate grid.

[0213] As an optional implementation, the processing module 802 is further used to: perform iterative search processing on feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein, for the pth submap of the K submaps, execute steps S1-S6:

[0214] S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold;

[0215] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold;

[0216] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set;

[0217] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0218] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is all 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0219] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers;

[0220] A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

[0221] As another optional implementation method, the processing module 802 is used to: perform semantic feature extraction on each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence; map the roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

[0222] As an optional implementation, the processing module 802 is further configured to:

[0223] S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold;

[0224] S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold;

[0225] S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set;

[0226] S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j);

[0227] S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, wherein the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1);

[0228] S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers;

[0229] A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

[0230] As an optional implementation, the processing module 802 is also used to: obtain multiple reference curb maps based on multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, the multiple reference curb maps are obtained by mapping the curb maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference curb maps correspond one-to-one to the multiple frames of laser point cloud data; obtain multiple preset areas from the multiple reference curb maps, the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference curb maps; determine the score of each feature point in the multiple preset areas based on the laser point cloud data sequence and the feature points in the multiple preset areas; obtain curb feature annotation data corresponding to each frame of laser point cloud data based on the score of each feature point in the multiple preset areas, and the curb feature annotation data is obtained by processing the feature points whose scores are not less than a sixth threshold.

[0231] It should be noted that the above Figure 8 The acquisition module 801 and the processing module 802 shown are used to execute the relevant steps of the above-mentioned roadside data labeling method.

[0232] In this embodiment, the roadside data labeling device is presented in the form of a module. Here, the "module" may refer to an application-specific integrated circuit (ASIC), a processor and memory that executes one or more software or firmware programs, an integrated logic circuit, and / or other devices that can provide the above functions.

[0233] In addition, the above acquisition module 801 and processing module 802 can be Fig. 9 The processor 902 of the roadside data labeling device shown is implemented.

[0234] Fig. 9 It is a schematic diagram of the hardware structure of the roadside data labeling device provided in an embodiment of the present application. Fig. 9 The roadside data labeling device 900 shown (the device 900 may be a computer device) includes a memory 901, a processor 902, a communication interface 903 and a bus 904. The memory 901, the processor 902 and the communication interface 903 are connected to each other through the bus 904.

[0235] The memory 901 may be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM).

[0236] The memory 901 can store programs. When the program stored in the memory 901 is executed by the processor 902, the processor 902 and the communication interface 903 are used to execute the various steps of the roadside data labeling method of the embodiment of the present application.

[0237] The processor 902 can adopt a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by the units in the curb data labeling device of the embodiment of the present application, or to execute the curb data labeling method of the method embodiment of the present application.

[0238] The processor 902 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the roadside data labeling method of the present application may be completed by an integrated logic circuit of hardware or software instructions in the processor 902. The above-mentioned processor 902 may also be a general-purpose processor, a digital signal processor (DigitalSignal Processing, DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (Field ProgrammableGate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 901, and the processor 902 reads the information in the memory 901, and combines its hardware to complete the functions required to be performed by the units included in the curb data labeling device of the embodiment of the present application, or executes the curb data labeling method of the method embodiment of the present application.

[0239] The communication interface 903 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the device 900 and other devices or a communication network. For example, data can be obtained through the communication interface 903 .

[0240] The bus 904 may include a path for transmitting information between various components of the device 900 (eg, the memory 901 , the processor 902 , and the communication interface 903 ).

[0241] It should be noted that although Fig. 9 The device 900 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the device 900 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the device 900 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the device 900 may also only include the devices necessary to implement the embodiments of the present application, and does not necessarily include Fig. 9 All devices shown in .

[0242] The present application provides a chip system, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes a method provided in any possible implementation manner of the first aspect.

[0243] An embodiment of the present application also provides an intelligent driving vehicle, including a travel system, a sensing system, a control system, and a computer system, wherein the computer system is used to execute a method provided in any possible implementation manner of the first aspect.

[0244] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0245] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0246] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the specific descriptions of the corresponding steps in the aforementioned method embodiments, and will not be repeated here.

[0247] It should be understood that in the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; wherein A and B can be singular or plural. Also, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, wherein a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be necessarily different. Meanwhile, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0248] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the division of the unit is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling, direct coupling, or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0249] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0250] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. The available medium may be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD), etc.

[0251] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A method for labeling roadside data, characterized in that: include: Acquire a body posture data sequence of the vehicle and a laser point cloud data sequence containing information about the surrounding environment of the vehicle, and obtain a laser high-precision map based on the body posture data sequence and the laser point cloud data sequence; Obtain a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence; Processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances; Obtaining curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence; The step of obtaining the curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence comprises: According to the multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, a plurality of reference roadside maps are obtained by mapping the roadside maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference roadside maps correspond one-to-one to the multiple frames of laser point cloud data; Acquire multiple preset areas from the multiple reference roadside maps, where the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference roadside maps; Determine a score for each feature point in the plurality of preset areas according to the laser point cloud data sequence and the feature points in the plurality of preset areas; The roadside feature annotation data corresponding to each frame of laser point cloud data is obtained according to the score of each feature point in the multiple preset areas, and the roadside feature annotation data is obtained by processing the feature points with scores not less than a sixth threshold.

2. The method according to claim 1, characterized in that The step of obtaining a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence includes: Processing the laser point cloud data sequence to obtain laser point cloud semantic information; Obtaining a road semantic map according to the laser point cloud semantic information and the laser high-precision map; The roadside feature points are extracted from the road semantic map to obtain a roadside feature point set in the road semantic map.

3. The method according to claim 2, characterized in that The road semantic map includes a first type of point cloud and a second type of point cloud, and the extraction of roadside feature points from the road semantic map to obtain a roadside feature point set in the road semantic map includes: Mapping the road semantic map into a two-dimensional grid map to obtain a mapped road semantic map; Acquire a candidate grid from the mapped road semantic map, wherein the candidate grid is a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold; A set of roadside feature points is obtained according to the first type of point cloud and the second type of point cloud in the candidate grid.

4. The method according to claim 2 or 3, characterized in that: The step of processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes: Iteratively search the feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein for the pth submap in the K submaps, execute steps S1-S6: S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold; S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold; S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set; S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j); S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1); S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers; A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

5. The method according to claim 1, characterized in that The step of obtaining a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence includes: Performing semantic feature extraction on each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence; The roadside feature point set of each frame of laser point cloud data in the laser point cloud data sequence is mapped to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

6. The method according to claim 5, characterized in that The step of processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances includes: S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold; S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold; S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set; S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j); S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1); S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers; A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

7. A roadside data labeling device, characterized in that: include: An acquisition module, used to acquire a body posture data sequence of the vehicle and a laser point cloud data sequence containing the surrounding environment information of the vehicle; Processing modules for: Obtaining a laser high-precision map according to the vehicle body posture data sequence and the laser point cloud data sequence; Obtain a roadside feature point set according to the laser high-precision map and the laser point cloud data sequence; Processing each feature point in the roadside feature point set to obtain a roadside map containing multiple roadside instances; Obtaining curb feature annotation data corresponding to each frame of laser point cloud data in the laser point cloud data sequence according to the curb map, the laser point cloud data sequence and the vehicle body posture data sequence; The processing module is further used for: According to the multiple frames of laser point cloud data in the laser point cloud data sequence and the vehicle body posture data sequence, a plurality of reference roadside maps are obtained by mapping the roadside maps in the coordinate systems where the multiple frames of laser point cloud data are respectively located, and the multiple reference roadside maps correspond one-to-one to the multiple frames of laser point cloud data; Acquire multiple preset areas from the multiple reference roadside maps, where the size of the preset areas is not less than the size of each frame of laser point cloud data, and the multiple preset areas correspond one-to-one to the multiple reference roadside maps; Determine a score for each feature point in the plurality of preset areas according to the laser point cloud data sequence and the feature points in the plurality of preset areas; The roadside feature annotation data corresponding to each frame of laser point cloud data is obtained according to the score of each feature point in the multiple preset areas, and the roadside feature annotation data is obtained by processing the feature points with scores not less than a sixth threshold.

8. The device according to claim 7, characterized in that The processing module is used for: Processing the laser point cloud data sequence to obtain laser point cloud semantic information; Obtaining a road semantic map according to the laser point cloud semantic information and the laser high-precision map; The roadside feature points are extracted from the road semantic map to obtain a roadside feature point set in the road semantic map.

9. The device according to claim 8, characterized in that The road semantic map includes a first type of point cloud and a second type of point cloud, and the processing module is further used to: Mapping the road semantic map into a two-dimensional grid map to obtain a mapped road semantic map; Acquire a candidate grid from the mapped road semantic map, wherein the candidate grid is a grid having the first type of point cloud and the second type of point cloud, and the height difference between the first type of point cloud and the second type of point cloud in the road semantic map is not greater than a first threshold; A set of roadside feature points is obtained according to the first type of point cloud and the second type of point cloud in the candidate grid.

10. The device according to claim 8 or 9, characterized in that The processing module is further used for: Iteratively search the feature points in the roadside feature point set of K submaps to obtain feature point sets of multiple roadside instances, wherein the K submaps are obtained by segmenting the road semantic map, and K is an integer not less than 2; wherein for the pth submap in the K submaps, execute steps S1-S6: S1. Acquire a feature point T(i, j) from the p-th submap, wherein the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a second threshold; S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a third threshold; S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a fourth threshold from the at least one point set; S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance from the feature point T(i,j+1) is not less than a fifth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j); S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the p-th submap, the feature point set of the i-th roadside instance in the p-th submap includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1); S6, let i=i+1, and repeat steps S1-S5 until the feature points in the p-th submap are traversed to obtain a feature point set of each roadside instance in the p-th submap, wherein the p-th submap is any one of the K submaps, and i and j are both positive integers; A roadside map including multiple roadside instances is obtained according to the feature point sets of multiple roadside instances in the K sub-maps.

11. The device according to claim 7, characterized in that The processing module is used for: Performing semantic feature extraction on each frame of laser point cloud data in the laser point cloud data sequence to obtain a roadside feature point set for each frame of laser point cloud data in the laser point cloud data sequence; The roadside feature point set of each frame of laser point cloud data in the laser point cloud data sequence is mapped to the laser high-precision map to obtain a roadside feature point set corresponding to the coordinate system of the laser high-precision map.

12. The device according to claim 11, characterized in that The processing module is further used for: S1. Acquire a feature point T(i, j) from a road feature point set corresponding to the coordinate system where the laser high-precision map is located, and the number of points in the neighborhood feature point set corresponding to the feature point T(i, j) is not less than a first threshold; S2. Obtain at least one point set according to the neighborhood feature point set corresponding to the feature point T(i, j) and the feature point T(i, j), wherein the angle formed by the line connecting any two feature points in each point set of the at least one point set and the feature point T(i, j) is not greater than a second threshold; S3, determining a feature point T(i, j+1) whose distance to the feature point T(i, j) is not less than a third threshold from the at least one point set; S4, obtaining a neighborhood feature point set corresponding to the feature point T(i,j+1), and determining a feature point T(i,j+2) whose distance to the feature point T(i,j+1) is not less than a fourth threshold from the neighborhood feature point set corresponding to the feature point T(i,j+1), wherein the neighborhood feature point set corresponding to the feature point T(i,j+1) does not overlap with the neighborhood feature point set corresponding to the feature point T(i,j); S5, let j=j+1, repeat step S4 until the number of points in the neighborhood feature point set corresponding to the feature point T(i,j+1) is 0, and obtain the feature point set of the i-th roadside instance in the roadside feature point set, the feature point set of the i-th roadside instance includes the feature point T(i,j), the neighborhood feature point set corresponding to the feature point T(i,j), the feature point T(i,j+1), and the neighborhood feature point set corresponding to the feature point T(i,j+1); S6, let i=i+1, and repeat steps S1-S5 until the feature points in the curb feature point set are traversed to obtain a feature point set for each curb instance in the curb feature point set, where i and j are both positive integers; A roadside map containing multiple roadside instances is obtained according to the feature point set of each roadside instance in the roadside feature point set.

13. A roadside data labeling device, characterized in that: It comprises a processor and a memory; wherein the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 6.

15. A computer program product, characterized in that When the computer program product runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

16. A chip system, characterized in that: The chip system is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method as described in any one of claims 1 to 6.

17. An intelligent driving vehicle, characterized in that: The invention comprises a traveling system, a sensing system, a control system and a computer system, wherein the computer system is used to execute the method according to any one of claims 1 to 6.

Images