Map element annotation method, device, equipment, medium and program product
By filtering and labeling map elements close to the location of the collected data in the lidar point cloud data and highly matched map elements, the problem of low accuracy of manual labeling in the prior art is solved, and more efficient and accurate map element labeling is achieved.
Patent Information
- Application Number
- CN202411081196.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-08
AI Technical Summary
In the prior art, map element labeling relies on manual operations, resulting in low labeling accuracy.
By generating a filter area based on the acquisition data location of the lidar point cloud data, combining the element coordinates in the vector data of the map element and the position distribution relationship of the screening area, the map elements are filtered to obtain a close and highly matched position from the acquisition data. Map elements and label them in lidar point cloud data.
Improve the accuracy and efficiency of map element annotation, reduce manual intervention, and enhance the reliability of the annotation.
Smart Images

Figure CN118608732B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a map element annotation method, device, equipment, medium and program product. Background Art
[0002] During the unmanned driving process, the laser radar installed on the vehicle will collect laser radar point cloud data. Then, it is necessary to generate map elements in the laser radar point cloud data, that is, to mark the map elements so that the vehicle can control the vehicle according to the marked content. The process of marking map elements is to use a neural network model to achieve the marking, and the neural network model is trained using a training data set. The data in the training data set is the collected laser radar point cloud data with the marked map elements.
[0003] In the prior art, the generation of a training data set is usually done by manually determining the positions of map elements based on the lidar point cloud data, and then marking the map elements in the lidar point cloud data to form a training data set.
[0004] In summary, the existing map element annotation methods are performed manually, resulting in low annotation accuracy. Summary of the invention
[0005] The embodiments of the present application provide a map element annotation method, apparatus, device, medium and program product to solve the problem of low annotation accuracy caused by manually annotating map elements in lidar point cloud data when constructing a training data set in the prior art.
[0006] In a first aspect, an embodiment of the present application provides a map element annotation method, comprising:
[0007] According to the data collection position of the laser radar point cloud data, a screening area corresponding to the data collection position is generated;
[0008] Filtering the plurality of map elements according to the element coordinates in the vector data of the plurality of map elements and the position distribution relationship of the filtering area to obtain at least one target map element;
[0009] According to the element coordinates of the at least one target map element, the target map element is marked in the lidar point cloud data.
[0010] In a possible implementation, the screening of the plurality of map elements according to the element coordinates in the vector data of the plurality of map elements and the position distribution relationship of the screening area to obtain at least one target map element includes:
[0011] Eliminate all map elements whose element coordinates are outside the screening area from among the multiple map elements to obtain a first map element;
[0012] According to the screening area, the element coordinates of each first map element are corrected to obtain an element coordinate set of each first map element;
[0013] The first map elements are screened according to the element coordinate set of each first map element to obtain at least one target map element.
[0014] In a possible implementation, the step of screening the first map elements according to the element coordinate set of each first map element to obtain at least one target map element includes:
[0015] Determining a first preset number of lane centerlines closest to the data collection location from the first map element, and determining a height screening range based on the first preset number of lane centerlines;
[0016] The first map elements are screened according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element.
[0017] In a possible implementation, the correcting the element coordinates of each of the first map elements according to the screening area to obtain the element coordinate set of the first map elements includes:
[0018] Performing graphic fitting according to the element coordinates of each first map element to obtain a graphic of each first map element;
[0019] For each first map element, the element coordinates of the first map element within the screening area and the coordinates of the intersection of the graphic of the first map element and the boundary of the screening area constitute an element coordinate set of the first map element.
[0020] In a possible implementation, determining the height screening range based on the first preset number of lane centerlines includes:
[0021] For each lane centerline of the first preset number of lane centerlines, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline;
[0022] Selecting a second preset number of interpolation points closest to the location of the collected data from the multiple interpolation points to obtain a target interpolation point of the lane centerline;
[0023] The height screening range is determined according to an average height of the target interpolation points of the first preset number of lane center lines.
[0024] In a possible implementation, the filtering of the first map elements according to the height filtering range and the element coordinate set of each first map element to obtain at least one target map element includes:
[0025] For each lane centerline in all first map elements, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline;
[0026] For each lane centerline in all first map elements, establishing a correspondence between the first map element corresponding to the lane centerline and the interpolation points according to the coordinates of the plurality of interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline;
[0027] The first map elements are screened according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, and the height screening range, to obtain at least one target map element.
[0028] In a possible implementation, establishing a correspondence between the first map element corresponding to the lane centerline and the interpolation points according to the coordinates of the plurality of interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline includes:
[0029] Calculate the distance between each first map element corresponding to the lane centerline and each interpolation point according to the coordinates of the plurality of interpolation points corresponding to the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline;
[0030] For each first map element corresponding to the lane centerline, an interpolation point among the multiple interpolation points that is closest to the first map element is established in a corresponding relationship with the first map element.
[0031] In a possible implementation, the first map elements are screened according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, and the height screening range, to obtain at least one target map element, including:
[0032] According to the correspondence between the first map element corresponding to each lane centerline in all the first map elements and the interpolation point, the first map element corresponding to the interpolation point whose height is not within the height screening range is taken as the first element to be deleted;
[0033] For each lane centerline in all first map elements, if the heights of multiple interpolation points of the lane centerline are not within the height screening range, the lane centerline is used as a second element to be deleted;
[0034] Eliminate the first to-be-deleted elements and the second to-be-deleted elements from all the first map elements to obtain a second map element;
[0035] For each second map element, deleting the element coordinates of the element coordinate set of the second map element whose height is not within the height screening range, to obtain a target map element and a target element coordinate set of the target map element;
[0036] The step of marking the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element comprises:
[0037] The target map element is marked in the lidar point cloud data according to the target element coordinate set of the at least one target map element.
[0038] In a possible implementation, the method further includes:
[0039] Convert the element coordinates of each target map element to obtain the two-dimensional image coordinates of each target map element;
[0040] According to the two-dimensional image coordinates of each target map element, the target map element is marked in the image captured by the camera.
[0041] In a second aspect, an embodiment of the present application provides a map element annotation device, comprising:
[0042] Processing modules for:
[0043] According to the data collection position of the laser radar point cloud data, a screening area corresponding to the data collection position is generated;
[0044] Filtering the plurality of map elements according to the element coordinates in the vector data of the plurality of map elements and the position distribution relationship of the filtering area to obtain at least one target map element;
[0045] The labeling module is used to label the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0047] Processor, memory, communication interface;
[0048] The memory is used to store executable instructions of the processor;
[0049] Wherein, the processor is configured to execute the map element labeling method described in any one of the first aspects by executing the executable instructions.
[0050] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the map element labeling method described in any one of the first aspects.
[0051] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the map element labeling method described in any one of the first aspects.
[0052] The embodiments of the present application provide a method, device, equipment, medium and program product for annotating map elements. After generating a screening area according to the location of the collected data of the laser radar point cloud data, multiple map elements are screened according to the element coordinates in the vector data of the multiple map elements and the position distribution relationship of the screening area to obtain at least one target map element, and then the target map element is annotated in the laser radar point cloud data according to the element coordinates of the at least one target map element. This solution screens the determined map elements according to the screening area corresponding to the location of the collected data. The screened map elements are closer to the data collection location and are map elements in the laser radar point cloud data. Then, the map elements are annotated according to the element coordinates of the screened map elements, which effectively improves the accuracy of the annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0054] Figure 1a A flowchart of the first embodiment of the map element annotation method provided in this application;
[0055] Figure 1b Schematic diagram of the collection scenario provided for this application;
[0056] Figure 1c A schematic diagram of the target interpolation points provided for this application;
[0057] Figure 1d A schematic diagram of the laser radar point cloud data for marking target map elements provided for this application;
[0058] Figure 1e A schematic diagram of the laser radar point cloud data annotated with the first map element provided in this application;
[0059] Figure 2a A flowchart of the second embodiment of the map element annotation method provided in this application;
[0060] Figure 2b A schematic diagram of the correspondence between the first map element and the interpolation point provided in this application;
[0061] Figure 3a A flowchart of the third embodiment of the map element annotation method provided in this application;
[0062] Figure 3b Image diagram 1 of the labeled map elements provided for this application;
[0063] Figure 3c Image diagram 2 of the labeled map elements provided for this application;
[0064] Figure 4 A schematic diagram of the structure of an embodiment of a map element annotation device provided in this application;
[0065] Figure 5 A schematic diagram of the structure of an electronic device provided in this application.
[0066] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0068] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0069] In the field of unmanned driving, after the vehicle collects lidar point cloud data, the map elements are annotated on the lidar point cloud data to form a training data set, and then the training data set is used for model training to obtain a neural network model. In order for the unmanned vehicle to subsequently input the collected lidar point cloud data into the neural network model to obtain lidar point cloud data annotated with map elements, and then control the vehicle's driving.
[0070] In the prior art, the generation of training data sets usually involves manually determining the locations of map elements based on LiDAR point cloud data, and then annotating map elements in the LiDAR point cloud data to form a training data set. Since everyone has a different understanding of LiDAR point cloud data, this can result in low annotation accuracy.
[0071] In view of the problems existing in the prior art, the inventors found in the process of studying the map element annotation method that in order to improve accuracy, a screening area can be generated according to the acquisition data position of the laser radar point cloud data, and the map elements can be screened in combination with the element coordinates of the vector data of the determined map elements to obtain map elements that are close to the acquisition data position of the laser radar point cloud data and whose height is in the same height range as the acquisition data position, and then the map elements are annotated in the laser radar point cloud data according to the element coordinates of the screened map elements. Based on the above invention concept, the map element annotation scheme in this application is designed.
[0072] The executor of the map element annotation method in the present application may be a computer, or a server, a terminal device, a vehicle, etc. The present application does not limit it and the following description will be made using a computer as an example.
[0073] The following is an example of an application scenario of the map element annotation method provided by this application.
[0074] For example, in one application scenario, the computer has acquired vector data of multiple map elements, and the vector data of each map element includes element coordinates. In order to construct a training data set, the vehicle collects lidar point cloud data at the data collection location, and then sends the data collection location and its corresponding collected lidar point cloud data to the computer, and the computer annotates the map elements on the lidar point cloud data.
[0075] The computer generates a screening area corresponding to the data collection location according to the data collection location of the laser radar point cloud data, removes all map elements whose coordinates are outside the screening area among the multiple map elements, and obtains a first map element.
[0076] Then, the element coordinates of the first map element are corrected to obtain an element coordinate set of the first map element.
[0077] A first preset number of lane centerlines closest to the data collection location are determined from the first map element, and a height screening range is determined based on the first preset number of lane centerlines.
[0078] The first map elements are then screened according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element and a target element coordinate set of the target map element.
[0079] The target map elements are annotated in the lidar point cloud data according to the target element coordinate set of the target map elements.
[0080] The computer then uses the laser radar point cloud data of multiple labeled target map elements to form a training data set, and then conducts model training to obtain a neural network model.
[0081] For example, in another application scenario, the computer has acquired vector data of multiple map elements, each of which includes element coordinates. The vector data of the multiple map elements are collected when the vehicle is traveling in the data collection area.
[0082] The unmanned vehicle drives in the collection area and sends the collected LiDAR point cloud data and the collected data location of the vehicle to the computer.
[0083] The computer generates a screening area corresponding to the data collection location according to the data collection location of the laser radar point cloud data. Then, according to the element coordinates in the vector data of the multiple map elements and the position distribution relationship of the screening area, the multiple map elements are screened to obtain at least one target map element; and then, according to the element coordinates of the at least one target map element, the target map element is marked in the laser radar point cloud data.
[0084] The computer sends the laser radar point cloud data of the marked target map elements to the unmanned vehicle, and the unmanned vehicle controls the vehicle driving according to the laser radar point cloud data of the marked target map elements.
[0085] It should be noted that the above scenarios are only examples of application scenarios provided by the embodiments of the present application. The embodiments of the present application do not limit the actual forms of the various devices included in the scenarios, nor do they limit the interaction methods between the devices. In the specific application of the solution, it can be set according to actual needs.
[0086] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0087] Figure 1a This is a flowchart of the first embodiment of the map element annotation method provided by the present application. The present application embodiment describes the situation in which a computer screens multiple map elements, obtains a target map element, and then annotates the target map element in the laser radar point cloud data according to the element coordinates of the target map element. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. Figure 1a As shown, the map element marking method specifically includes the following steps:
[0088] S101: Generate a screening area corresponding to the data collection location according to the data collection location of the laser radar point cloud data.
[0089] In this step, in order to complete the labeling of map elements for the lidar point cloud data, the computer obtains the collection data location of the lidar point cloud data and the vector data of multiple map elements, and then generates a screening area corresponding to the collection data location based on the collection data location.
[0090] The vector data of each map element includes element coordinates. The location of the collected data is represented by three-dimensional coordinates, and the element coordinates are three-dimensional coordinates.
[0091] It should be noted that when the coordinate systems of the acquisition data position and the element coordinates are different, at least one of them is transformed so that the coordinate systems of the two are the same.
[0092] It should be noted that the map elements may be one or any combination of lane center lines, lane lines, ground signs, signboards, lamp poles, etc. The embodiment of the present application does not limit the ground elements and may be determined according to actual conditions.
[0093] It should be noted that the vector data of each map element also includes one or any combination of element identification, element type, element color identification, collection time, collection parameters, etc. The embodiment of the present application does not limit the data included in the vector data, which can be determined according to actual conditions.
[0094] It should be noted that the screening area is an area including the location of the collected data. The screening area can be a cylindrical area with the location of the collected data as the center, a preset radius length as the radius, and a preset height as the height; it can also be a spherical area with the location of the collected data as the center and a preset radius length as the radius; it can also be a regular polyhedron area with the location of the collected data as the center and a preset side length as the side length. The preset radius length, preset side length, and preset height can be 5 meters, 10 meters, 30 meters, 50 meters, 70 meters, 150 meters, etc. The embodiment of the present application does not limit the screening area, preset radius length, preset side length, and preset height, and can be determined according to actual conditions.
[0095] S102: Filtering multiple map elements according to the element coordinates in the vector data of the multiple map elements and the position distribution relationship of the filtering area to obtain at least one target map element.
[0096] In this step, after the computer generates the screening area corresponding to the location of the collected data, the multiple map elements are screened according to the element coordinates in the vector data of the multiple map elements and the position distribution relationship of the screening area to obtain at least one target map element.
[0097] Specifically, the map elements whose coordinates of all elements in the multiple map elements are outside the screening area are removed to obtain the first map element. The number of element coordinates of each map element is multiple, and all element coordinates of a map element are outside the screening area, indicating that the map element is far away from the location of the collected data, and there is no point cloud of the map element in the collected LiDAR point cloud data. There is no need to mark the map element in the LiDAR point cloud data, so it is removed.
[0098] After obtaining the first map element, since each map element has multiple element coordinates, some of the element coordinates of the first map elements are not within the screening area. For the subsequent annotation effect, it is necessary to correct the element coordinates of each first map element according to the screening area to obtain the element coordinate set of the first map element.
[0099] First, graphic fitting is performed on the element coordinates of each first map element to obtain the graphic of each first map element.
[0100] Furthermore, for each first map element, the element coordinates of the first map element within the screening area and the coordinates of the intersection of the graphic of the first map element and the boundary of the screening area constitute an element coordinate set of the first map element.
[0101] In order to improve the annotation effect, the coordinates of the intersection of the graphic of the first map element and the boundary of the screening area are used as the element coordinates of the first map element.
[0102] It should be noted that the vector data of each map element also includes an element type, which is a line or a polygon. When the element type of the first map element is a polygon, a plurality of interpolation points are obtained by interpolation based on the intersection of the graphic and the boundary of the screening area, and then the coordinates of the interpolation points are used as the element coordinates of the first map element and added to the element coordinate set of the first map element.
[0103] After obtaining the element coordinate set of the first map element, in some scenes, some first map elements are not in the same height range as the data acquisition position, and there is no point cloud data of these first map elements in the lidar point cloud data. There is no need to mark these first map elements in the lidar point cloud data, and they need to be removed.
[0104] For example, Figure 1b The collection scene diagram provided for this application is as follows: Figure 1b As shown, the data collection location A is located under the viaduct, and there are signboards and lane lines on the viaduct. There is no point cloud of the signboards and lane lines in the lidar point cloud data collected at point A, and there is no need to mark the signboards and lane lines in the lidar point cloud data.
[0105] Therefore, according to the element coordinate set of each first map element, the first map elements are screened to obtain at least one target map element. A first preset number of lane centerlines closest to the data collection location are first determined from the first map element, and a height screening range is determined based on the first preset number of lane centerlines.
[0106] Straight line fitting or curve fitting can be performed based on the element coordinate set of each lane centerline in the first map element to obtain a fitting line for each lane centerline, and then the distance between the collected data position and the fitting line of each lane centerline is calculated, so as to select the lane centerline corresponding to the first preset number of fitting lines with the smallest distance to the collected data position.
[0107] It should be noted that the first preset number can be 1, 2, 3, etc. The embodiment of the present application does not limit the first preset number and can be set according to actual conditions.
[0108] For each lane centerline in the first preset number of lane centerlines, an equidistant interpolation process is performed on the lane centerline according to the element coordinate set of the lane centerline to obtain multiple interpolation points of the lane centerline. A second preset number of interpolation points closest to the data collection location is selected from the multiple interpolation points to obtain a target interpolation point of the lane centerline.
[0109] It should be noted that a method of selecting a second preset number of interpolation points closest to the data collection location from multiple interpolation points can be to store the coordinates of the interpolation points in the form of a K-dimensional tree (kd tree for short), and then determine the second preset number of interpolation points closest to the data collection location using a K nearest neighbor search algorithm.
[0110] It should be noted that the distance between multiple interpolation points of each lane centerline can be 5 meters, 10 meters, 15 meters, etc. The embodiment of the present application does not limit the distance between multiple interpolation points of each lane centerline, which can be determined according to actual conditions.
[0111] It should be noted that the second preset number can be 2, 3, 5, etc. The embodiment of the present application does not limit the second preset number and can be set according to actual conditions.
[0112] The height screening range is determined based on the average height of the target interpolation points of the first preset number of lane center lines. The average height is used as the height of the ground when the vehicle collects the laser radar point cloud data. The height screening range is determined based on the average height, making the height screening range more reasonable.
[0113] The height screening range is [HX, H+X], where H is the average height and X is the preset height difference. The preset height difference can be 3 meters, 5 meters, 10 meters, etc. The embodiment of the present application does not limit the preset height difference and can be set according to actual conditions.
[0114] For example, Figure 1c A schematic diagram of the target interpolation point provided for this application, such as Figure 1c As shown, the triangle represents the location of the collected data, the solid line is the lane centerline, and the first preset number and the second preset number are both 2. The figure includes lane centerline A and lane centerline B, which are the two lane centerlines closest to the location of the collected data. The points on the line are interpolation points, among which point A and point B are the target interpolation points of lane centerline A, and point C and point D are the target interpolation points of lane centerline B. According to the average height of point A, point B, point C and point D, the height screening range is determined.
[0115] In one implementation, the height screening range can be determined according to the height of the data collection location, and the height screening range is [LX, L+X], where L is the height of the data collection location, and X is a preset height difference. The preset height difference can be 3 meters, 5 meters, 10 meters, etc. The embodiment of the present application does not limit the preset height difference and can be set according to actual conditions.
[0116] After the height screening range is determined, the first map elements are screened according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element.
[0117] That is, the first map elements whose heights do not match the height screening range are eliminated, and the corresponding element coordinate sets of the remaining first map elements are updated to obtain the target map element and the target element coordinate set of the target map element.
[0118] S103: Marking a target map element in the laser radar point cloud data according to the element coordinates of at least one target map element.
[0119] In this step, after obtaining at least one target map element, the computer marks the target map element in the lidar point cloud data according to the element coordinates of the at least one target map element.
[0120] That is, for each target map element, the target map element is generated in the lidar point cloud data according to its target element coordinate set, and then the vector data of the target map element is associated with the generated target map element.
[0121] For example, Figure 1d A schematic diagram of the laser radar point cloud data for marking target map elements provided for this application, such as Figure 1d As shown, point A represents the location where data was collected. The black solid line in the figure and the closed figure formed by the black solid line represent the target map elements marked in the lidar point cloud data. It can be seen that the straight-ahead signs, lane lines, and lane center lines on the ground that are closer to point A are marked, and the map elements on the viaduct are not marked.
[0122] For example, Figure 1e A schematic diagram of the laser radar point cloud data for marking the first map element provided in this application, such as Figure 1e As shown, Figure 1d In contrast, the lane lines, lane center lines and signage on the viaduct are also marked in this figure, and these map elements do not have corresponding point clouds in the lidar point cloud data. Filtering the first map element by the height screening range can effectively improve the accuracy of the marked map elements.
[0123] The map element annotation method provided in this embodiment screens multiple map elements according to the data collection location of the laser radar point cloud data and the vector data of multiple map elements to obtain at least one target map element, and then annotates the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element. This solution screens through the determined map elements, and then annotates the map elements according to the element coordinates of the screened map elements, without manual annotation, which effectively improves the annotation accuracy and also improves the annotation efficiency.
[0124] Figure 2a This is a flow chart of the second embodiment of the map element annotation method provided by the present application. Based on the above embodiment, the present embodiment of the present application describes the situation in which a computer screens the first map element to obtain the target map element according to the height screening range and the element coordinate set of each first map element. Figure 2a As shown, the map element marking method specifically includes the following steps:
[0125] S201: For each lane center line in all first map elements, perform equidistant interpolation processing on the lane center line according to the element coordinate set of the lane center line to obtain multiple interpolation points of the lane center line.
[0126] In this step, after the computer determines the height screening range, in order to improve the screening efficiency, for each lane center line in all the first map elements, the lane center line is equidistantly interpolated according to the element coordinate set of the lane center line to obtain multiple interpolation points of the lane center line.
[0127] It should be noted that this step is similar to the process of determining multiple interpolation points of the lane centerline in step S101 of the first embodiment, and will not be described in detail here.
[0128] S202: For each lane centerline in all first map elements, a correspondence between the first map element corresponding to the lane centerline and the interpolation points is established according to the coordinates of multiple interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline.
[0129] In this step, after the computer determines multiple interpolation points of each lane centerline, for each lane centerline in all first map elements, a corresponding relationship between the first map element corresponding to the lane centerline and the interpolation points is established based on the coordinates of the multiple interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline.
[0130] Specifically, according to the coordinates of multiple interpolation points corresponding to the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline, the distance between each first map element corresponding to the lane centerline and each interpolation point is calculated.
[0131] Since the computer stores the correspondence between each lane centerline and other map elements, indicating the map elements that exist on the lane where the lane centerline is located except for the lane centerline, the first map element corresponding to the lane centerline can be determined.
[0132] It should be noted that the method for calculating the distance between a first map element and an interpolation point is: first calculate the average coordinates of all element coordinate points in the element coordinate set of the first map element, and then calculate the distance based on the average coordinates and the coordinates of the interpolation point.
[0133] For each first map element corresponding to the lane centerline, a corresponding relationship is established between an interpolation point among the multiple interpolation points that is closest to the first map element and the first map element.
[0134] For example, Figure 2b A schematic diagram of the correspondence between the first map element and the interpolation point provided in this application, such as Figure 2b As shown, there are two lane center lines in the figure, namely lane center line A and lane center line B. There are interpolation points A, B and C on lane center line A, and interpolation points D, E and F on lane center line B. The first map elements corresponding to lane center line A are elements 1 to 5, and the first map elements corresponding to lane center line B are elements 6 to 9.
[0135] The distance between element 1 and interpolation point A is the smallest, and a corresponding relationship is established between element 1 and interpolation point A. The distance between element 2 and interpolation point A is the smallest, and a corresponding relationship is established between element 2 and interpolation point A. The distance between element 3 and interpolation point B is the smallest, and a corresponding relationship is established between element 3 and interpolation point B. The distance between element 4 and interpolation point C is the smallest, and a corresponding relationship is established between element 4 and interpolation point C. The distance between element 5 and interpolation point C is the smallest, and a corresponding relationship is established between element 5 and interpolation point C. The distance between element 6 and interpolation point D is the smallest, and a corresponding relationship is established between element 6 and interpolation point D. The distance between element 7 and interpolation point E is the smallest, and a corresponding relationship is established between element 7 and interpolation point E. The distance between element 8 and interpolation point E is the smallest, and a corresponding relationship is established between element 8 and interpolation point E. The distance between element 9 and interpolation point F is the smallest, and a corresponding relationship is established between element 9 and interpolation point F.
[0136] S203: Filter the first map elements according to the corresponding relationship between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation point, and the height screening range, to obtain at least one target map element.
[0137] In this step, after the computer determines the correspondence between the first map element and the interpolation point, the first map elements are screened according to the correspondence between the first map element corresponding to each lane centerline in all the first map elements and the interpolation point, and the height screening range, to obtain at least one target map element.
[0138] Specifically, according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, the first map elements corresponding to the interpolation points whose heights are not within the height screening range are taken as the first elements to be deleted, indicating that there are no point clouds of these map elements in the collected lidar point cloud data and they need to be deleted.
[0139] For each lane centerline in all first map elements, if the heights of multiple interpolation points of the lane centerline are not within the height screening range, the lane centerline is taken as the second element to be deleted, indicating that there is no new point cloud of the lane center in the collected lidar point cloud data and it needs to be deleted.
[0140] The first to-be-deleted elements and the second to-be-deleted elements in all the first map elements are removed to obtain the second map element.
[0141] After obtaining the second map element, since each map element has multiple element coordinates, some of the element coordinates of the second map elements are not within the height screening range. For the subsequent annotation effect, for each second map element, the element coordinates of the element coordinate set of the second map element whose height is not within the height screening range are deleted to obtain the target map element and the target element coordinate set of the target map element.
[0142] For example, in Figure 2b Based on this, the heights of interpolation points A, B, and C are not within the height screening range, so elements 1 to 5 are deleted and lane centerline A is deleted.
[0143] Subsequently, the target map element is annotated in the lidar point cloud data according to the target element coordinate set of the at least one target map element.
[0144] In one implementation, after the height screening range is determined, for each first map element, if the height of each element coordinate in the element coordinate set of the first map element is not within the height screening range, the first map element is deleted to obtain a third map element. Then, for each third map element, the element coordinates in the element coordinate set of the third map element whose height is not within the height screening range are deleted to obtain a target map element and a target element coordinate set of the target map element.
[0145] The map element labeling method provided in this embodiment obtains the target map element by screening the first map element according to the height screening range, thereby improving the accuracy of the target map element.
[0146] Figure 3a This is a flow chart of the third embodiment of the map element annotation method provided by the present application. Based on the above embodiments, the present embodiment of the present application describes the situation where a computer annotates a target map element in an image captured by a camera. Figure 3a As shown, the map element marking method specifically includes the following steps:
[0147] S301: Convert the element coordinates of each target map element to obtain the two-dimensional image coordinates of each target map element.
[0148] When the vehicle collects lidar point cloud data, it also takes images through the camera simultaneously. This solution can annotate map elements in the images taken by the camera.
[0149] In this step, after the computer determines the target map element, in order to mark the map element in the image taken by the camera, the element coordinates of each target map element are first converted, that is, the element coordinates in the target element coordinate set of each target map element are converted to obtain the two-dimensional image coordinates of each target map element.
[0150] For example, when the element coordinates are in the front-left-up (FLU) coordinate system with the data acquisition position as the origin, the formula can be used. , calculate the two-dimensional image coordinates. Among them, X, Y, Z represent the three component coordinates of the element coordinates, and u and v represent the two component coordinates of the two-dimensional image coordinates. , represents the camera extrinsic parameters, , It represents the time alignment matrix between the camera and the lidar, z represents the depth value, and K is the camera intrinsic parameter.
[0151] S302: Annotate the target map elements in the image captured by the camera according to the two-dimensional image coordinates of each target map element.
[0152] In this step, after obtaining the two-dimensional image coordinates of each target map element, the computer marks the target map element in the image captured by the camera according to the two-dimensional image coordinates of each target map element.
[0153] For example, Figure 3b An image diagram of annotated map elements provided for this application is shown in FIG. Figure 3bAs shown, the target map elements in the image include a crosswalk sign, a lane line, and a speed limit sign of 50 km / h. The graphics composed of thick black lines and thick black lines in the figure represent the marked target map elements.
[0154] For example, Figure 3c The image diagram 2 of the annotated map elements provided for this application is as follows: Figure 3c As shown in the figure, the white lines or the graphics surrounded by white lines represent the marked target map elements. It can be seen that map elements such as lane lines, cables, pedestrian crossing signs, and signboards are marked. All map elements in the image are marked, which improves the accuracy of marking.
[0155] The map element labeling method provided in this embodiment improves labeling efficiency and accuracy by labeling the target map elements in the image taken by the camera according to the element coordinates of the target map elements after the coordinate conversion, and can also visually verify the labeling effect; it can also be used to construct a training data set to perform model training to obtain a model for target detection in images taken by the camera.
[0156] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0157] Figure 4 This is a schematic diagram of the structure of an embodiment of the map element annotation device provided by this application. Figure 4 As shown, the map element marking device 40 includes:
[0158] The processing module 41 is used for:
[0159] According to the data collection position of the laser radar point cloud data, a screening area corresponding to the data collection position is generated;
[0160] Filtering the plurality of map elements according to the element coordinates in the vector data of the plurality of map elements and the position distribution relationship of the filtering area to obtain at least one target map element;
[0161] The labeling module 42 is used to label the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element.
[0162] Furthermore, the processing module 41 is specifically used for:
[0163] Eliminate all map elements whose element coordinates are outside the screening area from among the multiple map elements to obtain a first map element;
[0164] According to the screening area, the element coordinates of each of the first map elements are corrected to obtain an element coordinate set of each of the first map elements;
[0165] The first map elements are screened according to the element coordinate set of each first map element to obtain at least one target map element.
[0166] Furthermore, the processing module 41 is further configured to:
[0167] Determining a first preset number of lane centerlines closest to the data collection location from the first map element, and determining a height screening range based on the first preset number of lane centerlines;
[0168] The first map elements are screened according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element.
[0169] Furthermore, the processing module 41 is specifically used for:
[0170] Performing graphic fitting according to the element coordinates of each first map element to obtain a graphic of each first map element;
[0171] For each first map element, the element coordinates of the first map element within the screening area and the coordinates of the intersection of the graphic of the first map element and the boundary of the screening area constitute an element coordinate set of the first map element.
[0172] Furthermore, the processing module 41 is specifically used for:
[0173] For each lane centerline of the first preset number of lane centerlines, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline;
[0174] Selecting a second preset number of interpolation points closest to the location of the collected data from the multiple interpolation points to obtain a target interpolation point of the lane centerline;
[0175] The height screening range is determined according to an average height of the target interpolation points of the first preset number of lane center lines.
[0176] Furthermore, the processing module 41 is specifically used for:
[0177] For each lane centerline in all first map elements, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline;
[0178] For each lane centerline in all first map elements, establishing a correspondence between the first map element corresponding to the lane centerline and the interpolation points according to the coordinates of the plurality of interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline;
[0179] The first map elements are screened according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, and the height screening range, to obtain at least one target map element.
[0180] Furthermore, the processing module 41 is specifically used for:
[0181] Calculate the distance between each first map element corresponding to the lane centerline and each interpolation point according to the coordinates of the plurality of interpolation points corresponding to the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline;
[0182] For each first map element corresponding to the lane centerline, an interpolation point among the multiple interpolation points that is closest to the first map element is established in a corresponding relationship with the first map element.
[0183] Furthermore, the processing module 41 is specifically used for:
[0184] According to the correspondence between the first map element corresponding to each lane centerline in all the first map elements and the interpolation point, the first map element corresponding to the interpolation point whose height is not within the height screening range is taken as the first element to be deleted;
[0185] For each lane centerline in all first map elements, if the heights of multiple interpolation points of the lane centerline are not within the height screening range, the lane centerline is used as a second element to be deleted;
[0186] Eliminate the first to-be-deleted elements and the second to-be-deleted elements from all the first map elements to obtain a second map element;
[0187] For each second map element, deleting the element coordinates of the element coordinate set of the second map element whose height is not within the height screening range, to obtain a target map element and a target element coordinate set of the target map element;
[0188] Furthermore, the marking module 42 is specifically used for:
[0189] According to the target element coordinate set of the at least one target map element, the target map element is marked in the lidar point cloud data.
[0190] Furthermore, the processing module 41 is also used to convert the element coordinates of each target map element to obtain the two-dimensional image coordinates of each target map element;
[0191] Furthermore, the labeling module 42 is further configured to label the target map elements in the image captured by the camera according to the two-dimensional image coordinates of each target map element.
[0192] The map element annotation device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0193] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 5 As shown, the electronic device 50 includes:
[0194] Processor 51, memory 52, and communication interface 53;
[0195] The memory 52 is used to store executable instructions of the processor 51;
[0196] The processor 51 is configured to execute the technical solution in any of the aforementioned method embodiments by executing the executable instructions.
[0197] Optionally, the memory 52 may be independent or integrated with the processor 51 .
[0198] Optionally, when the memory 52 is a device independent of the processor 51, the electronic device 50 may further include:
[0199] The bus 54 , the memory 52 and the communication interface 53 are connected to the processor 51 via the bus 54 and communicate with each other. The communication interface 53 is used to communicate with other devices.
[0200] Optionally, the communication interface 53 may be implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.
[0201] The bus 54 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0202] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0203] The electronic device is used to execute the technical solution in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.
[0204] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided by any of the aforementioned method embodiments is implemented.
[0205] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned method embodiments when executed by a processor.
[0206] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A map element annotation method, characterized in that: include: According to the data collection position of the laser radar point cloud data, a screening area corresponding to the data collection position is generated; Eliminate all map elements whose element coordinates in the vector data of the plurality of map elements are outside the screening area to obtain a first map element; Determining a first preset number of lane centerlines closest to the location of the collected data from the first map element; For each lane centerline of the first preset number of lane centerlines, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline; Selecting a second preset number of interpolation points closest to the location of the collected data from the multiple interpolation points to obtain a target interpolation point of the lane centerline; Determining a height screening range according to an average height of the target interpolation points of the first preset number of lane center lines; Filtering the first map element according to the height screening range to obtain at least one target map element; According to the element coordinates of the at least one target map element, the target map element is marked in the lidar point cloud data.
2. The method according to claim 1, characterized in that: After removing all the map elements whose coordinates in the vector data of the plurality of map elements are outside the screening area to obtain the first map element, the method further includes: According to the screening area, the element coordinates of each of the first map elements are corrected to obtain an element coordinate set of each of the first map elements.
3. The method according to claim 2, characterized in that The filtering of the first map element according to the height filtering range to obtain at least one target map element includes: The first map elements are screened according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element; the screening includes removing the first map elements whose heights do not match the height screening range, and updating the element coordinate sets of the remaining first map elements.
4. The method according to claim 2, characterized in that: The step of performing correction processing on the element coordinates of each of the first map elements according to the screening area to obtain an element coordinate set of each of the first map elements includes: Performing graphic fitting according to the element coordinates of each first map element to obtain a graphic of each first map element; For each first map element, the element coordinates of the first map element within the screening area and the coordinates of the intersection of the graphic of the first map element and the boundary of the screening area constitute an element coordinate set of the first map element.
5. The method according to claim 3, characterized in that: The step of screening the first map elements according to the height screening range and the element coordinate set of each first map element to obtain at least one target map element includes: For each lane centerline in all first map elements, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline; For each lane centerline in all first map elements, establishing a correspondence between the first map element corresponding to the lane centerline and the interpolation points according to the coordinates of the plurality of interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline; The first map elements are screened according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, and the height screening range, to obtain at least one target map element.
6. The method according to claim 5, characterized in that The establishing a correspondence between the first map element corresponding to the lane centerline and the interpolation points according to the coordinates of the plurality of interpolation points of the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline includes: Calculate the distance between each first map element corresponding to the lane centerline and each interpolation point according to the coordinates of the plurality of interpolation points corresponding to the lane centerline and the element coordinate set of the first map element corresponding to the lane centerline; For each first map element corresponding to the lane centerline, an interpolation point among the multiple interpolation points that is closest to the first map element is established in a corresponding relationship with the first map element.
7. The method according to claim 5, characterized in that The first map elements are screened according to the correspondence between the first map elements corresponding to each lane centerline in all the first map elements and the interpolation points, and the height screening range, to obtain at least one target map element, including: According to the correspondence between the first map element corresponding to each lane centerline in all the first map elements and the interpolation point, the first map element corresponding to the interpolation point whose height is not within the height screening range is taken as the first element to be deleted; For each lane centerline in all first map elements, if the heights of multiple interpolation points of the lane centerline are not within the height screening range, the lane centerline is used as a second element to be deleted; Eliminate the first to-be-deleted elements and the second to-be-deleted elements from all the first map elements to obtain a second map element; For each second map element, deleting the element coordinates of the element coordinate set of the second map element whose height is not within the height screening range, to obtain a target map element and a target element coordinate set of the target map element; The step of marking the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element comprises: According to the target element coordinate set of the at least one target map element, the target map element is marked in the lidar point cloud data.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Convert the element coordinates of each target map element to obtain the two-dimensional image coordinates of each target map element; According to the two-dimensional image coordinates of each target map element, the target map element is marked in the image captured by the camera.
9. A map element annotation device, characterized in that: include: Processing modules for: According to the data collection position of the laser radar point cloud data, a screening area corresponding to the data collection position is generated; Eliminate all map elements whose element coordinates in the vector data of the plurality of map elements are outside the screening area to obtain a first map element; Determining a first preset number of lane centerlines closest to the location of the collected data from the first map element; For each lane centerline of the first preset number of lane centerlines, performing equidistant interpolation processing on the lane centerline according to the element coordinate set of the lane centerline to obtain a plurality of interpolation points of the lane centerline; Selecting a second preset number of interpolation points closest to the location of the collected data from the multiple interpolation points to obtain a target interpolation point of the lane centerline; Determining a height screening range according to an average height of the target interpolation points of the first preset number of lane center lines; Filtering the first map element according to the height screening range to obtain at least one target map element; The labeling module is used to label the target map element in the laser radar point cloud data according to the element coordinates of the at least one target map element.
10. An electronic device, characterized in that: include: Processor, memory, communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the map element labeling method described in any one of claims 1 to 8 by executing the executable instructions.
11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the map element labeling method according to any one of claims 1 to 8 is implemented.
12. A computer program product, characterized in that It comprises a computer program, which is used to implement the map element labeling method according to any one of claims 1 to 8 when executed by a processor.
