Road element true value generation method and device, equipment and storage medium

Abstract

The invention discloses a road element truth value generation method and device, equipment and a storage medium, and relates to the technical field of maps, and the method comprises the steps: constructing a point cloud scene map for two-dimensional image data and three-dimensional point cloud data of a road according to the parking space posture information of a vehicle; marking road elements of the point cloud scene map, partitioning the marked point cloud scene map into multiple sections of road segment information according to a road structure and a road connection relation topological structure, and storing the road segment information into a map database; and retrieving in a graph database according to the received retrieval fragment to generate a road element true value. The point cloud scene map is constructed according to the two-dimensional image data and the three-dimensional point cloud data of the road, and then the marked point cloud scene map is partitioned and stored in the map database according to the road structure, so that the point cloud scene map can be applied to all road segment information after being marked once, tedious and repeated marking is avoided, and the marking efficiency is improved. And the road element truth value can be directly generated according to the retrieval segment retrieval graph database, so that the efficiency is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of map technology, and in particular to a method, device, equipment and storage medium for generating true values ​​of road elements. Background Art

[0002] Traditionally, the generation of road element truth values ​​relies on the acquisition of images by acquisition equipment, which are then pre-annotated by the model and manually reviewed. Due to their working principles, image acquisition equipment generally only captures two-dimensional images, lacks depth information, and lacks the ability to accurately construct the entire environment. For the same scene, different viewing angles, etc., it is difficult to use existing truth values ​​to generate corresponding truth values, and re-annotation is required.

[0003] In the existing map generation process, although laser radar is used to obtain three-dimensional location information, the entire scene can be constructed in the world coordinate system by continuously collecting point cloud fragments, vehicle motion information, posture sensors, etc. However, due to the characteristics of point clouds, the data density is extremely high, and the real-time requirements for storing associated indexes are extremely high. The process generally requires target detection and repeated labeling, and then the environmental map is obtained by indexing the matching of the labeled information with the database. This method of retrieving through environmental information depends on the accuracy of the acquisition equipment and the accuracy of the detection algorithm. There are obvious problems with the accuracy of the index, which requires repeated comparison and calculation, and the retrieval efficiency is low. Summary of the invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for generating true values ​​of road elements, aiming to solve the technical problems that the existing method of retrieving through environmental information relies on the accuracy of the acquisition equipment and the accuracy of the detection algorithm, requires repeated comparison and calculation during retrieval, and has low retrieval efficiency.

[0005] To achieve the above purpose, the present application proposes a method for generating true values ​​of road elements, the method comprising:

[0006] Collect 2D image data and 3D point cloud data of roads through vehicles;

[0007] Performing scene construction on the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map;

[0008] Annotating the point cloud scene map with road elements to obtain an annotated point cloud scene map;

[0009] Divide the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and store the obtained multiple road segment information in a graph database;

[0010] When a retrieval fragment is received, the graph database is searched according to the retrieval fragment to generate a true value of a road element.

[0011] In one embodiment, the step of constructing a scene based on the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map includes:

[0012] Performing time-series alignment and distortion removal on the two-dimensional image data and the three-dimensional point cloud data to obtain pre-processed road data;

[0013] Extracting features from the preprocessed road data to obtain feature descriptors;

[0014] Perform forward optimization prediction based on the odometer information of the vehicle to obtain the vehicle posture information;

[0015] Performing factor graph optimization on the feature descriptor according to the vehicle posture information to obtain local maps of different segments;

[0016] The local maps of the different fragments are merged into the same world coordinate system to generate a point cloud scene map.

[0017] In one embodiment, the step of dividing the annotated point cloud scene map into blocks according to a preset road structure and a road connection relationship topological structure, and storing the obtained multiple road segment information in a graph database includes:

[0018] Dividing the annotated point cloud scene map according to a preset road structure to obtain corresponding node type data, wherein the node type data includes road data, intersection data, lane node data, road object data, and road element data;

[0019] Extracting relationships from the annotated point cloud scene map according to a preset road connectivity relationship topology structure to obtain corresponding edge type data, wherein the edge type data includes connection relationships, position relationships, inclusion relationships, and intersection relationships;

[0020] The road data, the intersection data, the lane node data, the road object data, the road element data, the connection relationship, the position relationship, the inclusion relationship and the intersection relationship are used as multiple sections of road segment information, and the multiple sections of road segment information are stored in a graph database.

[0021] In one embodiment, after the step of storing the obtained multiple road segment information in the graph database, the method further includes:

[0022] Constructing a multi-segment road topology graph from the graph database according to the road data, the intersection data, the lane node data, the road object data and the road element data;

[0023] Based on the connection relationship, the position relationship, the inclusion relationship and the intersection relationship, the multiple road topology graphs are connected to obtain a road network topology map;

[0024] Accordingly, when receiving the search fragment, the step of searching in the graph database according to the search fragment to generate the true value of the road element includes:

[0025] When the search fragment is received, the graph database is searched according to the search fragment and the road network topology map to generate the true value of the road element.

[0026] In one embodiment, when receiving the search fragment, searching the graph database according to the search fragment and the road network topology map to generate the true value of the road element includes:

[0027] Extracting the pose information and picture information of the searched fragment;

[0028] Positioning the search segment in the road network topology map according to the position information, and determining an intersection node of the search segment in the point cloud scene map;

[0029] According to the intersection node, extracting the 3D true value corresponding to the search segment from the graph database;

[0030] Based on the image information, the 3D true value is converted into a 2D true value according to a vehicle body coordinate system, and the 2D true value is used as the road element true value corresponding to the retrieval segment.

[0031] In one embodiment, the step of positioning in the road network topology map according to the posture information and determining the intersection node of the search segment in the point cloud scene map includes:

[0032] Extracting the coordinate information of the vehicle from the position information;

[0033] Based on the coordinate information, traverse the road network topology map;

[0034] Determine the intersection node range of the search fragment in the road network topology map according to the traversal result;

[0035] According to the intersection node range and the coordinate information, positioning is performed from the graph database to determine the intersection node of the retrieval fragment in the point cloud scene map.

[0036] In one embodiment, after the step of traversing the road network topology map based on the coordinate information, the method further includes:

[0037] If the coordinate information is not retrieved within the intersection node range of the road network topology map, it is determined that the point cloud map corresponding to the search segment is not included in the graph database;

[0038] Perform SLAM mapping according to the retrieval fragments to obtain a target point cloud map;

[0039] Based on the target point cloud map, returning to the step of annotating the point cloud scene map with road elements to obtain the annotated point cloud scene map, until the information of multiple road segments corresponding to the target point cloud map is updated to the graph database;

[0040] According to the updated graph database, return to the step of extracting the pose information and the picture information of the search segment to determine the true value of the road element corresponding to the search segment.

[0041] In addition, to achieve the above purpose, the present application also proposes a road element true value generation device, the device comprising:

[0042] A data acquisition module, used for acquiring two-dimensional image data and three-dimensional point cloud data of the road through a vehicle;

[0043] A scene construction module, used to construct a scene for the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map;

[0044] A road labeling module, used to label road elements on the point cloud scene map to obtain a labeled point cloud scene map;

[0045] A graph database module, used to divide the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and store the obtained multiple road segment information in a graph database;

[0046] The segment retrieval module is used to, upon receiving a retrieval segment, search the graph database according to the retrieval segment to generate a true value of a road element.

[0047] In addition, to achieve the above-mentioned purpose, the present application also proposes a road element truth value generation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the road element truth value generation method as described above.

[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the road element true value generation method described above are implemented.

[0049] One or more technical solutions proposed in the present application have at least the following technical effects: The method for generating true values ​​of road elements in the present application includes: collecting two-dimensional image data and three-dimensional point cloud data of roads through a vehicle; constructing a scene for the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map; annotating the point cloud scene map with road elements to obtain an annotated point cloud scene map; dividing the annotated point cloud scene map into blocks according to a preset road structure and a road connectivity relationship topological structure, and storing the obtained multiple road segment information in a graph database; when a retrieval segment is received, searching the graph database according to the retrieval segment to generate a true value of the road element.

[0050] Since the present application first constructs a point cloud scene map based on the two-dimensional image data and three-dimensional point cloud data of the road, and then divides the point cloud scene map after annotating the road elements according to the road structure and the road connectivity relationship topological structure, and stores it in a graph database, the annotation can be applied to all road segment information once, thus eliminating the need for tedious manual annotation. Finally, the graph database can be searched based on the retrieval segment to directly generate the true value of the road elements under each working condition, greatly improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the method for generating the true value of road elements of the present application;

[0054] Figure 2 A schematic diagram of the content of the road element annotation provided in Example 1 of the present application;

[0055] Figure 3 A schematic diagram of the process of graph database element conversion provided in Example 1 of the present application;

[0056] Figure 4 A schematic diagram of the structure of the road network topology map provided in Example 1 of the present application;

[0057] Figure 5 A schematic diagram of a flow chart provided for the second embodiment of the method for generating the true value of road elements of the present application;

[0058] Figure 6 A schematic diagram of the process of fragment retrieval provided in Example 2 of the present application;

[0059] Figure 7 A schematic diagram of the overall process of the method for generating true values ​​of road elements provided in the second embodiment of the present application;

[0060] Figure 8 This is a schematic diagram of the module structure of the device for generating true values ​​of road elements according to an embodiment of the present application;

[0061] Fig. 9 Schematic diagram of the device structure of the hardware operating environment involved in the method for generating true values ​​of road elements in the embodiment of the present application.

[0062] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0064] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0065] It should be noted that the existing road element true value generation process has the following problems:

[0066] 1. There are inevitable differences in the data collected from different angles and at different times for the same scene (for example, the data was collected by cameras in different positions, with the data collected in the left lane for the first time and in the right lane for the second time). The true value of the first annotation is difficult to transfer to the data collected for the second time, and repeated manual annotation is required. Repeated manual annotation is time-consuming and laborious, and it is difficult to flexibly adapt to changes in the sensors of the acquisition equipment.

[0067] 2. The existing solution is to collect image data through cameras, retrieve data from the cloud database by detecting static key targets, and then return to the cloud map. This is highly dependent on the accuracy of detection and matching, and the real scene is changeable, making it difficult to achieve accurate and efficient scene matching.

[0068] 3. The scenes covered by the point cloud map are not necessarily complete. The existing solutions require a complex data collection and iteration process, which makes it difficult to meet the corresponding needs in a timely manner. In addition, the point cloud data is large and dense, and the traditional database storage method has data redundancy. For a given positioning information, repeated comparisons and calculations are required when searching, resulting in low retrieval efficiency.

[0069] To solve the above problems, this application proposes a method for generating true values ​​of road elements, which can be applied to all fragments after annotating the collected data once; at the same time, it has high performance and scalability when processing large-scale data (such as large-scale point cloud data) by managing the relationship between road elements through a graph database; for the task of generating true values, data collected at different locations in the same scene can directly generate the true values ​​under various working conditions based on the data annotated in the graph database, without the need for tedious manual annotation, thereby improving the richness of data and retrieval efficiency.

[0070] It should be noted that the execution subject of this embodiment can be a computing service device with data collection, point cloud map construction, map segmentation and data retrieval functions, such as a personal computer, a car computer, etc., or an electronic device capable of realizing the above functions, a road element truth value generation device (referred to as a truth value generation device) that executes the road element truth value generation method of this application, etc., and this embodiment does not limit this. The following takes the truth value generation device as an example to illustrate this embodiment and the following embodiments.

[0071] Based on this, the present application embodiment provides a method for generating true values ​​of road elements, referring to Figure 1 , Figure 1 A flowchart diagram of the first embodiment of the method for generating true values ​​of road elements in the present application.

[0072] In this embodiment, the road element true value generation method includes steps S10 to S50:

[0073] Step S10: Collecting two-dimensional image data and three-dimensional point cloud data of the road by a vehicle.

[0074] It should be noted that the data collection process can be carried out by a special vehicle (such as a truth collection vehicle) equipped with high-precision measurement and detection equipment (such as sensors including lidar and cameras). Accurate road information can be collected by the truth collection vehicle.

[0075] The two-dimensional image data may be composed of pixels on a plane and may represent road information, such as photos and pictures.

[0076] Exemplarily, the two-dimensional image data can be collected by a camera of a truth collection vehicle, and may include images at each moment, such as a front view road image, a rear view road image, and a side view video road image of the truth collection vehicle; the process can be collected at a resolution of 20 Hz, which is not limited in this embodiment.

[0077] The three-dimensional point cloud data may be a collection of three-dimensional coordinate information of a large number of points on the surface of road objects, and these point clouds represent the actual positions of various objects on the road. The three-dimensional point cloud data may be collected by a laser radar (e.g., a rotating laser radar) of a true value collection vehicle, and the process may be collected at a frequency of 10 Hz, which is not limited in this embodiment.

[0078] It should be noted that during the collection process, the vehicle's position information and attitude information can also be recorded and saved through the vehicle's own Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU).

[0079] In the specific implementation, after the vehicle collects the two-dimensional image data and three-dimensional point cloud data of the road through the camera and the rotating laser radar, it packages the recorded vehicle position information, vehicle posture information, the above two-dimensional image data and three-dimensional point cloud data, saves them in the form of data packets, and sends them to the truth value generation device. After receiving the above data, the truth value generation device starts the subsequent operations.

[0080] Step S20: constructing a scene for the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map.

[0081] It should be noted that the vehicle position information, i.e., the vehicle position information and the vehicle posture information, represents the position and posture of the vehicle during the process of collecting the two-dimensional image data and the three-dimensional point cloud data.

[0082] It is understandable that the point cloud scene map can be a map representation of the road and its surrounding environment, consisting of a large number of points, containing three-dimensional spatial coordinate information and other attribute information (road shape, terrain undulations, buildings, trees, transportation facilities, etc.).

[0083] In a specific implementation, after obtaining the above-mentioned two-dimensional image data and three-dimensional point cloud data, the truth value generation device can perform a series of preprocessing (such as time alignment, removal of redundant data, etc.) on the above-mentioned two-dimensional image data, three-dimensional point cloud data and self-vehicle posture information, and then construct a point cloud scene graph based on the preprocessed data to obtain a point cloud scene map.

[0084] In a feasible implementation, step S20 of this embodiment may include the steps of: performing time alignment and distortion removal on the two-dimensional image data and the three-dimensional point cloud data to obtain preprocessed road data; performing feature extraction on the preprocessed road data to obtain a feature descriptor; performing forward optimization prediction based on the odometer information of the vehicle to obtain the vehicle posture information; performing factor graph optimization on the feature descriptor based on the vehicle posture information to obtain local maps of different fragments; merging the local maps of different fragments into the same world coordinate system to generate a point cloud scene map.

[0085] It should be noted that time series alignment can be used to synchronize and match two different types of data, 2D image data and 3D point cloud data, in the time dimension. Because they may be collected by different sensors at different times, they need to be aligned in time series for the accuracy of subsequent processing and analysis.

[0086] Distortion removal can be used to correct various deformations or inaccurate data that may exist in 2D image data and 3D point cloud data. For example, 2D image data may have geometric distortion due to factors such as lenses, and point cloud data may have some unreasonable deformations due to measurement errors. By removing these distortions, the data can be made more accurate.

[0087] Specifically, for ease of understanding, the process of timing alignment and distortion removal includes: the truth generation device first synchronizes the timing data in the two-dimensional image data and the three-dimensional point cloud data through the timestamp, and uses linear interpolation to fill in the missing data points to ensure the accuracy of different sensor data at the same timestamp. Then, for the two-dimensional image data, the camera's internal parameters are used to perform distortion removal (such as radial and tangential distortion) through the distortion model to eliminate the influence of lens distortion and ensure the accuracy of subsequent feature extraction.

[0088] It is understandable that the feature descriptor can be a key feature of the road that is representative and distinguishable and is found from the pre-processed road data. Exemplarily, the feature descriptor can include the geometric shape of the road (such as curves, slopes, etc.), edge features, roadside signs, etc.

[0089] Specifically, for ease of understanding, after obtaining the preprocessed road data, feature extraction is required. This stage is divided into several key steps: First, ground point segmentation is performed. The truth generation device can identify and separate ground points from the point cloud in the preprocessed road data through a ground segmentation algorithm (such as the RANSAC algorithm or the Hough transform algorithm) to reduce the interference of the ground on feature extraction. Then, dynamic point verification is performed. Dynamic object detection algorithms (such as optical flow methods or deep learning-based methods) can be used to identify and remove dynamic points in the preprocessed road data to improve the accuracy of static environmental features. After completing the above-mentioned separation of ground points and removal of dynamic points, feature extraction is finally performed. This process can construct feature descriptors by extracting plane features (such as plane model fitting) and edge features (such as Canny edge detection) in the data. These feature descriptors will be used for subsequent matching and optimization.

[0090] It should be understood that the odometer information can be information related to the measurement and recording of the vehicle's moving distance or travel, and can include data such as the distance, speed, and direction traveled by the vehicle during the movement of the data collection. Since the self-vehicle position information and self-vehicle posture information recorded in the form of data packets are not converted and are original odometer information, the odometer information needs to be processed to obtain the self-vehicle posture information.

[0091] Specifically, for ease of understanding, after obtaining the above-mentioned feature descriptors, the truth generation device can perform inertial measurement unit (IMU) pre-integration, which combines the odometer information for forward optimization prediction: First, the IMU data (such as acceleration and angular velocity) in the odometer information is pre-integrated to obtain the IMU posture change data of the vehicle within a short time window. This process can effectively reduce the error accumulation caused by time delay. Then the wheel speed sensor information in the odometer information is combined with the above-mentioned IMU posture change data, and the state estimation is performed through methods such as Kalman filtering to obtain more accurate vehicle posture information.

[0092] It is understandable that factor graph optimization can be achieved by constructing a factor graph to integrate information from different sources (such as images and point clouds) to optimize the global pose.

[0093] Specifically, for ease of understanding, after obtaining the above-mentioned more accurate self-vehicle posture information, the truth generation device can use different constraint factors (such as GNSS factors, loop detection factors, ground constraint factors, etc., which are not limited in this embodiment) to construct factor graphs, and use these factor graphs to provide different information sources to help optimize the posture (wherein, the GNSS factor is used to provide global positioning information in an open environment. The loop detection factor is used to perform loop detection by identifying environmental features to reduce cumulative errors. The ground constraint factor enhances alignment accuracy by utilizing the geometric characteristics of ground points). Then, an optimization algorithm (such as the G2O or Ceres Solver database) is used in combination with the above-mentioned factor graph to globally optimize the feature descriptor, minimize the error function, and obtain local maps of different fragments at each moment.

[0094] Finally, since the local maps generated by the above-mentioned different fragment point clouds have the same coordinate system, the global pose information (such as the vehicle pose information) can be used to splice the local maps and merge them into the same world coordinate system to generate a complete 3D point cloud scene map of the entire road scene.

[0095] In this embodiment, the truth value generation device can perform the above-mentioned series of preprocessing, feature extraction, IMU pre-integration and factor graph optimization based on the two-dimensional image data of continuous frames, the three-dimensional point cloud data and the odometer information of the vehicle to construct a point cloud map generated by different fragments, and finally merge them into the same world coordinate system to generate an accurate point cloud scene map.

[0096] Step S30: annotating the point cloud scene map with road elements to obtain an annotated point cloud scene map.

[0097] It should be noted that the road element annotation is to distinguish the different main contents of the road in the point cloud scene map to facilitate the subsequent block division operation.

[0098] For example, the annotation content may include the following five categories: lane line annotation, roadside annotation, ground sign annotation, traffic light annotation, and traffic sign annotation; other methods may also be used for annotation, which is not limited in this embodiment. The specific annotation content is shown in Table 1:

[0099] Table 1

[0100] Lane lines Roadside Ground signs Traffic Lights Traffic signs Location Location Location Location Location ID ID ID ID ID Wear properties Curb category Ground Arrow Properties Number of bulbs Orientation Occlusion properties Stop Line Properties color category Single and double line type Sidewalk attributes pattern Virtual and real type No Parking Zone Properties Function Type

[0101] During the labeling process, only one manual labeling is required, and the obtained true value can be converted to any image acquisition device connected to the vehicle body (such as front view, rear view and side view) through matrix operation. The labeling results can remain highly consistent in continuous frame scenes.

[0102] In the specific implementation, the truth value generation device can annotate road elements based on the point cloud scene map, distinguish the different main contents of the road in the point cloud scene map, and obtain the annotated point cloud scene map to facilitate the subsequent block division operation. And because they are all in the same coordinate system, the annotation can be applied to all segments once, which greatly improves efficiency.

[0103] Step S40: dividing the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and storing the obtained multiple road segment information in a graph database.

[0104] It should be noted that the road structure can be the physical characteristics and components of the road itself, and may include aspects such as road grade (such as expressways, ordinary roads, urban roads, etc.), road surface conditions, number of lanes, width, slope, turning radius, etc.

[0105] The road connectivity topology can focus on the logical relationship between the interconnections and associations between roads. Through the road connectivity topology, an abstract network structure can be constructed with nodes (representing intersections, endpoints, etc. of roads) and lines (representing road sections), showing how different roads are interconnected, intersecting, and forming an overall connection.

[0106] Since the above-mentioned annotated point cloud scene map is too large in scale, it can be divided according to the above-mentioned road structure and road connection relationship topological structure, and multiple road segment information can be obtained and stored in the graph database.

[0107] In the specific implementation, refer to Figure 2 , Figure 2 Schematic diagram of the content of the road element annotation provided in Example 1 of the present application. After the data collection vehicle collects the two-dimensional image data and three-dimensional point cloud data of multiple segments (such as segment 1, segment 2, segment 3, etc.), they are merged into a unified world coordinate system through the above-mentioned mapping process to obtain a point cloud scene map. At this time, the point cloud scene map can be divided into road elements (for example, divided into intersections, roads, ramps, etc.) and combined with road connectivity relationships (such as connection relationships, intersection relationships, etc.), and multiple road segment information is obtained and stored in the graph database.

[0108] In a feasible implementation, step S40 of this embodiment may include the steps of: dividing the annotated point cloud scene map according to a preset road structure to obtain corresponding node type data, wherein the node type data includes road data, intersection data, lane node data, road object data, and road element data; performing relationship extraction on the annotated point cloud scene map according to a preset road connectivity relationship topological structure to obtain corresponding edge type data, wherein the edge type data includes connection relationship, position relationship, inclusion relationship, and intersection relationship; using the road data, the intersection data, the lane node data, the road object data, the road element data, the connection relationship, the position relationship, the inclusion relationship, and the intersection relationship as multiple road segment information, and storing the multiple road segment information in a graph database.

[0109] For example, to facilitate understanding of the meaning of road structure and road connection relationship topology, an example is given below. Each map in the point cloud scene map can be divided into roads and junctions according to whether there is an intersection. These two categories include several lane nodes and road objects. Adjacent intersections and roads are defined by connection relationships. The example structure is as follows:

[0110] For example: Road 1 - Lane Section 1 (solid line) - Lane Section 2 (solid line) - Lane Section 3 (curb) - Road Object 3 (arrow) - Road Object 4 (arrow). Intersection 1 - Lane Section 1 (solid line) - Lane Section 2 (solid line) - Road Object 1 (signal light) - Road Object 2 (arrow).

[0111] According to this division relationship, the road structure can be divided into:

[0112] 1. Road: Each road consists of several parts, including straight segments, curve segments, intersections, etc. Specific attributes may include road identifier (id), road name (name), road length (length), road type (type), and the part containing road lane information (laneSection).

[0113] 2. Junction: An intersection is an area where multiple roads meet, usually referring to the connection methods of different roads, traffic control facilities and vehicle driving paths. According to the function of the intersection in connecting different roads. Its attributes may include the intersection identifier (id), the type of intersection (type), the connection information between the intersection and adjacent roads (roadLinks), and whether there are traffic lights (trafficLight). Among them, the intersection starts from the center of the intersection and expands in the direction of each intersection until the intersection of the intersection road and the stop line (the sidewalk shall prevail if there is no stop line). The intersection needs to include road elements and road objects such as intersection lights, intersection traffic signs, sidewalks, stop lines, and the connection relationship between the intersection and each road node can also be recorded.

[0114] 3. Lane Sections: Lane sections are the lateral parts of lanes, usually with the same lane configuration in a certain section at a certain location. According to the function of lane sections, they can be divided into standard lane sections, lane sections with medians, ramp lane sections, and lane sections with traffic facilities. Its attributes may include the starting position (s) of the lane section, the lateral offset (lateraloffset) of the lane section, the road marking (roadMark) of the lane section, and the elevation (height) of the lane section. Among them, the lane section adjacent to the intersection is bounded by the intersection at one end, and the other end follows the following rules: when the lane attributes in the area change, when the length of the lane section exceeds 150m, the road is cut according to the above priority.

[0115] 4. Road Objects: static objects on the road, such as traffic signs, light poles, guardrails, signal lights, etc., which have an impact on traffic flow and vehicle driving behavior. Its attributes may include road object identifier (id), object type (type), object spatial position (position), object size (size), and object orientation (orientation).

[0116] 5. Road Elements: In each lane section, you can define the center line, left and right lanes and their boundaries, as well as ground signs, traffic signs and other element information. It can include various road element instances in the road section: lane lines, curbs, ground signs. The same road element may appear across road sections, and the ID needs to be kept consistent. The attribute transformation of the same lane line needs to record the coordinates of the attribute transformation point.

[0117] According to the above division relationship, the road connection relationship topology structure can be divided into:

[0118] 1. Connection relationship. The connection relationship may include the successor relationship (indicating the next road of the current road. The ID, connection point type and location information of the next road can be specified through the successor attribute) and the predecessor relationship (indicating the previous road node of the current road. The ID, connection point type and location information of the previous road can be specified through the predecessor attribute).

[0119] 2. Position relationship: It is used to establish the relationship between road surface elements, road surface objects and road nodes and intersections. They are associated through IDs, and the mapping relationship may be many-to-many.

[0120] 3. Containment relationship: A road is the result of combining multiple road segments and intersections. It may contain multiple road segments and intersections, which can be reflected by an ID list.

[0121] 4. Intersection relationship: A special connection relationship is a state or situation where two or more roads intersect or connect with each other in plane or space, which can be reflected by an ID list.

[0122] Therefore, according to the above description, a road can be composed of lane nodes, which contain Elements instances. Elements instances contain the coordinate range, type, size, own ID and associated lane node ID of the element. Lane node attributes contain Elements instance ID, lane node coordinate range, type, size, etc. Lane nodes are described by road structure entities for correlation, mainly including connectivity, direction, connection mode and connection points, etc. The above multi-segment road segment information is integrated into the graph database for element conversion, which can be divided into the following two categories. The instances and their fields are as follows:

[0123] 1. Node type data. Road data: the road itself, including basic information about the road; its attributes include identifier (id), name (name), total length (length), type (type), etc. Lane section data: an area of ​​a lane, usually an area on the road, each lane segment may have different road surface markings; its attributes include identifier (id), start position (startposition), end position (endposition), etc. Intersection data: represents the connection point of multiple roads, such as crossroads, T-junctions, etc.; its attributes include identifier (id), type (type), position (position), etc. Road object data: static objects on the road, such as traffic signs, roadblocks, etc.; its attributes include identifier (id), type (type), position (position), etc. Road element data: signs, lane lines, curbs, etc. on the road; its attributes include identifier (id), type (type), position (position), etc.

[0124] 2. Edge type data, where nodes are connected by edges, representing the relationship between nodes; each edge type can have different attributes. Including: Connection relationship: represents the connection relationship between two elements, usually representing the connection between roads and intersections, lanes and lane segments. Containment relationship: indicates that one element is part of another element, for example, lanes belong to roads, and lane segments belong to lanes. Position relationship: indicates that an object is located at a certain position, such as road markings, traffic signs, etc. are located on a certain road or lane segment. Intersection relationship: indicates that a road or lane intersects with other elements, such as an intersection or a road marking intersects with a road.

[0125] Exemplary, reference Figure 3 , Figure 3 A flow chart of the conversion of graph database elements provided in Example 1 of the present application. When storing information of multiple road segments into a graph database, nodes (such as the lane segments, intersections, road objects, road elements, and other node types mentioned above) may be created first, and then attributes (such as boundary coordinates, node types, node status, etc.) may be added to the nodes, and then the connection relationships between the nodes (such as connection relationships, inclusion relationships, position relationships, intersection relationships, etc.) may be added to each node to complete the data storage of the point cloud scene map.

[0126] In this embodiment, the collected data is managed by a graph database, and the connectivity between roads and intersections and various attribute information of road surface elements in the road can be obtained clearly and efficiently. The input location information only needs to be compared with the road range, and the roads within the range and the road segments that are connected to the road can be extracted to efficiently build a map around the vehicle. In addition, the vehicle positioning information has high accuracy and high retrieval matching precision.

[0127] Furthermore, after storing the information of multiple road segments in the graph database, the method also includes the steps of: constructing a multi-segment road topology map from the graph database based on the road data, the intersection data, the lane node data, the road object data and the road element data; connecting the multiple road topology maps based on the connection relationship, the position relationship, the inclusion relationship and the intersection relationship to obtain a road network topology map; accordingly, the step S50 includes: when receiving a retrieval segment, searching the graph database according to the retrieval segment and the road network topology map to generate a road element true value.

[0128] It should be noted that a road network topology map is a map used to represent the structure and connection relationship of a road network, focusing on the connection mode, nodes and overall topology between roads, rather than specific geographical details or precise geometric shapes. Through this map, the direction, intersections, branches and other relationships of roads can be clearly seen.

[0129] For example, to facilitate understanding of the road network topology map, refer to Figure 4 , Figure 4 A schematic diagram of the structure of a road network topology map provided in Example 1 of the present application. Intersection 1 includes arrow 1, lane line 1 and lane line 2 connected to road 1 through connection relationship 1; road 1 includes lane line 1, lane line 2 and road surface marking arrow 2, and is connected to road 2 through connection relationship 2; road 2 includes lane line 1, lane line 2 and road surface marking arrow 3, and is connected to intersection 2 through connection relationship 3; intersection 2 includes lane line 1, lane line 2 and arrow 4.

[0130] In this embodiment, by constructing a road network topology map, the graph database can manage the relationship between road elements, so that in subsequent fragment retrieval, relationship queries and graph traversal operations can be quickly performed based on the road network topology map, and it has high performance and scalability when processing large-scale data (i.e., large-scale point cloud data).

[0131] Step S50: When receiving the retrieval fragment, searching the graph database according to the retrieval fragment to generate a true value of the road element.

[0132] It should be noted that the search fragment may be search information input when performing a road image query, and may exist in the form of a picture.

[0133] The true value of a road element may be the actual value of various characteristics or attributes of the road, such as the actual value of specific parameters such as the length, width, slope, and curve radius of the road, which are stored in the point cloud scene map in the above-mentioned graph database.

[0134] In a specific implementation, when the truth value generating device receives a retrieval fragment, it can search in the graph database according to the retrieval fragment. When the element corresponding to the point cloud scene map is retrieved, the corresponding data can be extracted from the graph database to generate the road element truth value with the retrieval fragment.

[0135] In the technical solution provided in this embodiment, after the vehicle collects the two-dimensional image data and three-dimensional point cloud data of the road through the camera and the rotating laser radar, the vehicle will package the recorded self-vehicle position information, self-vehicle posture information, the above two-dimensional image data and three-dimensional point cloud data, save them in the form of a data packet, and send them to the truth generation device. After receiving the above data, the truth generation device can perform a series of preprocessing (such as time alignment, removal of redundant data, etc.) on the above two-dimensional image data, three-dimensional point cloud data and self-vehicle posture information, and then construct the point cloud scene graph according to the preprocessed data to obtain a point cloud scene map. Then, the road elements can be annotated based on the point cloud scene map, and the different main contents of the road in the point cloud scene map can be distinguished to obtain the annotated point cloud scene map, so as to facilitate the subsequent block division operation. And because they are all in one coordinate system, the annotation can be applied to all fragments once, which greatly improves the efficiency. At this time, the point cloud scene map can be divided into road elements (for example, by intersections, roads, ramps, etc.) and combined with road connectivity relationships (for example, connection relationships, intersection relationships, etc.), and multiple road segment information is obtained and stored in the graph database. Therefore, through the above operation, when the retrieval segment is received, it can be searched in the graph database according to the retrieval segment. When the element corresponding to the point cloud scene map is retrieved, the corresponding data can be extracted from the graph database, and the road element true value is generated with the retrieval segment. Since the present embodiment first constructs a point cloud scene map based on the two-dimensional image data and three-dimensional point cloud data of the road, and then divides the point cloud scene map after the road elements are annotated into blocks according to the road structure and the road connectivity relationship topological structure, and stores it in the graph database, the annotation can be applied to all road segment information once, so there is no need for tedious manual annotation. Finally, it can be searched in the graph database according to the retrieval segment to directly generate the road element true value under various working conditions, which greatly improves efficiency.

[0136] Based on the above first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above first embodiment can be referred to the above introduction, and will not be repeated in the following. Figure 5 , Figure 5 A flowchart diagram of the second embodiment of the method for generating true values ​​of road elements of the present application is provided.

[0137] In this example, step S50 includes steps S501 to S504:

[0138] Step S501: extracting the pose information and picture information of the search segment.

[0139] It should be noted that the pose information may be information related to the position and pose of the object or scene represented by the retrieval segment in the road. The image information may be various data and attributes such as pixel data, color, resolution, size, etc. included in the retrieval segment that describe the characteristics of the image itself.

[0140] Step S502: Locate according to the pose information in the road network topological map to determine the intersection node of the retrieval segment in the point cloud scene map.

[0141] It should be noted that the intersection node may be a key node in the road.

[0142] In a feasible implementation manner, step S502 of this embodiment includes the steps of: extracting the coordinate information of the vehicle in the pose information; traversing in the road network topological map based on the coordinate information; determining the intersection node range of the retrieval segment in the road network topological map according to the traversal result; and locating from the graph database according to the intersection node range and the coordinate information to determine the intersection node of the retrieval segment in the point cloud scene map.

[0143] For a given retrieval segment, the coordinate information in its pose information can be extracted, and the intersection node ranges of each road and intersection node in the graph database are retrieved through the coordinate information for preliminary screening, so as to determine a more accurate intersection node. Exemplarily, assume that at any moment for a given retrieval segment, the coordinate information of the vehicle is (x, y), and the graph database stores the range of each road and intersection node, that is, (x_min, y_min) and (x_max, y_max). By traversing all nodes, a node that meets x_min < x < x_max and y_min < y < y_max is found, and this node is the intersection node where the vehicle in the retrieval segment is located.

[0144] In this implementation manner, since the graph database stores the range information of each road segment, the intersection node range to which the vehicle of the retrieval segment belongs can be retrieved by comparing the input coordinate information. At the same time, since the graph database stores the connection relationship between each road segment and the road segment, as well as the information of the road surface elements included in the road segment, the information of the intersection node where the vehicle of the retrieval segment is located can be extracted. Therefore, the input position information only needs to compare the intersection node range, and extract the roads within the range and the road segments connected to this road to efficiently determine the intersection node.

[0145] Step S503: Extract the 3D ground truth corresponding to the retrieval segment from the graph database according to the intersection node.

[0146] Step S504: Based on the image information, the 3D true value is converted into a 2D true value according to the vehicle body coordinate system, and the 2D true value is used as the road element true value corresponding to the search segment.

[0147] It should be noted that the 3D truth value can be the actual data of the road in the point cloud scene map corresponding to the search segment in the three-dimensional space. The 2D truth value can be the accurate data on the two-dimensional plane corresponding to the 3D truth value converted to the search segment.

[0148] Exemplarily, for ease of understanding, in order to facilitate understanding of the conversion process between 3D true value and 2D true value, for any element on the point cloud scene map, take the ground arrow as an example to explain. At this time, multiple 3D points can be used to represent the range of the circumscribed cuboid of this road element. For the image collected by the vehicle at any time, the vehicle's position information and posture information can be obtained through sensor information. For the 2D image collected by a certain camera of the vehicle, there is a conversion relationship between 3D points and 2D points. The camera's internal parameter matrix can be recorded as K1 and the external parameter matrix can be recorded as K2. For the 3D point cloud coordinates, the image coordinate system (u, v) has the following conversion relationship:

[0149]

[0150]

[0151] Among them, K1 is the camera internal parameter; K2 is the camera external parameter; f x 、f y represents the focal length of the camera, which is the focal length in the x and y directions (in pixels); u0 and v0 are the coordinate values ​​of the principal point, that is, the coordinates of the center point of the image (in pixels); u and v are the values ​​of the image coordinate system; R is the rotation matrix, which describes the rotation of the camera coordinate system relative to the world coordinate system and defines the direction of the camera, that is, how the camera rotates to align with the world coordinate system; t is the translation vector, which describes the displacement of the origin of the camera coordinate system relative to the origin of the world coordinate system and defines the position of the center of the camera lens in the world coordinate system; X w , Y w , Z w is the coordinate value of the 3D point cloud; Z c It is the coordinate conversion relationship between 3D point cloud coordinates and image coordinate system.

[0152] The above-mentioned coordinate conversion can be realized through the exact coordinates in the sensor coordinate system without the need for additional human intervention.

[0153] For ease of understanding, refer to Figure 6 , Figure 6A flow chart of the fragment retrieval provided in Example 2 of the present application. Taking a 20-second retrieval fragment as an example, a vehicle may pass through two road fragments and an intersection, which are respectively within the range of Road 1, Road 2 and Intersection 1. According to the successive relationship of the road and intersection attributes, the corresponding point cloud map in the world coordinate system and its subordinate instances can be spliced. When the retrieval fragment is received, the truth value generation device can query the relevant fragment in the data graph database. First, the posture information and image information in the retrieval fragment are extracted, and the graph database is searched through the posture information. If the relevant information can be retrieved, the corresponding 3D truth value is extracted; finally, the 3D truth value and image information are converted into the vehicle body coordinate system, and the 2D truth value corresponding to the retrieval fragment can be generated and projected to obtain the scene map from the vehicle's perspective.

[0154] In this embodiment, the retrieval fragments collected by subsequent vehicles only need to input the posture information and the internal and external parameters of the sensor, and the scene map from the perspective of the vehicle around the vehicle at any time can be obtained through point cloud projection.

[0155] Furthermore, after the step of traversing the road network topology map based on the coordinate information, the present implementation also includes: if the coordinate information is not retrieved within the intersection node range of the road network topology map, determining that the point cloud map corresponding to the retrieval fragment is not included in the graph database; performing SLAM mapping according to the retrieval fragment to obtain a target point cloud map; based on the target point cloud map, returning to execute the step of annotating road elements on the point cloud scene map to obtain an annotated point cloud scene map, until the information of multiple road fragments corresponding to the target point cloud map is updated to the graph database; based on the updated graph database, returning to execute the step of extracting the posture information and image information of the retrieval fragment to determine the true value of the road element corresponding to the retrieval fragment.

[0156] It should be noted that SLAM (Simultaneous Localization and Mapping) is a technology that determines its own position in the environment and gradually builds a map of the environment.

[0157] In this embodiment, if Figure 6As shown in the figure, if the coordinate information is not retrieved in the graph database in the intersection node range of the road network topology map, it means that the point cloud map corresponding to the search fragment is not included in the graph database. At this time, SLAM mapping can be performed according to the search fragment to obtain the target point cloud map; then the road elements of the target point cloud map are re-annotated to obtain the annotated target point cloud map. Then the annotated target point cloud map is divided into blocks according to the preset road structure and road connection relationship topology structure, and the obtained multiple road fragment information is updated to the graph database. At this time, the graph database already contains the point cloud map corresponding to the search fragment. At this time, the graph database can be searched again through the posture information to extract the corresponding 3D true value; finally, the 3D true value and image information are converted into the vehicle body coordinate system, and the 2D true value corresponding to the search fragment can be generated and projected to obtain the scene map from the vehicle perspective. Therefore, for some scenes that do not exist in the graph database or are missing in the graph database, the search fragment can be updated to the online graph database by SLAM mapping and annotation, and the existing road relationship can be associated, which is more immediate than the traditional method.

[0158] For example, in order to help understand the implementation process of the road element true value generation method obtained by combining the above-mentioned embodiment 1 and embodiment 2, please refer to Figure 7 , Figure 7 The overall flow chart of the method for generating the true value of road elements provided in the second embodiment of the present application is as follows:

[0159] 1. The first step is data collection and annotation. The truth collection vehicle includes a laser radar and a camera. The collected road data can be divided into 20-second intervals to obtain multiple segments (such as segment 1, segment 2, and segment 3). Each segment is mapped by SLAM through the vehicle's navigation system to obtain a three-dimensional point cloud map of the driving range. The three-dimensional map corresponding to each segment is directly superimposed to remove duplicate points, and a point cloud scene map is obtained after multiple segments are spliced ​​together.

[0160] At this time, the point cloud scene map can be annotated with 3D road elements through 3D annotation tools (such as CloudCompare) (the previous fragments 1, 2, and 3 can be annotated separately to obtain the annotated fragments 1, 2, and 3). The annotation content may include road lines, curbs, road surface markings, traffic lights, and traffic signs, which are represented by a list of 3D points.

[0161] 2. Then divide the road structure. The road network can be divided based on functionality. The intersection is taken as the starting point. The intersection range is from the center of the intersection to the stop line. The intersection range is taken as the range of the intersection road node instance. The road node is intercepted in units of 100m from the intersection to the road along the road direction to establish a predecessor-successor relationship. The lines and signs marked within the road node range are extracted as the associated instances of the lane node. The divided elements may include: intersection, road-road, intersection-road, ramp, toll station / gas station, parking lot, etc.

[0162] Then, the above-divided elements are transformed into graph database elements, including creating nodes, creating attributes, and creating connection relationships to build a road network topology map. The specific process is as described above and will not be elaborated here.

[0163] 3. Truth query. Take a 20-second segment (such as segment N) as an example, extract the vehicle body position and posture information corresponding to the segment. When the posture information is lower than the true value of the image to be extracted, use linear interpolation to obtain the corresponding posture information under the image timestamp. Through the posture information, you can query and extract the road segment at the corresponding position in the graph database based on the node and connection relationship. If the node is in the graph database, you can extract the node annotation result and obtain the 3D point cloud true value of the path. Convert it to the vehicle coordinate system through the coordinate system. Then convert it to the image coordinate system through the camera's internal and external parameters IPM to obtain the true value of the corresponding road element in the image coordinate system.

[0164] If the node is not in the graph database, the continuous frame point cloud can be combined for SLAM mapping, and the obtained point cloud map can be annotated with road elements and updated and inserted into the graph database. Then the above truth value extraction process is performed to obtain the truth value.

[0165] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for generating true values ​​of road elements in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0166] This application also provides a road element true value generation device, please refer to Figure 8 , Figure 8 This is a schematic diagram of the module structure of the road element true value generating device according to an embodiment of the present application; the road element true value generating device comprises:

[0167] The data acquisition module 801 is used to acquire two-dimensional image data and three-dimensional point cloud data of the road through a vehicle;

[0168] A scene construction module 802 is used to construct a scene for the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map;

[0169] A road labeling module 803 is used to label road elements on the point cloud scene map to obtain a labeled point cloud scene map;

[0170] A graph database module 804 is used to divide the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and store the obtained multiple road segment information into a graph database;

[0171] The segment retrieval module 805 is used to search the graph database according to the retrieval segment when receiving the retrieval segment, and generate the true value of the road element.

[0172] As an implementation mode, the scene construction module 802 is also used to perform time-series alignment and distortion removal on the two-dimensional image data and the three-dimensional point cloud data to obtain preprocessed road data; perform feature extraction on the preprocessed road data to obtain a feature descriptor; perform forward optimization prediction based on the odometer information of the vehicle to obtain the vehicle posture information; perform factor graph optimization on the feature descriptor based on the vehicle posture information to obtain local maps of different fragments; merge the local maps of different fragments into the same world coordinate system to generate a point cloud scene map.

[0173] As an implementation mode, the graph database module 804 is also used to divide the annotated point cloud scene map according to a preset road structure to obtain corresponding node type data, and the node type data includes road data, intersection data, lane node data, road object data and road element data; perform relationship extraction on the annotated point cloud scene map according to a preset road connectivity relationship topological structure to obtain corresponding edge type data, and the edge type data includes connection relationship, position relationship, inclusion relationship and intersection relationship; use the road data, the intersection data, the lane node data, the road object data, the road element data, the connection relationship, the position relationship, the inclusion relationship and the intersection relationship as multiple road segment information, and store the multiple road segment information in the graph database.

[0174] As an implementation mode, the graph database module 804 is also used to construct a multi-segment road topology map from the graph database based on the road data, the intersection data, the lane node data, the road object data and the road element data; based on the connection relationship, the position relationship, the inclusion relationship and the intersection relationship, the multi-segment road topology map is connected to obtain a road network topology map; correspondingly, the fragment retrieval module 805 is also used to search the graph database according to the retrieval fragment and the road network topology map when receiving the retrieval fragment, and generate the road element true value.

[0175] As an implementation mode, the fragment retrieval module 805 is also used to extract the posture information and image information of the retrieval fragment; locate the retrieval fragment in the road network topology map according to the posture information, and determine the intersection node of the retrieval fragment in the point cloud scene map; extract the 3D true value corresponding to the retrieval fragment from the graph database according to the intersection node; based on the image information, convert the 3D true value into a 2D true value according to the vehicle body coordinate system, and use the 2D true value as the road element true value corresponding to the retrieval fragment.

[0176] As an implementation method, the fragment retrieval module 805 is also used to extract the coordinate information of the vehicle in the posture information; based on the coordinate information, traverse the road network topology map; determine the intersection node range of the retrieved fragment in the road network topology map according to the traversal result; locate from the graph database according to the intersection node range and the coordinate information, and determine the intersection node of the retrieved fragment in the point cloud scene map.

[0177] As an implementation mode, the fragment retrieval module 805 is also used to determine that the point cloud map corresponding to the retrieved fragment is not included in the graph database if the coordinate information is not retrieved within the intersection node range of the road network topology map; perform SLAM mapping based on the retrieved fragment to obtain a target point cloud map; based on the target point cloud map, return to execute the operation of annotating road elements on the point cloud scene map to obtain the annotated point cloud scene map, until the information of multiple road fragments corresponding to the target point cloud map is updated to the graph database; based on the updated graph database, return to execute the operation of extracting the posture information and image information of the retrieved fragment to determine the true value of the road element corresponding to the retrieved fragment.

[0178] Other embodiments or specific implementation methods of the road element true value generation device of the present application can refer to the above-mentioned method embodiments, which will not be repeated here.

[0179] The road element true value generation device provided by the present application adopts the road element true value generation method in the above embodiment, which can solve the technical problem that the existing method of searching through environmental information depends on the accuracy of the acquisition equipment and the accuracy of the detection algorithm, and requires repeated comparison and calculation during retrieval, resulting in low retrieval efficiency. Compared with the prior art, the beneficial effects of the road element true value generation device provided by the present application are the same as the beneficial effects of the road element true value generation method provided by the above embodiment, and the other technical features of the road element true value generation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0180] The present application provides a road element truth value generation device, which includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the road element truth value generation method in the above-mentioned embodiment 1.

[0181] Reference below Fig. 9 , Fig. 9 The device structure diagram of the hardware operating environment involved in the road element truth value generation method in the embodiment of the present application shows a schematic diagram of the structure of the road element truth value generation device suitable for implementing the embodiment of the present application. The road element truth value generation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The road element true value generating device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0182] like Fig. 9As shown, the road element true value generating device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the road element true value generating device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the road element truth value generation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a road element truth value generation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0183] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0184] The road element true value generation device provided by the present application adopts the road element true value generation method in the above embodiment, which can solve the technical problem that the existing method of searching through environmental information depends on the accuracy of the acquisition device and the accuracy of the detection algorithm, and requires repeated comparison and calculation during retrieval, resulting in low retrieval efficiency. Compared with the prior art, the beneficial effects of the road element true value generation device provided by the present application are the same as the beneficial effects of the road element true value generation method provided by the above embodiment, and the other technical features in the road element true value generation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0185] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0186] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0187] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for generating true values ​​of road elements in the above-mentioned embodiment.

[0188] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0189] The computer-readable storage medium may be included in the road element true value generating device; or may exist independently without being assembled into the road element true value generating device.

[0190] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0191] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0192] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0193] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned road element true value generation method, and can solve the technical problems that the existing method of searching through environmental information depends on the accuracy of the acquisition equipment and the accuracy of the detection algorithm, and requires repeated comparison and calculation during retrieval, resulting in low retrieval efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the road element true value generation method provided by the above-mentioned embodiment, and will not be repeated here.

[0194] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for generating true values ​​of road elements, characterized in that: The method comprises: Collect 2D image data and 3D point cloud data of roads through vehicles; Performing scene construction on the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map; Annotating the point cloud scene map with road elements to obtain an annotated point cloud scene map; Divide the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and store the obtained multiple road segment information in a graph database; When a retrieval fragment is received, the graph database is searched according to the retrieval fragment to generate a true value of a road element.

2. The method according to claim 1, characterized in that The step of constructing a scene based on the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map includes: Performing time-series alignment and distortion removal on the two-dimensional image data and the three-dimensional point cloud data to obtain pre-processed road data; Extracting features from the preprocessed road data to obtain feature descriptors; Perform forward optimization prediction based on the odometer information of the vehicle to obtain the vehicle posture information; Performing factor graph optimization on the feature descriptor according to the vehicle posture information to obtain local maps of different segments; The local maps of the different fragments are merged into the same world coordinate system to generate a point cloud scene map.

3. The method according to claim 1, characterized in that The step of dividing the annotated point cloud scene map into blocks according to the preset road structure and road connection relationship topological structure, and storing the obtained multiple road segment information in the graph database includes: Dividing the annotated point cloud scene map according to a preset road structure to obtain corresponding node type data, wherein the node type data includes road data, intersection data, lane node data, road object data, and road element data; Extracting relationships from the annotated point cloud scene map according to a preset road connectivity relationship topology structure to obtain corresponding edge type data, wherein the edge type data includes connection relationships, position relationships, inclusion relationships, and intersection relationships; The road data, the intersection data, the lane node data, the road object data, the road element data, the connection relationship, the position relationship, the inclusion relationship and the intersection relationship are used as multiple sections of road segment information, and the multiple sections of road segment information are stored in a graph database.

4. The method according to any one of claims 1 to 3, characterized in that After the step of storing the obtained multiple road segment information in the graph database, the method further includes: Constructing a multi-segment road topology graph from the graph database according to the road data, the intersection data, the lane node data, the road object data and the road element data; Based on the connection relationship, the position relationship, the inclusion relationship and the intersection relationship, the multiple road topology graphs are connected to obtain a road network topology map; Accordingly, when receiving the search fragment, the step of searching in the graph database according to the search fragment to generate the true value of the road element includes: When the search fragment is received, the graph database is searched according to the search fragment and the road network topology map to generate the true value of the road element.

5. The method according to claim 4, characterized in that The step of searching the graph database according to the search fragment and the road network topology map to generate the true value of the road element when the search fragment is received includes: Extracting the pose information and picture information of the searched fragment; Positioning the search segment in the road network topology map according to the position information, and determining an intersection node of the search segment in the point cloud scene map; According to the intersection node, extracting the 3D true value corresponding to the search segment from the graph database; Based on the image information, the 3D true value is converted into a 2D true value according to a vehicle body coordinate system, and the 2D true value is used as the road element true value corresponding to the retrieval segment.

6. The method according to claim 5, characterized in that The step of positioning in the road network topology map according to the posture information and determining the intersection node of the search segment in the point cloud scene map comprises: Extracting the coordinate information of the vehicle from the position information; Based on the coordinate information, traverse the road network topology map; Determine the intersection node range of the search fragment in the road network topology map according to the traversal result; According to the intersection node range and the coordinate information, positioning is performed from the graph database to determine the intersection node of the retrieval fragment in the point cloud scene map.

7. The method according to claim 6, characterized in that After the step of traversing the road network topology map based on the coordinate information, the method further includes: If the coordinate information is not retrieved within the intersection node range of the road network topology map, it is determined that the point cloud map corresponding to the search segment is not included in the graph database; Perform SLAM mapping according to the retrieval fragments to obtain a target point cloud map; Based on the target point cloud map, returning to the step of annotating the point cloud scene map with road elements to obtain the annotated point cloud scene map, until the information of multiple road segments corresponding to the target point cloud map is updated to the graph database; According to the updated graph database, return to the step of extracting the pose information and the picture information of the search segment to determine the true value of the road element corresponding to the search segment.

8. A road element true value generating device, characterized in that: The device comprises: A data acquisition module, used for acquiring two-dimensional image data and three-dimensional point cloud data of the road through a vehicle; A scene construction module, used to construct a scene for the two-dimensional image data and the three-dimensional point cloud data according to the vehicle's own posture information to obtain a point cloud scene map; A road labeling module is used to label road elements on the point cloud scene map to obtain a labeled point cloud scene map; A graph database module, used to divide the annotated point cloud scene map into blocks according to a preset road structure and road connection relationship topological structure, and store the obtained multiple road segment information in a graph database; The segment retrieval module is used to, upon receiving a retrieval segment, search the graph database according to the retrieval segment to generate a true value of a road element.

9. A road element true value generation device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for generating true values ​​of road elements according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the road element true value generation method according to any one of claims 1 to 7 are implemented.

Images