High-precision map generation method and device, electronic equipment and storage medium
By combining iterative training of map generation and correction of large models, the problem of map generation in the existing technology is solved, effective generation and correction of high-precision maps are achieved, and the accuracy and effectiveness of map production are improved.
Patent Information
- Application Number
- CN202510621738.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-14
- Publication Date
- 2025-08-05
AI Technical Summary
Due to the limitations of hardware conditions and technical means, existing end-to-end models based on BEV are difficult to generate high-precision map elements, resulting in limited map production levels.
By combining the combined role of map generation and map correction, map features are generated and corrected, including validity judgment and position deviation regression, and model iterative training is used to improve the accuracy and effectiveness of map features.
It significantly improves the map mapping level, can better meet the production requirements of high-precision maps, filter out redundant map elements and ensure the position accuracy of map feature points.
Smart Images
Figure CN120429375A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of high-precision map technology, and more specifically, to a method, device, electronic device, and storage medium for generating a high-precision map. Background Art
[0002] With the development of technology, high-precision maps with more accurate location information and richer road elements have gradually become the basic data for various location-based service systems such as autonomous driving and intelligent driving.
[0003] Taking autonomous driving as an example, the Bird's Eye View (BEV) solution is currently the primary method for generating online maps in real time. This solution relies on the Transformer architecture, first defining query features for map elements, and then using an end-to-end model to generate map elements corresponding to each query, typically represented as points and lines.
[0004] Currently, maps generated by existing end-to-end map generation solutions often differ from the requirements for high-precision maps. Due to real-time requirements, existing BEV-based end-to-end models are not suitable for generating high-precision map elements due to hardware and technical limitations, resulting in very limited map production capabilities. Summary of the Invention
[0005] The embodiments of the present disclosure at least provide a high-precision map generation method, device, electronic device and storage medium, which achieve continuous correction of map generation effects through the joint action of two large models, thereby better meeting the production requirements of high-precision maps.
[0006] In a first aspect, an embodiment of the present disclosure provides a method for generating a high-precision map, comprising:
[0007] Obtain various collected vehicle data;
[0008] Use the pre-trained map generation model to generate map elements from various vehicle-borne data and determine the map element data;
[0009] The map element data is corrected using a pre-trained map correction model to output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
[0010] In a possible implementation, after the corrected map data is output, the following closed-loop steps are executed cyclically:
[0011] generating new map data required for iterative model training of the map generation model based on the corrected map data;
[0012] Performing model iterative training on the map generation model based on the newly added map data to obtain an updated map generation model;
[0013] Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle-mounted data to determine new map element data, and correction processing is performed on the new map element data.
[0014] In a possible implementation, after the corrected map data is output, the following closed-loop steps are executed cyclically:
[0015] generating operation behavior data for converting the corrected map data into map data meeting preset requirements;
[0016] Performing model iterative training on the map correction model based on the operation behavior data to obtain an updated map correction model;
[0017] The input map element data is corrected based on the updated map generation macro model to output re-corrected map data.
[0018] In a possible implementation, after acquiring various vehicle-borne data, the vehicle-borne data is preprocessed in one or more of the following ways:
[0019] For a preset map area, determining the current frame vehicle data, the preceding and following frame vehicle data, and the multi-track vehicle data collected in the preset map area, and performing complementary processing on the current frame vehicle data for the same map element using the preceding and following frame vehicle data and the multi-track vehicle data based on a first calibration relationship between the preceding and following frame vehicle data and the current frame vehicle data and a second calibration relationship between the multi-track vehicle data and the current frame vehicle data;
[0020] For a preset map area, the vehicle direction in the vehicle data is set as the direction of the preset map area.
[0021] In the case where the on-board data includes an original image and an original 3D point cloud, orthophoto processing is performed based on the original image and the original 3D point cloud to generate an image orthophoto map and / or a laser orthophoto map as the on-board data.
[0022] In one possible implementation, the map correction model is trained according to the following steps:
[0023] Acquiring various sample vehicle data, map element data, and correction behavior data; the correction behavior data is used to indicate the judgment result of the validity of the map element itself and / or the position deviation regression result of the map element point;
[0024] Perform feature extraction and fusion based on the various sample vehicle data to generate a bird's-eye view BEV fusion feature;
[0025] Generating various map element features according to the BEV fusion features;
[0026] Based on the various map element features, map element data and correction behavior data, at least one round of model training is performed on the map generation model to be trained to obtain a pre-trained map correction model.
[0027] In a possible implementation, the map element features include map element semantic features, and the map element semantic features are determined according to the following steps:
[0028] Determining semantic features of each point constituting a map element based on the BEV fusion features;
[0029] The semantic features of each point are fully connected to determine the semantic features of the map elements.
[0030] In a possible implementation, the map element feature includes a map element position feature, and the map element position feature is determined according to the following steps:
[0031] Determining the position coding features of each point constituting a map element based on the BEV fusion features;
[0032] The position coding features of each point are fully connected to determine the position features of the map elements.
[0033] In a possible implementation, the map element features include map element category features for indicating map element categories and map element geometric features for indicating the shapes of map element sets.
[0034] In a possible implementation, the map correction model includes a validity judgment classifier, and the correction processing of the map element data using the pre-trained map correction model includes:
[0035] The map element data is input into the validity judgment classifier to determine the output valid map element, wherein the valid map element is used to indicate that the corresponding map element point is valid.
[0036] In a possible implementation, the map correction model includes a position regressor, and the correction processing of the map element data using the pre-trained map correction model includes:
[0037] The map element data is input into the position regressor, a position deviation value of the output map element point is determined, and the current position of the map element point is corrected based on the position deviation value to obtain a corrected map element point position.
[0038] In a second aspect, the present disclosure further provides a high-precision map generation device, comprising:
[0039] Acquisition module, used to acquire various collected vehicle data;
[0040] A generation module is used to generate map elements from various vehicle-borne data using a pre-trained map generation model to determine the map element data;
[0041] The correction module is used to use a pre-trained map correction model to correct the map element data and output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
[0042] In a possible implementation, the method further includes:
[0043] The first closed-loop module is used to cyclically execute the following closed-loop steps:
[0044] generating new map data required for iterative model training of the map generation model based on the corrected map data;
[0045] Performing model iterative training on the map generation model based on the newly added map data to obtain an updated map generation model;
[0046] Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle-mounted data to determine new map element data, and correction processing is performed on the new map element data.
[0047] In a possible implementation, the method further includes:
[0048] The second closed-loop module is used to cyclically execute the following closed-loop steps:
[0049] generating operation behavior data for converting the corrected map data into map data meeting preset requirements;
[0050] Performing model iterative training on the map correction model based on the operation behavior data to obtain an updated map correction model;
[0051] The input map element data is corrected based on the updated map generation macro model to output re-corrected map data.
[0052] In the third aspect, the present disclosure also provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a high-precision map generation method as described in any one of the first aspect and its various embodiments is performed.
[0053] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes a high-precision map generation method as described in any one of the first aspect and its various embodiments.
[0054] By using the above-mentioned high-precision map generation method, device, electronic device and storage medium, when various collected vehicle-mounted data are obtained, the pre-trained map generation large model can first be used to generate map elements for various vehicle-mounted data to determine the map element data, and then the pre-trained map correction large model can be used to correct the map element validity and / or map element point position of the map element data, so that the corrected map data can better filter out redundant map elements and ensure the position accuracy of the map element points. This will significantly improve the level of map making, thereby better meeting the production requirements of high-precision maps.
[0055] Other advantages of the present disclosure will be explained in more detail with reference to the following description and accompanying drawings.
[0056] It should be understood that the above description is only an overview of the technical solution of the present disclosure, so that the technical means of the present disclosure can be generally understood and then implemented in accordance with the contents of the description. In order to make the above and other purposes, features and advantages of the present disclosure more clearly understood, the following examples are used to illustrate specific embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope of protection. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work. Moreover, the same reference numerals are used to represent the same components throughout the drawings. In the drawings:
[0058] Figure 1A flowchart of a high-precision map generation method provided by an embodiment of the present disclosure is shown;
[0059] Figure 2 The following is an application flow chart of the high-precision map generation method provided by the embodiment of the present disclosure;
[0060] Figure 3 A workflow diagram for correcting a large model in a high-precision map generation method provided by an embodiment of the present disclosure is shown;
[0061] Figure 4 A schematic diagram of a high-precision map generation device provided by an embodiment of the present disclosure is shown;
[0062] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0063] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0064] In the description of the embodiments of the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the presence of the disclosed features, numbers, steps, actions, components, parts, or combinations thereof in the specification, and do not exclude the possibility of the presence of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0065] Unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. “And / or” in this article is only a way to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0066] The terms "first," "second," etc., are used solely to distinguish identical or similar technical features for ease of description and should not be construed as indicating or implying the relative importance or quantity of these technical features. Thus, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, unless otherwise specified, the term "plurality" means two or more than two.
[0067] Research has found that the maps generated by the current end-to-end map generation solutions are often different from the requirements of high-precision maps. First, the maps generated by the BEV solution used in autonomous driving often accept raw data as input, while the offline production of high-precision maps can post-process image preprocessing problems, posture problems, and calibration problems offline, and the output feature accuracy will be better. In addition, online maps can often only use the current frame and previous data, while offline map production can have a God's perspective and use subsequent frame data; secondly, due to real-time requirements and actual needs, the existing BEV-based end-to-end model is not suitable for generating high-precision map elements due to the limitations of existing hardware conditions and technical means; thirdly, the map generated by autonomous driving is not suitable for generating high-precision map elements. Figure 1 Generally, one model is used, while offline high-precision map production can use more models according to actual needs.
[0068] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, the present disclosure provides at least one high-precision map generation solution, which can continuously improve the production level of high-precision maps through continuous iteration of the model.
[0069] To facilitate understanding of this embodiment, a high-precision map generation method disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the high-precision map generation method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities. The electronic device includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a personal digital assistant (PDA), an in-vehicle device, a wearable device, etc. In some possible implementations, the high-precision map generation method can be implemented by a processor calling computer-readable instructions stored in a memory.
[0070] See also Figure 1 , which shows a flow chart of a high-precision map generation method provided by an embodiment of the present disclosure, the method comprising the following steps S101 to S103:
[0071] S101: Acquire various collected vehicle data;
[0072] S102: Using a pre-trained map generation model to generate map elements for various vehicle-borne data, and determining map element data;
[0073] S103: Use the pre-trained map correction model to correct the map element data and output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
[0074] To facilitate understanding of the high-precision map generation method provided by the embodiments of the present disclosure, the following briefly describes the application scenarios of this method. The high-precision map generation method provided by the embodiments of the present disclosure can be primarily applied to offline high-precision map production. Considering that existing solutions cannot meet the requirements of high-precision map production due to real-time and practical needs, the embodiments of the present disclosure provide a map generation solution that continuously improves the production level of high-precision maps through continuous iteration of the model.
[0075] The high-precision map generation method provided by the embodiment of the present disclosure can first generate map elements for the various vehicle-mounted data using a pre-trained map generation model to determine the map element data, and then use the pre-trained map correction model to correct the map element validity and / or map element point positions of the map element data, so that the corrected map data can better filter out redundant map elements and ensure the position accuracy of the map element points. This will significantly improve the level of map making and thus better meet the production requirements of high-precision maps.
[0076] The various vehicle-mounted data acquired here can include, for example, raw images, 3D point clouds, image matrices composed of surround-view cameras, image orthophotos, and orthophotos such as laser orthophotos. An image orthophoto can be a bird's-eye view generated using image information based on camera internal and external parameters, with pixels containing RGB information. A laser orthophoto can be a bird's-eye view generated using the position information of a laser point cloud, with pixels containing point cloud intensity information.
[0077] In order to achieve the generation of high-precision maps, the embodiment of the present disclosure pre-trains two major models, including a large map generation model and a large map correction model. The former is used to generate map element data based on vehicle-mounted data, and the latter is used to further correct the map elements generated by the model to further optimize the precision and accuracy of the map elements.
[0078] Before actually training or applying the model, Figure 2 As shown, the vehicle data can also be pre-processed to facilitate map generation.
[0079] The preprocessing involved here may include the following methods:
[0080] i. For the selected preset map area, the current frame vehicle data, previous and next frame vehicle data, and multi-track vehicle data collected in the preset map area can be obtained, and the current frame vehicle data can be supplemented based on the previous and next frames.
[0081] Specifically, a first calibration relationship between the vehicle-mounted data of the previous and next frames and the vehicle-mounted data of the current frame, and a second calibration relationship between the vehicle-mounted data of the multi-track and the vehicle-mounted data of the current frame can be pre-established here. Then, the vehicle-mounted data of the previous and next frames are projected onto the current vehicle-mounted data through the first calibration relationship to perform complement processing on the same map element, and the multi-track vehicle-mounted data are projected onto the current vehicle-mounted data through the second calibration relationship to perform complement processing on the same map element. Through the double complement processing of the previous and next frames and the multi-track, the processed vehicle-mounted data can better preserve the characteristics of the map elements. In this way, even if there are incomplete map elements in the current frame, the map elements themselves can be better restored through the complement processing, which will significantly improve the accuracy of subsequent map generation and its correction.
[0082] The front- and rear-frame vehicle data and multi-track vehicle data can be easily and accurately acquired offline. The use of front- and rear-frame vehicle data focuses on the temporal dimension and enriches the data integrity in the temporal dimension, while the use of multi-track vehicle data focuses on the spatial dimension and enriches the data integrity in the spatial dimension.
[0083] ii. Set the direction of the preset map area. During offline map creation, for example, the vehicle direction in the vehicle data can be set as the area direction, making it easier to process data in the same direction.
[0084] iii. Generate offline orthophotos. In addition to the original images and original 3D point clouds, image orthophotos and laser orthophotos can be generated based on these as additional inputs for the model.
[0085] In practical applications, a Maptr-like model can be used to train maps to generate large models. Specifically, map feature features can be defined first, and then the model outputs the corresponding map feature geometry and attribute information based on the map feature features as map feature data.
[0086] The map elements here include but are not limited to various lane lines, stop lines, ground symbols, road edges, etc. The geometric information is mainly represented by points and lines, describing the position, shape, direction, size and other information of the map elements. The attribute information describes the category, color, material and other information of the map elements.
[0087] like Figure 2As shown in the figure, the above preprocessing can be performed first, whether in the model training stage or the model prediction stage. In this way, the determined map element data can be used as new map data through post-processing and further corrected by the map correction model. The input of the map correction model comes from the output of the map generation model.
[0088] The reason why the map is produced here in combination with the correction processing of the map correction large model is mainly to avoid the problem of poor quality of the output map elements that may be caused by simply using the map to generate the large model, such as redundant map elements that need to be deleted during manual work, and poor position accuracy. The position accuracy of the output points is not high, and manual adjustment is required during actual work. Therefore, the map correction large model is needed for correction at this time.
[0089] The embodiment of the present disclosure mainly performs correction processing on the validity of map elements and the accuracy of the positions of map element points, and realizes map production with higher precision by combining validity and accuracy.
[0090] In order to further improve the accuracy of the generated map, the dual closed loop of the two models can be used to achieve continuous iteration of the model, thereby continuously improving the production level of high-precision maps.
[0091] On the one hand, the embodiments of the present disclosure can continuously use newly generated map data to iteratively update the map generation model. Specifically, the following closed-loop operations can be cyclically performed:
[0092] Step 1: Generate new map data required for iterative model training of the map generation model based on the corrected map data;
[0093] Step 2: Iteratively train the map generation model based on the newly added map data to obtain an updated map generation model;
[0094] Step 3: Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle data to determine new map element data, and the new map element data is corrected.
[0095] like Figure 2 The large closed-loop implementation process on the left shown in the figure determines that when new map data is identified, it can be added to the existing map data for the next training of the map generation model, and then the map generation model is iterated in a loop. At the same time, the map data added here is map data that has been manually worked on, so that the model training performance can be significantly improved through loop iteration.
[0096] On the other hand, the newly generated manual work data can be continuously used for iterative updates of the map correction model. Specifically, the following closed-loop steps can be executed cyclically:
[0097] Step 1: generating operation behavior data for converting the corrected map data into map data that meets preset requirements;
[0098] Step 2: Iteratively train the map correction model based on the operation behavior data to obtain an updated map correction model;
[0099] Step 3: Based on the updated map, the large model is generated to correct the input map element data to output the corrected map data.
[0100] like Figure 2 The small closed-loop implementation process shown on the right first generates the operational behavior data for transforming the corrected map data into map data that meets the preset requirements, and then updates the pre-trained map correction model based on the operational behavior data. The correction precision and accuracy are improved through the cyclic iteration of the map correction model.
[0101] Considering the key role of training a large map correction model in achieving high-precision mapping, the following will focus on the process of training a large map correction model. The embodiment of the present disclosure is mainly implemented through the following steps:
[0102] Step 1: Obtain various sample vehicle data, map element data, and correction behavior data; the correction behavior data is used to indicate the judgment result of the validity of the map element itself and / or the position deviation regression result of the map element point;
[0103] Step 2: Extract and fuse features based on various sample vehicle data to generate bird’s-eye view BEV fusion features;
[0104] Step 3: Generate various map element features based on BEV fusion features;
[0105] Step 4: Based on various map element features, map element data and correction behavior data, the map generation model to be trained is subjected to at least one round of model training to obtain a pre-trained map correction model.
[0106] The sample vehicle data used in the training process can refer to the aforementioned vehicle data, including but not limited to original images, original 3D, various orthophotos, etc., which will not be elaborated here.
[0107] When feature extraction and fusion are performed based on various sample vehicle data, BEV fusion features can be generated to determine various map element features. The map element features here include but are not limited to map element semantic features, map element location features, map element category features, and map element geometric features. These map element features are then used to perform one or more rounds of model training on the correction large model.
[0108] See also Figure 2 As shown in the figure, in the process of training large map correction models such as validity judgment classifiers and position regressors, it is necessary to rely on the operation behavior data (i.e., correction behavior data) generated by manual operations. The operation behavior data can be the operation records of the operators, which record the specific operation process from the pre-labeling results to the final result data. For example, the manual judgment results of map element redundancy and the manual adjustment results of map element positions can be used as the operation behavior data required for training.
[0109] Here, in order to understand the construction process of map element features, we will combine Figure 3 The application process of the corresponding map correction model is explained in detail.
[0110] like Figure 3 As shown in the figure, the vehicle data used in this example includes 3D point clouds, images, and laser orthophotos. Point cloud BEV features are generated by extracting features from the original point cloud. Common network structures include PointPillars and VoxelNet. Orthophoto BEV features can be determined by jointly extracting features from the image orthophoto and the laser orthophoto, such as using a ResNet-like backbone and FPN for feature extraction. In addition, other vehicle data can be included, and the corresponding BEV features can be extracted through an adapted feature extraction network. BEV feature maps from different sources need to be integrated through a feature fusion model.
[0111] The geometric shapes of the map elements involved in the embodiments of the present disclosure can be polylines and polygons, which can be used as geometric features of map elements. Polylines can refer to linear elements such as lane lines (solid lines, dashed lines), stop lines, guide lines, road edge lines, etc. in the map, and polygons can refer to ground symbols (arrows, speed limit signs, etc.), special road areas (grid no-parking areas, crosswalks), and other elements in the map.
[0112] Category characteristics of map elements, such as solid lines, dashed lines, edge lines, etc.
[0113] The semantic features of map elements can be determined through the following steps:
[0114] Step 1: Determine the semantic features of each point that constitutes the map element based on the BEV fusion feature;
[0115] Step 2: Fully connect the semantic features of each point to determine the semantic features of the map elements.
[0116] In practical applications, the semantic features of each point in the map element are first extracted: according to the coordinate position of the point, it is obtained based on the BEV fusion feature (graph) interpolation; then the semantic features of each point are connected together, and the semantic features of the map element are generated through the fully connected layer.
[0117] The location characteristics of map elements can be determined through the following steps:
[0118] Step 1: Determine the position coding features of each point that constitutes the map element based on the BEV fusion feature;
[0119] Step 2: Fully connect the location coding features of each point to determine the location features of the map elements.
[0120] In practical applications, for example, the position coding features of each point in the map element are first extracted through the sine-cosine function, and then the position coding features of the points are converted into map element position features through the fully connected layer.
[0121] The disclosed embodiment directly inputs the aforementioned various map element features into the map correction model and combines them with the corresponding correction behavior data to determine the correction result. Furthermore, the various map element features can be further fused to determine the validity of the map elements and the accuracy of the map element point locations based on the determined fused features. Specifically, the following methods are included:
[0122] Method 1: Input the map element data into the validity judgment classifier to determine the output valid map elements. The valid map elements are used to indicate that the corresponding map elements are valid.
[0123] Method 2: Input the map feature data into the position regressor, determine the position deviation value of the output map feature point, and correct the current position of the map feature point based on the position deviation value to obtain the corrected map feature point position.
[0124] like Figure 3 As shown, the high-precision map generation method provided by the embodiment of the present disclosure mainly performs two types of correction: feature validity judgment and map feature point position regression. Before performing specific corrections, map feature semantic features, map feature position features, map feature category features, and map feature geometric features can be fused together to determine fused features.
[0125] In the process of judging the validity of features, the fused features are input into the validity judgment classifier. If the feature is valid, the output is 1, and if it is invalid, the output is 0. For map features with an output of 0, the map feature needs to be deleted. In the process of regressing the position of map feature points, the fused features are input into the position regressor, and the position deviation value is output, that is, the difference between the current position of the map feature point and the actual point position. If the position accuracy is not high (the difference is large), the map feature point can be moved from one position to another.
[0126] When actually training the classifier to judge the effectiveness, cross entropy loss training can be used, and when actually training the position regressor, L1 loss training can be used.
[0127] It should be noted that before actually training the map to generate a large model, feature extraction and related preprocessing operations are also required based on the above operations. Please refer to the above description for details and will not be repeated here.
[0128] In addition, in order to improve data accuracy, the BEV feature resolution accuracy of the map correction large model needs to be higher than the BEV feature resolution of the map generation large model.
[0129] In this way, the embodiment of the present disclosure can effectively close the loop in the map generation process by performing multi-dimensional feature processing on the input data and combining the interaction of the two major models, thereby achieving a higher level of mapping and meeting the actual needs of high-precision maps.
[0130] In the description of this specification, the description with reference to the terms "some possible embodiments", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure, and the above terms do not necessarily represent the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.
[0131] About the method flow chart of the present disclosure embodiment, some operations are described as different steps performed in a certain order. Such flow chart is illustrative and not restrictive. Some steps described in this article can be grouped together and performed in a single operation, or some steps can be divided into multiple sub-steps and can be performed in an order different from that shown in this article. The various steps shown in the flow chart can be implemented in any way by any circuit structure and / or tangible mechanism (for example, by software running on a computer device, hardware (for example, the logical function implemented by a processor or chip), etc., and / or any combination thereof).
[0132] Those skilled in the art will understand that, in the method described in the above specific embodiments, the writing order of each step does not mean a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.
[0133] Based on the same inventive concept, a high-precision map generation device corresponding to the high-precision map generation method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned high-precision map generation method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0134] Reference Figure 4 FIG. 2 is a schematic diagram of a high-precision map generation device provided by an embodiment of the present disclosure, the device comprising: an acquisition module 201, a generation module 202, and a correction module 203; wherein,
[0135] An acquisition module 201 is used to acquire various collected vehicle-mounted data;
[0136] A generation module 202 is used to generate map elements from various vehicle-borne data using a pre-trained map generation model to determine map element data;
[0137] The correction module 203 is used to use the pre-trained map correction model to correct the map element data and output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
[0138] By using the above-mentioned high-precision map generation device, when various collected vehicle-mounted data are obtained, the pre-trained map generation large model can first be used to generate map elements for various vehicle-mounted data to determine the map element data, and then the pre-trained map correction large model can be used to correct the map element validity and / or map element point position of the map element data, so that the corrected map data can better filter out redundant map elements and ensure the position accuracy of the map element points. This will significantly improve the level of map making, thereby better meeting the production requirements of high-precision maps.
[0139] In a possible implementation, the method further includes:
[0140] The first closed-loop module 204 is configured to cyclically execute the following closed-loop steps:
[0141] Generate additional map data required for iterative model training of the map generation model based on the corrected map data;
[0142] Iteratively train the map generation model based on the newly added map data to obtain an updated map generation model;
[0143] Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle data to determine new map element data, and correction processing is performed on the new map element data.
[0144] In a possible implementation, the method further includes:
[0145] The second closed-loop module 205 is configured to cyclically execute the following closed-loop steps:
[0146] generating operation behavior data for converting the corrected map data into map data meeting preset requirements;
[0147] Perform iterative training on the map correction model based on the operation behavior data to obtain an updated map correction model;
[0148] The input map element data is corrected based on the updated map generation macro model to output re-corrected map data.
[0149] In a possible implementation, the method further includes:
[0150] The pre-processing module 206 is used to pre-process the vehicle data after acquiring various vehicle data in one or more of the following ways:
[0151] For a preset map area, determining the current frame vehicle data, the preceding and following frame vehicle data, and the multi-track vehicle data collected in the preset map area, and based on a first calibration relationship between the preceding and following frame vehicle data and the current frame vehicle data, and a second calibration relationship between the multi-track vehicle data and the current frame vehicle data, performing a complement process on the current frame vehicle data using the preceding and following frame vehicle data and the multi-track vehicle data;
[0152] For a preset map area, the vehicle direction in the vehicle data is set as the direction of the preset map area.
[0153] In the case where the on-board data includes original images and original 3D point clouds, orthophoto processing is performed based on the original images and original 3D point clouds to generate image orthophotos and / or laser orthophotos as the on-board data.
[0154] In one possible implementation, the correction module 203 is configured to train the map correction model according to the following steps:
[0155] Acquiring various sample vehicle data, map element data, and correction behavior data; the correction behavior data is used to indicate the judgment result of the validity of the map element itself and / or the position deviation regression result of the map element point;
[0156] Feature extraction and fusion are performed based on various sample vehicle data to generate bird’s-eye view BEV fusion features;
[0157] Generate various map element features based on BEV fusion features;
[0158] Based on various map element features, map element data and correction behavior data, at least one round of model training is performed on the map generation model to be trained to obtain a pre-trained map correction model.
[0159] In a possible implementation, the map element features include map element semantic features. The correction module 203 is configured to determine the map element semantic features according to the following steps:
[0160] Determine the semantic features of each point that constitutes the map element based on BEV fusion features;
[0161] The semantic features of each point are fully connected to determine the semantic features of map elements.
[0162] In a possible implementation, the map element features include map element position features, and the correction module 203 is configured to determine the map element position features according to the following steps:
[0163] Determine the location coding features of each point that constitutes the map element based on the BEV fusion feature;
[0164] The position coding features of each point are fully connected to determine the position features of map elements.
[0165] In a possible implementation, the map element features include map element category features for indicating map element categories and map element geometric features for indicating the shapes of map element sets.
[0166] In one possible implementation, the map correction model includes a validity judgment classifier and a correction module 203, which is configured to perform correction processing on map element data using the pre-trained map correction model according to the following steps:
[0167] The map element data is input into the validity judgment classifier to determine the output valid map elements. The valid map elements are used to indicate that the corresponding map element points are valid.
[0168] In one possible implementation, the map correction model includes a position regressor and a correction module 203 for correcting map feature data using a pre-trained map correction model according to the following steps:
[0169] The map feature data is input into the position regressor, the position deviation value of the output map feature point is determined, and the current position of the map feature point is corrected based on the position deviation value to obtain the corrected map feature point position.
[0170] It should be noted that the device in the embodiment of the present disclosure can implement each process of the embodiment of the aforementioned method and achieve the same effects and functions, which will not be repeated here.
[0171] The present disclosure also provides an electronic device, such as Figure 5 FIG. 3 is a schematic diagram of the electronic device structure provided by an embodiment of the present disclosure, including: a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions (e.g., Figure 4 The acquisition module 201, the generation module 202, the execution instructions corresponding to the correction module 203 in the device, etc., when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the following processing:
[0172] Obtain various collected vehicle data;
[0173] Use the pre-trained map generation model to generate map elements from various vehicle-borne data and determine the map element data;
[0174] The map element data is corrected using a pre-trained map correction model, and the corrected map data is output. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself, and position deviation regression processing for the map element points.
[0175] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the high-precision map generation method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.
[0176] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the high-precision map generation method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0177] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0178] Each embodiment of this disclosure is described in a progressive manner. The same or similar parts between the various embodiments can be referenced across them, and each embodiment focuses on the differences between the other embodiments. In particular, the descriptions of the apparatus, device, and computer-readable storage medium embodiments are simplified because they are generally similar to the method embodiments. For relevant details, please refer to the descriptions of the method embodiments.
[0179] The apparatus, equipment and computer-readable storage medium provided in the embodiments of the present disclosure correspond one-to-one to the method. Therefore, the apparatus, equipment and computer-readable storage medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus, equipment and computer-readable storage medium will not be repeated here.
[0180] It should be understood by those skilled in the art that the embodiments of the present disclosure can be implemented as methods and apparatus (devices or systems), or computer-readable storage media. Therefore, the present disclosure can be implemented entirely in hardware, entirely in software, or in a combination of software and hardware. Furthermore, the present disclosure can be implemented in the form of a computer-readable storage medium implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0181] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (equipment or system) and computer-readable storage media according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0182] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, wherein the instruction device implements the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.
[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0184] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0185] Memory can include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0186] Computer-readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. In addition, although the operations of the disclosed method are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all shown operations must be performed to achieve the desired result. In addition, certain steps may be omitted, multiple steps may be combined into a single step, and / or a step may be broken down into multiple sub-steps.
[0187] Although the spirit and principles of the present disclosure have been described above with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division into various aspects does not mean that the features of these aspects cannot be combined. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims
1. A high-precision map generation method, characterized in that: include: Obtain various collected vehicle data; Use the pre-trained map generation model to generate map elements from various vehicle-borne data and determine the map element data; The map element data is corrected using a pre-trained map correction model to output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
2. The method according to claim 1, characterized in that After outputting the corrected map data, the following closed-loop steps are executed cyclically: generating new map data required for iterative model training of the map generation model based on the corrected map data; Performing model iterative training on the map generation model based on the newly added map data to obtain an updated map generation model; Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle-mounted data to determine new map element data, and correction processing is performed on the new map element data.
3. The method according to claim 1, characterized in that After outputting the corrected map data, the following closed-loop steps are executed cyclically: generating operation behavior data for converting the corrected map data into map data meeting preset requirements; Performing model iterative training on the map correction model based on the operation behavior data to obtain an updated map correction model; The input map element data is corrected based on the updated map generation macro model to output re-corrected map data.
4. The method according to any one of claims 1 to 3, characterized in that After obtaining various vehicle-borne data, the vehicle-borne data is preprocessed in one or more of the following ways: For a preset map area, determining the current frame vehicle data, the preceding and following frame vehicle data, and the multi-track vehicle data collected in the preset map area, and performing complementary processing on the current frame vehicle data for the same map element using the preceding and following frame vehicle data and the multi-track vehicle data based on a first calibration relationship between the preceding and following frame vehicle data and the current frame vehicle data and a second calibration relationship between the multi-track vehicle data and the current frame vehicle data; For a preset map area, setting the vehicle direction in the vehicle data as the direction of the preset map area; In the case where the on-board data includes an original image and an original 3D point cloud, an image orthophoto and / or a laser orthophoto are generated based on the original image and the original 3D point cloud as the on-board data.
5. The method according to any one of claims 1 to 3, characterized in that Follow the steps below to train the map correction model: Acquiring various sample vehicle data, map element data, and correction behavior data; the correction behavior data is used to indicate the judgment result of the validity of the map element itself and / or the position deviation regression result of the map element point; Perform feature extraction and fusion based on the various sample vehicle data to generate a bird's-eye view BEV fusion feature; Generating various map element features according to the BEV fusion features; Based on the various map element features, map element data and correction behavior data, at least one round of model training is performed on the map generation model to be trained to obtain a pre-trained map correction model.
6. The method according to claim 5, characterized in that The map element features include map element semantic features, and the map element semantic features are determined according to the following steps: Determining semantic features of each point constituting a map element based on the BEV fusion features; The semantic features of each point are fully connected to determine the semantic features of the map elements.
7. The method according to claim 5, characterized in that The map element features include map element position features, and the map element position features are determined according to the following steps: Determining the position coding features of each point constituting a map element based on the BEV fusion features; The position coding features of each point are fully connected to determine the position features of the map elements.
8. The method according to claim 5, characterized in that The map element features include map element category features for indicating map element categories and map element geometric features for indicating the shapes of map element sets.
9. The method according to claim 5, characterized in that The map correction model includes a validity judgment classifier, and the map element data is corrected using the pre-trained map correction model, including: The map element data is input into the validity judgment classifier to determine the output valid map element, where the valid map element is used to indicate that the corresponding map element is valid.
10. The method according to claim 5, characterized in that The map correction model includes a position regressor, and the map element data is corrected using the pre-trained map correction model, including: The map element data is input into the position regressor, a position deviation value of the output map element point is determined, and the current position of the map element point is corrected based on the position deviation value to obtain a corrected map element point position.
11. A high-precision map generation device, characterized in that: include: Acquisition module, used to acquire various collected vehicle data; A generation module is used to generate map elements from various vehicle-borne data using a pre-trained map generation model to determine the map element data; The correction module is used to use a pre-trained map correction model to correct the map element data and output the corrected map data. The correction processing includes one or two of the following processing methods: validity judgment processing for the map element itself and position deviation regression processing for the map element point.
12. The device according to claim 11, characterized in that Also includes: The first closed-loop module is used to cyclically execute the following closed-loop steps: generating new map data required for iterative model training of the map generation model based on the corrected map data; Performing model iterative training on the map generation model based on the newly added map data to obtain an updated map generation model; Based on the updated map generation model, map element generation processing is performed on the re-acquired vehicle-mounted data to determine new map element data, and correction processing is performed on the new map element data.
13. The device according to claim 11, characterized in that Also includes: The second closed-loop module is used to cyclically execute the following closed-loop steps: generating operation behavior data for converting the corrected map data into map data meeting preset requirements; Performing model iterative training on the map correction model based on the operation behavior data to obtain an updated map correction model; The input map element data is corrected based on the updated map generation macro model to output re-corrected map data.
14. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the high-precision map generation method as described in any one of claims 1 to 10 is executed.
15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the high-precision map generation method according to any one of claims 1 to 10.