Lightweight multi-run mapping method and apparatus, device, and vehicle

Abstract

The present application relates to the technical field of electronic maps, and discloses a lightweight multi-run mapping method and apparatus, a device, and a vehicle. The lightweight multi-run mapping method comprises: acquiring a memory map created after a vehicle has performed multiple runs on a target route, the memory map comprising the target route; performing trajectory alignment between the target route in the memory map and a corresponding route in a historical map to obtain two aligned maps; identifying environmental difference information between the memory map and the historical map, and on the basis of the environmental difference information, extracting corresponding environmental features; and fusing the two aligned maps and the environmental features to generate an electronic map comprising the target route and the environmental features. According to the present method, road information and environmental information surrounding the road are all collected, thereby preventing omissions of route information caused by single-run data collection, and improving the completeness of mapped road information. In addition, surrounding environmental features are also fused, so that the fused electronic map can better reproduce a real map scene.

Description

A lightweight multi-trip mapping method, apparatus, equipment and vehicle

[0001] This application claims priority to Chinese Patent Application No. 202411217855.X, filed on August 30, 2024, entitled "A Lightweight Multi-Trip Mapping Method, Apparatus, Equipment and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of electronic map technology, specifically to a lightweight multi-trip mapping method, apparatus, equipment, and vehicle. Background Technology

[0003] With the widespread use of mobile devices, electronic maps have seamlessly integrated into people's daily lives. Whether it's smartphones, tablets, or in-car navigation systems, they provide convenient map services anytime, anywhere. Creating electronic maps requires collecting a large amount of basic geographic data, including road networks, topography, building layouts, and environmental information. This data may come from various sources, such as satellite remote sensing, aerial photography, ground surveying, and data collected from users' driving activities.

[0004] Currently, the method of creating maps using memorized routes involves collecting information such as road boundaries and road markings from a user's vehicle after a single trip along a preset route. However, single-trip memorized route mapping may not fully extract semantic information about the roads, such as lanes and traffic signs. There may be omissions, obstructions, or geometric errors in the collected route information, resulting in low accuracy or incomplete road information in the generated electronic map.

[0005] Summary of the Invention

[0006] In view of this, this application provides a lightweight multi-trip mapping method, apparatus, equipment and vehicle to solve the problems of low accuracy or incomplete road information in the created electronic maps.

[0007] Firstly, this application provides a lightweight multi-pass mapping method, which includes:

[0008] Obtain a memory map created after the vehicle has traveled multiple times on the target route, the memory map containing the target route;

[0009] Align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps.

[0010] Identify the environmental differences between the memory map and the historical map, and extract the corresponding environmental features based on the environmental differences.

[0011] The two aligned maps and the environmental features are merged to generate an electronic map that includes the target route and the environmental features.

[0012] The method provided in this paper uses sampling data after a vehicle travels multiple times on the target route to build a map. Compared with the method of building a map using sampling data from a single trip, this method can collect all information about the road and the surrounding environment, avoiding omissions of route information in single-trip sampling data and improving the completeness of road information in the map.

[0013] Furthermore, by aligning the created memory map and historical map with their trajectories, the target routes of the two aligned maps match, improving the accuracy of subsequent fusion mapping. It also incorporates surrounding environmental features, making the fused electronic map more accurately reflect the real-world map scene. Moreover, since only the two aligned trajectories and the intermediate results of the fusion mapping need to be stored, this method requires fewer storage resources compared to storing and reporting more route data, thus saving storage space.

[0014] In conjunction with the first aspect, in one possible implementation, the target route in the memory map is aligned with the corresponding route in the historical map to obtain two aligned maps, including:

[0015] The target route in the memory map is matched with the historical map to find whether there is at least one route in the historical map that matches the target route;

[0016] If so, the matching routes in the historical map are aligned with the target route to obtain two aligned maps.

[0017] In conjunction with the first aspect, in another possible implementation, the matching routes in the historical map are aligned with the target route to obtain two aligned maps, including:

[0018] Obtain the first set of keyframes corresponding to the matching routes in the historical map, and the second set of keyframes corresponding to the target route, respectively.

[0019] At least one keyframe in the first keyframe set and the second keyframe set is associated with a keyframe, map features are associated between the historical map and the memory map, and pose optimization is performed on the associated keyframe to obtain the two aligned maps.

[0020] In conjunction with the first aspect, in another possible implementation, keyframe association is performed on at least one keyframe in the first keyframe set and the second keyframe set, including: pairing keyframes in the first keyframe set and the second keyframe set according to the trajectory of the target route based on a hidden Markov model, to generate at least one keyframe pair.

[0021] The association of map elements between the historical map and the memory map includes: establishing an association relationship between map elements of the historical map and the memory map based on the at least one keyframe pair;

[0022] The step of performing pose optimization processing on at least one associated keyframe to obtain the aligned two maps includes: optimizing the pose of the keyframes in the first keyframe set and the second keyframe set based on the association relationship between the map features, to obtain the pose of the optimized trajectory; and obtaining the aligned two maps based on the pose of the optimized trajectory, the map features, and the association relationship between the map features.

[0023] In conjunction with the first aspect, in another possible implementation, establishing the association between map features of the historical map and the memory map based on the at least one keyframe pair includes:

[0024] Based on the at least one keyframe pair, determine the discrete and / or continuous elements of the historical map and the memory map, and establish the association between the discrete and / or continuous elements;

[0025] The discrete elements include at least one of the following:

[0026] Dashed lines are used to indicate a portion of lane dividers or lane markings;

[0027] A pedestrian crossing is used to indicate an area for pedestrians to cross the road.

[0028] A stop line is a marking line used to indicate that vehicles must stop and wait in front of this line;

[0029] Arrows are used to indicate direction;

[0030] The continuous elements include one or more of lane boundaries and road boundaries.

[0031] In conjunction with the first aspect, in another possible implementation, the step of optimizing the pose of the optimized trajectory by considering the relationships between the map features and the keyframes in the first and second keyframe sets to obtain the pose of the optimized trajectory includes:

[0032] Obtain an objective function that minimizes the positional differences between associated map features and keyframes;

[0033] The pose of the keyframes in each trajectory is adjusted using the objective function to obtain the pose of the optimized trajectory. The pose of the keyframes includes the position and orientation of the keyframes.

[0034] In conjunction with the first aspect, in yet another possible implementation, after obtaining the aligned two maps, the process further includes:

[0035] Check whether the absolute distance and distribution between the map features that have established the association are within a preset range; and check whether some or all of the important map features have been successfully associated with the pose in the trajectory; and check whether the difference between the pose of the optimized trajectory and the pose before optimization is within an allowable range.

[0036] If one or more results in the check are negative, the negative result will be re-associated or optimized to obtain two new maps, until the output check results meet the conditions.

[0037] If all results in the check are yes, then the two aligned maps are output.

[0038] In conjunction with the first aspect, in another possible implementation, acquiring the memory map created after the vehicle has traveled multiple times on the target route includes: acquiring sampling data of the vehicle traveling multiple times on the target route, the sampling data including route information of the target route; and creating the memory map based on the environmental information of the target route.

[0039] In conjunction with the first aspect, in yet another possible implementation, the sampling data also includes environmental information;

[0040] The step of identifying environmental differences between the memory map and the historical map, and extracting corresponding environmental features based on the environmental differences, includes:

[0041] Based on the environmental information, an identification algorithm is used to identify environmental differences between the memory map and the historical map, including construction detours and / or road signs.

[0042] Extract the corresponding environmental features based on the construction detour and / or the road signs;

[0043] The environmental features of the construction detour and / or the road signs are determined as environmental features for map fusion based on the weight values ​​of the environmental difference information.

[0044] Secondly, this application also provides a lightweight multi-trip mapping device, the device comprising:

[0045] The acquisition module is used to acquire a memory map created after the vehicle has traveled multiple times on the target route, and the memory map contains the target route;

[0046] The alignment module is used to align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps.

[0047] The identification module is used to identify environmental differences between the memory map and the historical map, and to extract corresponding environmental features based on the environmental differences.

[0048] The generation module is used to fuse the two aligned maps and the environmental features to generate an electronic map containing the target route and the environmental features.

[0049] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the lightweight multi-pass mapping method of the first aspect or any corresponding embodiment described above.

[0050] Optionally, the electronic device is a vehicle control unit.

[0051] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment.

[0052] In addition, this application provides a computer program product, including computer instructions for causing a computer to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment above.

[0053] Fifthly, this application also provides a vehicle including a vehicle-mounted controller, wherein the vehicle-mounted controller is used to execute the lightweight multi-trip mapping method described in the first aspect or any corresponding embodiment above.

[0054] The lightweight multi-trip mapping method, apparatus, equipment, and vehicle provided in this invention utilize sampling data from multiple trips along the target route to create a map. Compared to mapping using single-trip sampling data, this method can collect all road and surrounding environmental information, avoiding omissions of route information in single-trip sampling data and improving the completeness of road information in the map.

[0055] Furthermore, by aligning the created memory map and historical map with their trajectories, the target routes of the two aligned maps match, improving the accuracy of subsequent fusion mapping. It also incorporates surrounding environmental features, making the fused electronic map more accurately reflect the real-world map scene. Moreover, since only the two aligned trajectories and the intermediate results of the fusion mapping need to be stored, this method requires fewer storage resources compared to storing and reporting more route data, thus saving storage space. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 is a flowchart illustrating a lightweight multi-pass mapping method according to an embodiment of this application;

[0058] Figure 2a is a schematic diagram of a target route before trajectory alignment according to an embodiment of this application;

[0059] Figure 2b is a schematic diagram of the trajectory alignment of a target route according to an embodiment of this application;

[0060] Figure 3 is a flowchart illustrating another lightweight multi-pass mapping method according to an embodiment of this application;

[0061] Figure 4 is a flowchart illustrating another lightweight multi-pass mapping method according to an embodiment of this application;

[0062] Figure 5 is a structural block diagram of a lightweight multi-pass mapping device according to an embodiment of this application;

[0063] Figure 6 is a structural block diagram of another lightweight multi-pass mapping device according to an embodiment of this application;

[0064] Figure 7 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application;

[0065] Figure 8 is a structural schematic diagram of a vehicle according to an embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Electronic maps integrate advanced Geographic Information System (GIS), remote sensing technology, Global Positioning System (GPS), and Internet technology, transforming traditional paper maps into a highly interactive, information-rich, and real-time updated digital platform.

[0068] In the process of creating electronic maps, user vehicles typically collect data on single-trip routes and report this data to a server or network center, which then builds the map based on the data reported by the user vehicles. This mapping method has the following problems:

[0069] 1. Low accuracy: The geometric accuracy of single-trip mapping data is greatly affected by factors such as the performance of the data acquisition equipment and environmental changes. In addition, single-trip mapping may have omissions, obstructions or geometric accuracy errors in the collected route information, resulting in low accuracy or incomplete road information in the generated electronic map.

[0070] 2. Redundant Information Management: Merging multiple mapping data may generate a large amount of redundant information, increasing storage and computational burden. Furthermore, merging sampling data from multiple routes requires effective compensation and correction for geometric errors, further increasing computational load.

[0071] 3. Adaptability to environmental changes. The sampling environment is dynamically changing, and the mapping data collected at different time periods may vary significantly.

[0072] This application aims to detect areas of current change and update electronic maps by using sampled data from multiple memory learnings along the same route, thereby solving the problem of single-trip mapping and achieving high-quality fusion of lightweight multi-trip mapping data to meet the needs of AI-powered driving and other scenarios for high-quality, lightweight maps.

[0073] It should be noted that the lightweight multi-pass mapping method provided in this application can be implemented by a mapping device or equipment, which can be implemented as part or all of an electronic device through software, hardware, or a combination of both. This electronic device can be a vehicle-side control device, such as a vehicle controller, or it can be a terminal device or a server device. This application does not specifically limit the type of electronic device. In the following method embodiments, the implementation is always described using an electronic device as an example.

[0074] According to an embodiment of this application, a lightweight multi-pass mapping method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0075] This embodiment provides a lightweight multi-pass mapping method. "Lightweight" means that the data or information sampling devices, information acquisition, and processing algorithms in this technical solution are easier to obtain and require fewer computational resources compared to traditional methods. For example, the sampling device can be a user vehicle equipped with sensors. Compared to expensive professional mapping equipment such as LiDAR and mapping-grade GNSS (Global Navigation Satellite System), using a user vehicle is less costly and can be implemented in a general user's vehicle. Furthermore, in terms of sampling data and its processing, the method reduces the amount of data, computational resources required, and processing time compared to sampling data based on professional mapping equipment.

[0076] The method of this embodiment can be used in the aforementioned electronic devices. Figure 1 is a flowchart of a lightweight multi-pass mapping method according to an embodiment of this application. The process includes:

[0077] Step S101: Obtain the memory map created after the vehicle has traveled multiple times on the target route. The memory map contains the target route.

[0078] The target route is the route that the user vehicle expects to map, such as the route from starting point A to ending point B, i.e., the target route A→B. The user vehicle traveling from point A to point B along the target route is considered to have completed one trip on the target route. The user vehicle then travels from point A to point B again to complete a second trip on the target route, and so on. Multiple trips by the user vehicle from point A to point B are referred to as multiple trips on the target route. It should be noted that the vehicle traveling multiple trips on the target route can be the same user vehicle or different user vehicles; this embodiment does not impose such limitations.

[0079] One implementation includes: acquiring sampling data of a vehicle traveling multiple times on a target route, the sampling data including route information of the target route; and creating a memory map based on environmental information of the target route.

[0080] The sampling data is obtained by mass-produced vehicles or low-cost data collection equipment through multiple sampling trips along the target route A→B.

[0081] The route information may include the following: Route orientation: indicating the start and end points of the sampling route, as well as the major nodes passed through, which helps to understand the overall flow direction of the sampling activity. Horizontal alignment: including the specific locations and lengths of straight sections and curved sections (such as left and right turns), which helps to accurately reconstruct the sampling path on the map. Mileage markers: to accurately represent the total length of the route and the length of each segment, kilometer markers and hexagonal markers are usually set, with corresponding mileage values ​​marked. In addition, the route information also includes information on intersections and grade-separated structures.

[0082] Each time a user vehicle travels along the target route, it acquires a sample of data. This data is then reported to a server or vehicle controller, which retrieves and updates it in real time to generate a memory map. Methods for creating memory maps include, but are not limited to, SLAM (Simultaneous Localization and Mapping). SLAM is primarily used for real-time localization and map building in robots and autonomous vehicles. It can also compare real-time sensor data with existing map data to identify changes or errors in the map.

[0083] In addition, the sampling data may also include environmental information.

[0084] Step S102: Align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps.

[0085] Historical maps are one or more electronic maps pre-stored locally on a server or vehicle controller. Each electronic map includes map features containing the target route based on historical records. Historical maps can be generated on the server side after being reported by other user vehicles based on collected data, or generated and reported by other clients or third-party organizations.

[0086] The two aligned maps described above are designated as Map 1 and Map 2. Map 1 is an aligned map generated based on a memory map, while Map 2 is an aligned map generated based on a historical map. In this embodiment, there is one historical map, but in practice, there can be two or more; this embodiment does not impose any restrictions on this.

[0087] One approach to route alignment is to load data from memory maps and historical maps using GIS software (such as ArcGIS, QGIS, etc.). Within the software, spatial analysis tools are used to align the two routes. This alignment process includes algorithms such as keyframe association, keyframe pairing, path correction, and pose optimization.

[0088] Referring to Figures 2a and 2b, these are schematic diagrams of the target route in the memory map before and after alignment with the corresponding route in the historical map. Figure 2a is a schematic diagram of the trajectory before alignment, where the trajectories of the two routes differ significantly. Figure 2b is a schematic diagram of the trajectory after alignment, where the route trajectories of the two maps overlap or are basically overlapped.

[0089] Step S103: Identify the environmental differences between the memory map and the historical map, and extract the corresponding environmental features based on the environmental differences.

[0090] Environmental information refers to information on the memory map or historical map, including topography, roadside landmarks, and meteorological conditions. Topography includes the undulations and landforms (such as mountains, rivers, and lakes) along the sampling route, as well as the distribution of features (such as buildings, roads, and vegetation); this information helps in analyzing the geographical characteristics of the sampling area. Meteorological conditions include temperature, humidity, air pressure, wind speed, and wind direction at the time of sampling, which significantly affect the sampling results and therefore need to be recorded in detail. Roadside landmarks include information on road construction and detours, and road signs such as slow-down, no-parking, and signs indicating school access.

[0091] It should be understood that the environmental information may also include other special environmental factors, and this embodiment does not limit this.

[0092] Environmental difference information refers to the differences between environmental information on the memory map and environmental information on the historical map. For example, if the memory map includes a construction detour sign, but the historical map does not have the sign, then the construction detour sign can be considered as one type of environmental difference information.

[0093] The environmental feature refers to the feature of the map corresponding to the environmental difference. For example, if the environmental difference information is a construction detour sign, then the extracted environmental feature is: a sign icon or pattern of the construction detour sign on the map.

[0094] This step S103 specifically includes: based on the environmental information, identifying environmental difference information between the memory map and the historical map using an identification algorithm, the environmental difference information including construction detours and / or road signs; extracting corresponding environmental features based on the construction detours and / or road signs; and determining whether the environmental features of the construction detours and / or road signs are used as environmental features for map fusion based on the weight value of the environmental difference information.

[0095] Specifically, the recognition algorithm includes: image processing techniques (such as SIFT, SURF, ORB, etc.) to extract keyframes and local features from electronic map images, thereby describing salient parts of the image, such as signs, building outlines, etc. It also includes: feature matching algorithms and object detection algorithms, such as deep learning models (such as YOLO, SSD, Faster R-CNN, etc.) to detect objects such as signs and traffic signs in the electronic map. This embodiment does not limit the specific recognition algorithm.

[0096] The system can assign a weight value to all environmental differences related to construction detours and / or road signs. If the weight value exceeds or equals a threshold, the corresponding environmental feature is determined. For example, signs indicating construction detours or road signs can be used as environmental features for map fusion. If the weight value is less than the threshold, the environmental difference is ignored, and the environmental feature is not added to the map fusion process. Specifically, the weight value can be user-defined or determined by system calculation and evaluation. For example, the fusion weight value may be assigned different values ​​based on the weather conditions, traffic congestion, and time period at the time of collection. This embodiment does not impose any restrictions on this.

[0097] Step S104: Merge the two aligned maps and environmental features to generate an electronic map containing the target route and environmental features.

[0098] This step combines the two aligned maps obtained in step S102 and the environmental features output in step S103 with an image fusion algorithm to create a new electronic map, which includes the aforementioned target route and environmental features.

[0099] One possible implementation is to integrate the fused environmental features and target route information into an electronic map. The electronic map can be represented using various methods, such as raster or vector methods, the specific choice depending on the application scenario and accuracy requirements. The generated electronic map is then optimized and adjusted to ensure its clarity, readability, and accuracy. Finally, manual adjustments are made using map editing tools, or algorithms are used to automatically optimize the map's layout and display.

[0100] The lightweight multi-trip mapping method provided in this embodiment uses sampling data after a vehicle travels multiple times on the target route to build a map. Compared with the method of using single-trip sampling data to build a map, this method can collect all road and surrounding environmental information, avoid omissions of route information in single-trip sampling data, and improve the completeness of road information in the map.

[0101] Furthermore, by aligning the created memory map and historical map with their trajectories, the target routes of the two aligned maps match, improving the accuracy of subsequent fusion mapping. It also incorporates surrounding environmental features, making the fused electronic map more accurately reflect the real-world map scene. Moreover, since only the two aligned trajectories and the intermediate results of the fusion mapping need to be stored, this method requires fewer storage resources compared to storing and reporting more route data, thus saving storage space.

[0102] In one possible implementation of this embodiment, as shown in FIG3, step S102 above, which aligns the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps, specifically includes:

[0103] Step S1021: Match the target route in the memory map with the historical map.

[0104] Step S1022: Check if there is at least one route in the historical map that matches the target route.

[0105] The historical map may contain one or more routes. According to the matching algorithm, if any one of the routes matches the target route, it is determined as "yes"; if yes, then step S1023 is executed.

[0106] If no matching route is found, the result is "No," indicating that there is no route identical or similar to the target route in the current historical map search, thus ending the matching process. Alternatively, a different historical map can be used to search and match again.

[0107] Step S1023: Align the matching routes in the historical map with the target route to obtain two aligned maps.

[0108] In one possible implementation of this embodiment, step S1023, which aligns the matching routes in the historical map with the target route to obtain two aligned maps, specifically includes:

[0109] First, a first set of keyframes corresponding to the matching route in the historical map and a second set of keyframes corresponding to the target route are obtained respectively. The first set of keyframes consists of at least one keyframe from the historical map, and similarly, the second set of keyframes consists of at least one keyframe from the target route in the memory map. Referring to Figure 2a or Figure 2b above, each dot or triangle pattern in the figure is a keyframe, and Figures 2a and 2b consist of two maps composed of multiple images containing keyframes.

[0110] Then, keyframe association, map feature association between historical map and memory map are performed on at least one keyframe in the first keyframe set and the second keyframe set, and pose optimization processing is performed on at least one associated keyframe to obtain two aligned maps.

[0111] In this step, keyframes related to the target route in the first and second keyframe sets are processed in three stages: keyframe association, map feature association, and keyframe pose optimization, ultimately outputting two aligned maps. The following is a detailed explanation of the above three stages of processing.

[0112] ① Keyframe association processing

[0113] Specifically, as shown in Figure 4, the above-mentioned keyframe association of at least one keyframe in the first keyframe set and the second keyframe set includes:

[0114] Step S301: Based on the Hidden Markov Model (HMM), pair the keyframes in the first keyframe set and the second keyframe set according to the trajectory of the target route to generate at least one keyframe pair.

[0115] Specifically, a trained Hidden Mirror Model (HMM) is used to match keyframes of two trajectories. The matching process involves using the Viterbi algorithm to find the most probable sequence of hidden states (i.e., matching keyframe pairs). The output matching results include the pairing relationship for each keyframe (i.e., at least one keyframe pair) and possible errors or uncertainties. This step is used to subsequently match map features (lane lines, ground markings) based on this pairing relationship, forming matching constraints.

[0116] ② Map element association

[0117] This embodiment describes the process of associating map elements between the historical map and the memory map, as shown in Figure 4. The map element association includes:

[0118] Step S302: Based on the at least one keyframe pair, establish the association between map elements of the historical map and the memory map.

[0119] One specific implementation includes: determining discrete and / or continuous features of the historical map and the memory map based on the at least one keyframe pair, and establishing the association between the discrete and / or continuous features.

[0120] Discrete elements include at least one of the following: dashed lines, pedestrian crossings, stop lines, and arrows.

[0121] Dashed lines are used to indicate lane dividers or part of lane markings; crosswalks are areas for pedestrians to cross the road; stop lines are markings indicating where vehicles must stop and wait, commonly found at traffic light intersections; arrows are used to indicate direction, such as lane direction or turn indicators.

[0122] Continuous elements include one or more of the following: lane boundary and road boundary. The lane boundary defines the line separating lanes, typically a solid or dashed line; the road boundary defines the line separating the road from the surrounding area (such as sidewalks, grass, or other roads).

[0123] In this implementation, during the map feature association process, before using the Iterative Closest Point (ICP) algorithm for point cloud registration, the initial pose (i.e., the initial relative position and orientation between two point clouds) may be unknown or highly inaccurate. Directly using ICP may cause the algorithm to get stuck in a local optimum, failing to find the true best match. Therefore, ICP alignment is performed first, followed by pairing. In each iteration of ICP, the algorithm searches for the nearest point pairs and calculates an optimal transformation matrix (rotation and translation) based on these pairs. This transformation is then applied to update the position of the point clouds until a certain convergence condition is met (e.g., the transformation amount is less than a certain threshold). At this point, the optimal matching result is considered to have been found. Once the initial pose is sufficiently good, ICP iterations are no longer performed. This good initial pose is used as the final result, or minor fine-tuning is performed on it, which may be more efficient and accurate. This method dynamically adjusts the ICP algorithm usage strategy based on the quality of the initial pose to ensure accurate matching results under various conditions.

[0124] ③ Keyframe pose optimization

[0125] As shown in Figure 4, the above-mentioned pose optimization processing of at least one associated keyframe yields two aligned maps, specifically including:

[0126] Step S303: Optimize the pose of the relationships between map features and the keyframes in the first keyframe set and the second keyframe set to obtain the pose of the optimized trajectory.

[0127] One implementation includes: obtaining an objective function, and then using the objective function to adjust the pose of the keyframes in each trajectory to obtain the optimized trajectory pose, wherein the keyframe pose includes the position and orientation of the keyframe. The objective function is used to minimize the positional differences between associated map features and keyframes; this typically means minimizing a cost function that calculates a measure of the positional differences between all paired elements.

[0128] Step S304: Based on the pose of the optimized trajectory, map features, and the relationships between map features, the two aligned maps are obtained.

[0129] Specifically, each keyframe in the first and second keyframe sets contains location information (such as latitude and longitude) and possible direction information. Paired map elements include the correspondence between keyframes in multiple trajectories and specific elements on the electronic map (such as dashed lines, pedestrian crossings, stop lines, arrows, lane boundaries, and road boundaries).

[0130] After optimization using the algorithm, the position and orientation of each keyframe in each trajectory are adjusted, thereby better matching map features and satisfying various constraints. Specifically, the map features in the output remain unchanged after pose optimization, but their pairing relationships (i.e., which keyframes they are associated with) may become more accurate due to the optimization process.

[0131] Step S304 can be implemented as an iterative process, continuously trying different pose combinations to find the optimal solution. The iteration involves point-line constraints and point-to-point constraints. Point-line constraints: For linear elements such as boundaries, crosswalks, and stop lines, point-line constraints are used. This means that keyframes (as points) need to be optimized to maintain a certain distance and direction relationship with their paired linear elements (as lines). Point-to-point constraints: For point-like elements such as dashes and arrows, point-to-point constraints are used. This means that keyframes need to be optimized to be as close as possible in position to their paired point-like elements.

[0132] After the keyframe pose optimization described above, the matching relationship between the trajectory and map features is improved by adjusting the pose of the keyframes. Finally, the two aligned maps are output to improve the accuracy of map fusion.

[0133] In one possible implementation of this embodiment, after obtaining the two aligned maps in step S304, the method further includes an alignment self-check process. This process is used to check whether the aligned maps meet preset requirements.

[0134] Specifically, this includes: checking whether the absolute distance and distribution between map features that have established associations are within a preset range; checking whether some or all important map features have been successfully associated with the pose in the trajectory; and checking whether the difference between the pose of the optimized trajectory and the pose before optimization is within an allowable range.

[0135] If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps. That is, step S102 is repeated until the output check results meet the conditions. If all results in the check are positive, the aligned two maps are output.

[0136] Optionally, the absolute distances and distributions between the map features are checked to see if they are within a preset range. Specifically, this includes checking the physical distances between associated features and their distribution across the entire map. By calculating these distances and distributions, the accuracy of the associations can be assessed, such as whether there are abnormally long-distance associations or associations that are dense in some areas and sparse in others.

[0137] Check which map features have not been associated, i.e., check whether all important map features have been successfully associated with the pose in the trajectory. If there are unassociated features, it indicates missing data, limitations of the association algorithm, or a mismatch between the map and the trajectory.

[0138] In addition, the difference between the pose of the optimized trajectory and the pose before optimization is checked to see if it is within acceptable limits. This step compares the optimized trajectory (based on map features and pose optimization) with the unoptimized trajectory (e.g., based on GPS and inertial navigation data). This difference can reveal whether the optimization process effectively improved the accuracy of the trajectory, especially in areas with poor GPS signals. For areas with poor GPS signals, multiple iterative optimizations are performed: for areas with unstable GPS signals, the system may perform multiple iterative optimization processes to improve the accuracy of the alignment between the two maps.

[0139] During the self-check process, if it is found that the association between certain map features and the trajectory may be incorrect (for example, based on the detection results of the above steps), it may be necessary to return to the previous step S301 to re-pair and establish the association, and then execute the pose optimization steps S303 and S304 again.

[0140] The self-checking process continues until a certain convergence criterion is reached. For example, if the results of several consecutive iterations change very little, or if all the above check results are "yes", then the system will output the final optimized pose and map features, and output the two aligned maps. These output results will be used for subsequent map fusion and mapping.

[0141] The method provided in this embodiment only needs to store the reported data of two trajectories during the trajectory alignment and fusion mapping process of the memory map and the historical map, such as the trajectory corresponding to the target route and the latest uploaded trajectory, as well as the intermediate results of the fusion mapping. It does not need to store and report more route data, so this method requires less storage resources and can save storage space.

[0142] Furthermore, in this embodiment, the multi-pass fusion algorithm incrementally fuses the results of multiple passes of mapping, which greatly reduces the number of elements to be processed, lowers the processing complexity, and improves the efficiency of fusion mapping.

[0143] This embodiment also provides a lightweight multi-pass mapping device for implementing the above embodiments and optional implementations; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0144] This embodiment provides a lightweight multi-pass mapping device for implementing the method steps shown in Figures 1, 3 or 4 above. As shown in Figure 5, the device includes: an acquisition module 510, an alignment module 520, an identification module 530 and a generation module 540. In addition, the device may also include other more or fewer units or modules, such as storage units, etc. This embodiment does not limit this.

[0145] The acquisition module 510 is used to acquire a memory map created after the vehicle has traveled multiple times on the target route, and the memory map contains the target route.

[0146] Alignment module 520 is used to align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps.

[0147] The identification module 530 is used to identify environmental difference information between the memory map and the historical map, and extract corresponding environmental features based on the environmental difference information.

[0148] The generation module 540 is used to fuse the two aligned maps and the environmental features to generate an electronic map containing the target route and the environmental features.

[0149] In some optional implementations, the alignment module 520 is further configured to match the target route in the memory map with the historical map, and to search whether there is at least one route in the historical map that matches the target route; if so, the matching route in the historical map is aligned with the target route to obtain two aligned maps.

[0150] In some alternative implementations, the alignment module 520 is further configured to obtain a first set of keyframes corresponding to the matching route in the historical map and a second set of keyframes corresponding to the target route; perform keyframe association on at least one keyframe in the first set of keyframes and the second set of keyframes, map element association between the historical map and the memory map, and pose optimization processing on at least one associated keyframe to obtain the two aligned maps.

[0151] In some alternative implementations, the alignment module 520 is further configured to pair keyframes in the first keyframe set and the second keyframe set according to the trajectory of the target route based on the hidden Markov model, to generate at least one keyframe pair.

[0152] The alignment module 520 is further configured to establish a relationship between map elements of the historical map and the memory map based on the at least one keyframe pair.

[0153] The apparatus provided in this embodiment also includes an optimization module, which is not shown in FIG5.

[0154] The optimization module is used to optimize the pose of the relationships between map elements and the keyframes in the first keyframe set and the second keyframe set to obtain the pose of the optimized trajectory.

[0155] The alignment module 520 is also used to obtain the two aligned maps based on the pose of the optimized trajectory, map features, and the correlation between the map features.

[0156] In some alternative implementations, the alignment module 520 is further configured to determine discrete and / or continuous features of the historical map and the memory map based on the at least one keyframe pair, and to establish the association between the discrete and / or continuous features.

[0157] The discrete elements include at least one of the following:

[0158] Dashed lines are used to indicate part of lane dividers or markings; pedestrian crossings are used to indicate areas for pedestrians to cross the road; stop lines are used to indicate where vehicles must stop and wait; arrows are used to indicate direction.

[0159] Continuous elements include one or more of the lane boundary and the road boundary.

[0160] In some alternative implementations, the optimization module is specifically used to obtain an objective function, and then use the objective function to adjust the pose of the keyframes in each trajectory to obtain the optimized trajectory pose. The pose of the keyframes includes the position and orientation of the keyframes. The objective function is used to minimize the positional difference between the associated map features and the keyframes.

[0161] As shown in Figure 6, the device provided in this embodiment also includes an inspection module 550 and an output module 560.

[0162] The inspection module 550 is used to check whether the absolute distance and distribution between the map features that have established the association are within a preset range; and to check whether some or all of the important map features have been successfully associated with the pose in the trajectory; and to check whether the difference between the pose of the optimized trajectory and the pose before optimization is within an allowable range.

[0163] The output module 560 is used to re-associate or optimize the result of the detection module if one or more of the above results are negative, and to obtain two new maps, until the output detection result meets the conditions.

[0164] In addition, the output module 560 is also used to output the aligned two maps if the detection module checks all the above results and finds them to be true.

[0165] In some alternative implementations, the acquisition module 510 is specifically used to acquire sampling data from multiple trips of the vehicle along the target route, and to create the memory map based on the environmental information of the target route. The sampling data includes route information for the target route.

[0166] In some alternative implementations, the sampling data may also include environmental information.

[0167] The identification module 530 is specifically used to identify environmental difference information between the memory map and the historical map based on the environmental information and through an identification algorithm. The environmental difference information includes construction detours and / or road signs. The module extracts corresponding environmental features based on the construction detours and / or road signs. The module determines whether the environmental features of the construction detours and / or road signs are used as environmental features for map fusion based on the weight value of the environmental difference information.

[0168] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0169] In this embodiment, the navigation information generation device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0170] This application also provides an electronic device having the lightweight multi-pass mapping device shown in FIG5 or FIG6 above.

[0171] Please refer to Figure 7, which is a schematic diagram of the structure of an electronic device provided in an optional embodiment of this application. As shown in Figure 7, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 shows an example of a single processor 10.

[0172] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may also include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The programmable logic devices may be complex programmable logic devices (CLPs), field-programmable gate arrays (FPGAs), general-purpose array logic (GDAs), or any combination thereof.

[0173] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the lightweight multi-pass mapping method shown in the above embodiments.

[0174] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0175] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0176] The electronic device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30 and output device 40 can be connected via a bus or other means, as shown in Figure 7, which illustrates a connection via a bus.

[0177] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.

[0178] In addition, the electronic device in this embodiment may also include at least one communication interface for communicating with a vehicle or other devices.

[0179] Optionally, the electronic device is a vehicle controller or a vehicle control unit (VCU).

[0180] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; optionally, the storage medium may also include a combination of the above types of memory.

[0181] It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implement the lightweight multi-pass mapping method shown in the above embodiments.

[0182] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the lightweight multi-pass mapping method of any embodiment of this application.

[0183] Furthermore, referring to Figure 8, this application embodiment also provides a vehicle, which includes a vehicle controller and at least one sensor. In addition, the vehicle may also include other more or fewer devices / components such as a display screen, which is not limited in this embodiment.

[0184] The at least one sensor is used to collect environmental information, as well as driving trajectory information, road information, etc. The vehicle controller is used to execute the lightweight multi-trip mapping method shown in the above embodiments. For details, please refer to Figures 1, 3, and 4 of the aforementioned embodiments, which will not be repeated here.

[0185] The above embodiments are only used to illustrate the technical solutions of the embodiments of this application, and are not intended to limit them. Although the embodiments of this application have been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A lightweight multi-pass mapping method, characterized in that, The method includes: Obtain a memory map created after the vehicle has traveled multiple times on the target route, the memory map containing the target route; Align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps. Identify the environmental differences between the memory map and the historical map, and extract the corresponding environmental features based on the environmental differences. The two aligned maps and the environmental features are merged to generate an electronic map that includes the target route and the environmental features.

2. The method according to claim 1, characterized in that, The step of aligning the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps includes: The target route in the memory map is matched with the historical map to find whether there is at least one route in the historical map that matches the target route; If so, the matching routes in the historical map are aligned with the target route to obtain two aligned maps.

3. The method according to claim 2, characterized in that, Align the matching routes in the historical map with the target route to obtain two aligned maps, including: Obtain the first set of keyframes corresponding to the matching routes in the historical map, and the second set of keyframes corresponding to the target route, respectively. At least one keyframe in the first keyframe set and the second keyframe set is associated with a keyframe, map features are associated between the historical map and the memory map, and pose optimization is performed on the associated keyframe to obtain the two aligned maps.

4. The method according to claim 3, characterized in that, Keyframe association is performed on at least one keyframe in the first keyframe set and the second keyframe set, including: Based on the Hidden Markov Model, key frames in the first key frame set and the second key frame set are paired according to the trajectory of the target route to generate at least one key frame pair. The association of map features between the historical map and the memory map includes: Based on the at least one keyframe pair, establish the association between map elements of the historical map and the memory map; The step of performing pose optimization processing on at least one associated keyframe to obtain the aligned two maps includes: The pose of the optimized trajectory is obtained by optimizing the pose of the map features by performing pose optimization on the relationships between the map features and the keyframes in the first keyframe set and the second keyframe set. Based on the pose of the optimized trajectory, map features, and the relationships between the map features, the two aligned maps are obtained.

5. The method according to claim 4, characterized in that, The step of establishing the association between map features of the historical map and the memory map based on the at least one keyframe pair includes: Based on the at least one keyframe pair, determine the discrete and / or continuous elements of the historical map and the memory map, and establish the association between the discrete and / or continuous elements; The discrete elements include at least one of the following: Dashed lines are used to indicate a portion of lane dividers or lane markings; A pedestrian crossing is used to indicate an area for pedestrians to cross the road. A stop line is a marking line used to indicate that vehicles must stop and wait in front of this line; Arrows are used to indicate direction; The continuous elements include one or more of lane boundaries and road boundaries.

6. The method according to claim 4, characterized in that, The step of optimizing the pose of the optimized trajectory by processing the relationships between the map features and the keyframes in the first and second keyframe sets to obtain the pose of the optimized trajectory includes: Obtain an objective function that minimizes the positional differences between associated map features and keyframes; The pose of the keyframes in each trajectory is adjusted using the objective function to obtain the pose of the optimized trajectory. The pose of the keyframes includes the position and orientation of the keyframes.

7. The method according to claim 6, characterized in that, After obtaining the two aligned maps, the process further includes: Check whether the absolute distances and distributions between map features with established relationships are within preset ranges; and, Check whether some or all of the important map features have been successfully associated with the pose in the trajectory; and... Check whether the difference between the pose of the optimized trajectory and the pose before optimization is within the allowable range; If one or more results in the check are negative, the negative result will be re-associated or optimized to obtain two new maps, until the output check results meet the conditions. If all results in the check are yes, then the two aligned maps are output.

8. The method according to any one of claims 1-7, characterized in that, The acquisition of the memory map created after the vehicle has traveled multiple times on the target route includes: Acquire sampling data of the vehicle traveling multiple times on the target route, wherein the sampling data includes route information of the target route; The memory map is created based on the environmental information of the target route.

9. The method according to claim 8, characterized in that, The sampling data also includes environmental information; The step of identifying environmental differences between the memory map and the historical map, and extracting corresponding environmental features based on the environmental differences, includes: Based on the environmental information, an identification algorithm is used to identify environmental differences between the memory map and the historical map, including construction detours and / or road signs. Extract the corresponding environmental features based on the construction detour and / or the road signs; The environmental features of the construction detour and / or the road signs are determined as environmental features for map fusion based on the weight values ​​of the environmental difference information.

10. A lightweight multi-trip mapping device, characterized in that, The device includes: The acquisition module is used to acquire a memory map created after the vehicle has traveled multiple times on the target route, and the memory map contains the target route; The alignment module is used to align the target route in the memory map with the corresponding route in the historical map to obtain two aligned maps. The identification module is used to identify environmental differences between the memory map and the historical map, and to extract corresponding environmental features based on the environmental differences. The generation module is used to fuse the two aligned maps and the environmental features to generate an electronic map containing the target route and the environmental features.

11. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory and the processor are connected. The memory stores computer instructions, and the processor executes the computer instructions to perform the lightweight multi-pass mapping method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 9.

14. A vehicle, characterized in that, It includes a vehicle-mounted controller, which is used to execute the lightweight multi-trip mapping method according to any one of claims 1 to 9.

Images