A lightweight multi-pass mapping method, device, equipment and vehicle

Abstract

The present invention relates to the field of electronic map technology and discloses a lightweight multi-pass mapping method, apparatus, equipment, and vehicle. The lightweight multi-pass mapping method comprises: obtaining a memory map created after a vehicle travels multiple times on a target route, the memory map containing the target route; aligning the trajectory of the target route in the memory map with the corresponding route in a historical map to obtain two aligned maps; identifying environmental difference information between the memory map and the historical map, and extracting corresponding environmental features based on the environmental difference information; and fusing the two aligned maps and environmental features to generate an electronic map containing the target route and environmental features. This method collects all information about the road and the surrounding environment, preventing single-pass sampling data from omitting route information and improving the integrity of the road information for mapping. Furthermore, the surrounding environmental features are integrated, so that the integrated electronic map can better restore the real map scene.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic maps, and in particular to a lightweight multi-pass mapping method, device, equipment and vehicle. Background Art

[0002] With the ubiquity of mobile devices, electronic maps have become seamlessly integrated into people's daily lives. Whether on smartphones, tablets, or in-car navigation systems, they provide users with convenient map services anytime, anywhere. Creating electronic maps requires the collection of a vast amount of basic geographic data, including road networks, topography, building layouts, and environmental information. This data can come from a variety of sources, including satellite remote sensing, aerial photography, ground surveying, and user-generated data.

[0003] Currently, a method for mapping a route using memory is used. This involves creating an electronic map by collecting information about the preset route, such as road boundaries and road surface markings, after a user's vehicle has driven the route a single time. However, this single-trip memory mapping method may not fully extract semantic information about the road, such as lanes and traffic signs. The collected route information may be missing, obscured, or suffer geometric inaccuracies, resulting in low-accuracy or incomplete road information. Summary of the Invention

[0004] In view of this, the present invention provides a lightweight multi-pass mapping method, device, equipment and vehicle to solve the problem of low accuracy of created electronic maps or incomplete road information.

[0005] In a first aspect, the present invention provides a lightweight multi-pass mapping method, the method comprising:

[0006] Obtaining a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route;

[0007] Aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps;

[0008] Identifying environmental difference information between the memory map and the historical map, and extracting corresponding environmental features based on the environmental difference information;

[0009] The two aligned maps and the environmental features are fused to generate an electronic map including the target route and the environmental features.

[0010] The method provided in this aspect uses sampling data from multiple vehicle trips on a target route to build a map. Compared to the method of using single-trip sampling data to build a map, this method can collect all information about the road and the surrounding environment of the road, avoid omissions of route information in single-trip sampling data, and improve the integrity of the road information for mapping.

[0011] Furthermore, by aligning the paths of the created memory map and the historical map, the target routes of the two aligned maps match, improving the accuracy of subsequent fused mapping. It also incorporates features of the surrounding environment, making the fused electronic map more realistic. Furthermore, since only the two aligned paths and the intermediate results of the fused mapping need to be stored, this method requires less storage resources than storing and reporting more route data, thus saving storage space.

[0012] In conjunction with the first aspect, in a possible implementation, aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps includes:

[0013] matching the target route in the memory map with a history map, and searching whether there is at least one route in the history map that matches the target route;

[0014] If so, the matched route in the historical map is aligned with the target route to obtain two aligned maps.

[0015] In combination with the first aspect, in another possible implementation, aligning the matched routes in the historical map with the target route to obtain two aligned maps includes:

[0016] Respectively obtaining a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route;

[0017] Keyframe association is performed on at least one keyframe in the first keyframe set and the second keyframe set, map elements are associated between the historical map and the memory map, and pose optimization processing is performed on at least one associated keyframe to obtain the two aligned maps.

[0018] In conjunction with the first aspect, in another possible implementation, associating key frames on at least one key frame in the first key frame set and the second key frame set includes: pairing the key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair;

[0019] The associating the map elements between the historical map and the memory map comprises: establishing an associative relationship between the map elements of the historical map and the memory map based on the at least one key frame pair;

[0020] The performing pose optimization processing on at least one associated key frame to obtain the two aligned maps includes: performing pose optimization on the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the pose of the optimized trajectory; and obtaining the two aligned maps based on the pose of the optimized trajectory, the map elements, and the association relationship between the map elements.

[0021] In combination with the first aspect, in another possible implementation, establishing an association relationship between map elements of the historical map and the memory map based on the at least one key frame pair includes:

[0022] Determining discrete elements and / or continuous elements of the history map and the memory map based on the at least one key frame pair, and establishing association relationships between the discrete elements and / or the continuous elements;

[0023] The discrete elements include at least one of the following:

[0024] Dashed lines, used to indicate a portion of a lane divider or marking;

[0025] Crosswalks are used to indicate areas where pedestrians cross the road;

[0026] Stop line: a marking line used to indicate that vehicles must stop and wait before this line;

[0027] Arrows, used to indicate direction;

[0028] The continuous elements include: one or more of lane boundaries and road boundaries.

[0029] In conjunction with the first aspect, in another possible implementation, optimizing the associations between the map elements and the key frames in the first key frame set and the second key frame set to obtain the optimized trajectory pose includes:

[0030] Obtaining an objective function for minimizing a position difference between an associated map element and a keyframe;

[0031] The objective function is used to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, where the pose of the key frame includes the position and orientation of the key frame.

[0032] In combination with the first aspect, in another possible implementation, after obtaining the two aligned maps, the method further includes:

[0033] Checking whether the absolute distances and distributions between associated map elements are within a preset range; checking whether some or all important map elements have been successfully associated with the poses in the trajectory; and checking whether the difference between the pose of the optimized trajectory and the pose before optimization is within an allowable range;

[0034] If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps until the output check result meets the conditions;

[0035] If all the results in the check are yes, the two aligned maps are output.

[0036] In combination with the first aspect, in another possible implementation, the memory map created after the vehicle has traveled multiple times on the target route includes: obtaining sampling data of the vehicle traveling multiple times on the target route, the sampling data including route information of the target route; and creating the memory map based on environmental information of the target route.

[0037] In combination with the first aspect, in yet another possible implementation, the sampled data further includes environmental information;

[0038] The identifying of environmental difference information between the memory map and the historical map, and extracting corresponding environmental features according to the environmental difference information, includes:

[0039] identifying, based on the environmental information, environmental difference information between the memory map and the historical map using a recognition algorithm, the environmental difference information including construction diversions and / or road signs;

[0040] Extracting corresponding environmental features according to the construction diversion and / or the road signs;

[0041] It is determined whether the environmental features of the construction diversion and / or the road signs are used as environmental features for map fusion according to the weight values ​​of the environmental difference information.

[0042] In a second aspect, the present invention further provides a lightweight multi-pass mapping device, comprising:

[0043] an acquisition module, configured to acquire a memory map created after the vehicle has traveled multiple times on the target route, wherein the memory map includes the target route;

[0044] an alignment module, configured to align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps;

[0045] an identification module, configured to identify environmental difference information between the memory map and the historical map, and extract corresponding environmental features based on the environmental difference information;

[0046] A generation module is used to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features.

[0047] In a third aspect, the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute the lightweight multi-pass mapping method of the first aspect or any corresponding embodiment thereof.

[0048] Optionally, the electronic device is a vehicle controller.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment thereof.

[0050] In addition, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment thereof.

[0051] In a fifth aspect, the present invention further provides a vehicle, comprising a vehicle controller, wherein the vehicle controller is configured to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment thereof.

[0052] The lightweight multi-pass mapping method, device, equipment, and vehicle provided in this aspect use sampling data after the vehicle has traveled multiple times on the target route to build a map. Compared with the method of using single-pass sampling data to build a map, this method can collect all information about the road and the surrounding environment of the road, avoid omissions of route information in single-pass sampling data, and improve the integrity of the road information for mapping.

[0053] Furthermore, by aligning the paths of the created memory map and the historical map, the target routes of the two aligned maps match, improving the accuracy of subsequent fused mapping. It also incorporates features of the surrounding environment, making the fused electronic map more realistic. Furthermore, since only the two aligned paths and the intermediate results of the fused mapping need to be stored, this method requires less storage resources than storing and reporting more route data, thus saving storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 1 is a flow chart of a lightweight multi-pass mapping method according to an embodiment of the present invention;

[0056] Figure 2a is a schematic diagram of a target route before trajectory alignment according to an embodiment of the present invention;

[0057] Figure 2b is a schematic diagram of a target route after trajectory alignment according to an embodiment of the present invention;

[0058] Figure 3 is a flowchart of another lightweight multi-pass mapping method according to an embodiment of the present invention;

[0059] Figure 4 is a flowchart of another lightweight multi-pass mapping method according to an embodiment of the present invention;

[0060] Figure 5 This is a structural block diagram of a lightweight multi-pass mapping device according to an embodiment of the present invention;

[0061] Figure 6 This is a structural block diagram of another lightweight multi-pass mapping device according to an embodiment of the present invention;

[0062] Figure 7 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention;

[0063] Figure 8 2 is a schematic structural diagram of a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0065] Electronic maps integrate advanced Geographic Information System (GIS), remote sensing technology, Global Positioning System (GPS) and Internet technology, transforming traditional paper maps into a highly interactive, information-rich and real-time updated digital platform.

[0066] In the process of creating an electronic map, a user's car usually collects data on a single trip route and reports the data to a server or network center, which then builds a map based on the data reported by the user's car.

[0067] This mapping method has the following problems:

[0068] 1. Low accuracy: The geometric accuracy of single-trip mapping data is significantly affected by factors such as the performance of the acquisition equipment and environmental changes. In addition, single-trip mapping may result in omissions, occlusions, or geometric accuracy errors in the collected route information, resulting in low accuracy of the generated electronic map or incomplete road information.

[0069] 2. Redundant Information Management: The fusion of multiple mapping data passes can generate a large amount of redundant information, increasing storage and computational burdens. Furthermore, when fusing multiple route sampling data passes, effective compensation and correction for geometric errors are required, increasing computing power.

[0070] 3. Adaptability to environmental changes: The sampling environment changes dynamically, and mapping data collected in different time periods may vary significantly.

[0071] This invention aims to detect areas of current change and update electronic maps by memorizing and learning sample data from multiple times on the same route, solving the problem of single-trip mapping and achieving lightweight, high-quality fusion of multi-trip mapping data, thereby meeting the demand for high-quality, lightweight maps in scenarios such as AI ​​driving.

[0072] It should be noted that the lightweight multi-pass mapping method provided in the embodiments of this application can be executed by a mapping device or equipment. This mapping device or equipment can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. The electronic device can be a vehicle-side control device, such as a vehicle controller, or it can also be a terminal device or server device. The embodiments of this application do not specifically limit the electronic device. In the following method embodiments, the execution subject is an electronic device as an example.

[0073] According to an embodiment of the present invention, an embodiment of a lightweight multi-pass mapping method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system, such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown.

[0074] In this embodiment, a lightweight multi-pass mapping method is provided, which can be used in the above-mentioned electronic devices. Figure 1 : is a flowchart of a lightweight multi-pass mapping method according to an embodiment of the present invention, which includes:

[0075] Step S101: Obtain a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route.

[0076] The target route is the route that the user's car expects to map, such as the route from starting point A to end point B, that is, the target route A→B.

[0077] One implementation includes: obtaining sampling data of a vehicle traveling multiple times on a target route, the sampling data including route information of the target route; and creating a memory map based on the environmental information of the target route.

[0078] The sampling data is obtained by a mass-produced vehicle or low-cost collection equipment through multiple sampling trips on the target route A→B.

[0079] The route information may include the following: Route direction: used to indicate the starting point, end point and major nodes passed through the sampling route, which helps to understand the overall flow direction of the sampling activity. Plane line shape: including the specific location and length of straight segments and curved segments (such as left and right turns), which helps to accurately restore the sampling path on the map. Mileage pile number: In order to accurately indicate the total length of the route and the length of each section, kilometer piles and hundred-meter piles are usually set, and the corresponding mileage values ​​are marked. In addition, route information also includes information such as intersections and overpass structures.

[0080] Each time a user's vehicle passes through a target route, it obtains a sample of data. This data is then reported to the server or vehicle controller, which then acquires and updates it in real time to generate a memory map. Methods for creating memory maps include, but are not limited to, SLAM (Simultaneous Localization and Mapping). SLAM is primarily used for real-time positioning and mapping of robots and autonomous vehicles. It can also compare real-time sensor data with existing map data to identify changes or errors in the map.

[0081] In addition, the sampled data may also include environmental information.

[0082] Step S102: Align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps.

[0083] Historical maps are one or more pre-stored electronic maps on a server or vehicle controller. Each electronic map contains historically recorded map elements that include the target route. Historical maps can be generated on the server based on data collected and reported by other user vehicles, or generated and reported by other clients or third-party organizations.

[0084] After the two maps are aligned, it is assumed that Figure 1 and Map 2, where Figure 1 The first map is an aligned map generated based on the memory map, and the second map 2 is an aligned map generated based on the history map. In this embodiment, the number of history maps is one, but in fact, it can also be two or more, which is not limited in this embodiment.

[0085] One method for aligning routes involves loading data from memory maps and historical maps using GIS software (such as ArcGIS and QGIS). Within the software, spatial analysis tools are used to align the two routes. This alignment process includes algorithms such as keyframe association, keyframe pairing, path correction, and pose optimization.

[0086] See also Figure 2a and Figure 2b The following diagrams show the target route in the memory map and the corresponding route in the history map before and after alignment. Figure 2a To align the previous trajectory diagram, the trajectories of the two routes are quite different. Figure 2b This is a schematic diagram of the trajectory after alignment. After alignment, the route trajectories of the two maps overlap or basically overlap.

[0087] Step S103: identifying environmental difference information between the memory map and the history map, and extracting corresponding environmental features based on the environmental difference information.

[0088] Environmental information refers to the content on memory maps or historical maps involving topography, roadside landmarks, meteorological conditions, etc. Among them, topography includes: terrain undulations, landform features (such as mountains, rivers, lakes, etc.) and feature distribution (such as buildings, roads, vegetation, etc.) on both sides of the sampling route; this information helps to analyze the geographical environment characteristics of the sampling area. Meteorological conditions include: temperature, humidity, air pressure, wind speed, wind direction and other meteorological conditions at the time of sampling have an important impact on the sampling results, so detailed records are needed. Roadside landmarks include: construction diversion information, road signs, such as slow driving, no parking, school attachments and other signs / signs.

[0089] It should be understood that the environmental information may also include other special environmental factors, which is not limited in this embodiment.

[0090] Environmental difference information refers to the difference information between the environmental information on the memory map and the environmental information on the historical map. For example, if there is a construction diversion sign on the memory map but not on the historical map, then the construction diversion sign can be used as one of the environmental difference information.

[0091] The environmental feature refers to the feature of the map corresponding to the environmental difference. For example, if the environmental difference information is a construction diversion sign, the extracted environmental feature is: a sign logo or pattern of the construction diversion sign on the map.

[0092] This step S103 specifically includes: identifying environmental difference information between the memory map and the historical map through a recognition algorithm based on the environmental information, wherein the environmental difference information includes construction diversions and / or road signs; extracting corresponding environmental features based on the construction diversions and / or road signs; and determining whether the environmental features of the construction diversions and / or road signs should be used as environmental features for map fusion based on the weight values ​​of the environmental difference information.

[0093] Specifically, the recognition algorithm includes image processing techniques (such as SIFT, SURF, and ORB) to extract keyframes and local features from electronic map images, thereby describing salient features in the image, such as signs and building outlines. It also includes feature matching algorithms and object detection algorithms, such as deep learning models (such as YOLO, SSD, and Faster R-CNN) to detect objects such as signs and traffic signs in electronic maps. This embodiment does not limit the specific recognition algorithm.

[0094] The system can set a weight value for all environmental difference information of construction diversions and / or road signs. If the weight value exceeds or equals the threshold, the environmental feature corresponding to the environmental difference is determined, such as a sign indicating a construction diversion, or a road sign sign can be used as an environmental feature for map fusion. If the weight value is less than the threshold, the environmental difference is ignored and the environmental feature is not added to the map fusion process. Specifically, the setting of the weight value can be customized by the user or determined by system calculation and evaluation. For example, the fusion weight value will be assigned different fusion weight values ​​based on the weather conditions collected at the time, vehicle congestion, and time distance. This embodiment does not limit this.

[0095] Step S104: Fusing the two aligned maps and environmental features to generate an electronic map including the target route and environmental features.

[0096] In this step, the two aligned maps obtained in step S102 and the environmental features output in step S103 are fused according to an image fusion algorithm to generate a new electronic map. The electronic map includes the target route and environmental features.

[0097] One possible implementation involves integrating the fused environmental features and target route information into an electronic map. Electronic maps can be represented using various methods, such as raster and vector representations, depending on the application scenario and accuracy requirements. The generated electronic map is then optimized and adjusted to ensure clarity, readability, and accuracy. Finally, manual adjustments can be made using map editing tools, or algorithms can be used to automatically optimize the map's layout and display.

[0098] The lightweight multi-pass mapping method provided in this embodiment uses sampling data from multiple vehicle trips along a target route to build a map. Compared to mapping methods using single-pass sampling data, this method can collect all information about the road and its surrounding environment, avoiding omissions of route information in single-pass sampling data and improving the integrity of the mapped road information.

[0099] Furthermore, by aligning the paths of the created memory map and the historical map, the target routes of the two aligned maps match, improving the accuracy of subsequent fused mapping. It also incorporates features of the surrounding environment, making the fused electronic map more realistic. Furthermore, since only the two aligned paths and the intermediate results of the fused mapping need to be stored, this method requires less storage resources than storing and reporting more route data, thus saving storage space.

[0100] In a possible implementation of this embodiment, as Figure 3As shown, the above step S102 aligns the target route in the memory map with the corresponding route in the history map to obtain the two aligned maps, specifically including:

[0101] Step S1021: Match the target route in the memory map with the history map.

[0102] Step S1022: Check whether there is at least one route in the history map that matches the target route.

[0103] The historical map may contain one or more routes. According to the matching algorithm, as long as one of the one or more routes matches the target route, it is determined to be "yes"; if yes, step S1023 is executed.

[0104] If there is no matching route, the result is "No", indicating that there is no route identical or similar to the target route in the current historical map, and the matching process ends. Alternatively, another historical map may be used to search and match again.

[0105] Step S1023: Aligning the matched routes in the historical map with the target route to obtain two aligned maps.

[0106] In a possible implementation of this embodiment, step S1023 aligns the matching routes in the historical map with the target route to obtain two aligned maps, specifically including:

[0107] First, obtain the first key frame set corresponding to the matching route in the historical map and the second key frame set corresponding to the target route. The first key frame set consists of at least one key frame in the historical map, and similarly, the second key frame set consists of at least one key frame on the target route in the memory map. Figure 2a or Figure 2b As shown, each dot or triangle pattern in the figure is a key frame. Figure 2a and Figure 2b There are two maps composed of multiple images containing keyframes.

[0108] Then, key frame association is performed on at least one key frame in the first key frame set and the second key frame set, map elements are associated between the historical map and the memory map, and posture optimization processing is performed on at least one associated key frame to obtain two aligned maps.

[0109] In this step, the keyframes involved in the target route in the first and second keyframe sets are processed through three stages: keyframe association, map feature association, and keyframe pose optimization. Ultimately, two aligned maps are output. The following describes these three stages in detail.

[0110] ①Key frame association processing

[0111] Specifically, if Figure 4 As shown, the above-mentioned key frame association of at least one key frame in the first key frame set and the second key frame set includes:

[0112] Step S301: Pairing key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a Hidden Markov Model (HMM) to generate at least one key frame pair.

[0113] Specifically, the trained HMM model is used to match the keyframes of the two trajectories. The matching process involves using the Viterbi algorithm to find the most likely hidden state sequence (i.e., matching keyframe pairs). The matching results are output, including the pairing relationship of each keyframe (i.e., at least one keyframe pair) and any possible errors or uncertainties. This step is used to subsequently match map elements (lane lines, ground signs) based on this pairing relationship to form matching constraints.

[0114] ② Map element association

[0115] This embodiment describes the process of associating map elements between the historical map and the memory map. Figure 4 As shown, the map element association includes:

[0116] Step S302: establishing an association relationship between map elements of the history map and the memory map based on the at least one key frame pair.

[0117] A specific implementation includes: determining discrete elements and / or continuous elements of the history map and the memory map based on the at least one key frame pair, and establishing an association relationship between the discrete elements and / or continuous elements.

[0118] The discrete elements include at least one of a dotted line, a crosswalk, a stop line, and an arrow.

[0119] A dash is used to indicate a lane divider or part of a marking line; a crosswalk is used to indicate an area for pedestrians to cross the road; a stop line is used to indicate a marking line before which vehicles must stop and wait, and is commonly seen at traffic lights; and an arrow is used to indicate direction, such as the direction of travel in a lane or a turn indicator.

[0120] Continuous Elements include one or more of lane boundaries and road boundaries. Lane boundaries are lines that define the boundaries between lanes, usually solid or dashed lines; road boundaries are lines that define the boundaries between a road and surrounding areas (such as sidewalks, grass, or other roads).

[0121] In this embodiment, during the map feature association process, before using the Iterative Closest Point (ICP) algorithm for point cloud registration, the initial pose (i.e., the initial relative position and pose between the two point clouds) may be unknown or very inaccurate. Directly using ICP may cause the algorithm to fall into a local optimum and fail to find the true best match. Therefore, ICP alignment is performed first, followed by pairing. In each ICP iteration, the algorithm searches for the closest point pairs and calculates an optimal transformation matrix (rotation and translation) based on these points. This transformation is then applied to update the position of the point clouds until a certain convergence condition is reached (e.g., the transformation amount is less than a certain threshold), at which point the optimal match is considered to have been found. When the initial pose is sufficiently good, no further ICP iterations are performed. Using this good initial pose as the final result, or performing minor fine-tuning based on it, may be more efficient and accurate. This method dynamically adjusts the ICP algorithm application strategy based on the quality of the initial pose to ensure accurate matching results in all situations.

[0122] ③Key frame pose optimization

[0123] like Figure 4 As shown, the pose optimization process of at least one associated key frame is performed to obtain two aligned maps, specifically including:

[0124] Step S303: performing pose optimization on the association relationship between map elements and the key frames in the first key frame set and the second key frame set to obtain the pose of the optimized trajectory.

[0125] One embodiment includes obtaining an objective function, and then using the objective function to adjust the pose of the keyframes in each trajectory to obtain the pose of the optimized trajectory, where the pose of the keyframes includes the position and orientation of the keyframes. The objective function is used to minimize the position difference between the associated map features and the keyframes; this typically means minimizing a cost function that calculates some measure of the position difference between all paired elements.

[0126] Step S304: obtaining the two aligned maps based on the pose of the optimized trajectory, map elements, and the associations between the map elements.

[0127] Specifically, each keyframe in the first keyframe set and the second keyframe set contains location information (e.g., longitude and latitude) and possible direction information. Paired map elements include: a correspondence between keyframes in multiple trajectories and specific elements on the electronic map (e.g., dashed lines, crosswalks, stop lines, arrows, lane boundaries, and road boundaries).

[0128] After optimization using the optimization algorithm, the position and orientation of each keyframe in each trajectory are adjusted to better match map features and meet various constraints. Specifically, map features with optimized poses: The map features themselves in the output remain unchanged, but their pairing relationships (i.e., which keyframes they are associated with) may become more accurate due to the optimization process.

[0129] The specific implementation of step S304 can be an iterative process, which searches for the optimal solution by continuously trying different pose combinations. The iterative process involves point-line constraints and point-point constraints. Point-line constraints: For linear elements such as boundary, cross walk, and stopline, point-line constraints are used. This means that the keyframe (as a point) needs to be optimized to maintain a certain distance and direction relationship with its paired linear element (as a line). Point-point constraints: For point elements such as dash and arrow, point-point constraints are used. This means that the keyframe needs to be optimized to be as close as possible in position to its paired point element.

[0130] After the above keyframe pose optimization, the matching relationship between the trajectory and map features is improved by adjusting the keyframe pose, and finally the two aligned maps are output to improve the accuracy of map fusion.

[0131] In a possible implementation of this embodiment, after obtaining the two aligned maps in step S304, the method further includes an alignment self-check process, which is used to check whether the aligned maps meet preset requirements.

[0132] Specifically, it includes: checking whether the absolute distance and distribution between the associated map elements are within the preset range; checking whether some or all important map elements have been successfully associated with the posture in the trajectory; and checking whether the difference between the posture of the optimized trajectory and the posture before optimization is within the allowable range.

[0133] If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps, that is, the above step S102 is re-executed until the output check result meets the conditions. If all results in the check are positive, the two aligned maps are output.

[0134] Furthermore, the absolute distances and distributions between map elements are checked to see if they fall within pre-defined ranges. This includes the physical distances between associated elements and their distribution across the map. By calculating these distances and distributions, the accuracy of the association can be assessed, for example, to determine if there are unusual long-distance associations or associations that are dense in some areas but sparse in others.

[0135] Check which map features are not associated, that is, check whether all important map features have been successfully associated with the poses in the trajectory. If there are unassociated features, it indicates missing data, limitations of the association algorithm, or mismatch between the map and the trajectory.

[0136] In addition, it is checked whether the difference between the pose of the optimized trajectory and the pose before optimization is within the allowable range. This step compares the difference between the optimized trajectory (based on the optimization results of map features and pose) and the trajectory before optimization (such as based on GPS and inertial navigation data). This difference can reveal whether the optimization process has effectively improved the accuracy of the trajectory, especially for areas with poor GPS signals. For areas with poor GPS signals, multiple iterative optimizations will be performed: For areas with unstable GPS signals, the system may perform multiple iterative optimization processes to improve the accuracy of the alignment of the two maps.

[0137] During the self-check process, if it is found that the association between certain map elements and trajectories may be incorrect (for example, based on the detection results of the above steps), it may be necessary to return to the previous step S301 to re-pair and establish the association relationship, and execute the posture optimization steps S303 and S304 again.

[0138] The self-check process continues until a certain convergence criterion is reached. For example, if the results of several consecutive iterations show very little change, or if all the above check results are "yes", the system will output the final optimized pose and map elements, and output the two aligned maps. These output results will be used for subsequent map fusion and mapping.

[0139] The method provided in this embodiment only needs to store the reported data of two trajectories during the process of aligning and merging the memory map and the historical map. For example, the trajectory corresponding to the target route and the most recently uploaded trajectory, as well as the intermediate results of the fusion mapping, do not need to store and report more route data. Therefore, this method requires fewer storage resources and can save storage space.

[0140] In addition, in this embodiment, the multi-pass fusion algorithm performs incremental fusion between the results of multiple passes of mapping, the number of elements that need to be processed is greatly compressed, the processing complexity is reduced, and the efficiency of fusion mapping is improved.

[0141] This embodiment also provides a lightweight multi-pass mapping device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0142] This embodiment provides a lightweight multi-pass mapping device for achieving the above Figure 1 、 Figure 3 or Figure 4 The method steps shown are as follows: Figure 5 As shown, the device includes: an acquisition module 510, an alignment module 520, an identification module 530 and a generation module 540. In addition, the device may also include other more or fewer units or modules, such as a storage unit, etc., which is not limited in this embodiment.

[0143] The acquisition module 510 is configured to acquire a memory map created after the vehicle has traveled multiple times on the target route, wherein the memory map includes the target route.

[0144] The alignment module 520 is configured to align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps.

[0145] The identification module 530 is configured to identify environmental difference information between the memory map and the history map, and extract corresponding environmental features based on the environmental difference information.

[0146] The generating module 540 is configured to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features.

[0147] In some optional embodiments, the alignment module 520 is further configured to match the target route in the memory map with the historical map, and to search for at least one route in the historical map that matches the target route; if so, aligning the matching route in the historical map with the target route to obtain two aligned maps.

[0148] In other optional embodiments, the alignment module 520 is further specifically used to obtain a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route, respectively; perform key frame association on at least one key frame in the first key frame set and the second key frame set, associate map elements between the historical map and the memory map, and perform pose optimization processing on at least one associated key frame to obtain two aligned maps.

[0149] In some further optional implementations, the alignment module 520 is further configured to pair the key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair.

[0150] The alignment module 520 is further configured to establish an association relationship between map elements of the history map and the memory map based on the at least one key frame pair.

[0151] The device provided in this embodiment also includes an optimization module. Figure 5 Not shown.

[0152] The optimization module is used to optimize the association relationship between map elements and the key frames in the first key frame set and the second key frame set to obtain the pose of the optimized trajectory.

[0153] The alignment module 520 is further configured to obtain the two aligned maps based on the pose of the optimized trajectory, the map elements, and the associations between the map elements.

[0154] In some further optional embodiments, the alignment module 520 is further configured to determine the discrete elements and / or continuous elements of the historical map and the memory map based on the at least one key frame pair, and to establish an association relationship between the discrete elements and / or the continuous elements.

[0155] The discrete elements include at least one of the following:

[0156] A dashed line is used to indicate a lane divider or part of a marking line; a crosswalk is used to indicate an area for pedestrians to cross the road; a stop line is used to indicate a marking line before which vehicles must stop and wait; and an arrow is used to indicate direction.

[0157] Continuous features include one or more of lane boundaries and road boundaries.

[0158] In yet other optional embodiments, the optimization module is specifically configured to obtain an objective function, and to use the objective function to adjust the pose of the keyframes in each trajectory to obtain the pose of the optimized trajectory, wherein the pose of the keyframes includes the position and orientation of the keyframes. The objective function is configured to minimize the position difference between the associated map elements and the keyframes.

[0159] like Figure 6 As shown, the device provided in this embodiment further includes: an inspection module 550 and an output module 560.

[0160] The checking module 550 is used to check whether the absolute distance and distribution between the associated map elements are within a preset range; and to check whether some or all important map elements have been successfully associated with the posture in the trajectory; and to check whether the difference between the posture of the optimized trajectory and the posture before optimization is within an allowable range.

[0161] The output module 560 is used to re-associate or optimize the item with the negative result if one or more of the above results are negative in the detection module, and re-obtain two new maps until the output inspection result meets the conditions.

[0162] In addition, the output module 560 is further configured to output the two aligned maps if all the above results are yes when checked by the detection module.

[0163] In some other optional embodiments, the acquisition module 510 is specifically configured to acquire sampling data of multiple trips of the vehicle on the target route, and to create the memory map based on the environmental information of the target route, wherein the sampling data includes route information of the target route.

[0164] In some further optional implementations, the sampled data also includes environmental information.

[0165] The identification module 530 is specifically configured to identify environmental difference information between the memory map and the historical map based on the environmental information using a recognition algorithm, wherein the environmental difference information includes construction diversions and / or road signs; extract corresponding environmental features based on the construction diversions and / or road signs; and determine, based on weight values ​​of the environmental difference information, whether the environmental features of the construction diversions and / or road signs should be used as environmental features for map fusion.

[0166] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0167] The navigation information generating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0168] An embodiment of the present invention further provides an electronic device having the above Figure 5 or Figure 6 The lightweight multi-pass mapping device shown.

[0169] See also Figure 7, is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0170] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0171] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the lightweight multi-pass mapping method shown in the above embodiment.

[0172] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0174] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0175] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0176] In addition, the electronic device in this embodiment may further include at least one communication interface for communicating with a vehicle or other devices.

[0177] Optionally, the electronic device is a vehicle controller or a vehicle control unit (VCU).

[0178] The present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or as computer code that can be recorded on a storage medium, or downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the method described herein can be stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory.

[0179] It can be understood that a computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor or hardware, the lightweight multi-pass mapping method shown in the above embodiment is implemented.

[0180] An embodiment of the present invention provides a computer program product comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the lightweight multi-pass mapping method according to any embodiment of the present invention.

[0181] In addition, see Figure 8 An embodiment of the present invention further provides a vehicle, which includes a vehicle controller and at least one sensor. In addition, the vehicle may also include more or fewer other devices / components such as a display screen, which is not limited in this embodiment.

[0182] The at least one sensor is used to collect environmental information, driving trajectory information, road information, etc. The vehicle controller is used to execute the lightweight multi-pass mapping method shown in the above embodiment. For the specific process, please refer to the above embodiment. Figure 1 、 Figure 3 and Figure 4 The content is recorded, and this embodiment will not be repeated here.

[0183] The above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight multi-pass mapping method, characterized in that: The method comprises: Obtaining a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route; Aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps; Identifying environmental difference information between the memory map and the historical map, and extracting corresponding environmental features based on the environmental difference information; fusing the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features; The step of aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps includes: matching the target route in the memory map with a history map, and searching whether there is at least one route in the history map that matches the target route; If yes, respectively obtain a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route; Keyframe association is performed on at least one keyframe in the first keyframe set and the second keyframe set, map elements are associated between the historical map and the memory map, and pose optimization processing is performed on at least one associated keyframe to obtain the two aligned maps.

2. The method according to claim 1, characterized in that Associating key frames on at least one key frame in the first key frame set and the second key frame set includes: pairing key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair; The association of map elements between the historical map and the memory map includes: establishing an association relationship between map elements of the history map and the memory map based on the at least one key frame pair; The performing pose optimization processing on the at least one associated key frame to obtain the two aligned maps includes: Optimizing the poses of the associations between the map elements and the key frames in the first key frame set and the second key frame set to obtain the poses of the optimized trajectories; The two aligned maps are obtained according to the pose of the optimized trajectory, the map elements and the association relationship between the map elements.

3. The method according to claim 2, characterized in that The establishing, based on the at least one key frame pair, an association relationship between map elements of the historical map and the memory map includes: Determining discrete elements and / or continuous elements of the history map and the memory map based on the at least one key frame pair, and establishing association relationships between the discrete elements and / or the continuous elements; The discrete elements include at least one of the following: Dashed lines, used to indicate a portion of a lane divider or marking; Crosswalks are used to indicate areas where pedestrians cross the road; Stop line: a marking line used to indicate that vehicles must stop and wait before this line; Arrows, used to indicate direction; The continuous elements include: one or more of lane boundaries and road boundaries.

4. The method according to claim 3, characterized in that The performing pose optimization on the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the pose of the optimized trajectory includes: Obtaining an objective function for minimizing a position difference between an associated map element and a keyframe; The objective function is used to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, where the pose of the key frame includes the position and orientation of the key frame.

5. The method according to claim 4, characterized in that After obtaining the two aligned maps, the method further includes: Check whether the absolute distance and distribution between the associated map features are within the preset range; and, Checking whether some or all of the important map features have been successfully associated with the poses in the trajectory; and, Check whether the difference between the pose of the optimized trajectory and the pose before optimization is within an allowable range; If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps until the output check result meets the conditions; If all the results in the check are yes, the two aligned maps are output.

6. The method according to any one of claims 1 to 5, characterized in that The step of obtaining a memory map created after the vehicle has traveled multiple times on the target route includes: Acquire sampling data of a vehicle traveling multiple times on a target route, wherein the sampling data includes route information of the target route; The memory map is created according to the environmental information of the target route.

7. The method according to claim 6, characterized in that The sampled data also includes environmental information; The identifying of environmental difference information between the memory map and the historical map, and extracting corresponding environmental features according to the environmental difference information, includes: identifying, based on the environmental information, environmental difference information between the memory map and the historical map using a recognition algorithm, the environmental difference information including construction diversions and / or road signs; Extracting corresponding environmental features according to the construction diversion and / or the road signs; It is determined whether the environmental features of the construction diversion and / or the road sign are used as environmental features for map fusion according to the weight value of the environmental difference information.

8. A lightweight multi-pass mapping device, characterized in that: The device comprises: an acquisition module, configured to acquire a memory map created after the vehicle has traveled multiple times on the target route, wherein the memory map includes the target route; an alignment module, configured to align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps; an identification module, configured to identify environmental difference information between the memory map and the historical map, and extract corresponding environmental features based on the environmental difference information; a generating module, configured to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features; Among them, the alignment module is specifically used to match the target route in the memory map with the historical map, and search whether there is at least one route in the historical map that matches the target route; if so, obtain a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route; perform key frame association on at least one key frame in the first key frame set and the second key frame set, associate map elements between the historical map and the memory map, and perform posture optimization processing on at least one associated key frame to obtain the two aligned maps.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein the memory and the processor are connected; The memory stores computer instructions, and the processor executes the lightweight multi-pass mapping method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 7.

12. A vehicle, characterized in that: It includes a vehicle controller, which is used to execute the lightweight multi-pass mapping method according to any one of claims 1 to 7.

Images