Lightweight multi-trip mapping method, apparatus and device, and vehicle

By adopting a lightweight multi-trip map construction method in the creation of electronic maps, data is collected through multiple driving of vehicles and memory maps and historical maps to identify environmental differences, the problem of low accuracy of single-trip map construction is solved and the electronic map is improved accuracy and integrity, and save storage resources.

CN119085670AActive Publication Date: 2024-12-06GUANGZHOU XIAOPENG CONNECTIVITY TECH CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202411217855.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

In the existing electronic map creation method, single memory route construction cannot fully extract the semantic information of the road, resulting in the generated electronic map being low in accuracy or incomplete road information.

Method used

The lightweight multi-trip map construction method is adopted, and data is collected by driving on the target route multiple times, memory maps are obtained and historical maps are aligned for trajectory, environmental differences information is identified, environmental features are extracted, and electronic maps are fused.

Benefits of technology

It improves the accuracy of electronic maps and the integrity of road information, reduces storage resource requirements, and saves storage space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085670A_ABST
    Figure CN119085670A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of electronic maps, and discloses a lightweight multi-trip mapping method, device and equipment and a vehicle, the lightweight multi-trip mapping method comprises the following steps: obtaining a memory map created after the vehicle travels for multiple trips on a target route, the memory map comprising the target route; performing track alignment on the target route in the memory map and the corresponding route in the historical map to obtain two aligned maps; environment difference information between the memory map and the historical map is recognized, and corresponding environment features are extracted according to the environment difference information; and fusing the two aligned maps and the environment features to generate an electronic map containing the target route and the environment features. According to the method, road information and road surrounding environment information are all collected, the situation that route information is missed due to single sampling data is avoided, and the integrity of road information of mapping is improved. In addition, the features of the surrounding environment are fused, so that the fused electronic map can restore a real map scene better.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electronic maps, and in particular to a lightweight multi-pass mapping method, device, equipment and vehicle. Background Art

[0002] With the popularity of mobile devices, electronic maps have been seamlessly integrated into people's daily lives. Whether it is a smartphone, tablet or car navigation system, it can provide users with convenient map services anytime and anywhere. Creating electronic maps requires the collection of a large amount of basic geographic data, including road networks, topography, building layout, environmental information, etc. These data may come from multiple channels such as satellite remote sensing, aerial photography, ground measurement, and data collected by users while driving.

[0003] At present, the method of mapping by memorizing routes is adopted, that is, after the user's car travels on the preset route once, the information on the preset route is collected, such as road boundaries, road surface marking elements, etc., to create an electronic map. However, single-trip memorized route mapping may not fully extract the semantic information of the road, such as lanes, traffic signs, etc., and there are factors such as omissions, occlusions or geometric accuracy errors in the collected route information, resulting in low accuracy of the generated electronic map or incomplete road information. Summary of the invention

[0004] In view of this, the present invention provides a lightweight multi-pass mapping method, device, equipment and vehicle to solve the problem of low accuracy of the created electronic map or incomplete road information.

[0005] In a first aspect, the present invention provides a lightweight multi-pass mapping method, the method comprising:

[0006] Obtaining a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route;

[0007] Aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps;

[0008] Identifying environmental difference information between the memory map and the historical map, and extracting corresponding environmental features according to the environmental difference information;

[0009] The two aligned maps and the environmental features are fused to generate an electronic map including the target route and the environmental features.

[0010] The method provided in this aspect uses sampling data after the vehicle has traveled multiple times on the target route to build a map. Compared with the method of using single-trip sampling data to build a map, this method can collect all information about the road and the surrounding environment of the road, avoid omissions of route information in single-trip sampling data, and improve the integrity of the road information for map building.

[0011] In addition, by aligning the created memory map and the historical map, the target routes of the two aligned maps are matched, the map accuracy of the subsequent fusion map is improved, and the surrounding environment features are integrated, so that the fused electronic map can better restore the real map scene. In addition, only the two aligned trajectories and the intermediate results of the fusion map need to be stored, so compared with storing and reporting more route data, this method requires less storage resources, thereby saving storage space.

[0012] In combination with the first aspect, in a possible implementation, aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps includes:

[0013] Matching the target route in the memory map with a history map to find out whether there is at least one route in the history map that matches the target route;

[0014] If yes, trajectories of the matching route in the historical map and the target route are aligned to obtain two aligned maps.

[0015] In combination with the first aspect, in another possible implementation, aligning the matched route in the historical map with the target route to obtain two aligned maps includes:

[0016] Respectively obtaining a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route;

[0017] Key frame association is performed on at least one key frame in the first key frame set and the second key frame set, map elements are associated between the historical map and the memory map, and posture optimization processing is performed on at least one associated key frame to obtain the two aligned maps.

[0018] In combination with the first aspect, in another possible implementation, key frame association is performed on at least one key frame in the first key frame set and the second key frame set, including: pairing the key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair;

[0019] The associating the map elements between the historical map and the memory map comprises: establishing an associative relationship between the map elements of the historical map and the memory map according to the at least one key frame pair;

[0020] The performing posture optimization processing on at least one associated key frame to obtain the two aligned maps includes: performing posture optimization on the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the posture of the optimized trajectory; and obtaining the two aligned maps according to the posture of the optimized trajectory, the map elements and the association relationship between the map elements.

[0021] In combination with the first aspect, in another possible implementation, establishing an association relationship between map elements of the history map and the memory map according to the at least one key frame pair includes:

[0022] Determine the discrete elements and / or continuous elements of the history map and the memory map according to the at least one key frame pair, and establish an association relationship between the discrete elements and / or the continuous elements;

[0023] The discrete elements include at least one of the following:

[0024] A dashed line is used to indicate a portion of a lane divider or marking line;

[0025] Crosswalks are used to indicate areas where pedestrians cross the road;

[0026] Stop line, a marking line used to indicate that vehicles need to stop and wait before this line;

[0027] Arrows, used to indicate direction;

[0028] The continuous elements include: one or more of lane boundaries and road boundaries.

[0029] In combination with the first aspect, in another possible implementation, the performing posture optimization on the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the posture of the optimized trajectory includes:

[0030] Obtaining an objective function, the objective function being used to minimize the position difference between the associated map element and the key frame;

[0031] The objective function is used to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, where the pose of the key frame includes the position and orientation of the key frame.

[0032] In combination with the first aspect, in another possible implementation manner, after obtaining the two aligned maps, the method further includes:

[0033] Check whether the absolute distance and distribution between the associated map elements are within a preset range; and check whether some or all of the important map elements have been successfully associated with the poses in the trajectory; and check whether the difference between the pose of the trajectory after optimization and the pose before optimization is within an allowable range;

[0034] If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps until the output check result meets the conditions;

[0035] If all the results in the check are yes, the two aligned maps are output.

[0036] In combination with the first aspect, in another possible implementation, the memory map created after the vehicle has traveled multiple times on the target route includes: obtaining sampling data of the vehicle traveling multiple times on the target route, the sampling data including route information of the target route; and creating the memory map based on environmental information of the target route.

[0037] In combination with the first aspect, in yet another possible implementation, the sampled data further includes environmental information;

[0038] The identifying the environmental difference information between the memory map and the historical map, and extracting corresponding environmental features according to the environmental difference information, includes:

[0039] According to the environmental information, identifying environmental difference information between the memory map and the historical map through a recognition algorithm, the environmental difference information including construction diversion and / or road signs;

[0040] Extracting corresponding environmental features according to the construction diversion and / or the road signs;

[0041] It is determined whether the environmental features of the construction diversion and / or the road sign are used as environmental features for map fusion according to the weight value of the environmental difference information.

[0042] In a second aspect, the present invention further provides a lightweight multi-pass mapping device, the device comprising:

[0043] An acquisition module, used to acquire a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route;

[0044] An alignment module, used for aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps;

[0045] An identification module, used to identify environmental difference information between the memory map and the historical map, and extract corresponding environmental features according to the environmental difference information;

[0046] The generation module is used to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features.

[0047] In a third aspect, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the lightweight multi-pass mapping method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0048] Optionally, the electronic device is a vehicle controller.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment thereof.

[0050] In addition, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the lightweight multi-pass mapping method described in the above-mentioned first aspect or any corresponding embodiment thereof.

[0051] In a fifth aspect, the present invention further provides a vehicle, comprising a vehicle controller, wherein the vehicle controller is used to execute the lightweight multi-pass mapping method described in the first aspect or any corresponding embodiment thereof.

[0052] The lightweight multi-trip mapping method, device, equipment and vehicle provided in this aspect use sampling data after the vehicle has traveled multiple times on the target route to build a map. Compared with the method of using single-trip sampling data to build a map, this method can collect all information about the road and the surrounding environment of the road, avoid omissions of route information in single-trip sampling data, and improve the integrity of the road information for mapping.

[0053] In addition, by aligning the created memory map and the historical map, the target routes of the two aligned maps are matched, the map accuracy of the subsequent fusion map is improved, and the surrounding environment features are integrated, so that the fused electronic map can better restore the real map scene. In addition, only the two aligned trajectories and the intermediate results of the fusion map need to be stored, so compared with storing and reporting more route data, this method requires less storage resources, thereby saving storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 is a flowchart of a lightweight multi-pass mapping method according to an embodiment of the present invention;

[0056] Figure 2a is a schematic diagram of a target route before trajectory alignment according to an embodiment of the present invention;

[0057] Figure 2b is a schematic diagram of a target route after trajectory alignment according to an embodiment of the present invention;

[0058] Figure 3 is a flowchart of another lightweight multi-pass mapping method according to an embodiment of the present invention;

[0059] Figure 4 is a flowchart of another lightweight multi-pass mapping method according to an embodiment of the present invention;

[0060] Figure 5 is a structural block diagram of a lightweight multi-pass mapping device according to an embodiment of the present invention;

[0061] Figure 6 is a structural block diagram of another lightweight multi-pass mapping device according to an embodiment of the present invention;

[0062] Figure 7 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention;

[0063] Figure 8 is a schematic structural diagram of a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0065] Electronic maps integrate advanced Geographic Information System (GIS), remote sensing technology, Global Positioning System (GPS) and Internet technology, transforming traditional paper maps into a digital platform that is highly interactive, rich in information and updated in real time.

[0066] In the process of creating an electronic map, a user's car usually collects data on a single trip route and reports the data to a server or a network center, which then builds a map based on the data reported by the user's car.

[0067] This mapping method has the following problems:

[0068] 1. Low accuracy: The geometric accuracy of single-trip mapping data is greatly affected by factors such as the performance of the collection equipment and environmental changes. In addition, single-trip mapping may result in omissions, occlusions, or geometric accuracy errors in the collected route information, resulting in low accuracy of the generated electronic map or incomplete road information.

[0069] 2. Redundant information management: After the fusion of multiple mapping data, a large amount of redundant information may be generated, increasing the storage and computing burden. In addition, when fusion of multiple route sampling data, effective compensation and correction of geometric errors are required, which increases the computing burden.

[0070] 3. Adaptability to environmental changes: The sampling environment changes dynamically, and the mapping data collected in different time periods may be quite different.

[0071] The present invention aims to detect the current change areas and update the electronic map through the sampling data of multiple memory learning of the same route, solve the problem of single-trip mapping, and realize the high-quality fusion of lightweight multi-trip mapping data, thereby meeting the needs of AI driving and other scenarios for high-quality and lightweight maps.

[0072] It should be noted that the lightweight multi-pass mapping method provided in the embodiment of the present application can be executed by a mapping device or a mapping equipment, which can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. The electronic device can be a vehicle-side control device, such as a vehicle controller, or it can also be a terminal device or a server device. The embodiment of the present application does not specifically limit the electronic device. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0073] According to an embodiment of the present invention, an embodiment of a lightweight multi-pass mapping method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0074] In this embodiment, a lightweight multi-pass mapping method is provided, which can be used in the above-mentioned electronic device. Figure 1 is a flow chart of a lightweight multi-pass mapping method according to an embodiment of the present invention, the process comprising:

[0075] Step S101: Obtain a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route.

[0076] The target route is the route that the user's car expects to map, such as the route from starting point A to end point B, that is, the target route A→B.

[0077] An implementation method includes: obtaining sampling data of a vehicle traveling multiple times on a target route, the sampling data including route information of the target route; and creating a memory map based on the environmental information of the target route.

[0078] The sampling data is obtained by a mass-produced vehicle or a low-cost collection device after multiple sampling trips on the target route A→B.

[0079] The route information may include the following: Route direction: used to indicate the starting point, end point and major nodes of the sampling route, which helps to understand the overall flow direction of the sampling activity. Plane alignment: including the specific location and length of straight segments and curve segments (such as left and right turns), which helps to accurately restore the sampling path on the map. Mileage pile number: In order to accurately indicate the total length of the route and the length of each section, kilometer piles and hundred-meter piles are usually set, and the corresponding mileage values ​​are marked. In addition, route information also includes information such as intersections and overpass structures.

[0080] Each time the user's car passes through the target route, a sample of data can be obtained, and then the sampled data is reported to the server or the vehicle controller, which obtains and updates it in real time to generate a memory map. The method of creating a memory map includes but is not limited to the SLAM (Simultaneous Localization and Mapping) method. The SLAM method is mainly used for real-time positioning and map construction of robots and autonomous vehicles. It can also compare the real-time collected sensor data with the existing map data to identify changes or errors in the map.

[0081] In addition, the sampled data may also include environmental information.

[0082] Step S102: Align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps.

[0083] The historical map is one or more electronic maps pre-stored locally on the server or vehicle controller, each of which includes map elements of the target route recorded in the historical records. The source of the historical map can be generated on the server side after being reported by other user vehicles based on the collected data, or generated and reported by other clients or third-party organizations.

[0084] After the alignment of the two maps above, it is assumed that Figure 1 and Map 2, where Figure 1 is an aligned map generated based on the memory map, and another map 2 is an aligned map generated based on the history map. In this embodiment, the number of history maps is one, but in fact, it can also be two or more, which is not limited in this embodiment.

[0085] One implementation method of route trajectory alignment is to use GIS software (such as ArcGIS, QGIS, etc.) to load the data of memory maps and historical maps. In the software, two routes are aligned through spatial analysis tools, and the alignment process includes: key frame association, key frame pairing, path correction or posture optimization and other algorithm processing.

[0086] See also Figure 2a and Figure 2b The following is a schematic diagram of the target route in the memory map and the corresponding route in the history map before and after alignment. Figure 2a To align the previous trajectory diagram, the trajectories of the two routes are quite different. Figure 2b This is a schematic diagram of the trajectory after alignment. After alignment, the route trajectories of the two maps overlap or basically overlap.

[0087] Step S103: identifying the environmental difference information between the memory map and the history map, and extracting corresponding environmental features according to the environmental difference information.

[0088] Environmental information refers to the content on memory maps or historical maps involving topography, roadside landmarks, meteorological conditions, etc. Among them, topography includes: terrain undulations, geomorphic features (such as mountains, rivers, lakes, etc.) and feature distribution (such as buildings, roads, vegetation, etc.) on both sides of the sampling route; this information helps to analyze the geographical environment characteristics of the sampling area. Meteorological conditions include: temperature, humidity, air pressure, wind speed, wind direction and other meteorological conditions at the time of sampling have an important impact on the sampling results, so they need to be recorded in detail. Roadside landmarks include: construction diversion information, road signs, such as slow driving, no parking, school attachments and other signs / signs.

[0089] It should be understood that the environmental information may also include other special environmental factors, which are not limited in this embodiment.

[0090] Environmental difference information refers to the difference information between the environmental information on the memory map and the environmental information on the historical map. For example, if a construction diversion sign is included on the memory map but not on the historical map, the construction diversion sign can be used as one of the environmental difference information.

[0091] The environmental feature refers to the feature of the map corresponding to the environmental difference. For example, if the environmental difference information is a construction diversion sign, the extracted environmental feature is: a sign mark or pattern of the construction diversion sign on the map.

[0092] This step S103 specifically includes: according to the environmental information, identifying the environmental difference information between the memory map and the historical map through a recognition algorithm, the environmental difference information including construction diversions and / or road signs; extracting corresponding environmental features according to the construction diversions and / or road signs; and determining whether the environmental features of the construction diversions and / or road signs are used as environmental features for map fusion according to the weight values ​​of the environmental difference information.

[0093] Specifically, the recognition algorithm includes: image processing technology (such as SIFT, SURF, ORB, etc.) to extract key frames and local features from the electronic map image, and then be able to describe the significant parts of the image, such as signs, building outlines, etc. It also includes: feature matching algorithms, object detection algorithms, such as deep learning models (such as YOLO, SSD, Faster R-CNN, etc.) to detect objects such as signs and traffic signs in the electronic map. This embodiment does not limit the specific recognition algorithm.

[0094] The system can set a weight value for all environmental difference information of construction diversions and / or road signs. If the weight value exceeds or equals the threshold, the environmental feature corresponding to the environmental difference is determined, such as a sign indicating a construction diversion, or a road sign sign can be used as an environmental feature for map fusion. If the weight value is less than the threshold, the environmental difference is ignored and the environmental feature is not added to the map fusion process. Specifically, the setting of the weight value can be customized by the user or determined by system calculation and evaluation. For example, the fusion weight value will be assigned different fusion weight values ​​based on the weather conditions collected at the time, vehicle congestion, and the distance in time. This embodiment does not limit this.

[0095] Step S104: Fusing the two aligned maps and environmental features to generate an electronic map including the target route and environmental features.

[0096] In this step, the two aligned maps obtained in the above step S102 and the environmental features output in step S103 are fused according to the image fusion algorithm to generate a new electronic map, which includes the above target route and environmental features.

[0097] One possible implementation is to integrate the fused environmental features and target route information into an electronic map. Electronic maps can be represented by raster methods, vector methods, and other methods. The specific choice depends on the application scenario and accuracy requirements. The generated electronic map is optimized and adjusted to ensure the clarity, readability, and accuracy of the map. Finally, the map editing tool is used for manual adjustment, or the algorithm is used to automatically optimize the layout and display of the map.

[0098] The lightweight multi-trip mapping method provided in this embodiment uses sampling data after the vehicle has traveled multiple times on the target route to build a map. Compared with the method of using single-trip sampling data to build a map, this method can collect all information about the road and the surrounding environment of the road, avoid omissions of route information in single-trip sampling data, and improve the integrity of the road information for mapping.

[0099] In addition, by aligning the created memory map and the historical map, the target routes of the two aligned maps are matched, the map accuracy of the subsequent fusion map is improved, and the surrounding environment features are integrated, so that the fused electronic map can better restore the real map scene. In addition, only the two aligned trajectories and the intermediate results of the fusion map need to be stored, so compared with storing and reporting more route data, this method requires less storage resources, thereby saving storage space.

[0100] In a possible implementation of this embodiment, as Figure 3As shown, the above step S102, aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps, specifically includes:

[0101] Step S1021: Match the target route in the memory map with the history map.

[0102] Step S1022: Check whether there is at least one route in the history map that matches the target route.

[0103] The historical map may contain one or more routes. According to the matching algorithm, as long as one of the one or more routes matches the target route, it is determined to be "yes"; if yes, step S1023 is executed.

[0104] If there is no matching route, it is determined as "No", and it is determined that there is no route identical or similar to the target route in the currently searched historical map, and the matching process is terminated. Alternatively, other historical maps may be replaced for re-search and matching.

[0105] Step S1023: aligning the matched route in the historical map with the target route to obtain two aligned maps.

[0106] In a possible implementation of this embodiment, the above step S1023, aligning the matching route in the historical map with the target route to obtain two aligned maps, specifically includes:

[0107] First, obtain the first key frame set corresponding to the matching route in the historical map and the second key frame set corresponding to the target route. The first key frame set consists of at least one key frame in the historical map, and similarly, the second key frame set consists of at least one key frame on the target route in the memory map. Figure 2a or Figure 2b As shown in the figure, each dot or triangle pattern is a key frame. Figure 2a and Figure 2b There are two maps in the image consisting of multiple images containing key frames.

[0108] Then, key frame association is performed on at least one key frame in the first key frame set and the second key frame set, map element association is performed between the historical map and the memory map, and posture optimization processing is performed on at least one associated key frame to obtain two aligned maps.

[0109] In this step, the key frames involved in the target route in the first key frame set and the second key frame set are processed in three stages: key frame association, map element association, and key frame pose optimization, and finally two aligned maps are output. The above three-stage processing process is described in detail below.

[0110] ①Key frame association processing

[0111] Specifically, Figure 4 As shown, the above-mentioned key frame association of at least one key frame in the first key frame set and the second key frame set includes:

[0112] Step S301: based on a Hidden Markov Model (HMM), pairing key frames in the first key frame set and the second key frame set according to the trajectory of the target route to generate at least one key frame pair.

[0113] Specifically, the trained HMM model is used to match the key frames of the two trajectories. The matching process involves using the Viterbi algorithm to find the most likely hidden state sequence (i.e., matching key frame pairs). The matching results are output, including the pairing relationship of each key frame (i.e., at least one key frame pair) and possible errors or uncertainties. This step is used to match map elements (lane lines, ground markings) based on the pairing relationship in the future to form matching constraints.

[0114] ②Map element association

[0115] This embodiment describes the process of associating map elements between the historical map and the memory map. Figure 4 As shown, the map element association includes:

[0116] Step S302: establishing an association relationship between map elements of the history map and the memory map based on the at least one key frame pair.

[0117] A specific implementation includes: determining discrete elements and / or continuous elements of the history map and the memory map based on the at least one key frame pair, and establishing an association relationship between the discrete elements and / or continuous elements.

[0118] The discrete elements include at least one of a dotted line, a crosswalk, a stop line, and an arrow.

[0119] A dash is used to indicate a lane divider or part of a marking line; a crosswalk is used to indicate an area for pedestrians to cross the road; a stop line is used to indicate a marking line before which vehicles must stop and wait, and is commonly seen at traffic lights; an arrow is used to indicate direction, such as the direction of a lane or a turn.

[0120] Continuous Elements include one or more of lane boundaries and road boundaries. Lane boundaries are lines that define the boundaries between lanes, usually solid or dashed lines; road boundaries are lines that define the boundaries between roads and surrounding areas (such as sidewalks, grass, or other roads).

[0121] In this embodiment, when doing the map element association processing, before using the iterative closest point (ICP) algorithm for point cloud registration, the initial pose (i.e., the initial relative position and pose between two point clouds) may be unknown or very inaccurate. Directly using ICP may cause the algorithm to fall into a local optimal solution and fail to find the true best match, so ICP alignment is performed first and then pairing. In each iteration of ICP, the algorithm will find the closest point pair and calculate an optimal transformation matrix (rotation and translation) based on these point pairs, and then apply this transformation to update the position of the point cloud until a certain convergence condition is reached (such as the transformation amount is less than a certain threshold), at which time it is considered that the optimal matching result has been found. When the initial pose is good enough, ICP iteration will no longer be performed, and using this good initial pose as the final result, or making a small amount of fine-tuning on this basis, may be more efficient and accurate. This method dynamically adjusts the ICP algorithm usage strategy according to the quality of the initial pose to ensure that accurate matching results can be obtained in various situations.

[0122] ③Key frame pose optimization

[0123] like Figure 4 As shown, the above-mentioned pose optimization processing of at least one associated key frame to obtain two aligned maps specifically includes:

[0124] Step S303: optimizing the positions and postures of the associations between the map elements and the key frames in the first key frame set and the second key frame set to obtain the positions and postures of the optimized trajectories.

[0125] One implementation includes: obtaining an objective function, and then using the objective function to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, the pose of the key frame including the position and orientation of the key frame. The objective function is used to minimize the position difference between the associated map element and the key frame; this usually means minimizing a cost function that calculates some measure of the position difference between all paired elements.

[0126] Step S304: obtaining the two aligned maps according to the position and posture of the optimized trajectory, the map elements and the associations between the map elements.

[0127] Specifically, each key frame in the first key frame set and the second key frame set contains location information (such as longitude and latitude) and possible direction information. The paired map elements include: the correspondence between the key frames in the multiple trajectories and the specific elements on the electronic map (such as dotted lines, crosswalks, stop lines, arrows, lane boundaries, and road boundaries).

[0128] After optimization using the optimization algorithm, the position and orientation of each keyframe in each trajectory are adjusted to better match the map features and meet various constraints. Among them, the map features after pose optimization: the map features in the output themselves have not changed, but their pairing relationship (that is, which keyframes are associated with) may become more accurate due to the optimization process.

[0129] The specific implementation of step S304 can be an iterative process to find the optimal solution by constantly trying different pose combinations. The iterative process involves point-line constraints and point-point constraints. Point-line constraints: For linear elements such as boundary, cross walk and stopline, point-line constraints are used. This means that the keyframe (as a point) needs to be optimized to maintain a certain distance and direction relationship with its paired linear element (as a line). Point-point constraints: For point elements such as dash and arrow, point-point constraints are used. This means that the keyframe needs to be optimized to be as close as possible in position to the point element it is paired with.

[0130] After the above keyframe pose optimization, the matching relationship between the trajectory and the map elements is improved by adjusting the keyframe pose, and finally the two aligned maps are output to improve the accuracy of map fusion.

[0131] In a possible implementation of this embodiment, after obtaining the two aligned maps in step S304, the method further includes: an alignment self-checking process. The process is used to check whether the aligned maps meet preset requirements.

[0132] Specifically, it includes: checking whether the absolute distance and distribution between the associated map elements are within a preset range; and checking whether some or all of the important map elements have been successfully associated with the posture in the trajectory; and checking whether the difference between the posture of the trajectory after optimization and the posture before optimization is within an allowable range.

[0133] If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps, that is, the aforementioned step S102 is re-executed until the output check result meets the condition. If all the results in the check are positive, the aligned two maps are output.

[0134] Furthermore, it is checked whether the absolute distance and distribution between the map elements are within a preset range, specifically including: the physical distance between the associated elements and their distribution on the entire map. By calculating these distances and distributions, the accuracy of the association can be evaluated, such as whether there are abnormal long-distance associations or associations that are dense in local areas and sparse in other areas.

[0135] Check which map features are not associated, that is, check whether all important map features have been successfully associated with the poses in the trajectory. If there are unassociated features, it indicates missing data, limitations of the association algorithm, or mismatch between the map and the trajectory.

[0136] In addition, it is checked whether the difference between the pose of the optimized trajectory and the pose before optimization is within the allowed range. This step compares the difference between the optimized trajectory (based on the optimization results of map features and pose) and the trajectory before optimization (such as based on GPS and inertial navigation data). This difference can reveal whether the optimization process has effectively improved the accuracy of the trajectory, especially for areas with poor GPS signals. For areas with poor GPS signals, multiple iterative optimizations will be performed: For areas with unstable GPS signals, the system may perform multiple iterative optimization processes to improve the accuracy of the alignment of the two maps.

[0137] During the self-check process, if it is found that the association between certain map elements and trajectories may be incorrect (for example, based on the detection results of the above steps), it may be necessary to return to the previous step S301 to re-pair and re-establish the association relationship, and execute the posture optimization steps S303 and S304 again.

[0138] The self-check process continues until a certain convergence criterion is reached. For example, the results of several consecutive iterations change very little, or the above check results are all "yes", then the system will output the final optimized pose and map elements, and output the two aligned maps. These output results will be used for subsequent map fusion and mapping.

[0139] The method provided in this embodiment only needs to store the reported data of two trajectories, such as the trajectory corresponding to the target route and the latest uploaded trajectory, and the intermediate result of the fusion mapping, during the process of aligning and fusion mapping of the memory map and the history map. There is no need to store and report more route data. Therefore, this method requires less storage resources and can save storage space.

[0140] In addition, in this embodiment, the multi-pass fusion algorithm performs incremental fusion between the results of multiple passes of mapping, the number of elements that need to be processed is greatly compressed, the processing complexity is low, and the efficiency of fusion mapping is improved.

[0141] In this embodiment, a lightweight multi-pass mapping device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0142] This embodiment provides a lightweight multi-pass mapping device for realizing the above Figure 1 , Figure 3 or Figure 4 The method steps shown are as follows: Figure 5 As shown, the device includes: an acquisition module 510, an alignment module 520, an identification module 530 and a generation module 540. In addition, the device may also include other more or fewer units or modules, such as a storage unit, etc., which is not limited in this embodiment.

[0143] The acquisition module 510 is used to acquire a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route.

[0144] The alignment module 520 is used to align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps.

[0145] The identification module 530 is used to identify the environmental difference information between the memory map and the historical map, and extract corresponding environmental features according to the environmental difference information.

[0146] The generating module 540 is used to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features.

[0147] In some optional embodiments, the alignment module 520 is also used to match the target route in the memory map with the historical map, and to find out whether there is at least one route in the historical map that matches the target route; if so, the matching route in the historical map is aligned with the target route to obtain two aligned maps.

[0148] In some other optional embodiments, the alignment module 520 is further specifically used to respectively obtain a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route; perform key frame association on at least one key frame in the first key frame set and the second key frame set, associate map elements between the historical map and the memory map, and perform pose optimization processing on at least one associated key frame to obtain two aligned maps.

[0149] In some further optional implementations, the alignment module 520 is further configured to pair the key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair.

[0150] The alignment module 520 is further specifically configured to establish an association relationship between the map elements of the history map and the memory map according to the at least one key frame pair.

[0151] The device provided in this embodiment also includes an optimization module. Figure 5 Not shown in FIG.

[0152] The optimization module is used to optimize the association relationship between map elements and the key frames in the first key frame set and the second key frame set to obtain the position and posture of the optimized trajectory.

[0153] The alignment module 520 is further used to obtain the two aligned maps according to the posture of the optimized trajectory, the map elements and the association relationship between the map elements.

[0154] In some further optional embodiments, the alignment module 520 is further specifically used to determine the discrete elements and / or continuous elements of the historical map and the memory map based on the at least one key frame pair, and to establish an association relationship between the discrete elements and / or the continuous elements.

[0155] The discrete elements include at least one of the following:

[0156] A dotted line is used to indicate a lane divider or part of a marking line; a crosswalk is used to indicate an area for pedestrians to cross the road; a stop line is used to indicate a marking line before which vehicles must stop and wait; an arrow is used to indicate a direction.

[0157] Continuous features include: one or more of lane boundaries and road boundaries.

[0158] In some other optional implementations, the optimization module is specifically used to obtain an objective function, and use the objective function to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, wherein the pose of the key frame includes the position and orientation of the key frame. The objective function is used to minimize the position difference between the associated map element and the key frame.

[0159] like Figure 6 As shown, the device provided in this embodiment further includes: a checking module 550 and an output module 560 .

[0160] The checking module 550 is used to check whether the absolute distance and distribution between the associated map elements are within a preset range; and to check whether some or all of the important map elements have been successfully associated with the posture in the trajectory; and to check whether the difference between the posture of the optimized trajectory and the posture before optimization is within an allowable range.

[0161] The output module 560 is used to re-associate or optimize the item with a negative result when one or more of the above results are negative in the detection module, and re-obtain two new maps until the output inspection result meets the conditions.

[0162] In addition, the output module 560 is further configured to output the two aligned maps if all the above results are yes when checked by the detection module.

[0163] In some other optional implementations, the acquisition module 510 is specifically used to acquire sampling data of multiple trips of the vehicle on the target route, and to create the memory map according to the environmental information of the target route. The sampling data includes route information of the target route.

[0164] In some further optional implementations, the sampled data also includes environmental information.

[0165] The identification module 530 is specifically used to identify the environmental difference information between the memory map and the historical map through a recognition algorithm according to the environmental information, wherein the environmental difference information includes construction diversions and / or road signs; extract corresponding environmental features according to the construction diversions and / or the road signs; and determine whether the environmental features of the construction diversions and / or the road signs are used as environmental features for map fusion according to the weight values ​​of the environmental difference information.

[0166] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0167] The navigation information generating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0168] The embodiment of the present invention also provides an electronic device having the above Figure 5 or Figure 6 The lightweight multi-pass mapping device shown.

[0169] See also Figure 7, is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0170] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0171] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the lightweight multi-pass mapping method shown in the above embodiment.

[0172] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0174] The electronic device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0175] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0176] In addition, the electronic device in this embodiment may also include at least one communication interface for communicating with a vehicle or other devices.

[0177] Optionally, the electronic device is a vehicle controller, or a vehicle control unit (VCU).

[0178] The embodiment of the present invention also provides a computer-readable storage medium, and the method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or downloaded through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory.

[0179] It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor or hardware, the lightweight multi-pass mapping method shown in the above embodiment is implemented.

[0180] An embodiment of the present invention provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the lightweight multi-pass mapping method of any embodiment of the present invention.

[0181] In addition, see Figure 8 An embodiment of the present invention further provides a vehicle, which includes a vehicle controller and at least one sensor. In addition, the vehicle may also include other more or fewer devices / components such as a display screen, which is not limited in this embodiment.

[0182] The at least one sensor is used to collect environmental information, driving trajectory information, road information, etc. The vehicle controller is used to execute the lightweight multi-pass mapping method shown in the above embodiment. For the specific process, please refer to the above embodiment. Figure 1 , Figure 3 and Figure 4 The content is recorded, and this embodiment will not be repeated here.

[0183] The above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention are described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight multi-pass mapping method, characterized in that: The method comprises: Obtaining a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route; Aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps; Identifying environmental difference information between the memory map and the history map, and extracting corresponding environmental features according to the environmental difference information; The two aligned maps and the environmental features are fused to generate an electronic map including the target route and the environmental features.

2. The method according to claim 1, characterized in that The step of aligning the target route in the memory map with the corresponding route in the history map to obtain two aligned maps includes: Matching the target route in the memory map with a history map to find out whether there is at least one route in the history map that matches the target route; If yes, trajectories of the matching route in the historical map and the target route are aligned to obtain two aligned maps.

3. The method according to claim 2, characterized in that Aligning the matched route in the historical map with the target route to obtain two aligned maps, including: Respectively obtaining a first key frame set corresponding to the matching route in the historical map and a second key frame set corresponding to the target route; Key frame association is performed on at least one key frame in the first key frame set and the second key frame set, map elements are associated between the historical map and the memory map, and posture optimization processing is performed on at least one associated key frame to obtain the two aligned maps.

4. The method according to claim 3, characterized in that Associating at least one key frame in the first key frame set and the second key frame set with a key frame, comprising: Pairing the key frames in the first key frame set and the second key frame set according to the trajectory of the target route based on a hidden Markov model to generate at least one key frame pair; The association of map elements between the historical map and the memory map includes: establishing an association relationship between map elements of the history map and the memory map according to the at least one key frame pair; The performing posture optimization processing on the at least one associated key frame to obtain the two aligned maps includes: Performing posture optimization on the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the posture of the optimized trajectory; The two aligned maps are obtained according to the position and posture of the optimized trajectory, the map elements and the association relationship between the map elements.

5. The method according to claim 4, characterized in that The establishing of an association relationship between the map elements of the history map and the memory map according to the at least one key frame pair includes: Determine the discrete elements and / or continuous elements of the history map and the memory map according to the at least one key frame pair, and establish an association relationship between the discrete elements and / or the continuous elements; The discrete elements include at least one of the following: A dashed line is used to indicate a portion of a lane divider or marking line; Crosswalks are used to indicate areas where pedestrians cross the road; Stop line, a marking line used to indicate that vehicles need to stop and wait before this line; Arrows, used to indicate direction; The continuous elements include: one or more of lane boundaries and road boundaries.

6. The method according to claim 4, characterized in that The step of optimizing the association relationship between the map elements and the key frames in the first key frame set and the second key frame set to obtain the optimized trajectory posture includes: Obtaining an objective function, the objective function being used to minimize the position difference between the associated map element and the key frame; The objective function is used to adjust the pose of the key frame in each trajectory to obtain the pose of the optimized trajectory, where the pose of the key frame includes the position and orientation of the key frame.

7. The method according to claim 6, characterized in that After obtaining the two aligned maps, the method further includes: Check whether the absolute distance and distribution between the associated map elements are within the preset range; and, Checking whether some or all of the important map features have been successfully associated with the poses in the trajectory; and, Check whether the difference between the pose of the trajectory after optimization and the pose before optimization is within an allowable range; If one or more results in the check are negative, the negative result is re-associated or optimized to obtain two new maps until the output check result meets the conditions; If all the results in the check are yes, the two aligned maps are output.

8. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining a memory map created after the vehicle travels multiple times on the target route includes: Acquire sampling data of a plurality of trips of a vehicle on a target route, wherein the sampling data includes route information of the target route; The memory map is created according to the environmental information of the target route.

9. The method according to claim 8, characterized in that The sampled data also includes environmental information; The identifying the environmental difference information between the memory map and the historical map, and extracting corresponding environmental features according to the environmental difference information, includes: According to the environmental information, identifying environmental difference information between the memory map and the historical map through a recognition algorithm, the environmental difference information including construction diversion and / or road signs; Extracting corresponding environmental features according to the construction diversion and / or the road signs; It is determined whether the environmental features of the construction diversion and / or the road sign are used as environmental features for map fusion according to the weight value of the environmental difference information.

10. A lightweight multi-pass mapping device, characterized in that: The device comprises: An acquisition module, used to acquire a memory map created after the vehicle travels multiple times on the target route, wherein the memory map includes the target route; An alignment module, used to align the target route in the memory map with the corresponding route in the history map to obtain two aligned maps; An identification module, used to identify environmental difference information between the memory map and the historical map, and extract corresponding environmental features according to the environmental difference information; The generation module is used to fuse the two aligned maps and the environmental features to generate an electronic map including the target route and the environmental features.

11. An electronic device, characterized in that: comprising a memory and a processor, wherein the memory and the processor are connected; The memory stores computer instructions, and the processor executes the lightweight multi-pass mapping method according to any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the lightweight multi-pass mapping method according to any one of claims 1 to 9.

14. A vehicle, characterized in that: It includes a vehicle controller, which is used to execute the lightweight multi-pass mapping method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Cloud-based trajectory map generation method, device, equipment, and application program

    CN108235725A

  • Semantic mapping method based on track alignment

    CN112833892A

  • High-precision map generation method and device, electronic equipment, medium and program product

    CN114034295A

  • Map generation method and device, electronic equipment and storage medium

    CN114494618A

  • Autonomous parking system and method based on cloud sharing and map fusion

    CN115035728A