Global Map Acquisition Method, Intelligent Device and Computer-Readable Storage Medium

Abstract

This application relates to the field of autonomous driving technology, and specifically relates to a method for obtaining a global map, an intelligent device, and a computer-readable storage medium, aiming to solve the technical problem of how to obtain a globally consistent map for areas without high-precision map coverage. For this purpose, this application conducts multiple trips of map data collection on the area to be measured, and obtains the corresponding single-trip global map according to the map data collection results obtained in each trip; divides each single-trip global map to obtain multiple consecutive sub-maps of each single-trip global map; constructs a factor graph based on all the obtained sub-maps, and performs graph optimization on the sub-maps with the first constraint term and the second constraint term as factor constraints; according to the graph optimization results, splice the sub-maps to obtain the global map of the area to be measured, which can maintain the relative relationship within the sub-maps unchanged, solve the problem of global positioning jumps, and can obtain a globally consistent global map in areas without high-precision map coverage.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a global map acquisition method, an intelligent device, and a computer-readable storage medium. Background Art

[0002] Autonomous driving is attracting more and more attention. In the areas covered by high-precision maps in urban areas, advanced driver assistance has been well applied. At present, the mainstream solution is to collect multiple data through lidar or camera to cover the area where high-precision maps are needed for data fusion. Relying on front-end vision or laser odometer and combining loop detection to achieve closed-loop optimization, high-precision maps can be obtained.

[0003] In the internal roads such as parking lots, there is a lack of HD map coverage, and the need to obtain internal road area maps to make up for the lack of HD maps has become a concern in the field of self-driving. However, in order to achieve full-link autonomous driving, when the vehicle enters the internal road area from the urban high-precision map environment, it is necessary to connect the two to achieve a smooth transition from the HD map to the internal map, which requires the internal map to have a global error within meters. However, the internal roads in the urban area are usually located in urban canyons, and many buildings reflect and interfere with the GNSS signal, resulting in low GNSS signal confidence, many abnormal jumps and instability problems, and multiple acquisition results at the same location have large deviations. The unstable global positioning factor has a great impact on the global consistency of the map, making it difficult to obtain a map with high global consistency.

[0004] Accordingly, the art needs a new global map acquisition solution to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem of how to obtain a globally consistent map for an area not covered by a high-precision map.

[0006] In a first aspect, a method for obtaining a global map is provided, the method comprising:

[0007] Performing multiple trips of map data collection on the area to be tested, and obtaining a single trip global map corresponding to the map data collection result according to each trip of map data collection result, so as to obtain multiple single trip global maps;

[0008] Performing map division on each of the single-trip global maps respectively to obtain a plurality of continuous sub-maps of the single-trip global map;

[0009] A factor graph is constructed according to all the obtained submaps, and the submap is optimized according to the factor graph to optimize the position of the submap and obtain a graph optimization result; the factor node of the factor graph is the submap, and the factor constraint of the factor graph includes a first constraint item and a second constraint item; the first constraint item is a constraint item between the submaps obtained according to the overlap rate between the submaps; the second constraint item is a constraint item obtained according to the global positioning result corresponding to the submap;

[0010] According to the map optimization result, the sub-maps are stitched to obtain a global map of the area to be tested.

[0011] In a technical solution of the above-mentioned global map acquisition method, the method comprises acquiring the first constraint item according to the following steps:

[0012] According to the sub-map, obtaining a voxelized map of the sub-map;

[0013] For each voxelized map corresponding to each two of the sub-maps, obtaining the overlap between the two voxelized maps;

[0014] According to the overlap degree, a relative position between the two sub-maps is obtained as the first constraint item.

[0015] In a technical solution of the above-mentioned global map acquisition method, the step of acquiring a voxelized map of the sub-map according to the sub-map includes:

[0016] Splicing the submaps according to the positions of all key frames in all submaps to obtain a local map;

[0017] The local map is voxelized to obtain a voxelized map of the sub-map.

[0018] In a technical solution of the above-mentioned global map acquisition method, obtaining the relative position between the two sub-maps according to the overlap degree includes:

[0019] Comparing the overlap with a preset overlap threshold;

[0020] For the two submaps whose overlap is greater than the overlap threshold, obtaining the relative position between the two submaps according to the anchor frames of the submaps;

[0021] The anchor frame is one of the key frames of the sub-map.

[0022] In a technical solution of the above-mentioned global map acquisition method, acquiring the relative position between the two sub-maps according to the anchor frames of the sub-maps includes:

[0023] Transforming the two submaps to the anchor frame, performing submap registration, and obtaining the relative pose between the anchor frames as the relative pose between the two submaps;

[0024] In a technical solution of the above-mentioned global map acquisition method, the method includes acquiring the second constraint item according to the following steps:

[0025] For the sub-map having the global positioning result, a global positioning result of a frame with the smallest error among the global positioning results in the sub-map is obtained as the second constraint item.

[0026] In a technical solution of the above-mentioned global map acquisition method, the factor constraint further includes a third constraint item and a fourth constraint item; the third constraint item is a constraint item between the sub-maps obtained according to the loop detection result of the single-pass global map; the fourth constraint item is a constraint item obtained according to the relative posture between the sub-maps during the single-pass global map acquisition process;

[0027] The step of constructing a factor graph according to all the acquired sub-maps, and performing graph optimization on the sub-maps according to the factor graph, optimizing the positions of the sub-maps, and obtaining graph optimization results includes:

[0028] A factor graph is constructed according to all the obtained sub-maps, and the first constraint item, the second constraint item, the third constraint item and the fourth constraint item are used as factor constraints to construct the factor graph, perform graph optimization on the sub-map, optimize the position and posture of the sub-map, and obtain a graph optimization result.

[0029] In a technical solution of the above-mentioned global map acquisition method, the step of constructing a factor graph includes:

[0030] Obtaining an anchor frame of each sub-map; the anchor frame is one of the key frames of the sub-map;

[0031] The factor graph is constructed by taking the anchor frame as the factor node, and taking the first constraint item, the second constraint item, the third constraint item and the fourth constraint item as factor constraints.

[0032] In a technical solution of the above-mentioned global map acquisition method, the method further comprises acquiring the third constraint item according to the following steps:

[0033] For a key frame with a loop detection result, the loop detection result is converted to the anchor frame corresponding to the key frame according to the relative posture between the key frame and the anchor frame corresponding to the key frame, as the third constraint item.

[0034] In a technical solution of the above-mentioned global map acquisition method, the method further comprises acquiring the fourth constraint item according to the following steps:

[0035] According to the single-pass global map, relative positions between anchor frames of adjacent sub-maps in the single-pass global map are obtained as the fourth constraint item of the adjacent sub-maps.

[0036] In a technical solution of the above-mentioned global map acquisition method, the map optimization result is the optimized pose of the anchor frame of the submap;

[0037] The step of performing map stitching on the submaps according to the optimized positions and postures of the submaps to obtain a global map of the area to be measured includes:

[0038] For each submap, according to the optimized pose of the anchor frame and the relative pose between the anchor frame and other key frames in the submap, the optimized poses of other key frames are obtained;

[0039] According to the optimized posture, optimizing the postures of all key frames in the sub-map;

[0040] Map stitching is performed according to the optimized poses of all key frames to obtain a global map of the area to be tested.

[0041] In a technical solution of the above-mentioned global map acquisition method, the acquisition of a single-trip global map corresponding to the map data acquisition result according to each trip of map data acquisition results respectively includes:

[0042] For each trip of the map data collection result, performing local map registration according to each frame of data in the map data collection result to obtain a local map registration result;

[0043] The single-trip global map is obtained according to the global positioning result corresponding to the map data collection result and the local map registration result.

[0044] In a technical solution of the above-mentioned global map acquisition method, the method further includes performing back-end optimization on the single-trip global map according to the following steps:

[0045] For each single-trip global map, performing key frame screening on the single-trip global map to obtain the key frame of the single-trip global map;

[0046] Perform loop detection on all key frames of the single-pass global map and obtain loop detection results;

[0047] According to the loop detection result, backend optimization is performed on the key frames of the single-trip global map to obtain the single-trip global map after backend optimization.

[0048] In a technical solution of the global map acquisition method, the map division is performed on each of the single-trip global maps to obtain a plurality of continuous sub-maps of the single-trip global map, including:

[0049] For each of the single-pass global maps, dividing the single-pass global map based on a preset number of frames to obtain a plurality of continuous sub-maps;

[0050] The relative positions between the key frames in the sub-map remain fixed.

[0051] In a second aspect, a smart device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned global map acquisition method is implemented.

[0052] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned global map acquisition method.

[0053] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0054] In implementing the technical solution of the global map acquisition method provided by the present application, the present application collects map data for the area to be tested for multiple times, and obtains the corresponding single-trip global map according to the map data collection results obtained for each trip; divides each single-trip global map to obtain multiple continuous sub-maps of each single-trip global map; constructs a factor graph based on all the obtained sub-maps, and optimizes the sub-maps with the first constraint item and the second constraint item as factor constraints to obtain the graph optimization result; the first constraint item is the constraint item between the sub-maps obtained according to the overlap between the sub-maps; the second constraint item is the constraint item obtained according to the global positioning result corresponding to the sub-map; according to the graph optimization result, the sub-maps are spliced ​​to obtain the global map of the area to be tested. Through the above configuration, the present application uses the sub-map as the minimum optimization unit of the factor graph, and can maintain the relative relationship within the sub-map unchanged. At the same time, the first constraint item between the submaps is obtained according to the overlap between the submaps. As a factor constraint of the factor graph, it can reduce the data difference between multiple single-trip global maps; based on the second constraint item obtained according to the global positioning result corresponding to the submap, as a factor constraint of the factor graph, it can effectively solve the problem of global positioning jump in scenes such as urban canyons. The global map acquisition method based on the present application can ensure the consistency of the global map acquisition process while ensuring globality, and ensure that a global map with global consistency can be obtained in areas not covered by high-precision maps, thereby realizing the connection with high-precision maps, so that automatic driving can achieve smooth transition when entering internal roads from urban areas, and can still achieve lane-level positioning and navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0056] Figure 1 is a schematic flow chart of the main steps of a method for obtaining a global map according to an embodiment of the present application;

[0057] Figure 2 It is a schematic diagram of the main system block diagram of a global map acquisition method according to an implementation method of an embodiment of the present application;

[0058] Figure 3 is a schematic diagram of the main structure of a sub-map according to an implementation method of an embodiment of the present application;

[0059] Figure 4 is a schematic diagram of a factor graph constructed based on a submap according to an implementation of an embodiment of the present application;

[0060] Figure 5is a schematic diagram of a global map comparison effect obtained by a global map acquisition method according to an example of an embodiment of the present application;

[0061] Figure 6 is a schematic flow chart of the main steps of step S101 according to an implementation of an embodiment of the present application;

[0062] Figure 7 It is a schematic flow chart of the main steps of obtaining the first constraint item according to an implementation method of an embodiment of the present application;

[0063] Figure 8 is a schematic flow chart of the main steps of step S104 according to an implementation of an embodiment of the present application;

[0064] Fig. 9 It is a schematic diagram of the main structure of a smart device according to an embodiment of the present application.

[0065] Reference numerals:

[0066] 11: memory; 12: processor. DETAILED DESCRIPTION

[0067] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0068] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0069] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0070] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.

[0071] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0072] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a method for obtaining a global map according to an embodiment of the present application. Figure 1 As shown, the global map acquisition method in the embodiment of the present application mainly includes the following steps S101 to S104.

[0073] Step S101: performing multiple passes of map data collection on the area to be tested, and obtaining a single pass global map corresponding to the map data collection result according to each pass of map data collection result, so as to obtain multiple single pass global maps.

[0074] In this embodiment, multiple passes of map data collection may be performed on the area to be tested to obtain multiple passes of data collection results. Based on each pass of data collection results, a single pass global map is obtained, thereby obtaining multiple single pass global maps.

[0075] In one implementation, map data can be collected based on sensors such as a laser radar, a camera, and an IMU (Inertial Measurement Unit) of a smart device. Map data can be collected while a smart device equipped with the above sensors is driving in a test area. Multiple trips of map data collection can be achieved based on multiple smart devices, or multiple trips of map data collection can be achieved based on the same smart device driving in the test area multiple times.

[0076] In one implementation, a global positioning factor in a single-trip global map acquisition process may be acquired based on a GNSS (Global Navigation Satellite System) signal.

[0077] Step S102: dividing each single-trip global map into two parts to obtain a plurality of continuous sub-maps of the single-trip global map.

[0078] In this embodiment, each single-pass global map may be divided into two parts to obtain a plurality of continuous sub-maps of the single-pass global map.

[0079] In one embodiment, please refer to the attached Figure 3 , Figure 3 FIG. 1 is a schematic diagram of the main structure of a sub-map according to an implementation of an embodiment of the present application. Figure 3 As shown, a preset number of frames can be set, and based on the preset number of frames, each single-trip global map is divided into multiple continuous sub-maps (sub-maps) of each single-trip global map (i.e., data 1, data 2, ..., data n). Figure 1 、Zi Di Figure 2 , ..., submap m). Those skilled in the art can set the value of the preset number of frames according to the needs of actual applications. Among them, the relative positions between the key frames in the submap are fixed. That is, the submap can be understood as a rigid body, and the relative positions between the key frames in the submap are fixed.

[0080] Step S103: construct a factor graph based on all the obtained submaps, and perform graph optimization on the submaps based on the factor graph, optimize the position of the submaps, and obtain the graph optimization result; the factor nodes of the factor graph are submaps, and the factor constraints of the factor graph include a first constraint item and a second constraint item; the first constraint item is a constraint item between submaps obtained according to the overlap rate between submaps; the second constraint item is a constraint item obtained according to the global positioning result corresponding to the submap.

[0081] In this embodiment, a factor graph can be constructed based on all submaps, and the factor graph can be constructed with the first constraint item and the second constraint item as factor constraints, and the position of the submap can be optimized to obtain a graph optimization result. The first constraint item between the submaps can be obtained based on the overlap rate between the submaps. The second constraint item can be obtained based on the global positioning result corresponding to the submap.

[0082] In one implementation, a global positioning result may be obtained based on a GNSS signal.

[0083] In one implementation, the anchor frame of the submap can be used as a node of the factor graph, and the graph optimization result can be obtained by optimizing the position of the anchor frame of the submap. The anchor frame of the submap is one of the key frames of the submap. Figure 3As shown, the anchor frame is the anchor point of the submap. One of the key frames of the submap can be used as the anchor frame, and the anchor frame is used as a factor node of the factor graph, that is, the state quantity to be optimized. The submap is the smallest optimization unit in the graph optimization process, and the inside of the submap can be considered as a rigid body, that is, the relative posture between the key frames in the submap is unchanged. In other words, the anchor frame of the submap can be constrained with other key frames through relative posture (that is, relative constraint factor), and the anchor frame can realize the transfer of optimized posture through relative constraint factor. Among them, the anchor frame can be the frame with the best global positioning result in the submap key frame, or it can be a middle frame in the submap key frame. Of course, those skilled in the art can also determine the anchor frame of the submap according to the needs of actual applications, which are all within the protection scope of this application.

[0084] In one implementation, for a sub-map having a global positioning result, a global positioning result of a frame with the smallest error among the global positioning results in the sub-map may be obtained as the second constraint item.

[0085] In one implementation, a global positioning result may be obtained based on a GNSS signal.

[0086] Specifically, in urban internal roads, GNSS signals are often unstable and have poor confidence, and in some internal road scenes (such as underground parking lots, etc.), GNSS signals may not exist. Therefore, the second constraint item can be added only for the submap with GNSS signals. By obtaining the covariance of the GNSS signal, the frame with the smallest covariance of the GNSS signal can be selected as the second constraint item of the submap to provide a global positioning result for the submap.

[0087] Step S104: According to the map optimization result, the sub-maps are stitched together to obtain a global map of the area to be tested.

[0088] In this embodiment, the sub-maps may be spliced ​​based on the graph optimization result to obtain a global map of the area to be tested.

[0089] In one implementation, the graph optimization result may be the optimized pose of the anchor frame in the submap. Based on the optimized pose of the anchor frame and the relative pose between the key frames within the submap, the optimized pose of all key frames may be obtained, thereby achieving pose optimization of all key frames in the submap.

[0090] Based on the methods described in steps S101 to S104 above, the embodiment of the present application collects static map data of the area to be tested for multiple times, and obtains the corresponding single-trip global map according to the map data collection results obtained in each trip; divides each single-trip global map to obtain multiple continuous sub-maps of each single-trip global map; constructs a factor graph based on all the obtained sub-maps, and optimizes the sub-maps with the first constraint item and the second constraint item as factor constraints to obtain the graph optimization result; the first constraint item is the constraint item between the sub-maps obtained according to the overlap between the sub-maps; the second constraint item is the constraint item obtained according to the global positioning result corresponding to the sub-map; according to the graph optimization result, the sub-maps are spliced ​​to obtain the global map of the area to be tested. Through the above configuration, the embodiment of the present application uses the sub-map as the minimum optimization unit of the factor graph, and can maintain the relative relationship within the sub-map unchanged. At the same time, the first constraint item between the submaps is obtained according to the overlap between the submaps as a factor constraint of the factor graph, which can reduce the data difference between multiple single-trip global maps; the second constraint item is obtained based on the global positioning result corresponding to the submap as a factor constraint of the factor graph, which can effectively solve the problem of global positioning jump in scenes such as urban canyons. The global map acquisition method based on the embodiment of the present application obtains a global map, which can ensure the consistency of the global map acquisition process while ensuring globality, and ensure that a global map with global consistency can be obtained in areas not covered by high-precision maps, thereby realizing the connection with high-precision maps, so that automatic driving can achieve smooth transition when entering internal roads from urban areas, and can still achieve lane-level positioning and navigation.

[0091] Step S101, step S103 and step S104 are further described below.

[0092] In one implementation of the present application, see the attached Figure 6 , Figure 6 is a schematic diagram of the main steps of step S101 according to an implementation of an embodiment of the present application, such as Figure 6 As shown, step S101 may further include the following steps S1011 and S1012:

[0093] Step S1011: for each map data collection result, perform local map registration according to each frame of data in the map data collection result to obtain a local map registration result.

[0094] In this embodiment, for each trip of map data collection, the map data can be preprocessed and time-synchronized first. Based on the type of sensor for map data collection, the corresponding mileage calculation method is selected to achieve feature extraction of the map data collection results and feature association with the local map, so as to achieve registration of each frame of map data collection results to the local map and obtain the local map registration result. The mileage calculation method commonly used in this field can be applied to achieve local map registration. For example, for the map data collection results collected by lidar, mileage calculation methods such as ICP (Iterative Closest Point) and NDT (Normal Distributions Transform) algorithms can be selected. For the map data collection results collected by the camera, mileage calculation methods such as feature point method and direct method can be used.

[0095] Step S1012: Obtain a single-trip global map according to the global positioning result and the local map registration result corresponding to the map data collection result.

[0096] In this embodiment, the global positioning result corresponding to the map data collection result can be used as a global positioning factor, the local map registration result can be optimized, and the local map registration result can be converted to a global coordinate system to obtain a single-trip global map.

[0097] In one implementation, a global positioning result of the map collection data may be obtained based on GNSS signals.

[0098] In one embodiment, continue to refer to the attached Figure 6 ,like Figure 6 As shown, step S101 may further include the following steps S1013 to S1015:

[0099] Step S1013: for each single-trip global map, perform key frame screening on the single-trip global map to obtain the key frames of the single-trip global map.

[0100] In this implementation, spatial clustering may be performed on a single-pass global map, and one frame within a preset distance may be retained as a key frame.

[0101] In a specific example, the preset distance is 3 meters. Those skilled in the art can also set the value of the preset distance according to the needs of actual applications.

[0102] Step S1014: Perform loop detection on all key frames of the single-pass global map to obtain loop detection results.

[0103] In this embodiment, the corresponding loop detection algorithm can be selected according to the type and distribution of sensors for map data collection, loop detection can be performed on all key frames of the single-pass global map, loop pairs can be screened, and the relative pose of the loop pairs can be calculated as the loop detection results. For example, for the map data collection results collected by the camera, the bag-of-words model can be applied for loop detection; for the map data collection results collected by the lidar, M2DP (multiview 2Dprojection) can be applied for loop detection. Of course, other loop detection algorithms commonly used in the field can also be used for loop detection, which are all within the scope of protection of this application.

[0104] It should be noted that the current loop detection algorithm needs to achieve a balance between precision and recall rate. At the same time, precision is more important in the process of backend optimization. Once a false loop is introduced, it will greatly affect the effect of backend optimization. Therefore, the loop detection of the embodiment of the present application needs to ensure higher precision, so the recall rate of the loop detection is low at this time.

[0105] Step S1015: According to the loop detection result, back-end optimization is performed on the key frames of the single-trip global map to obtain the single-trip global map after back-end optimization.

[0106] In this embodiment, loop detection results can be received and back-end optimization can be performed on key frames of a single-pass global map, thereby achieving closed-loop optimization and reducing the posture errors between multiple single-pass global maps.

[0107] In one implementation according to the embodiment of the present application, see the attached Figure 7 , Figure 7 FIG. 1 is a flow chart of the main steps of obtaining the first constraint item according to an implementation method of an embodiment of the present application. Figure 7 As shown, the first constraint item can be obtained according to the following steps S201 to S203:

[0108] Step S201: According to the sub-map, a voxelized map of the sub-map is obtained.

[0109] In this embodiment, step S201 may further include the following steps S2011 and S2012:

[0110] Step S2011: stitching the sub-maps according to the positions and postures of all key frames in all sub-maps to obtain a local map.

[0111] In this embodiment, the submaps can be spliced ​​according to the positions between all keyframes in the submap and the point clouds of the keyframes to obtain a local map. The point clouds of the keyframes can be obtained based on sensor acquisition or converted based on data collected by the sensor.

[0112] Step S2012: voxelize the local map to obtain a voxelized map of the sub-map.

[0113] In this embodiment, the local map may be voxelized to obtain a voxelized map of each sub-map.

[0114] In one implementation, a hash table may be constructed to store the voxelized maps of the sub-maps.

[0115] Step S202: for each voxelized map corresponding to two sub-maps, obtain the overlap between the two voxelized maps.

[0116] In this embodiment, the overlap between voxelized maps corresponding to every two sub-maps may be obtained.

[0117] In one implementation, the intersection-over-union ratio between the voxelized maps corresponding to every two sub-maps may be calculated as the overlap between the two sub-maps.

[0118] Step S203: According to the degree of overlap, the relative position between the two sub-maps is obtained as the first constraint item.

[0119] In this implementation, step S203 may further include the following steps S2031 and S2032:

[0120] Step S2031: Compare the overlap degree with a preset overlap degree threshold.

[0121] In this embodiment, an overlap threshold may be set, and the overlap between submaps may be compared with the overlap threshold. Those skilled in the art may set the value of the overlap threshold according to actual application requirements.

[0122] Step S2032: For two sub-maps whose overlap is greater than the overlap threshold, the relative position between the two sub-maps is obtained according to the anchor frames of the sub-maps.

[0123] In this embodiment, for two submaps with an overlap greater than an overlap threshold, the two submaps can be transformed to the corresponding anchor frames, and the submaps can be aligned to obtain the relative pose between the anchor frames as the relative pose between the two submaps. The relative pose is used as the first constraint item between the factor nodes corresponding to the two submaps. Since closed-loop optimization is achieved by relying on loop detection results during the back-end optimization of a single-trip global map, a map with good consistency has been obtained, but the single-trip global map after back-end optimization and the global position still have a large deviation. Therefore, the overlap between the single-trip global maps after back-end optimization can be obtained, the submaps that overlap in space can be aligned, the relative pose can be calculated, and the loop pairs with an overlap greater than the overlap threshold can be recalled as implicit loops, thereby making up for the problem of insufficient recall rate in the aforementioned loop detection.

[0124] The sub-map registration process may include extracting features from two sub-maps and then correlating the features to achieve sub-map registration.

[0125] In one implementation of the embodiment of the present application, the factor constraint also includes a third constraint item and a fourth constraint item; the third constraint item is a constraint item between submaps obtained according to the loop detection result of a single-pass global map; the fourth constraint item is a constraint item obtained according to the relative posture between submaps during the process of obtaining a single-pass global map. Step S103 can be further configured as follows:

[0126] A factor graph is constructed based on all the obtained sub-maps, and the first constraint item, the second constraint item, the third constraint item and the fourth constraint item are used as factor constraints to construct the factor graph, perform graph optimization on the sub-map, optimize the position of the sub-map, and obtain the graph optimization result.

[0127] In one implementation, the anchor frame of each sub-map may be obtained, the anchor frame may be used as a node of the factor graph, and the first constraint item, the second constraint item, the third constraint item, and the fourth constraint item may be used as factor constraints to construct the factor graph.

[0128] In one embodiment, during the process of loop closure detection on key frames of a single-pass global map, for key frames with loop closure detection results, the loop closure detection results are converted to the anchor frames corresponding to the key frames according to the relative pose between the key frames and the anchor frames corresponding to the key frames, as the third constraint item.

[0129] In one implementation, based on the single-pass global map, relative positions between anchor frames of adjacent sub-maps in the single-pass global map are obtained as the fourth constraint item of the adjacent sub-maps.

[0130] For details, please refer to the attached Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a factor graph constructed based on a submap according to an implementation of an embodiment of the present application. Figure 4 As shown, the implicit loop factor is the first constraint 1, the global positioning factor is the second constraint 2, the explicit loop factor is the third constraint 3, and the odometer factor is the fourth constraint 4. S1, S2, S3, ..., Sk, Sk+1, Sk+2, Sk+3, ..., Sp are factor graph nodes, and data 1, ..., data n are single-trip global maps. Each submap is used as the minimum optimization unit for graph optimization, and the anchor frame is used as the optimization state quantity of the submap, so that the number of factor nodes in the factor graph is reduced from the key frame to the submap level, and the amount of calculation is smaller. In addition, during the graph optimization process, adding explicit loop factors, global positioning factors, odometer factors, and implicit loop factors can achieve efficient optimization of submaps. The anchor frame ensures that the submap is a rigid body and its internal consistency does not change through the relative posture between the anchor frame and other key frames in the submap. The global position of the entire submap is adjusted through the global positioning factor, and the consistency constraints between submaps are further strengthened through explicit loop factors and implicit loop factors, and the consistency constraints between adjacent submaps are strengthened through the odometer factor. Therefore, through the above factor graph, in the process of optimizing the submap nodes, not only the consistency within the submap and the consistency between submaps can be ensured, but also the global position of the submap can be adjusted to achieve an efficient map optimization process.

[0131] In one implementation of the present application, see the attached Figure 8 , Figure 8 FIG. 1 is a flow chart of the main steps of step S104 according to an implementation of an embodiment of the present application. Figure 8 As shown, step S104 may further include the following steps S1041 to S1043:

[0132] Step S1041: For each sub-map, according to the optimized pose of the anchor frame and the relative pose between the anchor frame and other key frames in the sub-map, the optimized poses of other key frames are obtained.

[0133] In this embodiment, the optimized poses of other key frames in the submap can be obtained according to the optimized poses of the anchor frames of the submap in the graph optimization result and the relative poses between the anchor frames and other key frames in the submap.

[0134] Step S1042: Optimize the poses of all key frames in the sub-map according to the optimized poses.

[0135] In this implementation, the poses of all key frames in the submap may be optimized according to the optimized poses, and the positions of the key frames may be updated.

[0136] Step S1043: performing map stitching according to the optimized positions and postures of all key frames to obtain a global map of the area to be measured.

[0137] In this embodiment, map stitching can be achieved based on the optimized key frame positions, thereby obtaining a global map of the area to be measured.

[0138] For details, please refer to the attached Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a global map comparison effect obtained by a global map acquisition method according to an example of an embodiment of the present application. Figure 5 As shown, Figure 5 The scattered points indicated by number 5 are the global positioning results. It can be seen that the global positioning results have abnormal jumps and the differences between different data are large. Figure 5 The number 6 in the middle shows the global map after back-end optimization of only a single-trip global map. It can be seen from the figure that the problems of data jumps and large differences between different data have not been solved, and the global positioning result destroys the consistency between single-trip global maps. Figure 5 The global map obtained by the global map acquisition method according to the embodiment of the present application is shown in the middle label 7, which can realize the fusion of global positioning between different single-trip global maps, and is robust to abnormal jumps in global positioning results. In the process of adjusting the single-trip global map, the consistency between each single-trip global map and within the single-trip global map is ensured, thereby outputting a global map of the area to be tested without ghosting. The global map of the area to be tested obtained by the global map acquisition method of the embodiment of the present application can have a global error within the meter level, and can be connected with a high-precision map, so as to achieve smooth switching when the automatic driving is from the urban area to the internal road, and can still achieve lane-level positioning and navigation on the internal road.

[0139] In one embodiment, please refer to the attached Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a main system block diagram of a global map acquisition method according to an implementation method of an embodiment of the present application. Figure 2As shown, a single-trip global map of each trip map data collection result can be made based on the multi-trip map data collection results (data 1, data 2, ..., data n) of the area to be tested. For the obtained single-trip global map, key frames can be screened, and loop detection can be performed on all single-trip global maps. The single-trip global map is optimized based on the loop detection results. Submaps are extracted from the single-trip global map after back-end optimization, and the overlap between submaps is calculated. Submaps with overlap greater than the overlap threshold are registered to obtain implicit loop factors. The anchor frames of the submaps are used as factor nodes, and the explicit loop factors, implicit loop factors, global positioning factors, and odometer factors in the loop detection results are used as factor constraints to perform graph optimization. Based on the graph optimization results, pose propagation is performed in the submaps to obtain the optimized pose of each map. Map splicing is performed based on the optimized pose to obtain the global map of the area to be tested.

[0140] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0141] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0142] Another aspect of the present application also provides a computer-readable storage medium.

[0143] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the global map acquisition method of the above method embodiment, and the program may be loaded and run by a processor to implement the above global map acquisition method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0144] Another aspect of the present application also provides a smart device.

[0145] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. The intelligent device described in the present application may include a driving device, a smart car, a robot, and the like. Fig. 9 , Fig. 9 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.

[0146] In some embodiments of the present application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in the present application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments.

[0147] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A method for obtaining a global map, characterized in that: The method comprises: Performing multiple trips of map data collection on the area to be tested, and obtaining a single trip global map corresponding to the map data collection result according to each trip of map data collection result, so as to obtain multiple single trip global maps; Performing map division on each of the single-trip global maps respectively to obtain a plurality of continuous sub-maps of the single-trip global map; A factor graph is constructed according to all the obtained submaps, and the submap is optimized according to the factor graph to optimize the position of the submap and obtain a graph optimization result; the factor node of the factor graph is the submap, and the factor constraint of the factor graph includes a first constraint item and a second constraint item; the first constraint item is a constraint item between the submaps obtained according to the overlap rate between the submaps; the second constraint item is a constraint item obtained according to the global positioning result corresponding to the submap; According to the map optimization result, the sub-maps are stitched together to obtain a global map of the area to be tested; The method comprises obtaining the first constraint item according to the following steps: According to the sub-map, obtaining a voxelized map of the sub-map; For each voxelized map corresponding to each two of the sub-maps, obtaining the overlap between the two voxelized maps; Compare the overlap with a preset overlap threshold; for two submaps whose overlap is greater than the overlap threshold, transform the two submaps to the anchor frames of the submaps, perform submap registration, and obtain the relative position between the anchor frames of the two submaps as the first constraint item; The anchor frame is one of the key frames of the sub-map.

2. The global map acquisition method according to claim 1, characterized in that: The step of obtaining a voxelized map of the sub-map according to the sub-map includes: Splicing the submaps according to the positions of all key frames in all submaps to obtain a local map; The local map is voxelized to obtain a voxelized map of the sub-map.

3. The global map acquisition method according to claim 1, characterized in that: The method comprises obtaining the second constraint item according to the following steps: For the sub-map having the global positioning result, the global positioning result of a frame with the smallest global positioning result error in the sub-map is obtained as the second constraint item.

4. The global map acquisition method according to claim 1, characterized in that: The factor constraint also includes a third constraint item and a fourth constraint item; the third constraint item is a constraint item between the sub-maps obtained according to the loop detection result of the single-pass global map; the fourth constraint item is a constraint item obtained according to the relative postures between the sub-maps during the single-pass global map acquisition process; The step of constructing a factor graph according to all the acquired sub-maps, and performing graph optimization on the sub-maps according to the factor graph, optimizing the positions of the sub-maps, and obtaining graph optimization results includes: A factor graph is constructed according to all the obtained sub-maps, and the first constraint item, the second constraint item, the third constraint item and the fourth constraint item are used as factor constraints to construct the factor graph, perform graph optimization on the sub-map, optimize the position and posture of the sub-map, and obtain a graph optimization result.

5. The global map acquisition method according to claim 4, characterized in that: The constructing of the factor graph comprises: Obtaining an anchor frame of each sub-map; the anchor frame is one of the key frames of the sub-map; The factor graph is constructed by taking the anchor frame as the factor node, and taking the first constraint item, the second constraint item, the third constraint item and the fourth constraint item as factor constraints.

6. The global map acquisition method according to claim 5, characterized in that: The method further comprises obtaining the third constraint item according to the following steps: For a key frame with a loop detection result, the loop detection result is converted to the anchor frame corresponding to the key frame according to the relative posture between the key frame and the anchor frame corresponding to the key frame, as the third constraint item.

7. The global map acquisition method according to claim 5, characterized in that: The method further comprises obtaining the fourth constraint item according to the following steps: According to the single-pass global map, relative positions between anchor frames of adjacent sub-maps in the single-pass global map are obtained as the fourth constraint item of the adjacent sub-maps.

8. The global map acquisition method according to claim 1, characterized in that: The graph optimization result is the optimized pose of the anchor frame of the submap; The step of performing map stitching on the submaps according to the map optimization result to obtain a global map of the area to be tested includes: For each submap, according to the optimized pose of the anchor frame and the relative pose between the anchor frame and other key frames in the submap, the optimized poses of other key frames are obtained; According to the optimized posture, optimizing the postures of all key frames in the sub-map; Map stitching is performed according to the optimized poses of all key frames to obtain a global map of the area to be tested.

9. The global map acquisition method according to claim 1, characterized in that: The step of obtaining a single-trip global map corresponding to the map data collection result according to each trip of the map data collection result comprises: For each trip of the map data collection result, performing local map registration according to each frame of data in the map data collection result to obtain a local map registration result; The single-trip global map is obtained according to the global positioning result corresponding to the map data collection result and the local map registration result.

10. The global map acquisition method according to claim 9, characterized in that: The method further comprises performing back-end optimization on the single-trip global map according to the following steps: For each single-trip global map, performing key frame screening on the single-trip global map to obtain the key frame of the single-trip global map; Perform loop detection on all key frames of the single-pass global map and obtain loop detection results; According to the loop detection result, backend optimization is performed on the key frames of the single-trip global map to obtain the single-trip global map after backend optimization.

11. The global map acquisition method according to claim 1, characterized in that: The map division is performed on each of the single-trip global maps to obtain a plurality of continuous sub-maps of the single-trip global map, including: For each of the single-pass global maps, dividing the single-pass global map based on a preset number of frames to obtain a plurality of continuous sub-maps; The relative positions between the key frames in the sub-map remain fixed.

12. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the global map acquisition method according to any one of claims 1 to 11 is implemented.

13. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the global map acquisition method according to any one of claims 1 to 11.

Images