A method, device and terminal device for map construction

Abstract

This application is applicable to the field of localization and mapping technology, and provides a map construction method, device and terminal device. The method includes: respectively determining a first point cloud map corresponding to a first area and second point cloud maps corresponding to at least one area to be stitched; determining and optimizing the initial pose data of the origin in the second point cloud map in the first point cloud map to obtain initial optimized pose data; performing data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map; performing global optimization on the first point cloud map and the converted second point cloud map, and stitching them to obtain a global point cloud map. This application realizes the offline correction of the point cloud map of a location and the stitching of multiple point cloud maps based on the key frame data and positioning data of the pre-constructed point cloud map, improving the accuracy and efficiency of the offline correction and expansion of the point cloud map of a relatively large area.

Description

Technical Field

[0001] This application belongs to the technical field of localization and mapping, and particularly relates to a map construction method, device, and terminal device. Background Art

[0002] In an unmanned driving system, a globally consistent point cloud map based on the external environment is the most important prior information.

[0003] Related localization and mapping methods are usually multi-sensor fusion map construction methods. The above methods have the problems of low map construction success rate or large error between the constructed map and the real environment for some scenarios (such as tunnels, long corridors, etc.); moreover, when new driving routes need to be expanded, map construction needs to be repeated, with a large amount of calculation, low map construction efficiency, and unstable accuracy and low quality of the constructed map. Summary of the Invention

[0004] Embodiments of this application provide a map construction method, device, and terminal device, which can solve the problems of large calculation amount, low map construction efficiency, unstable accuracy of the constructed map, and low quality existing in related methods.

[0005] In a first aspect, embodiments of this application provide a map construction method, including:

[0006] Determine a first point cloud map corresponding to a first area and second point cloud maps corresponding to at least one area to be stitched respectively;

[0007] Determine the initial pose data of the origin in the first point cloud map of the second point cloud map;

[0008] Optimize the initial pose data to obtain initial optimized pose data;

[0009] Perform data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map;

[0010] Perform global optimization on the first point cloud map and the converted second point cloud map, and stitch them to obtain a global point cloud map.

[0011] In one embodiment, determining a first point cloud map corresponding to a first area includes:

[0012] Determine a pre-constructed first initial point cloud map corresponding to the first area;

[0013] When it is detected that there is an abnormality in the first initial point cloud map, determine first key frame data to be corrected;

[0014] Performing point cloud registration based on the first key frame data to be corrected to obtain a first point cloud registration result, and performing global optimization based on the first point cloud registration result to obtain the first point cloud map.

[0015] In one embodiment, determining a second point cloud map corresponding to a second region includes:

[0016] Respectively determining pre-constructed second initial point cloud maps corresponding to each of the second regions;

[0017] When it is detected that there is an abnormality in the second initial point cloud map, determining second key frame data to be corrected;

[0018] Performing point cloud registration based on the second key frame data to be corrected to obtain a second point cloud registration result, and performing global optimization based on the second point cloud registration result to obtain the second point cloud map.

[0019] In one embodiment, the data conversion of the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map includes:

[0020] Based on the initial optimized pose data, converting the second key frame data in the second point cloud map to the coordinate system of the first point cloud map to obtain a converted second point cloud map.

[0021] In one embodiment, the optimization of the initial pose data to obtain initial optimized pose data includes:

[0022] Optimizing the initial pose data based on a first preset algorithm to obtain optimized initial pose data; wherein, the first preset algorithm includes a high-low resolution algorithm;

[0023] Performing iterative optimization on the optimized initial pose data based on a second preset algorithm to obtain initial optimized pose data; wherein, the second preset algorithm includes a normal distribution transformation algorithm and an iterative closest point algorithm.

[0024] In one embodiment, the determination of the initial pose data of the origin in the first point cloud map in the second point cloud map includes:

[0025] Determining first key frame data and first positioning data in the first point cloud map;

[0026] Determining second key frame data and second positioning data in the second point cloud map; wherein, the second positioning data includes origin positioning data;

[0027] Selecting target key frame data that meets a preset condition according to the origin positioning data;

[0028] Based on the first positioning data, the first key frame data, the second positioning data, and the target key frame data, calculate and determine the initial pose data of the origin in the first point cloud map under the second point cloud map.

[0029] In one embodiment, the globally optimizing the first point cloud map and the transformed second point cloud map and stitching them to obtain a global point cloud map includes:

[0030] Determine the third key frame data to be corrected in the first point cloud map and the second point cloud map;

[0031] Perform point cloud registration on the third key frame data to be corrected, and globally optimize the first point cloud map and the second point cloud map based on the point cloud registration result to obtain an optimized first point cloud map and an optimized second point cloud map;

[0032] Stitch the optimized first point cloud map and the optimized second point cloud map to obtain a global point cloud map.

[0033] In a second aspect, an embodiment of the present application provides a map construction device, including:

[0034] A map determination module, configured to respectively determine a first point cloud map corresponding to a first region and a second point cloud map corresponding to at least one region to be stitched;

[0035] A data determination module, configured to determine the initial pose data of the origin in the first point cloud map in the second point cloud map;

[0036] A pose optimization module, configured to optimize the initial pose data to obtain initial optimized pose data;

[0037] A data conversion module, configured to perform data conversion on the second point cloud map based on the initial optimized pose data to obtain a transformed second point cloud map;

[0038] A map stitching module, configured to globally optimize the first point cloud map and the transformed second point cloud map and stitch them to obtain a global point cloud map.

[0039] In one embodiment, the map determination module includes:

[0040] A first data acquisition unit, configured to determine a pre-constructed first initial point cloud map corresponding to the first region;

[0041] A first data detection unit, configured to determine first key frame data to be corrected when detecting that the first initial point cloud map is abnormal;

[0042] The first map correction unit is configured to perform point cloud registration based on the first key frame data to be corrected to obtain a first point cloud registration result, and perform global optimization based on the first point cloud registration result to obtain the first point cloud map.

[0043] In one embodiment, the map determination module further includes:

[0044] The second data acquisition unit is configured to respectively determine pre-constructed second initial point cloud maps corresponding to each of the second regions;

[0045] The second data detection unit is configured to determine second key frame data to be corrected when detecting an abnormality in the second initial point cloud map;

[0046] The second map correction unit is configured to perform point cloud registration based on the second key frame data to be corrected to obtain a second point cloud registration result, and perform global optimization based on the second point cloud registration result to obtain the second point cloud map.

[0047] In one embodiment, the data conversion module is specifically configured to:

[0048] Based on the initial optimized pose data, convert the second key frame data in the second point cloud map to the coordinate system of the first point cloud map to obtain a converted second point cloud map.

[0049] In one embodiment, the pose optimization module includes:

[0050] The first pose optimization unit is configured to optimize the initial pose data based on a first preset algorithm to obtain optimized initial pose data; wherein, the first preset algorithm includes a high-low resolution algorithm;

[0051] The second pose optimization unit is configured to iteratively optimize the optimized initial pose data based on a second preset algorithm to obtain initial optimized pose data; wherein, the second preset algorithm includes a normal distribution transformation algorithm and an iterative closest point algorithm.

[0052] In one embodiment, the data determination module includes:

[0053] The first data determination unit is configured to determine first key frame data and first positioning data in the first point cloud map;

[0054] The second data determination unit is configured to determine second key frame data and second positioning data in the second point cloud map; wherein, the second positioning data includes origin positioning data;

[0055] The first data selection unit is configured to select target key frame data that meets a preset condition according to the origin positioning data;

[0056] A calculation unit, configured to calculate and determine initial pose data of the origin in the first point cloud map in the second point cloud map based on the first positioning data, the first key frame data, the second positioning data, and the target key frame data.

[0057] In one embodiment, the map stitching module includes:

[0058] A second data selection unit, configured to determine third key frames to be corrected in the first point cloud map and the second point cloud map;

[0059] A map optimization unit, configured to perform point cloud registration on the third key frames to be corrected, and perform global optimization on the first point cloud map and the second point cloud map based on the point cloud registration result, to obtain an optimized first point cloud map and an optimized second point cloud map;

[0060] A map stitching unit, configured to stitch the optimized first point cloud map and the optimized second point cloud map to obtain a global point cloud map.

[0061] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the map construction method according to any one of the above first aspects is implemented.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, storing a computer program, where when the computer program is executed by a processor, the map construction method according to any one of the above first aspects is implemented.

[0063] In a fifth aspect, an embodiment of the present application provides a computer program product, which when running on a terminal device, causes the terminal device to execute the map construction method according to any one of the above first aspects.

[0064] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: By using the first point cloud map corresponding to the first area and the second point cloud map corresponding to at least one area to be stitched, the initial pose data of the origin in the first point cloud map in the second point cloud map is determined and optimized to obtain initial optimized pose data. Based on the initial optimized pose data, data conversion is performed on the second point cloud map to obtain a converted second point cloud map. Global optimization is performed on the first point cloud map and the converted second point cloud map, and stitching is performed to obtain a global point cloud map, realizing offline correction of the point cloud map and stitching of multiple point cloud maps, and improving the accuracy and efficiency of offline correction and amplification of the point cloud map of a larger area.

[0065] It should be understood that the beneficial effects of the second to fifth aspects described above can be referred to the relevant descriptions in the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0067] Figure 1 is a schematic flowchart of the map construction method provided by the embodiment of the present application;

[0068] Figure 2 is a schematic flowchart of step S102 of the map construction method provided by the embodiment of the present application;

[0069] Figure 3 is a schematic flowchart of step S105 of the map construction method provided by the embodiment of the present application;

[0070] Figure 4 is a schematic structural diagram of the map construction device provided by the embodiment of the present application;

[0071] Figure 5 is a schematic structural diagram of the terminal device provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0073] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0074] It should also be understood that the term " / and" as used in the specification and the appended claims of the present application refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0075] As used in the description of the present application and the appended claims, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0076] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0077] The reference to "one embodiment" or "some embodiments" or the like described in the description of the present application means that a specific feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0078] The map construction method provided by the embodiments of the present application can be applied to terminal devices such as mobile service robots, autonomous vehicles, mobile phones, tablet computers, in-vehicle devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0079] Figure 1 A schematic flowchart of the map construction method provided by the present application is shown. By way of example and not limitation, this method can be applied to the in-vehicle device described above.

[0080] S101: respectively determine a first point cloud map corresponding to a first region and second point cloud maps corresponding to at least one region to be stitched.

[0081] Specifically, when a vehicle based on an unmanned driving system is in motion, it is necessary to determine the current position information of the vehicle based on a globally consistent point cloud map constructed in advance. When the driving range of the vehicle is large, more regional maps need to be constructed correspondingly. It is necessary to perform global consistency processing on multiple constructed maps, or when the vehicle travels to an area where no map has been built, it is necessary to build a map for the area where no map has been built and perform global consistency optimization with the pre-constructed map to facilitate the vehicle to position and travel based on the globally consistent point cloud map. First, it is necessary to determine a first point cloud map corresponding to a first area, and determine a second point cloud map corresponding to at least one other area to be stitched. Among them, the first area can be a pre-constructed regional point cloud map used as a stitching reference. For example: the first area is the central area within a larger area.

[0082] For example, if the city center of City A is Area a, then the pre-constructed point cloud map of Area a in City A is used as the first point cloud map. It is necessary to obtain the second point cloud maps of Area b, Area c, and Area d in City A respectively, and perform global consistency optimization on the point cloud maps of Area a, Area b, Area c, and Area d based on the point cloud data and GPS positioning data of the point cloud map of Area a, and stitch them to obtain the globally consistent map of City A to facilitate the positioning and travel of the unmanned vehicle.

[0083] S102. Determine the initial pose data of the origin in the first point cloud map in the second point cloud map.

[0084] Specifically, based on the first positioning data of the first point cloud map and the second positioning data of the second point cloud map, calculate and determine the initial pose data of the origin of the second point cloud map in the coordinate system of the first point cloud map.

[0085] S103. Optimize the initial pose data to obtain the initial optimized pose data.

[0086] Specifically, optimize the initial pose data through a first preset algorithm and a second preset algorithm to obtain the initial optimized pose data of the origin in the second point cloud map in the coordinate system of the first point cloud map.

[0087] S104. Perform data conversion on the second point cloud map based on the initial optimized pose data to obtain the converted second point cloud map.

[0088] In one embodiment, the step S104 of performing data conversion on the second point cloud map based on the initial optimized pose data to obtain the converted second point cloud map includes:

[0089] Based on the initial optimized pose data, convert the second key frame data in the second point cloud map to the coordinate system of the first point cloud map to obtain the converted second point cloud map.

[0090] Specifically, based on the second key frame data and the initial optimized pose data in the second point cloud map, all the key frame data (specifically, the key frame pose data) of the second point cloud map is converted into key frame conversion data in the coordinate system of the first point cloud map.

[0091] S105. Perform global optimization on the first point cloud map and the converted second point cloud map, and splice them to obtain a global point cloud map.

[0092] Specifically, based on the first key frame data in the first point cloud map and the key frame conversion data in the second point cloud map, third key frame data to be corrected is selected, and global optimization processing is performed on the first point cloud map and the second point cloud map based on the third key frame data to be corrected, and the first point cloud map and the second point cloud map after global optimization are spliced to obtain a global point cloud map.

[0093] In one embodiment, determining the first point cloud map corresponding to the first region includes:

[0094] Determine a pre-constructed first initial point cloud map corresponding to the first region;

[0095] When it is detected that there is an abnormality in the first initial point cloud map, determine the first key frame data to be corrected;

[0096] Perform point cloud registration based on the first key frame data to be corrected to obtain a first point cloud registration result, and perform global optimization based on the first point cloud registration result to obtain the first point cloud map.

[0097] Specifically, determine a pre-constructed first initial point cloud map corresponding to the first region. The first initial point cloud map carries all the key frame pose data and corresponding key frame point cloud data corresponding to the first region, as well as the GPS positioning data of the origin in the coordinate system of the first point cloud map. Identify and determine whether there is an abnormal scene in the first initial point cloud map (for example: whether there is obvious scene overlap, or whether there are inconsistent adjacent scenes, such as adjacent scenes not being on the same plane). When it is detected that there is an abnormal scene in the first initial point cloud map, determine the first key frame data to be corrected, perform point cloud registration on the first key frame data to be corrected in the abnormal scene to obtain a corresponding first point cloud registration result, and add the first point cloud registration result as a constraint condition to the global optimizer for global optimization to obtain the first point cloud map corresponding to the first region and including the corrected first key frame data.

[0098] In one embodiment, determining the second point cloud map corresponding to the second region includes:

[0099] Respectively determine pre-constructed second initial point cloud maps corresponding to each second region;

[0100] When an anomaly is detected in the second initial point cloud map, determine the second key frame data to be corrected;

[0101] Perform point cloud registration based on the second key frame data to be corrected to obtain a second point cloud registration result, and perform global optimization based on the second point cloud registration result to obtain the second point cloud map.

[0102] Specifically, respectively determine the pre-constructed second initial point cloud maps corresponding to each second region. The second initial point cloud maps carry all the key frame pose data and the corresponding key frame point cloud data corresponding to the regions to be stitched, as well as the GPS positioning data of the origin in the coordinate system of the second point cloud map. Identify and determine whether there is an abnormal scene in the second initial point cloud map. When an abnormal scene is detected in the second initial point cloud map, determine the second key frame data to be corrected, perform point cloud registration on the second key frame data in the abnormal scene to obtain the corresponding second point cloud registration result, and add the second point cloud registration result as a constraint condition to the global optimizer for global optimization to obtain the second point cloud map corresponding to the second region and containing the corrected second key frame data.

[0103] In one embodiment, the optimization of the initial pose data to obtain the initial optimized pose data includes:

[0104] Optimize the initial pose data based on a first preset algorithm to obtain optimized initial pose data; wherein, the first preset algorithm includes a high-low resolution algorithm;

[0105] Iteratively optimize the optimized initial pose data based on a second preset algorithm to obtain the initial optimized pose data; wherein, the second preset algorithm includes a normal distribution transform algorithm and an iterative closest point algorithm.

[0106] Specifically, use the initial pose data of the origin in the second point cloud map in the coordinate system of the first point cloud map as input data, calculate the corresponding initial optimized pose based on the first preset algorithm for the above input data, and loop and input it into the second preset algorithm to perform iterative optimization processing on the above initial optimized pose to obtain the corresponding initial optimized pose. The first preset algorithm includes, but is not limited to, a high-low resolution algorithm; the second preset algorithm includes, but is not limited to, a normal distribution transform algorithm (Normal Distributions Transform, NDT) and an iterative closest point algorithm (Iterative Closest Point, ICP).

[0107] Specifically, due to certain errors in GPS positioning data and the lack of heading angles in some GPS positioning devices, the accuracy of the initially calculated pose data is not high. During the driving process of the vehicle, due to changes in data such as the driving direction and height of the vehicle, there are certain errors in the corresponding pose data in the x, y, and z directions. It is set to calculate the correct initial optimized pose data through a high-low resolution algorithm:

[0108] High-resolution iterative optimization: First, perform iterative search within a first preset range corresponding to the position of the initial pose data (for example, N meters away from the initial pose data, where N is a positive integer greater than 1). After obtaining new pose data each time, transform the point cloud data of the current frame into the global coordinate system based on the new pose data, and perform point cloud registration between the transformed current-frame point cloud data and the key frame data in the local global point cloud map (i.e., the first map). When it is detected that the point cloud registration result is less than a set first preset registration threshold (the first preset registration threshold can be specifically set according to actual needs), it is determined that the preliminary pose optimization is completed, and the corresponding optimized initial pose data is output.

[0109] Specifically, when it is detected that the point cloud registration result is greater than the first preset registration threshold and less than the second preset registration threshold (the second preset registration threshold is greater than the first preset registration threshold), perform low-resolution iterative optimization: Perform iterative search within a second preset range corresponding to the position of the initial pose data (for example, the second preset range is an area M meters away from the initial pose data, where M is less than N). After obtaining new pose data each time, transform the point cloud data of the current frame into the global coordinate system based on the new pose data, and perform point cloud registration between the transformed current-frame point cloud data and the key frame data in the local global point cloud map (i.e., the first map). When it is detected that the point cloud registration result is less than the set first preset registration threshold, it is determined that the preliminary pose optimization is completed, and the corresponding optimized initial pose data is output.

[0110] Specifically, when it is detected that the point cloud registration result is greater than the second preset registration threshold, search within the first preset range corresponding to the position of the initial pose data to determine a new pose data as the initial pose data, and perform high-resolution iterative optimization.

[0111] As Figure 2 shown, in one embodiment, the step S102 of determining the initial pose data of the origin in the first point cloud map in the second point cloud map includes:

[0112] S1021. Determine the first key frame data and the first positioning data in the first point cloud map;

[0113] S1022. Determine the second key frame data and the second positioning data in the second point cloud map; wherein, the second positioning data includes the origin positioning data;

[0114] S1023. Select target key frame data that meets the preset conditions according to the origin positioning data;

[0115] S1024. Based on the first positioning data, the first key frame data, the second positioning data, and the target key frame data, calculate and determine the initial pose data of the origin in the first point cloud map in the second point cloud map.

[0116] Specifically, determine all the key frame data carried in the first point cloud map (hereinafter referred to as the first key frame data) and the first positioning data, and determine all the key frame data carried in the second point cloud map (hereinafter referred to as the second key frame data) and the second positioning data (the second positioning data includes the origin positioning data of the origin in the second point cloud map); among them, the first positioning data is the GPS positioning data collected based on the GPS positioning device when constructing the first point cloud map; the second positioning data is the GPS positioning data collected based on the GPS positioning device when constructing the second point cloud map. The key frame data includes but is not limited to key frame point cloud data and key frame pose data.

[0117] Specifically, the preset conditions can be specifically set according to actual needs. In this embodiment, the preset condition is set to K second key frame data with a distance less than the preset distance threshold centered on the origin positioning data of the origin in the second point cloud map. That is, based on the origin positioning data of the origin in the second point cloud map as the center, select K second key frame data with a distance less than the preset distance threshold as the target key frame data. Among them, the preset distance threshold can be determined according to factors such as the actual range of the second point cloud map and user needs. The preset distance threshold is in a direct proportional relationship with the growth of the actual range of the second point cloud map. K is a positive integer greater than 1, and the value of K is in a direct proportional relationship with the growth of the preset distance threshold. For example, the actual range of the second point cloud map is 10KM 2 , and the corresponding preset distance threshold is 10m; the actual range of the second point cloud map is 30km 2 , and the corresponding preset distance threshold is 30m.

[0118] Specifically, based on the first positioning data, the first key data, the target key frame data of the first point cloud map, and the second positioning data of the second point cloud map, calculate the initial pose data of the origin in the coordinate system of the first point cloud map in the second point cloud map.

[0119] As Figure 3 shown, in one embodiment, the step S105 of globally optimizing the first point cloud map and the converted second point cloud map and stitching them to obtain a global point cloud map includes:

[0120] S1051. Determine the third key frame data to be corrected in the first point cloud map and the second point cloud map;

[0121] S1052. Perform point cloud registration on the third key frame data to be corrected, and globally optimize the first point cloud map and the second point cloud map based on the point cloud registration result to obtain an optimized first point cloud map and an optimized second point cloud map;

[0122] S1053. Stitch the optimized first point cloud map and the optimized second point cloud map to obtain a global point cloud map.

[0123] Specifically, identify the first point cloud map and the second point cloud map to determine the environmental similarity between the first point cloud map and the second point cloud map. When detecting an area of environmental overlap between the first point cloud map and the second point cloud map (for example, based on 3D point cloud loop detection, the Scancontext algorithm detects the consistency of the first point cloud map and the second point cloud map, and when detecting an area with the same scene in the first point cloud map and the second point cloud map, it is determined as an area of environmental overlap), select the corresponding first key frame data and second key frame data in this area of environmental overlap as the third key frame data to be corrected; perform point cloud registration on the selected third key frame data to be corrected (specifically, register the first key frame data in the area of environmental overlap in the first point cloud map with the second key frame data in the corresponding area of environmental overlap in the second point cloud map), and add the point cloud registration result as a constraint condition to the global optimizer (the global optimizer includes but is not limited to g2o, and by continuously iteratively optimizing the pose data of the first key frame data in the area of environmental overlap and the second key frame data in the area of environmental overlap, the relative position error amount is made close to 0) to obtain an optimized first point cloud map and an optimized second point cloud map globally, and stitch the optimized first point cloud map and the optimized second point cloud map globally to obtain a global point cloud map.

[0124] In this embodiment, through the first point cloud map corresponding to the first area and the second point cloud map corresponding to at least one area to be stitched, the initial pose data of the origin in the second point cloud map in the first point cloud map is determined and optimized to obtain the initial optimized pose data. Based on the initial optimized pose data, data conversion is performed on the second point cloud map to obtain the converted second point cloud map. The first point cloud map and the converted second point cloud map are globally optimized and stitched to obtain a global point cloud map, realizing offline correction of the point cloud map and stitching of multiple point cloud maps, and improving the accuracy and efficiency of offline correction and expansion of the point cloud map for a larger area.

[0125] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0126] Corresponding to the map construction method described in the above embodiments, Figure 4 The structural block diagram of the map construction device provided by the embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiment of the present application are shown.

[0127] Referring to Figure 4 , the map construction device 100 includes:

[0128] A map determination module 101, configured to respectively determine a first point cloud map corresponding to a first area and a second point cloud map corresponding to at least one area to be stitched;

[0129] A data determination module 102, configured to determine the initial pose data of the origin in the first point cloud map in the second point cloud map;

[0130] A pose optimization module 103, configured to optimize the initial pose data to obtain initial optimized pose data;

[0131] A data conversion module 104, configured to perform data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map;

[0132] A map stitching module 105, configured to perform global optimization on the first point cloud map and the converted second point cloud map, and stitch them to obtain a global point cloud map.

[0133] In one embodiment, the map determination module includes:

[0134] A first data acquisition unit, configured to determine a pre-constructed first initial point cloud map corresponding to the first area;

[0135] A first data detection unit, configured to determine first key frame data to be corrected when detecting that there is an abnormality in the first initial point cloud map;

[0136] A first map correction unit, configured to perform point cloud registration based on the first key frame data to be corrected to obtain a first point cloud registration result, and perform global optimization based on the first point cloud registration result to obtain the first point cloud map.

[0137] In one embodiment, the map determination module further includes:

[0138] A second data acquisition unit, configured to respectively determine pre-constructed second initial point cloud maps corresponding to each of the second areas;

[0139] A second data detection unit, configured to determine second key frame data to be corrected when detecting that there is an abnormality in the second initial point cloud map;

[0140] A second map correction unit, configured to perform point cloud registration based on the second key frame data to be corrected to obtain a second point cloud registration result, and perform global optimization based on the second point cloud registration result to obtain the second point cloud map.

[0141] In one embodiment, the pose optimization module includes:

[0142] A first pose optimization unit, configured to optimize the initial pose data based on a first preset algorithm to obtain optimized initial pose data; wherein, the first preset algorithm includes a high-low resolution algorithm;

[0143] A second pose optimization unit, configured to iteratively optimize the optimized initial pose data based on a second preset algorithm to obtain initial optimized pose data; wherein, the second preset algorithm includes a normal distribution transformation algorithm and an iterative closest point algorithm.

[0144] In one embodiment, the data conversion module is specifically configured to:

[0145] Based on the initial optimized pose data, convert the second key frame data in the second point cloud map to the coordinate system of the first point cloud map to obtain a converted second point cloud map.

[0146] In one embodiment, the data determination module includes:

[0147] A first data determination unit, configured to determine first key frame data and first positioning data in the first point cloud map;

[0148] A second data determination unit, configured to determine second key frame data and second positioning data in the second point cloud map; wherein, the second positioning data includes origin positioning data;

[0149] A first data selection unit, configured to select target key frame data that meets a preset condition according to the origin positioning data;

[0150] A calculation unit, configured to calculate and determine the initial pose data of the origin in the second point cloud map in the first point cloud map based on the first positioning data, the first key frame data, the second positioning data, and the target key frame data.

[0151] In one embodiment, the map stitching module includes:

[0152] A second data selection unit, configured to determine third key frame data to be corrected in the first point cloud map and the second point cloud map;

[0153] A map optimization unit, configured to perform point cloud registration on the third key frame data to be corrected, and globally optimize the first point cloud map and the second point cloud map based on the point cloud registration result, so as to obtain an optimized first point cloud map and an optimized second point cloud map;

[0154] A map stitching unit, configured to stitch the optimized first point cloud map and the optimized second point cloud map to obtain a global point cloud map.

[0155] In this embodiment, through the first point cloud map corresponding to the first area and the second point cloud map corresponding to at least one area to be stitched, the initial pose data of the origin in the second point cloud map in the first point cloud map is determined and optimized to obtain the initial optimized pose data. Based on the initial optimized pose data, data conversion is performed on the second point cloud map to obtain a converted second point cloud map. The first point cloud map and the converted second point cloud map are globally optimized and stitched to obtain a global point cloud map, realizing offline correction of the point cloud map and stitching of multiple point cloud maps, and improving the accuracy and efficiency of offline correction and amplification of the point cloud map of a larger area.

[0156] It should be noted that for the information interaction, execution process, etc. between the above-mentioned device / unit, since it is based on the same concept as the method embodiment of the present application, for its specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details are not described herein again.

[0157] Figure 5 It is a schematic structural diagram of the terminal device provided in this embodiment. As Figure 5 shown, the terminal device 5 of this embodiment includes: at least one processor 50 ( Figure 5 only one is shown in the figure), a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50. When the processor 50 executes the computer program 52, the steps in any of the above-mentioned map construction method embodiments are implemented.

[0158] The terminal device 5 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that Figure 5 merely examples of the terminal device 5 are given, and do not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0159] The so-called processor 50 may be a Central Processing Unit (CPU), and the processor 50 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0160] In some embodiments, the memory 51 may be an internal storage unit of the terminal device 5, such as the hard disk or memory of the terminal device 5. In some other embodiments, the memory 51 may also be an external storage device of the terminal device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the terminal device 5. Further, the memory 51 may also include both the internal storage unit and the external storage device of the terminal device 5. The memory 51 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program, etc. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0162] An embodiment of the present application also provides a network device, which includes at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0163] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, which when executed by a processor can implement the steps in the above method embodiments.

[0164] An embodiment of the present application provides a computer program product, which when running on a mobile terminal enables the mobile terminal to implement the steps in the above method embodiments when executed.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above embodiment methods of the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0166] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0167] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0168] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0169] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A method for constructing a map, characterized in that, Including: Respectively determine a first point cloud map corresponding to a first region and second point cloud maps corresponding to at least one region to be stitched; Determine the initial pose data of the origin in the first point cloud map in the second point cloud map, including: determining first key frame data and first positioning data in the first point cloud map; determining second key frame data and second positioning data in the second point cloud map; wherein, the second positioning data includes origin positioning data; select target key frame data that meets a preset condition according to the origin positioning data; based on the first positioning data, the first key frame data, the second positioning data and the target key frame data, calculate and determine the initial pose data of the origin in the first point cloud map in the second point cloud map; Optimize the initial pose data to obtain initial optimized pose data; Perform data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map; Perform global optimization on the first point cloud map and the converted second point cloud map and stitch them to obtain a global point cloud map.

2. The map construction method according to claim 1, characterized in that, Determine a first point cloud map corresponding to a first region, including: Determine a pre-constructed first initial point cloud map corresponding to the first region; When it is detected that there is an abnormality in the first initial point cloud map, determine first key frame data to be corrected; Perform point cloud registration based on the first key frame data to be corrected to obtain a first point cloud registration result, and perform global optimization based on the first point cloud registration result to obtain the first point cloud map.

3. The map construction method according to claim 1, characterized in that, Determine a second point cloud map corresponding to a second region, including: Respectively determine pre-constructed second initial point cloud maps corresponding to each second region; When it is detected that there is an abnormality in the second initial point cloud map, determine second key frame data to be corrected; Perform point cloud registration based on the second key frame data to be corrected to obtain a second point cloud registration result, and perform global optimization based on the second point cloud registration result to obtain the second point cloud map.

4. The map construction method according to claim 1, characterized in that, The performing data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map includes: Based on the initial optimized pose data, convert the second key frame data in the second point cloud map to the coordinate system of the first point cloud map to obtain a converted second point cloud map.

5. The map construction method according to claim 1, wherein The optimizing the initial pose data to obtain initial optimized pose data includes: Optimize the initial pose data based on a first preset algorithm to obtain optimized initial pose data; wherein, the first preset algorithm includes a high-low resolution algorithm; Perform iterative optimization on the optimized initial pose data based on a second preset algorithm to obtain initial optimized pose data; wherein, the second preset algorithm includes a normal distribution transformation algorithm and an iterative closest point algorithm.

6. The map construction method according to claim 1, characterized in that, The performing global optimization on the first point cloud map and the converted second point cloud map and stitching them to obtain a global point cloud map includes: Determine third key frame data to be corrected in the first point cloud map and the second point cloud map; Perform point cloud registration on the third key frame data to be corrected, and perform global optimization on the first point cloud map and the second point cloud map based on the point cloud registration result to obtain an optimized first point cloud map and an optimized second point cloud map; Stitch the optimized first point cloud map and the optimized second point cloud map to obtain a global point cloud map.

7. A map construction device, characterized in that, Including: A map determination module for respectively determining a first point cloud map corresponding to a first region and a second point cloud map corresponding to at least one region to be stitched; A data determination module for determining the initial pose data of the origin in the first point cloud map in the second point cloud map; The data determination module includes: a first data determination unit for determining first key frame data and first positioning data in the first point cloud map; a second data determination unit for determining second key frame data and second positioning data in the second point cloud map; wherein the second positioning data includes origin positioning data; a first data selection unit for selecting target key frame data that meets a preset condition according to the origin positioning data; a calculation unit for calculating and determining the initial pose data of the origin in the first point cloud map in the second point cloud map based on the first positioning data, the first key frame data, the second positioning data, and the target key frame data; A pose optimization module for optimizing the initial pose data to obtain initial optimized pose data; A data conversion module for performing data conversion on the second point cloud map based on the initial optimized pose data to obtain a converted second point cloud map; A map stitching module for performing global optimization on the first point cloud map and the converted second point cloud map and stitching them to obtain a global point cloud map.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1 to 6 is implemented.

Images