Map data management method and device, computer readable storage medium and robot
By using pose clustering and modular splitting of feature point descriptors, the problem of managing massive amounts of map data in traditional VSLAM systems is solved, achieving efficient data management and fast indexing, and adapting to the relocalization needs of complex and large-scale scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UBTECH ROBOTICS CORP LTD
- Filing Date
- 2026-02-26
- Publication Date
- 2026-06-05
Smart Images
Figure CN122156393A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of robotics technology, and in particular relates to a map data management method, apparatus, computer-readable storage medium, and robot. Background Technology
[0002] Visual Simultaneous Localization and Mapping (VSLAM) technology is the core support for robots to achieve autonomous navigation. It collects environmental data through sensors, builds environmental maps in real time, and determines the robot's own pose.
[0003] Traditional VSLAM systems typically employ a monolithic storage architecture, integrating all map data into a single file. In ultra-large-scale scenarios, the number of feature points surges dramatically (up to thousands of feature points in a single frame, and tens of thousands of keyframes in a scene covering millions of square meters), resulting in a dramatic increase in the amount of feature point descriptor data. This leads to extremely large map data volumes (reaching tens or even hundreds of gigabytes), making effective management difficult. Summary of the Invention
[0004] In view of this, embodiments of this application provide a map data management method, apparatus, computer-readable storage medium, and robot to solve the problem of the difficulty in effectively managing large amounts of map data in the prior art.
[0005] A first aspect of this application provides a map data management method, which may include:
[0006] Acquire the pose information of each keyframe in the target map data; wherein, the target map data is the map data for the robot to perform visual synchronous localization and map construction; Based on the pose information, cluster each keyframe in the target map data to obtain keyframe clustering information; Based on the keyframe clustering information, the feature point descriptors of each keyframe in the target map data are split to obtain each feature point descriptor map block. The keyframe clustering information is used as an index for each feature point descriptor map block; wherein, each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map block.
[0007] In one specific implementation of the first aspect, after using the keyframe clustering information as an index for each feature point descriptor map patch, it may further include: During the robot's relocalization process, the global image descriptor of the robot's current frame is extracted; The global image descriptor of the current frame is matched with the global image descriptors of each key frame in the target map data, and several candidate key frames are selected based on the matching results. The first feature point descriptor map block is determined based on the keyframe clustering information, and the first feature point descriptor map block is loaded into memory; wherein, the first feature point descriptor map block is the feature point descriptor map block corresponding to the candidate keyframe; The robot's repositioning pose is determined based on the first feature point descriptor map block and the pose information.
[0008] In one specific implementation of the first aspect, determining the robot's relocalization pose based on the first feature point descriptor map block and the pose information may include: Extract the feature point descriptor of the current frame; The feature point descriptor of the current frame is matched with the feature point descriptors of each key frame in the first feature point descriptor map block to obtain the feature point matching result. The robot's repositioning pose is obtained by calculating the pose based on the feature point matching results and the pose information.
[0009] In one specific implementation of the first aspect, after determining the robot's repositioning pose, the method may further include: Based on the keyframe clustering information, several second feature point descriptor map blocks are determined; wherein, the second feature point descriptor map block is a feature point descriptor map block corresponding to the relocation pose; The second feature point descriptor map block is loaded into memory for use in the robot's localization process.
[0010] In one specific implementation of the first aspect, after loading the second feature point descriptor map block into memory, it may further include: During the robot's localization process, the robot's real-time pose is acquired, and the feature point descriptor map blocks loaded into memory are updated based on the real-time pose.
[0011] In one specific implementation of the first aspect, updating the feature point descriptor map block loaded into memory based on the real-time pose may include: Several third feature point descriptor map blocks are determined based on the keyframe clustering information; wherein, the third feature point descriptor map block is a feature point descriptor map block whose distance from the real-time pose is greater than a preset distance threshold and has been loaded into memory. Unload the third feature point descriptor map block from memory.
[0012] In one specific implementation of the first aspect, updating the feature point descriptor map block loaded into memory based on the real-time pose may include: Several fourth feature point descriptor map blocks are determined based on the keyframe clustering information; wherein, the fourth feature point descriptor map block is a feature point descriptor map block whose distance from the real-time pose is less than a preset distance threshold and which has not been loaded into memory. The fourth feature point descriptor map block is loaded into memory.
[0013] A second aspect of the embodiments of this application provides a map data management device, which may include: The pose information acquisition module is used to acquire the pose information of each keyframe in the target map data; wherein, the target map data is the map data for the robot to perform visual synchronous localization and map construction. The keyframe clustering module is used to cluster each keyframe in the target map data according to the pose information to obtain keyframe clustering information. The map tile splitting module is used to split the feature point descriptors of each key frame in the target map data according to the key frame clustering information to obtain map tiles of each feature point descriptor. The map tile indexing module is used to use the keyframe clustering information as an index for each feature point descriptor map tile; wherein each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map tile.
[0014] In one specific implementation of the second aspect, the map data management device may further include: A global image descriptor extraction module is used to extract the global image descriptor of the current frame of the robot during the robot's relocalization process. The global image descriptor matching module is used to match the global image descriptor of the current frame with the global image descriptors of each key frame in the target map data, and to filter out several candidate key frames based on the matching results. The first map tile loading module is used to determine a first feature point descriptor map tile based on the keyframe clustering information and load the first feature point descriptor map tile into memory; wherein, the first feature point descriptor map tile is a feature point descriptor map tile corresponding to the candidate keyframe. The repositioning pose determination module is used to determine the robot's repositioning pose based on the first feature point descriptor map block and the pose information.
[0015] In one specific implementation of the second aspect, the repositioning pose determination module may be specifically used to: extract the feature point descriptor of the current frame; match the feature point descriptor of the current frame with the feature point descriptors of each key frame in the first feature point descriptor map block to obtain the feature point matching result; and perform pose calculation based on the feature point matching result and the pose information to obtain the repositioning pose of the robot.
[0016] In one specific implementation of the second aspect, the map data management device may further include: The map tile preloading module is used to determine several second feature point descriptor map tiles based on the keyframe clustering information; wherein, the second feature point descriptor map tile is a feature point descriptor map tile corresponding to the relocalization pose; and the second feature point descriptor map tile is loaded into memory for use in the robot's localization process.
[0017] In one specific implementation of the second aspect, the map data management device may further include: The map tile update module is used to acquire the real-time pose of the robot during the robot's localization process and update the feature point descriptor map tiles loaded into memory based on the real-time pose.
[0018] In one specific implementation of the second aspect, the map tile update module may include: The map tile unloading unit is used to determine a number of third feature point descriptor map tiles based on the keyframe clustering information; wherein the third feature point descriptor map tile is a feature point descriptor map tile whose distance from the real-time pose is greater than a preset distance threshold and has been loaded into memory; and unload the third feature point descriptor map tile from memory.
[0019] In one specific implementation of the second aspect, the map tile update module may include: The map tile loading unit is used to determine several fourth feature point descriptor map tiles based on the keyframe clustering information; wherein, the fourth feature point descriptor map tile is a feature point descriptor map tile whose distance from the real-time pose is less than a preset distance threshold and which has not been loaded into memory; and to load the fourth feature point descriptor map tile into memory.
[0020] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described map data management methods.
[0021] A fourth aspect of this application provides a robot including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described map data management methods.
[0022] The fifth aspect of this application provides a computer program product that, when run on a robot, causes the robot to perform the steps of any of the above-described map data management methods.
[0023] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment obtains the pose information of each keyframe in the target map data; wherein, the target map data is map data for robot visual synchronous localization and map construction; the keyframes in the target map data are clustered according to the pose information to obtain keyframe clustering information; the feature point descriptors of each keyframe in the target map data are split according to the keyframe clustering information to obtain each feature point descriptor map block; the keyframe clustering information is used as the index of each feature point descriptor map block; wherein, each keyframe cluster in the keyframe clustering information corresponds to one feature point descriptor map block. In this application embodiment, the modular splitting of feature point descriptors is achieved through pose clustering, splitting the originally large amount of full feature point descriptors into multiple smaller map blocks, and the rapid indexing of map blocks is achieved through keyframe clustering information, thereby enabling more effective management of map data. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of a three-level VSLAM map data system in an embodiment of this application; Figure 2 This is a flowchart of one embodiment of a map data management method according to the present application. Figure 3 This is a flowchart of one embodiment of a robot relocation process in this application. Figure 4 This is a flowchart of one embodiment of a robot localization process in this application. Figure 5 This is a structural diagram of one embodiment of a map data management device according to the present application. Figure 6 This is a schematic block diagram of a robot according to an embodiment of this application. Detailed Implementation
[0026] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0028] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0029] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0030] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."
[0031] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0032] Visual Simultaneous Localization and Mapping (VSLAM) technology is the core support for robots to achieve autonomous navigation. It collects environmental data through sensors, builds environmental maps in real time, and determines the robot's own pose.
[0033] Traditional VSLAM systems typically employ a monolithic storage architecture, integrating all map data into a single file. In ultra-large-scale scenarios, the number of feature points surges dramatically (up to thousands of feature points in a single frame, and tens of thousands of keyframes in a scene covering millions of square meters), resulting in a dramatic increase in the amount of feature point descriptor data. This leads to extremely large map data volumes (reaching tens or even hundreds of gigabytes), making effective management difficult.
[0034] In view of this, embodiments of this application provide a map data management method, apparatus, computer-readable storage medium, and robot to solve the problem of the difficulty in effectively managing large amounts of map data in the prior art.
[0035] The execution subject of this application embodiment can be a robot. The robot realizes the modular splitting of feature point descriptors through pose clustering, splitting the original massive full feature point descriptors into multiple smaller map tiles, and realizes fast indexing of map tiles through keyframe clustering information, thereby enabling more effective management of map data.
[0036] In this embodiment of the application, a system such as [example] can be pre-constructed. Figure 1 The three-level VSLAM map data system shown provides a foundation for subsequent map data management.
[0037] The first level is the keyframe global descriptor file (global_descriptor), which can be stored in HDF5 or other formats. It can include global image descriptors for each keyframe (such as global features based on ResNet), which is used to achieve fast image-level matching and relocation of candidate frames. It has a small data volume and fast matching speed.
[0038] The second level is the keyframe pose and association map file (map_info), which can be stored using Protocol Buffers or other formats. It can include the pose information of each keyframe (3D coordinates, roll angle, pitch angle, yaw angle), the topological association between keyframes, the 3D coordinates of map points, and the correspondence between feature points and keyframes, etc., to provide pose reference for pose clustering and localization.
[0039] The third level is the feature descriptor map information file, which can be stored using Protocol Buffers or other formats. It can include deep learning descriptors of all feature points in each keyframe (such as feature descriptors based on SuperPoint), and is the core data for achieving accurate pose calculation, accounting for more than 90% of the total map data.
[0040] Based on the aforementioned three-level VSLAM map data system, one embodiment of a map data management method in this application may include, for example: Figure 2 The process shown: Step S201: Obtain the pose information of each keyframe in the target map data.
[0041] The target map data is the map data used by the robot for VSLAM, and this target map data adopts the three-level VSLAM map data system mentioned above.
[0042] In this embodiment of the application, the pose information of each keyframe can be extracted from the keyframe pose and associated map file (map_info) of the target map data.
[0043] Step S202: Cluster each keyframe in the target map data according to the pose information to obtain keyframe clustering information.
[0044] In this embodiment, keyframes with similar poses can be grouped into the same keyframe cluster based on a preset clustering algorithm. The specific clustering algorithm used can be flexibly set according to actual conditions, and may include, but is not limited to, density-based clustering algorithms such as Density-Based Spatial Clustering of Applications with Noise (DBSCAN). This embodiment does not specifically limit this. The clustering parameters (such as neighborhood radius, minimum number of keyframes, etc.) can be adaptively adjusted according to the scene scale to ensure that the clustering results conform to the spatial distribution characteristics of the scene.
[0045] After clustering is completed, the results (including but not limited to cluster number, identifier (ID) of keyframes contained in each keyframe cluster, cluster center pose, cluster region range, etc.) can be used as keyframe clustering information and stored in a clustering information map file (clustering_map). This file can be stored using Protocol Buffers or other formats.
[0046] Step S203: Based on the keyframe clustering information, the feature point descriptors of each keyframe in the target map data are split to obtain each feature point descriptor map block.
[0047] For each keyframe cluster, the feature point descriptors of all keyframes in that cluster can be found in the feature point descriptor map information file (feature_descriptor) of the target map data. These feature point descriptors are then extracted from the feature point descriptor map information file (feature_descriptor) and treated as an independent file, namely, a feature point descriptor map block (feature_descriptor_i). Here, i is the cluster index. For example, the feature point descriptor map block corresponding to the first keyframe cluster can be represented as feature_descriptor_1, the feature point descriptor map block corresponding to the second keyframe cluster can be represented as feature_descriptor_2, the feature point descriptor map block corresponding to the third keyframe cluster can be represented as feature_descriptor_3, and so on.
[0048] In this way, the original feature point descriptor map information file (feature_descriptor) can be split into multiple feature point descriptor map blocks with smaller data volumes. Each map block contains only the feature point descriptors of the corresponding clustering region, thereby realizing the modular splitting of feature point descriptors.
[0049] Step S204: Use the keyframe clustering information as the index of each feature point descriptor map block.
[0050] Each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map tile. The keyframe clustering information enables rapid indexing of map tiles, thereby enabling more effective management of map data and providing a basis for rapid map tile search in subsequent relocation and localization processes.
[0051] Because of the modular map storage scheme described above, when local environmental changes occur in a large scene (such as the movement of shelves or the addition of obstacles), only the feature descriptor map block (feature_descriptor_i) of the corresponding cluster area needs to be updated. There is no need to rebuild the entire map, which greatly reduces the cost of map updates and maintenance and effectively improves the practicality and economy of the system.
[0052] In one specific implementation of this application's embodiments, based on the aforementioned modular map storage scheme, the robot can, as shown in... Figure 3 The process shown allows for more efficient relocation: Step S301: During the robot's relocalization process, extract the global image descriptor of the robot's current frame.
[0053] Step S302: Match the global image descriptor of the current frame with the global image descriptors of each key frame in the target map data, and select several candidate key frames based on the matching results.
[0054] In this embodiment, a preset similarity calculation algorithm can be used to match the global image descriptor of the current frame with the global descriptors of all keyframes in the keyframe global descriptor file (global_descriptor), and the N keyframes with the highest similarity can be selected as candidate keyframes. The specific similarity calculation algorithm used can be flexibly set according to actual conditions, and may include, but is not limited to, similarity calculation algorithms based on Euclidean distance, etc. This embodiment does not specifically limit this. The specific value of N can be flexibly set according to actual conditions; for example, it can be set to 5 to 10 or other values, and this embodiment does not specifically limit this.
[0055] Step S303: Determine the first feature point descriptor map block based on the keyframe clustering information, and load the first feature point descriptor map block into memory.
[0056] The first feature point descriptor map block is the feature point descriptor map block corresponding to the candidate keyframe.
[0057] In this embodiment of the application, keyframe clustering information in the clustering information map file (clustering_map) can be loaded. Based on the identifier (ID) of the selected candidate keyframes, the corresponding clustering sequence number and clustering region information are queried to determine the feature point descriptor map block (feature_descriptor_i) to be loaded, i.e. the first feature point descriptor map block, and these first feature point descriptor map blocks are loaded into memory.
[0058] Step S304: Determine the robot's repositioning pose based on the first feature point descriptor map block and pose information.
[0059] In this embodiment, feature point descriptors of the current frame can be extracted, and based on a preset feature point matching algorithm, the feature point descriptors of the current frame can be matched with the feature point descriptors of each keyframe in the first feature point descriptor map tile to obtain the feature point matching result. The specific feature point matching algorithm used can be flexibly set according to the actual situation, and may include, but is not limited to, feature point matching algorithms based on the Fast Library for Approximate Nearest Neighbors (FLANN) matcher, etc. This embodiment does not specifically limit this.
[0060] After obtaining the feature point matching results, the pose information of each keyframe in the keyframe pose and the associated map file (map_info) can be combined to perform pose calculation based on a preset pose calculation algorithm, thereby obtaining the robot's relocalization pose. The specific pose calculation algorithm used can be flexibly set according to the actual situation, and may include, but is not limited to, the Perspective-n-Point (PNP) algorithm, etc. The embodiments of this application do not specifically limit this.
[0061] In the aforementioned relocation process, a three-level relocation mechanism of "global coarse matching - clustering fine localization - feature fine matching" was adopted. First, candidate keyframes were quickly filtered using a small amount of global image descriptors, avoiding the massive computational burden of matching all feature points. Then, the required map tiles were accurately loaded using clustering information, eliminating the need to load all feature point descriptors, thereby reducing the amount of data loading and matching computation during the relocation process. The relocation latency was reduced from several seconds in traditional schemes to hundreds of milliseconds, greatly improving processing efficiency and effectively meeting the real-time requirements of relocation. At the same time, focusing on feature matching of neighboring cluster regions reduced the probability of false matching, greatly improving the relocation success rate and adapting to the relocation needs of complex and large-scale scenarios.
[0062] In one specific implementation of this application embodiment, after determining the robot's repositioning pose, the following can be used in the subsequent positioning process: Figure 4 The map tile management strategy shown is "preload-dynamic update": Step S401: Determine several second feature point descriptor map blocks based on the keyframe clustering information.
[0063] The second feature point descriptor map block is the feature point descriptor map block corresponding to the relocation pose.
[0064] Step S402: Load the second feature point descriptor map block into memory for use in the robot's localization process.
[0065] In this embodiment, based on the repositioning pose, keyframe clustering information in the clustering information map file (clustering_map) can be queried to determine the K clusters with the closest poses and their corresponding K feature point descriptor map blocks (feature_descriptor_i), i.e., the second feature point descriptor map blocks. These second feature point descriptor map blocks are then loaded into memory to complete the preloading process, ensuring the accuracy and stability of the initial positioning. The specific value of K can be flexibly set according to the actual situation; for example, it can be set to 3 to 5 or other values. This embodiment does not specifically limit this value.
[0066] Step S403: During the robot's localization process, acquire the robot's real-time pose and update the feature point descriptor map blocks loaded into memory based on the real-time pose.
[0067] In this embodiment, several third feature point descriptor map blocks can be determined based on keyframe clustering information, and these third feature point descriptor map blocks can be unloaded from memory. The third feature point descriptor map block is a feature point descriptor map block whose distance from the real-time pose is greater than a preset distance threshold and which has already been loaded into memory. The specific value of the distance threshold can be flexibly set according to actual conditions, and this embodiment does not impose specific limitations on it.
[0068] In this embodiment, several fourth feature point descriptor map blocks can be determined based on keyframe clustering information, and these fourth feature point descriptor map blocks can be loaded into memory. The fourth feature point descriptor map blocks are those whose distance from the real-time pose is less than a distance threshold and which are not yet loaded into memory.
[0069] Through this dynamic adaptive map tile loading / unloading positioning memory management strategy, the robot monitors its pose changes in real time during the localization process. When the robot moves away from a certain cluster area, it automatically unloads the feature point descriptor map tile corresponding to that area, releasing memory; when the robot approaches a new cluster area, it automatically loads the feature point descriptor map tile corresponding to that area. This allows for real-time unloading of old map tiles and loading of new ones during robot movement, maintaining only a small number of feature point descriptor map tiles in memory at all times. It ensures sufficient neighboring feature point data to support localization, keeping memory usage stable within a reasonable range and avoiding memory overload caused by loading the entire map. This overcomes the bottleneck of uncontrolled memory usage under ultra-large maps, enabling the robot to maintain stable positioning accuracy even in ultra-large scenes of millions of square meters, thus adapting to the hardware memory limitations of embedded robots.
[0070] The technical solutions in this application do not rely on specific feature extraction or matching algorithms and are compatible with existing mainstream VSLAM algorithms (such as ORB-SLAM3, LIO-SAM, etc.), exhibiting good compatibility and scalability. Simultaneously, the modular map storage and on-demand loading strategy reduces the demand for hardware storage and computing power, making it adaptable to various low-cost embedded robots. This significantly lowers the barrier to entry for large-scale SLAM technology and allows for wide application in warehousing and logistics, large venue inspection, urban services, and other fields.
[0071] In summary, this application embodiment obtains the pose information of each keyframe in the target map data; wherein, the target map data is the map data used by the robot for visual synchronous localization and map construction; the keyframes in the target map data are clustered according to the pose information to obtain keyframe clustering information; the feature point descriptors of each keyframe in the target map data are split according to the keyframe clustering information to obtain each feature point descriptor map block; the keyframe clustering information is used as the index of each feature point descriptor map block; wherein, each keyframe cluster in the keyframe clustering information corresponds to one feature point descriptor map block. In this application embodiment, the modular splitting of feature point descriptors is achieved through pose clustering, splitting the originally large amount of full feature point descriptors into multiple smaller map blocks, and the rapid indexing of map blocks is achieved through keyframe clustering information, thereby enabling more effective management of map data.
[0072] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0073] Corresponding to the map data management method described in the above embodiments, Figure 5 This illustration shows a structural diagram of one embodiment of a map data management device provided in this application.
[0074] In this embodiment, a map data management device may include: The pose information acquisition module 501 is used to acquire the pose information of each key frame in the target map data; wherein, the target map data is the map data for the robot to perform visual synchronous localization and map construction. The keyframe clustering module 502 is used to cluster each keyframe in the target map data according to the pose information to obtain keyframe clustering information. The map tile splitting module 503 is used to split the feature point descriptors of each key frame in the target map data according to the key frame clustering information to obtain each feature point descriptor map tile. The map tile index module 504 is used to use the keyframe clustering information as an index for each feature point descriptor map tile; wherein, each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map tile.
[0075] In one specific implementation of this application embodiment, the map data management device may further include: A global image descriptor extraction module is used to extract the global image descriptor of the current frame of the robot during the robot's relocalization process. The global image descriptor matching module is used to match the global image descriptor of the current frame with the global image descriptors of each key frame in the target map data, and to filter out several candidate key frames based on the matching results. The first map tile loading module is used to determine a first feature point descriptor map tile based on the keyframe clustering information and load the first feature point descriptor map tile into memory; wherein, the first feature point descriptor map tile is a feature point descriptor map tile corresponding to the candidate keyframe. The repositioning pose determination module is used to determine the robot's repositioning pose based on the first feature point descriptor map block and the pose information.
[0076] In one specific implementation of this application embodiment, the repositioning pose determination module can be specifically used to: extract the feature point descriptor of the current frame; match the feature point descriptor of the current frame with the feature point descriptors of each key frame in the first feature point descriptor map block to obtain the feature point matching result; and perform pose calculation based on the feature point matching result and the pose information to obtain the repositioning pose of the robot.
[0077] In one specific implementation of this application embodiment, the map data management device may further include: The map tile preloading module is used to determine several second feature point descriptor map tiles based on the keyframe clustering information; wherein, the second feature point descriptor map tile is a feature point descriptor map tile corresponding to the relocalization pose; and the second feature point descriptor map tile is loaded into memory for use in the robot's localization process.
[0078] In one specific implementation of this application embodiment, the map data management device may further include: The map tile update module is used to acquire the real-time pose of the robot during the robot's localization process and update the feature point descriptor map tiles loaded into memory based on the real-time pose.
[0079] In one specific implementation of this application embodiment, the map tile update module may include: The map tile unloading unit is used to determine a number of third feature point descriptor map tiles based on the keyframe clustering information; wherein the third feature point descriptor map tile is a feature point descriptor map tile whose distance from the real-time pose is greater than a preset distance threshold and has been loaded into memory; and unload the third feature point descriptor map tile from memory.
[0080] In one specific implementation of this application embodiment, the map tile update module may include: The map tile loading unit is used to determine several fourth feature point descriptor map tiles based on the keyframe clustering information; wherein, the fourth feature point descriptor map tile is a feature point descriptor map tile whose distance from the real-time pose is less than a preset distance threshold and which has not been loaded into memory; and to load the fourth feature point descriptor map tile into memory.
[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0082] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0083] Figure 6 A schematic block diagram of a robot provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0084] like Figure 6 As shown, the robot 6 in this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps described in the various map data management method embodiments above, for example... Figure 2 Steps S201 to S204 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 504 are shown.
[0085] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the robot 6.
[0086] Those skilled in the art will understand that Figure 6 This is merely an example of robot 6 and does not constitute a limitation on robot 6. It may include more or fewer parts than shown, or combine certain parts, or different parts. For example, robot 6 may also include input / output devices, network access devices, buses, etc.
[0087] The processor 60 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0088] The memory 61 can be an internal storage unit of the robot 6, such as a hard drive or memory. The memory 61 can also be an external storage device of the robot 6, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the robot 6. The memory 61 is used to store the computer program and other programs and data required by the robot 6. The memory 61 can also be used to temporarily store data that has been output or will be output.
[0089] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0092] In the embodiments provided in this application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0094] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0095] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0096] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A map data management method, characterized in that, include: Acquire the pose information of each keyframe in the target map data; wherein, the target map data is the map data for the robot to perform visual synchronous localization and map construction; Based on the pose information, cluster each keyframe in the target map data to obtain keyframe clustering information; Based on the keyframe clustering information, the feature point descriptors of each keyframe in the target map data are split to obtain each feature point descriptor map block. The keyframe clustering information is used as an index for each feature point descriptor map block; wherein, each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map block.
2. The map data management method according to claim 1, characterized in that, After using the keyframe clustering information as an index for each feature point descriptor map patch, the method further includes: During the robot's relocalization process, the global image descriptor of the robot's current frame is extracted; The global image descriptor of the current frame is matched with the global image descriptors of each key frame in the target map data, and several candidate key frames are selected based on the matching results. The first feature point descriptor map block is determined based on the keyframe clustering information, and the first feature point descriptor map block is loaded into memory; wherein, the first feature point descriptor map block is the feature point descriptor map block corresponding to the candidate keyframe; The robot's repositioning pose is determined based on the first feature point descriptor map block and the pose information.
3. The map data management method according to claim 2, characterized in that, Determining the robot's repositioning pose based on the first feature point descriptor map block and the pose information includes: Extract the feature point descriptor of the current frame; The feature point descriptor of the current frame is matched with the feature point descriptors of each key frame in the first feature point descriptor map block to obtain the feature point matching result. The robot's repositioning pose is obtained by calculating the pose based on the feature point matching results and the pose information.
4. The map data management method according to any one of claims 2 to 3, characterized in that, After determining the robot's repositioning pose, the process also includes: Based on the keyframe clustering information, several second feature point descriptor map blocks are determined; wherein, the second feature point descriptor map block is a feature point descriptor map block corresponding to the relocation pose; The second feature point descriptor map block is loaded into memory for use in the robot's localization process.
5. The map data management method according to claim 4, characterized in that, After loading the second feature point descriptor map tile into memory, the process also includes: During the robot's localization process, the robot's real-time pose is acquired, and the feature point descriptor map blocks loaded into memory are updated based on the real-time pose.
6. The map data management method according to claim 5, characterized in that, The step of updating the feature point descriptor map block loaded into memory based on the real-time pose includes: Several third feature point descriptor map blocks are determined based on the keyframe clustering information; wherein, the third feature point descriptor map block is a feature point descriptor map block whose distance from the real-time pose is greater than a preset distance threshold and has been loaded into memory. Unload the third feature point descriptor map block from memory.
7. The map data management method according to claim 5, characterized in that, The step of updating the feature point descriptor map block loaded into memory based on the real-time pose includes: Several fourth feature point descriptor map blocks are determined based on the keyframe clustering information; wherein, the fourth feature point descriptor map block is a feature point descriptor map block whose distance from the real-time pose is less than a preset distance threshold and which has not been loaded into memory. The fourth feature point descriptor map block is loaded into memory.
8. A map data management device, characterized in that, include: The pose information acquisition module is used to acquire the pose information of each keyframe in the target map data; wherein, the target map data is the map data for the robot to perform visual synchronous localization and map construction. The keyframe clustering module is used to cluster each keyframe in the target map data according to the pose information to obtain keyframe clustering information. The map tile splitting module is used to split the feature point descriptors of each key frame in the target map data according to the key frame clustering information to obtain map tiles of each feature point descriptor. The map tile indexing module is used to use the keyframe clustering information as an index for each feature point descriptor map tile; wherein each keyframe cluster in the keyframe clustering information corresponds to a feature point descriptor map tile.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the map data management method as described in any one of claims 1 to 7.
10. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the map data management method as described in any one of claims 1 to 7.