A method, system, device and medium for updating a management positioning map

Abstract

The present application relates to a method, system, device and medium for updating and managing a positioning map. The method includes: obtaining the positioning map and initializing the positioning map; then detecting the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source and detection method; and performing corresponding update management on the positioning map according to the detection result. Through the present application, the problem of how to update and manage the positioning map to adapt it to the changed scene is solved, and the positioning accuracy and user experience are improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and in particular to a method, system, device, and medium for updating and managing a positioning map. Background Art

[0002] With the rapid development of computer technology, augmented reality (AR) technology has gradually penetrated into more industries and application scenarios. Among the various technologies relied on by AR applications, the construction of a locatable map of the real-world scene is the most fundamental and crucial link, because AR applications generally need to automatically locate AR devices in the scene, and the necessary prerequisite for achieving automatic positioning is a locatable map and a positioning algorithm.

[0003] Generally speaking, the automatic positioning process of an AR device can be abstracted and simplified as: obtaining scene content and device status → querying map spatial points → constructing constraints between spatial points and scene content and solving for the device pose. According to different ways of obtaining scene content and different types of device status, the corresponding positioning map and positioning algorithm will have different implementation forms. For example, if the device is equipped with a camera device, such as a mobile phone, AR glasses, etc., the scene content can be obtained by taking pictures, and the corresponding positioning map is also constructed from image data, and map points can be queried through visual features. Finally, when solving for the pose, a camera projection constraint can be used to establish a solution algorithm, such as the PnP (Perspective-n-Points) algorithm. This kind of positioning map and positioning algorithm are regarded as pure vision positioning problems. In addition to taking pictures with a camera device, the device may also be equipped with a GPS module to obtain the current GPS position information of the device. Then, the positioning algorithm at this time can fuse GPS and visual information for use; for another example, the positioning device may also include a lidar or TOF module, then the scene content can be obtained through point cloud information, and the corresponding positioning map may be mainly based on point clouds and point cloud features, and the positioning algorithm may also change from PnP to ICP (Iterative Closest Point). For another example, a Bluetooth positioning map of the scene can also be made by a professional supplier, that is, scene perception and positioning can be performed through the Bluetooth module of the device. Of course, the above-mentioned pure vision, lidar, Bluetooth and other technologies can be used in combination to construct a multi-sensor positioning map, so as to learn from each other's strengths and achieve more stable, accurate, and fast device positioning.

[0004] However, in the related art, no matter which of the above positioning maps is adopted, the problems of map update and management need to be solved. Because the on-site environment will gradually change over time, making the scene information collected when the map is first constructed may no longer match the information perceived by the current device at all, resulting in inability to locate. For example, when a certain store scene is photographed during map construction, if this store is renovated and upgraded after a period of time, then when the device takes a picture here again at this time, it is very difficult to locate successfully.

[0005] Currently, for the problem in the related art of how to update and manage the positioning map to adapt it to the changed scene, no effective solution has been proposed. Summary of the Invention

[0006] The embodiments of the present application provide a method, a system, a device and a medium for updating and managing a positioning map, so as to at least solve the problem in the related art of how to update and manage the positioning map to adapt it to the changed scene.

[0007] In a first aspect, the embodiments of the present application provide a method for updating and managing a positioning map, and the method includes:

[0008] Obtain a positioning map and initialize the positioning map;

[0009] Detect the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, the detection source and the detection method;

[0010] Perform corresponding update management on the positioning map according to the detection result.

[0011] In some of these embodiments, before obtaining the positioning map, the method includes:

[0012] Obtain a scene image, construct a scene map through the scene image, extract global features and local features from the scene image respectively, and associate each local feature with the corresponding global feature and 3D point to obtain the positioning map.

[0013] In some of these embodiments, initializing the positioning map includes:

[0014] Perform clustering and chunking on the positioning map to obtain different map chunks. Among them, the map elements included in each map chunk include map points, local features and global features associated with each map point;

[0015] Detect the map elements included in each map chunk through a positioning ability detection algorithm.

[0016] In some of these embodiments, detecting the positioning map through a positioning ability detection algorithm, or detecting the map elements included in each map block through a positioning ability detection algorithm includes:

[0017] Input a query image, identify the abnormal source of the query image through a positioning algorithm, determine whether each query image is successfully positioned, and determine the map block to which each query image belongs in the positioning map;

[0018] Perform a hierarchical status detection on the map block with an abnormal source.

[0019] In some of these embodiments, performing a hierarchical status detection on the map block with an abnormal source includes:

[0020] First-level global detection: Calculate the true value candidate map of the query image. If there is no true value candidate map for the query image, then it is determined that the detected map block has problems with insufficient spatial points, global features, and local features, and add the query image without a true value candidate map to the first set; otherwise, if there is a corresponding true value candidate map for the query image, then calculate the correct rate of the global feature query. If the correct rate is lower than the preset threshold, it is determined that the detected map block has a problem with insufficient global feature accuracy, and add the query image with a true value candidate map to the second set;

[0021] Second-level local detection: In the case where the query image passes the first-level global detection, calculate the number of effective 2D-2D feature matches of the query image in the positioning map through local feature query. If the number of effective 2D-2D feature matches is lower than the preset threshold, it is determined that the detected map block has a problem with insufficient local feature quantity, and add the query image and its corresponding true value candidate map to the third set;

[0022] Third-level 2D-3D matching detection: In the case where the query image passes the first-level global detection and the second-level local detection, calculate the number of 3D points associated with the 2D features of the query image. If the number is lower than the preset threshold, it is determined that the detected map block has a problem with insufficient spatial point quantity, and add the query image and its corresponding true value candidate map to the third set; otherwise, if the query image still fails to be positioned in the case of passing the 2D-3D matching detection, it is determined that the detected map block has a problem with insufficient map point accuracy, and add the query image and the associated map points to the fourth set;

[0023] Fourth-level redundancy detection: Calculate the effective utilization rate of each map feature in each map block respectively. If the effective utilization rate of a map feature is lower than a preset threshold, it is determined that the map feature is redundant, and the map feature is added to the fifth set. Among them, the fourth-level redundancy detection is always performed throughout the hierarchical detection process.

[0024] In some embodiments, the corresponding update management of the positioning map according to the detection result includes:

[0025] Obtain the detection results at all levels, and perform positioning map update management operations on the map blocks containing different sets to solve the problems existing in the map blocks detected in the detection results at all levels.

[0026] In some embodiments, perform map update management operations on the map blocks containing different sets to solve the problems existing in the map blocks detected in the detection results at all levels:

[0027] If the map block contains a non-empty first set, perform target area registration expansion according to the query images in the first set;

[0028] If the map block contains a non-empty second set, determine the scene area to be updated according to the query images in the second set, and improve the global query accuracy rate through the method of positioning map feature expansion;

[0029] If the map block contains a non-empty third set, use the ground truth candidate map corresponding to the query images in the third set as the expansion source, split the images in the expansion source into sub-images, extract local feature points for each sub-image separately, and map them back to the corresponding source images to expand to obtain new 2D feature points and generate corresponding 3D points;

[0030] If the map block contains a non-empty fourth set, improve the accuracy of the scene area where the map point accuracy needs to be improved according to different reasons;

[0031] If the map block contains a non-empty fifth set, selectively delete the map features in the fifth set while maintaining the integrity of the positioning map structure.

[0032] In a second aspect, an embodiment of the present application provides a system for updating and managing a positioning map. The system includes:

[0033] An initialization module, configured to obtain a positioning map and initialize the positioning map;

[0034] A positioning detection module, configured to detect the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source, and detection method;

[0035] An update management module, configured to perform corresponding update management on the positioning map according to the detection result.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for updating and managing a positioning map as described in the first aspect above is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a storage medium, on which a computer program is stored. When the program is executed by a processor, the method for updating and managing a positioning map as described in the first aspect above is implemented.

[0038] Compared with the related art, the method for updating and managing a positioning map provided by the embodiment of the present application obtains a positioning map and initializes the positioning map; then, the positioning map is detected through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source, and detection method; corresponding update management is performed on the positioning map according to the detection result, solving the problem of how to update and manage the positioning map to adapt to the changed scenario, and improving the positioning accuracy and user experience. Description of the Drawings

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0040] Figure 1 is a flowchart of the method for updating and managing a positioning map according to an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of the association of each feature element of the positioning map according to an embodiment of the present application;

[0042] Figure 3 is a structural block diagram of the system for updating and managing a positioning map according to an embodiment of the present application;

[0043] Figure 4 is an internal structural schematic diagram of an electronic device according to an embodiment of the present application. Detailed Embodiments

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without creative efforts belong to the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as insufficient disclosure of the content of the present application.

[0045] Reference to "embodiment" in the present application means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in the present application should have the ordinary meaning understood by those of ordinary skill in the technical field to which the present application belongs. The words such as "a", "one", "kind", "the" and the like involved in the present application do not indicate a quantity limitation and can represent a single or plural number. The terms "including", "comprising", "having" and any variations thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but may further include unlisted steps or units, or may further include other steps or units inherent to these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in the present application means greater than or equal to two. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0047] This embodiment provides a method for updating and managing a positioning map. Figure 1is a flowchart of a method for updating and managing a positioning map according to an embodiment of the present application. As Figure 1 shown, the process includes the following steps:

[0048] First of all, it should be noted that the following embodiments take a pure vision positioning map as an example to update and manage the positioning map. However, the method of the present application can also be applied to other types of positioning maps, without specific limitation.

[0049] Step S101, obtain the positioning map and initialize the positioning map;

[0050] Figure 2 is a schematic diagram of the association of each feature element of the positioning map according to an embodiment of the present application. Preferably, before obtaining the positioning map, it is necessary to construct the positioning map. For a pure vision positioning map, the specific construction process includes: obtaining a scene image, constructing a scene sparse map, that is, a sparse point cloud map, according to the scene image through the SFM algorithm, and then extracting the global features and local features of the scene image respectively. Finally, each local feature is associated with the corresponding global feature and 3D point to obtain the positioning map. As Figure 2 shown, the squares represent local features, the dashed ellipses represent global features, and the circles represent 3D points in the map. Each global feature can be used as a global label and associated with multiple local feature points, while each local feature point can be associated with a 3D point.

[0051] Furthermore, obtain the constructed positioning map and initialize the positioning map. Preferably, the specific initialization process includes:

[0052] First, perform clustering and partitioning on the positioning map to obtain different map blocks. Among them, the map elements included in each map block include map points, local features and global features associated with each map point. It should be noted that the clustering and partitioning method in this embodiment is not specifically limited. For example, a top view plan of the positioning map can be generated, and on the plan, the map is divided into several regular map blocks. For example, the map covered by a square with an area of 5m×5m is used as a map block for segmentation. Among them, the map points in each map block, as well as the associated local features and global features, are all the map elements of the map block;

[0053] Then, detect the map elements included in each map block through a positioning ability detection algorithm, set the states of each map element according to the detection results, and multiple states can be set superimposed. The state types include: normal and available, insufficient quantity, insufficient accuracy, and redundancy. Table 1 is an example table of the initialization state setting of a certain positioning map according to an embodiment of the present application, as shown in Table 1 below:

[0054] Table 1

[0055] Map point Global feature Local feature Normally available √ √ Insufficient quantity √ Insufficient accuracy √ Redundancy exists

[0056] Step S102, detect the positioning map through the positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source, and detection method;

[0057] Preferably, in this embodiment, during the initialization or operation and use stage of the positioning map, according to the input detection source, perform positioning detection on the positioning map to update the state of the positioning map, that is, update and set the states of each map element in the positioning map. The state types include: normal and available, insufficient quantity, insufficient accuracy, and redundancy. Then obtain the current detection result of the positioning map.

[0058] Specifically, the parts that need to be clarified during the detection process mainly include: detection timing, detection source, and detection method;

[0059] Among them, for the detection time: Except for the need to perform the positioning ability detection for the first time during map initialization, the detection frequency and time can be set flexibly according to the actual situation. For example, the detection can be performed regularly at a fixed time, or the detection can be performed after receiving a sufficient number of positioning failure feedbacks.

[0060] For the detection source: The detection source refers to the input source of the detection algorithm, including internal sources and external sources. Among them, the internal sources include: the pictures taken during map construction, and the external sources collected during previous map updates; while the external source refers to the external images collected after the previous map update and before the current map update, such as the query images uploaded by users during normal use, or the test images specially collected by relevant testers, etc.

[0061] For the detection method: The detection method here is the positioning ability detection algorithm in this embodiment. For different map elements and states, the detection means are not the same. The following exemplarily describes a complete detection process. The specific steps are as follows:

[0062] S1. Abnormal source identification: Input the query image, and perform abnormal source identification on the query image through the positioning algorithm to determine whether each query image is successfully positioned. If the query image is generated during the user's daily use, then this step directly follows the online running positioning map for online execution; if the query image comes from the pictures batch-collected during a certain map test, then this step can be executed offline in batches;

[0063] S2. Abnormal Map Block Annotation: Determine the map block to which each query image belongs in the positioning map. Specifically, if a query image is successfully positioned, the map block it belongs to can be directly determined based on the map points it observes; conversely, if a query image fails to be positioned, the true value candidate search algorithm needs to be used to find the true value candidate map of the image in the map, and the map block to which the query image belongs is determined based on the true value candidate map. However, if the query image fails to be positioned and no true value candidate map can be found, other information needs to be used to determine the map block, such as gps information, device vio (visual-inertial-odometry) tracking information, or even manual annotation, etc.;

[0064] S3. Status Detection: Perform hierarchical status detection on each map block with abnormal sources. Specifically, the detection steps are as follows:

[0065] (1) First-level Global Detection: Calculate the true value candidate map of the query image. If there is no true value candidate map for the query image, it is determined that the map block to which the query image belongs has problems with insufficient spatial points, global features, and local features, and the above query images without true value candidate maps are added to the first set, that is, the registration and expansion target set; conversely, if there is a corresponding true value candidate map for the query image, calculate the accuracy rate of the global feature query. If the accuracy rate is lower than the preset threshold, it is determined that the map block to which the query image belongs has problems with insufficient global feature accuracy, and the above query images with true value candidate maps are added to the second set, that is, the global feature update target set;

[0066] (2) Second-level Local Detection: When the query image passes the first-level global detection, calculate the number of effective 2D-2D feature matches of the query image in the positioning map through local feature query. If the number of effective 2D-2D feature matches is lower than the preset threshold, it is determined that the map block to which the query image belongs has problems with insufficient local feature quantity, and the query image and its corresponding true value candidate map are added to the third set, that is, the 2D-3D expansion target set;

[0067] (3) Third-level 2D-3D Matching Detection: When the query image passes the first-level global detection and the second-level local detection, calculate the number of 3D points associated with the 2D features of the query image. If the number is lower than the preset threshold, it is determined that the map block to which the query image belongs has problems with insufficient spatial point quantity, and the query image and its corresponding true value candidate map are added to the third set, that is, the 2D-3D expansion target set; conversely, if the query image still fails to be positioned when the 2D-3D matching detection passes, it is determined that the detected map block has problems with insufficient map point accuracy, and the query image and the associated map points are added to the fourth set, that is, the map point accuracy improvement target set;

[0068] Fourth-level redundancy detection: Calculate the effective utilization rate of each map feature in each map block. If the effective utilization rate of a map feature is lower than the preset threshold, it is determined that the map feature is redundant, and the map feature is added to the fifth set, that is, the target set to be deleted. The calculation formula for the effective utilization rate of map points is shown in the following formula (1):

[0069]

[0070] Among them, the denominator represents the number of times a certain map point appears in the frustum of the query map with successful positioning, and the numerator represents the number of times it is actually used in the query map with successful positioning.

[0071] The calculation formula for the effective utilization rate of global features is shown in the following formula (2):

[0072]

[0073] Among them, the denominator represents the number of times a certain global feature appears in the candidate list of the query map with successful positioning, and the numerator represents the number of times it provides effective local feature matching in the query map with successful positioning.

[0074] The calculation formula for the effective utilization rate of local features is shown in the following formula (3):

[0075]

[0076] Among them, the denominator represents the number of times a certain local feature is matched in the query map with successful positioning, and the numerator represents the number of times it provides effective map points in the query map with successful positioning.

[0077] Finally, it should be pointed out that the first-level global detection, the second-level local detection, and the third-level 2D-3D matching detection in the detection steps are progressive, but the fourth-level redundancy detection is always executed throughout the hierarchical detection process.

[0078] In addition, it should be noted that in this embodiment, the method for obtaining the ground truth candidate map of the query image is not unique. The preferred method is to perform local feature matching and two-view geometry verification on the query image and all map images in the positioning map; when the verification passes and the number of matches is greater than the preset threshold, it can be considered that the matched image is similar to the query image. At this time, sort the map images that pass the verification and matching in descending order according to the number of local feature matches to obtain the ground truth candidate frame list of the query image. Through the obtained ground truth candidate frame list, the global query result can be evaluated, that is, the accuracy rate of the global query can be calculated.

[0079] Let the query image be imgQ, the query result be resQ = {img1, img2,..., imgX}, the length of the query result be K, and the ground truth candidate frame list be resGT = {img1, img2,..., imgY}. Then the calculation formula for the accuracy rate of the global query result is shown in the following formula (4):

[0080]

[0081] where |{·}| represents the number of elements in the set.

[0082] In this embodiment, a detailed description and determination are made for the possible positioning failure problem in the positioning detection, which is beneficial to the subsequent update management of the positioning map and improves the positioning accuracy;

[0083] Step S103, perform corresponding update management on the positioning map according to the detection result.

[0084] Preferably, in this embodiment, the detection results at all levels obtained in step S102 are acquired, and an update management operation of the positioning map is performed on the map blocks containing different sets to solve the problems existing in the map blocks detected in the detection results at all levels. The specific update operations include:

[0085] 1. Registration and expansion: If the map block contains a non-empty first set, target area registration and expansion are performed according to the query images in the first set to solve the problem that the map block detected in the first-level global detection has insufficient spatial points, global features, and local features. The specific method is as follows: According to the first set, register and expand the query images listed in the target set, determine the scene area to be expanded, obtain the captured images of the target area, and ensure that there is sufficient visual correlation between these images and the images of the existing map; then, through the method of map construction, these images are added to the positioning map. A typical addition process includes: using the incremental SFM algorithm to construct sparse map points, then extracting global features and local features of the images, and finally merging the newly added map elements into the existing positioning map resources;

[0086] 2. Global feature update: If the map block contains a non-empty second set, determine the scene area to be updated according to the query images in the second set, and improve the global query accuracy rate through the method of expanding the positioning map features to solve the problem that the map block detected in the first-level global detection has insufficient global feature accuracy;

[0087] 3. 2D-3D Expansion: If the map block contains a non-empty third set, use the ground truth candidate map corresponding to the query image in the third set as the expansion source. Split the images in the expansion source into sub-images, for example, using squares of different sizes and positions to split all source images into several sub-images. The sub-images can overlap or not. Then, extract local feature points for each sub-image separately and map them back to the corresponding source images one by one. In this way, a large number of new 2D feature points can be obtained. Further, based on the expanded 2D feature points, a large number of new 3D points can be generated through triangulation and BA algorithms in the map construction method. Finally, the newly added 2D-3D points can be merged into the positioning map resources. Through the above method, the problem of insufficient local feature quantity in the map block detected in the second-level local detection or the problem of insufficient spatial point quantity in the map block detected in the third-level 2D-3D matching detection can be solved;

[0088] 4. Map Point Accuracy Improvement: If the map block contains a non-empty fourth set, improve the accuracy of the scene area where the map point accuracy needs to be improved through different solutions according to different reasons. Suppose it is found through analysis that a certain scene area is caused by the inappropriate triangulation algorithm selected during map construction. Then, in this stage, a more suitable triangulation algorithm can be used to regenerate the map points in this part and delete the corresponding old map points to complete the update.

[0089] 5. Map Element Deletion: If the map block contains a non-empty fifth set, selectively delete the map elements in the fifth set while maintaining the integrity of the positioning map structure. Specifically, when deleting a map point, the local features associated with this map point should also be deleted accordingly; local features can be directly deleted, but when it is found that all local features associated with a map point are deleted, this map point should also be deleted. Similarly, when all local features associated with a global feature are deleted, this global feature should also be deleted; finally, global features can also be directly deleted, but when there are no global features associated with a local feature, this local feature should still be retained because local features are more critical than global features, and global features are only used to accelerate the search process of local features.

[0090] Through the above steps S101 to S103, this embodiment analyzes and determines the possible reasons for positioning failure during map query and positioning at different levels, and proposes corresponding solutions, solving the problem of how to update and manage the positioning map to adapt to the changed scene, and improving the positioning accuracy and user experience.

[0091] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0092] This embodiment also provides a system for updating and managing a positioning map. This system is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0093] Figure 3 is a structural block diagram of a system for updating and managing a positioning map according to an embodiment of the present application. As Figure 3 shown, the system includes an initialization module 31, a positioning detection module 32, and an update management module 33:

[0094] The initialization module 31 is used to obtain a positioning map and initialize the positioning map; the positioning detection module 32 is used to detect the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source, and detection method; the update management module 33 is used to perform corresponding update management on the positioning map according to the detection result.

[0095] Through the above system, this embodiment analyzes and determines the reasons for possible positioning failures during map query and positioning at different levels, and proposes corresponding solutions, solving the problem of how to update and manage the positioning map to adapt to the changed scenario, and improving the positioning accuracy and user experience.

[0096] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and alternative implementation manners, and will not be repeated here.

[0097] In addition, it should be noted that the above-mentioned each module can be a functional module or a program module, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned each module can be located in the same processor; or the above-mentioned each module can also be located in different processors in any combination form.

[0098] This embodiment also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments.

[0099] Optionally, the above electronic device may further include a transmission device and an input / output device. The transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0100] In addition, in combination with the method for updating and managing a positioning map in the above embodiments, an embodiment of the present application can be implemented by providing a storage medium. A computer program is stored on the storage medium; when the computer program is executed by a processor, it implements any one of the methods for updating and managing a positioning map in the above embodiments.

[0101] In one embodiment, a computer device is provided. The computer device may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a method for updating and managing a positioning map. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0102] In one embodiment, Figure 4 is a schematic internal structure diagram of an electronic device according to an embodiment of the present application, as Figure 4 shown, an electronic device is provided. The electronic device may be a server, and its internal structure diagram may be as Figure 4 shown. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected through an internal bus. The non-volatile memory stores an operating system, a computer program, and a database. The processor is used to provide computing and control capabilities. The network interface is used to communicate with an external terminal through a network connection. The internal memory is used to provide an environment for the operation of the operating system and the computer program. The computer program, when executed by the processor, implements a method for updating and managing a positioning map. The database is used to store data.

[0103] Those skilled in the art can understand, Figure 4The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0104] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0105] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not conflict, they should be considered to be within the scope described in this specification.

[0106] The above embodiments only represent several implementation manners of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application should be subject to the appended claims.

Claims

1. A method for updating and managing a positioning map, characterized in that, The method includes: obtaining a positioning map and initializing the positioning map; detecting the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map. Among them, the parts that need to be clarified during the detection process include the detection timing, detection source, and detection method. The detecting the positioning map through the positioning ability detection algorithm includes: inputting a query image, identifying abnormal sources of the query image through a positioning algorithm, determining whether each query image is successfully positioned, and determining the map block to which each query image belongs in the positioning map, and performing a hierarchical status detection on the map block with abnormal sources. The performing a hierarchical status detection on the map block with abnormal sources includes: The first-level global detection: calculating the true value candidate map of the query image. If there is no true value candidate map for the query image, then it is determined that there are problems with insufficient spatial points, global features, and local features in the detected map block, and the query image without the true value candidate map is added to the first set. Conversely, if there is a corresponding true value candidate map for the query image, then calculate the correct rate of the global feature query. If the correct rate is lower than the preset threshold, it is determined that there is a problem with insufficient global feature accuracy in the detected map block, and the query image with the true value candidate map is added to the second set. The second-level local detection: when the query image passes the first-level global detection, calculating the number of effective 2D-2D feature matches of the query image in the positioning map through local feature query. If the number of effective 2D-2D feature matches is lower than the preset threshold, it is determined that there is a problem with insufficient local feature quantity in the detected map block, and the query image and its corresponding true value candidate map are added to the third set. The third-level 2D-3D matching detection: when the query image passes the first-level global detection and the second-level local detection, calculating the number of 3D points associated with the 2D features of the query image. If the number is lower than the preset threshold, it is determined that there is a problem with insufficient spatial point quantity in the detected map block, and the query image and its corresponding true value candidate map are added to the third set. Conversely, if the query image is still not successfully positioned when passing the 2D-3D matching detection, it is determined that there is a problem with insufficient map point accuracy in the detected map block, and the query image and the associated map points are added to the fourth set. The fourth-level redundancy detection: calculating the effective usage rate of each map element in each map block respectively. If the effective usage rate of a map element is lower than the preset threshold, it is determined that the map element is redundant, and the map element is added to the fifth set. Among them, the fourth-level redundancy detection is always performed throughout the hierarchical detection process. Performing corresponding update management on the positioning map according to the detection result, including: judging whether the corresponding map block needs to be expanded, updated, or deleted according to the detection results at each level.

2. The method according to claim 1, characterized in that, Before obtaining the positioning map, the method includes: Obtain a scene image, construct a scene map through the scene image, extract global features and local features from the scene image respectively, and associate each local feature with the corresponding global feature and 3D point to obtain the positioning map.

3. The method according to claim 1, characterized in that, Initializing the positioning map includes: Performing clustering and partitioning on the positioning map to obtain different map blocks, where the map elements included in each map block include map points, local features and global features associated with each map point; Detecting the map elements included in each map block through a positioning ability detection algorithm.

4. The method according to claim 3, characterized in that, Detecting the map elements included in each map block through a positioning ability detection algorithm includes: Input a query image, identify abnormal sources for the query image through a positioning algorithm, determine whether each query image is successfully positioned, and determine the map block to which each query image belongs in the positioning map; Performing a hierarchical status detection on the map blocks with abnormal sources.

5. The method according to claim 1, characterized in that, Performing corresponding update management on the positioning map according to the detection results includes: Obtaining the detection results at all levels, and performing positioning map update management operations on the map blocks containing different sets to solve the problems existing in the map blocks detected in the detection results at all levels.

6. The method according to claim 5, characterized in that, Performing map update management operations on the map blocks containing different sets to solve the problems existing in the map blocks detected in the detection results at all levels: If the map block contains a non-empty first set, perform target area registration and expansion according to the query images in the first set; If the map block contains a non-empty second set, determine the scene area to be updated according to the query images in the second set, and improve the global query accuracy rate through the method of positioning map feature expansion; If the map block contains a non-empty third set, use the true value candidate map corresponding to the query images in the third set as an expansion source, split the images in the expansion source into sub-images, extract local feature points for each sub-image separately, and map them back to the corresponding source images to expand new 2D feature points and generate corresponding 3D points; If the map block contains a non-empty fourth set, improve the accuracy of the scene area where the map point accuracy needs to be improved according to different reasons; If the map block contains a non-empty fifth set, selectively delete the map elements in the fifth set while maintaining the integrity of the positioning map structure.

7. A system for updating and managing a positioning map, characterized in that, The system includes: An initialization module, configured to obtain a positioning map and initialize the positioning map; A positioning detection module, configured to detect the positioning map through a positioning ability detection algorithm to obtain the current detection result of the positioning map, where the parts that need to be clarified during the detection process include the detection timing, detection source and detection method, and detecting the positioning map through the positioning ability detection algorithm includes: Input a query image, identify abnormal sources for the query image through a positioning algorithm, determine whether each query image is successfully positioned, and determine the map block to which each query image belongs in the positioning map, and perform a hierarchical status detection on the map blocks with abnormal sources. The performing a hierarchical status detection on the map blocks with abnormal sources includes: First-level global detection: Calculate the ground truth candidate map of the query image. If there is no ground truth candidate map for the query image, it is determined that there are problems with insufficient spatial points, global features, and local features in the detected map block, and the query image without the ground truth candidate map is added to the first set. Conversely, if there is a corresponding ground truth candidate map for the query image, calculate the accuracy rate of the global feature query. If the accuracy rate is lower than the preset threshold, it is determined that there is a problem with insufficient global feature accuracy in the detected map block, and the query image with the ground truth candidate map is added to the second set. Second-level local detection: When the query image passes the first-level global detection, calculate the number of valid 2D-2D feature matches of the query image in the positioning map through local feature query. If the number of valid 2D-2D feature matches is lower than the preset threshold, it is determined that there is a problem with insufficient local feature quantity in the detected map block, and the query image and its corresponding ground truth candidate map are added to the third set. Third-level 2D-3D matching detection: When the query image passes the first-level global detection and the second-level local detection, calculate the number of 3D points associated with the 2D features of the query image. If the number is lower than the preset threshold, it is determined that there is a problem with insufficient spatial point quantity in the detected map block, and the query image and its corresponding ground truth candidate map are added to the third set. Conversely, if the query image still fails to be positioned when the 2D-3D matching detection passes, it is determined that there is a problem with insufficient map point accuracy in the detected map block, and the query image and the associated map points are added to the fourth set. Fourth-level redundancy detection: Calculate the effective utilization rate of each map element in each map block respectively. If the effective utilization rate of a map element is lower than the preset threshold, it is determined that the map element is redundant, and the map element is added to the fifth set. Among them, the fourth-level redundancy detection is always executed throughout the hierarchical detection process. Update management module, configured to perform corresponding update management on the positioning map according to the detection results, including: judging whether the corresponding map block needs to be expanded, updated, or deleted according to the detection results at each level.

8. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the method for updating and managing the positioning map according to any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method for updating and managing the positioning map according to any one of claims 1 to 6 when running.

Images