Positioning methods and related devices
By acquiring and processing point cloud and pose information in augmented reality (AR) space using augmented reality technology, and determining coordinate system transformation relationships, the problem of content asynchrony in multi-user AR space was solved, and AR content synchronization was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 北京数原数字化城市研究中心
- Filing Date
- 2025-11-21
- Publication Date
- 2026-06-26
AI Technical Summary
In augmented reality (AR) spaces with multiple users and different sessions, content states can easily become out of sync, leading to inconsistent experiences.
By acquiring spatial information of the scaleless AR space and the scaled AR space of the target space, and using point cloud information and pose information to determine the coordinate system transformation relationship, all scaled AR spaces are unified into the same first scaled AR space, thus achieving content synchronization.
It solves the problem of asynchronous content states when multiple users participate in AR space across sessions, enabling all users to share the content of the same AR space.
Smart Images
Figure CN121639792B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of augmented reality technology, specifically relating to a positioning method and related equipment. Background Technology
[0002] Augmented Reality (AR) is a technology that overlays and merges virtual scenes or information with the real physical environment, presenting them interactively to the user, thereby creating a space where the virtual and the real share the same space.
[0003] Virtual objects often need to be anchored in physical space (such as a corner or desktop). When multiple users participate in different sessions, the spatial coordinates of multiple users (who are in different locations in the space) and cross sessions (who come in after a period of time, resulting in different locations) may drift, causing the content state to be out of sync and the experience to be inconsistent across multiple devices. Summary of the Invention
[0004] The purpose of this application is to provide a positioning method and related equipment that can solve the problem of asynchronous content states in AR spaces involving multiple users and different sessions.
[0005] In a first aspect, embodiments of this application provide a positioning method, the method comprising:
[0006] The spatial information of the scale-free augmented reality (AR) space corresponding to the target space and the spatial information of the first scaled AR space are obtained, as well as the first positioning image of the target space is obtained. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information.
[0007] Based on the first positioning image, the point cloud information, and the first pose information, the first positioning pose information of the first positioning image in the scale-free AR space is determined.
[0008] Based on the first positioning pose information, the first pose information and the second pose information, a first coordinate system transformation relationship is determined. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference.
[0009] Having obtained the second positioning pose information of the target object in the second scaled AR space, the third positioning pose information of the target object in the first scaled AR space is determined based on the second positioning pose information and the first coordinate system transformation relationship.
[0010] Secondly, embodiments of this application provide a positioning device, the device comprising:
[0011] The first acquisition module is used to acquire spatial information of the scale-free augmented reality (AR) space corresponding to the target space and spatial information of the first scaled AR space, as well as to acquire the first positioning image of the target space. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information.
[0012] The first determining module is used to determine the first positioning pose information of the first positioning image in the scale-free AR space based on the first positioning image, the point cloud information and the first pose information.
[0013] The second determining module is used to determine a first coordinate system transformation relationship based on the first positioning pose information, the first pose information and the second pose information. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference.
[0014] The third determining module is used to determine the third positioning pose information of the target object in the first scaled AR space based on the second positioning pose information and the first coordinate system transformation relationship, after obtaining the second positioning pose information of the target object in the second scaled AR space.
[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, they implement the steps of the positioning method as described in the first aspect.
[0016] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the positioning method as described in the first aspect.
[0017] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the positioning method as described in the first aspect.
[0018] In this embodiment, spatial information of a scale-free augmented reality (AR) space and spatial information of a first scaled AR space corresponding to a target space are obtained, as well as a first positioning image of the target space. The spatial information of the scale-free AR space includes point cloud information and a first pose information, and the spatial information of the first scaled AR space includes a second pose information. Based on the first positioning image, the point cloud information, and the first pose information, a first positioning pose information of the first positioning image in the scale-free AR space is determined. Based on the first positioning pose information, the first pose information, and the second pose information, a first coordinate system transformation relationship is determined. This first coordinate system transformation relationship indicates the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space, where the second scaled AR space is a space constructed with the first positioning image as a reference. When the second positioning pose information of the target object in the second scaled AR space is obtained, a third positioning pose information of the target object in the first scaled AR space is determined based on the second positioning pose information and the first coordinate system transformation relationship. In particular, for multi-user, cross-session scenarios, by first establishing a scale-free AR space and then restoring the scale, all different scaled AR spaces are unified into the same first scaled AR space, enabling all users to share AR content in the same space, thus solving the problem of asynchronous content states in AR spaces for multiple users and different sessions. Attached Figure Description
[0019] Figure 1 One of the flowcharts of the positioning method provided in the embodiments of this application;
[0020] Figure 2 A schematic diagram of the execution system architecture for the positioning method provided in the embodiments of this application;
[0021] Figure 3 A second schematic flowchart illustrating the positioning method provided in this application embodiment;
[0022] Figure 4 This is a schematic diagram of the positioning device provided in the embodiments of this application;
[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0026] The positioning method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios. Figure 1 This is a flowchart illustrating the positioning method. Figure 2 This is a schematic diagram of the execution system architecture of the positioning method.
[0027] like Figure 1 As shown, the positioning method in this application embodiment includes the following steps:
[0028] Step 101: Obtain spatial information of the scale-free augmented reality (AR) space and the spatial information of the first scaled AR space corresponding to the target space, and obtain the first positioning image of the target space. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information.
[0029] In this context, the target space refers to a real-world space, such as a room or a stadium. The characteristic of scale-free AR space is that the scale between the scale-free AR space and the target space is unknown; that is, the conversion relationship between distances in the scale-free AR space and the real world is unknown. Conversely, the characteristic of scaled AR space is that the scale between the scaled AR space and the target space is determined. In this application, the scale between the scaled AR space and the target space can be 1:1.
[0030] Point clouds are data structures in three-dimensional space that use a set of discrete points to represent the features of an object's surface or environment.
[0031] Pose is the collective term for an object's position (translation) and orientation in three-dimensional space.
[0032] The aforementioned point cloud information and pose information refer to the collection of all relevant information about point clouds and poses included in the AR space.
[0033] The following description is provided regarding the acquisition of the first localization image of the target space:
[0034] The scale-free AR space and the first scaled AR space are AR spaces pre-constructed for the target space. When a user enters the target space, the device can collect images of the target space. Understandably, the images collected by the device are a part of the target space. For example, when a user enters a room with a mobile phone and opens an AR application, the mobile phone camera can capture an image corresponding to the user's current position and the angle the camera is facing. This is the first positioning image.
[0035] Step 102: Based on the first positioning image, the point cloud information, and the first pose information, determine the first positioning pose information of the first positioning image in the scale-free AR space.
[0036] Due to the characteristics of point cloud information, point cloud information that matches the first positioning image can be found in the point cloud information. Then, based on the correspondence between the point cloud information and the first pose information, the pose information corresponding to the first positioning image can be determined from the first pose information, thus determining the first positioning pose information of the first positioning image in the scale-free AR space.
[0037] Step 103: Based on the first positioning pose information, the first pose information and the second pose information, determine the first coordinate system transformation relationship. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference.
[0038] The following description is provided for the first scaled AR space and the second scaled AR space:
[0039] The first scaled AR space is a pre-constructed AR space based on the target space. It can be understood as the space obtained from the first scaled space construction of the target space, with its coordinate origin potentially being the center of the target space. The second scaled AR space is constructed after the first scaled space is built. In scenarios involving multiple users (each user opening the AR application after entering the target space) or across sessions (a user opening a new session in the target space (or reopening the application after exiting)), the second scaled AR space is constructed based on the user's standing position. Understandably, because the user's standing position differs, the coordinate origin of the second scaled AR space is different from that of the first scaled AR space. Without additional processing, because the coordinate origin of the second scaled AR space differs from that of the first scaled AR space, the AR space content seen by each user will be different in multi-user and cross-session scenarios, resulting in positional shifts.
[0040] In this step, the transformation relationship between the first scaled AR space and the scale-free AR space can be determined based on the first pose information and the second pose information. Since the position of the first localization image in the second scaled AR space is known, and combined with the first localization pose information of the first localization image in the scale-free AR space, the transformation relationship between the first scaled AR space and the second scaled AR space can be determined through the "bridge" of the scale-free AR space.
[0041] Step 104: After obtaining the second positioning pose information of the target object in the second scaled AR space, based on the second positioning pose information and the first coordinate system transformation relationship, determine the third positioning pose information of the target object in the first scaled AR space.
[0042] In this step, the target object can be a virtual object that the user wants to place in the AR space. The user can place the target object in the current second-scaled AR space to obtain second positioning information. Based on the first coordinate system transformation relationship between the first-scaled AR space and the second-scaled AR space, the target object can be transformed into third positioning pose information in the first-scaled AR space. In subsequent multi-user and cross-session scenarios, the pose of the target object in the first-scaled AR space can be viewed simultaneously, instead of being viewed based on different AR spaces, thus achieving content synchronization in the AR space.
[0043] In the above approach, for multi-user, cross-session scenarios, by first establishing a scale-free AR space and then restoring the scale, all different scaled AR spaces are unified into the same first scaled AR space, enabling all users to share AR content in the same space. This solves the problem of asynchronous content states in AR spaces for multiple users and different sessions.
[0044] Optionally, determining the transformation relationship of the first coordinate system based on the first positioning pose information, the first pose information, and the second pose information includes:
[0045] Based on the first pose information and the second pose information, a second coordinate system transformation relationship is determined between the scale-free AR space and the first scaled AR space;
[0046] Based on the first positioning pose information and the second coordinate system transformation relationship, the fourth positioning pose information of the first positioning image in the first scaled AR space is determined.
[0047] Based on the fourth positioning pose information and the second pose information, the transformation relationship of the first coordinate system is determined.
[0048] Please refer to the above. Figure 2 as well as Figure 3 In the mapping stage and the anchor placement stage, in the mapping stage, a scale-free AR space (which can be constructed through Visual Positioning Service (VPS) map construction and (3D Gaussian Splatting, 3DGS) construction) and a first scaled AR space (which can be constructed through related technologies, referred to as UniCity in this application, and the corresponding second pose information can be referred to as UniCity-map-pose) are constructed for the target space. The second coordinate system transformation relationship between the scale-free AR space and the first scaled AR space is calculated based on vps-pose and unicity-map-pose, namely the Sim(3) similarity transformation relationship sim3, sim3 includes rotation relationship R, translation relationship T, and scale relationship S.
[0049] During the anchor placement phase, in a target space where no target object anchor has been placed, the user enters the project space for positioning and uploads the first positioning image (query image) to the cloud. The Visual Positioning Service (VPS) system performs online real-time positioning (i.e., in the above embodiment, based on the first positioning image, the point cloud information, and the first pose information, the first positioning pose information (location pose) of the first positioning image in the scale-free AR space is determined, and in this embodiment, based on the first pose information and the second pose information, the second coordinate system transformation relationship between the scale-free AR space and the first scaled AR space is determined; based on the first positioning pose information and the second coordinate system transformation relationship, the fourth positioning pose information of the first positioning image in the first scaled AR space is determined), thus obtaining the fourth positioning pose information (UniCity-map-pose).
[0050] Combining step 104 above, UniCity opens a new second-scaled AR spatial coordinate system for each session. Based on unicity-new-pose (second pose information) and unicity-map-pose (fourth positioning pose information), the transformation relationship △T_1 between the second-scaled AR spatial coordinate system unicity new session and the first-scaled AR spatial coordinate system unicity map session is obtained. All poses under unicity new session are transformed to the unicity map coordinate system through △T_1. After the user freely selects to place the anchor, the anchor pose is transformed to the unicity map coordinate system through △T_1, and the cloud anchor management system records the anchor pose in the unicity map coordinate system.
[0051] By determining the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space through this embodiment, the target object can be unified under the first scaled AR space, so as to synchronize the AR content of different users.
[0052] Optionally, obtaining the spatial information of the scale-free augmented reality (AR) space corresponding to the target space and the spatial information of the first scaled AR space includes:
[0053] Acquire scanned images of the target space;
[0054] AR space is constructed based on the scanned image to obtain the scale-free AR space, the first scaled AR space, and the spatial information of the scale-free AR space and the first scaled AR space.
[0055] Optionally, determining the first positioning pose information of the first positioning image in the scale-free AR space based on the first positioning image, the point cloud information, and the first pose information includes:
[0056] The point cloud information is vectorized and stored to obtain a global descriptor set and a local feature set. The global descriptor set is used to store the global feature vector of the image, and the local feature set is used to store the local feature vectors corresponding to preset points in the image.
[0057] The first pose information is vectorized and stored to obtain a set of spatial locations, which stores the feature vectors corresponding to the AR spatial locations.
[0058] Obtain the first global descriptor vector and the first local feature vector of the first localized image;
[0059] Select a second set of global descriptor vectors from the global descriptor set whose similarity to the first global descriptor vector is greater than a first preset similarity.
[0060] Based on the second global descriptor vector set and the local feature set, a second local feature set is determined. The second local feature set is the set of local features in the local feature set that correspond to the candidate image set. The candidate image set is the set of images corresponding to the second global descriptor vector set.
[0061] Based on the first local feature vector and the second local feature set, a third local feature vector is determined. The third local feature vector is a feature vector in the second local feature set whose similarity to the first local feature vector is greater than a second preset similarity.
[0062] Based on the third local feature vector and the set of spatial locations, the first positioning pose information is determined.
[0063] In this embodiment, compared to storing AR spatial data built on a traditional VPS in H5 (Hierarchical DataFormat version 5) file format, in a multi-user environment, the traditional storage method limits data processing performance. As the number of users increases, the file size grows, slowing down data query and location speeds. In this embodiment, through... Figure 2 The execution system shown performs vectorization and structured storage on the multi-source heterogeneous data generated by the VPS. Specifically, the process involves constructing a unified vector representation from the image local feature vectors stored in the original features.h5 file, the global descriptor vectors in the original global-netvlad.h5 file, the 3D point cloud coordinate vectors in the original points3D.txt file, and the camera extrinsic parameters and 2D feature point pixel position vectors in the original images.txt file, using multimodal fusion technology.
[0064] A multi-set storage architecture is established in the Milvus vector database, including a global descriptor set, a local feature set, and a spatial location set. The global descriptor set stores global feature vectors of the image and uses an HNSW index structure to achieve efficient approximate nearest neighbor search. The local feature set stores local feature vectors corresponding to key points of the image and uses an IVF_PQ index to balance retrieval accuracy and speed. The spatial location set stores three-dimensional spatial anchor point information and uses a DISKANN index to support fast retrieval of large-scale spatial data.
[0065] In this embodiment, the system employs a hierarchical vector retrieval strategy to significantly improve visual localization speed. First, in the coarse-grained global retrieval stage, the first global descriptor vector of the first localized image is used to quickly filter out a second set of global descriptor vectors with a similarity greater than a first preset similarity from the global descriptor set. Second, in the fine-grained local matching stage, detailed feature matching is performed on the candidate image set in the local feature set, and the SuperGlue / LightGlue algorithm is used to establish efficient feature correspondence.
[0066] The query vector generation process includes: extracting the global descriptor vector and local feature vector of the query image in real time using a deep learning model, and constructing a multimodal query vector representation including timestamps. Global retrieval adopts an approximate nearest neighbor search based on L2 distance metric, while local matching adopts a bidirectional matching strategy based on feature similarity to ensure the accuracy and robustness of the matching results.
[0067] The system implements an adaptive index selection mechanism, dynamically configuring the optimal index strategy based on the data size. For small datasets, the HNSW index is used to balance memory usage and retrieval efficiency; for medium-sized datasets, the IVF_PQ index is used to provide high-quality approximate nearest neighbor search; and for massive datasets with hundreds of millions of records, the DISKANN index is used to support large-scale disk-level vector retrieval.
[0068] To improve system performance, a multi-level caching system is established, including: a first-level cache that stores hot query vectors and uses a Redis in-memory database to achieve millisecond-level response; a second-level cache that stores frequently accessed feature vectors and uses memory mapping technology to achieve fast access; and a third-level cache that uses an LRU eviction policy to store the most recent query results, significantly reducing the response latency of repeated queries.
[0069] Optionally, after obtaining the second positioning pose information of the target object in the second scaled AR space, and determining the third positioning pose information of the target object in the first scaled AR space based on the second positioning pose information and the first coordinate system transformation relationship, the method further includes:
[0070] In the case of acquiring the second positioning image, based on the second positioning image and the point cloud information, the fifth positioning pose information of the second positioning image in the scale-free AR space is determined;
[0071] Based on the fifth positioning pose information, the first pose information and the second pose information, a third coordinate system transformation relationship is determined. The third coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the third scaled AR space. The third scaled AR space is a space constructed with the second positioning image as a reference.
[0072] Based on the third positioning pose information and the third coordinate system transformation relationship, the sixth positioning pose information of the target object in the third scaled AR space is determined;
[0073] The target object is rendered based on the sixth positioning pose information.
[0074] Optionally, determining the transformation relationship of the third coordinate system based on the fifth positioning pose information, the first pose information, and the second pose information includes:
[0075] Based on the first pose information and the second pose information, a second coordinate system transformation relationship is determined between the scale-free AR space and the first scaled AR space;
[0076] Based on the fifth positioning pose information and the second coordinate system transformation relationship, the seventh positioning pose information of the second positioning image in the first scaled AR space is determined;
[0077] Based on the seventh positioning pose information and the second pose information, the transformation relationship of the third coordinate system is determined.
[0078] Please see Figure 3 The experience phase shown is similar in principle to the anchor placement phase. Within the target space where anchors have been placed, the user enters the target space for localization. Similarly, the second localization image (query image) is uploaded to the cloud for VPS localization, obtaining the location pose in the VPS coordinate system. This pose is then transformed to the Unicity map coordinate system using Sim3 transformation. Subsequently, based on the Unicity-new-pose and Unicity-map-pose obtained from this localization, the transformation relationship ΔT_2 between the current third-scale AR space and the first-scale AR space coordinate system is calculated. By transforming all poses in the third-scale AR space to the first-scale AR space Unicity map coordinate system using ΔT_2, the anchors and their target objects located in this coordinate system can be correctly rendered. By unifying all poses to the Unicity map coordinate system, all users can share the same augmented reality space.
[0079] In addition, it should be noted that, Figure 2 The execution system shown has the following characteristics:
[0080] Deep integration of cross-platform SDK: Based on the native Android / iOS framework and Unity engine, combined with WeChat Mini Program plugins, it realizes unified scheduling of AR capabilities across multiple terminals, supporting full coverage of scenarios from mobile terminals to lightweight Mini Programs; it builds a multimodal fusion cloud-native augmented reality service system, which makes full use of modular design and distributed storage optimization in architecture, taking into account data security, scalability and maintainability, and providing solid support for the efficient deployment of large-scale AR services;
[0081] High-precision visual positioning: By integrating 2D-3D feature matching (SuperPoint / aliked / xfeat, SuperGlue / LightGlue, etc.), geometric verification (RANSAC / PnP), and VPS services, it achieves low-threshold centimeter-level real-time positioning and innovatively introduces a vector database to accelerate large-scale point cloud retrieval; it uses the Milvus vector database to store and manage data, enabling efficient storage, management, and retrieval of unstructured data represented in vector form, allowing users to quickly retrieve high-precision real-time positioning when requesting location services;
[0082] Cloud-edge collaborative architecture: It adopts Docker+Kubernetes containerized deployment and automatic scaling mechanism to achieve rapid response of core VPS functions such as image retrieval, feature matching, and pose calculation in high-concurrency environment. It combines relational database (metadata management) and file storage system (image / point cloud) to build elastic expansion capability that supports hundreds of millions of feature indexes.
[0083] Dynamic spatial awareness: Persistent alignment of virtual and real spaces is achieved through the Anchor management system and VPS, and the IMU-VIO tightly coupled algorithm is integrated to maintain tracking stability in low-texture scenes.
[0084] The entire system operates on a cyclical workflow, from map creation to user experience and project status updates, with each stage involving data uploading, storage, processing, and computation. The mobile application and WeChat mini-program handle user interaction and initial data processing, while the spatial-xr-server service acts as an intermediate layer, responsible for data storage and some computation. The VPS service provides deeper data processing and computational support, primarily handling spatial map creation and pose calculation. The system's architecture fully leverages modular design and distributed storage optimization, balancing data security, scalability, and maintainability, providing solid support for the efficient deployment of large-scale AR services.
[0085] like Figure 4 As shown, this application embodiment also provides a positioning device 400, which includes:
[0086] The first acquisition module 401 is used to acquire spatial information of the scale-free augmented reality (AR) space corresponding to the target space and spatial information of the first scaled AR space, as well as to acquire the first positioning image of the target space. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information.
[0087] The first determining module 402 is used to determine the first positioning pose information of the first positioning image in the scale-free AR space based on the first positioning image, the point cloud information and the first pose information.
[0088] The second determining module 403 is used to determine a first coordinate system transformation relationship based on the first positioning pose information, the first pose information and the second pose information. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference.
[0089] The third determining module 404 is used to determine the third positioning pose information of the target object in the first scaled AR space based on the second positioning pose information and the first coordinate system transformation relationship, after obtaining the second positioning pose information of the target object in the second scaled AR space.
[0090] Optionally, the second determining module 403 includes:
[0091] The first determining unit is used to determine the second coordinate system transformation relationship between the scale-free AR space and the first scaled AR space based on the first pose information and the second pose information.
[0092] The second determining unit is used to determine the fourth positioning pose information of the first positioning image in the first scaled AR space based on the first positioning pose information and the second coordinate system transformation relationship.
[0093] The third determining unit is used to determine the transformation relationship of the first coordinate system based on the fourth positioning pose information and the second pose information.
[0094] Optionally, the first acquisition module 401 includes:
[0095] The first acquisition unit is used to acquire a scanned image of the target space;
[0096] A spatial construction unit is used to construct AR space based on the scanned image to obtain the scale-free AR space, the first scaled AR space, and the spatial information of the scale-free AR space and the spatial information of the first scaled AR space.
[0097] The point cloud information includes a global descriptor set and a local feature set. The first pose information includes a spatial position set. The global descriptor set is used to store global feature vectors of the image. The local feature set is used to store local feature vectors corresponding to preset points in the image. The spatial position set stores feature vectors corresponding to AR spatial positions.
[0098] Optionally, the first determining module 402 includes:
[0099] The second acquisition unit is used to acquire the first global descriptor vector and the first local feature vector of the first localized image;
[0100] The first selection unit is used to select from the global descriptor set a second set of global descriptor vectors whose similarity to the first global descriptor vector is greater than a first preset similarity.
[0101] The fourth determining unit is used to determine a second local feature set based on the second global descriptor vector set and the local feature set, wherein the second local feature set is a set of local features in the local feature set that correspond to the candidate image set, and the candidate image set is a set of images corresponding to the second global descriptor vector set;
[0102] The fifth determining unit is used to determine a third local feature vector based on the first local feature vector and the second local feature set, wherein the third local feature vector is a feature vector in the second local feature set whose similarity to the first local feature vector is greater than a second preset similarity.
[0103] The sixth determining unit is used to determine the first positioning pose information based on the third local feature vector and the spatial position set.
[0104] Optionally, the positioning device 400 further includes:
[0105] The fourth determining module is used to determine the fifth positioning pose information of the second positioning image in the scale-free AR space based on the second positioning image and the point cloud information when the second positioning image is acquired.
[0106] The fifth determining module is used to determine the third coordinate system transformation relationship based on the fifth positioning pose information, the first pose information and the second pose information. The third coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the third scaled AR space. The third scaled AR space is a space constructed with the second positioning image as a reference.
[0107] The sixth determining module determines the sixth positioning pose information of the target object in the third scaled AR space based on the third positioning pose information and the third coordinate system transformation relationship.
[0108] The rendering module renders the target object based on the sixth positioning pose information.
[0109] Optionally, the fifth determining module includes:
[0110] The seventh determining unit determines the second coordinate system transformation relationship between the scale-free AR space and the first scaled AR space based on the first pose information and the second pose information.
[0111] The eighth determining unit determines the seventh positioning pose information of the second positioning image in the first scaled AR space based on the fifth positioning pose information and the second coordinate system transformation relationship.
[0112] The ninth determining unit determines the transformation relationship of the third coordinate system based on the seventh positioning pose information and the second pose information.
[0113] It should be noted that the positioning device 400 provided in this application embodiment can achieve the following: Figure 1 The entire technical process of the positioning method shown in the embodiment, and the same technical effect, will not be described again here to avoid repetition.
[0114] The positioning device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. Non-mobile electronic devices can also be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the scope of the device.
[0115] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501 and a memory 502. The memory 502 stores a program or instructions that can run on the processor 501. When the program or instructions are executed by the processor 501, they implement the various steps of the above-described positioning method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0116] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0117] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described positioning method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0118] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0119] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the positioning method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0122] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A positioning method, characterized in that, The method includes: The spatial information of the scale-free augmented reality (AR) space corresponding to the target space and the spatial information of the first scaled AR space are obtained, as well as the first positioning image of the target space is obtained. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information. Based on the first positioning image, the point cloud information, and the first pose information, the first positioning pose information of the first positioning image in the scale-free AR space is determined. Based on the first positioning pose information, the first pose information and the second pose information, a first coordinate system transformation relationship is determined. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference. Having obtained the second positioning pose information of the target object in the second scaled AR space, the third positioning pose information of the target object in the first scaled AR space is determined based on the second positioning pose information and the first coordinate system transformation relationship.
2. The method as described in claim 1, characterized in that, The step of determining the transformation relationship of the first coordinate system based on the first positioning pose information, the first pose information, and the second pose information includes: Based on the first pose information and the second pose information, a second coordinate system transformation relationship is determined between the scale-free AR space and the first scaled AR space; Based on the first positioning pose information and the second coordinate system transformation relationship, the fourth positioning pose information of the first positioning image in the first scaled AR space is determined. Based on the fourth positioning pose information and the second pose information, the transformation relationship of the first coordinate system is determined.
3. The method as described in claim 1, characterized in that, The acquisition of spatial information of the scale-free augmented reality (AR) space corresponding to the target space and spatial information of the first scaled AR space includes: Acquire a scanned image of the target space.
4. The method as described in claim 3, characterized in that, The step of determining the first positioning pose information of the first positioning image in the scale-free AR space based on the first positioning image, the point cloud information, and the first pose information includes: The point cloud information is vectorized and stored to obtain a global descriptor set and a local feature set. The global descriptor set is used to store the global feature vector of the image, and the local feature set is used to store the local feature vectors corresponding to preset points in the image. The first pose information is vectorized and stored to obtain a set of spatial locations, which stores the feature vectors corresponding to the AR spatial locations. Obtain the first global descriptor vector and the first local feature vector of the first localized image; Select a second set of global descriptor vectors from the global descriptor set whose similarity to the first global descriptor vector is greater than a first preset similarity. Based on the second global descriptor vector set and the local feature set, a second local feature set is determined. The second local feature set is the set of local features in the local feature set that correspond to the candidate image set. The candidate image set is the set of images corresponding to the second global descriptor vector set. Based on the first local feature vector and the second local feature set, a third local feature vector is determined. The third local feature vector is a feature vector in the second local feature set whose similarity to the first local feature vector is greater than a second preset similarity. Based on the third local feature vector and the set of spatial locations, the first positioning pose information is determined.
5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the second positioning pose information of the target object in the second scaled AR space, and determining the third positioning pose information of the target object in the first scaled AR space based on the second positioning pose information and the first coordinate system transformation relationship, the method further includes: In the case of acquiring the second positioning image, based on the second positioning image and the point cloud information, the fifth positioning pose information of the second positioning image in the scale-free AR space is determined; Based on the fifth positioning pose information, the first pose information and the second pose information, a third coordinate system transformation relationship is determined. The third coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the third scaled AR space. The third scaled AR space is a space constructed with the second positioning image as a reference. Based on the third positioning pose information and the third coordinate system transformation relationship, the sixth positioning pose information of the target object in the third scaled AR space is determined; The target object is rendered based on the sixth positioning pose information.
6. The method as described in claim 5, characterized in that, The step of determining the transformation relationship of the third coordinate system based on the fifth positioning pose information, the first pose information, and the second pose information includes: Based on the first pose information and the second pose information, a second coordinate system transformation relationship is determined between the scale-free AR space and the first scaled AR space; Based on the fifth positioning pose information and the second coordinate system transformation relationship, the seventh positioning pose information of the second positioning image in the first scaled AR space is determined; Based on the seventh positioning pose information and the second pose information, the transformation relationship of the third coordinate system is determined.
7. A positioning device, characterized in that, The device includes: The first acquisition module is used to acquire spatial information of the scale-free augmented reality (AR) space corresponding to the target space and spatial information of the first scaled AR space, as well as to acquire the first positioning image of the target space. The spatial information of the scale-free AR space includes point cloud information and first pose information, and the spatial information of the first scaled AR space includes second pose information. The first determining module is used to determine the first positioning pose information of the first positioning image in the scale-free AR space based on the first positioning image, the point cloud information and the first pose information. The second determining module is used to determine a first coordinate system transformation relationship based on the first positioning pose information, the first pose information and the second pose information. The first coordinate system transformation relationship is used to indicate the coordinate system transformation relationship between the first scaled AR space and the second scaled AR space. The second scaled AR space is a space constructed with the first positioning image as a reference. The third determining module is used to determine the third positioning pose information of the target object in the first scaled AR space based on the second positioning pose information and the first coordinate system transformation relationship, after obtaining the second positioning pose information of the target object in the second scaled AR space.
8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the positioning method as described in any one of claims 1 to 6.
9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the positioning method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the positioning method as described in any one of claims 1 to 6.