Spatial pose anchoring method, apparatus, and XR device

Abstract

The embodiment of the application discloses a spatial pose anchoring method and device and an XR device, relates to the technical field of computer vision, and is used for solving the problem that in the existing spatial pose anchoring mode, it is difficult to accurately realize global expression of a specific object attitude and multiple device anchoring references are not unified. The method comprises the following steps: acquiring current frame image data collected by a sensor, matching the current frame image data with target image data, and obtaining homonymous point pairs; performing spatial pose solving on the current frame image data according to a target object coordinate system and the homonymous point pairs, and realizing spatial pose anchoring; wherein the target coordinate system is obtained by constructing the target image data based on an intelligent device in a current environment.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a spatial pose anchoring method, apparatus and XR device. Background Technology

[0002] With the development of science and technology, many technological fields such as augmented reality, virtual reality, and mixed reality require virtual-real interaction in real-world scenarios, accurately overlaying virtual objects onto specific locations in the real world. This necessitates precise knowledge of the device's pose relative to the real-world environment. Therefore, spatial pose anchoring for intelligent devices is a crucial aspect in many fields.

[0003] Existing intelligent devices primarily rely on a combination of multiple sensors for state estimation to acquire information such as the device's position, velocity, and attitude. For example, SLAM technology acquires state information, but this information uses a local coordinate system. On one hand, this coordinate system is unsuitable for globally representing the attitude of a specific object; on the other hand, if the device's positioning information is lost, the current reference system needs to be repositioned to a specific global coordinate system to uniformly represent all states. Furthermore, SLAM-based visual odometry and GPS positioning often depend on global environmental features. This dependence is susceptible to interference from the natural environment, resulting in low positioning accuracy and ineffective anchoring. In addition, existing intelligent devices may construct independent coordinate systems based on their own SLAM, leading to inconsistencies in pose calibration between multiple intelligent devices and resulting in low overall consistency in inter-device collaboration. Summary of the Invention

[0004] To address the problems of existing spatial pose anchoring methods, which struggle to accurately represent the posture of a specific object globally and lack unified anchoring references across multiple devices, this application provides a spatial pose anchoring method, apparatus, and XR device through one or more embodiments.

[0005] One or more embodiments of this application employ the following technical solutions:

[0006] In a first aspect, one or more embodiments of this application provide a spatial pose anchoring method, the method comprising:

[0007] Acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain corresponding point pairs;

[0008] Based on the target object coordinate system and the corresponding point pair, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring; wherein, the target coordinate system is obtained by constructing the target image data based on the intelligent device in the current environment.

[0009] Optionally, in one or more embodiments of this application, spatial pose determination is performed on the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring, specifically including:

[0010] According to the construction rules of the target object coordinate system, the current frame image data within the same point is mapped to obtain the matching relationship of the same point pair;

[0011] Based on the matching relationship of the same point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship.

[0012] Optionally, in one or more embodiments of this application, based on the matching relationship of the corresponding point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship, specifically including:

[0013] Based on the matching relationship and sensor intrinsic parameters, the transformation matrix from the current sensor coordinate system to the target object coordinate system is determined, and the spatial pose transformation relationship is obtained.

[0014] Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined; wherein, the anchoring type includes: single-device anchoring and multi-device anchoring;

[0015] Based on the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each of the intelligent devices, the spatial pose anchoring of each of the intelligent devices is realized.

[0016] Optionally, in one or more embodiments of this application, after solving the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring, the method further includes:

[0017] Acquire historical frame image data to determine the historical pose information corresponding to the historical frame image data;

[0018] Based on the historical pose information and the current pose information corresponding to the current frame image data, as well as the spatial pose transformation relationship, the cumulative transformation relationship is determined;

[0019] Based on the cumulative transformation relationship, the historical frame image data is transformed into the target object coordinate system.

[0020] Optionally, in one or more embodiments of this application, before performing spatial pose calculation on the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring, the method further includes:

[0021] Determine the current environment in order to load target image data of a specific target object within that current environment;

[0022] Establish a mapping relationship between the pixel coordinates and physical coordinates of the target image data, so as to obtain the target object coordinate system fixed to the specific target object according to the mapping relationship.

[0023] Optionally, in one or more embodiments of this application, after loading target image data of a specific target object in the current environment, the method further includes:

[0024] The target image data is denoised, and edge detection is performed on the denoised target image data to obtain the target candidate region of the specific target object;

[0025] Based on the local peak detection results of the target candidate region, the target area of ​​the specific target object is determined;

[0026] Feature points of the target region are extracted to generate a first feature descriptor based on the feature points.

[0027] Optionally, in one or more embodiments of this application, matching the current frame image data with the target image data to obtain corresponding point pairs specifically includes:

[0028] Obtain keyframe information corresponding to each current frame image data, and identify whether the current frame image contains the specific target object; wherein, the keyframe information includes: pose information and a second feature descriptor;

[0029] If so, feature matching is performed based on the first feature descriptor and the second feature descriptor to obtain the same point pairs between the target image data and the current frame image data.

[0030] Optionally, in one or more embodiments of this application, after acquiring the current frame image data collected by the sensor, the method further includes:

[0031] Monitor upper-layer application commands to obtain the target scenario corresponding to the upper-layer application commands;

[0032] Based on the target scene and the current scene of the current frame image data, determine whether the current frame image data should undergo world-frame transformation;

[0033] If so, the pose information of the current frame image data is transformed according to the transformation relationship corresponding to the world system transformation.

[0034] Secondly, one or more embodiments of this application provide a spatial pose anchoring device, the device comprising:

[0035] The matching unit is used to acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain the same point pair;

[0036] An anchoring unit is used to solve the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring; wherein, the target coordinate system is obtained by constructing the target image data based on the intelligent device in the current environment.

[0037] Thirdly, one or more embodiments of this application provide an XR device, the device comprising:

[0038] At least one processor; and,

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: perform any of the spatial pose anchoring methods described above.

[0041] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0042] By utilizing the corresponding point pairs between the current frame image and the target image for pose determination, localization can be directly achieved based on the spatial features of the target object, reducing interference from irrelevant factors in the environment and thus improving the accuracy of spatial pose anchoring for intelligent devices. Furthermore, through precise mapping between the target object coordinate system and the corresponding point pairs, stable transformation from a local coordinate system to a global coordinate system is achieved for the intelligent device, solving the core problem of inconsistent pose references in multi-device collaboration. Constructing a target coordinate system based on the target image data of a specific target object facilitates the subsequent use of this target coordinate system as a global reference system, enabling the pose information of the intelligent device to be transformed into this coordinate system for unified representation. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0044] Figure 1 A schematic diagram of a spatial pose anchoring architecture provided in an embodiment of this application;

[0045] Figure 2A schematic flowchart illustrating a spatial pose anchoring method provided in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the overall process of a spatial pose anchoring method provided in an embodiment of this application;

[0047] Figure 4 This application provides a schematic flowchart of a method for spatial pose anchoring of various smart devices.

[0048] Figure 5 This is a schematic diagram of the structure of a spatial pose anchoring device provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an XR device provided in an embodiment of this application. Detailed Implementation

[0050] This application provides a spatial pose anchoring method, apparatus, and XR device.

[0051] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0052] This application provides a spatial pose anchoring method, apparatus, and XR device.

[0053] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0055] The spatial pose anchoring method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, such as Figure 1As shown, the application environment may include: a smart device terminal 101, a sensor 102, and a specific target object 103. The smart device terminal 101 can be an augmented reality (AR) / virtual reality (VR) headset, a robot, a drone, or other hardware devices that require alignment with the actual physical scene or virtual-real interaction, or software running on the aforementioned hardware devices. Furthermore, the smart device terminal can be a single smart device terminal or multiple smart device terminals working collaboratively in different scenarios. The sensor 102 communicates with the smart device terminal 101 and can be used to collect data from the specific target object 103 in the current environment, obtaining the current frame image data. Then, based on the communication between the sensor 102 and the smart device terminal 101, the current frame image data is uploaded to the smart device terminal 101, enabling the smart device terminal 101 to match the pre-stored target image data in the current environment with the current frame image data collected by the sensor 102 to obtain corresponding point pairs. The smart sensor terminal 101 is also used to construct a target object coordinate system based on the pre-stored target image data in the current environment. Then, based on the target object coordinate system and the corresponding point pairs, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring. Specifically, when multiple smart terminal devices 101 are anchored based on the same specific target object, the target object coordinate system can be constructed by any one of the multiple smart terminal devices 101, and other smart terminal devices 101 can obtain and use this target object coordinate system based on the interaction between the smart terminal devices 101.

[0056] like Figure 2 As shown in the figure, this application provides a schematic flowchart of a spatial pose anchoring method. Figure 2The method described can be applied to various scenarios requiring high-precision spatial positioning and attitude estimation, including but not limited to: augmented reality and virtual reality scenarios, such as in AR glasses or mobile AR applications, where precise spatial anchoring of virtual objects is achieved by identifying specific target objects such as QR codes, landmarks, or natural objects, ensuring stable alignment between virtual content and the real world. In VR environments, real-time pose tracking of devices such as headsets and controllers is achieved based on the target object coordinate system, improving immersion and reducing drift issues. In XR applications, the target object coordinate system enables placing a virtual person on a card in a real-world scene for interactive use, and placing a Building Information Model (BIM) on an actual construction site to guide construction and acceptance. In robot navigation and autonomous movement scenarios, for example, robots in unknown environments can correct their pose by identifying preset target objects such as landmarks or special patterns, with multiple robots sharing the same target object coordinate system, enabling collaborative positioning and task allocation processes such as those for warehouse logistics robots. In intelligent unmanned industrial automation scenarios, for example, during AR-assisted maintenance, engineers can wear AR devices and obtain maintenance guidance spatial anchoring based on markers on the device, corresponding virtual coordinates to actual physical coordinates, improving operational accuracy. The spatial pose anchoring method, based on the program execution of a smart device, includes the following steps: S201-S203.

[0057] S201: Acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain a pair of points with the same name.

[0058] The pose information acquired by intelligent devices through technologies such as VIO is based on a local coordinate system and cannot be directly associated with the global reference system in the physical scene, i.e., the target object coordinate system. Therefore, to solve the problem of inconsistent expression of information under different reference systems, a data foundation is provided for the transformation from a local to a global system. In this embodiment, the current frame image data collected in real time by various intelligent devices such as XR glasses and intelligent unmanned devices through sensors such as cameras is acquired. After the current frame image data undergoes the same preprocessing as the target image data, the feature information to be matched is obtained. Through a feature matching algorithm, corresponding point pairs are found in the current frame image data and the target image data. It can be understood that a corresponding point pair is the pixel point corresponding to the same physical feature point in the two images. For example, the pixel coordinates of a corner point on the target object in the pre-stored image and the pixel coordinates of the same corner point in the current frame image form a corresponding point pair. Since the feature points in the target image data have been associated with the target object coordinate system through the mapping from pixel coordinates to physical coordinates, and the feature points of the current frame image correspond to the pixel coordinates of the smart device sensor, the two can be matched to establish a point-to-point relationship between the 3D coordinates in the target object coordinate system and the 2D pixel coordinates of the current frame image.

[0059] Furthermore, in some embodiments, after acquiring the current frame image data collected by the sensor, the method further includes:

[0060] S301: Monitor upper-layer application instructions to obtain the target scenario corresponding to the upper-layer application instructions;

[0061] S302: Based on the target scene and the current scene of the current frame image data, determine whether the current frame image data should undergo world-frame transformation;

[0062] S303: If so, then the pose information of the current frame image data is transformed according to the transformation relationship corresponding to the world system transformation.

[0063] Smart devices may run multiple applications simultaneously, and different applications may require different world coordinate systems due to varying user needs. Furthermore, in certain scenarios, when a user or device switches from one scene to another, such as switching from an indoor AR game to outdoor navigation, it's crucial to quickly determine whether to switch reference frames to avoid pose representation confusion. Therefore, to address these issues, in this embodiment, after acquiring the current frame image data from various sensors, the smart device monitors upper-layer application commands to obtain the target scene corresponding to those commands. Based on whether the target scene corresponding to the current upper-layer application command matches the current scene in the current frame image data, it determines whether a world frame transformation is needed for the current frame image data. For example, in a certain application scenario, if the upper-layer application command determines that the AR glasses are switching from furniture placement mode to outdoor navigation mode, it can be determined that the current scene in the placement mode (with the room as the frame of reference) is inconsistent with the target scene in the outdoor navigation mode (with geographic coordinates as the frame of reference). In this case, the virtual content needs to be automatically re-anchored. Therefore, if a transformation occurs, to ensure real-time pose data synchronization, the pose information of the current frame image data is transformed according to the transformation relationship corresponding to the world frame transformation. The current and historical poses are also uniformly corrected based on the transformation relationship to ensure spatiotemporal continuity.

[0064] This process detects whether a world frame transformation is needed. When necessary, it transforms all pose information corresponding to the current frame image data. By intelligently monitoring upper-layer application commands and dynamically matching the target scene, it achieves flexible switching and seamless integration of world coordinate systems across multiple application scenarios. This effectively solves the problem of pose representation chaos caused by the failure to transform the world frame in a fixed coordinate system in a timely manner. When a scene change requirement is detected, a world frame transformation is automatically triggered, uniformly correcting the current frame and historical pose data. This ensures spatial consistency between virtual and real content, maintains spatiotemporal continuity across scenes, and provides efficient global pose management capabilities for multi-tasking intelligent devices.

[0065] Specifically, in some embodiments, the step S201 above, which involves matching the current frame image data with the target image data to obtain corresponding point pairs, specifically includes:

[0066] S401: Obtain keyframe information corresponding to each current frame image data, and identify whether the current frame image contains the specific target object; wherein, the keyframe information includes: pose information and a second feature descriptor;

[0067] S402: If so, feature matching is performed based on the first feature descriptor and the second feature descriptor to obtain the same point pair between the target image data and the current frame image data.

[0068] To transform the local coordinate system of the current frame image data to the target object coordinate system, and to avoid the problems of high computational cost and susceptibility to interference associated with traditional methods that directly search for the target object in the entire image, as described in the embodiments of this application... Figure 3 The process first acquires keyframe information corresponding to each current frame image data and identifies whether the current frame image contains the specific target object. By using keyframe filtering and a pre-detection mechanism that determines whether a target object is present, the inefficient computation problem of searching for the target object in the entire image is reduced. It should be noted that the keyframe information includes: pose information (VIO pose), feature points, and a second feature descriptor. Then, if the current frame image is determined to contain the specific target object based on the above judgment, feature matching is performed according to the first and second feature descriptors to obtain corresponding point pairs between the target image data and the current frame image data. For example, if the descriptor is of binary type, Hamming distance can be used for matching; for floating-point type descriptors, nearest neighbor ratio testing can be used for matching, thus retaining point pairs with matching scores higher than a threshold to obtain corresponding point pairs.

[0069] S202: Based on the target object coordinate system and the corresponding point pair, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring; wherein, the target coordinate system is obtained by the intelligent device constructing the target image data in the current environment. The intelligent device constructing the target coordinate system can be the executing entity of this application, or it can be pre-constructed by other intelligent devices. It is understood that the method of this application can be that after the executing entity constructs the target coordinate system, it enters the virtual space corresponding to the target coordinate system again through the target image data, or the executing entity enters the virtual space corresponding to the target coordinate system constructed by other terminal devices through the target image data, thereby realizing the synchronization of virtual spaces between multiple devices.

[0070] The poses acquired by current smart devices through VIO or SLAM are typically based on local coordinate systems, which cannot directly represent the position of the target object in the real physical world. This results in inaccurate overlay of virtual content or inability for devices to coordinate localization. Furthermore, traditional methods directly use 2D-2D matching to solve poses, which are susceptible to mismatched points, leading to pose jitter or drift. Additionally, the observed poses of different devices for the same target object may not be directly aligned due to differences in local coordinate systems, making interaction difficult. Therefore, to address these issues, after obtaining corresponding point pairs based on step S201, the 3D-2D correspondence between the target object coordinate system and the corresponding point pairs enables 3D-2D data association of the corresponding points. Once the correct data association is obtained, the relative pose can be solved. In this embodiment, based on the target coordinate system and the corresponding point pairs, the spatial pose of the current frame image data is solved to obtain the relative pose of the current frame image data and the target image data. This enables the target object pose output or the transformation of the reference reference system, completing the spatial pose anchoring of each smart device. This process achieves stable transformation of intelligent devices from a local coordinate system to a global coordinate system through precise mapping between the target object coordinate system and corresponding point pairs, solving the core problem of inconsistent pose references in multi-device collaboration. It should be noted that the target object coordinate system is pre-constructed by the intelligent device or other intelligent devices in the current environment based on the target image data.

[0071] The target object coordinate system constructed here is necessary because when existing smart devices perform spatial pose anchoring, the carrier state information acquired by the smart devices through technologies such as SLAM is usually based on a local coordinate system. This local coordinate system is difficult to directly express the posture of a specific object in the global scene, making it difficult to uniformly manage and interact with the information required by the user's smart devices. Furthermore, when device positioning information is lost, the local coordinate system needs to be realigned to the global reference system, but currently, especially in scenarios involving multiple smart devices working together, there is a lack of an accurate way to align to the global reference system. Therefore, in order to unify the global coordinate system and provide a unified benchmark for the collaboration of multiple smart devices, this embodiment loads the target image data of a specific target object through a specified path of the smart device, and constructs a target coordinate system based on this target image data. Constructing the target coordinate system based on the target image data helps to subsequently use this target coordinate system as a global reference system, allowing the posture information of the smart device to be transformed into this coordinate system for unified expression. The specific target object can be set according to the user needs in different scenarios, such as a QR code, a poster, etc.

[0072] In some feasible embodiments, the target coordinate system can be a right-handed global coordinate system with the x-axis pointing to the right and the y-axis pointing downwards, fixedly connected to the specific target object. The origin of the target object coordinate system is usually set at the center of the target object, and its axis is consistent with the actual scene direction, so that subsequent pose transformations can directly reflect the relative relationship between the device and the physical scene.

[0073] Intelligent devices acquire target image data of specific objects and then construct a target object coordinate system based on the target image data. This can transform specific objects into a global reference system, solving problems such as the inability of local coordinate systems to express globally, difficulties in multi-device collaboration, and the lack of virtual-real interaction benchmarks. It provides a unified coordinate framework for subsequent pose anchoring and a stable and reliable foundation for achieving accurate spatial positioning and interaction.

[0074] In some embodiments, before step S202 above, which calculates the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair, the method further includes:

[0075] S501: Determine the current environment to load target image data of a specific target object in the current environment;

[0076] S502: Establish a mapping relationship between the pixel coordinates and physical coordinates of the target image data, so as to obtain the target object coordinate system fixed to the specific target object according to the mapping relationship.

[0077] The specific target object is a custom marker used to provide a reference for spatial pose anchoring of smart devices. For example, in an AR / VR device, if a virtual object needs to be precisely superimposed onto a specific location in a real-world scene, such as a card, then in order to provide an anchor point for virtual-real fusion and ensure alignment between the virtual content and the physical world, this embodiment determines the current environment of the smart device and loads target image data corresponding to the specific target object pre-stored in the database within that environment. For example, if in an AR / VR device, a virtual object needs to be precisely superimposed onto a specific location in a real-world scene, such as a card, the specific target object in this scene could be a QR code, barcode, or other custom pattern. Then, a mapping relationship between the pixel coordinates of the target image data and the physical coordinates is established, so as to obtain the target object coordinate system fixed to the specific target object based on this mapping relationship. The mapping relationship between the pixel coordinates of the target image data and the physical coordinates can be established as follows:

[0078] ;

[0079] ;

[0080] .

[0081] Where u and v are the coordinates of the detected feature pixels, pixelcols and pixelrows are the pixel width and pixel height of the target image, markerWidth and markerHeight are the actual physical width and actual physical height of the target image data, and x and y are the coordinates in the transformed target object coordinate system.

[0082] By establishing a clear mapping relationship, the target image data of a specific object is associated with its actual physical data, thereby constructing a global coordinate system strictly fixed to the specific target object. This mapping method ensures the accuracy of pose calculation, enabling intelligent devices to achieve stable spatial pose anchoring based on a unified physical reference system. It effectively solves the problem of aligning the local coordinate system with the global scene, while supporting the high-precision requirements of multi-device collaboration and virtual-real interaction. Furthermore, the mapping process described above directly establishes an intuitive correspondence between pixel coordinates (u, v) and target object coordinate system coordinates (x, y) based on the pixel width and height of the target image and the actual physical width and height. This avoids complex spatial geometric calculations, reduces the difficulty and computational cost of coordinate transformation, and facilitates efficient implementation in intelligent devices.

[0083] In one feasible embodiment, during the process of determining the current environment of the smart device and loading target image data of a specific target object in the current environment: since different smart devices may be deployed in different environments and affected by different lighting conditions, the current environment data of each smart device can be obtained. Then, the current environment data is matched with the environment adaptation range corresponding to each specific target object in the pre-set database. Thus, the specific target object is determined through dynamic environment adaptation, which improves the robustness and practicality of detection. This enables the spatial pose anchoring method to be reliably and accurately implemented under different lighting, distance and occlusion scenarios.

[0084] Furthermore, in some embodiments, after loading the target image data of a specific target object in the current environment in step S301, the method further includes:

[0085] S601: Denoise the target image data and perform edge detection on the denoised target image data to obtain the target candidate region of the specific target object;

[0086] S602: Determine the target region of the specific target object based on the local peak detection results of the target candidate region;

[0087] S603: Extract feature points of the target region to generate a first feature descriptor based on the feature points.

[0088] The target image data pre-stored in the storage unit of a smart device may contain interference such as shadows, dirt, or background noise. Therefore, noise reduction processing can be used to eliminate image noise in the target image data, improving the signal-to-noise ratio of the specific target object. Then, Canny or Sobel operators are used to detect edges in the denoised target image data, thereby generating target candidate regions through connected component analysis. The gradient magnitude of the target image data is calculated in each target candidate region to obtain local peak detection results. The local maximum points are retained to determine the target region of the specific target object. For example, irrelevant text and patterns are excluded from a poster, and only the logo area is retained as the target region corresponding to the specific target object. Then, feature points of the target image data are extracted within the target region, and a first feature descriptor is generated based on the feature points, providing highly discriminative data for subsequent 3D-2D matching and ensuring the accuracy of pose calculation. This process reduces interference from non-target regions by determining the target region corresponding to the specific target object and generating the first feature descriptor only based on this target region. Furthermore, it is understandable that, due to the invariance of feature points, it can be ensured that the features of the pre-stored target image data and the current frame image data collected in real time by the smart device can still be effectively matched under different shooting angles and distances, which helps to provide a reliable correspondence for subsequent 3D-2D matching.

[0089] In some feasible embodiments, determining the target region of the specific target object based on the local peak detection results of the target candidate region can be achieved by determining the target region of the specific target object based on the local peak detection results combining the gradient magnitude and phase of the target candidate region. Specifically:

[0090] In complex scenarios, the target candidate regions identified by the edge detection process using Canny or Sobel operators may be misidentified due to noise or texture interference. Therefore, to accurately distinguish between real target edges and noise and improve the reliability of target region localization, this embodiment performs Sobel or Scharr convolution on the target candidate region image to obtain horizontal and vertical gradients. Then, the gradient magnitude corresponding to each target candidate region image is obtained by taking the square root of the sum of the squares of the horizontal and vertical gradients. The gradient phase corresponding to each target candidate region image is obtained based on the arctangent of the ratio of the vertical to the horizontal gradient. The gradient magnitude represents edge intensity; a larger value indicates a higher likelihood of a target boundary. The gradient phase represents edge direction, such as 0° for vertical edges and 90° for horizontal edges. Local peaks in the target candidate region are detected based on the obtained gradient magnitude and gradient phase, and the target region of a specific object is determined based on these local peak detection results. This method upgrades single-dimensional judgment to multi-dimensional collaborative verification through joint analysis of gradient magnitude and phase, significantly improving the accuracy and robustness of target area detection.

[0091] In some embodiments, spatial pose calculation is performed on the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring of each of the smart devices, specifically including:

[0092] S701: Based on the construction rules of the target object coordinate system, map the current frame image data within the corresponding points to obtain the matching relationship of the corresponding point pairs;

[0093] S702: Based on the matching relationship of the same point pairs, obtain the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system, and realize the spatial pose anchoring of each of the smart devices.

[0094] First, based on a pre-constructed target object coordinate system, such as a right-handed coordinate system with the QR code center as the origin, the X-axis pointing to the right, and the Y-axis pointing downwards, the detected 2D image feature points are mapped to 3D physical space through perspective projection. This establishes a precise transformation from pixel coordinates (u,v) to physical coordinates (x,y,z), ensuring that each matching point pair has true physical coordinates. Then, based on the matching relationship of corresponding point pairs, the pose of the smart device relative to the target object coordinate system is solved from the matching relationship. This determines the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system, completing the transformation from the local coordinate system to the global coordinate system and achieving spatial pose anchoring of each smart device. Compared with traditional SLAM positioning methods, this process provides an absolute measurement reference through the target object coordinate system of a specific target object, eliminating accumulated errors.

[0095] In one application scenario, assuming AR glasses are needed to precisely overlay virtual device parameter labels onto a real-world transformer nameplate, a right-handed coordinate system can be constructed using the QR code on the nameplate as the target object: the origin is the center of the QR code; the X-axis is parallel to the long side of the QR code to the right; the Y-axis is parallel to the short side of the QR code downwards; and the Z-axis is perpendicular to the nameplate surface outwards. Then, the AR glasses camera captures the current frame image data, detects the QR code, and extracts keyframe data to construct a feature descriptor. This feature descriptor is matched with the feature descriptor of the target image data to obtain corresponding point pairs. Assuming the corresponding point pairs are the four corner points, a pose transformation can be calculated based on existing pose transformation algorithms and converted into a transformation matrix. This transformation matrix is ​​then used to transform the virtual device label from the local coordinate system to the target object coordinate system, achieving spatial pose anchoring of the smart device.

[0096] Furthermore, such as Figure 4 In some embodiments, step S702 above, based on the matching relationship of the corresponding point pairs, obtains the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system, thereby realizing the spatial pose anchoring of each of the smart devices, specifically including:

[0097] S801: Based on the matching relationship and sensor intrinsic parameters, determine the transformation matrix from the current sensor coordinate system to the target object coordinate system to obtain the spatial pose transformation relationship;

[0098] S802: Based on the shared markers of the target image data and the interaction data of each smart device, determine the anchoring type corresponding to the smart device; wherein, the anchoring type includes: single-device anchoring and multi-device anchoring;

[0099] S803: Based on the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each of the intelligent devices, realize the spatial pose anchoring of each of the intelligent devices.

[0100] Based on steps S301-S303 above, the mapping relationship from pixel coordinates to physical coordinates of the target image data has been determined. Therefore, in the corresponding point pairs, the feature points of the target image data have been converted into 3D coordinates in the target object coordinate system through the mapping relationship from pixel coordinates to physical coordinates, while the feature points of the current frame image correspond to the 2D pixel coordinates of the smart device sensor. Then, by combining the sensor intrinsic parameters and using the existing 3D-2D pose calculation method, the transformation matrix of the current sensor coordinate system relative to the target object coordinate system, i.e., the spatial pose transformation relationship, can be solved. Then, based on the shared markers of the target image data and the interaction data of each device, it is determined whether the anchoring type corresponding to the smart device belongs to single-device anchoring or multi-device anchoring. It can be understood that: single-device anchoring only requires converting the pose information of a single smart device to the target object coordinate system, such as a single XR glasses superimposing a virtual person onto a real card; multi-device anchoring requires unifying the pose information of multiple smart devices to the same target object coordinate system, such as multiple intelligent unmanned devices working collaboratively on a construction site, anchored based on the same BIM model. Then, based on the determined anchoring type, spatial pose transformation relationship, and the local coordinate system corresponding to each intelligent device, the spatial pose anchoring of each intelligent device is realized.

[0101] Current smart devices acquire pose information based on local coordinate systems using technologies like VIO, which cannot be directly correlated with the global reference system of the physical scene. This process, however, solves the problem of accurate conversion from a local to a global system by determining the transformation matrix from the current sensor coordinate system to the target object coordinate system, enabling device pose to be expressed under a unified reference system. By differentiating anchoring types, it also addresses the issue of single anchoring logic being unable to adapt to multiple scenarios, ensuring accurate positioning of a single device and consistency in multi-device collaboration. Furthermore, the local coordinate systems of multiple smart devices are independent and may have accumulated errors; direct correlation would lead to collaboration deviations. However, this process, through a unified transformation of the target object coordinate system, solves the problem of inconsistent pose reference systems among multiple devices, ensuring spatial coordination in inter-device interactions.

[0102] In one feasible embodiment, based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined, specifically including:

[0103] The process involves acquiring the shared tag corresponding to the specific target object to determine the scope of shared devices corresponding to that target object. Simultaneously, interaction data from each smart device is acquired to identify associated smart devices with collaborative tasks in the current scenario. If an associated smart device exists within the shared device scope, its anchoring type is determined to be multi-device anchoring. If the associated device does not belong to the shared device scope, the sharing permissions for that associated device are determined based on the data type of the interaction data and the target object data corresponding to the historical anchoring data of the associated device. If the sharing permissions are greater than preset permissions, the sharing tag is granted to the associated device, and the anchoring type is determined to be multi-device. Conversely, if the sharing permissions are less than preset permissions, the anchoring type is single-device.

[0104] In this feasible embodiment, based on the physical range limitation of shared markers and real-time analysis of interactive data, the system can intelligently identify effective collaborative device groups, ensuring accurate synchronization of pose data. Furthermore, the hierarchical permission strategy, through historical anchoring data learning and data type verification, enables rapid authorization of temporary devices while ensuring system security. This allows the anchoring process to maintain high-precision collaboration of the core device group while dynamically expanding temporary working devices, facilitating interaction between multiple intelligent devices anchored to the same target coordinate system through the same specific target object.

[0105] In one feasible embodiment, during the spatial pose anchoring process of each intelligent device according to the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each intelligent device, for single-device anchoring, the pose information of the intelligent device in the local coordinate system can be directly transformed to the target object coordinate system based on the transformation matrix obtained in step S801. For multi-device anchoring, the pose of each device in its local coordinate system is first transformed to the target object coordinate system through the spatial pose transformation relationship between each device and the target object. Then, the deviation is calibrated based on the interaction data between the devices to ensure that the poses of all devices are consistent in the global reference system, thus achieving multi-device spatial pose anchoring. By distinguishing between single-device and multi-device anchoring types, the technical solution can be adapted to different application scenarios, improving the versatility and practicality of the solution. After transforming the pose information of all devices to the target object coordinate system, the problem that information in the local coordinate system cannot be globally expressed is solved. Moreover, the unified target object coordinate system enables virtual content to be stably superimposed on the real scene. At the same time, multiple devices can achieve precise collaboration based on the same reference system, enhancing the application value of intelligent devices in XR, industrial collaboration, and other scenarios.

[0106] Furthermore, in some embodiments, after solving the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring, the method further includes:

[0107] S901: Acquire historical frame image data to determine the historical pose information corresponding to the historical frame image data;

[0108] S902: Determine the cumulative transformation relationship based on the historical pose information, the current pose information corresponding to the current frame image data, and the spatial pose transformation relationship;

[0109] S903: Based on the cumulative transformation relationship, transform the historical frame image data to the target object coordinate system.

[0110] During movement, the VIO or SLAM system of a smart device continuously outputs pose information in a local coordinate system. After detecting a target and completing pose anchoring for the current frame, the historical frame data remains in the original local coordinate system, leading to inconsistent pose representations between frames and hindering continuous trajectory analysis or global optimization. Furthermore, the historical pose in the local coordinate system may accumulate errors due to sensor drift. Therefore, in this embodiment, based on step S901 above, the historical frame image data of the smart device is obtained. After obtaining the historical pose information corresponding to the historical frame image data, the cumulative transformation relationship is determined by comparing the historical pose information with the current pose information corresponding to the current frame image data, and the spatial pose transformation relationship. Here, historical pose information refers to the historical frame pose data output by the VIO or SLAM system based on the original local coordinate system before the smart device detects the target; current pose information refers to the pose data of the current frame in the original local coordinate system after the target is detected; and the spatial pose transformation relationship is the transformation matrix from the current frame coordinate system to the target coordinate system obtained through the above steps, such as matching corresponding point pairs and pose calculation. Since both historical and current pose information are based on the same original local coordinate system, the relative transformation matrix from the historical frame to the current frame can be calculated using the pose difference between the two. Then, the cumulative transformation relationship can be obtained by combining this relative transformation matrix with the spatial pose transformation relationship. Finally, the cumulative transformation relationship is used to uniformly transform all historical poses to the target object coordinate system, forming continuous image data without jumps.

[0111] This process resolves the issue of frame-to-frame pose reference system splitting caused by historical frames remaining in their original local coordinate system and the current frame being anchored to the target object coordinate system. By defining the cumulative transformation relationship and uniformly converting historical poses to the target object coordinate system, this process unifies the pose information of all frames under the target object coordinate system. This allows virtual objects in virtual-real fusion scenarios such as XR to maintain a stable superposition with real-world objects as the device moves or over time, avoiding virtual content jitter or misalignment caused by coordinate system jumps. Furthermore, historical poses in local coordinate systems are prone to error accumulation due to sensor drift in VIO or SLAM systems, while the target object coordinate system is based on a fixed reference in the physical world. However, by converting historical poses to this coordinate system through cumulative transformation, the stability of the global reference system can constrain and correct errors, improving the reliability of long-term pose data.

[0112] This application also provides a spatial pose anchoring device for a smart device, such as... Figure 5 As shown, a spatial pose anchoring device for a smart device includes:

[0113] The matching unit 501 is used to acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain a pair of points with the same name;

[0114] Anchoring unit 502 is used to solve the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring; wherein, the target coordinate system is obtained by constructing the target image data by the intelligent device in the current environment.

[0115] In some embodiments, spatial pose estimation is performed on the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring, specifically including:

[0116] According to the construction rules of the target object coordinate system, the current frame image data within the same point is mapped to obtain the matching relationship of the same point pair;

[0117] Based on the matching relationship of the same point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship.

[0118] In some embodiments, based on the matching relationship of the corresponding point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship, specifically including:

[0119] Based on the matching relationship and sensor intrinsic parameters, the transformation matrix from the current sensor coordinate system to the target object coordinate system is determined, and the spatial pose transformation relationship is obtained.

[0120] Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined; wherein, the anchoring type includes: single-device anchoring and multi-device anchoring;

[0121] Based on the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each of the intelligent devices, the spatial pose anchoring of each of the intelligent devices is realized.

[0122] In some embodiments, after solving the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring, the method further includes:

[0123] Acquire historical frame image data to determine the historical pose information corresponding to the historical frame image data;

[0124] Based on the historical pose information and the current pose information corresponding to the current frame image data, as well as the spatial pose transformation relationship, the cumulative transformation relationship is determined;

[0125] Based on the cumulative transformation relationship, the historical frame image data is transformed into the target object coordinate system.

[0126] In some embodiments, before performing spatial pose calculation on the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring, the method further includes:

[0127] Determine the current environment in order to load target image data of a specific target object within that current environment;

[0128] Establish a mapping relationship between the pixel coordinates and physical coordinates of the target image data, so as to obtain the target object coordinate system fixed to the specific target object according to the mapping relationship.

[0129] In some embodiments, after loading target image data of a specific target object in the current environment, the method further includes:

[0130] The target image data is denoised, and edge detection is performed on the denoised target image data to obtain the target candidate region of the specific target object;

[0131] Based on the local peak detection results of the target candidate region, the target area of ​​the specific target object is determined;

[0132] Feature points of the target region are extracted to generate a first feature descriptor based on the feature points.

[0133] In some embodiments, matching the current frame image data with the target image data to obtain pairs of corresponding points specifically includes:

[0134] Obtain keyframe information corresponding to each current frame image data, and identify whether the current frame image contains the specific target object; wherein, the keyframe information includes: pose information and a second feature descriptor;

[0135] If so, feature matching is performed based on the first feature descriptor and the second feature descriptor to obtain the same point pairs between the target image data and the current frame image data.

[0136] In some embodiments, after acquiring the current frame image data collected by the sensor, the method further includes:

[0137] Monitor upper-layer application commands to obtain the target scenario corresponding to the upper-layer application commands;

[0138] Based on the target scene and the current scene of the current frame image data, determine whether the current frame image data should undergo world-frame transformation;

[0139] If so, the pose information of the current frame image data is transformed according to the transformation relationship corresponding to the world system transformation.

[0140] This application embodiment also provides an XR device, the device including:

[0141] At least one processor; and,

[0142] A memory communicatively connected to the at least one processor; wherein,

[0143] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: perform any of the spatial pose anchoring methods described above.

[0144] In a certain embodiment of this application, such as Figure 6 As shown, the XR device may include: a computing unit 601, a ROM 102, a RAM 103, a bus 604, an I / O interface 605, a front-end VIO 106, a sensor input unit 607, an output unit 608, a storage unit 609, and a communication unit 610.

[0145] The computing unit 601 can execute the spatial pose anchoring processing and analysis based on a computer program stored in the read-only memory 602 or a computer program loaded from the storage unit 610 into the random access memory (RAM) 603. The RAM 603 can also store various programs and data required for the operation of the XR device. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0146] Multiple components in the XR device are connected to I / O interface 605, including: a front-end VIO 606 connected to an input unit 607, which can be various sensors; an output unit 608, such as various types of displays, speakers, etc.; a storage unit 609, such as a disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless transceiver, etc. The communication unit 610 allows the smart device to interact with other XR devices through computer networks such as the Internet and / or various telecommunications networks, exchanging information or data.

[0147] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence computing chips, various computing units running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. During the execution of the spatial pose anchoring method, the computing unit 601 acquires target image data of a specific target object to construct a target object coordinate system corresponding to the specific target object; acquires current frame image data collected by a sensor and matches the current frame image data with the target image data to obtain corresponding point pairs; and performs spatial pose calculation on the current frame image data based on the target object coordinate system and the corresponding point pairs, thus realizing the various processing steps of spatial pose anchoring. For example, in some embodiments, the spatial pose anchoring method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 609. In some embodiments, part or all of the computer program can be loaded and / or installed on the XR device via ROM 602 and / or communication unit 610. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps in the following embodiments of this application can be performed. In other feasible embodiments, computing unit 601 may be configured in any other suitable manner to perform the spatial pose anchoring method of this application.

[0148] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0149] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] The above description is merely one or more embodiments of this application and is not intended to limit this application. For those skilled in the art, various modifications and variations can be made to one or more embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this application should be included within the scope of the claims of this application.

Claims

1. A spatial pose anchoring method, characterized in that, The method includes: Acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain corresponding point pairs; Based on the target object coordinate system and the corresponding point pair, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring; wherein, the target object coordinate system is obtained by constructing the target image data based on the intelligent device in the current environment; Based on the target object coordinate system and the corresponding point pair, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring, specifically including: According to the construction rules of the target object coordinate system, the current frame image data within the same point is accurately mapped from pixel coordinates to physical coordinates to obtain the matching relationship of the same point pair; Based on the matching relationship of the same point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship; Based on the matching relationship of the corresponding point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, and spatial pose anchoring is performed based on the spatial pose transformation relationship, specifically including: Based on the matching relationship and sensor intrinsic parameters, the transformation matrix from the current sensor coordinate system to the target object coordinate system is determined, and the spatial pose transformation relationship is obtained. Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined; wherein, the anchoring type includes: single-device anchoring and multi-device anchoring; Based on the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each of the intelligent devices, the spatial pose anchoring of each of the intelligent devices is realized; Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined, specifically including: Obtain the shared tag corresponding to a specific target object, so as to determine the range of shared devices corresponding to the specific target object based on the shared tag, and obtain the interaction data of each smart device, so as to determine the associated smart devices with collaborative tasks in the current scenario based on the interaction data; If the associated smart device exists within the scope of the shared device, then the anchoring type corresponding to the smart device is determined to be multi-device anchoring; If it is determined that the associated smart device does not belong to the scope of the shared device, then the sharing permission corresponding to the associated smart device is determined based on the data type of the interaction data and the target data corresponding to the historical anchoring data of the associated smart device. After solving the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring, the method further includes: Acquire historical frame image data to determine the historical pose information corresponding to the historical frame image data; Based on the historical pose information and the current pose information corresponding to the current frame image data, as well as the spatial pose transformation relationship, the cumulative transformation relationship is determined; Based on the cumulative transformation relationship, the historical frame image data is transformed into the target object coordinate system.

2. The spatial pose anchoring method according to claim 1, characterized in that, Before performing spatial pose calculation on the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring, the method further includes: Determine the current environment in order to load target image data of a specific target object within that current environment; Establish a mapping relationship between the pixel coordinates and physical coordinates of the target image data, so as to obtain the target object coordinate system fixed to the specific target object according to the mapping relationship.

3. The spatial pose anchoring method according to claim 2, characterized in that, After loading the target image data of a specific target object in the current environment, the method further includes: The target image data is denoised, and edge detection is performed on the denoised target image data to obtain the target candidate region of the specific target object; Based on the local peak detection results of the target candidate region, the target area of ​​the specific target object is determined; Feature points of the target region are extracted to generate a first feature descriptor based on the feature points.

4. The spatial pose anchoring method according to claim 3, characterized in that, Matching the current frame image data with the target image data to obtain corresponding point pairs specifically includes: Obtain keyframe information corresponding to each current frame image data, and identify whether the current frame image contains the specific target object; wherein, the keyframe information includes: pose information and a second feature descriptor; If so, feature matching is performed based on the first feature descriptor and the second feature descriptor to obtain the same point pairs between the target image data and the current frame image data.

5. The spatial pose anchoring method according to claim 1, characterized in that, After acquiring the current frame image data collected by the sensor, the method further includes: Monitor upper-layer application commands to obtain the target scenario corresponding to the upper-layer application commands; Based on the target scene and the current scene of the current frame image data, determine whether the current frame image data should undergo world-frame transformation; If so, the pose information of the current frame image data is transformed according to the transformation relationship corresponding to the world system transformation.

6. A spatial pose anchoring device, characterized in that, The device includes: The matching unit is used to acquire the current frame image data collected by the sensor, and match the current frame image data with the target image data to obtain the same point pair; An anchoring unit is used to solve the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pair to achieve spatial pose anchoring; wherein, the target object coordinate system is obtained by constructing the target image data based on the intelligent device in the current environment; Based on the target object coordinate system and the corresponding point pair, the spatial pose of the current frame image data is solved to achieve spatial pose anchoring, specifically including: According to the construction rules of the target object coordinate system, the current frame image data within the same point is mapped to obtain the matching relationship of the same point pair; Based on the matching relationship of the same point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, so as to perform spatial pose anchoring based on the spatial pose transformation relationship; Based on the matching relationship of the corresponding point pairs, the spatial pose transformation relationship of the current frame image data relative to the target object coordinate system is obtained, and spatial pose anchoring is performed based on the spatial pose transformation relationship, specifically including: Based on the matching relationship and sensor intrinsic parameters, the transformation matrix from the current sensor coordinate system to the target object coordinate system is determined, and the spatial pose transformation relationship is obtained. Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined; wherein, the anchoring type includes: single-device anchoring and multi-device anchoring; Based on the anchoring type, the spatial pose transformation relationship, and the local coordinate system corresponding to each of the intelligent devices, the spatial pose anchoring of each of the intelligent devices is realized; Based on the shared tags of the target image data and the interaction data of each smart device, the anchoring type corresponding to the smart device is determined, specifically including: Obtain the shared tag corresponding to a specific target object, so as to determine the range of shared devices corresponding to the specific target object based on the shared tag, and obtain the interaction data of each smart device, so as to determine the associated smart devices with collaborative tasks in the current scenario based on the interaction data; If the associated smart device exists within the scope of the shared device, then the anchoring type corresponding to the smart device is determined to be multi-device anchoring; If it is determined that the associated smart device does not belong to the scope of the shared device, then the sharing permission corresponding to the associated smart device is determined based on the data type of the interaction data and the target data corresponding to the historical anchoring data of the associated smart device. After solving the spatial pose of the current frame image data based on the target object coordinate system and the corresponding point pairs to achieve spatial pose anchoring, the method further includes: Acquire historical frame image data to determine the historical pose information corresponding to the historical frame image data; Based on the historical pose information and the current pose information corresponding to the current frame image data, as well as the spatial pose transformation relationship, the cumulative transformation relationship is determined; Based on the cumulative transformation relationship, the historical frame image data is transformed into the target object coordinate system.

7. An XR device, characterized in that, The XR device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to: perform the spatial pose anchoring method according to any one of claims 1-5.

Images