A three-dimensional space anchor point generation method, device, equipment and medium

Abstract

The application discloses a three-dimensional space anchor point generation method and device, equipment and medium, and relates to the technical field of augmented reality, which comprises the following steps: acquiring a plurality of target two-dimensional pointing positioning results generated by a user through different indication methods in real time, and converting the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on an imaging plane; based on the current camera pose of a local preset camera, the two-dimensional pixel coordinates are back-projected into target three-dimensional space rays in a preset three-dimensional coordinate system; the intersection point operation is performed on the target three-dimensional space rays and environmental geometric information to obtain corresponding three-dimensional coordinate candidate points and corresponding uncertainties; and based on the uncertainties, the three-dimensional coordinate candidate points and historical three-dimensional space anchors are weighted and fused to obtain target three-dimensional space anchors. Through back-projection of the pixel coordinates into space rays in the preset three-dimensional coordinate system based on the camera pose, under the condition of meeting the tracking effectiveness, the virtual content jitter caused by parallax is significantly reduced.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and in particular to a method, apparatus, device, and medium for generating three-dimensional spatial anchor points. Background Technology

[0002] In the field of augmented reality technology, lightweight AR / AI smart glasses generally use voice commands combined with various pointing methods (such as eye pointing, gesture pointing, head pointing, or touch / remote control pointing) to select target objects in the real world and overlay corresponding virtual information in order to achieve human-computer interaction.

[0003] Current augmented reality methods involve overlaying virtual information tags onto the camera image or displaying the corresponding pixel coordinates on the imaging plane. However, this method suffers from a problem: when the user's head or eyes move, the parallax effect causes the overlaid virtual information to fail to adhere firmly to the real object, resulting in noticeable visual jitter. Furthermore, it cannot form stable 3D spatial anchor points across frames. Therefore, how to form stable 3D spatial anchor points remains a technical problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for generating three-dimensional spatial anchor points, which can back-project pixel coordinates into spatial rays in a preset three-dimensional coordinate system based on camera pose, significantly reducing virtual content jitter caused by parallax while ensuring effective tracking. The specific solution is as follows: In a first aspect, this application provides a method for generating three-dimensional spatial anchor points, applied to AR glasses, including: The target signal parsing method is used to acquire several two-dimensional target pointing and positioning results generated by the user through different indication methods in real time, and the two-dimensional target pointing and positioning results are converted into two-dimensional pixel coordinates on the imaging plane. Based on the current camera pose of the local preset camera, the two-dimensional pixel coordinates are back-projected into a target three-dimensional spatial ray in a preset three-dimensional coordinate system; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera based on the initial camera pose of the environment, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. The intersection point operation between the target's three-dimensional spatial ray and the environmental geometric information is performed to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points; Based on the aforementioned uncertainty, the candidate three-dimensional coordinates are weighted and fused with several historical three-dimensional spatial anchors to obtain the target three-dimensional spatial anchor.

[0005] Optionally, before acquiring several target two-dimensional pointing and positioning results generated by the user through different indication methods in real time using the target signal parsing method, the method further includes: The focal length and lens distortion of the preset camera lens are corrected, the positional and directional relationships of the local preset motion sensor, preset eye tracker and camera lens are determined, and the preset three-dimensional coordinate system is established for the current environment based on synchronous positioning and mapping technology.

[0006] Optionally, converting the target two-dimensional pointing positioning result into two-dimensional pixel coordinates on the imaging plane includes: If the user's instruction method is eye gaze, then the two-dimensional pixel coordinates of the user's gaze on the imaging plane are directly obtained; If the user's instruction method is a gesture instruction, then calculate the intersection point of the dotted line extending from the user's finger and the camera screen, and determine the coordinates of the intersection point on the imaging plane as the two-dimensional pixel coordinates; If the user's instruction method is voice instruction, then the natural language recognition model is used to identify the target object indicated by the user, and the coordinates of the center point of the outline of the target object on the imaging plane are determined as the two-dimensional pixel coordinates.

[0007] Optionally, the operation of finding the intersection point between the target's three-dimensional spatial ray and the environmental geometry includes: If the environmental geometric information is the depth information at the two-dimensional pixel coordinates, then a point on the target three-dimensional spatial ray with a distance corresponding to the depth information is determined, and the point with a distance corresponding to the depth information is determined as the candidate point of the three-dimensional coordinates; If the environmental geometry information is the plane parameter of any physical plane in the environment, then the plane intersection point of the target three-dimensional spatial ray and the physical plane is calculated based on the plane parameter, and the plane intersection point is determined as the candidate point of the three-dimensional coordinates; If the environmental geometry information is a sparse three-dimensional point cloud of the environment, then the target point closest to the target three-dimensional spatial ray is determined in the sparse three-dimensional point cloud, and the target point is determined as the candidate point of the three-dimensional coordinates.

[0008] Optionally, after weighted fusing the candidate 3D coordinates with several historical 3D spatial anchors based on the uncertainty, the method further includes: The position of the target three-dimensional space anchor point is determined. If the target three-dimensional space anchor point is located on any physical plane, the position of the target three-dimensional space anchor point is constrained to the any physical plane. If the target 3D spatial anchor point is located on any target object, then the position of the target 3D spatial anchor point is bound to the visual features of the target object.

[0009] Optionally, after weighted fusing the candidate 3D coordinates with several historical 3D spatial anchors based on the uncertainty, the method further includes: The first position of the target three-dimensional spatial anchor point in the preset three-dimensional coordinate system is converted into the second position in the camera coordinate system at the current moment; The target three-dimensional spatial anchor point is projected onto the screen according to the second position, so as to draw and overlay virtual content on the target three-dimensional spatial anchor point.

[0010] Optionally, the three-dimensional spatial anchor point generation method further includes: The target 3D spatial anchor point and the environmental feature information associated with the target 3D spatial anchor point are stored so that when the user re-enters the same environment, virtual content is displayed based on the target 3D spatial anchor point and the environmental feature information.

[0011] Secondly, this application provides a three-dimensional spatial anchor point generation device for use in AR glasses, comprising: The positioning result conversion module is used to acquire in real time a number of target two-dimensional pointing positioning results generated by the user through different indication methods using the target signal parsing method, and convert the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on the imaging plane. The coordinate back projection module is used to back project the two-dimensional pixel coordinates into a target three-dimensional spatial ray in a preset three-dimensional coordinate system based on the current camera pose of the local preset camera; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera on the environment according to the initial camera pose, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. The candidate point acquisition module is used to perform an intersection operation between the target's three-dimensional spatial ray and the environmental geometric information to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points. The spatial anchor point acquisition module is used to perform weighted fusion of the candidate three-dimensional coordinates and several historical three-dimensional spatial anchor points based on the uncertainty, so as to obtain the target three-dimensional spatial anchor point.

[0012] Thirdly, this application provides an electronic device, comprising: Memory is used to store computer programs; A processor is used to execute the computer program to implement the aforementioned three-dimensional spatial anchor point generation method.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the aforementioned three-dimensional spatial anchor point generation method.

[0014] This application first uses a target signal analysis method to acquire several target two-dimensional pointing positioning results generated by the user through different indication methods in real time, and converts the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on the imaging plane. Then, based on the current camera pose of the local preset camera, the two-dimensional pixel coordinates are back-projected into a target three-dimensional spatial ray in a preset three-dimensional coordinate system. The preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera based on the initial camera pose of the environment. The target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. Then, the intersection point operation of the target three-dimensional spatial ray and the geometric information of the environment is performed to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points. Finally, based on the uncertainty, the three-dimensional coordinate candidate points and several historical three-dimensional spatial anchor points are weighted and fused to obtain the target three-dimensional spatial anchor points. Therefore, this application achieves compatible processing of multiple interaction sources by uniformly converting the two-dimensional pointing results generated by different pointing methods into screen pixel coordinates, thus solving the problem that two-dimensional visual output is difficult to directly use for three-dimensional spatial anchoring; by back-projecting pixel coordinates into spatial rays in a preset three-dimensional coordinate system based on camera pose, it realizes the mapping from the image plane to world coordinates, significantly reducing virtual content jitter caused by parallax under the condition of effective tracking; and by weighted fusion based on uncertainty, it achieves suppression of observation noise and stable optimization of anchor point state, solving the problem of anchor point jump caused by interference in a single frame. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a three-dimensional spatial anchor point generation method disclosed in this application; Figure 2 This is a schematic diagram of a three-dimensional spatial anchor point generation device disclosed in this application; Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Current augmented reality methods suffer from the inability to form stable 3D spatial anchor points across frames. To address this, this application provides a 3D spatial anchor point generation method that back-projects pixel coordinates into spatial rays in a preset 3D coordinate system based on camera pose. Under the condition of effective tracking, this significantly reduces virtual content jitter caused by parallax.

[0019] See Figure 1 As shown, this embodiment of the invention discloses a three-dimensional spatial anchor point generation method, applied to AR glasses, including: Step S11: Use the target signal analysis method to acquire several target two-dimensional pointing positioning results generated by the user through different indication methods in real time, and convert the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on the imaging plane.

[0020] This embodiment, based on the pinhole camera model, spatial geometry principles, and probability estimation theory in computer vision, treats the user's two-dimensional pointing results on the displayed screen as noisy observations of the real scene. It constructs a multi-layered processing framework that includes geometric back-projection, scene geometric intersection, and probabilistic fusion stabilization, generating and continuously maintaining three-dimensional spatial anchor points in the world coordinate system online. Its core lies in unifying and normalizing the two-dimensional pointing information generated by various interaction methods (including but not limited to gaze, gesture rays, touch clicks, or bounding boxes / masks output by visual positioning) into pixel observation points on the camera image plane. and their corresponding confidence levels By smoothing and removing outliers from the observation data within a short time window, a series of time-series-based two-dimensional pointing observation data is generated.

[0021] In this embodiment, before using the target signal analysis method to obtain several target two-dimensional pointing positioning results generated by the user through different indication methods in real time, the method further includes: correcting the lens focal length and lens distortion of the preset camera, determining the positional and directional relationships between the local preset motion sensor, preset eye tracker and camera lens, and establishing the preset three-dimensional coordinate system for the current environment based on synchronous positioning and map building technology.

[0022] The above process, also known as equipment preparation, specifically includes: 1. Camera Calibration: Used to establish the lens characteristics (such as focal length and distortion) of the system's cameras so that the geometric relationship between the images detected by the system's cameras and the real world is correct.

[0023] 2. Sensor Calibration: Determine the position and orientation of components such as the motion sensor IMU (Inertial Measurement Unit), eye tracker, and gesture camera relative to the main camera. This ensures that when the user turns their head or points with their finger, the system can accurately identify the position of that "pointing line" in the camera's view.

[0024] 3. Create a world map: Start the SLAM (Simultaneous Localization and Mapping) system to create an initial 3D map (world coordinate system, i.e., a preset 3D coordinate system) for the current environment. All subsequent anchor points will be fixed on this map.

[0025] In addition, before obtaining the user's pointing, this implementation can also perceive the environment in real time (continuous localization and mapping), with the aim of updating its own location and environmental information in real time while moving.

[0026] Specific content: 1. Tracking self-pose: Using VIO (Visual-Inertial Odometry) / SLAM technology, continuously calculate the precise position and orientation of the glasses (camera) in the world map established in step one.

[0027] 2. Understanding Scene Geometry: Continuously enriching the understanding of the surrounding environment, for example, identifying the presence of a tabletop (a plane) or a corner in the distance (some sparse point clouds). The above simple geometric information forms the basis for subsequent calculations.

[0028] In this embodiment, the process of converting the target two-dimensional pointing positioning result into two-dimensional pixel coordinates on the imaging plane may specifically include: if the user's indication method is eye gaze, then the two-dimensional pixel coordinates of the user's gaze on the imaging plane are directly obtained; if the user's indication method is gesture indication, then the intersection point of the dotted line extending from the user's finger and the camera screen is calculated, and the coordinates of the screen intersection point on the imaging plane are determined as two-dimensional pixel coordinates; if the user's indication method is voice indication, then the target object indicated by the user is identified using a natural language recognition model, and the coordinates of the center point of the target object's outline on the imaging plane are determined as two-dimensional pixel coordinates.

[0029] That is, the target semantics are determined based on the speech recognition or semantic understanding results, and the target region is located in the image by the visual perception module; the center point or centroid of the target region is determined as two-dimensional pixel coordinates.

[0030] Specifically, 1. Analyze the pointing signal: Eye tracking: Calculates which pixel (u, v) on the screen (i.e., the imaging plane) the user's gaze is fixed on.

[0031] Gesture pointing: Calculate the intersection point (u, v) between the virtual ray extended from the user's finger and the camera image, i.e., the two-dimensional pixel coordinates.

[0032] AI Recognition: The AI ​​model is instructed to recognize objects spoken or naturally pointed to by the user and output a bounding box or outline, with the system taking the center point as (u, v).

[0033] 2. Assess the confidence level of the target: For example, assess the user's blinking behavior, the user's hand tremors, and the confidence level of the AI ​​model's object recognition results.

[0034] This embodiment also evaluates the confidence level of various pointing methods of the user. If the confidence level of a certain pointing method is lower than the preset confidence threshold, then the pointing method will not be used.

[0035] In other words, the unified modeling and calculation method for the two-dimensional pointing positioning results in this embodiment is as follows: Input definition (unifying the different sources): Two-dimensional pointing and positioning results can come from: finger pointing, gestures, gaze, touch cursors, voice-triggered screen center crosshairs, etc. Regardless of the source, they are uniformly represented as: pixel coordinates (i.e., two-dimensional pixel coordinates): Confidence level: Pixel measurement covariance matrix: ; in, The variance (in pixels) of the measurement error for pixel coordinate u (horizontal / column direction); The variance (in pixels) of the measurement error for pixel coordinate v (vertical / row direction); In addition, to reduce vibration, you can... Perform window filtering and threshold removal, and use the moving average: Threshold: If Then the frame will be downgraded or discarded. For the k-th frame or the k-th time sampling point, the corresponding two-dimensional pointing observation vector is the pixel coordinates obtained by gesture or eye tracking in the k-th frame.

[0036] The weights used for subsequent solutions are: ; in, This serves as a stability measure.

[0037] It should be noted that this embodiment also performs distortion correction processing on the two-dimensional pixel coordinates: right First, perform camera distortion correction to obtain (Distortion parameters have been calibrated and anti-distortion mapping has been performed).

[0038] By performing distortion correction, the reliability of the corrected two-dimensional pixel coordinates can be guaranteed, thereby ensuring the reliability of the subsequent process of converting the two-dimensional pixel coordinates into three-dimensional spatial rays using the camera intrinsic parameter matrix K.

[0039] The camera intrinsic parameter matrix is ​​as follows: ; in, and In the diagram, C represents the principal point coordinates, which are also the pixel positions where the optical axis intersects the imaging plane. and In this context, f represents the x and y directions, respectively.

[0040] In summary, this embodiment designs a unified framework for converting 2D pointing signals to 3D anchor points. Regardless of whether the 2D pointing signal originates from eye tracking, gesture interaction, or a bounding box or mask output by an AI visual positioning model, it can be normalized and then integrated into this system. This modular design enhances the method's versatility and facilitates the integration of multiple interaction modalities.

[0041] By constructing a unified 2D pointing model, these different formats are compressed into pixel observations and confidence scores. Specifically, the bounding box B is centered or weighted as (u,v), and the confidence score is obtained from the detection. The mask M is centered at the region centroid (u,v), and the confidence score is defined by the average activation value. The gaze and gesture are directly projected onto the imaging plane pixels. Then, the above 2D observations are fed into the same "backprojection + geometric intersection + probabilistic fusion" process to obtain a unified 3D anchor point output. The purpose of the above process is to fully reuse existing 2D detection / segmentation / visual localization capabilities without completely rewriting the perception module; however, on the output side, all 2D results are elevated to steady-state 3D anchor points in the world coordinate system, improving the stability and consistency of AR overlay.

[0042] Step S12: Based on the current camera pose of the local preset camera, the two-dimensional pixel coordinates are back-projected into a target three-dimensional spatial ray in a preset three-dimensional coordinate system; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera based on the initial camera pose of the environment, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates.

[0043] Because the real world has depth of field, this embodiment needs to back-project the two-dimensional pixel coordinates into a target three-dimensional spatial ray in a preset three-dimensional coordinate system, so as to transform the two-dimensional pixel coordinates into three-dimensional spatial anchor points. Specifically, according to the camera calibration parameters, a virtual ray is emitted from the center of the camera lens, passing through the pixel point (u, v), and pointing towards the real world, so as to perform the intersection point calculation operation later.

[0044] During the geometric backprojection stage, the pre-calibrated camera intrinsic parameter matrix K is used to map the pixel observation points at each time step. By back-projecting onto the camera coordinate system, a unit viewing direction vector is obtained. Then, it combines the current camera pose transformation matrix calculated in real time by a visual inertial odometry (VIO) or simultaneous localization and mapping (SLAM) system. Transform the line-of-sight vector to the world coordinate system, forming a line with the position of the camera's optical center in the world coordinate system. With the origin as the origin and the direction vector as the direction vector A ray pointing in three-dimensional space. Specifically: Line of sight (ray direction) from a pixel to a unit in the camera coordinate system: Construct homogeneous pixels: ; in, and These are the x and y coordinates of the observation point in frame t, respectively. This is the transpose of the matrix.

[0045] Normalized camera coordinates (unnormalized directions): ; in, The direction of the ray is not yet normalized.

[0046] Unit direction (camera coordinate system): ; Ray from camera coordinate system to world or map coordinate system: Camera pose obtained from VIO / SLAM (camera to world): ; in, R is the rotation matrix, representing how the axes of the camera coordinate system are rotated to the world coordinate system. t is the translation vector, representing the position of the camera origin in the world coordinate system, which is also the position of the camera center or optical center.

[0047] The origin and direction of the ray are as follows: ; ; This leads to the world-system ray parameter equations: ; Among them, the above These are the weighting parameters. At this point, the 2D pointing result has been ray-transformed into a set of weighted rays. (i.e., the target 3D spatial ray), providing computable input for the following 3D anchor point solution.

[0048] This embodiment utilizes in-camera pose participation to uniformly backproject each two-dimensional observation into a ray in the world coordinate system. Then, through multi-frame ray geometric consistency (weighted ray consistency WRC, WRC + depth / plane fusion WRDPF), the unique 3D point in the world coordinate system is directly obtained. And maintain it as the anchor point state.

[0049] The result is that virtual content is no longer attached to pixels in a single frame, but rather to three-dimensional points in world coordinates; when the camera pose is updated, only the observed coordinates change, the anchor point itself remains unchanged, and the overlaid content is... Simply reproject the image; parallax and significant jitter are eliminated at the source. This directly solves the defect that "virtual content does not move with the camera along with the object due to only two-dimensional overlay."

[0050] Step S13: Perform an intersection operation on the target three-dimensional spatial ray and the environmental geometric information to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points.

[0051] In this embodiment, the operation of finding the intersection point between the target three-dimensional spatial ray and the environmental geometric information includes: if the environmental geometric information is the depth information at the two-dimensional pixel coordinates, then a point on the target three-dimensional spatial ray with a distance corresponding to the depth information is determined, and the point with a distance corresponding to the depth information is determined as a candidate point for three-dimensional coordinates; if the environmental geometric information is the plane parameter of any physical plane in the environment, then the plane intersection point between the target three-dimensional spatial ray and any physical plane is calculated based on the plane parameter, and the plane intersection point is determined as the candidate point for three-dimensional coordinates; if the environmental geometric information is a sparse three-dimensional point cloud of the environment, then the target point closest to the target three-dimensional spatial ray is determined in the sparse three-dimensional point cloud, and the target point is determined as a candidate point for three-dimensional coordinates.

[0052] In other words, the "scene geometry intersection" layer in this embodiment employs a multi-strategy parallel design: when reliable depth is available, depth back projection is used to obtain candidate points. When there is no depth but a stable plane is detected, it is related to the plane. The intersection is used to obtain 3D points; when only sparse point clouds or meshes exist, ray casting or nearest-point fitting is used to find the point on the map geometry that best matches the ray. Then, WRDPF is used to unify multiple priors such as "ray + depth + plane" into a single objective function, using weight parameters. Automatic balancing.

[0053] The reasons can be summarized as follows: the final determination of 3D points no longer strongly depends on a single sensor (such as a depth map), but allows for the selection of different geometric intersection paths under different hardware combinations and scene conditions; when depth is missing, it can still be solved using a plane or map; and when the depth is reliable, it will be given higher weight. This reduces the rigid requirements on heavy-duty depth hardware at the architectural level, solving the existing bottleneck that prevents stable 3D rendering without good depth.

[0054] The strategy for finding the intersection point is as follows: Best case (with depth information): If the device has a depth sensor (such as ToF), directly read the depth value at the point, and intercept the point at the corresponding distance on the ray, which is the three-dimensional coordinate (i.e., the candidate point of the three-dimensional coordinate).

[0055] Common scenario (with a plane): If the system identifies in step two that the user is pointing to a plane (such as a desktop or wall), then it calculates the intersection of the line of sight and the plane.

[0056] Minimum scenario (sparse points only): If none of the above applies, then find the point closest to this line of sight in the existing sparse environmental point cloud as an approximation.

[0057] Assessing uncertainty: Simultaneously estimate the "error range" of the 3D coordinates. For example, evaluate the accuracy of depth measurements and the effectiveness of plane fitting.

[0058] In other words, during the scene geometry intersection stage, this embodiment utilizes various lightweight scene geometry information acquired by the device itself (e.g., local depth maps from monocular / binocular / time-of-flight methods, local plane sets maintained by the VIO / SLAM system, sparse feature point clouds, or low-density meshes) to couple the generated 3D spatial rays with the 3D environment structure, thereby solving for specific 3D spatial candidate points. Specific intersection strategies include: If a reliable depth value exists in the current pixel observation area Then the distance is directly intercepted on the back-projected ray. Spatial points are used as 3D candidate points.

[0059] If the observation region is parameterized as If a ray is covered by a local plane, then the intersection of the ray and that plane is calculated as a 3D candidate point.

[0060] If the scene geometry consists only of sparse point clouds or meshes, then ray casting or nearest-neighbor fitting algorithms are used to find spatial points that minimize reprojection error as candidate points. During this process, the algorithm simultaneously estimates the 3D observation points for that frame based on pixel observation noise, depth measurement error, plane fitting residuals, or the quality of the map itself. The uncertainty is expressed as a covariance matrix. .

[0061] This solution does not heavily rely on dedicated depth sensing hardware that increases size, weight, and power consumption. Its design allows the system to flexibly select from various lightweight geometric intersection strategies (such as based on known planes, sparse maps, or region depth inference) according to the device's actual computing power, power budget, and scene conditions, achieving a flexible balance between computing resource requirements and overlay effects.

[0062] Step S14: Based on the uncertainty, the candidate three-dimensional coordinates are weighted and fused with several historical three-dimensional spatial anchors to obtain the target three-dimensional spatial anchor.

[0063] In this embodiment, due to potential errors and fluctuations in a single calculation, a weighted fusion of new and old observations is performed to stabilize the anchor point. Specifically: 1. New anchor point: If this is the first time creating the anchor point, the 3D point calculated in the above steps will be used as the initial position.

[0064] 2. Old Anchor Point Update: If an anchor point already exists, the new observations are weighted and merged with the old location. The more reliable the new observations (lower uncertainty), the greater their impact on the final location.

[0065] 3. Consistency Check: Project the currently maintained anchor point position back onto the current camera view and check if it matches the user's latest pointing position. If the deviation is too large, it indicates a pointing error or tracking loss, in which case updates are paused or the weight of new data is reduced.

[0066] That is, in the probabilistic fusion stabilization stage, this embodiment regards the "spatial anchor point" to be estimated as a hidden state, and uses probabilistic fusion methods such as weighted least squares batch optimization or Kalman filtering to recursively fuse and update the state of multiple three-dimensional observation points on the time series.

[0067] Within a single optimization window, a consistent optimal 3D point satisfying multi-frame geometric constraints is obtained by minimizing the weighted sum of distances from the anchor point to multiple historical observation rays. .

[0068] The process of finding the anchor point from the two-dimensional pointer is as follows: The user continuously points to a short period of time (e.g., 6 hours) Within 20 frames, the consistency of rays across multiple frames is used to directly solve for 3D points in a closed loop, avoiding heavy 3D reconstruction or dense mesh intersection.

[0069] 1. Objective: To find a 3D point that is "closest to all rays": For the t-th frame ray The perpendicular distance vector from any point P to this ray can be written as: ; in, Let be the transpose of the unit direction vector of the ray in the world coordinate system in frame t.

[0070] in It is the matrix projected onto the normal space (for a unit vector d, Symmetric and idempotent.

[0071] Define weighted least squares energy (minimize weighted ray consistency WRC): ; Where W points to the set of window frames.

[0072] 2. Derivation of closed-form solution: Expand: ; in, This represents the projection matrix. Perform the transpose. E(P) is the weighted total perpendicular distance squared from the anchor point to all rays. Minimizing E(P) will yield a consistent 3D anchor point.

[0073] Here, E(P) is the cost function or loss function, which determines how well a given anchor point (P) matches all rays within the window. Minimizing E(P) is equivalent to finding the 3D anchor point position that best explains the pointing of multiple frames simultaneously.

[0074] because ,get: ; Differentiate P with respect to zero: ; The linear equation is obtained by rearranging: ; make Then we have: ; Where b is a 3×1 ray consistency accumulation vector.

[0075] To prevent parallel rays, the Tikhonov regularity is used: ; in, For regularization parameters, take... Magnitude.

[0076] The computational steps at the implementation level are as follows: accumulate M and b frame by frame (only perform a few 3×3 matrix multiplications and additions per frame), and solve a 3×3 linear equation after the window ends. The computational complexity is very suitable for real-time operation on lightweight glasses.

[0077] In short, this embodiment only requires one back-projection calculation of the unit direction vector and constant-level matrix and vector addition operations (mainly 3×3 matrix operations) in the single-frame processing stage. In the sliding window-based batch optimization, the core operation is only to solve a small-scale 3×3 linear equation system, the solution of which can be expressed as... In the cross-frame state filtering update stage, only the operation of three-dimensional vectors and 3×3 covariance matrix is ​​involved.

[0078] This decouples computational complexity, making the overall algorithm complexity independent of the input image resolution and the number of candidate boxes, enabling real-time processing even on low-computing-power embedded devices. While maintaining or moderately lightweighting the performance of the front-end perceptual network, efficient back-end geometric and probabilistic fusion processing alone can significantly improve the overlay stability and visual experience of 3D spatial anchor points.

[0079] 3. Integrate depth and planar geometric priors: If the device has depth (binocular / ToF / structured light) or lightweight monocular depth, depth points can be constructed: ; in It represents pixel depth (i.e., depth information at two-dimensional pixel coordinates).

[0080] If plane detection (AR SLAM) exists, the plane equation (world system) is: It can also be used as a constraint.

[0081] WRDPF (Rayet + Depth + Plane Blend): ; in It is the weighted average of depth points within the available window.

[0082] It can also be derived as a closed linear system: ; And make the weights adaptive: ; in, For hyperparameters, It is a scaling factor, which is set manually to map the confidence level to an appropriate scale.

[0083] in For greater credibility, This represents the reliability of the plane fitting.

[0084] In summary, the core algorithm advantages of this embodiment are multi-frame ray consistency + depth / plane fusion + regularized closed-form solution; it acquires multiple frames of 2D pixel coordinates and generates multiple spatial rays within a preset time window; by minimizing the weighted distance from the 3D point to the multiple spatial rays, candidate 3D spatial anchor points are obtained. The solution includes depth constraint terms and plane constraint terms, and is obtained using a regularized closed-form solution.

[0085] In addition, in this embodiment, after weighted fusion of the candidate three-dimensional coordinates and several historical three-dimensional spatial anchors based on uncertainty, the method further includes: determining the position of the target three-dimensional spatial anchor; if the target three-dimensional spatial anchor is located on any physical plane, then constraining the position of the target three-dimensional spatial anchor to any physical plane; if the target three-dimensional spatial anchor is located on any target object, then binding the position of the target three-dimensional spatial anchor to the visual features of the target object.

[0086] The purpose of this step is to make the anchor point more stable and intelligent by utilizing additional information, specifically: 1. Planar constraint: If the anchor point is known to be on a plane (such as a cup on a table), then the anchor point will be forcibly pulled back to the nearest plane point to prevent the anchor point from floating in the air.

[0087] 2. Target Association: If the AI ​​identifies a complete object, it can associate the anchor point with the object's visual features. This way, even if the object moves, the anchor point will move accordingly.

[0088] By defining and maintaining 3D spatial anchor points in the world coordinate system, the position of virtual information is directly bound to real-world objects. When the user's head movement causes a change in the viewing pose, only the rendering viewpoint changes, while the world coordinates of the anchor points themselves remain unchanged. This ensures that augmented reality information can be stably pasted onto the real target object, effectively overcoming the inherent visual jitter defects of traditional 2D image spatial overlay methods.

[0089] In addition, after weighted fusion of candidate 3D coordinate points and several historical 3D spatial anchor points based on uncertainty, the process also includes: converting the first position of the target 3D spatial anchor point in the preset 3D coordinate system into the second position in the camera coordinate system at the current moment; and projecting the target 3D spatial anchor point onto the screen according to the second position, so as to draw and overlay virtual content on the target 3D spatial anchor point.

[0090] The above process is to ensure the stable display of virtual information. Specifically: 1. Coordinate transformation: Transform the stable anchor point position in the world coordinate system (i.e., the first position) to the position in the camera coordinate system at the current moment (i.e., the second position).

[0091] 2. Projection and Rendering: The 3D coordinates are projected onto the screen, and virtual labels or models are drawn there. The system also considers occlusion, causing virtual content to be obscured by real objects to enhance realism.

[0092] 3. Determine the orientation: Assign a reasonable orientation to the anchor point (such as perpendicular to the plane it is on) so that the virtual UI always faces the user.

[0093] Specifically, this embodiment also discloses the specific process of obtaining the attitude of the three-dimensional coordinate anchor point: AR anchor points are typically in pose (P, R). If there is a planar normal n, the local z-axis of the anchor point can be set to n, and then the world orientation can be used. Constructing an orthogonal basis: ; If there is no plane normal, then the orientation facing the camera can be used: ; in, This represents the position of the camera's optical center in the world coordinate system at the current moment or in the latest frame.

[0094] Construct R again using the same method.

[0095] The above yields one measurement value. Additionally, to ensure stable AR overlay, timing filtering, anomaly removal, and cross-session maintenance are required for the anchor points.

[0096] 1. Error and uncertainty propagation: When using WRDPF linear systems: ; Under the least squares assumption (noise variance) The anchor point covariance matrix can be approximated as: ; Window residual estimation: ; 2. Adaptive filter update: Set the current anchor point state Covariance matrix This measurement Covariance matrix .

[0097] Using static point Kalman update: ; in, The Kalman gain matrix is ​​equivalent to a weight allocator. Estimate the location of the anchor points after this update (posterior mean). This represents the uncertainty of the updated anchor point (posterior covariance matrix).

[0098] If computational power is taken into account, the equivalent EMA coefficient can be obtained using a scalar approximation: ; The trace of the tr() matrix is ​​the sum of the diagonals of a 3×3 covariance matrix Σ. In the covariance matrix, the trace can be understood as the sum of the uncertainties in the x, y, and z directions. This means that the 3×3 Kalman gain is approximated as a scalar weight.

[0099] Anomaly removal: Mahalanobis distance gating (anti-jump point): ; like If the chi-square threshold is not met, the measurement is deemed abnormal: weighting is reduced, the window is extended, or the solution is retried.

[0100] In continuous estimation across time intervals, a covariance matrix-driven update strategy is employed: when the uncertainty of a new observation is small, the confidence level is high, and the residual with the predicted state is small, the anchor point position is updated rapidly, quickly converging to the user's actual target; when the observation noise is high or significant anomalies exist, its update weight is automatically reduced, and mechanisms such as Mahalanobis distance gating are used to reject or slowly absorb it, thereby effectively suppressing instantaneous jumps in the anchor point. Finally, the state of the anchor point is determined by its mean μ and covariance matrix in the world coordinate system. The format is continuously maintained and bound to the map coordinate system of the SLAM system.

[0101] The principle behind this embodiment is to treat two-dimensional pointing as "noisy observation" and solve this problem through the following two-layer mechanism: 1. Multi-frame geometric consistency within a window: Within a short time window, multiple rays are introduced, and the weighted minimum perpendicular distance from the ray (WRC / WRDPF) is used to obtain the geometric consistency that simultaneously satisfies multi-frame constraints. Errors / jitter in a single frame will be recovered by other frames, and the three-dimensional position will no longer be determined by a single point.

[0102] 2. Cross-window probability filtering: This is the result obtained each time. Estimating the covariance matrix and the historical anchor state Perform Kalman / EMA updates; when the observation covariance matrix is ​​large, the filter gain automatically decreases, having a limited impact on the anchor point; when the observation bias is too large, the observation is directly rejected or weakened through Mahalanobis distance gating.

[0103] Therefore, even with slight fluctuations in SLAM pose, gesture / view jitter, or local depth noise, the anchor point position will not jump immediately, but will instead slowly and smoothly approach the object the user is actually pointing to. This fundamentally solves the anchor point jitter, drift, and occasional jump problems caused by existing "one-time ray intersection + modeling without uncertainty".

[0104] Furthermore, this embodiment can also store the target three-dimensional spatial anchor point and the environmental feature information associated with the target three-dimensional spatial anchor point, so that when the user re-enters the same environment, virtual content can be displayed based on the target three-dimensional spatial anchor point and environmental feature information.

[0105] The above process is designed to ensure that the virtual tag remains in place the next time the user puts on the glasses. Specifically, it includes: 1. Save: Save the anchor point along with the features of its surrounding environment (such as planar parameters and visual feature points).

[0106] 2. Restoration: When the user re-enters the environment, the system will automatically restore the previously saved virtual tags and accurately paste them back onto the original objects after re-identifying the surrounding scene through relocation.

[0107] Specifically, to avoid anchor point drift caused by world coordinate drift, the anchor point can be stored in the SLAM map coordinate system m: ; in, This refers to the position of the anchor point in the map coordinate system m, which is the coordinate that is to be saved or persisted.

[0108] When SLAM performs loop closure / relocation update, it generates an increment. At the same time, the anchor point is updated: ; This ensures that the AR overlay maintains spatial consistency after map optimization.

[0109] Therefore, this application achieves compatible processing of multiple interaction sources by uniformly converting the two-dimensional pointing results generated by different pointing methods into screen pixel coordinates, thus solving the problem that two-dimensional visual output is difficult to directly use for three-dimensional spatial anchoring; by back-projecting pixel coordinates into spatial rays in a preset three-dimensional coordinate system based on camera pose, it realizes the mapping from the image plane to world coordinates, significantly reducing virtual content jitter caused by parallax under the condition of effective tracking; and by weighted fusion based on uncertainty, it achieves suppression of observation noise and stable optimization of anchor point state, solving the problem of anchor point jump caused by interference in a single frame.

[0110] See Figure 2 As shown, this embodiment of the invention discloses a three-dimensional spatial anchor point generation device, applied to AR glasses, comprising: The positioning result conversion module 11 is used to acquire in real time a number of target two-dimensional pointing positioning results generated by the user through different indication methods using the target signal analysis method, and convert the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on the imaging plane. The coordinate back projection module 12 is used to back project the two-dimensional pixel coordinates into a target three-dimensional spatial ray in a preset three-dimensional coordinate system based on the current camera pose of the local preset camera; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera on the environment according to the initial camera pose, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. The candidate point acquisition module 13 is used to perform an intersection operation between the target three-dimensional spatial ray and the environmental geometric information to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points. The spatial anchor point acquisition module 14 is used to perform weighted fusion of the candidate three-dimensional coordinates and several historical three-dimensional spatial anchor points based on the uncertainty, so as to obtain the target three-dimensional spatial anchor point.

[0111] In some specific embodiments, the location result conversion module 11 further includes: The coordinate system establishment unit is used to correct the lens focal length and lens distortion of the preset camera, determine the positional and directional relationships between the local preset motion sensor, preset eye tracker and camera lens, and establish the preset three-dimensional coordinate system for the current environment based on synchronous positioning and mapping technology.

[0112] In some specific embodiments, the location result conversion module 11 may specifically include: The coordinate acquisition unit is used to directly acquire the two-dimensional pixel coordinates of the user's gaze on the imaging plane if the user's indication method is eye gaze. The first coordinate determination unit is used to calculate the intersection point of the dotted line extending from the user's finger and the camera screen if the user's indication method is a gesture indication, and to determine the coordinates of the intersection point on the imaging plane as the two-dimensional pixel coordinates. The second coordinate determination unit is used to identify the target object indicated by the user using a natural language recognition model if the user's instruction method is voice instruction, and to determine the coordinates of the center point of the outline of the target object on the imaging plane as the two-dimensional pixel coordinates.

[0113] In some specific embodiments, the candidate point acquisition module 13 may specifically include: The first candidate point determination unit is used to determine, if the environmental geometric information is the depth information at the two-dimensional pixel coordinates, a point on the target three-dimensional spatial ray with a distance corresponding to the depth information, and to determine the point with a distance corresponding to the depth information as the three-dimensional coordinate candidate point. The second candidate point determination unit is used to calculate the plane intersection point of the target three-dimensional spatial ray and the physical plane based on the plane parameters of any physical plane in the environment if the environmental geometric information is a plane parameter of any physical plane in the environment, and determine the plane intersection point as the three-dimensional coordinate candidate point. The third candidate point determination unit is used to determine the target point in the sparse three-dimensional point cloud that is closest to the target three-dimensional spatial ray if the environmental geometric information is a sparse three-dimensional point cloud of the environment, and to determine the target point as the three-dimensional coordinate candidate point.

[0114] In some specific embodiments, the spatial anchor point acquisition module 14 further includes: The position constraint unit is used to determine the position of the target three-dimensional space anchor point. If the target three-dimensional space anchor point is located on any physical plane, the position of the target three-dimensional space anchor point is constrained to the any physical plane. The position binding unit is used to bind the position of the target three-dimensional spatial anchor point to the visual features of the target object if the target three-dimensional spatial anchor point is located on any target object.

[0115] In some specific embodiments, the spatial anchor point acquisition module 14 further includes: The position transformation unit is used to transform the first position of the target three-dimensional space anchor point in the preset three-dimensional coordinate system into the second position in the camera coordinate system at the current moment; An anchor point projection unit is used to project the target three-dimensional space anchor point onto the screen according to the second position, so as to draw and overlay virtual content on the target three-dimensional space anchor point.

[0116] In some specific embodiments, the three-dimensional spatial anchor point generation device further includes: The information storage module is used to store the target three-dimensional spatial anchor point and the environmental feature information associated with the target three-dimensional spatial anchor point, so that when the user re-enters the same environment, virtual content can be displayed based on the target three-dimensional spatial anchor point and the environmental feature information.

[0117] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0118] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional spatial anchor point generation method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0119] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0120] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0121] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the three-dimensional spatial anchor point generation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0122] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned three-dimensional spatial anchor point generation method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0124] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0126] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating three-dimensional spatial anchor points, characterized in that, Applications in AR glasses include: The target signal parsing method is used to acquire several two-dimensional target pointing and positioning results generated by the user through different indication methods in real time, and the two-dimensional target pointing and positioning results are converted into two-dimensional pixel coordinates on the imaging plane. Based on the current camera pose of the local preset camera, the two-dimensional pixel coordinates are back-projected into a target three-dimensional spatial ray in a preset three-dimensional coordinate system; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera based on the initial camera pose of the environment, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. The operation of finding the intersection points between the target's three-dimensional spatial rays and the environmental geometry includes: If the environmental geometric information is the depth information at the two-dimensional pixel coordinates, then a point on the target three-dimensional spatial ray with a distance corresponding to the depth information is determined, and the point with a distance corresponding to the depth information is determined as the candidate point of the three-dimensional coordinates; If the environmental geometry information is the plane parameter of any physical plane in the environment, then the plane intersection point of the target three-dimensional spatial ray and the physical plane is calculated based on the plane parameter, and the plane intersection point is determined as the candidate point of the three-dimensional coordinates; If the environmental geometric information is a sparse three-dimensional point cloud of the environment, then the target point closest to the target three-dimensional spatial ray is determined in the sparse three-dimensional point cloud, and the target point is determined as the candidate point of the three-dimensional coordinates, so as to obtain the corresponding candidate point of the three-dimensional coordinates and the uncertainty corresponding to the candidate point of the three-dimensional coordinates; based on the pixel observation noise, depth measurement error, plane fitting residual or the quality of the map itself, the uncertainty of the three-dimensional observation point is quantitatively estimated and expressed as a covariance matrix; Based on the aforementioned uncertainty, the candidate three-dimensional coordinates are weighted and fused with several historical three-dimensional spatial anchors to obtain the target three-dimensional spatial anchor. Based on the weighted least squares batch optimization or Kalman filter probabilistic fusion method, multiple three-dimensional observation points on the time series are recursively fused and their states are updated.

2. The three-dimensional spatial anchor generation method of claim 1, wherein, Before acquiring the two-dimensional pointing and positioning results of several targets generated by the user through different indication methods in real time using the target signal parsing method, the method further includes: The focal length and lens distortion of the preset camera lens are corrected, the positional and directional relationships of the local preset motion sensor, preset eye tracker and camera lens are determined, and the preset three-dimensional coordinate system is established for the current environment based on synchronous positioning and mapping technology.

3. The three-dimensional spatial anchor generation method of claim 1, wherein, The step of converting the target two-dimensional pointing positioning result into two-dimensional pixel coordinates on the imaging plane includes: If the user's instruction method is eye gaze, then the two-dimensional pixel coordinates of the user's gaze on the imaging plane are directly obtained; If the user's instruction method is a gesture instruction, then calculate the intersection point of the dotted line extending from the user's finger and the camera screen, and determine the coordinates of the intersection point on the imaging plane as the two-dimensional pixel coordinates; If the user's instruction method is voice instruction, then the natural language recognition model is used to identify the target object indicated by the user, and the coordinates of the center point of the outline of the target object on the imaging plane are determined as the two-dimensional pixel coordinates.

4. The three-dimensional spatial anchor generation method of claim 1, wherein, After weighting and fusing the candidate 3D coordinates with several historical 3D spatial anchors based on the uncertainty, the method further includes: The position of the target three-dimensional space anchor point is determined. If the target three-dimensional space anchor point is located on any physical plane, the position of the target three-dimensional space anchor point is constrained to the any physical plane. If the target 3D spatial anchor point is located on any target object, then the position of the target 3D spatial anchor point is bound to the visual features of the target object.

5. The three-dimensional spatial anchor generation method of claim 1, wherein, After weighting and fusing the candidate 3D coordinates with several historical 3D spatial anchors based on the uncertainty, the method further includes: The first position of the target three-dimensional spatial anchor point in the preset three-dimensional coordinate system is converted into the second position in the camera coordinate system at the current moment; The target three-dimensional spatial anchor point is projected onto the screen according to the second position, so as to draw and overlay virtual content on the target three-dimensional spatial anchor point.

6. The method for generating three-dimensional spatial anchor points according to any one of claims 1 to 5, characterized in that, Also includes: The target 3D spatial anchor point and the environmental feature information associated with the target 3D spatial anchor point are stored so that when the user re-enters the same environment, virtual content is displayed based on the target 3D spatial anchor point and the environmental feature information.

7. A three-dimensional spatial anchor point generation device, characterized in that, Applications in AR glasses include: The positioning result conversion module is used to acquire in real time a number of target two-dimensional pointing positioning results generated by the user through different indication methods using the target signal parsing method, and convert the target two-dimensional pointing positioning results into two-dimensional pixel coordinates on the imaging plane. The coordinate back projection module is used to back project the two-dimensional pixel coordinates into a target three-dimensional spatial ray in a preset three-dimensional coordinate system based on the current camera pose of the local preset camera; wherein, the preset three-dimensional coordinate system is a three-dimensional coordinate system constructed by the preset camera on the environment according to the initial camera pose, and the target three-dimensional spatial ray is a virtual ray emanating from the center of the lens of the preset camera and passing through the two-dimensional pixel coordinates. The candidate point acquisition module is used to perform an intersection operation between the target's three-dimensional spatial ray and the environmental geometric information to obtain the corresponding three-dimensional coordinate candidate points and the uncertainty corresponding to the three-dimensional coordinate candidate points; based on pixel observation noise, depth measurement error, plane fitting residual or the quality of the map itself, the uncertainty of the three-dimensional observation points is quantitatively estimated and expressed as a covariance matrix; The spatial anchor point acquisition module is used to perform weighted fusion of the candidate three-dimensional coordinates and several historical three-dimensional spatial anchor points based on the uncertainty to obtain the target three-dimensional spatial anchor point; and to perform recursive fusion and state update of multiple three-dimensional observation points on the time series based on the weighted least squares batch optimization or Kalman filter probabilistic fusion method. The candidate point acquisition module includes: The first candidate point determination unit is used to determine, if the environmental geometric information is the depth information at the two-dimensional pixel coordinates, a point on the target three-dimensional spatial ray with a distance corresponding to the depth information, and to determine the point with a distance corresponding to the depth information as the three-dimensional coordinate candidate point. The second candidate point determination unit is used to calculate the plane intersection point of the target three-dimensional spatial ray and the physical plane based on the plane parameters of any physical plane in the environment if the environmental geometric information is a plane parameter of any physical plane in the environment, and determine the plane intersection point as the three-dimensional coordinate candidate point. The third candidate point determination unit is used to determine the target point in the sparse three-dimensional point cloud that is closest to the target three-dimensional spatial ray if the environmental geometric information is a sparse three-dimensional point cloud of the environment, and to determine the target point as the three-dimensional coordinate candidate point.

8. An electronic device, characterized in that, include: Memory is used to store computer programs; A processor for executing the computer program to implement the three-dimensional spatial anchor point generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the three-dimensional spatial anchor point generation method as described in any one of claims 1 to 6.

Images