Fusion positioning method, system, electronic device and storage medium based on local tracking

Abstract

The present application relates to a fusion positioning method based on local tracking, wherein the method includes obtaining the first pose at the first moment, the current pose at the current moment, and the relative motion trajectory of the local tracking system between the first moment and the current moment; determining the weight coefficients of the first pose and the current pose respectively based on a preset strategy, and optimizing the current pose according to the first pose and its weight coefficient, the current pose and its weight coefficient, and the relative motion trajectory. The low-precision 3Dof positioning result is updated and optimized by the high-precision tracking and positioning result, and under the premise of ensuring the positioning accuracy, the positioning interaction in the AR scene can be realized efficiently and at low cost without any prior information.

Description

Technical Field

[0001] The present application relates to the field of augmented reality, and in particular to a fusion positioning method, system, computer device and computer-readable storage medium based on local tracking. Background Art

[0002] To achieve a large-scale augmented reality (AR) experience, SLAM or SFM+scale recovery technology is used to pre-reconstruct the target scene in 3D to obtain the corresponding AR map of the scene. Furthermore, by integrating visual positioning with local tracking, the transformation relationship between the high-frame-rate camera and the AR map is calculated. Based on this transformation relationship, the local tracking pose is converted into a pose in the AR map coordinate system, allowing users to experience the AR content in the space based on this pose.

[0003] Due to the complex acquisition routes and high computational costs of technologies such as SLAM and SFM+scale restoration, it is difficult to build a large experience area at low cost. In addition, for specific scenes such as rich vegetation, water surfaces, high reflectivity, and weak textures, visual mapping and positioning are not applicable due to the excessive complexity of the scenes or the excessive number of interference factors.

[0004] Since it is difficult to build a large-scale AR experience area at low cost to achieve visual positioning, related technologies usually perform positioning correction based on low-precision 3Dof positioning information combined with additional prior information. For example, in the field of autonomous driving, image recognition technology is used to identify specific objects in the scene, such as curbs and traffic signs, and the recognition results are used to perform timed correction on the low-precision 3Dof positioning results.

[0005] However, the implementation of the above method requires the presence of specific identifiable objects at fixed times or points in the scene, and needs to be implemented based on image recognition technology. Therefore, it is not only difficult to apply it in batches in any common scene, but the image recognition process also requires a lot of computing power, and the training process of the image recognition model is also relatively complicated. Summary of the Invention

[0006] The embodiments of the present application provide a fusion positioning method, system, computer device and computer-readable storage medium based on local tracking to at least solve the problems of high cost and large computing power consumption of positioning methods in related technologies.

[0007] In a first aspect, an embodiment of the present application provides a fusion positioning method based on local tracking, the method comprising:

[0008] Obtaining a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of the local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map;

[0009] The weight coefficients of the first posture and the current posture are determined based on a preset strategy, and the current posture is optimized according to the first posture and its weight coefficient, the current posture and its weight coefficient, and the relative motion trajectory.

[0010] In some embodiments, determining weight coefficients of the first posture and the current posture based on a preset strategy includes:

[0011] Obtaining uncertainties of the first posture and the current posture respectively, wherein the uncertainties include position uncertainty and angle uncertainty;

[0012] According to the uncertainties of the first posture and the current posture, weight coefficients of the first posture and the current posture are calculated respectively, wherein the weight coefficients include a position weight coefficient and an angle weight coefficient.

[0013] In some embodiments, optimizing the current posture includes optimizing a position component and optimizing an angle component.

[0014] In some embodiments, optimizing the position component of the current pose includes:

[0015] Obtaining a weighted posture mean of the first posture and the current posture according to the first posture and its position weight coefficient, the current posture and its position weight coefficient;

[0016] Determining a direction vector between the first posture and the current posture, and determining a tracking trajectory length between the first moment and the current moment based on the relative motion trajectory;

[0017] The position component of the current posture is optimized according to the posture mean, the direction vector and the tracking trajectory length.

[0018] In some embodiments, optimizing the angle component of the current posture according to the angle weight coefficients of the first posture and the current posture includes:

[0019] Converting the angle components of the first posture and the current posture into quaternion representations respectively to obtain a first angle component and a second angle component;

[0020] Obtaining a median angle component between the first posture and the current posture by performing a spherical linear difference on the first angle component and the second angle component;

[0021] Performing a quaternion transformation on the relative motion trajectory, determining a tracking angle variable between the first moment and the current moment according to the transformed relative motion trajectory, and determining a median tracking angle variable corresponding to the tracking angle variable through a spherical linear interpolation;

[0022] Based on the median angle component and the median tracking angle variable, an angle component optimization result of the current posture is obtained.

[0023] In some embodiments, performing a quaternion transformation on the relative motion trajectory, and determining a tracking angle variable between the first moment and the current moment based on the transformed relative motion trajectory includes:

[0024] acquiring, according to the relative motion trajectory, a tracking angle component at the first moment and a tracking angle component at the current moment;

[0025] Converting the tracking angle component at the first moment and the tracking angle component at the current moment into quaternion representations respectively to obtain a first tracking angle component and a second tracking angle component;

[0026] A tracking angle variable is determined based on the first tracking angle component and the second tracking angle component.

[0027] In some embodiments, the method further comprises:

[0028] Obtaining a pitch angle and a roll angle at a target moment from the relative motion trajectory, and determining a three-dimensional vector formed by the pitch angle and the roll angle;

[0029] Determine a first rotation matrix from the pitch angle and the roll angle to a gravity direction according to the three-dimensional vector;

[0030] Determining a second rotation matrix corresponding to an angle component from the local positioning pose of the target at the time, and obtaining a gravity-aligned rotation matrix based on the second rotation matrix in quaternion form and the first rotation matrix;

[0031] The gravity-aligned transformation matrix is ​​spliced ​​with the tracking posture at the target moment to obtain the positioning posture after gravity alignment at the target moment.

[0032] In a second aspect, an embodiment of the present application provides a fusion positioning system based on local tracking, the system comprising: an acquisition module and an optimization module, wherein:

[0033] The acquisition module is used to obtain a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of the local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map;

[0034] The optimization module is used to determine the weight coefficients of the first posture and the current posture respectively based on a preset strategy, and optimize the current posture according to the first posture and its weight coefficient, the current posture and its weight coefficient, and the relative motion trajectory.

[0035] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0037] Compared with related technologies, the fusion positioning method based on local tracking provided by the embodiment of the present application obtains the first pose at the first moment, the current pose at the current moment, and the relative motion trajectory of the local tracking system between the first moment and the current moment; determines the weight coefficients of the first pose and the current pose respectively based on the preset strategy, and optimizes the current pose according to the first pose and its weight coefficient, the current pose and its weight coefficient, and the relative motion trajectory. The low-precision 3Dof positioning results are updated and optimized through the high-precision tracking and positioning results. Under the premise of ensuring the positioning accuracy, it is possible to realize positioning interaction in AR scenarios efficiently and at low cost without any prior information. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 2 is a schematic diagram of an application environment of a fusion positioning method based on local tracking according to an embodiment of the present application;

[0040] Figure 2 This is a flow chart of a fusion positioning method based on local tracking according to an embodiment of the present application.

[0041] Figure 3 is a structural block diagram of a fusion positioning system based on local tracking according to an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of performing posture optimization on position components according to an embodiment of the present application;

[0043] Figure 5 is a schematic diagram of performing posture optimization on angle components according to an embodiment of the present application;

[0044] Figure 6 Schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0046] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0047] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0048] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "current", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0049] It should be understood that the terms herein may be technical means for implementing a part of the present invention or other summary technical terms. For example, the terms may include:

[0050] AR (Argument Reality): Augmented reality is a technology that cleverly integrates virtual information with the real world. It simulates computer-generated virtual information such as text, images, models, videos, and then applies them to the real world. The two types of information complement each other to achieve "enhancement" of the real world.

[0051] Pose: position and attitude (orientation), for example: in two dimensions it is generally 3Dof (x, y, yaw), in three dimensions it is generally 6Dof (x, y, z, yaw, pitch, roll), the last three elements describe the attitude of the object, where yaw is the heading angle, rotating around the Z axis; pitch is the pitch angle, rotating around the Y axis; roll is the roll angle, rotating around the X axis.

[0052] 3Dof positioning: uses GPS, Bluetooth, Wi-Fi and other positioning modules, combined with electronic compass and other orientation positioning modules, to obtain 3-degree-of-freedom (x, y, yaw) position information. This positioning method usually has large accuracy errors.

[0053] Local positioning system: A system that performs positioning in a local coordinate system, such as ARKit for iOS phones, ARCore for Android phones, AREngine for Huawei phones, and stereo-VIO commonly used in AR glasses, with small accuracy errors.

[0054] The fusion positioning method based on local tracking provided in this application can be applied in Figure 1 In the application environment shown, Figure 1 FIG. 1 is a schematic diagram of an application environment of a fusion positioning method based on local tracking according to an embodiment of the present application. Figure 1 As shown, the terminal device 10 moves in the target scene and obtains a low-precision 3Dof positioning result from the server 11 at a fixed interval during the movement. At the same time, the terminal device 10 runs a tracking system locally to obtain the relative motion trajectory between the two positioning results. Each time positioning is performed, the terminal device 10 can perform a weighted operation based on the error accuracy of the above-mentioned multiple positioning results to optimize the current positioning result. It should be noted that the above-mentioned terminal device 10 can be a smart device such as a smartphone, a tablet computer, and AR glasses, and the server 11 includes but is not limited to deploying a combination of one or more Bluetooth positioning services, GPS positioning services, and WIFI positioning services.

[0055] Figure 2 is a flow chart of a fusion positioning method based on local tracking according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0056] S201, obtaining a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of a local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map;

[0057] In this embodiment, the first posture and the current posture are postures with a temporal relationship, and the specific time interval between the two is not specifically limited in this embodiment.

[0058] In addition, the first pose and the current pose are both the poses of the camera field of view corresponding to the AR map, which can be but not limited to 3Dof positioning poses obtained through GPS, Bluetooth, and WiFi, or 6Dof positioning poses in AR interaction scenarios with lower precision requirements.

[0059] It should be noted that the first pose can be the positioning pose obtained directly by the positioning system in the initial state (first positioning), or it can be the positioning pose at any moment during the movement. The first pose can be obtained directly by the 3Dof positioning system, or it can be the positioning result after the 3Dof is positioned and corrected by the local positioning system. For example, when the application scenario of the present application solution is an AR interaction scenario during the user's walking process, the first pose and the current pose can be the positioning pose in the user's initial state and the positioning pose at the 5th second, or the positioning pose at the 30th second and the 35th second during the user's walking process.

[0060] A local tracking system uses camera and inertial measurement unit (IMU) data to estimate the camera's position and motion trajectory. It uses continuous camera images and IMU signals to calculate the camera's position and orientation in three-dimensional space. In this embodiment, the local tracking system is used to obtain the relative motion trajectory of the terminal device in the local coordinate system between a first moment and the current moment. This relative motion trajectory consists of a series of continuous image frames and the local positioning pose corresponding to each image frame.

[0061] S202, based on a preset strategy, respectively determine the weight coefficients of the first pose and the current pose, and optimize the current pose according to the first pose and its weight coefficient, the current pose and its weight coefficient, and the relative motion trajectory.

[0062] In this embodiment, in the process of determining the weight coefficients of the two positioning results respectively through the above preset strategy, the accuracy of the two positioning results is taken as the main consideration, and the weight of each positioning result is determined according to the accuracy of the positioning results. Therefore, the positioning result with higher accuracy can obtain a higher weight, while the sample with lower accuracy will obtain a lower weight.

[0063] The weight coefficients of the two positioning results are obtained through the above-mentioned weight configuration method, and then the two 3Dof positioning results and the positioning results of the tracking system are fused according to the weight coefficients. Compared with the traditional method of multi-information fusion through Kalman filtering, particle filtering, etc., while being able to achieve a similar fusion effect, the technical means adopted in this application scheme have lower computational complexity, and therefore less computing power consumption. Whether on the terminal side or on the server side, positioning fusion can be achieved more efficiently.

[0064] Through the above steps S201 to S202, compared with the positioning methods in the prior art, this embodiment uses low-precision 3Dof positioning to obtain the position and posture, and combines it with a high-precision local positioning system for correction. Without the need for complex, time-consuming and labor-intensive three-dimensional map construction, efficient and accurate user position positioning can be achieved. At the same time, because the application solution only uses local tracking as a constraint to improve the accuracy of 3Dof positioning posture, it does not need to obtain prior information through image recognition, navigation path and other means. Therefore, the application cost is relatively low and it can be applied to various common scenarios.

[0065] In addition, the present application solution adopts a process of configuring weight values ​​based on the size of accuracy and performing fusion according to the weight values ​​in the process of fusing multiple positioning results. Since the information fusion process does not require high-computing-power-consuming operations such as matrix inversion and the calculation frequency is low, the overall computing power consumption is also lower. This solution can be deployed on a lightweight computing platform, such as webAR.

[0066] In some embodiments, considering that the positioning posture includes position and orientation, the solution of the present application calculates weight coefficients for position and orientation respectively, specifically including the following steps:

[0067] Step 1: Obtain the uncertainty of the first pose and the current pose respectively. The uncertainty includes position uncertainty and angle uncertainty. The uncertainty is used to describe the error size of the state (such as pose) and is usually expressed by standard deviation (one-dimensional variables) or covariance matrix (multi-dimensional variables). The larger the value, the less reliable it is, and the smaller the value, the more reliable it is.

[0068] Step 2: Calculate the weight coefficients of the first pose and the current pose according to their uncertainties, where the weight coefficients include position weight coefficient and angle weight coefficient.

[0069] It can be understood that in this step, weight coefficients need to be configured for the position component and the angle component of the posture respectively.

[0070] This embodiment determines the weight coefficient based on the accuracy of the positioning result. The weights of the first and current moments are assigned as the inverse of their respective accuracy values. The smaller the variance, the larger the inverse. Therefore, positioning poses with smaller accuracy values ​​have larger weights, while positioning poses with larger accuracy values ​​have smaller weights. This weight distribution method can better reflect the degree of positioning accuracy, so that moments with higher positioning accuracy have a greater impact on the final position estimate.

[0071] Optionally, configuring weight coefficients for the first pose and the second pose includes:

[0072] The xy (position) weight coefficient of two consecutive 3dof positioning results is calculated using the following expression:

[0073] sum=1.0 / xy_t0_std^2+1.0 / xy_t1_std^2

[0074] weight_t0=(1.0 / xy_t0_std^2) / sum

[0075] weight_t1=(1.0 / xy_t1_std^2) / sum

[0076] Among them, xy_t0_std and xy_t1_std are the standard deviations of the first pose and the current pose in the xy direction, respectively, which are used to indicate the accuracy of the positioning result in the xy direction; sum represents the sum of the reciprocals of the two accuracy parameters in the xy direction; weight_t0 and weight_t1 represent the xy direction weight coefficients of the first and second positioning results, respectively.

[0077] The yaw (angle) weight coefficient of two consecutive 3Dof positioning results is calculated using the following expression:

[0078] sum'=1.0 / yaw_t0_std^2+1.0 / yaw_t1_std^2

[0079] weight_t0'=(1.0 / yaw_t0_std^2) / sum'

[0080] weight_t1'=(1.0 / yaw_t1_std^2) / sum'

[0081] Among them, yaw_t0_std and yaw_t1_std are the standard deviations of the first and current poses in yaw, respectively, which are used to indicate the accuracy of the positioning result in the yaw direction. sum' represents the sum of the reciprocals of the two accuracy values ​​in yaw; weight_t0' and weight_t1' represent the yaw weight coefficients of the first and second positioning results, respectively.

[0082] In some embodiments, optimizing the current posture according to the first posture and its weight coefficient, the current posture and its weight coefficient, and the relative motion trajectory includes: optimizing the position component and optimizing the angle component.

[0083] It should be noted that traditional algorithms such as Kalman filtering optimize position and rotation in the same 4*4 matrix. Therefore, this type of method requires correlation between position and rotation. In the 3Dof positioning results used in this embodiment, position comes from GPS, Bluetooth, and WIFI, and rotation comes from an electronic compass. That is, the position and rotation in the positioning results are obtained through different systems, and there is no correlation between the two. In this application, weight parameters are configured based on the accuracy of each positioning result in position and angle, and the position component and angle component are optimized separately according to the position weight coefficient and angle weight coefficient. This is more suitable for 3Dof positioning results and can also obtain more accurate optimization results.

[0084] Specifically: the bit component optimization includes the following steps:

[0085] Step 1: According to the first pose and its position weight coefficient, the current pose and its position weight coefficient, obtain the weighted pose mean of the first pose and the current pose;

[0086] Step 2: Determine the direction vector between the first pose and the current pose, and determine the tracking trajectory length between the first moment and the current moment based on the relative motion trajectory;

[0087] Step 3: Optimize the position component of the current pose based on the pose mean, direction vector and tracking trajectory length.

[0088] Optionally, optimize the position component of the current pose using the following expression:

[0089] xy_mid=weight_t0*xy_t0+weight_t1*xy_t1

[0090] xy_dir=(xy_t1-xy_t0) / norm((xy_t1-xy_t0))

[0091] track_length=norm(track_t1-track_t0)

[0092] xy_t1_new=xy_mid+0.5*track_length*xy_dir

[0093] Among them, xy_t0 and xy_t1 are the position coordinates corresponding to the first pose and the current pose respectively, xy_mid is the weighted pose mean, xy_dir represents the direction vector from xy_t0 to xy_t1, track_t0 and track_t1 represent the relative motion trajectory obtained by local tracking between two positioning, track_length is the length of the relative motion trajectory, and xy_t1_new is the position component of the current pose after optimization.

[0094] Furthermore, the angle component optimization includes the following steps:

[0095] Step 1: Convert the angle components of the first pose and the current pose into quaternion representations to obtain the first angle component and the second angle component;

[0096] Step 2, obtain the median angle component between the first pose and the current pose by performing spherical linear interpolation on the first angle component and the second angle component;

[0097] Step 3, performing quaternion transformation on the relative motion trajectory, determining the tracking angle variable between the first moment and the current moment according to the relative motion trajectory after the transformation, and determining the median tracking angle variable corresponding to the tracking angle variable;

[0098] Step 4: Based on the median angle component and the median tracking angle variable, the angle component optimization result of the current posture is obtained.

[0099] The above step 3 specifically includes the following steps:

[0100] Step 3.1, according to the relative motion trajectory, obtain the tracking angle component at the first moment and the tracking angle component at the current moment respectively;

[0101] Step 3.2, convert the tracking angle component at the first moment and the tracking angle component at the current moment into quaternion representations respectively to obtain the first tracking angle component and the second tracking angle component;

[0102] Step 3.3, determine the tracking angle variable based on the first tracking angle component and the second tracking angle component.

[0103] Optionally, optimize the angle component of the current pose using the following expression:

[0104] Quaterniond yaw_t0,yaw_t1;

[0105] Quaterniond yaw_mid=yaw_t0.slerp(weight_t0',yaw_t1);

[0106] Quaterniond track_yaw_t0,track_yaw_t1;

[0107] Quaterniond track_delta_yaw=track_yaw_t0^(-1)*track_yaw_t1;

[0108] Quaterniond track_delta_yaw_half=identity.slerp(0.5,track_delta_yaw);

[0109] Quaterniond yaw_t1_new=yaw_mid*track_delta_yaw_half;

[0110] Among them, Quaterniond is used to represent quaternions, which can be written in the form of q=w+xi+yj+zk, where w, x, y and z are real numbers, and i, j and k are imaginary units that satisfy the multiplication rules.

[0111] In this embodiment, quaternions are used to represent rotational transformations in three-dimensional space. Combining this with spherical linear interpolation can avoid ambiguity in the rotation matrix and achieve smoother interpolation. Furthermore, because quaternions only require four floating-point numbers for storage, the technical approach of this embodiment is more efficient than other approaches.

[0112] Furthermore, in the above expression, Quaterniond is used to identify the data as quaternion, yaw_t0 and yaw_t1 are the angle components in the first pose and the current pose, yaw_mid is the median angle component obtained by spherical linear interpolation (Slerp) of yaw_t0 and yaw_t1; track_delta_yaw is the difference between the two rotation changes calculated using the ^ operator, that is, the tracking angle variable in the two tracking states of Quaterniond type;

[0113] track_delta_yaw_half interpolates the tracking angle variable to half of the unit quaternion (weight is 0.5) to obtain the median tracking angle variable of Quaterniond type; yaw_t1_new is the Quaterniond type yaw_t1_new variable obtained based on the two variables yaw_mid and track_delta_yaw_half, that is, the optimization result of the current positioning pose in the angle component.

[0114] In some of the embodiments, in actual applications, the posture of the camera may change, which may result in different directions and rotations in the positioning results. Therefore, the camera needs to be gravity aligned so that the direction of the collected data in the camera coordinate system is consistent with the direction in the world coordinate system, which facilitates subsequent data processing and calculation.

[0115] When the solution of this embodiment is used for 3Dof positioning, since there are only three degrees of freedom (x, y, and yaw), the gravity-aligned pose can be obtained from the tracking pose at the corresponding moment. For example, the first pose (pose_t0_to_armap) of the device in the map coordinate system at the current moment and the first tracking pose (pose_t0_to_trackworld) in the tracking coordinate system are used to determine the pose after gravity alignment, specifically including:

[0116] Step 1: Obtain the pitch angle and roll angle at the first moment from the relative motion trajectory, and determine the three-dimensional vector formed by the pitch angle and roll angle;

[0117] Step 2: Determine a first rotation matrix from the pitch angle and roll angle to the gravity direction based on the three-dimensional vector, and convert the first rotation matrix into a quaternion representation;

[0118] Step 3: Determine the second rotation matrix corresponding to the angle component from the current posture, and obtain the gravity-aligned rotation matrix based on the second rotation matrix in quaternion form and the first rotation matrix;

[0119] Step 4: Concatenate the gravity-aligned transformation matrix with the tracking pose at the current moment in the relative motion trajectory to obtain the positioning pose after gravity alignment at the current moment.

[0120] Optionally, taking the first moment as an example, the rotation matrix after gravity alignment at the first moment is obtained by the following expression:

[0121] pitch_roll_vec3=[pose_t0_to_trackworld[2,0],pose_t0_to_trackworld[2,1],pose_t0_to_trackworld[2,2]];

[0122] Quaterniond quat_pitch_roll=setFromTwoVectors(pitch_roll_vec3,[0,0,1])Matrix3d ​​rot_yaw=AngleAxisd(yaw,[0,0,1]);

[0123] Matrix3d ​​rot_t0_to_armap=rot_yaw*quat_pitch_roll.toRotationMatrix();

[0124] The implementation logic of the above expression can be expressed as:

[0125] 1. Get the pitch and roll vectors from the tracking pose, i.e. pitch_roll_vec3 = [pose_t0_to_trackworld[2,0],pose_t0_to_trackworld[2,1],pose_t0_to_trackworld[2,2]];

[0126] 2. Calculate the rotation matrix from pitch and roll to the direction of gravity, that is, Quaterniond quat_pitch_roll = setFromTwoVectors(pitch_roll_vec3,[0,0,1]);

[0127] 3. Get the rotation matrix of yaw at the first moment, that is, Matrix3d ​​rot_yaw = AngleAxisd(yaw, [0, 0, 1]);

[0128] 4. Multiply the rotation matrix formed by the current yaw by the rotation matrix from pitch and roll to the direction of gravity, that is, rot_t0_to_armap = rot_yaw * quat_pitch_roll.toRotationMatrix();

[0129] 5. Concatenate the gravity-aligned rotation matrix and the position in the tracking pose to get the gravity-aligned pose at the first moment.

[0130] This embodiment also provides a fusion positioning system based on local tracking, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated. As used below, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0131] Figure 3 is a structural block diagram of a fusion positioning system based on local tracking according to an embodiment of the present application, such as Figure 3 As shown, the system includes an acquisition module 30 and an optimization module 31, wherein:

[0132] The acquisition module 30 is used to obtain a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of the local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map;

[0133] The optimization module 31 is used to determine the weight coefficients of the first pose and the current pose based on a preset strategy, and optimize the current pose according to the first pose and its weight coefficient, the current pose and its weight coefficient, and the relative motion trajectory.

[0134] This system corrects the user's posture by combining low-precision 3Dof positioning with a high-precision local positioning system. It does not require complex, time-consuming and labor-intensive three-dimensional map construction, and can achieve efficient and accurate user posture positioning. At the same time, since the application proposal only uses local tracking as a constraint to improve the accuracy of 3Dof positioning, there is no need to obtain prior information through image recognition, navigation paths, etc., so the application cost is also low and it can be applied to various common scenarios.

[0135] It should be noted that since the positioning error of two consecutive 3Dof positionings in a short period of time is much larger than the cumulative error of the tracking system, in actual engineering applications, the positioning error of the local tracking system between two consecutive 3Dof positionings can be considered negligible. Specifically:

[0136] like Figure 4 As shown in the figure, assuming that the standard deviation of xy of each 3Dof positioning is 4m, assuming that the walking speed is a constant 1m / s, and the interval between two 3Dof positioning is 5s, the walking distance is 5m. According to the cumulative error of local tracking of 1%, the cumulative error is 0.05m. The maximum position error of 3Dof positioning is 8 / 0.05=160 times the cumulative error of local tracking. Since the position error of 3Dof is much larger than the position error of the tracking system, the error of the local tracking system in the position component between two consecutive 3Dof positioning can be ignored.

[0137] like Figure 5 As shown, assuming that the standard deviation of the orientation yaw of each 3dof positioning is 17°, the yaw angle difference between the two 3dof positioning is magn_delta_yaw = m±2*17°, and the angular error of local tracking is very small track_delta_yaw = n±0.1°. The maximum orientation error of 3dof positioning is 34 / 0.1 = 340 times the cumulative error of local tracking. Therefore, between the two 3dof positioning, the error of the local tracking system in the angular component can also be ignored.

[0138] In one embodiment, Figure 6is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 6 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected via an internal bus, wherein the non-volatile memory stores an operating system, a computer program, and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with external terminals via a network connection, the internal memory is used to provide an environment for the operation of the operating system, the computer program, when executed by the processor, implements a fusion positioning method based on local tracking, and the database is used to store data.

[0139] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

[0141] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A fusion positioning method based on local tracking, characterized in that: The method comprises: Obtaining a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of the local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map; The weight coefficients of the first posture and the current posture are determined based on a preset strategy, and the current posture is optimized according to the first posture and its weight coefficient, the current posture and its weight coefficient, and the relative motion trajectory.

2. The method according to claim 1, characterized in that Determining weight coefficients of the first posture and the current posture based on a preset strategy includes: Obtaining uncertainties of the first posture and the current posture respectively, wherein the uncertainties include position uncertainty and angle uncertainty; According to the uncertainties of the first posture and the current posture, weight coefficients of the first posture and the current posture are calculated respectively, wherein the weight coefficients include a position weight coefficient and an angle weight coefficient.

3. The method according to claim 2, characterized in that Optimizing the current posture includes: optimizing the position component and optimizing the angle component.

4. The method according to claim 3, characterized in that Optimizing the position component of the current posture includes: Obtaining a weighted pose mean of the first pose and the current pose according to the first pose and its position weight coefficient, the current pose and its position weight coefficient; Determining a direction vector between the first posture and the current posture, and determining a tracking trajectory length between the first moment and the current moment based on the relative motion trajectory; The position component of the current posture is optimized according to the posture mean, the direction vector and the tracking trajectory length.

5. The method according to claim 3, characterized in that Optimizing the angle component of the current posture according to the angle weight coefficients of the first posture and the current posture includes: Converting the angle components of the first posture and the current posture into quaternion representations respectively to obtain a first angle component and a second angle component; Obtaining a median angle component between the first posture and the current posture by performing spherical linear interpolation on the first angle component and the second angle component converted into quaternions; Performing a quaternion transformation on the relative motion trajectory, determining a tracking angle variable between the first moment and the current moment according to the transformed relative motion trajectory, and determining a median tracking angle variable corresponding to the tracking angle variable through a spherical linear interpolation; Based on the median angle component and the median tracking angle variable, an angle component optimization result of the current posture is obtained.

6. The method according to claim 5, characterized in that Performing quaternion transformation on the relative motion trajectory, and determining a tracking angle variable between the first moment and the current moment according to the transformed relative motion trajectory includes: acquiring, according to the relative motion trajectory, a tracking angle component at the first moment and a tracking angle component at the current moment; Converting the tracking angle component at the first moment and the tracking angle component at the current moment into quaternion representations respectively to obtain a first tracking angle component and a second tracking angle component; A tracking angle variable is determined based on the first tracking angle component and the second tracking angle component.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a pitch angle and a roll angle at a target moment from the relative motion trajectory, and determining a three-dimensional vector formed by the pitch angle and the roll angle; Determine a first rotation matrix from the pitch angle and the roll angle to a gravity direction according to the three-dimensional vector; Determining a second rotation matrix corresponding to an angle component from the local positioning pose of the target at the time, and obtaining a gravity-aligned rotation matrix based on the second rotation matrix in quaternion form and the first rotation matrix; The gravity-aligned transformation matrix is ​​spliced ​​with the tracking posture at the target moment to obtain the positioning posture after gravity alignment at the target moment.

8. A fusion positioning system based on local tracking, characterized in that: The system includes: an acquisition module and an optimization module, wherein: The acquisition module is used to obtain a first pose at a first moment, a current pose at a current moment, and a relative motion trajectory of the local tracking system between the first moment and the current moment, wherein the first pose and the current pose are positioning poses corresponding to the camera field of view in the AR map; The optimization module is used to determine the weight coefficients of the first posture and the current posture respectively based on a preset strategy, and optimize the current posture according to the first posture and its weight coefficient, the current posture and its weight coefficient, and the relative motion trajectory.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images