Decision-level fusion method for multi-positioning systems in augmented reality

Abstract

The application relates to a decision-level fusion method of multiple positioning systems in augmented reality, which is applied to a large-space augmented reality scene and comprises the following steps: in the case that an AR device receives a second positioning result, a backend obtains a fusion positioning result according to the second positioning result and a current positioning result, and judges whether to perform positioning update based on the uncertainty of the current positioning result and the uncertainty of the fusion positioning result; if yes, the current positioning result is updated to the fusion positioning result for AR interaction, wherein the second positioning result is derived from any type of positioning system. In the application, different positioning systems are regarded as black boxes, and the uncertainty is used as a unified selection standard to judge whether to perform positioning update, and the process does not need to involve algorithms and is simple and efficient. The problems of large operation amount and poor efficiency of the multi-positioning system fusion method in the related art are solved, the device-side algorithm consumption is reduced, and the fusion efficiency of the positioning result is improved.

Description

Technical Field

[0001] This application relates to the field of augmented reality, and in particular to a decision-level fusion method, system, computer device, and computer-readable storage medium for multi-location systems in augmented reality. Background Technology

[0002] In the field of large-scene argumented reality (LSAR), users can experience AR through terminal devices such as mobile phones and AR glasses. The large scene may include indoor and outdoor connected scenes.

[0003] In large-scale scenarios, user terminals use preset positioning maps for positioning. For example, indoor locations are equipped with Bluetooth and visual maps, while outdoor locations are equipped with GPS and visual maps. As long as the coordinate system is aligned and managed, Bluetooth and visual positioning can be used simultaneously indoors, and GPS and visual positioning can be used simultaneously outdoors to achieve fusion positioning and seamlessly connect multiple specific scenarios.

[0004] In related technologies, the fusion of multiple positioning systems is generally based on learning algorithms for fault diagnosis, and specific multi-positioning system fusion is carried out through filtering and optimization algorithms. This process consumes a lot of computing power and has a slow response speed.

[0005] Currently, no effective solution has been proposed to address the issue of poor efficiency in multi-positioning system fusion methods under LSAR scenarios in related technologies. Summary of the Invention

[0006] This application provides a decision-level fusion method, system, computer device, and computer-readable storage medium for augmented reality multi-positioning systems, to at least solve the problem of poor efficiency of multi-positioning system fusion methods in LSAR scenarios in related technologies.

[0007] In a first aspect, embodiments of this application provide a decision-level fusion method for multiple positioning systems in augmented reality, applied to large-space augmented reality scenarios, the method comprising:

[0008] AR devices perform AR interactions in the target scene based on the current location results;

[0009] Upon receiving the second positioning result, the backend of the AR device obtains a fused positioning result based on the second positioning result and the current positioning result, and...

[0010] Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, determine whether to perform a positioning update; if so,

[0011] The current positioning result is updated to the fused positioning result for AR interaction, wherein the second positioning result comes from any type of positioning system.

[0012] In some embodiments, before the AR device performs AR interaction in the target scene based on the current positioning result, the method further includes:

[0013] The uncertainty of different types of positioning systems is evaluated offline, and the coordinate systems of the different types of positioning systems are aligned to the AR world coordinate system of the target scene.

[0014] In some embodiments, the positioning system includes: a Bluetooth positioning system, a visual positioning system, and a global positioning system.

[0015] In some embodiments, the current positioning result includes a fusion result of one or more of the following: the pose acquired by the AR device tracking system, the pose sent by the Bluetooth positioning system, the pose sent by the visual positioning system, and the pose sent by the global positioning system.

[0016] In some embodiments, determining whether to perform a location update based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result includes:

[0017] When the uncertainty of the current positioning result is less than the uncertainty of the fused positioning result, it is determined that the positioning update is required;

[0018] If the uncertainty of the current positioning result is greater than or equal to the uncertainty of the fused positioning result, it is determined that no positioning update is required.

[0019] In some embodiments, the need for the positioning update is determined by the following equation:

[0020] uncertainty_current_to_arworld<uncertainty_t1_to_arworld

[0021] Here, uncertainty_current_to_arworld is the uncertainty of the current localization result aligned to AR world coordinates, and uncertainty_t1_to_arworld is the uncertainty of the fused localization result aligned to AR world coordinates at time t1.

[0022] In some embodiments, the method further includes obtaining the current uncertainty generated due to the cumulative offset of the tracking system by linear extrapolation, including:

[0023] Obtain the initial uncertainty of the positioning results sent by the positioning system in the initial state;

[0024] The cumulative trajectory length of the tracking system is obtained by accumulating the trajectory length values ​​of the tracking system over multiple preset time periods;

[0025] The uncertainty change value of the tracking system is obtained based on the cumulative trajectory length and the cumulative error constant, wherein the cumulative error constant represents the cumulative error generated by the tracking system for each preset distance moved, which is obtained through offline evaluation;

[0026] Based on the initial uncertainty and the uncertainty change value of the tracking system, the current uncertainty generated by the cumulative offset of the tracking system is obtained.

[0027] Secondly, this application provides a decision-level fusion system for multiple positioning systems in augmented reality, applied in large-space augmented reality scenarios. The system includes: an AR device and multiple positioning systems, wherein:

[0028] The AR device is used to perform AR interaction in the target scene based on the current positioning result, and

[0029] Upon receiving the second positioning result, the backend is instructed to obtain a fused positioning result based on the second positioning result and the current positioning result, and...

[0030] Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, determine whether to perform a positioning update; if so,

[0031] The current positioning result is updated to the fused positioning result for AR interaction, wherein the second positioning result comes from any type of positioning system.

[0032] Thirdly, embodiments of this application provide a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0033] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0034] Compared to related technologies, the decision-level fusion method for multiple positioning systems in augmented reality provided in this application involves the AR device receiving a second positioning result, obtaining a fused positioning result based on the second and current positioning results, and determining whether to update the positioning based on the uncertainty of the current and fused positioning results. If so, the current positioning result is updated to the fused positioning result for AR interaction. The second positioning result originates from any type of positioning system. This application treats different positioning systems as black boxes, disregarding how the positioning data source is obtained, and uses uncertainty as a unified selection criterion to determine whether to update the positioning. This process requires no filtering or optimization algorithms and is simple and efficient. It solves the problems of high computational load and poor efficiency in multi-positioning system fusion methods under LSAR scenarios in related technologies, reducing device-side computing power consumption while improving the fusion efficiency of positioning results. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a schematic diagram of the application environment of the decision-level fusion method for multi-positioning systems in augmented reality according to the embodiments of this application;

[0037] Figure 2 This is a flowchart of a decision-level fusion method for a multi-location system in augmented reality according to an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of a continuous positioning update determination and fusion logic according to an embodiment of this application;

[0039] Figure 4 This is a structural block diagram of a decision-level fusion system for multiple positioning systems in augmented reality according to an embodiment of this application;

[0040] Figure 5 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0042] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0043] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0044] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0045] In this document, it should be understood that the terms used may be technical means used to implement part of the present invention or other summary technical terms. For example, the terms may include:

[0046] AR (Argument Reality): Augmented Reality is a technology that cleverly integrates virtual information with the real world. It simulates and applies computer-generated text, images, 3D models, videos, and other virtual information to the real world. The two types of information complement each other, thus "enhancing" the real world.

[0047] Pose: Position and orientation (facing), for example, in 2D it is generally (x, y, yaw), and in 3D it is generally (x, y, z, yaw, pitch, roll), including 6 degrees of freedom, 6 of (6 Degrees Of Freedom). The last three elements describe the object's orientation, where yaw is the heading angle, rotating around the Z-axis; pitch is the pitch angle, rotating around the Y-axis; and roll is the roll angle, rotating around the X-axis.

[0048] Uncertainty describes the magnitude of error in a state (such as a pose), and is usually represented by standard deviation (a one-dimensional variable) or covariance matrix (a multi-dimensional variable). A larger value indicates a less reliable result, while a smaller value indicates a more reliable result. Uncertainty describes the error interval and is mathematically represented as a 6x6 covariance matrix. This application uses a simplified version that only includes the uncertainty of the translation part, i.e., uncertainty_a_to_b is a 3x1 vector.

[0049] Positioning System: A system that determines the position and orientation of an object in a map coordinate system. The positioning result is represented as "the pose of object a in map b at time t".

[0050] Global Positioning System (GPS): A system that provides positioning based on a geographic coordinate system. GPS includes systems such as GPS, BeiDou, and Galileo that use artificial satellites for positioning. In this embodiment, GPS is used as the abbreviation.

[0051] Local Position System: A system that performs positioning in a local coordinate system. In this embodiment, the term "track" is used as a euphemism, such as ARKit on iOS phones, ARCore on Android phones, AREngine on Huawei phones, and stereo-VIO commonly used in AR glasses. The cumulative pose error is generally expressed as x%, meaning that the cumulative error is xm for every 100m movement.

[0052] Visual Positioning System: This system uses images for positioning in a visual map coordinate system. The uncertainty of the position is generally 0.2m. In this field, it can be referred to as "cloud positioning", "visual cloud positioning", "VPS", or "VC (visual cloud)".

[0053] Bluetooth Positioning System: A system that uses Bluetooth for positioning in a Bluetooth map coordinate system. In this embodiment, it can be referred to as BT. The position uncertainty varies depending on the sensor. The position uncertainty of ordinary Bluetooth 4.0 is generally 4m, while the position uncertainty of AOA (Angle of Arrival) Bluetooth is generally 0.5m.

[0054] Coordinate system transformation (abbreviated as T): can be represented by the formula X_b=T_a_to_b*X_a. Using the coordinate system transformation matrix T_a_to_b, a point X in coordinate system a can be transformed to coordinate system b to obtain X_b.

[0055] The decision-level fusion method for multi-positioning systems in augmented reality provided in this application can be applied to, for example... Figure 1 In the application environment shown, Figure 1 This is a schematic diagram illustrating the application environment of the decision-level fusion method for multi-positioning systems in augmented reality according to embodiments of this application, such as... Figure 1 As shown, users can interact with the target scene through AR device 10. The target scene is a large scene that connects indoor and outdoor spaces, such as scenic spots, parks, and exhibition halls. Different locations in the scene may be provided with positioning results by different positioning systems. For example, some indoor scenes may be provided with positioning results by Bluetooth positioning system 20 or visual positioning system 40, while some outdoor scenes may be provided with positioning results by GPS positioning system 20 and visual positioning system 40.

[0056] When a user moves from one location to another, there may be multiple location sources. The AR device 10 can, based on the decision-level fusion scheme provided in this application embodiment, use the uncertainty of pose as a unified standard to determine whether a pose update is needed at the current moment, and to re-fuse the pose and uncertainty. The decision-level fusion scheme provided in this embodiment is applied in the field of large-scene augmented reality, and the AR devices involved include, but are not limited to, smartphones, tablets, and AR glasses.

[0057] Figure 2 This is a flowchart of a decision-level fusion method for a multi-location system in augmented reality according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0058] S201, offline evaluation of uncertainty of different types of positioning systems;

[0059] The aforementioned different types of positioning systems can be Bluetooth positioning systems, visual positioning systems, or GPS positioning systems; how to evaluate uncertainty is not related to the core inventive point of this application, and can be achieved through conventional means in the field, therefore, it will not be elaborated on in this embodiment.

[0060] Specifically, the uncertainties (positioning accuracy) of various types of positioning systems in this field are as follows:

[0061] a. The uncertainty of a regular GPS at its highest accuracy outdoors is approximately 4m, while dual-antenna GPS offers even higher accuracy;

[0062] b. The normal accuracy of ordinary Bluetooth positioning is about 3m, and the normal accuracy of AOA Bluetooth positioning is about 0.5m;

[0063] c. The normal accuracy of visual positioning is approximately 0.1m;

[0064] d. Tracking systems: iOS's ARKit has an accuracy of about 2%, Android's ARCore has an accuracy of about 3%, and other tracking systems will not be discussed further.

[0065] S202, align the coordinate systems of multiple positioning systems to the AR world coordinate system of the target scene;

[0066] It should be noted that coordinate system alignment is a prerequisite for implementing the technical solution of this application. An object can be represented in two coordinate systems (this object has two different coordinate values), and the process of finding the transformation relationship between the same object in these two coordinate systems is the process of coordinate system alignment.

[0067] Specifically, in this embodiment, the alignment process uses the AR world coordinate system as a reference to determine the transformation relationship between the coordinate systems of other positioning systems and this reference coordinate system. Since coordinate system alignment methods are also conventional techniques in this field, the specific implementation process will not be described in detail in this embodiment.

[0068] S203, Select the positioning system, determine whether the positioning result needs to be updated based on the uncertainty, and if so, perform AR interaction in the target scene based on the fused positioning result.

[0069] If this is the first location attempt, the current location result will be used directly. Specifically, there is no particular limitation on which type of location result to use; it can be determined based on factors such as the user's location in the target scene or the user's usage status. For example, if the user is in an indoor scene and a visual positioning system is deployed in the scene, the visual positioning result can be used for the first location attempt. It should be noted that all types of location results have been aligned to the AR world coordinate system.

[0070] If this is not the first time the positioning has been performed, a positioning selection is required. Specifically, in this embodiment, regardless of which system the positioning result originates from, the uncertainty of the positioning result is used as the unified selection criterion. The uncertainty of the current positioning result is compared with the uncertainty of the fused positioning result to determine whether a pose update is needed.

[0071] Specifically, if the uncertainty of the current location result and the newly received location result is less than the uncertainty of the current location result (i.e., the accuracy of the fused location result is better than the current one), then it is determined that an update is needed and the fused location result is used for AR interaction; otherwise, there is no need to update and the current location result is still used for AR interaction experience.

[0072] Specifically, Figure 3 This is a schematic diagram of a continuous positioning update determination and fusion logic according to an embodiment of this application, as shown below. Figure 3 As shown:

[0073] At time t0 (the moment the AR experience begins), the user obtains the location result through visual positioning and then enters the AR world;

[0074] During the AR experience at times t1-t4, the pose obtained by the AR device tracking system is used. It can be seen that the uncertainty of the current positioning result also increases as the pose error of the tracking system accumulates over time.

[0075] At time t5, the user has just left the indoor environment and obtained a GPS location result. The AR device runs a positioning selection algorithm, determines that the uncertainty provided by the tracking system is still less than the uncertainty after fusing with the GPS positioning result, and outputs a command that no positioning update is needed. The AR experience on the device is still based on the positioning result output by the tracking system.

[0076] At time t7, the AR device acquires a positioning result sent by the visual positioning system (at this time, the error of the tracking system further increases), continues to run the positioning selection algorithm, and determines that the current uncertainty is greater than the uncertainty of the positioning result after fusion with the visual positioning result. Therefore, it outputs an instruction to immediately update the positioning and uses the positioning pose after fusion with the visual positioning result for the AR experience.

[0077] Users continuously perform AR experiences by superimposing the above positioning poses with the tracking poses. At some point in the future, if the uncertainty of the current positioning result is greater than the uncertainty after fusing with the GPS positioning result, then the current positioning result and the GPS positioning result will be fused together and used for the AR experience on the device.

[0078] It should be noted that, Figure 3 The continuous determination and fusion logic shown is merely a specific example of the solution in this application; in actual applications, the types of positioning results, the timing of acquisition, and the determination and fusion logic may be more diverse and not limited to the above example. It should be understood that any determination based on uncertainty as a unified standard for positioning updates, thereby achieving a decision-level fusion solution, should fall within the reasonable protection scope of the technical solution in this application.

[0079] Furthermore, after calculating the fused pose, optimization algorithms can be used to construct a pose-graph optimization model of the pose itself. However, it should be noted that the core point of this patent is to discuss how to decide which positioning result to use when one or more positioning results exist simultaneously, and how to optimize the pose is unrelated to the core inventive point of this application.

[0080] Through the steps S201 to S204 above, compared with the related technologies that use filtering or optimization algorithms for positioning learning, judgment, and fusion, the technical solution of this application treats different positioning systems as black boxes, disregarding how the positioning data source is obtained and the scene conditions. Uncertainty is used as a unified selection criterion. When multiple positioning system results are available simultaneously, it is only necessary to determine whether the uncertainty of the current positioning result is less than the uncertainty of the fusion result to simply and efficiently determine which positioning system's result to use. Compared with the traditional approach that requires filtering and optimization algorithms, combined with specific scenes and multi-dimensional positioning data, and a large number of logical operations to make a decision, the solution of this application is simpler and more efficient in implementation while ensuring positioning accuracy, and consumes less computing power on the device side.

[0081] In some embodiments, determining whether to perform a location update based on the uncertainty of the current location result and the uncertainty of the fused location result includes:

[0082] When the uncertainty of the current positioning result is less than the uncertainty of the fused positioning result, it is determined that the positioning update is required;

[0083] If the uncertainty of the current positioning result is greater than or equal to the uncertainty of the fused positioning result, it is determined that no positioning update is required.

[0084] Specifically, in terms of technical implementation, the need to update the location is determined using the following formula 1:

[0085] Formula 1: uncertainty_current_to_arworld<uncertainty_t1_to_arworld

[0086] Here, uncertainty_current_to_arworld is the uncertainty of the current localization result aligned to AR world coordinates, and uncertainty_t1_to_arworld is the uncertainty of the fused localization result aligned to AR world coordinates at time t1.

[0087] It is understandable that if the above relationship holds true, then no update is needed; otherwise, the location needs to be updated.

[0088] Furthermore, after determining that fusion is necessary, the positioning results (poses) are fused using the following formula: T_current_to_arworld=T_t0_to_arworld*T_current_to_t0. Further, the fused poses are used for AR interaction on the device, such as triggering AR effects based on the poses, multi-device collaboration, etc.

[0089] Uncertainty is linearly fused using the following formula: uncertainty_current_to_arworld=uncertainty_t0_to_arworld+uncertainty_current_to_t0;

[0090] In some embodiments, for positioning systems such as vision, Bluetooth, and GPS, the uncertainty is already determined when the positioning action occurs; while for tracking systems, the uncertainty accumulates with the accumulation of time / track distance.

[0091] Based on the above characteristics, the current uncertainty generated by the cumulative offset of the tracking system is obtained through linear extrapolation, including:

[0092] 1. Obtain the initial uncertainty of the positioning results sent by the positioning system in the initial state;

[0093] 2. The cumulative trajectory length of the tracking system is obtained by summing the trajectory length values ​​of multiple preset time periods. The formula is as follows:

[0094] trajectory_length_t0_to_tN=length_t0_to_t1+length_t1_to_t2+...+length_tN-1_to_tN;

[0095] 3. Based on the cumulative trajectory length and the cumulative error constant, the uncertainty change value of the tracking system is obtained. The cumulative error constant represents the cumulative error generated by the tracking system for each preset distance traveled, and it is obtained through offline evaluation. The formula is as follows:

[0096] uncertainty_tN_to_t0 = trajectory_length_t0_to_tN * x%, where x% is the uncertainty parameter, which is obtained through offline measurement. The uncertainty parameter of each tracking system is unique and fixed.

[0097] 4. Based on the initial uncertainty and the change in uncertainty of the tracking system, obtain the current uncertainty caused by the cumulative offset of the tracking system, expressed by the following formula:

[0098] uncertainty_tN_to_arworld=uncertainty_t0_to_arworlrld+uncertainty_tN_to_t0.

[0099] This embodiment also provides a decision-level fusion system for multi-positioning systems in augmented reality. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] Figure 4 This is a structural block diagram of a decision-level fusion system for a multi-positioning system in augmented reality according to an embodiment of this application, such as... Figure 4 As shown, the system includes AR devices and various types of positioning systems, among which,

[0101] AR devices are used to perform AR interactions in a target scene based on the current location results, and

[0102] Upon receiving the second positioning result, the backend is instructed to obtain a fused positioning result based on the second positioning result and the current positioning result, and...

[0103] Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, determine whether to perform a positioning update. If yes,

[0104] The current location result is updated to a fused location result for AR interaction, where the second location result comes from any type of location system.

[0105] In one embodiment, Figure 5 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 5 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 5 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operating system to run, the computer programs are executed by the processor to implement a decision-level fusion method for multi-location systems in augmented reality, and the database stores data.

[0106] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0107] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0108] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A decision-level fusion method for multiple positioning systems in augmented reality, characterized in that, When applied to large-space augmented reality scenarios, the method includes: The AR device performs AR interaction in the target scene based on the current positioning result, wherein the current positioning result is obtained by the tracking system on the AR device; the current uncertainty generated by the cumulative offset of the tracking system is obtained through linear extrapolation, including: Obtain the initial uncertainty of the positioning results sent by the positioning system in the initial state; The cumulative trajectory length of the tracking system is obtained by accumulating the trajectory length values ​​of the tracking system over multiple preset time periods; The uncertainty change value of the tracking system is obtained based on the cumulative trajectory length and the cumulative error constant, wherein the cumulative error constant represents the cumulative error generated by the tracking system for each preset distance moved, which is obtained through offline evaluation; Based on the initial uncertainty and the change in uncertainty of the tracking system, the current uncertainty generated by the cumulative offset of the tracking system is obtained; Upon receiving the second positioning result, the backend of the AR device obtains a fused positioning result based on the second positioning result and the current positioning result, and... Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, it is determined whether to perform a positioning update. Specifically, if the uncertainty of the current positioning result and the second positioning result after fusion is less than the uncertainty of the current positioning result, it is determined that the positioning update is required; if so, the current positioning result is updated to the fused positioning result for AR interaction, wherein the second positioning result comes from any type of positioning system.

2. The method according to claim 1, characterized in that, Before the AR device performs AR interaction in the target scene based on the current positioning result, the method further includes: The uncertainty of different types of positioning systems is evaluated offline, and the coordinate systems of the different types of positioning systems are aligned to the AR world coordinate system of the target scene.

3. The method according to claim 2, characterized in that, The positioning system includes: Bluetooth positioning system, visual positioning system and global positioning system.

4. The method according to claim 3, characterized in that, The current positioning result includes: the fusion result of one or more of the following: the pose acquired by the AR device tracking system, the pose sent by the Bluetooth positioning system, the pose sent by the visual positioning system, and the pose sent by the global positioning system.

5. The method according to claim 1, characterized in that, Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, determine whether to perform a positioning update, including: When the uncertainty of the current positioning result is greater than or equal to the uncertainty of the fused positioning result, it is determined that the positioning update is required; If the uncertainty of the current positioning result is less than the uncertainty of the fused positioning result, it is determined that no positioning update is required.

6. The method according to claim 5, characterized in that, The following equation is used to determine whether the positioning update is needed: Here, uncertainty_current_to_arworld is the uncertainty of the current localization result aligned to AR world coordinates, and uncertainty_t1_to_arworld is the uncertainty of the fused localization result aligned to AR world coordinates at time t1.

7. A decision-level fusion system for multiple positioning systems in augmented reality, characterized in that, For applications in large-space augmented reality scenarios, the system includes: AR devices and various types of positioning systems, wherein: The AR device is used to perform AR interaction in a target scene based on the current positioning result, wherein the current positioning result is obtained by the AR device's on-device tracking system; The current uncertainty generated by the cumulative offset of the tracking system is obtained through linear extrapolation, including: Obtain the initial uncertainty of the positioning results sent by the positioning system in the initial state; The cumulative trajectory length of the tracking system is obtained by accumulating the trajectory length values ​​of the tracking system over multiple preset time periods; The uncertainty change value of the tracking system is obtained based on the cumulative trajectory length and the cumulative error constant, wherein the cumulative error constant represents the cumulative error generated by the tracking system for each preset distance moved, which is obtained through offline evaluation; Based on the initial uncertainty and the change in uncertainty of the tracking system, the current uncertainty generated by the cumulative offset of the tracking system is obtained; And, upon receiving the second positioning result, instructing the backend to obtain a fused positioning result based on the second positioning result and the current positioning result, and, Based on the uncertainty of the current positioning result and the uncertainty of the fused positioning result, it is determined whether to perform a positioning update. Specifically, if the uncertainty of the current positioning result and the second positioning result after fusion is less than the uncertainty of the current positioning result, it is determined that the positioning update is required; if so, the current positioning result is updated to the fused positioning result for AR interaction, wherein the second positioning result comes from any type of positioning system.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images