Augmented reality remote guidance system

Abstract

The invention discloses an augmented reality remote guidance system, which comprises an AR display module, a remote cooperation engine and a dynamic environment modeling unit, and is characterized in that the AR display module, the remote cooperation engine and the dynamic environment modeling unit are used for real-time superposition of virtual operation guidance and remote expert annotation pictures; a multi-modal interactive guidance framework is constructed and is suitable for self-service terminal operation guidance and complex business auxiliary scenes; according to the invention, the dynamic environment modeling and multi-modal interaction engine is constructed, eye movement tracking, semantic segmentation and quantum security technologies are creatively fused, and the bottleneck of environmental adaptability and information security of a traditional AR guidance system is broken through.

Description

Technical Field

[0001] The present invention relates to the technical field of self-service terminals, and in particular to an augmented reality remote guidance system. Background Art

[0002] Traditional AR remote guidance systems are limited by static environment modeling and one-way command transmission. This makes it easy for virtual guidance to misalign with physical objects in dynamic operational scenarios, and they lack a closed-loop, real-time collaboration between experts and users. Existing technologies rely on pre-recorded videos or fixed process guidance, making them difficult to adapt to the real-time changes in complex business scenarios and lacking secure transmission mechanisms. Summary of the Invention

[0003] The purpose of the present invention is to provide an augmented reality remote guidance system to solve the above problems in the prior art, thereby solving all or one of the above problems in the prior art.

[0004] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: The present invention provides an augmented reality remote guidance system, comprising: AR display module, remote collaboration engine and dynamic environment modeling unit. The AR display module, the remote collaboration engine and the dynamic environment modeling unit are used to build a multimodal interactive guidance framework by superimposing virtual operation instructions and remote expert annotation images in real time, which is suitable for self-service terminal operation guidance and complex business assistance scenarios.

[0005] As an improved solution, the AR display module is specifically used to adopt light field rendering technology to achieve geometric space alignment of virtual guidance and physical environment, and support dynamic layered rendering of gaze points based on eye tracking.

[0006] As an improved solution, the remote collaboration engine is deployed with a real-time gesture recognition algorithm and a voice command parsing engine, which is specifically used to support end-to-end synchronous transmission of expert-side annotation trajectories and voice instructions.

[0007] As an improved solution, the dynamic environment modeling unit integrates a multimodal sensor fusion architecture, and is specifically used to construct a three-dimensional model of the dynamic operation space through LiDAR point cloud and plane semantic segmentation, and compensate for environmental lighting changes in real time.

[0008] As an improved solution, the augmented reality remote guidance system is also provided with a fault tolerance enhancement mechanism for automatically activating the AR marker projection submodule when the ambient occlusion rate exceeds a threshold, thereby generating an interactive virtual anchor point on the physical surface.

[0009] As an improved solution, the augmented reality remote guidance system also includes an adaptive interface generation engine, which is used to dynamically adjust the guidance information density according to the user's operating proficiency, and support a progressive interaction mode from full-process guidance to key step prompts.

[0010] As an improved solution, the augmented reality remote guidance system also deploys a multimodal interaction intention understanding module to integrate eye movement trajectory, gesture speed and voice semantic features, predict user operation intentions in real time and preload related guidance content.

[0011] As an improved solution, the augmented reality remote guidance system also has an end-to-end encrypted transmission channel, which is used to achieve secure distribution of AR annotation data and control instructions using dynamic key splitting technology, and supports quantum security algorithm upgrade interface.

[0012] As an improved solution, the augmented reality remote guidance system also integrates an energy consumption perception optimization engine, which is used to dynamically switch the AR rendering accuracy level according to the remaining power of the device and the network quality, thereby maintaining the continuous availability of the core guidance function.

[0013] As an improved solution, the augmented reality remote guidance system is also equipped with a context-aware auxiliary engine, which is used to generate personalized guidance strategies through historical operation records and business rule libraries, and support compliance verification in vertical scenarios such as financial bill verification.

[0014] The beneficial effect of the technical solution of the present invention is: by constructing dynamic environment modeling and multimodal interaction engines, the present invention innovatively integrates eye tracking, semantic segmentation and quantum security technologies, breaking through the environmental adaptability and information security bottlenecks of traditional AR guidance systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 2 is a schematic diagram of the architecture of the augmented reality remote guidance system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0018] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0019] The terms "first," "second," and the like in the specification and claims herein and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0020] Embodiment, this embodiment provides an augmented reality remote guidance system, such as Figure 1 Shown, including: AR display module, remote collaboration engine and dynamic environment modeling unit. The AR display module, the remote collaboration engine and the dynamic environment modeling unit are used to build a multimodal interactive guidance framework by superimposing virtual operation instructions and remote expert annotation images in real time, which is suitable for self-service terminal operation guidance and complex business assistance scenarios.

[0021] As an improved solution, the AR display module is specifically used to adopt light field rendering technology to achieve geometric space alignment of virtual guidance and physical environment, and support dynamic layered rendering of gaze points based on eye tracking.

[0022] As an improved solution, the remote collaboration engine is deployed with a real-time gesture recognition algorithm and a voice command parsing engine, which is specifically used to support end-to-end synchronous transmission of expert-side annotation trajectories and voice instructions.

[0023] As an improved solution, the dynamic environment modeling unit integrates a multimodal sensor fusion architecture, and is specifically used to construct a three-dimensional model of the dynamic operation space through LiDAR point cloud and plane semantic segmentation, and compensate for environmental lighting changes in real time.

[0024] As an improved solution, the augmented reality remote guidance system is also provided with a fault tolerance enhancement mechanism for automatically activating the AR marker projection submodule when the ambient occlusion rate exceeds a threshold, thereby generating an interactive virtual anchor point on the physical surface.

[0025] As an improved solution, the augmented reality remote guidance system also includes an adaptive interface generation engine, which is used to dynamically adjust the guidance information density according to the user's operating proficiency, and support a progressive interaction mode from full-process guidance to key step prompts.

[0026] As an improved solution, the augmented reality remote guidance system also deploys a multimodal interaction intention understanding module to integrate eye movement trajectory, gesture speed and voice semantic features, predict user operation intentions in real time and preload related guidance content.

[0027] As an improved solution, the augmented reality remote guidance system also has an end-to-end encrypted transmission channel, which is used to achieve secure distribution of AR annotation data and control instructions using dynamic key splitting technology, and supports quantum security algorithm upgrade interface.

[0028] As an improved solution, the augmented reality remote guidance system also integrates an energy consumption perception optimization engine, which is used to dynamically switch the AR rendering accuracy level according to the remaining power of the device and the network quality, thereby maintaining the continuous availability of the core guidance function.

[0029] As an improved solution, the augmented reality remote guidance system is also equipped with a context-aware auxiliary engine, which is used to generate personalized guidance strategies through historical operation records and business rule libraries, and support compliance verification in vertical scenarios such as financial bill verification.

[0030] It should be noted that the above examples are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0031] It should also be understood that in the embodiments herein, the term "and / or" merely describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" could represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0032] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0033] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices, or units, or can be an electrical, mechanical, or other form of connection.

[0034] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments herein.

[0035] In addition, the functional units in the various embodiments herein may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0036] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0037] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An augmented reality remote guidance system, characterized in that: include: AR display module, remote collaboration engine and dynamic environment modeling unit. The AR display module, the remote collaboration engine and the dynamic environment modeling unit are used to build a multimodal interactive guidance framework by superimposing virtual operation instructions and remote expert annotation images in real time, which is suitable for self-service terminal operation guidance and complex business assistance scenarios.

2. The augmented reality remote guidance system according to claim 1, characterized in that: The AR display module is specifically further used to adopt light field rendering technology to achieve geometric space alignment of virtual guidance and physical environment, and support dynamic layered rendering of gaze points based on eye tracking.

3. The augmented reality remote guidance system according to claim 1, characterized in that: The remote collaboration engine is deployed with a real-time gesture recognition algorithm and a voice command parsing engine, and is specifically used to support the end-to-end synchronous transmission of expert-side annotation trajectories and voice instructions.

4. The augmented reality remote guidance system according to claim 1, characterized in that: The dynamic environment modeling unit integrates a multimodal sensor fusion architecture, and is specifically used to construct a three-dimensional model of the dynamic operation space through LiDAR point cloud and plane semantic segmentation, and compensate for environmental lighting changes in real time.

5. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system is also provided with a fault tolerance enhancement mechanism for automatically activating the AR marker projection submodule when the ambient occlusion rate exceeds a threshold, thereby generating an interactive virtual anchor point on the physical surface.

6. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system also includes an adaptive interface generation engine for dynamically adjusting the guidance information density according to the user's operating proficiency, and supporting a progressive interaction mode from full-process guidance to key step prompts.

7. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system is also deployed with a multimodal interaction intention understanding module, which is used to integrate eye movement trajectory, gesture speed and voice semantic features, predict user operation intentions in real time and preload related guidance content.

8. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system also has an end-to-end encrypted transmission channel, which is used to achieve secure distribution of AR annotation data and control instructions using dynamic key splitting technology, and supports quantum security algorithm upgrade interface.

9. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system also integrates an energy consumption perception optimization engine, which is used to dynamically switch the AR rendering accuracy level according to the remaining power of the device and the network quality, thereby maintaining the continuous availability of the core guidance function.

10. The augmented reality remote guidance system according to claim 1, characterized in that: The augmented reality remote guidance system is also equipped with a context-aware auxiliary engine for generating personalized guidance strategies through historical operation records and a business rule library, and supporting compliance verification in vertical scenarios such as financial bill verification.

Images