Visual finger difference operation method based on AR glasses

Abstract

The invention relates to the technical field of visual finger difference operation, in particular to a visual finger difference operation method based on AR (Augmented Reality) glasses, which greatly improves the ability of operators to understand and execute an operation process by accurately fusing virtual guidance information with a real environment, and improves the operation efficiency by utilizing a deep learning driven finger difference action recognition technology. The system can monitor and confirm pointing actions of operators in real time, manual misoperation is reduced, operation accuracy and safety are effectively guaranteed, meanwhile, timeliness and effectiveness of information transmission are improved through dynamic task prompt and diversified interaction modes (such as voice and gesture control), the operation process is smoother and easy to control, and operation efficiency is improved. The method is suitable for various complex industrial and service scenes, the intelligent management of the operation process is enhanced, the overall operation efficiency and quality level are remarkably improved, and a solid technical support is provided for enterprises to realize digital transformation and intelligent manufacturing.

Description

Technical Field

[0001] The present invention relates to the technical field of visual pointing and difference operations, and in particular to a visual pointing and difference operation method based on AR glasses. Background Art

[0002] In modern industrial manufacturing, equipment maintenance, logistics, and warehousing, operators often need to complete a series of precise or highly sequential tasks according to specific standard operating procedures (SOPs). To ensure accuracy and consistency, traditional methods generally rely on paper work instructions or electronic devices (such as tablets) to provide graphic instructions, and operators must review the information and perform the corresponding operations. However, this model has many limitations in practice, such as a disconnect between the operating process and the instructions, operators' frequent glances down to check for information, which affects efficiency, and the lack of timely information transfer during multi-person collaboration. These problems can easily lead to errors, missed steps, and even safety hazards. Especially in equipment maintenance or complex assembly scenarios, where the variety of work objects and complex steps are numerous, real-time guidance and accurate confirmation are difficult to achieve with paper documents or graphic information alone. Therefore, how to use technology to achieve intelligent prompts, process tracking, and action confirmation during the operation process has become a key research focus in the fields of intelligent manufacturing and smart operations and maintenance.

[0003] With the rapid development of augmented reality (AR) technology, especially the increasing popularity of lightweight AR glasses, the research and application of integrating AR with work guidance is becoming a growing trend. AR glasses are capable of overlaying virtual information in the user's field of view, enabling real-time positioning and guidance of work targets, paths, tools, and more, significantly enhancing the operator's understanding of the situation and response efficiency. However, simply displaying static AR prompts is insufficient to fully address the issue of operation confirmation, making the introduction of the concept of "pointing confirmation" particularly important. Originally originating in the railway and aviation industries, pointing confirmation combines visual gaze with physical pointing, significantly reducing the risk of human error. Therefore, if visual pointing recognition technology can be integrated into AR glasses, the system can proactively detect whether the operator has accurately identified and executed the task objective, forming a closed-loop feedback mechanism. This not only enables intelligent and standardized work process control, but also records operator behavior in real time and provides behavioral correction support, possessing significant industrial application prospects and technological value.

[0004] In view of the above situation, in order to overcome the above technical problems, the present invention designs a visual pointing operation method based on AR glasses to solve the above technical problems. Summary of the Invention

[0005] The technical purpose of this invention is to design a visual pointing operation method based on AR glasses, which greatly improves the operator's understanding and execution ability of the operation process by accurately integrating virtual guidance information with the real environment.

[0006] In order to achieve the above technical objectives, the present invention provides the following technical solutions: The visual pointing method based on AR glasses includes the following steps: Step 1: Operation initialization step: The current operation scene is identified through the built-in sensors of the AR glasses or the external access system, the corresponding operation process data is automatically loaded, and the virtual operation guidance interface is initialized on the display interface of the AR glasses; Step 2: Task prompt step: Based on the current work process status, AR glasses generate and display visual prompt information for the next work task in real time, including arrows, icons, text or videos, to guide the operator's operation direction and target object; Step 3: Finger-to-finger motion recognition: The system uses the AR glasses' camera to capture images of the operator's hand movements, uses image processing algorithms to identify their finger-to-finger motions, and determines whether the finger-to-finger confirmation of the prompted object is complete, ensuring that the operator focuses on the correct task. Step 4: Status feedback: After identifying the finger-pointing action and verifying that it meets the operating requirements, the system records the completion status of the step and provides feedback through voice, vibration, or visual means. If the finger-pointing action does not meet the requirements, the operator is prompted to correct the operation again. Step 5: Operation monitoring and data storage: The motion recognition data, finger difference trajectory, time nodes and other information during the entire operation process are uploaded to the back-end database in real time for operation process monitoring, quality traceability and personnel behavior analysis to ensure the standardization and traceability of the operation process.

[0007] Preferably, in the operation initialization step, the AR glasses can quickly identify the operation scene through the NFC module, QR code recognition or Wi-Fi positioning, and automatically load the corresponding operation process and visual auxiliary resources from the local or cloud server according to the recognition results; at the same time, the system can automatically filter information not related to the operation according to the user's identity authority, ensuring the simplicity of the operation interface and the pertinence of the information, and effectively improving the user's operating efficiency and operation concentration.

[0008] Preferably, in the task prompt step, the virtual guidance information dynamically adjusts its display form and interaction method according to the complexity of the work content, provides 3D animation demonstration for key steps, and only provides text prompts for routine steps, and synchronously outputs the prompt content in combination with the voice broadcast function; at the same time, AR glasses can determine whether the user has read the prompt information through eye tracking, and automatically delay or repeat the prompt if not, thereby improving work safety and accuracy.

[0009] Preferably, the finger difference action recognition step utilizes a human key point detection algorithm constructed by a deep learning model to perform multi-parameter judgment on the finger pointing angle, speed, and duration to distinguish normal finger differences from erroneous operation behaviors; and supports user personalized training models to adapt to the operating habits of different people, further improving the recognition accuracy and system versatility.

[0010] Preferably, in the status feedback step, the feedback mechanism can be set by the user to use a combination of feedback methods, such as using only voice prompts or supplemented by vibration, image color change, etc.; in a multi-person collaboration scenario, the system can also synchronously broadcast the current step status to the AR glasses of other collaborators to achieve synchronous control of multi-person operations and improve collaboration efficiency.

[0011] Preferably, in the operation monitoring and data storage steps, the system not only records the completion time and finger confirmation status of each operation step, but also includes multi-dimensional behavioral data such as the operator's line of sight, ambient brightness, and operation posture; the data is encrypted and uploaded to the edge computing server in real time for preprocessing, and periodically synchronized to the cloud for long-term archiving and analysis, which is used to generate operation quality score reports and behavior improvement suggestions to support enterprises in operation process optimization and personnel performance evaluation.

[0012] Preferably, the display module equipped with the AR glasses supports the simultaneous display of real-life images and virtual guidance information, and has spatial positioning capabilities, which can anchor the guidance content on the surface of real objects to achieve virtual-real fusion; at the same time, users can use gestures or voice commands to zoom, rotate, hide and other operations on the displayed content, meeting the needs of visual information interaction in different work scenarios and improving the controllability and flexibility of the operation.

[0013] Preferably, the system has an automatic fault tolerance and prompt correction mechanism. When the operator fails to complete the finger difference confirmation or performs an erroneous operation within the predetermined time, the system will automatically pause the process and issue an error prompt, and reactivate the task prompt of the step; if three errors occur consecutively, the system will suggest that the user call a remote expert to intervene, and assist in solving the operation problem by remotely sharing the perspective through AR glasses, so as to ensure the continuity and safety of the operation.

[0014] Preferably, this method is applicable to a variety of industrial and service scenarios, including but not limited to aviation maintenance, power inspection, warehouse sorting, assembly production, medical surgery assistance and other fields; its core finger difference recognition and task prompt mechanism can be modularly configured according to industry needs, and supports API interface and third-party system (such as ERP, MES, WMS, etc.) docking, further expanding the scope of application and system compatibility.

[0015] Preferably, the system supports operator identity recognition and operation authority control, confirms the user's identity through face recognition, voice recognition or iris recognition technology, and automatically loads their historical operation records and preference settings; during the operation process, the system automatically limits the visual prompts of high-risk operation steps according to the personnel qualification level, and provides novice personnel with more detailed operation guidance and feedback mechanisms, thereby ensuring operation safety while improving personnel training efficiency.

[0016] The beneficial effects of the present invention are as follows: (1) The visual pointing operation method based on AR glasses of the present invention greatly improves the operator's understanding and execution ability of the operation process by accurately integrating virtual guidance information with the real environment. Utilizing deep learning-driven pointing action recognition technology, the system can monitor and confirm the operator's pointing action in real time, reduce the occurrence of human error, and effectively ensure the accuracy and safety of the operation. At the same time, dynamic task prompts and diversified interaction methods (such as voice, gesture control, etc.) improve the timeliness and effectiveness of information transmission, making the operation process smoother and easier to control. This method is applicable to a variety of complex industrial and service scenarios, enhances the intelligent management of the operation process, significantly improves the overall operation efficiency and quality level, and provides solid technical support for enterprises to achieve digital transformation and intelligent manufacturing.

[0017] (2) The multi-dimensional data collection and real-time monitoring functions integrated in this invention can systematically record the operator's behavioral trajectory, environmental status and operational feedback, providing a reliable data basis for subsequent quality assessment and process optimization. The personalized identity recognition and authority control mechanism ensures operational safety, supports the precise management and training needs of personnel with different qualifications, and effectively reduces the risk of novice operation errors. The remote expert assistance function further enhances the system's emergency response capabilities and ensures continuity and safety in complex operating environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] The above and other aspects of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0020] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] like Figure 1 As shown, the visual pointing method based on AR glasses includes the following steps: Step 1: Operation initialization step: The current operation scene is identified through the built-in sensors of the AR glasses or the external access system, the corresponding operation process data is automatically loaded, and the virtual operation guidance interface is initialized on the display interface of the AR glasses; Step 2: Task prompt step: Based on the current work process status, AR glasses generate and display visual prompt information for the next work task in real time, including arrows, icons, text or videos, to guide the operator's operation direction and target object; Step 3: Finger-to-finger motion recognition: The system uses the AR glasses' camera to capture images of the operator's hand movements, uses image processing algorithms to identify their finger-to-finger motions, and determines whether the finger-to-finger confirmation of the prompted object is complete, ensuring that the operator focuses on the correct task. Step 4: Status feedback: After identifying the finger-pointing action and verifying that it meets the operating requirements, the system records the completion status of the step and provides feedback through voice, vibration, or visual means. If the finger-pointing action does not meet the requirements, the operator is prompted to correct the operation again. Step 5: Operation monitoring and data storage: The motion recognition data, finger difference trajectory, time nodes and other information during the entire operation process are uploaded to the back-end database in real time for operation process monitoring, quality traceability and personnel behavior analysis to ensure the standardization and traceability of the operation process.

[0022] During the initialization step, the AR glasses first quickly locate and assess the current work environment using a variety of scene recognition technologies, including but not limited to using a built-in NFC module to identify specific workstation tags, using the camera to identify QR codes or barcodes to obtain task code information, and accurately locating the work area by comparing Wi-Fi signal strength with a location database. After identifying the work scene, the system automatically retrieves the corresponding work process data and visual aid resource files, such as work drawings, operating instructions, and tool guide animations, from a local cache or a remote cloud server based on the recognition results. Furthermore, the system intelligently filters tasks based on the user's identity and permissions, displaying only those tasks that align with their job responsibilities and operational qualifications, and automatically blocking irrelevant or high-authority operations, thereby effectively avoiding information redundancy or misleading information. This mechanism not only improves the simplicity of the interface and the accuracy of prompts, but also significantly enhances the operator's efficiency and focus, laying the foundation for subsequent task execution and action recognition. It is particularly suitable for critical work scenarios such as industrial and medical operations, where efficient information exchange is crucial.

[0023] During the task prompting step, the system dynamically adjusts the presentation and interaction of virtual guidance information based on the complexity and execution requirements of the specific task. For high-risk or critical steps, the guidance displayed by the AR glasses uses high-precision 3D animations to intuitively illustrate the spatial relationship of the operating objects and the key movements, allowing the operator to clearly grasp the key steps in a virtual and real-world overlay. For routine or low-risk, simple operations, simple text prompts are used to reduce visual clutter. The system also integrates a real-time voice announcement function that simultaneously outputs the name of the current step, precautions, and the next action point, ensuring that the user can fully understand the instructions through hearing even when both hands are focused on the operation. To further improve the effectiveness of the prompt information, the AR glasses' built-in eye tracking module monitors the operator's gaze point and duration. If the system detects that the user fails to read or pay attention to the prompt content in a timely manner, it automatically triggers a delay or repeat prompt mechanism, extending the display time of the virtual guidance or replaying the key animation until the user is confirmed to have accurately received the instructions. This mechanism can not only significantly reduce operational errors caused by ignoring or incomplete reading of information, but also ensure operational safety and process accuracy in complex environments.

[0024] During the finger-to-finger motion recognition step, the system builds a human keypoint detection algorithm based on deep learning technology. The AR glasses' high-definition camera collects and analyzes the operator's hand and upper limb movements in real time. This algorithm accurately locates the positions of multiple key points, such as the fingers, palm, wrist, and elbow, and combines them with a time series model to determine multi-dimensional parameters such as their motion trajectory, pointing angle, motion duration, and movement speed. Based on this, the system intelligently recognizes the captured finger-to-finger motion using a rules engine and a sample training model. This effectively distinguishes standard finger-to-finger confirmation motions from unintentional non-operational behaviors such as false touches and feints, significantly improving the accuracy of command confirmation. Furthermore, given the significant individual differences in body language, movement amplitude, and operating rhythm among different operators, the system supports personalized training: users can enter their own standard finger-to-finger motion samples through a guided program during initial use or regular calibration. The system then constructs a user-specific recognition model based on these samples. This feature not only improves the adaptability and stability of the recognition algorithm in real-world use, but also enhances the system's compatibility with complex operating environments and diverse user groups, ensuring the continuity of the overall operational process and the practicality of the intelligent assistance system.

[0025] During the status feedback step, the system provides a variety of feedback mechanisms. Users can select or combine feedback methods based on their specific work environment, personal habits, and task nature, enabling a more flexible operation confirmation experience. For example, in quiet environments or when operations require more precision, users can select voice prompts as the primary feedback method, with the AR glasses announcing the current step status or whether the operation is correct. In contrast, in high-noise environments or when work requires high concentration, the vibration feedback module can be activated, with brief vibration signals from the frame or headset indicating task status changes. Furthermore, graphical elements in the AR interface can provide visual feedback. For example, after the current step is completed, the relevant guidance icon automatically changes color or fades out, providing intuitive visual confirmation. Especially for complex workflows involving multiple people, the system supports simultaneous status broadcasting, which allows the status of a step completed by one person to be shared in real time with the AR glasses of other team members, ensuring that all participants have a consistent understanding of the current process node. This mechanism not only effectively avoids information miscommunication and collaboration delays, but also improves team collaboration efficiency and overall task progress control. It is particularly suitable for demanding scenarios requiring high levels of collaboration, such as engineering maintenance, medical surgical assistance, and emergency rescue.

[0026] During the operation monitoring and data storage steps, the system collects and records key behavioral data throughout the entire operation. This includes not only the completion time of each operation step, whether the finger-checking action was performed and its accuracy, but also the operator's gaze trajectory, head movement direction, hand movement path, working posture (such as standing, bending, crouching, etc.), and external factors such as the ambient brightness and noise level at the work site. This data is acquired in real time via the multi-sensor module built into the AR glasses and undergoes preliminary encryption and data cleansing on a local or nearby edge computing server to ensure data privacy and processing efficiency. The processed data is regularly uploaded to a cloud platform for centralized storage and in-depth analysis. Based on historical operation data, the system automatically generates operation quality score reports, operation behavior analysis reports, and efficiency deviation maps to identify potential operation bottlenecks and employee behavior issues and provide improvement suggestions. Furthermore, this analysis can be integrated with the company's performance evaluation system to provide managers with quantitative reference indicators to assist in process optimization, personnel retraining planning, and job adjustment recommendations, thereby improving overall operation quality, efficiency, and intelligent management, and building a data-driven lean operation system.

[0027] The high-performance display module equipped with these AR glasses not only displays high-definition real-world scenes but also overlays virtual guidance information in real time, achieving visual fusion within the augmented reality environment. This display module utilizes optical waveguide or freeform optical technology to ensure that virtual images and real-world scenes naturally overlap within the user's field of view, enhancing immersion and spatial perception. Leveraging built-in spatial positioning and object recognition modules, the system accurately identifies key objects in the work environment and precisely anchors corresponding guidance content, such as operating procedure prompts, precautionary annotations, and tool path navigation, to the surface or its surroundings, ensuring that guidance information remains within the user's field of view and is not shifted by head or body movement. Furthermore, users can interact with the displayed content through natural gestures or voice commands, such as using a "zoom in" gesture to view local structural details or a voice command like "next" to switch to subsequent steps. Temporarily unnecessary information can also be hidden, maintaining a simple interface and focused information. This interactive mechanism significantly improves the efficiency and flexibility of information acquisition during work processes and is particularly suitable for complex work scenarios such as industrial maintenance, assembly and debugging, and medical assistance, which require highly integrated visual assistance, dynamic command response, and multi-step operation control.

[0028] The system features a comprehensive automatic fault-tolerance and prompt-correction mechanism designed to maximize the accuracy and safety of operational processes. During an operation, if the system detects that an operator fails to complete a critical finger-to-point confirmation action within a preset time window or performs an incorrect operation that does not comply with specifications, it immediately responds by automatically pausing the current process to prevent further error propagation. Simultaneously, the system provides clear, real-time error notifications to the user via AR glasses, including voice alarms, visual flashing prompts, or vibration feedback, alerting the operator to correct operational deviations. While paused, the system automatically reactivates the task guidance for the current step, helping the user understand and complete the operation, ensuring each step is accurate. If the user fails to complete a step correctly three times in a row, the system intelligently identifies a significant operational risk or technical obstacle and proactively recommends that the user call an expert for assistance via remote assistance. By remotely sharing the real-time perspective captured by the AR glasses, the expert can gain a direct understanding of the on-site situation and guide the operator in problem diagnosis and corrective actions. This remote collaboration mechanism not only greatly improves the efficiency of solving problems under complex or special working conditions, but also ensures the continuity and safety of operations, avoids major accidents and economic losses caused by human errors or knowledge blind spots, and further promotes the widespread application of intelligent operation assistance systems in high-risk industries.

[0029] This method has broad applicability, covering a wide range of critical operational scenarios across various industries and services, meeting the diverse needs of different sectors. These include, but are not limited to, aircraft inspection and maintenance, power equipment inspection and troubleshooting, cargo sorting and inventory management in warehousing and logistics, assembly line operations in manufacturing, and complex surgical assistance procedures in the medical field, among other high-precision, high-risk scenarios. The method's core technologies—finger-difference motion recognition and a dynamic task prompting mechanism—feature a highly modular design, enabling flexible configuration and customization based on the specific operational processes and standards of each industry, adapting to operational requirements of varying complexity and expertise. Furthermore, the system supports open APIs, facilitating seamless integration with existing enterprise management and execution systems, such as enterprise resource planning (ERP), manufacturing execution systems (MES), and warehouse management systems (WMS). This compatibility not only enables the effective integration of operational data into enterprise information systems, promoting data sharing and collaborative management, but also further expands the method's application boundaries and enhances its versatility and scalability. Through deep integration across multiple industries and systems, this method not only optimizes operational processes but also provides solid technical support for improving operational efficiency, safety management, and digital transformation for enterprises.

[0030] The system features advanced operator identification and access control. By integrating multimodal biometric technologies such as facial recognition, voice recognition, and iris recognition, it accurately and efficiently verifies the current user's identity. Once successfully identified, the system automatically accesses the user's historical work history, operating habits, and preferences to customize a personalized work interface and guidance. This not only improves the user experience but also provides a foundation for subsequent data analysis and performance evaluation. During actual operations, the system intelligently adjusts visible work steps and operational permissions based on the operator's qualification level and authorization scope. For steps involving high-risk or safety-critical aspects, the system automatically restricts access to those steps by unqualified personnel, preventing accidents caused by incorrect operations. Furthermore, for novice or inexperienced operators, the system provides more detailed operational guidance and multi-level real-time feedback to help them quickly master skills and reduce operational errors. This precise access control and personalized coaching mechanism not only effectively ensures operational safety and improves work quality, but also significantly enhances personnel training efficiency and promotes standardized and intelligent management of enterprise operational processes.

[0031] Various modifications to the present disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but should be given the widest scope consistent with the principles and novel features disclosed herein. Although one or more exemplary embodiments of the present disclosure have been described with reference to the accompanying drawings, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined in the appended claims.

Claims

1. A visual pointing method based on AR glasses, characterized in that: The following steps are involved: Step 1: Operation initialization step, using the built-in sensors of the AR glasses or an external access system to identify the current operation scene, automatically load the corresponding operation process data, and initialize the virtual operation guidance interface on the display interface of the AR glasses; Step 2: Task prompt step: Based on the current work process status, AR glasses generate and display visual prompt information for the next work task in real time, including arrows, icons, text or videos, to guide the operator's operation direction and target object; Step 3: Finger-to-finger motion recognition: The system uses the camera of the AR glasses to collect images of the operator's hand movements, uses image processing algorithms to identify their finger-to-finger motions, and determines whether the finger-to-finger confirmation of the prompted object is complete, ensuring that the operator focuses on the correct work target. Step 4: Status feedback step. After identifying the finger-pointing action and verifying that it meets the operating requirements, the system records the completion status of the step and provides feedback through voice, vibration, or visual means. If the finger-pointing action does not meet the requirements, the operator is prompted to correct the operation again. Step 5: Operation monitoring and data storage step. The motion recognition data, finger difference trajectory, time nodes and other information of the entire operation process are uploaded to the back-end database in real time for operation process monitoring, quality traceability and personnel behavior analysis to ensure the standardization and traceability of the operation process.

2. The visual pointing method based on AR glasses according to claim 1, characterized in that: In the operation initialization step, the AR glasses use the NFC module 、 QR code recognition or Wi-Fi positioning can be used to quickly identify the work scene. Based on the recognition results, the corresponding work process and visual assistance resources are automatically loaded from the local or cloud server. The system can automatically filter information not related to the work based on user identity permissions.

3. The visual pointing method based on AR glasses according to claim 1, characterized in that: In the task prompt step, the virtual guidance information dynamically adjusts its display form and interaction method according to the complexity of the work content, provides 3D animation demonstration for key steps, and only provides text prompts for routine steps, and synchronously outputs the prompt content in combination with the voice broadcast function; at the same time, AR glasses can determine whether the user has read the prompt information through eye tracking, and automatically delay or repeat the prompt if the user has not read it.

4. The visual pointing method based on AR glasses according to claim 1, characterized in that: The finger difference action recognition step uses a human key point detection algorithm built by a deep learning model to perform multi-parameter judgment on the finger pointing angle, speed, and duration to distinguish normal finger difference from erroneous operation behavior; Supports user-personalized training models to adapt to the operating habits of different people.

5. The visual pointing method based on AR glasses according to claim 1, characterized in that: In the status feedback step, the feedback mechanism can be set by the user in a combination of feedback methods, such as using only voice prompts or supplemented by vibration, image color change, etc.; in a multi-person collaboration scenario, the system can also synchronously broadcast the current step status to the AR glasses of other collaborators.

6. The visual pointing method based on AR glasses according to claim 1, characterized in that: In the operation monitoring and data storage steps, the system records the completion time and finger confirmation status of each operation step, as well as multi-dimensional behavioral data such as the operator's line of sight, ambient brightness, and operation posture; the data is encrypted and uploaded to the edge computing server in real time for preprocessing, and periodically synchronized to the cloud for long-term archiving and analysis, so as to generate operation quality score reports and behavior improvement suggestions.

7. The visual pointing method based on AR glasses according to claim 1, characterized in that: The display module equipped with the AR glasses supports the simultaneous display of real-life images and virtual guidance information, and has spatial positioning capabilities, which can anchor the guidance content on the surface of real objects to achieve virtual-real fusion; at the same time, users can use gestures or voice commands to zoom, rotate, hide and other operations on the displayed content to meet the needs of visual information interaction in different work scenarios.

8. The visual pointing method based on AR glasses according to claim 1, characterized in that: The system has an automatic fault tolerance and prompt correction mechanism. If the operator fails to complete the finger difference confirmation within the predetermined time or performs an incorrect operation, the system will automatically pause the process and issue an error prompt, while reactivating the task prompt for that step. If three consecutive errors occur, the system will suggest that the user call a remote expert to intervene and assist in solving the operational problem by remotely sharing the perspective through AR glasses.

9. The visual pointing method based on AR glasses according to claim 1, characterized in that: This method is applicable to a variety of industrial and service scenarios, including but not limited to aviation maintenance, power inspection, warehouse sorting, assembly production, medical surgery assistance and other fields.

10. The visual pointing method based on AR glasses according to claim 1, characterized in that: The system supports operator identity recognition and operation permission control, confirms the user's identity through facial recognition, voice recognition or iris recognition technology, and automatically loads their historical operation records and preference settings; during the operation process, the system automatically limits the visual prompts of high-risk operation steps based on the personnel's qualification level, and provides more detailed operation guidance and feedback mechanisms for novice personnel.

Images