AR glasses control system and method

Through multimodal data fusion technology, the AR glasses system processes user input in real time, solving the problem of misidentification of AR glasses in complex environments, improving interaction accuracy and robustness, and optimizing user experience.

CN120428854AInactive Publication Date: 2025-08-05SHIJIAZHUANG TIEDAO UNIV
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202510479003.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-04
Publication Date
2025-08-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When existing AR glasses are used in complex environments, voice input is affected by noise interference, gesture obstruction and eye movement data, resulting in misidentification or incorrect triggering of operations, which makes the user experience poor.

Method used

Multimodal data fusion technology is used to capture facial images through cameras for authentication, combine voiceprint data with voice assistants, build a three-dimensional environmental model with depth sensors, track gestures and identify predefined gestures, combine eye tracking sensors to obtain gaze direction, analyze user intentions through natural language processing, and perform priority under multimodal data fusion.

Benefits of technology

Improve the interactive accuracy and robustness of AR glasses, ensure smooth operation and immediacy, optimize user experience, and reduce input delays and misoperation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120428854A_ABST
    Figure CN120428854A_ABST
Patent Text Reader

Abstract

The invention discloses an AR glasses control system and method, and belongs to the field of artificial intelligence and AR glasses application, and the method comprises the steps: capturing a user face image, combining a voice assistant, collecting user voiceprint data, and carrying out the matching through a recurrent neural network; constructing a three-dimensional environment model, positioning and marking obstacles and interactive objects, and identifying objects in the environment; the method comprises the following steps of: tracking a gesture, acquiring a key point coordinate by adopting an OpenPose algorithm, identifying a predefined gesture by utilizing a support vector machine, and mapping the gesture into a specific operation instruction; analyzing the user intention based on natural language processing, and calling a corresponding function module according to an analysis result; acquiring a gazing direction of a user by using an eye movement tracking sensor, calculating a screen focus position through an algorithm, and dynamically highlighting options or displaying information according to a gazing point position; through multi-modal data fusion, gestures, voices and fixation point input are analyzed in a unified manner. By accurately understanding the intention of the user, the interaction precision of the AR glasses is improved, and the user experience is optimized.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Operation intention recognition method, system and equipment based on multi-modal fusion and medium

    CN120873982A

  • Robot multi-mode fusion control system and method based on head-mounted AR equipment

    CN121232978A

  • Sight line point coordinate calculation method and system based on multi-modal fusion

    CN121541788A

  • Barrier-free interaction method and equipment for government affair service

    CN121680622A

  • Intelligent watch interaction method and system based on gesture action

    CN121979397A