A system for virtual reality glasses based on artificial intelligence
By integrating multimodal sensors and artificial intelligence algorithms, the virtual reality glasses system can judge the user's micro-context in real time and generate a visual consensus window, solving the problem of static permission allocation in multi-user interaction and realizing efficient group collaboration and natural decision-making.
Patent Information
- Application Number
- CN202611323404.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing virtual reality glasses systems cannot perceive users' attention levels and task contexts during multi-user interactions, resulting in static allocation of interaction permission levels that cannot be negotiated, easily leading to misoperations and low-privilege users passively accepting requests.
The system employs an AI-based virtual reality glasses system that integrates eye-tracking sensors, EEG sensors, voice sensors, and inertial measurement units. Through a context arbitrator and a distributed intent fusion engine, it judges the user's micro-context in real time, generates a visual consensus window, collects micro-behavioral feedback, and dynamically allocates computing resources to achieve collaborative display among groups.
It improves the smoothness and efficiency of team collaboration, reduces the learning cost and operational burden for users, and automatically reaches consensus through micro-behavioral feedback, thus naturalizing the group decision-making process.
Smart Images

Figure CN123114259A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of virtual reality glasses technology, and more particularly to a virtual reality glasses system based on artificial intelligence. Background Technology
[0002] This research aims to construct an AI-based virtual reality glasses system. By integrating biosensors, mobile terminals, cloud computing, and data visualization technologies, it seeks to achieve real-time, dynamic monitoring, intelligent analysis, and intuitive graphical presentation of multi-dimensional human physiological indicators such as heart rate, blood pressure, blood sugar, body fat percentage, muscle mass, and athletic ability. Its core objective is to provide users with a convenient and efficient self-health management tool to monitor changes in physical condition, identify potential health risks, and develop scientific exercise and nutrition plans. Simultaneously, it provides structured, continuous, and traceable health data support for medical institutions, health management organizations, and research units to optimize treatment pathways, conduct precise health interventions, and promote innovation in physical health research. Ultimately, this will improve health literacy and quality of life at the individual level, reduce medical costs and disease burden, and promote the popularization of public health awareness, support the implementation of the Healthy China strategy, and drive the digital transformation and high-quality development of the health industry. Finally, it aims to achieve a paradigm shift from disease treatment to health management, constructing a closed-loop health management system encompassing prevention, monitoring, intervention, and improvement. This system possesses significant public health value, social benefits, and economic significance.
[0003] In existing technologies, when multiple users attempt to manipulate the same virtual object simultaneously, the system pre-assigns an interaction permission level to each user. When user A attempts to move an object while user B simultaneously attempts to rotate it, the system compares their permission levels, fully executing the instructions of the higher interaction permission level and ignoring or blocking the instructions of the lower interaction permission level. This approach relies entirely on static, pre-defined role labels, representing a typical form of hard priority preemption. It cannot perceive the user's current focus or task context, nor can it engage in any form of negotiation. If the higher interaction permission level user makes a mistake or has an unreasonable intent, other users can only passively accept it. Therefore, this paper proposes an artificial intelligence-based virtual reality glasses system. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an artificial intelligence-based virtual reality glasses system.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: A system for virtual reality glasses based on artificial intelligence, comprising: Central data buffer: Connected to multiple physiological and environmental parameter acquisition modules, including at least eye-tracking sensors, EEG sensors, voice sensors, and inertial measurement units, to uniformly receive and process raw acquired data, extract and broadcast multimodal feature data including eye-tracking features, EEG features, voice features, and motion features; The context arbitrator is connected to the central data buffer. The context arbitrator determines the user's current micro-context in real time based on the motion characteristics and multimodal characteristics data. The micro-context includes at least the individual motion state and the group cooperation state. Based on the determination result, the arbitrator dynamically allocates system computing resources. The distributed intent fusion engine is connected to the context arbitrator and the central data cache. The distributed intent fusion engine is activated when the context arbitrator determines that the current state is a group collaboration state. It calculates the context awareness priority by analyzing the attention focus, task role and physiological focus index of multiple users, detects the display intent of users, and generates a visual consensus window when multiple users' display intents conflict. It automatically reaches a consensus by simultaneously displaying conflicting intent options in the augmented reality field of view and collecting the micro-behavioral feedback of users within a preset time, and outputs a group collaborative display instruction. The environment-adaptive dynamic interaction module is connected to the context arbitrator and the central data buffer. The environment-adaptive dynamic interaction module dynamically integrates and switches interaction paradigms according to the individual's motion state determined by the context arbitrator. When it is determined that the user is walking or moving, it reduces the reliance on fine gestures and eye-tracking clicks and enhances the analysis of the motor imagery component in the EEG signal to generate control commands. When it is determined that the user is stationary or focused, it enables a fine operation mode based on micro-gestures and eye tracking to generate control commands. Edge-cloud collaborative processing module: Deployed on a local AR chip and a cloud server, the local AR chip runs an ultra-lightweight eye-tracking-electroencephalogram joint encoder compressed using knowledge distillation technology, which is used to quickly determine the rough direction of the user's intention and generate corresponding rough intention labels and confidence data to be sent to the cloud, and preload potential holographic content based on the rough intention labels. The cloud server is deployed with a hybrid expert model, which is used to receive and decode multimodal complex data including speech to generate accurate display intentions. The federated continuous learning module is connected to the local AR chip and the cloud server respectively. The federated continuous learning module records the user's interaction feedback data with the system in real time locally, continuously updates the parameters of the local lightweight user habit model, and uploads the model parameter update data to the cloud server under encrypted conditions. The cloud server is used to aggregate the model parameter update data from multiple terminals, generate the updated basic model, and distribute it to each terminal periodically. The physiological closed-loop display control module is connected to the central data buffer. The physiological closed-loop display control module monitors visual fatigue indicators obtained from the central data buffer in real time, including micro-fluctuations, blink frequency and alpha waves of brain waves. Based on a preset mapping library, it dynamically adjusts the visual parameters of the holographic display. When the accumulated fatigue exceeds the safety threshold, it generates and outputs intelligent rest prompts, including information softening or display position guidance. The visual synthesizer is connected to the distributed intent fusion engine, the environment adaptive dynamic interaction module, the edge-cloud collaborative processing module, and the physiological closed-loop display control module. The visual synthesizer receives the group collaborative display instruction, the control instruction, the precise display intent, and the intelligent rest prompt instruction, and performs unified rendering and overlay on all instructions to drive the AR display module on the smart contact lenses to perform holographic display.
[0006] The above technical solution further includes: Furthermore, the specific method for generating the visual consensus window in the distributed intent fusion engine is as follows: When the system detects a conflict between the first user's intent and the second user's intent, for example, when the first user's intent is to zoom in on details while the second user's intent is to switch perspectives, it does not immediately execute either instruction. Instead, it simultaneously merges the preview images of the first user's intent and the second user's intent in a semi-transparent manner in the AR field of view of the smart contact lens, and guides the user to conduct non-verbal negotiation through visual cue elements. The system collects the gaze duration and eye movement trajectory convergence of most users within a subsequent preset time window as the micro-behavioral feedback, and automatically selects the display mode that has been watched for the longest time as the group collaborative display instruction based on the analysis results.
[0007] Furthermore, the individual's movement state includes walking or moving patterns and stationary or focused patterns; In walking or moving mode, the environment adaptive dynamic interaction module analyzes the EEG signals of the user imagining movement in a specific direction and combines them with head rotation data to generate control commands for switching the augmented reality interface, thereby avoiding accidental eye tracking or gesture recognition caused by body shaking. In static or focused mode, the piezoelectric or electromyographic sensors integrated on the smart contact lenses capture the user's subtle finger pinching movements and, combined with the point of gaze, enable precise selection and dragging of virtual interface elements.
[0008] Furthermore, the intent prediction mechanism in the edge-cloud collaborative processing module is as follows: while the ultra-lightweight eye-tracking and EEG co-encoder on the local AR chip outputs coarse intent labels and confidence data, it predicts the user's potential next interaction target based on the current eye movement trajectory, and sends a resource request to the cloud server in advance or preloads holographic display content related to the predicted target from the local cache, so as to shorten the end-to-end system response delay when the user subsequently issues a voice command.
[0009] Furthermore, the local personalized fine-tuning in the federated continuous learning module is based on the user's active correction of the system's automatically recommended content or corrective feedback on the intent recognition results as an online data stream, which updates the weight parameters of the local lightweight user habit model in real time and in small increments. After aggregating model update data from a large number of anonymous users, the cloud server discovers a collective habit drift pattern and optimizes the adaptive bias of the base model accordingly, enabling the periodically distributed model to adapt to the long-term evolution patterns of a wide range of users.
[0010] Furthermore, the dynamic adjustment of visual parameters by the physiological closed-loop display control module includes: When the user's visual fatigue index is detected to be rising, the intensity of blue light of a specific wavelength in the holographic display is automatically and smoothly reduced to improve the contrast of the displayed content, or the focal plane of the display is dynamically fine-tuned according to the real-time pupil size data to make it more in line with the user's current ciliary muscle accommodation state. When the accumulated fatigue level exceeds the safety threshold, the intelligent rest prompt instruction uses a visual synthesizer to present non-critical information in a softened manner in the non-central area of the AR field of vision, or guides the display position of critical information to a preset area below the field of vision, so as to force the user's eye muscles to relax.
[0011] Furthermore, the rules for the dynamic allocation of system computing resources by the context arbitrator include: When the user is determined to be walking based on the inertial measurement unit data, computing resources are prioritized for allocation to the environment adaptive dynamic interaction module, and the group collaboration detection function of the distributed intent fusion engine is paused. When the multimodal feature data indicates that the user is in a static collaborative state with overlapping gaze points among multiple users, the distributed intent fusion engine is activated, and the federated continuous learning module is simultaneously supported in accumulating data and updating the model in the background.
[0012] Furthermore, the unified rendering and overlay process of the visual synthesizer for multiple instructions includes: simultaneously receiving and parsing the visual consensus window display requirement from the distributed intent fusion engine and the contrast reduction adjustment requirement from the physiological closed-loop display control module; rendering the visual consensus window with visual parameters that conform to the reduced contrast through an image processing algorithm; and presenting a low-contrast consensus window image on the AR display module that both achieves group negotiation consensus and meets the current user's visual health protection requirements.
[0013] The present invention has the following beneficial effects: In this invention, a distributed intent fusion engine is introduced to proactively generate a visual consensus window when intent conflicts are detected. Consensus is automatically reached by collecting micro-behavioral feedback from the group. Once consensus is reached, the distributed intent fusion engine immediately generates a clear group collaborative display instruction, making the AR display content a result of team decision-making. This greatly improves the smoothness and efficiency of team collaboration. By replacing traditional button or voice voting with micro-behavioral feedback, the group decision-making process becomes seamless and natural, reducing the user's learning cost and operational burden. Attached Figure Description
[0014] Figure 1 This is a system block diagram of a virtual reality glasses system based on artificial intelligence proposed in this invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] Please see Figure 1 As shown, the present invention is a system for virtual reality glasses based on artificial intelligence, comprising: Central data buffer: Connected to multiple physiological and environmental parameter acquisition modules, including at least eye-tracking sensors, EEG sensors, voice sensors, and inertial measurement units, to uniformly receive and process raw acquired data, extract and broadcast multimodal feature data including eye-tracking features, EEG features, voice features, and motion features; It should be noted that the specific analysis process for the central data cache is as follows: Establish a multi-source sensor data access channel: Establish a standardized data interface with multiple physiological and environmental parameter acquisition modules, which include at least an eye-tracking sensor, an EEG sensor, a voice sensor, and an inertial measurement unit. Through the standardized data interface, raw data from each sensor is received simultaneously at a preset sampling frequency, and a unified timestamp is added to each frame of raw data to ensure strict synchronization of multimodal data in the time dimension. Perform preprocessing operations on raw data: Perform targeted preprocessing operations on the various types of raw acquired data: Denoise removal, saccade removal and smoothing filtering are performed on the raw eye movement data acquired by the eye movement sensor; artifact removal, bandpass filtering and baseline correction are performed on the raw EEG signal data acquired by the EEG sensor; endpoint detection, pre-emphasis and noise reduction are performed on the raw speech data acquired by the speech sensor; Kalman filtering and attitude calculation are performed on the raw motion data acquired by the inertial measurement unit to eliminate sensor noise and environmental interference. Multimodal feature data extraction: Based on the preprocessed clean data, feature parameters of various modalities are calculated by a preset feature extraction algorithm: eye movement features including fixation point coordinates, saccade velocity, pupil diameter, fixation duration and saccade trajectory are extracted from eye movement data; EEG features including power spectral density of each frequency band, event-related potential, alpha wave rhythm and EEG symmetry index are extracted from EEG signal data; speech features including Mel frequency cepstral coefficients, fundamental frequency, formants and speech energy are extracted from speech data; and motion features including triaxial acceleration, angular velocity, walking cadence and posture angle are extracted from motion data. Construct a unified format feature data package: The extracted eye-tracking features, EEG features, speech features, and motion features are encapsulated according to a preset data structure to form a standardized multimodal feature data package; The multimodal feature data package retains the added unified timestamp identifier and attaches a data source identifier and feature quality confidence label to support the upper-layer module in evaluating the credibility of the feature data; Establish a central feature data cache queue: Create a circular cache queue with a fixed capacity in local memory, and store the generated multimodal feature data packets into the cache queue in chronological order. When a new data packet is stored and the cache queue is full, the oldest data packet is automatically discarded according to the first-in-first-out principle, so as to always retain the complete feature data within the most recent time window and meet the real-time requirements of each upper-layer module for historical data query. Implement a feature data subscription and broadcasting mechanism: Establish a data distribution service based on a publish-subscribe model to receive feature subscription requests from upper-layer functional modules, which include at least a distributed intent fusion engine, an environment-adaptive dynamic interaction module, an edge-cloud collaborative processing module, a federated continuous learning module, and a physiological closed-loop display control module; based on the subscription conditions registered by each module, extract the corresponding feature data from the central feature data cache queue in real time, and broadcast it in parallel to all subscribed modules through a high-speed data bus; The context arbitrator is connected to the central data buffer. The context arbitrator determines the user's current micro-context in real time based on the motion characteristics and multimodal characteristics data. The micro-context includes at least the individual motion state and the group cooperation state. Based on the determination result, the arbitrator dynamically allocates system computing resources. It should be noted that the specific analysis process of the situational arbitrator is as follows: Establish a data connection with the central data cache and subscribe to multimodal feature data: The context arbitrator establishes a real-time communication connection with the central data cache, and the context arbitrator initiates a data subscription request to the central data cache to obtain specific data types required for subsequent judgment. The subscribed data includes, but is not limited to: motion features reflecting the user's limb movements (mainly from the solution data of the inertial measurement unit), and multimodal feature data including eye movement trajectory, pupil diameter, EEG spectrum, and voice activity detection. Preliminary determination of individual movement state based on motion characteristics: The context arbitrator receives the motion characteristic data in real time and runs a lightweight motion state machine. This state machine calculates the user's stride frequency, walking speed, and body posture stability based on time-series data from the accelerometer and gyroscope. Based on a preset threshold algorithm, the user's individual movement state is initially determined to be one of the following categories: Walking or moving state: When the step frequency exceeds the first threshold and the posture changes frequently; Still or focused state: When the cadence is zero and the amplitude of posture change is below the second threshold for more than a preset duration; Transitional state: A state between the two mentioned above, such as standing with slight movement; Deep identification of group collaboration status by integrating multimodal feature data: Based on the initial determination of individual movement status, the context arbitrator further calls the subscribed multimodal feature data to identify whether the current state is a group collaboration state. The specific identification logic is as follows: Spatial convergence analysis: The context arbitrator analyzes the eye movement characteristics of multiple users and calculates the coordinates of their gaze points in three-dimensional space. If the gaze point coordinates of more than a preset number (e.g., two or more) of users highly overlap within a preset spatial distance threshold and the duration exceeds the gaze time threshold, a group attention event is triggered. Interaction Intent Synchronization Analysis: The context arbitrator obtains the user's rough intent tags output from the edge-cloud collaborative processing module. If multiple users output intent tags related to the currently co-focused object (such as "interested" or "confused") within the same time window, the possibility of group collaboration is further confirmed. Context triggering: If the system calendar, location information, or application layer data indicates that the current state is in a preset collaboration mode such as a multi-person meeting or collaborative design, the context arbitrator will increase its sensitivity to the identification of the group collaboration status. Based on the above analysis, the context arbitrator ultimately determines whether the current micro-context is a group collaboration state and records the user identifiers participating in the collaboration and the target areas of common interest. Based on the micro-context judgment results, a dynamic allocation strategy for system computing resources is implemented: the context arbitrator merges the determined individual movement state with the determined group cooperation state to form the final micro-context judgment result. Subsequently, according to preset resource allocation priority rules, it issues instructions to the operating system or task scheduler to dynamically allocate system computing resources. Scenario 1 (walking or moving, but not in a group): Priority protection module: Environment adaptive dynamic interaction module, the context arbitrator allocates the highest CPU / GPU time slice and memory bandwidth to it to ensure the real-time performance of motion image analysis and anti-shake processing in walking mode; Resource limiting or pause module: Distributed intent fusion engine. Due to the extremely low probability of group collaboration in walking scenarios, the situation arbitrator pauses the wake-up and detection services of this module to save power consumption and reduce the frequency of non-critical fatigue analysis in the physiological closed-loop display control module. Scenario 2 (Still or focused state, and in a group collaborative state): Priority guarantee module: Distributed intent fusion engine. The context arbitrator is activated and fully supports the operation of this engine, reserving sufficient resources for processing the fusion and consensus negotiation of multi-user intents. Collaborative Support Module: Federated Continuous Learning Module, the context arbitrator allows it to use idle computing power in the background for local data accumulation and model fine-tuning.
[0017] Standard support modules: The environment adaptive dynamic interaction module switches to static mode, and the physiological closed-loop display control module resumes full-function operation, jointly ensuring user experience; Scenario 3 (Still or focused state, and single-person mode without group collaboration): Balanced allocation: The situational arbitrator adopts fair scheduling, supports the environment-adaptive dynamic interaction module to perform fine operation recognition, allows the federated continuous learning module to perform online incremental learning, and maintains the normal monitoring of the physiological closed-loop display control module; The distributed intent fusion engine is connected to the context arbitrator and the central data cache. The distributed intent fusion engine is activated when the context arbitrator determines that the current state is a group collaboration state. It calculates the context awareness priority by analyzing the attention focus, task role and physiological focus index of multiple users, detects the display intent of users, and generates a visual consensus window when multiple users' display intents conflict. It automatically reaches a consensus by simultaneously displaying conflicting intent options in the augmented reality field of view and collecting the micro-behavioral feedback of users within a preset time, and outputs a group collaborative display instruction. It should be noted that the specific analysis process of the distributed intent fusion engine is as follows: Activation condition judgment and data acquisition: The distributed intent fusion engine continuously monitors the status signals from the context arbitrator. When the context arbitrator determines that the current scene meets the preset threshold of the group collaboration state (for example, detecting that the gaze points of multiple users highly overlap on the same physical space or virtual object, and the time synchronization exceeds the preset duration), it sends an activation command to the distributed intent fusion engine. Upon receiving the activation command, the distributed intent fusion engine immediately establishes a data subscription connection with the central data cache to acquire in real time the attention focus data of each user participating in the collaboration (such as gaze point coordinates and gaze duration), preset or dynamically identified task role labels (such as explainer, observer, operator), and physiological attention indicators reflecting their cognitive load (such as β wave power, pupil diameter change rate, and blink inhibition degree in EEG signals). Context-aware priority calculation: Based on the acquisition of the above multi-user data, the distributed intent fusion engine starts the context-aware priority calculation thread to quantify the voice or intent priority of each user in the current collaborative task in real time. The engine integrates and calculates the concentration of each user's attention focus (such as the percentage of time spent looking at a certain object), the basic weight coefficient corresponding to their task role (for example, the narrator role has a higher basic weight when dealing with the current narration object), and their real-time physiological focus index (as a dynamic adjustment factor for the basic weight) to obtain the dynamic intent priority score of each individual user. For example, in a scenario where a team is collaborating to view a holographic design drawing, the user who is explaining the drawing while holding a physical model has a higher base weight as the explainer, and their highly focused physiological concentration index further increases this score. In contrast, other users who are listening and observing have a relatively lower current priority score. Multi-user display intent detection and conflict identification: The distributed intent fusion engine simultaneously parses the control command stream of each user obtained from the central data cache, including the coarse intent output by the environment adaptive dynamic interaction module and the precise display intent output by the edge-cloud collaborative processing module; The engine parses each user's intent into a specific display operation request. Then, the engine enters the conflict detection logic: when at least two users' display intents are detected to be targeting the same virtual object or the same spatial area within the same time window, and these intents are mutually exclusive in terms of display effect, it is determined that a display intent conflict has occurred. Generate and present a visual consensus window: Once an intent conflict is detected, the distributed intent fusion engine does not immediately execute the instructions of either party, but instead generates a visual consensus window; The engine performs image-level fusion processing on the intention options of both sides in the conflict, generating a semi-transparent, split-screen or picture-in-picture preview image. This preview image is rendered by the visual synthesizer and finally overlaid on the AR field of view of each relevant user's smart contact lenses, usually located in the vicinity of the conflict focus. At the same time, the distributed intent fusion engine overlays subtle visual cue elements on the visual consensus window, such as adding a dynamically flashing border to each intent option, or using a bright halo to indicate which option has received more group attention. These visual cuees are designed to guide users to express their preferences through natural physiological responses. Micro-behavior collection and consensus achievement: Within a preset time after the visual consensus window is presented, the distributed intent fusion engine intensively collects the micro-behavior feedback of all relevant users through the central data cache. The micro-behavior feedback includes, but is not limited to: the duration of each user's continuous gaze on different intent options (i.e., the total time that the gaze stays on a certain option area), the convergence of eye movement trajectory (i.e., whether the gazes of multiple people gradually focus on the same option), and weak recognition signals captured by EEG or EMG sensors (such as the EMG characteristics caused by slight nodding movements). The engine performs real-time statistical analysis on the collected group micro-behavior data. For example, it calculates the cumulative gaze duration of all users in the zoom-in detail option area and compares it with the cumulative gaze duration in the view switching option area. Combined with the calculated dynamic intent priority score (for example, giving higher weight to the presenter's gaze behavior), the engine automatically selects the intent option that has received the longest attention or the strongest group recognition signal. Outputting a group collaborative display instruction: After consensus is reached, the distributed intent fusion engine immediately generates a clear group collaborative display instruction, which includes the final winning display content and display parameters. This group collaborative display instruction is sent to the visual synthesizer. The environment-adaptive dynamic interaction module is connected to the context arbitrator and the central data buffer. The environment-adaptive dynamic interaction module dynamically integrates and switches interaction paradigms according to the individual's motion state determined by the context arbitrator. When it is determined that the user is walking or moving, it reduces the reliance on fine gestures and eye-tracking clicks and enhances the analysis of the motor imagery component in the EEG signal to generate control commands. When it is determined that the user is stationary or focused, it enables a fine operation mode based on micro-gestures and eye tracking to generate control commands. It should be noted that the specific analysis process for the environment adaptive dynamic interaction module is as follows: Real-time acquisition and analysis of user's multimodal feature data and individual motion state: Acquire preprocessed multimodal feature data, and simultaneously receive the individual motion state label output by the context arbitrator based on the motion feature data and multimodal feature data for comprehensive judgment; The corresponding interaction paradigm is dynamically selected and activated based on the individual's motion state label: The environment-adaptive dynamic interaction module has a built-in interaction paradigm selector, which takes the individual's motion state label as input and performs the following logical judgment: If an individual's motion state label is "walking" or "moving," then the interaction paradigm selector activates the motion-enhanced interaction mode. In this mode, the system executes the following sub-steps: Reduce the weight of unstable modalities: Automatically reduce the parsing weight of eye-tracking-based gaze-click commands and fine gesture recognition commands, as these modalities are prone to false triggering when the body is swaying; Enhanced motor imagery EEG analysis: Real-time acquisition of sensorimotor rhythm features from EEG signals from the central data buffer, and analysis of EEG patterns indicating whether the user has imagined motor intentions such as moving left, right, forward, or confirmation through a locally deployed lightweight motor imagery decoding model; Multimodal fusion verification: The decoded motion imagery intention and head rotation direction data are correlated over time. If the two trends are consistent (for example, the user imagines to move to the left while the head turns slightly to the left), the corresponding control command is generated. If the individual's motion state label is static or focused: then the interaction paradigm selector activates the fine-grained operation interaction mode. In this mode, the system executes the following sub-steps: Activate the micro-gesture acquisition channel: wake up the piezoelectric sensor or electromyography sensor integrated into the smart contact lens or peripheral device and connected to the central data buffer, and start high-frequency acquisition of the user's tiny muscle activity signals around the eyes or hands. Establish a gaze-micro-gesture linkage mechanism: track the user's gaze in real time to determine its hovering position in the AR virtual interface; at the same time, perform pattern recognition on the signals collected by the micro-gesture sensor to determine whether the user has made preset micro-gesture actions such as slight pinching, tapping or swiping. Generate precise operation commands: When the user's gaze is detected to be stable on an interactive virtual object, and a valid confirmation micro-gesture (such as pinching) is recognized within a preset time window, a selection control command for the virtual object is generated. If a sliding micro-gesture is recognized while the gaze is moving, a corresponding drag or scroll control command is generated. Output control commands and feed them back to the visual synthesizer: The environment adaptive dynamic interaction module will send the generated control commands, along with the confidence data of the commands, to the visual synthesizer in real time. Edge-cloud collaborative processing module: Deployed on a local AR chip and a cloud server, the local AR chip runs an ultra-lightweight eye-tracking-electroencephalogram joint encoder compressed using knowledge distillation technology, which is used to quickly determine the rough direction of the user's intention and generate corresponding rough intention labels and confidence data to be sent to the cloud, and preload potential holographic content based on the rough intention labels. The cloud server is deployed with a hybrid expert model, which is used to receive and decode multimodal complex data including speech to generate accurate display intentions. It should be noted that the specific analysis process for the edge-cloud collaborative processing module is as follows: By deploying an ultra-lightweight eye-tracking and EEG co-encoder on a local AR chip, the user's eye movement data and EEG signal data can be acquired in real time. The ultra-lightweight eye-tracking and EEG co-encoder uses knowledge distillation technology to compress a preset multimodal large model, which is used to quickly infer the input eye-tracking data and EEG signal data to determine the user's rough direction in a scenario without voice interaction, and to generate corresponding rough intention labels and confidence data representing the reliability of the judgment. The local AR chip sends the generated coarse intent label and confidence data to the cloud server. At the same time, the local AR chip performs intent prediction based on the potential interaction target indicated by the coarse intent label, and preloads holographic content data associated with the potential interaction target from local cache or through the network. The cloud server receives coarse intent labels and confidence data from the local AR chip, and simultaneously receives collected user voice data. The cloud server calls the hybrid expert model deployed on it, takes the received voice data, the coarse intent label and the confidence data as input, performs deep fusion and fine decoding of multimodal features, and generates precise display intent to guide AR display. The cloud server sends the generated precise display intent to the local AR chip, which then combines the pre-loaded holographic content to perform precise holographic projection display. The federated continuous learning module is connected to the local AR chip and the cloud server respectively. The federated continuous learning module records the user's interaction feedback data with the system in real time locally, continuously updates the parameters of the local lightweight user habit model, and uploads the model parameter update data to the cloud server under encrypted conditions. The cloud server is used to aggregate the model parameter update data from multiple terminals, generate the updated basic model, and distribute it to each terminal periodically. The physiological closed-loop display control module is connected to the central data buffer. The physiological closed-loop display control module monitors visual fatigue indicators obtained from the central data buffer in real time, including micro-fluctuations, blink frequency and alpha waves of brain waves. Based on a preset mapping library, it dynamically adjusts the visual parameters of the holographic display. When the accumulated fatigue exceeds the safety threshold, it generates and outputs intelligent rest prompts, including information softening or display position guidance. The visual synthesizer is connected to the distributed intent fusion engine, the environment adaptive dynamic interaction module, the edge-cloud collaborative processing module, and the physiological closed-loop display control module. The visual synthesizer receives the group collaborative display instruction, the control instruction, the precise display intent, and the intelligent rest prompt instruction, and performs unified rendering and overlay on all instructions to drive the AR display module on the smart contact lenses to perform holographic display. In one embodiment, the specific method for generating the visual consensus window in the distributed intent fusion engine is as follows: When the system detects a conflict between the first user's intent and the second user's intent, for example, when the first user's intent is to zoom in on details while the second user's intent is to switch perspectives, it does not immediately execute either instruction. Instead, it simultaneously merges the preview images of the first user's intent and the second user's intent in a semi-transparent manner in the AR field of view of the smart contact lens, and guides the user to conduct non-verbal negotiation through visual cue elements. The system collects the gaze duration and eye movement trajectory convergence of most users within a subsequent preset time window as the micro-behavioral feedback, and automatically selects the display mode that has been watched for the longest time as the group collaborative display instruction based on the analysis results.
[0018] In one embodiment, the individual's movement state includes walking or moving patterns and stationary or focused patterns; In walking or moving mode, the environment adaptive dynamic interaction module analyzes the EEG signals of the user imagining movement in a specific direction and combines them with head rotation data to generate control commands for switching the augmented reality interface, thereby avoiding accidental eye tracking or gesture recognition caused by body shaking. In static or focused mode, the piezoelectric or electromyographic sensors integrated on the smart contact lenses capture the user's subtle finger pinching movements and, combined with the point of gaze, enable precise selection and dragging of virtual interface elements.
[0019] In one embodiment, the intent prediction mechanism in the edge-cloud collaborative processing module is as follows: while outputting coarse intent labels and confidence data, the ultra-lightweight eye-tracking and EEG co-encoder on the local AR chip predicts the user's potential next interaction target based on the current eye movement trajectory, and sends a resource request to the cloud server in advance or preloads holographic display content related to the predicted target from the local cache, so as to shorten the end-to-end system response delay when the user subsequently issues a voice command.
[0020] In one embodiment, the local personalized fine-tuning in the federated continuous learning module is based on the user's active correction of the system's automatically recommended content or corrective feedback on the intent recognition results as an online data stream, which updates the weight parameters of the local lightweight user habit model in real time and in small increments. After aggregating model update data from a large number of anonymous users, the cloud server discovers a collective habit drift pattern and optimizes the adaptive bias of the base model accordingly, enabling the periodically distributed model to adapt to the long-term evolution patterns of a wide range of users.
[0021] In one embodiment, the dynamic adjustment of visual parameters by the physiological closed-loop display control module includes: When the user's visual fatigue index is detected to be rising, the intensity of blue light of a specific wavelength in the holographic display is automatically and smoothly reduced to improve the contrast of the displayed content, or the focal plane of the display is dynamically fine-tuned according to the real-time pupil size data to make it more in line with the user's current ciliary muscle accommodation state. When the accumulated fatigue level exceeds the safety threshold, the intelligent rest prompt instruction uses a visual synthesizer to present non-critical information in a softened manner in the non-central area of the AR field of vision, or guides the display position of critical information to a preset area below the field of vision, so as to force the user's eye muscles to relax.
[0022] In one embodiment, the rules for the context arbitrator to dynamically allocate system computing resources include: When the user is determined to be walking based on the inertial measurement unit data, computing resources are prioritized for allocation to the environment adaptive dynamic interaction module, and the group collaboration detection function of the distributed intent fusion engine is paused. When the multimodal feature data indicates that the user is in a static collaborative state with overlapping gaze points among multiple users, the distributed intent fusion engine is activated, and the federated continuous learning module is simultaneously supported in accumulating data and updating the model in the background.
[0023] In one embodiment, the unified rendering and overlay process of the visual synthesizer for multiple instructions includes: simultaneously receiving and parsing the visual consensus window display requirement from the distributed intent fusion engine and the contrast reduction adjustment requirement from the physiological closed-loop display control module; rendering the visual consensus window with visual parameters that conform to the reduced contrast using an image processing algorithm; and presenting a low-contrast consensus window image on the AR display module that both achieves group negotiation consensus and meets the current user's visual health protection requirements.
[0024] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. An artificial intelligence-based system for virtual reality glasses, characterized by, include: Central data buffer: Connects to multiple physiological and environmental parameter acquisition modules to extract and broadcast multimodal feature data, including eye-tracking features, electroencephalogram features, speech features, and motion features; Context arbitrator: Based on the motion characteristics and multimodal feature data, it determines the current micro-context of the user in real time and dynamically allocates system computing resources based on the determination result; Distributed Intent Fusion Engine: Detects the user's display intent. When multiple users' display intents conflict, it generates a visual consensus window. Based on the user's gaze duration and eye movement trajectory convergence as micro-behavioral feedback, it selects the display method that has been watched for the longest time as the group collaborative display instruction. Environment-adaptive dynamic interaction module: Based on the individual's motion state determined by the context arbitrator, dynamically integrate and switch interaction paradigms; Edge-cloud collaborative processing module: Deployed on local AR chip and cloud server, the local AR chip determines the rough direction of the user's intention and generates corresponding rough intention label and confidence data and sends them to the cloud, and preloads potential holographic content based on the rough intention label. The cloud server receives multimodal complex data and decodes it to generate accurate display intention. Federated Continuous Learning Module: Records interactive feedback data locally in real time, continuously updates the parameters of the local lightweight user habit model, and uploads the model parameter update data to the cloud server under encrypted conditions. The cloud server is used to aggregate model parameter update data from multiple terminals, generate an updated basic model, and distribute it to each terminal periodically. Physiological closed-loop display control module: Real-time monitoring of visual fatigue indicators obtained from the central data buffer, and dynamic adjustment of visual parameters of holographic display based on a preset mapping library; Visual compositor: performs unified rendering and overlay of all instructions to achieve holographic display. 2.The system for virtual reality glasses based on artificial intelligence according to claim 1, wherein, The specific method for generating the visual consensus window in the distributed intent fusion engine is as follows: When the system detects a conflict between the first user's intent and the second user's intent, it simultaneously merges the preview images of the first user's intent and the second user's intent in a semi-transparent manner in the AR field of view of the smart contact lens, and guides the user to conduct non-verbal negotiation through visual prompts. Within a subsequent preset time window, the fixation duration and eye movement trajectory convergence of most users are collected as micro-behavioral feedback, and the display method that receives the longest attention is selected as the group collaborative display instruction. 3.The artificial intelligence-based system for virtual reality glasses according to claim 1, characterized in that, The individual's movement state includes walking or moving patterns and stationary or focused patterns; In walking or moving mode, the environment adaptive dynamic interaction module analyzes the EEG signals of the user imagining movement in a specific direction and combines them with head rotation data to generate control commands for switching the augmented reality interface. In static or focused mode, the smart contact lenses capture subtle finger pinching movements of the user through piezoelectric or electromyographic sensors integrated on the lenses, combined with the point of gaze. 4.The system for virtual reality glasses based on artificial intelligence according to claim 1, wherein, The intent prediction mechanism in the edge-cloud collaborative processing module is as follows: while outputting rough intent labels and confidence data, the eye-tracking and EEG co-encoder on the local AR chip predicts the user's potential next interaction target based on the current eye movement trajectory, and sends a resource request to the cloud server in advance or preloads holographic display content related to the predicted target from the local cache.
5. The system for virtual reality glasses based on artificial intelligence according to claim 1, characterized in that, The federated continuous learning module updates the weight parameters of the local lightweight user habit model based on the user's active correction of the system's automatically recommended content or corrective feedback on the intent recognition results as an online data stream. After aggregating model update data from anonymous users, the cloud server discovers a collective habit drift pattern and optimizes the adaptive bias of the base model.
6. The system for virtual reality glasses based on artificial intelligence according to claim 1, characterized in that, The dynamic adjustment of visual parameters by the physiological closed-loop display control module includes: When an increase in the user's visual fatigue index is detected, the intensity of blue light of a specific wavelength in the holographic display is reduced, the contrast of the displayed content is increased, or the focal plane of the display is dynamically fine-tuned based on real-time pupil size data. When the accumulated fatigue level exceeds the safety threshold, the intelligent rest prompt instruction uses a visual synthesizer to present non-critical information in a softened manner in the non-central area of the AR field of view, or guides the display position of critical information to a preset area below the field of view.
7. The system for virtual reality glasses based on artificial intelligence according to claim 1, characterized in that, The rules for dynamically allocating system computing resources by the context arbitrator include: When the user is determined to be walking based on the inertial measurement unit data, computing resources are prioritized for allocation to the environment adaptive dynamic interaction module, and the group collaboration detection function of the distributed intent fusion engine is paused. When the multimodal feature data indicates that the user is in a static collaborative state with overlapping gaze points among multiple users, the distributed intent fusion engine is activated, and the federated continuous learning module is simultaneously supported in accumulating data and updating the model in the background.
8. The system for virtual reality glasses based on artificial intelligence according to claim 1, characterized in that, The unified rendering and overlay process of the visual synthesizer for multiple instructions includes: simultaneously receiving and parsing the visual consensus window display requirements from the distributed intent fusion engine and the contrast reduction adjustment requirements from the physiological closed-loop display control module; rendering the visual consensus window with visual parameters that conform to the reduced contrast using an image processing algorithm; and presenting the low-contrast consensus window image on the AR display module.
