Hybrid EEG-BCI with eye tracker for cursor control during computer navigation

Abstract

A hybrid EEG-BCI with an eye tracker for cursor control in computer navigation, consisting of: an eye tracker device, an EEG cap, a microprocessor a cloud storage and a web application the user wears the EEG cap and positions the eye tracker for accurate measurements, and the system is calibrated by collecting baseline EEG and eye tracking data to adjust the interface.

Description

AREA OF INVENTION

[0001] This invention relates to a hybrid EEG-BCI with eye tracker for cursor control in computer navigation. BACKGROUND OF THE INVENTION

[0002] The proposed solution integrates a hybrid brain-computer interface (BCI) with electroencephalography (EEG) and eye-tracking technology to improve cursor control in computer navigation. This system aims to enable users to interact intuitively and efficiently, particularly for people with limited mobility. To allow people with severe motor impairments to control a computer cursor, we propose a hybrid system that integrates an EEG-based brain-computer interface (BCI) and eye-tracking technology. This innovative approach allows users to control the cursor via brain signals and gaze direction, thus enabling intuitive and independent computer navigation. By combining these non-invasive methods, the system improves accessibility and allows users to interact more effectively with the technology.

[0003] US12001602B2 Brain-computer interface with adjustments for fast, accurate and intuitive user interaction

[0004] Research gap: A hybrid EEG-BCI with eye tracking combines brain signals and eye movements for precise cursor control, while a conventional BCI focuses solely on interpreting brain activity for fast and intuitive interactions. The hybrid offers improved navigation precision, while the dedicated BCI prioritizes rapid signal processing.

[0005] US20220404910A1 Brain-computer interface with high-speed eye-tracking functions

[0006] Research gap: A brain-computer interface (BCI) with high-speed eye tracking integrates real-time eye movement data, thereby improving user interaction. This technology enables precise tracking of the user's gaze, resulting in faster and more intuitive control of devices or applications. It can be used in games, assistive technologies for people with disabilities, and research, creating a seamless connection between visual attention and brain signals to enhance responsiveness and accuracy.

[0007] US10712820B2 Systems and methods for a hybrid brain interface for robot swarms using EEG signals and an input device

[0008] Research gap: Emotiv's system focuses primarily on thought control via EEG. While it enables basic control of digital devices, it is not tailored to smart home environments. Our system offers not only thought-based control but also eye-tracking precision, making it suitable for a wider range of devices and living environments. Summary of the invention

[0009] The hybrid EEG-BCI with eye tracker integrates data on brainwave activity and eye movements to improve cursor control in computer navigation. The EEG captures neural signals related to intention, while the eye tracker provides precise gaze direction. This combination enables smoother and more intuitive control. Users can select elements or navigate user interfaces by focusing on targets and using brain signals to execute commands. The goal is to improve accessibility for users with limited mobility and create a more seamless interaction experience. DETAILED DESCRIPTION OF THE INVENTION

[0010] The hybrid EEG-BCI with eye tracker integrates data on brainwave activity and eye movements to improve cursor control in computer navigation. The EEG captures neural signals related to intention, while the eye tracker provides precise gaze direction. This combination enables smoother and more intuitive control. Users can select elements or navigate user interfaces by focusing on targets and using brain signals to execute commands. The goal is to improve accessibility for users with limited mobility and create a more seamless interaction experience. THE ENTIRE PROCESS IN BRIEF:

[0011] The device consists of 3 main parts. 1. Introduction 2nd version 3. Introduction: During the introductory phase of using a hybrid EEG-BCI and eye tracker, the user wears the EEG cap and positions the eye tracker for accurate measurements. The system then calibrates itself by acquiring baseline EEG and eye-tracking data to adapt the user interface. After the navigation task and instructions are presented, the user can learn to control the cursor through brain activity and gaze tracking in short practice sessions, thus creating the conditions for effective interaction.

[0012] Implementation: A hybrid EEG-brain-computer interface (BCI) and an eye tracker enhance cursor navigation by combining brainwave activity and eye movements. The EEG captures signals related to cursor movement intentions, while the eye tracker identifies gaze direction. Real-time algorithms process this input to translate neural and visual signals into cursor movements, enabling intuitive navigation. The system requires initial calibration for optimal accuracy and provides users with continuous feedback to refine their control.

[0013] Conclusion: In the conclusion phase, which utilizes a hybrid EEG-BCI and an eye tracker for cursor navigation, the process ends once the user has successfully completed the task and receives confirmation through visual or auditory feedback. Users can review their performance metrics and gain insights into their interaction. After removing the EEG cap and eye tracker, the system saves relevant settings and calibration data for future sessions, ensuring a personalized experience. This phase allows users to reflect on their performance and prepare for subsequent tasks.

[0014] Process: The development process for the hybrid EEG-BCI with eye tracker for cursor control in computer navigation begins with initiation. This involves defining objectives, conducting market research, and creating a project plan. The implementation phase follows, encompassing the design and development of prototypes integrating brain-computer interface (BCI) and eye-tracking technologies. User testing gathers feedback, leading to iterative improvements and system integration with eye trackers. Finally, the system's effectiveness is evaluated, necessary adjustments are made, and comprehensive documentation is created. The project culminates in market launch, which includes user training and ongoing support to ensure optimal operation.

[0015] The hybrid EEG-BCI system combines brainwave activity and eye-tracking data for more precise and intuitive cursor control. Once the user puts on the EEG cap and positions the eye tracker, the system first calibrates itself to baseline values, capturing the user's brainwave patterns and eye movements at rest. This allows the system to adapt to the user's individual brain activity and gaze direction. Once calibration is complete, the user can interact with the system. The EEG cap captures brain signals indicating the user's intention to move the cursor, such as when they are thinking about moving the cursor in a particular direction. Simultaneously, the eye tracker monitors the user's gaze and determines which part of the screen they are focusing on. The data from both devices is then sent to the microprocessor for processing.

[0016] The microprocessor then combines brainwave activity (representing intention) with gaze direction (identifying the target) and translates this into corresponding cursor movements on the computer screen. For example, if the user focuses on a specific symbol and simultaneously intends to select it (indicated by a change in brain activity), the system moves the cursor to that symbol and executes a click or selection.

[0017] The calibration phase is crucial to ensure the system is precisely attuned to the user's brain activity and gaze direction. During this phase, the user is instructed to focus on specific areas of the screen while EEG and eye-tracking devices collect baseline data. This process ensures the system understands the user's individual patterns and compensates for any inconsistencies, resulting in a more personalized experience.

[0018] After calibration, the user can interact with the system by focusing on specific elements on the screen. The system's hybrid nature—the combination of eye-tracking and EEG signals—ensures that it always responds to the user's needs and intentions. Continuous feedback helps the user refine cursor control. As the user becomes more accustomed to controlling the computer via brain signals and gaze direction, the system becomes increasingly intuitive. Execution phase

[0019] During the execution phase, the system operates continuously, processing EEG and eye-tracking data in real time. The combination of brain activity and gaze direction enables smooth and precise cursor control. For example, if the user wants to move the cursor to a specific button, they can look at the button (detected by the eye tracker) while simultaneously thinking about the cursor movement, which is detected by the EEG cap. The system processes these inputs and moves the cursor to the target.

[0020] During this phase, the system also provides the user with real-time feedback by displaying cursor movements on the screen and confirming actions such as clicks or selections. This continuous interaction allows the user to refine their control over time, making the system increasingly efficient and intuitive.

[0021] Once the user has completed a task, such as navigating to a specific location or selecting an item, the completion phase begins. During this phase, successful task completion is confirmed through visual or audible signals. The system can display user performance metrics, such as cursor control accuracy or the time required to complete the task.

[0022] The system then saves all relevant data, including calibration settings and interaction history, to cloud storage. This allows future sessions to be personalized based on past interactions. Users can also view historical data to analyze their performance or identify areas for improvement. Once the task is complete and the data is saved, the system can be shut down or reconfigured for the next user. Advantages of the invention 1. Improved user accuracy: Users can select targets more accurately by combining gaze direction with neural signals, thereby reducing the likelihood of misclicks. 2. Improved interaction speed: The integration enables faster navigation through interfaces, as users can execute commands with minimal delay after focusing on a target. 3. Improved accessibility: People with limited mobility can experience a greater sense of independence, as they can interact with the technology more easily and efficiently. 4. User satisfaction and engagement: Positive feedback from users regarding the intuitive nature of the system can lead to greater satisfaction and stronger engagement with digital content. 5. Shortening the learning curve: With a more natural interaction method, users may find it easier to learn and adapt to the technology, leading to faster onboarding. 6. Real-time feedback and adjustment: The system could provide real-time feedback and help users adjust their focus and intention more effectively during use. 7. Broader application potential: Beyond accessibility, this technology could also be used in games, virtual reality and other interactive environments, thus expanding its impact. 8. Data collection for personalization: Continuous use can generate valuable data that can be analyzed to further personalize the user experience and improve the system over time. 9. Multitasking ability: Users may be able to perform multiple tasks more efficiently because they can control the cursor and execute commands simultaneously through their gaze and brain signals. 10. Research opportunities: The system could open up new avenues for further research into brain-computer interaction, neural processing, and the relationship between eye movements and cognitive intention. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 12001602B2

[0003] US 20220404910A1

[0005] US 10712820B2

[0007]

Claims

[1] A hybrid EEG-BCI with an eye tracker for cursor control in computer navigation, consisting of: an eye tracker device, an EEG cap, a microprocessor a cloud storage and a web application the user wears the EEG cap and positions the eye tracker for accurate measurements, and the system is calibrated by collecting baseline EEG and eye tracking data to adjust the interface. [2] The hybrid EEG-BCI according to claim 1, wherein the system presents the user with a navigation task and instructions, who is then familiarized in short practice sessions with controlling the cursor by brain activity and gaze, thus creating the basis for effective interaction. [3] The hybrid EEG-BCI according to claim 1, wherein the microcontroller processes the raw EEG and eye-tracking data to determine the cursor movement and performs actions based on the combined brain signals and gaze direction. [4] The hybrid EEG-BCI according to claim 1, wherein the system provides confirmation of task completion by visual or acoustic feedback once the user has successfully completed the task. [5] The hybrid EEG-BCI according to claim 1, wherein the execution phase comprises the design and development of prototypes integrating brain-computer interface (BCI) and eye-tracking technologies, followed by user testing to gather feedback and iterative improvements. [6] The hybrid EEG-BCI according to claim 1, wherein during the final phase the effectiveness of the system is evaluated, necessary adjustments are made and comprehensive documentation is created to improve future interactions. [7] The hybrid EEG-BCI according to claim 1, wherein the web application is used to display the results of the eye-tracking device and is connected to the eye-tracking device for real-time data transmission via a Bluetooth connection. [8] The hybrid EEG-BCI according to claim 1, wherein cloud storage is used to collect and store historical data for future analysis, thereby enabling continuous improvement of system performance. [9] The hybrid EEG-BCI according to claim 1, wherein the system integrates a user interface that enables real-time tracking and adjustments based on brain activity and gaze direction to improve the accuracy of cursor control. [10] The hybrid EEG-BCI according to claim 1, wherein the system enables users with limited mobility to navigate computer interfaces with increased precision by using both brainwave signals and eye movements, thereby improving overall user accessibility and interaction.