Electroencephalogram eye tracker based on brain-computer interface, calibration method and device thereof, computer equipment and storage medium

Abstract

The invention provides an electroencephalogram eye tracker based on a brain-computer interface, a calibration method and device of the electroencephalogram eye tracker, computer equipment and a storage medium. According to the scheme, a stable stimulation source is provided for data acquisition by orderly displaying the calibration points, the eye movement data and the electroencephalogram signals are synchronously acquired, and the user state information is acquired from multiple dimensions. The attention index is quantified according to the electroencephalogram signal, the data quality is preliminarily controlled, the target calibration point is displayed again, the data is updated, and errors caused by inattention of the user are corrected. And finally, high-quality eye movement data is utilized to calibrate and compensate eye tracker system errors, individual differences and environmental influences, and the calibration robustness is greatly improved, so that the calibrated electroencephalogram eye tracker can more accurately detect the watching position of the user, and reliable data support is provided for subsequent research and application based on eye movement tracking.

Description

Technical Field

[0001] This application relates to the technical field of brain-computer eye movement devices, and particularly to an electroencephalogram eye movement instrument based on a brain-computer interface, and its calibration method, device, computer equipment, and storage medium. Background Art

[0002] With the rapid development of Virtual Reality (VR) technology, it has been widely used in fields such as education, medical care, and entertainment. In these application scenarios, it is often necessary to combine the eye movement data of users to optimize the usage experience, and high-quality eye movement calibration helps to improve the accuracy of eye movement tracking. However, how to effectively ensure the calibration quality of users still faces multiple challenges. Traditional eye movement calibration techniques usually rely on a preset static calibration point sequence, and establish a mapping model by collecting the coordinate data of the user's fixation points. This solution has the problem of poor robustness. Summary of the Invention

[0003] The purpose of this application aims to solve at least one of the above technical defects, especially the problem of poor robustness existing in the prior art.

[0004] In a first aspect, this application provides a calibration method for an electroencephalogram eye movement instrument, including:

[0005] Successively display multiple calibration points to the user;

[0006] Use the electroencephalogram eye movement instrument to respectively and real-time collect the user's eye movement data and electroencephalogram signals;

[0007] For any one calibration point, determine the corresponding attention index according to the electroencephalogram signal;

[0008] If the attention index is lower than the first threshold, determine the calibration point as the target calibration point;

[0009] Redisplay the target calibration point, and update the corresponding eye movement data when the attention index is higher than the first threshold;

[0010] Calibrate according to the eye movement data.

[0011] In one embodiment, determining the corresponding attention index according to the electroencephalogram signal includes:

[0012] Determine the power spectral density of the beta wave according to the electroencephalogram signal, and determine the attention index according to the power spectral density of the beta wave.

[0013] In one embodiment, redisplaying the target calibration point includes:

[0014] Determine the extension ratio according to the fluctuation degree of the attention index; the greater the fluctuation degree, the greater the extension ratio;

[0015] Extend the display time of the target calibration point according to the extension ratio.

[0016] In one of the embodiments, calibrating according to the eye movement data further includes:

[0017] For any calibration point, determine the coordinates of the stable fixation point according to the eye movement data;

[0018] Perform calibration according to the difference between the coordinates of the stable fixation point and the set coordinates corresponding to the calibration point.

[0019] In one of the embodiments, during the process of sequentially presenting multiple calibration points to the user, it further includes:

[0020] When the fixation position enters the set area range corresponding to the calibration point, send a cognitive trigger stimulus to the user.

[0021] In one of the embodiments, for any calibration point, determining the coordinates of the stable fixation point according to the eye movement data includes:

[0022] Determine the eye movement speed according to the eye movement data;

[0023] Determine the time point when the eye movement speed is lower than the second threshold as the stable fixation time, and the fixation position when the eye movement speed is lower than the second threshold as the initial position;

[0024] Determine the cognitive generation time of the target cognitive component in the electroencephalogram signal; the target cognitive component is the cognitive component with the closest occurrence time to the stable fixation time, and the cognitive component is the electroencephalogram component generated when the visual stimulus appears;

[0025] If the difference between the cognitive generation time and the stable fixation time is greater than the third threshold, correct the initial position to obtain the coordinates of the stable fixation point;

[0026] Otherwise, use the initial position as the stable fixation coordinates.

[0027] In one of the embodiments, correcting the initial position to obtain the coordinates of the stable fixation point includes:

[0028] Determine the eye movement trajectory between the cognitive generation time and the stable fixation time according to the eye movement data;

[0029] Determine the correction amplitude according to the ratio of the absolute value of the time difference between the cognitive generation time and the stable fixation time to the third threshold;

[0030] Obtain the target correction amount according to the correction amplitude and the reference correction amount;

[0031] Starting from the initial position, offset along the eye movement trajectory according to the target correction amount to obtain the coordinates of the stable fixation point.

[0032] Second aspect, the present application provides a calibration device for an electroencephalogram electrooculogram instrument, including:

[0033] A display module for sequentially displaying a plurality of calibration points to the user;

[0034] An acquisition module for respectively and real-time acquiring the user's eye movement data and electroencephalogram signals by using the electroencephalogram electrooculogram instrument;

[0035] An attention index determination module for determining a corresponding attention index according to the electroencephalogram signal for any one calibration point;

[0036] If the attention index is lower than a first threshold, the calibration point is determined as a target calibration point;

[0037] A re-display module for re-displaying the target calibration point and updating the corresponding eye movement data when the attention index is higher than the first threshold;

[0038] A calibration module for calibrating according to the eye movement data.

[0039] Third aspect, the present application provides a computer device, including one or more processors and a memory. When the computer-readable instructions stored in the memory are executed by the one or more processors, the steps of the calibration method of the electroencephalogram electrooculogram instrument based on a brain-computer interface in any one of the above embodiments are executed.

[0040] Fourth aspect, the present application provides an electroencephalogram electrooculogram instrument, including:

[0041] A VR glasses main body;

[0042] An eye movement tracking module arranged inside the VR glasses main body for acquiring the user's eye movement data;

[0043] An electroencephalogram acquisition module arranged inside the VR glasses main body for acquiring the user's electroencephalogram signals;

[0044] The computer device in any one of the above embodiments is respectively connected to the VR glasses main body, the eye movement tracking module and the electroencephalogram acquisition module.

[0045] Fifth aspect, the present application provides a storage medium in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the calibration method of the electroencephalogram electrooculogram instrument in any one of the above embodiments.

[0046] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0047] This solution provides a stable stimulus source for data acquisition by presenting calibration points in an orderly manner, synchronously collecting eye movement data and electroencephalogram (EEG) signals, and obtaining user state information from multiple dimensions. Based on brain-computer interface technology, it quantifies the attention index according to the EEG signal to initially control the data quality, redisplay the target calibration points and update the data, and corrects the errors caused by the user's inattentiveness. Finally, it calibrates using high-quality eye movement data to compensate for the system error, individual differences, and environmental impact of the eye tracker, greatly improving the calibration robustness, enabling the calibrated EEG eye tracker to more accurately detect the user's fixation position, and providing reliable data support for subsequent eye movement tracking-based research and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0049] Figure 1 It is a schematic flowchart of a calibration method for an EEG eye tracker based on a brain-computer interface provided by an embodiment of the present application;

[0050] Figure 2 It is a schematic flowchart of determining a stable fixation coordinate in an embodiment of the present application;

[0051] Figure 3 It is a schematic flowchart of correcting a stable fixation coordinate in an embodiment of the present application;

[0052] Figure 4 It is a schematic structural diagram of an EEG eye tracker in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. The embodiments described in the specification are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0054] The present application provides a calibration method for an EEG eye tracker based on a brain-computer interface. Please refer to Figure 1 , which includes steps S102 to S112.

[0055] S102, sequentially present multiple calibration points to the user.

[0056] It is understandable that the calibration points are specific position points presented to the user through the display part of the electroencephalogram (EEG) and electrooculogram (EOG) instrument for calibrating the EEG and EOG instrument. In VR glasses or other related application scenarios, these calibration points are usually presented in the user's field of view in the form of specific visual stimuli, such as a small circle, square, or other easily recognizable graphics. By gazing at these calibration points, the user provides a reference benchmark for the eye movement data and EEG signals of the EEG and EOG instrument. The display method can be based on screen display technology, presenting these calibration points sequentially in the virtual environment of the VR glasses or on an external display.

[0057] Sequentially presenting multiple calibration points to the user is the starting step of the calibration process of the EEG and EOG instrument. Its purpose is to provide multiple different gazing targets for subsequent collection of the user's eye movement data and EEG signals, so that the instrument can establish a mapping relationship between the user's gazing position and the corresponding physiological signals. By having the user gaze at the calibration points at different positions, the EEG and EOG instrument can record the user's eye movement data and EEG signals in different gazing states. These data will be used for subsequent analysis of the user's attention state and calibration of the gazing detection accuracy of the eye movement instrument.

[0058] In a system based on VR glasses, the calibration points can be presented through the built-in display module of the VR glasses. Developers can use VR development toolkits (such as Unity or Unreal Engine) to create a calibration scene, and generate calibration points sequentially in the scene according to a preset order. For example, a three-dimensional space coordinate system centered on the user's head can be set, and calibration points are generated at different coordinate positions, and the display time and order of the calibration points are controlled through programming. In actual operation, calibration points are usually set at different horizontal, vertical, and depth positions to comprehensively cover the user's field of view. For example, in the horizontal direction, several calibration points are evenly distributed from the leftmost to the rightmost of the user's field of view; in the vertical direction, calibration points are set from the topmost to the bottommost of the field of view; in the depth direction, calibration points at different distances can be set to simulate gazing situations at different distances.

[0059] S104, Use the EEG and EOG instrument to respectively and real-time collect the user's eye movement data and EEG signals.

[0060] It is understandable that the EEG and EOG instrument is respectively configured with an eye movement tracking module and an EEG acquisition module, so as to support the simultaneous measurement of the user's eye movement data and EEG signals. The eye movement data includes information such as the movement trajectory of the eyeball, the gazing point position, the gazing time, the eyeball rotation speed, and the change in pupil diameter. These data reflect the transfer and focus of the user's visual attention. The EEG signal is an electrophysiological signal generated by the activity of brain neurons and can be collected through electrodes placed on the scalp. In the field of brain-computer interface technology, common EEG signal frequency bands include 、 , , and (above 30 Hz), electroencephalogram (EEG) signals in different frequency bands are related to different cognitive states and attention levels of the user. Real-time acquisition means that during the process when the user is looking at the calibration point, the EEG eye tracker continuously obtains eye movement data and EEG signals to capture the changes in the user's physiological state at different moments.

[0061] Real-time acquisition of the user's eye movement data and EEG signals is a key link in the calibration process. Eye movement data can directly reflect the user's fixation behavior. By analyzing the eye movement data, it can be determined whether the user accurately fixates on the calibration point and the stability of the fixation. The EEG signal, on the other hand, provides information about the user's attention state from the neurophysiological level. For example, when the user's attention is concentrated, the power in the β frequency band of the EEG signal may increase, while the power in the α frequency band may decrease. By simultaneously collecting these two types of data, a more comprehensive understanding of the user's physiological and cognitive states when looking at the calibration point can be obtained. The collected data will be used in subsequent steps to determine the user's attention metrics and calibrate the parameters of the EEG eye tracker.

[0062] In specific implementation, the user needs to wear the EEG eye tracker well to ensure that the eye tracking camera can clearly capture the movement of the eyeballs, and the EEG electrodes are in good contact with the head to obtain stable EEG signals. During the process when the user is looking at the calibration point, the eye tracking camera acquires eye images at a high frame rate (such as 120 Hz or higher), analyzes the movement characteristics of the eyeballs through image processing algorithms, and thus calculates the eye movement data. At the same time, the EEG electrodes transmit the collected EEG signals to an amplifier for amplification and analog-to-digital conversion, and then transmit the digitized EEG signals to a computer through a data cable for storage and analysis. In actual operation, to ensure the accuracy and stability of the data, it is usually necessary to calibrate and calibrate the EEG eye tracker. For example, calibrate the eye tracking camera to compensate for lens distortion, and perform impedance detection on the EEG electrodes to ensure good signal transmission.

[0063] S106. For any calibration point, determine the corresponding attention metric according to the EEG signal.

[0064] It can be understood that the attention metric is a quantified value or parameter used to reflect the degree of attention concentration of the user when looking at a certain calibration point. It is obtained by analyzing and processing the EEG signal. Common EEG features used to calculate the attention metric include the power spectral density of specific frequency bands, the power ratio between different frequency bands, event-related potential (ERP) components, etc. For example, the power spectral density of the frequency band, the frequency band and the power ratio of the frequency band ( Ratios, etc. are often used as attention indicators. When this ratio increases, it usually indicates an improvement in the user's attention level. Different attention indicators are applicable to different application scenarios and analysis purposes. By selecting the appropriate attention indicator, the user's attention state can be evaluated more accurately.

[0065] Determining the corresponding attention indicator based on the electroencephalogram (EEG) signal is to quantify the user's attention level when fixating on the calibration point. Since the EEG signal can reflect the neural activities of the brain, and the concentration or distraction of attention will cause changes in the neural activity pattern of the brain, the user's attention state can be inferred by analyzing the EEG signal. During the calibration process of the EEG - based eye tracker, it is very important to understand the user's attention level at each calibration point. If the user's attention is not concentrated at a certain calibration point, the collected eye movement data and EEG signal may be inaccurate, thus affecting the calibration accuracy. By determining the attention indicator, the calibration process can be adjusted according to the attention level in subsequent steps, such as re - presenting the calibration point corresponding to the inattentive state. This step plays a role in connecting the EEG signal acquisition and the evaluation and adjustment of the calibration point in the whole calibration process, providing a basis for subsequent calibration optimization.

[0066] The MNE (MNE - Python) library can be used to analyze the EEG signal and calculate the attention indicator. First, read the collected EEG signal data and pre - process the data, including filtering (removing high - frequency noise and low - frequency drift), artifact removal (removing interference signals caused by muscle activity, electro - oculogram, etc.). Then, calculate the power spectral density of the target frequency band. For example, calculate the power spectral density of the frequency band (13 - 30 Hz) and frequency band (8 - 13 Hz). Thus, the attention indicator is obtained based on the power spectral density of the target frequency band.

[0067] S108, if the attention indicator is lower than the first threshold, the calibration point is determined as the target calibration point.

[0068] It can be understood that the first threshold is a preset numerical standard used to judge whether the user's attention reaches an acceptable level when fixating on a certain calibration point. The target calibration point refers to the calibration point that is determined to need re - processing when the attention indicator is lower than the first threshold. Since the user's attention is not concentrated when fixating on these calibration points, it may lead to inaccurate collected eye movement data and EEG signals. Therefore, these calibration points need to be specially processed, such as re - presenting to obtain more accurate data.

[0069] In actual operation, the setting of the first threshold usually needs to be determined through preliminary pre-experiments. Researchers will let a group of users conduct an electroencephalogram (EEG) eye tracker calibration experiment under standard conditions, collect their EEG signals when looking at the calibration points, and calculate the attention index. Calibration is performed based on the EEG data under different attentions, and according to the change in the fixation detection error of the calibrated eye tracker during actual application, the appropriate first threshold is found. For example, if it is found that when the ratio is lower than 1.5, the fixation detection error of the calibrated eye tracker increases significantly during actual application, then 1.5 can be used as the first threshold.

[0070] S110, redisplay the target calibration point and update the corresponding eye movement data when the attention index is higher than the first threshold.

[0071] It can be understood that redisplaying the target calibration point means presenting the previously determined target calibration point (i.e., the calibration point where the user's attention is not concentrated during fixation) to the user again, allowing the user to refixate, in order to obtain more accurate eye movement data and EEG signals. Updating the corresponding eye movement data means replacing the previously collected potentially inaccurate data with the newly collected eye movement data when the user refixates on the target calibration point and the attention index is higher than the first threshold. This can ensure that the eye movement data used for calibrating the eye tracker is obtained when the user's attention is concentrated, thereby improving the accuracy of calibration.

[0072] Redisplaying the target calibration point and updating the eye movement data is a corrective measure for data quality problems during the calibration process. When it is found that the attention index corresponding to a certain calibration point is lower than the first threshold, it indicates that there may be errors in the previously collected eye movement data and EEG signals at this point because the user's attention is not concentrated during fixation. By redisplaying the target calibration point, giving the user another chance to fixate, and updating the eye movement data when the user's attention is concentrated (i.e., the attention index is higher than the first threshold), the reliability of the data can be improved. This step forms a closed-loop feedback mechanism with the previous steps, and by correcting the data collected under the condition of inattentiveness, the robustness and accuracy of the entire calibration process are improved.

[0073] S112, calibrate according to the eye movement data.

[0074] It can be understood that calibrating according to the eye movement data means using the screened and updated eye movement data to adjust the parameters of the EEG eye tracker, so that the eye tracker can more accurately detect the user's fixation position. The calibration process usually involves establishing the mapping relationship between eye movement data (such as eye rotation angle, fixation point coordinates, etc.) and positions in the actual physical space or virtual space. Through calibration, the errors of the eye tracker can be compensated, and the accuracy and reliability of the eye tracker in subsequent use can be improved.

[0075] This solution provides a stable stimulus source for data acquisition through the orderly display of calibration points, synchronously collects eye movement data and electroencephalogram (EEG) signals, and obtains user state information from multiple dimensions. Based on brain-computer interface technology, the attention index is quantified according to the EEG signal to initially control the data quality, redisplay the target calibration points and update the data, and correct the errors caused by the user's inattentiveness. Finally, using high-quality eye movement data for calibration compensates for the system error, individual differences, and environmental impact of the eye tracker, greatly improving the calibration robustness, enabling the calibrated EEG eye tracker to more accurately detect the user's fixation position, and providing reliable data support for subsequent eye movement tracking-based research and applications.

[0076] In one embodiment, determining the corresponding attention index according to the EEG signal includes: determining the wave power spectral density according to the wave power spectral density and determining the attention index. It can be understood that the EEG signal is an electrophysiological signal generated during the activity of brain neurons. In the field of EEG research, the EEG signal is divided into different frequency bands according to the frequency range, where the frequency range of the wave is usually 13 - 30 Hz. The wave power spectral density is used to measure the energy distribution of the wave at different frequencies, which reflects the energy intensity characteristics of the wave across the entire frequency band. The power spectral density is an index that quantifies the energy distribution of a signal in the frequency domain. In EEG signal analysis, by calculating the wave power spectral density, the neural activity intensity of the brain in the wave frequency band can be understood. Among many EEG frequency bands, the wave is widely considered to be closely related to high-level neural activities such as attention, alertness, and cognitive processing. When an individual is in a state of concentrated attention, the neural activity pattern of the brain changes, and the energy in the wave frequency band usually increases, that is, the wave power spectral density rises. By accurately calculating the wave power spectral density and determining the attention index based on this, the attention level of an individual can be effectively quantified. This process plays a key role in the calibration process of the EEG eye tracker.

[0077] In one embodiment, redisplaying the target calibration point includes: determining the extension ratio according to the fluctuation degree of the attention index. The greater the fluctuation degree, the greater the extension ratio. Extend the display time of the target calibration point according to the extension ratio.

[0078] It can be understood that the fluctuation degree of the attention index refers to the degree of intensity of the change of the attention index over time during the process of the user's fixation on the target calibration point. It reflects the stability of the user's attention concentration state. For example, if there is a large increase or decrease in the attention index within a short period of time, it indicates a large fluctuation degree; if the attention index is relatively stable, the fluctuation degree is small. The extension ratio is a coefficient used to adjust the display time of the target calibration point. It is related to the fluctuation degree of the attention index. The greater the fluctuation degree, the greater the extension ratio. Its function is to reasonably adjust the display time according to the stability of the user's attention.

[0079] In the calibration of the electroencephalogram electrooculogram instrument, the stability of the user's attention is crucial for collecting accurate eye movement data and electroencephalogram signals. When the fluctuation of the attention index is large, it indicates that the user's attention is difficult to be stably concentrated, and may not be able to fully fixate on the target calibration point within the normal display time, resulting in inaccurate collected data. Determining the extension ratio by analyzing the fluctuation degree of the attention index is to give users with unstable attention more time to concentrate. This step is closely coordinated with the subsequent steps. First, determine the extension ratio to provide a basis for extending the display time of the target calibration point, thereby improving the reliability and accuracy of the calibration data and making the entire calibration process more scientific and reasonable. After the extension ratio is determined, on the basis of the original display time, increase a certain duration according to the extension ratio to give users more time to concentrate on fixating on the target calibration point.

[0080] In one of the embodiments, calibrating according to the eye movement data includes: for any calibration point, determining the coordinates of the stable fixation point according to the eye movement data. Calibrating according to the difference between the coordinates of the stable fixation point and the set coordinates corresponding to the calibration point.

[0081] It can be understood that the stable fixation point coordinates are determined from the eye movement data and represent the coordinates of the position where the eyeball focuses when the user is stably fixating on a calibration point. The set coordinates corresponding to the calibration point are the position coordinates of the pre-set calibration point in a specific coordinate system (such as the screen coordinate system or the virtual space coordinate system) and are the reference standard for calibration. Calibration adjusts the parameters of the electroencephalogram (EEG) eye tracker based on the difference between the stable fixation point coordinates and the set coordinates to improve the accuracy of the eye tracker in detecting the user's fixation position. In the use of the EEG eye tracker, due to the errors of the instrument itself, individual physiological differences, and environmental factors, the fixation position detected by the eye tracker may deviate from the actual position. By collecting eye movement data to determine the stable fixation point coordinates and comparing them with the set coordinates of the calibration point, this deviation can be quantified. Calibrating the EEG eye tracker based on this difference can compensate for the systematic error and make the eye tracker more accurately detect the user's fixation position in subsequent use. These two steps cooperate with each other. First, determine the stable fixation point coordinates to obtain the actual fixation position information, and then obtain the error information by comparing with the set coordinates, providing a basis for calibration, so as to achieve the purpose of improving the detection accuracy of the eye tracker.

[0082] In one of the embodiments, during the process of sequentially presenting multiple calibration points to the user, it further includes: when the fixation position enters the set area range corresponding to the calibration point, a cognitive trigger stimulus is sent to the user. The fixation position refers to the specific position where the user's line of sight focuses during the observation process and can be accurately captured by the eye tracker. The set area range corresponding to the calibration point is a specific spatial range pre-defined around each calibration point, which is a regional definition relative to the position of the calibration point and is used to determine whether the user's fixation is close to or enters the area related to the calibration point. The cognitive trigger stimulus is a form of stimulus that can trigger a specific cognitive response of the user. In this scenario, when the user's fixation position enters the set area range, the system will send such a stimulus to the user, and its forms include but are not limited to visual stimuli (such as color change, flashing of the calibration point), auditory stimuli (such as specific prompt sounds), or tactile stimuli (such as slight vibration of the wearable device), as long as it can stimulate the appearance of cognitive components in the EEG signal.

[0083] During the calibration process of the EEG eye tracker, it is crucial to show the user the calibration point and guide the user to look accurately. By setting the area range corresponding to the calibration point, a cognitive trigger stimulus is issued when the user's gaze position enters the range, which can strengthen the user's attention and cognition of the calibration point. This stimulus can produce reactions at the user's physiological and psychological levels, attracting the user's attention and making him more focused on the calibration point. For example, when the user sees a sudden change in the color of the calibration point or hears a specific prompt tone, he will unconsciously focus on the calibration point, thereby improving the accuracy and stability of the gaze. This process helps to improve the user's participation and cooperation in the calibration process, and is closely related to the subsequent collection of accurate eye movement data and EEG signals. Accurate gaze can make the collected data more reflect the user's true visual and cognitive state, provide a reliable basis for the subsequent adjustment of calibration parameters based on these data, and ensure the accuracy and effectiveness of the calibration results.

[0084] In one embodiment, see Figure 2 , for any calibration point, the stable gaze point coordinates are determined according to the eye movement data, including steps S202 to S210.

[0085] S202: Determine eye movement speed according to the eye movement data.

[0086] Eye movement data is a collection of information about eye movement obtained with the help of eye tracking technology. It covers the trajectory of the change of the eye's position coordinates in space over time, the change of pupil diameter, and the relevant data of different types of eye movement patterns such as saccade, fixation and tracking. In the application scenarios of devices such as EEG eye trackers or VR glasses, these data are of key significance for analyzing the user's visual attention allocation and transfer path. Eye movement velocity is a physical quantity that quantifies the rate of change of the eye's position per unit time. It intuitively reflects the speed of eye movement and is usually measured in units such as pixels per second (for screen-based systems) or angles per second (for systems involving spatial positioning). By analyzing eye movement velocity, it is possible to infer whether the user is in a fast scanning state to search for an area of interest during observation, or in a relatively stable fixation state to obtain detailed information about a specific target. In the overall process of determining the coordinates of a stable fixation point, the determination of eye movement velocity is a key starting point. Its core principle is based on the close relationship between eye movement characteristics and visual cognitive behavior. When a user focuses on a specific target, the eye movement pattern will change accordingly, and the eye movement speed in the stable gaze stage is significantly lower than that in the scanning stage. By accurately calculating the eye movement speed, it is possible to effectively distinguish these two different eye movement states, and then filter out the time period that may correspond to stable gaze behavior. Specifically, the eye movement speed can be determined based on the rate of change of the gaze position in the eye movement data.

[0087] S204, determine the time points when the eye movement speed is lower than the second threshold as the stable fixation time, and determine the fixation position when the eye movement speed is lower than the second threshold as the initial position.

[0088] It can be understood that the second threshold is a preset speed value used to determine whether the eyeball is in a stable fixation state. When the eye movement speed is lower than this threshold, it is considered that the eyeball is in a relatively stable state and may be performing a fixation behavior. The stable fixation time refers to the time points when the eye movement speed is lower than the second threshold, which represents the time period during which the eyeball relatively stably fixates on a certain position. The initial position is the fixation position of the user during the stable fixation time and is the basis for determining the coordinates of the stable fixation point subsequently. When the eyeball is in a stable fixation state, the eye movement speed usually maintains at a relatively low level, which is determined by the physiological characteristics of the human visual system. By setting the second threshold, the stable fixation stage can be effectively identified from the continuous eye movement data. The determination of the stable fixation time and the initial position provides key time and space reference points for subsequent analysis in combination with electroencephalogram (EEG) signals.

[0089] S206, determine the cognitive generation time of the target cognitive component in the EEG signal.

[0090] It can be understood that the cognitive component is an EEG response component with specific time and frequency characteristics generated by the brain after cognitive stimuli (such as visual, auditory, tactile, etc. stimuli) act on the human body. Common cognitive components include P300 (a type of event-related potential, usually appearing about 300 milliseconds after the stimulus presentation, related to cognitive processing, attention allocation, etc.), N100 (a negative potential component, reflecting early sensory processing and attention orientation), etc. The target cognitive component is the component among all cognitive components whose appearance time is closest to the previously determined stable fixation time, and it is considered to be closely related to the brain cognitive activities accompanied by the stable fixation behavior. The cognitive generation time is the specific moment when the target cognitive component appears in the EEG signal time series.

[0091] During the calibration of the electroencephalogram electrooculogram (EEG-EOG) device, the eye movement device determines the stable time of the fixation point by analyzing the eye movement speed (e.g., using the speed threshold method), that is, when the eye movement speed is lower than a pre-set threshold, it is determined that the fixation has become stable. However, limited by the sampling rate of the eye movement device, or in the case of a brief jitter of the eyeball after a saccade, the determined stable fixation time may be delayed. To determine the magnitude of the lag time, based on the above embodiments, when the fixation position enters the area near the calibration point, a stimulus will be sent to the user, such as a change in the calibration point. If the user's attention is on the calibration point at this time and it is determined that the fixation has been completed, a target cognitive component will be generated. Therefore, there may be a time difference between the appearance time of the target cognitive component and the stable fixation time. If the difference between the two is small, it can be considered that the eye movement data is relatively reliable, and the determined stable fixation time can better reflect the actual situation; if the difference is large, it indicates that the eye movement data may deviate from the true fixation point situation due to delay problems or errors in the eye movement analysis algorithm.

[0092] S208, if the difference between the cognitive generation time and the stable fixation time is greater than the third threshold, correct the initial position to obtain the stable fixation point coordinates.

[0093] It can be understood that the third threshold is a pre-set time interval value used to determine whether the difference between the cognitive generation time and the stable fixation time exceeds the acceptable range. It can be determined based on a large amount of experimental data and research on normal visual-cognitive synchrony. When the difference between the two is greater than this threshold, it indicates that there is a significant deviation between the stable fixation time determined by the eye movement data and the occurrence time of the actual cognitive activity of the brain. When the difference between the cognitive generation time and the stable fixation time is greater than the third threshold, this means that the stable fixation time determined based on the eye movement data may be biased, which in turn leads to inaccuracy of the initial position determined based on this. This deviation may be due to the sampling rate limitation of the eye movement device, the eye movement analysis error caused by the brief jitter after a saccade, or other interference factors. By correcting the initial position, these errors can be compensated, so that the finally obtained stable fixation point coordinates are more in line with the fixation position corresponding to the actual cognitive activity of the brain.

[0094] S210, otherwise, use the initial position as the stable fixation coordinates.

[0095] It can be understood that in this step, if the difference between the cognitive generation time and the stable fixation time is not greater than the third threshold, it indicates that the initial position determined based on the eye movement data can more accurately reflect the stable fixation position of the eyeball. Therefore, the initial position is directly used as the stable fixation coordinates for subsequent EEG-EOG device calibration and other operations.

[0096] In one embodiment, please refer to Figure 3, the initial position is corrected to obtain the coordinates of the stable fixation point, including steps S302 to S308.

[0097] S302. Determine the eye movement trajectory between the cognitive generation time and the stable fixation time according to the eye movement data.

[0098] It can be understood that the eye movement trajectory is the path formed by the eye moving in space during the time period between the cognitive generation time and the stable fixation time, usually represented by a series of continuous position coordinates. The eye movement trajectory can reflect the movement path of the eye from the occurrence of cognitive stimulation to the achievement of a stable fixation state. When there is a difference between the cognitive generation time and the stable fixation time, the eye movement trajectory contains information about the adjustment of the eye to achieve stable fixation. By accurately determining the eye movement trajectory during this period, it can provide a key basis for subsequent correction of the initial position according to the degree of difference. Assume that the original eye movement data obtained from the eye tracker is stored in a two-dimensional array, each row represents the eye position at a certain time point, and the cognitive generation time and the stable fixation time are known. First, determine the index range of the eye movement data between these two time points. Then, extract the eye movement position data within this range, and these data points connected together form the eye movement trajectory.

[0099] S304. Determine the correction amplitude according to the ratio of the absolute value of the time difference between the cognitive generation time and the stable fixation time to the third threshold.

[0100] It can be understood that the absolute value of the time difference refers to the absolute value of the difference between the cognitive generation time and the stable fixation time. This value reflects the deviation degree between the stable fixation time determined by the eye movement device and the time when the brain actually generates a corresponding response due to cognitive stimulation. The correction amplitude is a quantified value, which is determined according to the ratio of the absolute value of the time difference to the third threshold, and is used to calculate the degree of correction of the initial position in the follow-up. This amplitude determines the size of the offset of the initial position on the eye movement trajectory, and is an important parameter for accurately correcting the initial position to obtain the coordinates of the stable fixation point. When there is a difference between the cognitive generation time and the stable fixation time, the larger the absolute value of the time difference, the greater the deviation between the stable fixation time determined by the eye movement data and the actual cognitive activity time of the brain, and then the greater the need to correct the initial position. By comparing the absolute value of the time difference with the third threshold and determining the correction amplitude in the form of a ratio, it is possible to dynamically adjust the correction amount according to the deviation degree. Since there are differences in the eye movement characteristics (such as saccade speed, stable fixation maintenance time) of different users and device parameters (such as sampling rate, noise level), the physical meaning of the absolute value of the time difference changes with the scene. By calculating the correction amplitude with the ratio to the third threshold, different magnitudes of time differences can be converted into the relative correction coefficient in the interval, making the correction logic universal.

[0101] S306. Obtain the target correction amount based on the correction amplitude and the reference correction amount.

[0102] It can be understood that the reference correction amount is a preset basic value, which is used to combine with the correction amplitude to calculate the final target correction amount. The target correction amount is a specific value obtained by comprehensively considering the correction amplitude and the reference correction amount. It will be directly used to offset the initial position to obtain more accurate stable fixation point coordinates. The setting of the reference correction amount is usually based on the estimation of the error range of eye movement data and experimental experience. Different application scenarios and devices may require adjustment of this reference value. This step further calculates the specific correction value on the basis of determining the correction amplitude. By multiplying the reference correction amount by the correction amplitude (or combining according to a specific algorithm) to obtain the target correction amount, the general correction requirements (reflected by the reference correction amount) and the specific correction requirements caused by the time difference (reflected by the correction amplitude) can be comprehensively considered.

[0103] S308. Starting from the initial position, offset along the eye movement trajectory according to the target correction amount to obtain the stable fixation point coordinates.

[0104] This step is the final execution link of the entire correction process. Its principle is that the eye movement trajectory can reflect the trend of eye movement, and the target correction amount quantifies the degree of adjustment required for the initial position. Starting from the initial position and offsetting along the eye movement trajectory according to the target correction amount can optimize the initial position according to the actual situation of eye movement and the error direction and degree reflected by the time difference, so as to obtain coordinates closer to the true stable fixation point. For example, if the eye movement trajectory shows a trend towards the upper right and the target correction amount is positive, then offset the initial position towards the upper right by the corresponding distance along this trend. This step closely cooperates with the previous steps, combines the determined eye movement trajectory, the calculated target correction amount and the initial position, and realizes the conversion from the initially estimated stable fixation position to more accurate stable fixation point coordinates, which plays a decisive role in improving the calibration accuracy of the electroencephalogram eye tracker.

[0105] The present application provides a calibration device for an electroencephalogram eye tracker based on a brain-computer interface, including a display module, a collection module, an attention index determination module, a target calibration point determination module, a re-display module, and a calibration module. The display module is used to sequentially display a plurality of calibration points to the user. The collection module is used to respectively and real-time collect the user's eye movement data and electroencephalogram signals by using the electroencephalogram eye tracker. The attention index determination module is used to determine the corresponding attention index according to the electroencephalogram signal for any calibration point. The target calibration point determination module is used to determine the calibration point as the target calibration point if the attention index is lower than the first threshold. The re-display module is used to re-display the target calibration point and update the corresponding eye movement data when the attention index is higher than the first threshold. The calibration module is used to perform calibration according to the eye movement data.

[0106] For the specific limitations of the calibration device of the electroencephalogram eye tracker based on the brain-computer interface, reference can be made to the limitations of the calibration method of the electroencephalogram eye tracker based on the brain-computer interface in the above text, which will not be elaborated here. Each module in the above calibration device of the electroencephalogram eye tracker based on the brain-computer interface can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in the form of hardware or be independent of it, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0107] The present application provides a computer device, including one or more processors and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the steps of the calibration method of the electroencephalogram eye tracker based on the brain-computer interface in any one of the above embodiments are executed.

[0108] The present application provides an electroencephalogram eye tracker based on the brain-computer interface. Please refer to Figure 4 , including a VR glasses main body 10, an eye movement tracking module 20, an electroencephalogram acquisition module 30, and the computer device in any one of the above embodiments. The computer device is respectively connected to the VR glasses main body, the eye movement tracking module 20, and the electroencephalogram acquisition module 30.

[0109] The VR glasses main body 10 is the core basic bearing component of the electroencephalogram eye tracker. It is a virtual reality head-mounted display device designed to build an immersive virtual environment experience for users. The VR glasses main body 10 corresponds two display screens to the human eyes respectively, and uses an optical lens system to process the display screen, so as to present a virtual scene with a sense of depth and three-dimensionality in front of the user's eyes, making the user have a real feeling as if being in the virtual world. The optical imaging system of the VR glasses main body 10 refracts the picture on the display screen through a lens and forms a clear image on the user's retina. The VR glasses main body 10 adopts a head-mounted design to fit the contour of the human head and ensure the comfort and stability of wearing.

[0110] The eye tracking module 20 is a key functional module for real-time collection of user eye movement data. Its main function is to capture information such as the movement trajectory of the user's eyeball, the position of the fixation point, and the change in pupil diameter, providing basic data support for subsequent data analysis and applications. From a principle perspective, the eye tracking module 20 usually adopts an optical-based method, mainly including the corneal reflection method and the pupil-corneal reflection vector method. The corneal reflection method is to set an infrared light source and a camera inside the main body 10 of the VR glasses. The light emitted by the infrared light source is reflected after hitting the corneal surface of the user's eyeball, and the reflected light is captured by the camera. According to the position change of the reflected light on the image sensor of the camera and the known relative position relationship between the light source and the camera, the movement angle and fixation direction of the eyeball are calculated. The pupil-corneal reflection vector method is to simultaneously detect the position of the pupil and the position of the corneal reflection point, and determine the movement state of the eyeball through the vector relationship between the two. A fill light can also be set on the eye tracking module 20. The eye tracking module 20 is set inside the main body 10 of the VR glasses, and its specific position is usually distributed in the area of the glasses main body close to the human eye. Considering the need to accurately capture the movement information of the eyeball, the infrared light source array and the high-frame-rate camera are arranged at specific angles and layouts, generally at the upper and lower edges or left and right edges of the inner frame of the glasses main body. Such a layout can ensure that the light emitted by the infrared light source can effectively irradiate the corneal surface of the eyeball, and at the same time the camera can clearly capture the eyeball image containing the pupil and the corneal reflection point, so as to accurately collect eye movement data.

[0111] The eye tracking module 20 usually consists of an infrared light source array, a high-frame-rate camera, an image signal processing chip, and related algorithm software. The infrared light source array generally uses multiple near-infrared light-emitting diodes (LEDs). The near-infrared light emitted by them is almost invisible to the human eye and will not cause harm to the human eye. At the same time, it can effectively enhance the intensity of the corneal reflection signal and improve the detection accuracy. The high-frame-rate camera is responsible for taking eyeball images at a relatively high frequency (usually above 100Hz) to ensure that the rapid movement details of the eyeball can be captured. The image signal processing chip performs real-time processing on the images collected by the camera, including operations such as image enhancement, noise reduction, and feature extraction, and extracts key features such as the pupil and the corneal reflection point. The related algorithm software, based on these features, uses complex mathematical models and calculation methods to accurately calculate the movement parameters and fixation point coordinates of the eyeball.

[0112] The electroencephalogram (EEG) acquisition module 30 is an important part of brain-computer interface technology and is the core component for collecting the user's EEG signals. EEG signals are bioelectric signals generated during the activities of brain neurons. They contain rich information about brain activities and can reflect various aspects of human psychological states, cognitive processes, and physiological functions. Based on the generation and conduction mechanisms of bioelectric signals, when brain neurons are in the processes of excitation and inhibition, ion flows occur, resulting in weak potential changes outside the cells. These potential changes are conducted through tissues such as the scalp and skull to the scalp surface, forming detectable EEG signals. The EEG acquisition module 30 contacts the head skin through electrodes, collects and amplifies these weak EEG signals, and then through processing such as filtering and analog-to-digital conversion, converts them into digital signals that can be recognized and processed by a computer.

[0113] The EEG acquisition module 30 mainly consists of electrodes, a signal amplifier, a filter, an analog-to-digital converter, and a data transmission module, etc. The electrodes are the components that directly contact the skin and should have good electrical conductivity and stability to effectively collect EEG signals. The signal amplifier is used to amplify weak EEG signals and usually has a high gain and common-mode rejection ratio to improve the signal-to-noise ratio of the signals; the filter is used to remove noise and interference components in the EEG signals, such as power frequency interference, electromyogram interference, etc.; the analog-to-digital converter converts the amplified and filtered analog EEG signals into digital signals; the data transmission module transmits the digital EEG signals to a computer device for subsequent analysis and processing. The EEG acquisition module 30 is also arranged inside the VR glasses main body 10, and its electrode part needs to contact the scalp to collect EEG signals. Therefore, the electrodes are usually distributed in the key areas where the VR glasses main body 10 contacts the head, such as near the forehead and temples. These positions can better acquire EEG signals generated in different regions of the brain. The layout of the electrodes will be designed according to the requirements of EEG signal acquisition.

[0114] This application provides a storage medium in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the calibration method of the EEG eye tracker in any of the above embodiments.

[0115] This application provides a storage medium in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the calibration method of the EEG eye tracker in any of the above embodiments.

[0116] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0118] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A calibration method for an electroencephalogram electrooculogram instrument based on a brain-computer interface, characterized in that, Including: Sequentially presenting multiple calibration points to the user; Using the electroencephalogram (EEG) and electrooculogram (EOG) instrument to respectively and real-time collect the user's eye movement data and EEG signals; For any one of the calibration points, determining a corresponding attention index according to the EEG signal; If the attention index is lower than a first threshold, determining the calibration point as a target calibration point; Redisplaying the target calibration point and updating the corresponding eye movement data when the attention index is higher than the first threshold; Performing calibration according to the eye movement data.

2. The calibration method according to claim 1, wherein The determining the corresponding attention index according to the EEG signal includes: Determine according to the electroencephalogram signal the wave power spectral density, and determine the attention index according to the wave power spectral density.

3. The calibration method according to claim 1, wherein The redisplaying the target calibration point includes: Determining an extension ratio according to the fluctuation degree of the attention index; the greater the fluctuation degree, the greater the extension ratio; Extending the display time of the target calibration point according to the extension ratio.

4. The calibration method according to claim 1, wherein The performing calibration according to the eye movement data further includes: For any one of the calibration points, determining a stable fixation point coordinate according to the eye movement data; Performing calibration according to the difference between the stable fixation point coordinate and the set coordinate corresponding to the calibration point.

5. The calibration method according to claim 4, characterized in that During the process of sequentially presenting multiple calibration points to the user, it further includes: When the fixation position enters the set area range corresponding to the calibration point, sending a cognitive trigger stimulus to the user.

6. The calibration method according to claim 5, characterized in that, The for any one of the calibration points, determining a stable fixation point coordinate according to the eye movement data includes: Determining an eye movement speed according to the eye movement data; Determining the time point when the eye movement speed is lower than a second threshold as the stable fixation time, and determining the fixation position when the eye movement speed is lower than the second threshold as the initial position; Determining the cognitive generation time of a target cognitive component in the EEG signal; the target cognitive component is the cognitive component with the closest occurrence time to the stable fixation time, and the cognitive component is the EEG component generated when a cognitive stimulus appears; If the difference between the cognitive generation time and the stable fixation time is greater than a third threshold, correcting the initial position to obtain the stable fixation point coordinate; Otherwise, using the initial position as the stable fixation coordinate.

7. The calibration method according to claim 6, wherein The correcting the initial position to obtain the stable fixation point coordinate includes: Determining an eye movement trajectory between the cognitive generation time and the stable fixation time according to the eye movement data; Determining a correction amplitude according to the ratio of the absolute value of the time difference between the cognitive generation time and the stable fixation time to the third threshold; Obtaining a target correction amount according to the correction amplitude and a reference correction amount; Starting from the initial position, offsetting along the eye movement trajectory according to the target correction amount to obtain the stable fixation point coordinate.

8. A calibration device for an electroencephalogram electrooculogram based on a brain-computer interface, characterized in that, Including: A display module for sequentially presenting multiple calibration points to the user; An acquisition module for using the EEG and EOG instrument to respectively and real-time collect the user's eye movement data and EEG signals; An attention index determination module for, for any one of the calibration points, determining a corresponding attention index according to the EEG signal; If the attention index is lower than a first threshold, determining the calibration point as a target calibration point; A re - display module, configured to re - display the target calibration point and update the corresponding eye movement data when the attention index is higher than the first threshold; A calibration module, configured to perform calibration according to the eye movement data.

9. A computer device, characterized in that, Comprising one or more processors and a memory, wherein computer - readable instructions are stored in the memory, and when the computer - readable instructions are executed by the one or more processors, the steps of the calibration method of the electroencephalogram - eye tracker according to any one of claims 1 - 7 are executed.

10. An electroencephalogram eye tracker based on a brain-computer interface, characterized in that, Comprising: The VR glasses main body; An eye movement tracking module, disposed inside the VR glasses main body, for collecting the user's eye movement data; An electroencephalogram acquisition module, disposed inside the VR glasses main body, for collecting the user's electroencephalogram signals; The computer device according to claim 9, which is respectively connected to the VR glasses main body, the eye movement tracking module, and the electroencephalogram acquisition module.

11. A storage medium, characterized in that, Computer - readable instructions are stored in the storage medium, and when the computer - readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the calibration method of the electroencephalogram - eye tracker according to any one of claims 1 - 7.

Images