Eye tracker assisted SSVEP frequency comparison method and device

Abstract

The invention discloses an eye tracker assisted SSVEP frequency comparison method and device. The method comprises the following steps: acquiring first image information, orbit region information and fixation point region information; the gaze point area information comprises first gaze point area information, second gaze point area information, third gaze point area information and fourth gaze point area information; processing the first image information and the orbit region information to obtain fixation point coordinate information; and carrying out fusion processing on the fixation point coordinate information and the fixation point region information to obtain a target frequency. Therefore, according to the method and the device, the watching direction of the testee is calibrated through the eye tracker, and the stimulation frequency comparison range is narrowed in real time, so that the accuracy of SSVEP is improved.

Description

Technical Field

[0001] The present invention relates to the field of SSVEP, and in particular to an eye tracker-assisted SSVEP frequency comparison method and device. Background Art

[0002] In daily life or work scenarios, fewer SSVEP stimuli cannot meet people's needs. However, as the SSVEP stimulus sources become smaller or their number increases, the occipital lobe area of ​​the subject will not be able to produce strong resonance, which makes it difficult to accurately compare the subject's measured EEG frequency with the observed stimulation point frequency, thereby limiting the performance and practical application of SSVEP under multi-task conditions. To solve this problem, more and more researchers are focusing on SSVEP decoding and encoding algorithms. However, no researchers have considered combining eye trackers with SSVEP to partition the stimulation interface in real time to improve the accuracy of SSVEP under multi-stimulus conditions. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an eye tracker-assisted SSVEP frequency comparison method and device, which calibrates the subject's gaze direction through the eye tracker and narrows the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present invention discloses an eye tracker-assisted SSVEP frequency comparison method, the method comprising:

[0005] S1, obtaining first image information, orbital region information and gaze point region information; the gaze point region information includes first gaze point region information, second gaze point region information, third gaze point region information and fourth gaze point region information;

[0006] S2, processing the first image information and the orbital area information to obtain gaze point coordinate information;

[0007] S3, fusing the gaze point coordinate information and the gaze point area information to obtain a target frequency.

[0008] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the first image information and the orbital region information to obtain gaze point coordinate information includes:

[0009] S21, processing the first image information and the orbital area information to obtain corneal center coordinate information and a visual axis vector;

[0010] S22, normalizing the visual axis vector to obtain a visual axis unit vector;

[0011] S23, using an eye movement fixation point calculation model, calculating and processing the corneal center coordinate information and the visual axis unit vector to obtain fixation point coordinate information;

[0012] The eye movement fixation point calculation model is:

[0013] ZSD=JM+μ·SZ+∈;

[0014] Wherein, ZSD is the coordinate information of the gaze point, JM is the coordinate information of the corneal center, SZ is the visual axis unit vector, μ is the visual axis gaze point distance, and ∈ is the first offset vector.

[0015] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the fusing the gaze point coordinate information and the gaze point area information to obtain the target frequency includes:

[0016] S31, processing the gaze point coordinate information and the gaze point area information to obtain SSVEP frequency information and stimulation frequency information; the stimulation frequency information includes a plurality of stimulation frequencies;

[0017] S32: Analyze and process the SSVEP frequency information and the stimulation frequency information to obtain a target frequency.

[0018] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the gaze point coordinate information and the gaze point area information to obtain SSVEP frequency information and stimulation frequency information includes:

[0019] S311, processing the gaze point coordinate information and the gaze point area information to obtain information of the area to be processed;

[0020] S312, processing the to-be-processed region information to obtain stimulation region information and stimulation frequency information; the stimulation region information includes a plurality of stimulation frequency regions;

[0021] S313: Perform visual stimulation processing using the stimulation area information and the stimulation frequency information to obtain SSVEP frequency information.

[0022] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the gaze point coordinate information and the gaze point area information to obtain the area information to be processed includes:

[0023] Determine whether the X-axis coordinate value of the gaze point coordinate information is greater than 0, and obtain a first determination result;

[0024] When the first judgment result is yes, determining whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, and obtaining a second judgment result;

[0025] When the second judgment result is yes, determining the first gaze point area information as area information to be processed;

[0026] When the second judgment result is no, determining the third gaze point area information as the area information to be processed;

[0027] When the first judgment result is no, determining whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, to obtain a third judgment result;

[0028] When the third judgment result is yes, determining the second gaze point area information as the area information to be processed;

[0029] When the third judgment result is no, the fourth gaze point area information is determined to be the area information to be processed.

[0030] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the analyzing and processing the SSVEP frequency information and the stimulation frequency information to obtain the target frequency includes:

[0031] S321, analyzing and processing the SSVEP frequency information to obtain sub-band component information; the sub-band component information includes a plurality of sub-band components;

[0032] S322: Calculate and process the sub-band component information and the stimulation frequency information to obtain a target frequency.

[0033] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the calculating and processing the sub-band component information and the stimulation frequency information to obtain the target frequency includes:

[0034] S3221, performing calculation processing on the stimulation frequency information to obtain sine and cosine reference information; the sine and cosine reference information includes a plurality of sine and cosine reference template information; the stimulation frequency in the stimulation frequency information corresponds one-to-one to the sine and cosine reference template information in the sine and cosine reference information;

[0035] S3222: Using a first SSVEP calculation model, perform calculation processing on the subband component information and the sine and cosine reference information to obtain correlation coefficient matrix information; the correlation coefficient matrix information includes a plurality of correlation coefficient matrices;

[0036] Wherein, the first SSVEP calculation model is:

[0037]

[0038] Where, XG is the correlation coefficient matrix information, XG j is the j-th correlation coefficient matrix in the correlation coefficient matrix information, is the transpose of the ith sub-band component in the sub-band component information, SBT i is the projection of the i-th sub-band component in the sub-band component information, is the transpose of the jth sine-cosine reference template information in the sine-cosine reference information, ZYT j is the projection of the j-th sin-cosine reference template information in the sin-cosine reference information, N is the number of the sub-band components in the sub-band component information, M is the number of the sin-cosine reference template information in the sin-cosine reference information, and ρ represents a correlation coefficient;

[0039] S3223, using a second SSVEP calculation model, performing calculation processing on the correlation coefficient matrix information to obtain SSVEP correlation coefficient information; the SSVEP correlation coefficient information includes a plurality of SSVEP correlation coefficients;

[0040] Among them, the second SSVEP calculation model is:

[0041]

[0042] Where, is the SSVEP correlation coefficient information, is the jth SSVEP correlation coefficient in the SSVEP correlation coefficient information, XG is the correlation coefficient matrix information, XG j is the j-th correlation coefficient matrix in the correlation coefficient matrix information, δ1 and δ2 are respectively a first weight parameter and a second weight parameter, o is a smoothing index, a and b are first constant parameters and a second constant parameter, N is the number of subband components in the subband component information, and M is the number of sin-cosine reference template information in the sin-cosine reference information;

[0043] S3224 , determining a target frequency based on the SSVEP correlation coefficient information; the target frequency is the stimulation frequency corresponding to the largest SSVEP correlation coefficient in the SSVEP correlation coefficient information.

[0044] A second aspect of an embodiment of the present invention discloses an eye tracker-assisted SSVEP frequency comparison device, the device comprising:

[0045] An acquisition module is used to acquire first image information, orbital region information, and gaze point region information; the gaze point region information includes first gaze point region information, second gaze point region information, third gaze point region information, and fourth gaze point region information;

[0046] a first calculation module, configured to process the first image information and the orbital region information to obtain gaze point coordinate information;

[0047] The second calculation module is used to fuse the gaze point coordinate information and the gaze point area information to obtain a target frequency.

[0048] A third aspect of an embodiment of the present invention discloses another eye tracker-assisted SSVEP frequency comparison device, the device comprising:

[0049] processor;

[0050] a memory coupled to the processor and storing executable program code;

[0051] The processor calls the executable program code stored in the memory to execute part or all of the steps of the eye tracker-assisted SSVEP frequency comparison method disclosed in the first aspect of the embodiment of the present invention.

[0052] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all steps of the eye tracker-assisted SSVEP frequency comparison method disclosed in the first aspect of the embodiment of the present invention.

[0053] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0054] In an embodiment of the present invention, first image information, orbital region information, and fixation region information are obtained; the fixation region information includes first fixation region information, second fixation region information, third fixation region information, and fourth fixation region information; the first image information and the orbital region information are processed to obtain fixation coordinate information; and the fixation coordinate information and the fixation region information are fused to obtain a target frequency. This shows that the present application calibrates the subject's gaze direction using an eye tracker and narrows the stimulus frequency comparison range in real time, which is beneficial to improving the accuracy of SSVEP. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 Schematic diagram of the flow of an eye tracker-assisted SSVEP frequency comparison method disclosed in an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of information about the eye orbital region where the pupil is located using an eye tracker disclosed in an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the partitions corresponding to the gaze point area information and the eye socket area information disclosed in an embodiment of the present invention;

[0059] Figure 4 This is a structural diagram of an eye tracker-assisted SSVEP frequency comparison device disclosed in an embodiment of the present invention;

[0060] Figure 5 This is a structural diagram of another eye tracker-assisted SSVEP frequency comparison device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0063] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] This invention discloses an eye tracker-assisted SSVEP frequency comparison method and device. This method uses an eye tracker to calibrate the subject's gaze direction and narrow the stimulus frequency comparison range in real time, thereby improving the accuracy of the SSVEP. These are described in detail below.

[0065] Example 1

[0066] See also Figure 1 , Figure 1 : This is a flow chart of an eye tracker-assisted SSVEP frequency comparison method disclosed in an embodiment of the present invention. Figure 1 The described eye tracker-assisted SSVEP frequency comparison method is applied to an eye tracker-assisted SSVEP frequency comparison device, such as a local server or cloud server for optimizing and managing eye tracker-assisted SSVEP frequency comparison, and the present invention does not limit this. Figure 1 As shown, the eye tracker-assisted SSVEP frequency comparison method may include the following operations:

[0067] S1, obtaining first image information, orbital region information and gaze point region information; the gaze point region information includes first gaze point region information, second gaze point region information, third gaze point region information and fourth gaze point region information;

[0068] S2, processing the first image information and the orbital area information to obtain gaze point coordinate information;

[0069] S3, fusing the gaze point coordinate information and the gaze point area information to obtain the target frequency.

[0070] It should be noted that the first image information is an image of the user's eyes captured in real time by the eye tracker's camera. These images usually contain important features such as pupils and corneal reflection points.

[0071] It should be noted that the orbital region information includes the first orbital region information, the second orbital region information, the third orbital region information and the fourth orbital region information. Figure 2 , the first orbital area information is Figure 2 In the upper right area, the second orbital area information is Figure 2 The upper left area in the third orbital area is Figure 2In the lower right area, the fourth orbital area information is Figure 2 The lower left area in

[0072] It should be noted that, see Figure 3 , the fixation area information is the stimulation interface of SSVEP, and the first fixation area information in the fixation area information is Figure 3 The upper right area in the second gaze point area is Figure 3 The upper left area in the third gaze point area is Figure 3 The lower right area in the fourth gaze point area is Figure 3 The upper left area of ​​.

[0073] It should be noted that, in an embodiment of the present invention, the first orbital area information, the second orbital area information, the third orbital area information and the fourth orbital area information in the orbital area information correspond one to one with the first gaze point area information, the second gaze point area information, the third gaze point area information and the fourth gaze point area information in the gaze point area information, respectively. When it is detected that the pupil position is in a certain orbital area information in the orbital area information (such as the first orbital area information), when the user's vision is subsequently stimulated, the corresponding gaze point area information in the gaze point area information (such as the first gaze point area information) is selected for stimulation to obtain SSVEP frequency information.

[0074] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0075] In an optional embodiment, the first image information and the orbital region information are processed to obtain gaze point coordinate information, including:

[0076] S21, processing the first image information and the orbital area information to obtain corneal center coordinate information and a visual axis vector;

[0077] S22, normalizing the visual axis vector to obtain a visual axis unit vector;

[0078] S23, using an eye movement fixation point calculation model, calculating and processing the corneal center coordinate information and the visual axis unit vector to obtain fixation point coordinate information;

[0079] Among them, the eye movement fixation point calculation model is:

[0080] ZSD=JM+μ·SZ+∈;

[0081] Where ZSD is the coordinate information of the gaze point, JM is the coordinate information of the corneal center, SZ is the visual axis unit vector, μ is the visual axis gaze point distance, and ∈ is the first offset vector.

[0082] It should be noted that the first offset vector is set by the user, and may also be obtained based on historical data, which is not specifically limited in the embodiment of the present invention.

[0083] It should be noted that the visual axis gaze point distance is the distance between the corneal center coordinate information and the gaze point being gazed. The visual axis gaze point distance is set by the user and can also be measured in the on-site environment. The embodiment of the present invention does not make any specific limitations.

[0084] It should be noted that processing the first image information and the orbital region information to obtain the corneal center coordinate information and the visual axis vector is performed using an eye tracker and a gaze tracking model to process the first image information and the orbital region information to obtain the corneal center coordinate information and the visual axis vector. The specific details are not limited in this embodiment of the present invention. Processing the first image information and the orbital region information using an eye tracker and a gaze tracking model can not only obtain the corneal center coordinate information and the visual axis vector, but also obtain the optical axis vector.

[0085] It should be noted that, see Figure 2 The coordinate system in this application is a three-dimensional coordinate system. The three-dimensional coordinate system in this application takes the intersection of the first orbital area information, the second orbital area information, the third orbital area information and the fourth orbital area information as the origin, and the horizontal axis ( Figure 2 The intersection of the upper right area and the lower right area in the figure) is taken from right to left as the X axis, and the vertical axis ( Figure 2 The intersection of the upper right area and the upper left area in the image is constructed from bottom to top as the Y axis, and the direction of the eye pointing to the gaze point (and perpendicular to the plane formed by the XY axis) as the Z axis.

[0086] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0087] In another optional embodiment, the first image information and the orbital region information are processed to obtain gaze point coordinate information, including:

[0088] S210, processing the first image information and the orbital area information to obtain corneal center coordinate information, a visual axis vector, and an optical axis vector;

[0089] It should be noted that the first image information and the orbital area information are processed to obtain the corneal center coordinate information, the visual axis vector and the optical axis vector. The first image information and the orbital area information are processed using an eye tracker and a gaze tracking model to obtain the corneal center coordinate information, the visual axis vector and the optical axis vector. Specifically, the embodiments of the present invention do not limit this.

[0090] S220, normalizing the visual axis vector and the optical axis vector to obtain a visual axis unit vector and an optical axis unit vector;

[0091] It should be noted that the visual axis vector and the optical axis vector are normalized respectively to obtain the visual axis unit vector and the optical axis unit vector, which are obtained by normalizing the visual axis normalization calculation model;

[0092] Among them, the normalized calculation model of the visual axis is:

[0093]

[0094]

[0095] Where SZ′ is the visual axis unit vector, SZX′, SZY′, and SZZ′ are the X-axis coordinate values, Y-axis coordinate values, and Z-axis coordinate values ​​of the visual axis vector, respectively; GZ′ is the optical axis unit vector, GZX′, GZY′, and GZZ′ are the X-axis coordinate values, Y-axis coordinate values, and Z-axis coordinate values ​​of the optical axis vector, respectively;

[0096] S230, using a gaze point calculation model, calculating and processing the corneal center coordinate information, the visual axis unit vector, and the optical axis unit vector to obtain gaze point coordinate information;

[0097] Among them, the gaze point calculation model is:

[0098] ZSD′=JM′+μ′·(α·SZ′+(1-α)·GZ′);

[0099] 0≤α≤1;

[0100] Where ZSD′ is the coordinate information of the gaze point, JM′ is the coordinate information of the corneal center, SZ′ is the visual axis unit vector, GZ′ is the optical axis unit vector, μ′ is the distance of the visual axis gaze point, and α is the weight factor.

[0101] It should be noted that the weight factor is set by the user or obtained based on historical data, and the embodiment of the present invention does not limit this.

[0102] It should be noted that the visual axis gaze point distance is the distance between the corneal center coordinate information and the gaze point being gazed. The visual axis gaze point distance is set by the user and can also be measured in the on-site environment. The embodiment of the present invention does not make any specific limitations.

[0103] It should be noted that the optical axis vector can capture more real biological information in the visual environment and can resist the influence of noise and other interference factors on the gaze point calculation. Therefore, by combining the visual axis vector and the optical axis vector, the accuracy and reliability of the gaze point coordinate information can be further improved.

[0104] It should be noted that the visual axis vector is a vector pointing from the corneal center coordinate information to the gaze point, and the optical axis vector refers to a vector passing through the optical center of the eyeball (usually located at the back of the lens) and the corneal center.

[0105] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0106] In another optional embodiment, normalizing the viewing axis vector to obtain the viewing axis unit vector includes:

[0107] Using the visual axis normalization calculation model, the visual axis vector is normalized to obtain the visual axis unit vector;

[0108] Among them, the visual axis normalization calculation model is:

[0109]

[0110] Where SZ is the viewing axis unit vector, SZX, SZY, and SZZ are the X-axis, Y-axis, and Z-axis coordinate values ​​of the viewing axis vector, respectively, and SZM is the modulus of the viewing axis vector.

[0111] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0112] In another optional embodiment, the gaze point coordinate information and the gaze point area information are fused to obtain the target frequency, including:

[0113] S31, processing the gaze point coordinate information and the gaze point area information to obtain SSVEP frequency information and stimulation frequency information; the stimulation frequency information includes a plurality of stimulation frequencies;

[0114] S32, analyzing and processing the SSVEP frequency information and the stimulation frequency information to obtain the target frequency.

[0115] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0116] In yet another optional embodiment, the gaze point coordinate information and the gaze point area information are processed to obtain SSVEP frequency information and stimulation frequency information, including:

[0117] S311, processing the gaze point coordinate information and the gaze point area information to obtain information of the area to be processed;

[0118] S312, processing the area information to be processed to obtain stimulation area information and stimulation frequency information; the stimulation area information includes several stimulation frequency areas;

[0119] It should be noted that the processing of the area information to be processed obtains the stimulation area information and stimulation frequency information; the stimulation area information includes several stimulation frequency areas: the area information to be processed is divided into several equidistant small areas, each small area is a stimulation area information, each small area corresponds to a stimulation frequency, and the stimulation frequency difference between each small area is 4L HZ, where L is a positive integer. Figure 3 The upper right area is the information of the area to be processed. The upper right area is divided into 4 small areas, where the stimulation frequencies of the 4 small areas are 11HZ, 15HZ, 19HZ and 23HZ, and the stimulation frequency difference between the small areas is 4HZ.

[0120] It should be noted that the stimulation frequency is the frequency of the stimulation area information when stimulating vision, such as using a light source with a flashing frequency of 11 Hz in a small area with a stimulation frequency of 11 Hz. The specific stimulation frequency is set by the user or obtained based on historical data, and is not limited in the embodiments of the present invention.

[0121] S313, using the stimulation area information and the stimulation frequency information, performing visual stimulation processing to obtain SSVEP frequency information.

[0122] It should be noted that the stimulation area information and the stimulation frequency information are used to stimulate the vision and obtain the SSVEP frequency information. Signal stimulation is performed alternately on each stimulation area information according to the stimulation frequency corresponding to the stimulation area information. For example, a light source with a corresponding stimulation frequency is used. When the signal stimulation is performed, the user's vision is always fixed on the stimulation area information. At this time, the EEG device is used to collect the user's SSVEP frequency signal, and the collected SSVEP frequency signal is the SSVEP frequency information.

[0123] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0124] In an optional embodiment, processing the gaze point coordinate information and the gaze point area information to obtain the area information to be processed includes:

[0125] Determine whether the X-axis coordinate value of the gaze point coordinate information is greater than 0, and obtain a first determination result;

[0126] When the first judgment result is yes, determining whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, and obtaining a second judgment result;

[0127] When the second judgment result is yes, determining the first gaze point area information as the area information to be processed;

[0128] When the second judgment result is no, determining the third gaze point area information as the area information to be processed;

[0129] When the first judgment result is no, determining whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, to obtain a third judgment result;

[0130] When the third judgment result is yes, determining the second gaze point area information as the area information to be processed;

[0131] When the third judgment result is no, the fourth gaze point area information is determined to be the area information to be processed.

[0132] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0133] In an optional embodiment, analyzing and processing the SSVEP frequency information and the stimulation frequency information to obtain the target frequency includes:

[0134] S321, analyzing and processing the SSVEP frequency information to obtain sub-band component information; the sub-band component information includes a plurality of sub-band components;

[0135] S322: Calculate and process the sub-band component information and the stimulation frequency information to obtain a target frequency.

[0136] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0137] In an optional embodiment, the SSVEP frequency information is analyzed and processed to obtain subband component information, including:

[0138] S3211, preprocessing the SSVEP frequency information to obtain preprocessed SSVEP frequency information;

[0139] S3212: Process the pre-processed SSVEP frequency information to obtain sub-band component information.

[0140] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0141] In an optional embodiment, preprocessing the SSVEP frequency information to obtain preprocessed SSVEP frequency information includes:

[0142] S32111, using a bandpass filter, performing signal denoising processing on the SSVEP frequency information to obtain first frequency information;

[0143] It should be noted that the use of a bandpass filter can improve the signal-to-noise ratio (SNR) of a signal by filtering out unnecessary frequency components, thereby making subsequent processing or analysis more accurate.

[0144] S32112, reconstructing the first frequency information using an independent component analysis model to obtain second frequency information;

[0145] It should be noted that the independent component analysis model is a FastICA function, which is used to extract independent components from multivariate signals.

[0146] It should be noted that the independent component analysis model can identify and remove artifacts (such as eye movement artifacts and electromyography artifacts), thereby improving the cleanliness of the data and the reliability of the analysis results.

[0147] S32113: Process the second frequency information using an adaptive noise cancellation model to obtain pre-processed SSVEP frequency information.

[0148] It should be noted that the adaptive noise cancellation model is adaptive noise cancellation ANC, which is a signal processing technology designed to separate the original signal from the noise-contaminated signal.

[0149] It should be noted that the use of an adaptive noise cancellation model can monitor and cancel environmental noise in real time, and adaptively adjust the filter to cope with changing noise characteristics, thereby maintaining signal clarity.

[0150] It should be noted that the use of bandpass filters (BPFs), independent component analysis (ICA), and adaptive noise cancellation (ANC) can significantly improve the overall signal processing effect. This integrated approach can effectively enhance the signal-to-noise ratio, improve signal quality, and make the separation and processing of complex signals more precise.

[0151] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0152] In an optional embodiment, the pre-processed SSVEP frequency information is processed to obtain sub-band component information, including:

[0153] S32121, using a wavelet decomposition function, calculating and processing the preprocessed SSVEP frequency information to obtain preprocessed subband component information; the preprocessed subband component information includes a plurality of preprocessed subband components;

[0154] S32122, using a filter bank decomposition function, processing the preprocessed subband component information to obtain subband component information;

[0155] It should be noted that the combination of wavelet and filter bank decomposition described above allows for more effective analysis of the multi-level characteristics of preprocessed SSVEP frequency information. Wavelet decomposition allows for the extraction of signal features at different time scales, while filter bank decomposition allows for the refinement of specific frequency ranges, enhancing the ability to detect SSVEP responses. Combining these two methods can improve the accuracy and reliability of the analysis.

[0156] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0157] In an optional embodiment, calculating and processing the sub-band component information and the stimulation frequency information to obtain the target frequency includes:

[0158] S3221, calculating and processing the stimulation frequency information to obtain sine and cosine reference information; the sine and cosine reference information includes a plurality of sine and cosine reference template information; the stimulation frequency in the stimulation frequency information corresponds one to one with the sine and cosine reference template information in the sine and cosine reference information;

[0159] It should be noted that the above calculation and processing of the stimulation frequency information to obtain the sine and cosine reference information is to calculate and process each stimulation frequency in the stimulation frequency information to obtain the sine and cosine reference template information corresponding to the stimulation frequency;

[0160] It should be noted that each stimulation frequency in the stimulation frequency information is calculated and processed to obtain the sine and cosine reference template information corresponding to the stimulation frequency. This can be obtained by calculation and processing using built-in functions in MATLAB or the CCA algorithm. Specifically, this embodiment of the present invention does not limit this.

[0161] S3222: Using a first SSVEP calculation model, perform calculations on the subband component information and the sine and cosine reference information to obtain correlation coefficient matrix information; the correlation coefficient matrix information includes a plurality of correlation coefficient matrices;

[0162] Among them, the first SSVEP calculation model is:

[0163]

[0164] Where, XG is the correlation coefficient matrix information, XG j is the jth correlation coefficient matrix in the correlation coefficient matrix information, is the transpose of the i-th subband component in the subband component information, SBT i is the projection of the i-th subband component in the subband component information, is the transpose of the jth sine-cosine reference template information in the sine-cosine reference information, ZYT j is the projection of the j-th sine-cosine reference template information in the sine-cosine reference information, N is the number of subband components in the subband component information, M is the number of sine-cosine reference template information in the sine-cosine reference information, and ρ represents the correlation coefficient;

[0165] It should be noted that ρ represents the correlation coefficient, such as express SBT i and The correlation coefficient is calculated as follows:

[0166]

[0167] Where, express and The correlation coefficient of express and The covariance of LA and LB are respectively and The standard deviation of .

[0168] S3223, using the second SSVEP calculation model, calculating and processing the correlation coefficient matrix information to obtain SSVEP correlation coefficient information; the SSVEP correlation coefficient information includes a plurality of SSVEP correlation coefficients;

[0169] Among them, the second SSVEP calculation model is:

[0170]

[0171] Where, is the SSVEP correlation coefficient information, is the jth SSVEP correlation coefficient in the SSVEP correlation coefficient information, XG is the correlation coefficient matrix information, XG j is the j-th correlation coefficient matrix in the correlation coefficient matrix information, δ1 and δ2 are the first weight parameter and the second weight parameter respectively, o is the smoothing index, a and b are the first constant parameter and the second constant parameter, N is the number of subband components in the subband component information, and M is the number of sine and cosine reference template information in the sine and cosine reference information;

[0172] It should be noted that the first weight parameter, the second weight parameter and the smoothing index are set by the user, or are obtained based on historical data, and are not limited in this embodiment of the present invention. For example, o is 2.

[0173] It should be noted that the first constant parameter and the second constant parameter are set by the user or obtained based on historical data, and the embodiment of the present invention does not limit this.

[0174] S3224 , determining a target frequency based on the SSVEP correlation coefficient information; the target frequency is the stimulation frequency corresponding to the largest SSVEP correlation coefficient in the SSVEP correlation coefficient information.

[0175] It can be seen that the eye tracker-assisted SSVEP frequency comparison method described in the embodiment of the present invention is implemented to calibrate the subject's gaze direction through the eye tracker and narrow the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0176] Example 2

[0177] See also Figure 4, Figure 4 This is a schematic diagram of the structure of an eye tracker-assisted SSVEP frequency comparison device disclosed in an embodiment of the present invention. Figure 4 The described eye tracker-assisted SSVEP frequency comparison device is applied to an eye tracker-assisted SSVEP frequency comparison optimization system, such as a local server or cloud server for eye tracker-assisted SSVEP frequency comparison, which is not limited in the embodiment of the present invention. Figure 4 As shown, the eye tracker-assisted SSVEP frequency comparison device includes:

[0178] An acquisition module 201 is configured to acquire first image information, orbital region information, and gaze point region information; the gaze point region information includes first gaze point region information, second gaze point region information, third gaze point region information, and fourth gaze point region information;

[0179] A first calculation module 202 is used to process the first image information and the orbital region information to obtain gaze point coordinate information;

[0180] The second calculation module 203 is used to fuse the gaze point coordinate information and the gaze point area information to obtain a target frequency.

[0181] It can be seen that the eye tracker-assisted SSVEP frequency comparison device described in the embodiment of the present invention calibrates the subject's gaze direction through the eye tracker and narrows the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0182] Example 3

[0183] See also Figure 5 , Figure 5 : is a structural diagram of another eye tracker-assisted SSVEP frequency comparison device disclosed in an embodiment of the present invention. Figure 5 The described eye tracker-assisted SSVEP frequency comparison device is applied to an eye tracker-assisted SSVEP frequency comparison optimization system, such as a local server or cloud server for eye tracker-assisted SSVEP frequency comparison, which is not limited in the embodiment of the present invention. Figure 5 As shown, the eye tracker-assisted SSVEP frequency comparison device includes:

[0184] Processor 301;

[0185] A memory 302 coupled to the processor 301 and storing executable program code;

[0186] The processor 301 calls the executable program code stored in the memory 302 to execute part or all of the steps of the eye tracker-assisted SSVEP frequency comparison method of the first embodiment.

[0187] It can be seen that the eye tracker-assisted SSVEP frequency comparison device described in the embodiment of the present invention calibrates the subject's gaze direction through the eye tracker and narrows the stimulus frequency comparison range in real time, which is conducive to improving the accuracy of SSVEP.

[0188] Example 4

[0189] An embodiment of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all steps of the eye tracker-assisted SSVEP frequency comparison method of embodiment 1.

[0190] Example 5

[0191] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps in the eye tracker-assisted SSVEP frequency comparison method described in Example 1.

[0192] The system embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0193] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0194] Finally, it should be noted that the eye tracker-assisted SSVEP frequency comparison method and device disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An eye tracker-assisted SSVEP frequency comparison method, characterized in that: The method comprises: S1, obtaining first image information, eye socket area information and gaze point area information; the gaze point area information includes first gaze point area information, second gaze point area information, third gaze point area information and fourth gaze point area information; S2, processing the first image information and the orbital area information to obtain gaze point coordinate information; S3, fusing the gaze point coordinate information and the gaze point area information to obtain a target frequency.

2. The eye tracker-assisted SSVEP frequency comparison method according to claim 1, characterized in that: The processing of the first image information and the orbital region information to obtain the gaze point coordinate information includes: S21, processing the first image information and the orbital area information to obtain corneal center coordinate information and a visual axis vector; S22, normalizing the visual axis vector to obtain a visual axis unit vector; S23, using an eye movement fixation point calculation model, calculating and processing the corneal center coordinate information and the visual axis unit vector to obtain fixation point coordinate information; Wherein, the eye movement fixation point calculation model is: ZSD=JM+μ·SZ+∈; Wherein, ZSD is the gaze point coordinate information, JM is the corneal center coordinate information, SZ is the visual axis unit vector, μ is the visual axis gaze point distance, and ∈ is the first offset vector.

3. The eye tracker-assisted SSVEP frequency comparison method according to claim 1, characterized in that: The fusing the gaze point coordinate information and the gaze point area information to obtain a target frequency includes: S31, processing the fixation point coordinate information and the fixation point region information to obtain SSVEP frequency information and stimulation frequency information; the stimulation frequency information includes a plurality of stimulation frequencies; S32, analyzing and processing the SSVEP frequency information and the stimulation frequency information to obtain a target frequency.

4. The eye tracker-assisted SSVEP frequency comparison method according to claim 3, characterized in that: The processing of the fixation point coordinate information and the fixation point region information to obtain SSVEP frequency information and stimulation frequency information includes: S311, processing the gaze point coordinate information and the gaze point area information to obtain the area information to be processed; S312, processing the to-be-processed region information to obtain stimulation region information and stimulation frequency information; the stimulation region information includes a plurality of stimulation frequency regions; S313, using the stimulation area information and the stimulation frequency information, performing visual stimulation processing to obtain SSVEP frequency information.

5. The eye tracker-assisted SSVEP frequency comparison method according to claim 4, characterized in that: The processing of the gaze point coordinate information and the gaze point area information to obtain the area information to be processed includes: Determine whether the X-axis coordinate value of the gaze point coordinate information is greater than 0, and obtain a first determination result; When the first judgment result is yes, determine whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, and obtain a second judgment result; When the second judgment result is yes, determining the first gaze point area information as area information to be processed; When the second judgment result is no, determining the third gaze point area information as the area information to be processed; When the first judgment result is no, judging whether the Y-axis coordinate value of the gaze point coordinate information is greater than 0, and obtaining a third judgment result; When the third judgment result is yes, determining the second gaze point area information as the area information to be processed; When the third judgment result is no, the fourth gaze point area information is determined to be the area information to be processed.

6. The eye tracker-assisted SSVEP frequency comparison method according to claim 3, characterized in that: The analyzing and processing the SSVEP frequency information and the stimulation frequency information to obtain a target frequency includes: S321, analyzing and processing the SSVEP frequency information to obtain sub-band component information; the sub-band component information includes a plurality of sub-band components; S322, calculating and processing the sub-band component information and the stimulation frequency information to obtain a target frequency.

7. The eye tracker-assisted SSVEP frequency comparison method according to claim 6, characterized in that: The calculating and processing the sub-band component information and the stimulation frequency information to obtain the target frequency includes: S3221, calculating and processing the stimulation frequency information to obtain sine and cosine reference information; the sine and cosine reference information includes a plurality of sine and cosine reference template information; the stimulation frequency in the stimulation frequency information corresponds to the sine and cosine reference template information in the sine and cosine reference information one by one; S3222, using a first SSVEP calculation model, calculating and processing the subband component information and the sine and cosine reference information to obtain correlation coefficient matrix information; the correlation coefficient matrix information includes a plurality of correlation coefficient matrices; Wherein, the first SSVEP calculation model is: Where, XG is the correlation coefficient matrix information, XG j is the jth correlation coefficient matrix in the correlation coefficient matrix information, is the transposition of the ith subband component in the subband component information, SBT i is the projection of the ith sub-band component in the sub-band component information, ZY j T is the transpose of the jth sine-cosine reference template information in the sine-cosine reference information, ZYT j is the projection of the j-th sin-cosine reference template information in the sin-cosine reference information, N is the number of the sub-band components in the sub-band component information, M is the number of the sin-cosine reference template information in the sin-cosine reference information, and ρ represents a correlation coefficient; S3223, using a second SSVEP calculation model, calculating and processing the correlation coefficient matrix information to obtain SSVEP correlation coefficient information; the SSVEP correlation coefficient information includes a plurality of SSVEP correlation coefficients; Wherein, the second SSVEP calculation model is: In the formula, is the SSVEP correlation coefficient information, is the jth SSVEP correlation coefficient in the SSVEP correlation coefficient information, XG is the correlation coefficient matrix information, XG j is the jth correlation coefficient matrix in the correlation coefficient matrix information, δ1 and δ2 are respectively the first weight parameter and the second weight parameter, o is the smoothing index, a and b represent the first constant parameter and the second constant parameter, N is the number of the subband components in the subband component information, and M is the number of the sine-cosine reference template information in the sine-cosine reference information; S3224, determining a target frequency based on the SSVEP correlation coefficient information; the target frequency is the stimulation frequency corresponding to the largest SSVEP correlation coefficient in the SSVEP correlation coefficient information.

8. An eye tracker-assisted SSVEP frequency comparison device, characterized in that: The device comprises: An acquisition module, used to acquire first image information, orbital region information and fixation region information; the fixation region information includes first fixation region information, second fixation region information, third fixation region information and fourth fixation region information; A first calculation module is used to process the first image information and the orbital area information to obtain gaze point coordinate information; The second calculation module is used to fuse the gaze point coordinate information and the gaze point area information to obtain a target frequency.

9. An eye tracker-assisted SSVEP frequency comparison device, characterized in that: The device comprises: processor; a memory coupled to the processor and storing executable program code; The processor calls the executable program code stored in the memory to execute the eye tracker-assisted SSVEP frequency comparison method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the eye tracker-assisted SSVEP frequency comparison method as described in any one of claims 1-7.

Images