Recognition method of ar-ssvep in strong light environment based on iterative learning strategy

Abstract

The application discloses a kind of AR-SSVEP identification methods under strong light environment based on iterative learning strategy, comprising the following steps: A: the SSVEP data of multiple subjects under different illumination intensity including the illumination intensity to be identified is collected;B: generating template data;C: using CCA algorithm to process the 1st template data;D: the target obtained in step C and the stimulation of all targets in the 2nd template data are processed in turn;E: the stimulation of all targets in the remaining template data is processed in turn;F: all stimulations under all targets of all subjects under the illumination intensity to be identified are processed in turn according to the method in step E;The identification result of the data to be identified is obtained.The application can improve the accuracy of filter model, so as to accurately obtain the identification result of AR-SSVEP under strong light environment.

Description

Technical Field

[0001] This invention relates to the field of AR-SSVEP recognition, and more particularly to an AR-SSVEP recognition method based on an iterative learning strategy in a strong light environment. Background Technology

[0002] Brain-computer interface (BCI) is a novel technology that allows humans or other animals to directly connect their brains to external devices without relying on peripheral nerves and muscles, enabling the brain to directly control these devices. It represents a new communication method between the brain and external devices.

[0003] When a person is exposed to repetitive, continuous visual stimuli (flickering), the nervous system generates transient visual evoked potentials through linear superposition, resulting in electrical activity at the same frequency as the visual flicker—this is known as steady-state visual evoked potential (SSVEP). Different frequencies of stimulation produce SSVEP signals of different frequencies, thus SSVEP signals can be used as commands to control external devices. The advantages of high signal-to-noise ratio (SNR), high information transfer rate (ITR), low subject training requirements, and high accuracy make the application of SSVEP signals in the field of brain-computer interfaces (BCI) simpler. SSVEP is one of the most commonly used EEG signals in BCI control systems.

[0004] Traditional SSVEP-BCI systems mostly present the evoked scintillation blocks of SSVEP on a computer screen. Due to the relatively fixed location and large size of the device, users can only complete the interactive task sitting or standing in a specific room, which greatly limits the application scope of the system. Furthermore, during the experiment, subjects need to shift their attention back and forth between the stimulation screen and their normal field of vision, increasing their experimental burden.

[0005] With the continuous development of Augmented Reality (AR) technology, AR glasses have gradually entered people's lives. AR glasses are a typical product of AR technology development, integrating key AR components into a single pair of glasses, greatly improving portability and practicality. Due to their portability, AR glasses are increasingly attracting the attention of researchers in the field of Brain-Induced Communication (BCI). Coupled with the development of wireless EEG signal acquisition devices, AR may become the best way to bring BCI systems out of the laboratory and into daily life. In the AR-based SSVEP-BCI system, the SSVEP stimulation interface can be projected onto the real environment through AR devices, overcoming the limitations of computer screen displays and achieving portability and wearability. This also makes user interaction with the outside world more natural and comfortable. The application of AR technology has greatly expanded the research field of the SSVEP-BCI system and broadened its application scenarios.

[0006] Currently, research on SSVEP-BCI in AR applications is still in its early stages. Existing studies simply port the PC-based presentation to AR glasses without adapting external conditions for AR glasses. These studies employ mild experimental conditions, making it difficult to determine the factors influencing AR-BCI performance and their mechanisms. As research into AR-BCI technology deepens, there is a strong desire to expand its application beyond laboratory conditions, hoping to provide a better user experience. A key factor in non-laboratory conditions is lighting. To avoid its impact on experiments, laboratory lighting intensity is usually set low. However, strong lighting not only affects human vision but also the display system of AR glasses, significantly limiting the application and development of AR-BCI.

[0007] In recent years, traditional CS-SSVEP recognition algorithms have all employed spatial filters to optimize multi-channel EEG data in order to improve the recognition accuracy of SSVEP-BCI. Therefore, the method for obtaining the spatial filter is particularly important. Traditional spatial filter acquisition methods mostly use statistical learning to obtain the spatial filter of the EEG signal, requiring no training data and offering fast recognition speed, making them widely used in SSVEP-BCI systems with fewer targets. However, these traditional spatial filter acquisition methods are not specific to any particular SSVEP signal; the data they process is usually collected under fixed conditions and does not consider the influence of environmental factors on the data. Since lighting conditions have a significant impact on SSVEP signals in AR-BCI, traditional spatial filter acquisition methods are not suitable for AR-SSVEP application scenarios where lighting factors are present. Summary of the Invention

[0008] The purpose of this invention is to provide a method for recognizing AR-SSVEPs in strong light environments based on an iterative learning strategy. This method can improve the accuracy of the filter model, thereby accurately obtaining the recognition results of AR-SSVEPs in strong light environments.

[0009] The present invention adopts the following technical solution:

[0010] A method for recognizing AR-SSVEPs in strong light environments based on an iterative learning strategy includes the following steps:

[0011] A: SSVEP data were collected from multiple subjects under Q different light intensities, including the light intensity to be identified, where the light intensity to be identified was L. p SSVEP data X p This is the data to be identified, which will be lower than the light intensity L to be identified. p The following data X1, X2, ..., X p-1 As supplementary data;

[0012] B: Using the auxiliary data obtained in step A, generate template data for SSVEP data of each subject under different light intensities. Template data refers to the average value obtained by averaging the collected SSVEP data according to the number of collection rounds;

[0013] C: Use the CCA algorithm on template data After processing, the template data is finally obtained. Optimization parameters S1, autocovariance matrix and cross-covariance matrix And template data of all participants The target with the highest accuracy in identifying all targets.

[0014] D: Following the method in step C, sequentially process the targets obtained in step C. and template data The stimuli from all targets are processed to obtain template data. Optimization parameters S2, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets.

[0015] E: Following the method in step D, process the template data sequentially. to The stimuli from all targets are processed to obtain template data. The optimization parameters S(p-1) and autocovariance matrix and cross-covariance matrix And all template data of all subjects. The target with the highest accuracy in identifying all targets.

[0016] F: Let L be the light intensity to be identified. p The data to be identified X below p There are h stimuli in total, namely The target obtained in step E As the first target, it is matched with the scrambled data X to be identified. p The h targets are combined into h+1 targets, and then the illumination intensity L to be identified is determined according to the method in step E. p All stimuli under all targets for all subjects were processed sequentially; the identification labels for all stimuli under all targets were obtained, which are the data to be identified, X. p The recognition results.

[0017] In step A, different light intensities L1, L2, ..., L are first achieved using lighting equipment. p ..., L Q Then, Q SSVEP data points were collected for each subject under different light intensities, namely X1, X2, ..., X... p ... X Q Light intensity L p Corresponding SSVEP data X p For the data to be identified, select light intensities L1, L2, ..., L... p-1 The corresponding SSVEP data X1, X2, ..., X p-1 As auxiliary data; each SSVEP data is T×C×S×B dimensional data, where T represents the number of targets, C represents the number of channels during acquisition, S represents the number of sampling points, and B represents the number of data acquisition rounds.

[0018] Step C includes the following specific steps:

[0019] C1: Set template data There are T objectives in total, namely

[0020] C2: Template data The first target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value;

[0021] First, set the template data. The initial parameter value is 0, which is the optimization parameter. Autocovariance matrix and cross-covariance matrix The initial values ​​are all 0;

[0022] Then, the template data is obtained using the CCA algorithm. The first target Stimulus identification tags and filter And calculate

[0023] Finally, using the obtained template data and Calculated and The final value, and using and The final value of the target is calculated. of and The final value;

[0024] The formula is as follows:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] Where n represents the nth light intensity; m represents the index of the stimulus; and t represents the tth stimulus. For reference signal, f k denoted as the frequency of the k-th target, N is the number of sub-bands divided into the template data, and s is the time constant; and These are the autocovariance matrices. and cross-covariance matrix Initial value; matrix u T Let be the transpose of matrix u; matrix u contains The filter matrix, The first column of u; matrix w T Let w be the transpose of matrix w; matrix w contains The filter matrix, The first column of w; matrix v TLet v be the transpose of matrix v; matrix v contains The filter matrix, The first column of v;

[0031] C3: Template data The second target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value, and the final coefficient.

[0032] First, the target calculated in step C2 of and The final value, as the target of and The initial value;

[0033] Then, the template data is obtained using the CCA algorithm. The second target Stimulus identification tags and And using the method in step C2, the target is calculated. of and The final value;

[0034] Finally, calculate the final coefficients.

[0035]

[0036]

[0037]

[0038]

[0039] Among them, X [t] This represents the data being processed; when determining the recognition result, the frequency of the reference signal corresponding to the highest correlation coefficient is the recognition result.

[0040] C4: Following the method in step C3, process the template data. The third target Up to the Tth target The stimuli were processed separately to obtain template data. Optimization parameters S1, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets.

[0041] Among them, the target of and The initial value is the target. of and The final value; the target obtained of and As template data Optimization parameters S1, autocovariance matrix and cross-covariance matrix

[0042] In step C4, template data for each subject... Process it and use the final coefficients obtained. The identification result for each target is determined, and the identification accuracy is calculated by comparing the identification result with the true label of the stimulus. All template data of all subjects are selected and recorded. The target with the highest accuracy in identifying all targets.

[0043] In step D, template data is set. There are g objectives in total, namely The target obtained in step D With template data All objectives in the middle form g+1 objectives, which are as follows: Process the stimuli of g+1 targets sequentially according to the method in step D;

[0044] In processing the target At that time, the target of and The initial value is the template data obtained in step C4. S1, and Target of and The initial value is the target. of and The final value; and so on; finally, the target will be obtained. of and As template data Optimization parameters S2, autocovariance matrix and cross-covariance matrix

[0045] Similarly, template data for each participant The data was processed, and all template data from all subjects were selected and recorded. The target with the highest accuracy in identifying all targets.

[0046] In step F, the target of and The initial value is the template data. S(p-1), and and target of and The final value is used as the next stimulus. and The initial value.

[0047] This invention optimizes data under strong light conditions using data from low-light conditions and employs a specially designed cross-condition iterative processing method to provide initial optimization parameters for iterative learning under different lighting conditions. This allows the invention to learn not only the features of different target data but also the data features under different conditions, enabling effective optimization and correction of the filter. Furthermore, this invention features a specially designed first-target optimization method for further optimization and correction of the filter. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0050] like Figure 1 As shown, the AR-SSVEP recognition method based on iterative learning strategy in a strong light environment according to the present invention includes the following steps:

[0051] A: SSVEP data of multiple subjects under Q different light intensities, including the light intensity to be identified, are collected. The SSVEP data under the light intensity to be identified is the data to be identified, and the data below the light intensity to be identified is used as auxiliary data.

[0052] In this invention, the light intensity to be identified is L. p First, different light intensities L1, L2, ..., L are achieved through lighting equipment. p ..., LQ Then, Q SSVEP data points were collected for each subject under different light intensities, namely X1, X2, ..., X... p ... X Q Light intensity L p Corresponding SSVEP data X p This refers to the data to be identified, where light intensities L1, L2, ..., L are selected. p-1 The corresponding SSVEP data X1, X2, ..., X p-1 As auxiliary data; each SSVEP data is T×C×S×B dimensional data, where T represents the number of targets, C represents the number of channels during acquisition, S represents the number of sampling points, and B represents the number of data acquisition rounds;

[0053] B: Using the auxiliary data obtained in step A, generate template data for SSVEP data of each subject under different light intensities;

[0054] In step B, the results obtained in step A for each subject under different light intensities L1, L2, ..., L... p-1 The following SSVEP data X1, X2, ..., X p-1 Template data for each subject was created. The template data refers to the average value obtained by averaging the collected SSVEP data according to the number of collection rounds.

[0055] In this invention, template data is used to provide initial parameters for calculating filters for the data to be identified. Using template data not only utilizes data information under different targets, but also information under different light intensities. Furthermore, the initial parameters calculated in this way are calculated by gradually superimposing them from weak light intensity to strong light intensity, so that the final calculated filter is a progressively optimized filter, which can effectively improve the accuracy of SSVEP signal recognition results.

[0056] C: Use the CCA algorithm on template data After processing, the template data is finally obtained. Optimization parameters S1, autocovariance matrix and cross-covariance matrix And template data of all participants The target with the highest accuracy in identifying all targets.

[0057] Step C includes the following specific steps:

[0058] C1: Set template data There are T objectives in total, namely

[0059] C2: Template data The first target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value;

[0060] First, set the template data. The initial parameter value is 0, which is the optimization parameter. Autocovariance matrix and cross-covariance matrix The initial values ​​are all 0;

[0061] Then, the template data is obtained using the CCA algorithm. The first target Stimulus identification tags and filter And calculate The CCA algorithm is a standard technique in this field and will not be described in detail here.

[0062] Finally, using the obtained template data and Calculated and The final value, and using and The final value of the target is calculated. of and The final value is calculated using the following formula:

[0063]

[0064]

[0065]

[0066]

[0067]

[0068] Where n represents the nth light intensity; m represents the index of the stimulus; and t represents the tth stimulus. For reference signal, f k denoted as the frequency of the k-th target, N is the number of sub-bands divided into the template data, and s is the time constant; and These are the autocovariance matrices. and cross-covariance matrix Initial value; matrix u T Let be the transpose of matrix u; matrix u contains The filter matrix, The first column of u; matrix w T Let w be the transpose of matrix w; matrix w contains The filter matrix, The first column of w; matrix v T Let v be the transpose of matrix v; matrix v contains The filter matrix, The first column of v;

[0069] C3: Template data The second target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value, and the final coefficient.

[0070] First, the target calculated in step C2 of and The final value, as the target of and The initial value;

[0071] Then, the template data is obtained using the CCA algorithm. The second target Stimulus identification tags and And using the method in step C2, the target is calculated. of and The final value;

[0072] Finally, the following three correlation coefficients were calculated:

[0073]

[0074]

[0075]

[0076] Among them, X [t] This indicates that the data being processed is obtained by adding the three correlation coefficients mentioned above to obtain the final coefficient. When determining the recognition result, the frequency of the reference signal corresponding to the largest correlation coefficient is the recognition result.

[0077] C4: Following the method in step C3, process the template data. The third target Up to the Tth target The stimuli were processed separately to obtain template data. Optimization parameters S1, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets.

[0078] Among them, the target of and The initial value is the target. of and The final value; and so on, the target of and The initial value is the target. of and The final value; finally, the target will be obtained. of and As template data Optimization parameters S1, autocovariance matrix and cross-covariance matrix

[0079] Following the above method, template data for each subject... Perform the processing; then use the final coefficients obtained in step C3. The identification result for each target is determined, and the identification accuracy is calculated by comparing the identification result with the true label of the stimulus. All template data of all subjects are selected and recorded. The target with the highest accuracy in identifying all targets.

[0080] D: Following the method in step C, sequentially process the targets obtained in step C. and template data The stimuli from all targets are processed to obtain template data. Optimization parameters S2, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets.

[0081] In this embodiment, template data is set. There are g objectives in total, namely The target obtained in step D With template data All objectives in the middle form g+1 objectives, which are as follows: Process the stimuli of g+1 targets sequentially according to the method in step D;

[0082] In processing the target At that time, the target of and The initial value is the template data obtained in step C4. S1, and Target of and The initial value is the target. of and The final value; and so on; finally, the target will be obtained. of and As template data Optimization parameters S2, autocovariance matrix and cross-covariance matrix

[0083] Similarly, template data for each participant The data was processed, and all template data from all subjects were selected and recorded. The target with the highest accuracy in identifying all targets.

[0084] E: Following the method in step D, process the template data sequentially. to The stimuli from all targets are processed to obtain template data. The optimization parameters S(p-1) and autocovariance matrix and cross-covariance matrix And all template data of all subjects. The target with the highest accuracy in identifying all targets. At this point, the auxiliary data processing is complete.

[0085] F: Let L be the light intensity to be identified. p The data to be identified X below p There are h stimuli in total, namely The target obtained in step E As the first target, it is matched with the scrambled data X to be identified. p The h targets in the dataset form h+1 targets, and the data to be identified X... p Each target contains group B stimuli, and then the light intensity L to be identified is determined according to the method in step E. p All stimuli under all targets for all subjects were processed sequentially.

[0086] During processing, the target of and The initial value is the template data. S(p-1), and and target of and The final value is used as the next stimulus. and The initial value is obtained; finally, the identification labels of all stimuli under all targets are obtained, which is the data to be identified, X. p The recognition results.

[0087] Compared to the OACCA algorithm (Online Adaptive CCA, OACCA), this invention optimizes data under strong light conditions using data from low-light conditions and employs a specially designed cross-condition iterative processing method to provide initial optimization parameters for iterative learning under different lighting conditions. This invention uses the final values ​​of the optimization parameters, autocovariance matrix, and cross-covariance matrix calculated for one target under a certain lighting condition as the initial values ​​for the optimization parameters, autocovariance matrix, and cross-covariance matrix of the next target under the same lighting condition. By combining this with comprehensive analysis and optimization of data from multiple low-light conditions, it can not only learn the characteristics of different target data but also the data characteristics under different conditions, thus enabling effective optimization and correction of the filter.

[0088] This invention also features a specially designed first-target optimization method. During cross-condition iteration under different lighting conditions, the target with the highest recognition accuracy among all targets in all template data under the previous lighting condition is combined with all targets under the next lighting condition to form a new set, and these sets are analyzed and calculated sequentially. During the calculation, the target with the highest accuracy is used as transitional data for cross-condition iteration (two different lighting conditions). The optimized parameters, autocovariance matrix, and crosscovariance matrix of the previous template data are used as the initial parameters for the target with the highest accuracy. Furthermore, the final parameters of the target with the highest accuracy are used as the initial parameters for the next stimulus, enabling further optimization and correction of the filter. This invention optimizes the first target in the target data, allowing for a relatively accurate filter model to be obtained in the early stages of filter learning.

Claims

1. A method for recognizing AR-SSVEPs in strong light environments based on an iterative learning strategy, characterized in that, Includes the following steps: A: SSVEP data were collected from multiple subjects under Q different light intensities, including the light intensity to be identified, where the light intensity to be identified was L. p SSVEP data X p This is the data to be identified, which will be lower than the light intensity L to be identified. p The following data X1, X2, ..., X p-1 As supplementary data; B: Using the auxiliary data obtained in step A, generate template data for SSVEP data of each subject under different light intensities. Template data refers to the average value obtained by averaging the collected SSVEP data according to the number of collection rounds; C: Use the CCA algorithm on template data After processing, the template data is finally obtained. Optimization parameters S1, autocovariance matrix and cross-covariance matrix And template data of all participants The target with the highest accuracy in identifying all targets. D: Following the method in step C, sequentially process the targets obtained in step C. and template data The stimuli of all targets are processed to obtain template data. Optimization parameters S2, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets. E: Following the method in step D, process the template data sequentially. to The stimuli from all targets are processed to obtain template data. The optimization parameters S(p-1) and autocovariance matrix and cross-covariance matrix And all template data of all subjects. The target with the highest accuracy in identifying all targets. F: Let L be the light intensity to be identified. p The data to be identified X below p There are h stimuli in total, namely The target obtained in step E As the first target, it is matched with the scrambled data X to be identified. p The h targets are combined into h+1 targets, and then the illumination intensity L to be identified is determined according to the method in step E. p All stimuli under all targets for all subjects were processed sequentially; the identification labels for all stimuli under all targets were obtained, which are the data to be identified, X. p The recognition results.

2. The AR-SSVEP recognition method based on iterative learning strategy under strong light conditions according to claim 1, characterized in that: In step A, different light intensities L1, L2, ..., L are first achieved using lighting equipment. p ..., L Q Then, Q SSVEP data points were collected for each subject under different light intensities, namely X1, X2, ..., X... p ... X Q Light intensity L p Corresponding SSVEP data X p For the data to be identified, select light intensities L1, L2, ..., L... p-1 The corresponding SSVEP data X1, X2, ..., X p-1 As auxiliary data; each SSVEP data is T×C×S×B dimensional data, where T represents the number of targets, C represents the number of channels during acquisition, S represents the number of sampling points, and B represents the number of data acquisition rounds.

3. The AR-SSVEP recognition method based on iterative learning strategy under strong light conditions according to claim 1, characterized in that, Step C includes the following specific steps: C1: Set template data There are T objectives in total, namely C2: Template data The first target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value; First, set the template data. The initial parameter value is 0, which means the optimization parameter... Autocovariance matrix and cross-covariance matrix The initial values ​​are all 0; Then, the template data is obtained using the CCA algorithm. The first target Stimulus identification tags and filter And calculate Finally, using the obtained template data and Calculated and The final value, and using and The final value of the target is calculated. of and The final value; The formula is as follows: Where n represents the nth light intensity; m represents the index of the stimulus; and t represents the tth stimulus. For reference signal, f k denoted as the frequency of the k-th target, N is the number of sub-bands divided into the template data, and s is the time constant; and These are the autocovariance matrices. and cross-covariance matrix Initial value; matrix u T Let be the transpose of matrix u; matrix u contains The filter matrix, The first column of u; matrix w T Let w be the transpose of matrix w; matrix w contains The filter matrix, The first column of w; matrix v T Let v be the transpose of matrix v; matrix v contains The filter matrix, The first column of v; C3: Template data The second target The stimulus is processed to obtain the target. Optimization parameters Autocovariance matrix and cross-covariance matrix The final value, and the final coefficient. First, the target calculated in step C2 of and The final value, as the target of and The initial value; Then, the template data is obtained using the CCA algorithm. The second target Stimulus identification tags and And using the method in step C2, the target is calculated. of and The final value; Finally, calculate the final coefficients. Among them, X [t] This represents the data being processed; when determining the recognition result, the frequency of the reference signal corresponding to the highest correlation coefficient is the recognition result. C4: Following the method in step C3, process the template data. The third target Up to the Tth target The stimuli were processed separately to obtain template data. Optimization parameters S1, autocovariance matrix and cross-covariance matrix and all template data The target with the highest accuracy in identifying all targets. Among them, the target of and The initial value is the target. of and The final value; the target obtained of and As template data Optimization parameters S1, autocovariance matrix and cross-covariance matrix 4. The AR-SSVEP recognition method based on iterative learning strategy under strong light conditions according to claim 3, characterized in that: In step C4, template data for each subject... Process it and use the final coefficients obtained. The recognition result for each target is determined, and the recognition accuracy is calculated by comparing the recognition result with the true label of the stimulus. All template data of all subjects are selected and recorded. The target with the highest accuracy in identifying all targets.

5. The AR-SSVEP recognition method based on iterative learning strategy under strong light conditions according to claim 3, characterized in that: In step D, template data is set. There are g objectives in total, namely The target obtained in step D With template data All objectives in the middle form g+1 objectives, which are as follows: Process the stimuli of g+1 targets sequentially according to the method in step D; In processing the target At that time, the target of and The initial value is the template data obtained in step C4. S1, and Target of and The initial value is the target. of and The final value; and so on; finally, the target will be obtained. of and As template data Optimization parameters S2, autocovariance matrix and cross-covariance matrix Similarly, template data for each participant The data was processed, and all template data from all subjects were selected and recorded. The target with the highest accuracy in identifying all targets.

6. The AR-SSVEP recognition method based on iterative learning strategy under strong light conditions according to claim 5, characterized in that: In step F, the target of and The initial value is the template data. S(p-1), and and target of and The final value is used as the next stimulus. and The initial value.

Images