Visual and auditory brain-computer interface spelling system and method based on space and semantic consistency

Abstract

The invention relates to a visual and auditory brain-computer interface spelling system and method based on space and semanteme consistency. In the spelling system, a visual and auditory stimulation device generates visual and auditory stimulation about target characters to a testee by adopting a visual and auditory experimental paradigm based on space and semanteme consistency; an electroencephalogram signal acquisition device acquires electroencephalogram signals of a testee; a computer performs data preprocessing on the electroencephalogram signal data to obtain preprocessed electroencephalogram signal data; and the computer performs feature extraction and classification on the preprocessed electroencephalogram signal data to obtain a classification result, and sends the classificationresult to an output display device. Spatial features and semantic features of visual stimulation and auditory stimulation are combined, spatial and semantic consistency of visual and auditory bimodalstimulation is achieved, compared with single visual stimulation or auditory stimulation, larger-amplitude ERPs can be induced, and therefore the performance of the brain-computer interface is improved.

Description

technical field [0001] The invention relates to the technical field of brain-computer interface, in particular to an audio-visual brain-computer interface spelling system and method based on spatial and semantic consistency. Background technique [0002] The brain-computer interface is a new type of direct interaction system between the brain and the computer, which establishes a direct communication channel between the brain and external devices for patients with muscular atrophy (especially patients with amyotrophic lateral sclerosis) . The current common brain-computer interface system is to record EEG signals related to event-related potentials (Event-related Potentials, ERPs) on the scalp, and to recognize commands or output characters by analyzing EEG signals. The P300-based speller is a character-spelling system based on oddball event-related potentials. Existing P300-based spelling systems mainly include the following: vision-based, auditory-based and tactile-based...

AI Technical Summary

Problems solved by technology

[0003] Based on this, it is necessary to address the unsatisfactory performance of the current vision-based brain-computer interface, and the limitations of the vision-based brain-computer interface spelling system for patients whose eye muscle control may be impaired or degenerate over time, and provide a An audio-visual brain-computer interface spelling system based on spatial and semantic consistency and its method, the audio-visual brain-computer interface spelling system based on spatial and semantic consistency can not only improve the performance of the visual-based brain-computer interface spelling system, but also improve its Versatility (suitable for both hearing-limited and visually-limited patients)

Images