Multi-sensory-mode BCI-VR control method and system and VR equipment

Abstract

The invention discloses a multi-sensory-mode BCI-VR control method and system and VR equipment, and the method comprises the steps: obtaining an EEG signal which is triggered by a VR scene and comprises a plurality of sensory signals corresponding to different sensory modes; performing feature extraction and classification recognition processing on the sensory signals to obtain EEG classification information corresponding to the sensory signals; and combining the EEG classification information to obtain an identification result of the EEG signal. According to the method, the traditional VR equipment is combined with the multi-sensory-mode BCI, and the SSVEP stimulation is fused to simulate the real environment, so that the adaptability and fatigue feeling of direct repeated stimulation on the brain of the user caused by the fact that the SSVEP single-visual-mode BCI depends on target object flicker stimulation are relieved, the visual fatigue is relieved, and the use experience of the user is improved.

Description

technical field

[0001] The present invention relates to the technical field of artificial intelligence, in particular to a multi-sensory mode BCI-VR control method, system and VR equipment. Background technique

[0002] Brain Computer Interface (BCI) enables humans to convert their own brain signals into instructions to control external devices without the participation of the peripheral nervous system and muscle tissue. BCI has a good application prospect in the field of medical rehabilitation. It provides patients with severe movement disorders with new auxiliary motor functions and external communication channels, that is, they can control wheelchairs and type without body movements or language.

[0003] Electroencephalogram (Electroencephalogram, EEG) has become the most commonly used brain signal for BCI due to its advantages of non-invasiveness, high time resolution, low equipment price and portability. Among them, the Steady State Visual Evoked Potentials (SSVEP) is ...

Examples

Embodiment 1

[0044] see figure 1 , the present embodiment provides a multi-sensory mode BCI-VR control method, including steps:

[0045] S1. Acquire an EEG signal, which is triggered by a VR scene and includes sensory signals corresponding to different sensory modalities.

[0046] VR (Virtual Reality, Virtual Reality) is a computer simulation system that provides users with an immersive three-dimensional virtual reality environment and a variety of sensory simulation experiences. The VR scene is a target object constructed based on virtual reality technology.

[0047] In this embodiment, different sensory modalities may include visual modalities, auditory modalities, and motor imagery modalities; correspondingly, EEG signals include SSVEP visual signals, P300 auditory signals, and MI motor imagery signals corresponding to the above sensory modalities.

[0048] S2. Perform feature extraction and classification recognition processing on the sensory signals, and obtain EEG classification inf...

Embodiment 2

[0090] see figure 2 , the present invention provides a multi-sensory modality BCI-VR system, as attached image 3 As shown, the multi-sensory BCI-VR system includes: an acquisition module 100 , an information preprocessing module 200 , an information processing module 300 , an information combination module 400 and a trigger module 500 .

[0091]Wherein, the acquisition module 100 is used to acquire EEG signals; the EEG signals are triggered by VR scenes and include sensory signals corresponding to different sensory modalities; wherein, in this embodiment 1, different sensory modalities include: visual modalities, auditory modalities The EEG signal includes one-to-one corresponding SSVEP visual signal, P300 auditory signal and MI motor imagery signal.

[0092] The VR scene is a target object constructed based on virtual reality technology; different target objects flicker according to different flicker frequencies, and flicker triggers the brain to generate SSVEP visual sign...

Embodiment 3

[0121] This embodiment provides a VR device, including the multi-sensory BCI-VR system in Embodiment 2. Preferably, the VR device is a VR game device. see image 3 , shows an application schematic diagram of a VR game device.

[0122] Specifically, in an application scenario, the VR game device constructs a virtual protagonist and target objects based on the game scene type of the VR game; different virtual protagonists and target objects flash according to different flicker frequencies, such as the left hand, right hand, The head and feet flicker at different flicker frequencies to trigger the brain to generate SSVEP visual signals corresponding to the visual sensory modes. The left hand, right hand, head and feet of the virtual protagonist are stimulated according to different low-probability sound frequencies to trigger the brain to generate auditory sensory modes. Corresponding to the P300 auditory signal, the user also generates a motor imagery MI signal. Based on the ...

technical field

[0001] The present invention relates to the technical field of artificial intelligence, in particular to a multi-sensory mode BCI-VR control method, system and VR equipment. Background technique

[0002] Brain Computer Interface (BCI) enables humans to convert their own brain signals into instructions to control external devices without the participation of the peripheral nervous system and muscle tissue. BCI has a good application prospect in the field of medical rehabilitation. It provides patients with severe movement disorders with new auxiliary motor functions and external communication channels, that is, they can control wheelchairs and type without body movements or language.

[0003] Electroencephalogram (Electroencephalogram, EEG) has become the most commonly used brain signal for BCI due to its advantages of non-invasiveness, high time resolution, low equipment price and portability. Among them, the Steady State Visual Evoked Potentials (SSVEP) is ...

Examples

Embodiment 1

[0044] see figure 1 , the present embodiment provides a multi-sensory mode BCI-VR control method, including steps:

[0045] S1. Acquire an EEG signal, which is triggered by a VR scene and includes sensory signals corresponding to different sensory modalities.

[0046] VR (Virtual Reality, Virtual Reality) is a computer simulation system that provides users with an immersive three-dimensional virtual reality environment and a variety of sensory simulation experiences. The VR scene is a target object constructed based on virtual reality technology.

[0047] In this embodiment, different sensory modalities may include visual modalities, auditory modalities, and motor imagery modalities; correspondingly, EEG signals include SSVEP visual signals, P300 auditory signals, and MI motor imagery signals corresponding to the above sensory modalities.

[0048] S2. Perform feature extraction and classification recognition processing on the sensory signals, and obtain EEG classification inf...

Embodiment 2

[0090] see figure 2 , the present invention provides a multi-sensory modality BCI-VR system, as attached image 3 As shown, the multi-sensory BCI-VR system includes: an acquisition module 100 , an information preprocessing module 200 , an information processing module 300 , an information combination module 400 and a trigger module 500 .

[0091]Wherein, the acquisition module 100 is used to acquire EEG signals; the EEG signals are triggered by VR scenes and include sensory signals corresponding to different sensory modalities; wherein, in this embodiment 1, different sensory modalities include: visual modalities, auditory modalities The EEG signal includes one-to-one corresponding SSVEP visual signal, P300 auditory signal and MI motor imagery signal.

[0092] The VR scene is a target object constructed based on virtual reality technology; different target objects flicker according to different flicker frequencies, and flicker triggers the brain to generate SSVEP visual sign...

Embodiment 3

[0121] This embodiment provides a VR device, including the multi-sensory BCI-VR system in Embodiment 2. Preferably, the VR device is a VR game device. see image 3 , shows an application schematic diagram of a VR game device.

[0122] Specifically, in an application scenario, the VR game device constructs a virtual protagonist and target objects based on the game scene type of the VR game; different virtual protagonists and target objects flash according to different flicker frequencies, such as the left hand, right hand, The head and feet flicker at different flicker frequencies to trigger the brain to generate SSVEP visual signals corresponding to the visual sensory modes. The left hand, right hand, head and feet of the virtual protagonist are stimulated according to different low-probability sound frequencies to trigger the brain to generate auditory sensory modes. Corresponding to the P300 auditory signal, the user also generates a motor imagery MI signal. Based on the ...

Images