A walking intention induction method and device based on an emotional brain machine and a storage medium

Abstract

The present application relates to a kind of walking intention induction method, device and storage medium based on emotional brain machine, wherein the method comprises the following steps: configuration experimental environment;Create virtual walking environment, the behavior of the subject in real world is mapped to virtual walking scene;According to emotional stimulation induction process, set walking path and integral reward strategy in virtual walking scene;Based on different walking scene emotional type, set emotional stimulation task;Real-time detection running emotional stimulation task, collect the electroencephalogram signal of subject in emotional stimulation induction process and segment and obtain electroencephalogram signal segment;The electroencephalogram signal segment is preprocessed;According to different walking scene emotional type, the electroencephalogram signal segment after preprocessing is matched with corresponding label according to time stamp, and the electroencephalogram signal data with label is output.Compared with prior art, the present application has the advantages of electroencephalogram data standardization, high electroencephalogram signal quality.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing, and in particular to a method, apparatus, and storage medium for inducing walking intention based on emotion-based brain-computer interfaces. Background Technology

[0002] Brain-computer interface (BCI) technology is a novel human-computer interaction method that allows humans to communicate with the external environment by outputting control signals through electronic devices such as computers, without the involvement of the peripheral nervous system and muscle tissue. Emotional brain-computer interfaces (BCIs) are a way to identify and regulate specific emotions in the human brain using BCI technology. However, for traditional BCI tasks, the functions and mechanisms of the hundreds of billions of neurons in the brain have not been sufficiently understood and explored, which limits the accurate recognition of brain signals by BCI systems. Furthermore, brain signals are subject to multiple interferences from the environment and the body's own physiological signals, making it even more difficult to effectively acquire task-related signals.

[0003] Human motor intention recognition is an important application of brain-computer interfaces (BCIs), and the common method is to acquire data through motor imagery. Motor preparation is the brain's pre-thought preparation for a related action, primarily utilizing EEG data before the start of the movement to detect the intention. Compared to motor imagery, motor preparation elicits a faster EEG response, a shorter signal response window, and more pronounced changes. Therefore, detecting the characteristics of motor preparation-evoked EEG can significantly improve the efficiency of BCIs based on motor intention recognition. In motor intention recognition based on motor imagery EEG, the test subject is instructed not to move, but rather to imagine the process of movement; this process is called the motor imagery paradigm. Afterward, the acquired EEG signals are preprocessed and input into various machine learning models for different forms of classification.

[0004] EEG recognition requires a large amount of high-quality data. With data acquisition equipment already mature, the quality of EEG data is of paramount importance.

[0005] Patent CN 110025452 B discloses an invention of a tactile feedback ankle joint function training system based on a brain-computer interface, including a training chair for subjects and a subject interface module, including a display screen and a host, capable of providing animations based on steady-state visual induction for subjects to watch. This invention uses a brain-computer interface to extract the subject's electroencephalogram (EEG) signals for tactile training, achieving active intention training; training based on steady-state visual induction technology simulates the gait cycle during walking, which helps subjects train their proprioception and abnormal gait. However, some problems still exist in the application of motor intention recognition, such as significant noise in the collected data, large differences in data characteristics among different test subjects, and low recognition accuracy due to inconsistent data collection paradigms. Due to different paradigms in the motor imagery process, the classification model has low generality and the classification results are not reliable. In some walking intention recognition tasks, data sources other than EEG signals have been used, such as cerebral cortical blood oxygen concentration and human myogenic electrical signals. However, changing the signal source has not fundamentally solved the problem of brain signals being affected by the environment and subjective human consciousness. Summary of the Invention

[0006] The purpose of this invention is to provide a method, device, and storage medium for inducing walking intention based on emotional brain-computer interfaces, so as to provide a standardized method for acquiring EEG data of walking intention and improve the quality of EEG signals and signal-to-noise ratio in brain-computer walking intention recognition.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for inducing walking intention based on emotion-based brain-computer interfaces includes the following steps:

[0009] Configure the experimental environment: Configure the experimental environment according to the requirements of brain-computer data acquisition, ensure that the subjects are in a quiet indoor environment, away from high-power electrical appliances, and create an atmosphere of concentration as much as possible. At the same time, it is required that there be no head or limb shaking or other movements to avoid changes in brain signals caused by muscle contractions that are not related to the task.

[0010] Create a virtual walking environment to map the behavior of subjects in the real world to the virtual walking scene: create a virtual walking scene according to the brain-computer task (walking intention) induced as needed, and display the current virtual scene on the head-mounted display to provide feedback on the currently mapped virtual walking behavior to the subject.

[0011] Based on the emotional stimulation process, set walking paths and point reward strategies in virtual walking scenarios;

[0012] Set emotional stimulation tasks based on different emotional types in different walking scenarios;

[0013] The system monitors emotional stimuli in real time, collects the subjects' EEG signals during the emotional stimulation induction process, and obtains EEG signal segments in segments.

[0014] Preprocessing of EEG signal segments;

[0015] Based on different emotional types in different walking scenarios, the preprocessed EEG signal segments are matched with corresponding labels according to their timestamps, and labeled EEG signal data is output.

[0016] The virtual walking scenario includes:

[0017] A virtual walking environment that includes a clear walking path and destination;

[0018] A virtual walking character is used to simulate walking, and the display shows real-time feedback to the subject from a camera that follows the virtual walking character.

[0019] The emotional types of the walking scene include walking and stopping.

[0020] The emotional stimulation task includes: guiding a virtual walking character to move along a walking path and walk to a destination, and guiding the virtual character to stop walking when the distance to the destination is less than a pre-configured threshold.

[0021] The points reward strategy is to award points based on a pre-configured score for each completed emotional stimulation task.

[0022] The EEG signals were acquired using a 32-channel BCIduino device with a sampling frequency of 1000Hz.

[0023] The preprocessing of the EEG signal segments involves filtering the EEG signal segments using a 5th-order Butterworth bandpass filter with a filtering range of 10-12Hz and 20-24Hz.

[0024] The tags correspond one-to-one with the emotional type of the walking scene, including walking and stopping.

[0025] A walking intention induction device based on emotion brain-computer interface includes a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the method described above.

[0026] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) This invention creates a virtual walking environment as a guide and simulates the addition of a reward and punishment mechanism as an emotional stimulus, mapping the behavior of the subjects in the real world to the virtual reality environment, thereby obtaining standardized EEG data of walking intentions without subjectivity.

[0029] (2) The standardized walking intention EEG data collected by this invention can be applied to the problem of brain-computer walking intention recognition, which can solve the problems of low EEG signal quality and low signal-to-noise ratio to a certain extent. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention;

[0031] Figure 2 The process is triggered by emotional stimulation;

[0032] Figure 3 This is a schematic diagram of a virtual walking scene;

[0033] Figure 4 This is a schematic diagram of brain lead pathways. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0035] A method for inducing walking intention based on emotion-brain-computer interface, such as Figure 1 As shown, it includes the following steps:

[0036] 1) Configure the experimental environment;

[0037] The implementation environment of this embodiment requires the entire testing process to be completed in a quiet indoor environment. To ensure the electromagnetic environment and signal acquisition quality, the subject must be kept away from communication devices such as mobile phones and laptops. The subject sits on an armchair 1 meter away from the monitor, with both hands placed on the armrests, and is required to keep their muscles relaxed, without any head or limb movements, to avoid changes in EEG signals caused by muscle contractions unrelated to the task.

[0038] 2) Create a virtual walking environment to map the subjects' behavior in the real world into the virtual walking scene;

[0039] Based on the brain-computer interface task (walking intention) induced as needed, a virtual walking scenario is created. The head-mounted display shows the current virtual scenario and provides feedback on the currently mapped virtual walking behavior to the subject. The virtual scenario then changes further based on the brain-computer interface data acquisition task.

[0040] 3) Set walking paths and point reward strategies in virtual walking scenarios based on the process of emotional stimulation;

[0041] This embodiment uses the Unity3D engine to create a virtual walking scene, such as Figure 3 As shown, it includes:

[0042] A virtual walking environment that includes a clear walking path and destination;

[0043] A virtual walking character is used to simulate walking, and the display shows real-time feedback to the subject from a camera that follows the virtual walking character.

[0044] The points reward strategy rewards points based on a pre-allocated score for each completed emotional stimulation task.

[0045] 4) Set emotional stimulation tasks based on different emotional types in different walking scenarios;

[0046] The emotional types in walking scenarios include walking, stopping, turning left, turning right, and walking quickly. In this embodiment, the walking and stopping types are tested.

[0047] Set emotionally stimulating tasks based on the emotional type of the walking scenario:

[0048] Task 1 (Walking Forward Task): Guide the virtual walking character to move along the walking path and walk to the destination;

[0049] The corresponding points reward policy is that the points start at 0, and 1 point is added for each time the destination is reached;

[0050] Task 2 (Stop Task): Guide the virtual character to stop walking when the distance to the destination is less than a pre-configured threshold;

[0051] The corresponding points reward policy is to add 1 point for correctly stopping walking at the appropriate location.

[0052] By completing integral tasks and processing scenarios, participants tend to feel emotionally satisfied and mentally focused, thereby improving problems such as low EEG signal quality and low signal-to-noise ratio in walking imagery problems.

[0053] 5) Real-time monitoring of the emotional stimulation task, collecting the subject's EEG signals during the emotional stimulation induction process and obtaining EEG signal segments in segments;

[0054] In this embodiment, the test is divided into 10 groups, with each group having an emotional stimulation cycle of 80 seconds. Each group includes one walking forward task and one stopping task, and the process is as follows: Figure 2 As shown.

[0055] One set of virtual walking steps is set as follows:

[0056] 1) From 0 to 20 seconds, provide a complete demonstration of the virtual scenario before the test. Introduce the task to the participant and instruct them to relax their body and adjust their breathing. Announce the initial score and the score to be achieved. In other words, inform the participant that there will be one round of walking, including a score to reach 2 meters. This task is playable and not under the participant's control, but this does not need to be explained to the participant.

[0057] 2) From 20 to 30 seconds, a virtual task prompt is given, explaining the objects in the scene related to the task.

[0058] 3) From 30 to 60 seconds, a virtual walking simulation begins, with the task of walking along a road towards a house. Participants are guided to imagine walking based on the current virtual scene. Simultaneously, 2-second EEG signal data segments are collected at 30s, 36s, 42s, 48s, and 54s.

[0059] 4) Between 60 and 70 seconds, the subject is prompted that they have completed the virtual task of approaching the house and are about to move on to the next task.

[0060] 5) From 70 to 80 seconds, the virtual walking playback enters the final stage. The virtual character gets closer and closer to the house and finally stops at 80 seconds. The subjects are guided to imagine stopping walking based on the current scene in the virtual environment. At the same time, 2-second EEG signal data segments are collected at 70s, 72s, 74s, 76s, and 78s.

[0061] During the test, a quiet environment was maintained around the participants. After each virtual walking task, the participants were required to rest for 20 seconds and the virtual walking scene was reset.

[0062] In this embodiment, a 32-channel BCIduino device is used to acquire EEG data, with a sampling frequency of 1000Hz. The electrode caps are placed using the international 10 / 20 system. The 10 electrode channels acquired are F1, F2, FC1, FC2, FC3, FC4, C1, C2, C3, and C4. The reference electrode is Cz. Figure 4 As shown.

[0063] EEG acquisition can be done using an EEG cap or other methods. The EEG acquisition channel should correspond to the motor and sensory areas. Changing the acquisition method and acquisition channel does not change the EEG induction process and can bring the same effect.

[0064] 6) Preprocess the EEG signal segments;

[0065] Research on brain-computer interface (BCI) primarily focuses on identifying changes in EEG signals under different states, correlating external information with EEG, and ultimately enabling EEG to directly control external devices, thus achieving interaction between the human brain and external devices. A typical BCI system consists of EEG signal acquisition and signal processing (including signal preprocessing, feature extraction, and recognition / classification). Due to external factors, the acquired raw EEG data contains irrelevant elements, such as unavoidable power line interference at 50Hz. EEG signal preprocessing is a method to separate segments of the acquired signal that are related to walking intention. In this embodiment, a 5th-order Butterworth bandpass filter is used to filter the EEG segments, with a filtering range of mu (10-12Hz) and beta (20-24Hz), corresponding to the motor sensory areas of the brain.

[0066] 7) Based on different emotional types in different walking scenarios, the preprocessed EEG signal segments are matched with corresponding labels according to the timestamp, and labeled EEG signal data is output.

[0067] In motion intention recognition, Common Spatial Pattern (CSP) and Independent Component Analysis (ICA) are generally used to extract features from EEG signals. Commonly used machine learning methods for this problem include Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Backpropagation Neural Network (BPNN), and Deep Learning (DL). These machine learning methods all require labeled EEG signal segments. In this embodiment, a real-time emotional stimulus detection task is performed. Based on different emotional types in walking scenarios, each subject's EEG signal segment is simultaneously labeled with a corresponding label. The label corresponds one-to-one with the emotional type of the walking scenario. In this embodiment, the labels include walking and stopping. Labels may also include turning left, turning right, walking fast, walking slowly, etc.

[0068] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0069] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A walking intention inducing method based on an emotional brain machine, characterized in that, The method comprises the following steps: configuring an experimental environment; creating a virtual walking environment to map the behavior of a subject in the real world to a virtual walking scene; setting a walking path and an integral reward strategy in the virtual walking scene according to an emotional stimulus induction process; setting an emotional stimulus task based on different walking scene emotional types; real-time detecting a running emotional stimulus task, collecting an electroencephalogram signal of a subject in an emotional stimulus induction process and segmenting to obtain an electroencephalogram signal segment; preprocessing the electroencephalogram signal segment; based on different walking scene emotional types, matching corresponding labels to the preprocessed electroencephalogram signal segment according to a time stamp, and outputting labeled electroencephalogram signal data; the virtual walking scene comprises: a virtual walking environment containing a clear walking path and a destination; a virtual walking character for simulating walking, and a display real-time feedback picture of a subject from a camera following the virtual walking character; the emotional stimulus task comprises guiding the virtual walking character to move along the walking path and walk to the destination, and guiding the virtual character to stop walking when the distance to the destination is less than a preconfigured threshold; the integral reward strategy is an integral reward of a preconfigured score for each completed emotional stimulus task.

2. The walking intention evoking method based on emotional brain machine according to claim 1, characterized in that, The walking scene emotional types include walking and stopping. 3.The walking intention evoking method based on emotional brain-computer in claim 1, wherein, The electroencephalogram signal is collected by a 32-channel BCIduino device with a sampling frequency of 1000 Hz.

4. The walking intention evoking method based on emotional brain machine according to claim 1, characterized in that, The preprocessing of the electroencephalogram signal segment is filtering the electroencephalogram signal segment by a 5th order Butterworth band-pass filter, and the filtering range is 10-12 Hz and 20-24 Hz.

5. The walking intention evoking method based on emotional brain machine according to claim 1, characterized in that, The labels correspond one-to-one to the walking scene emotional types, including walking and stopping.

6. An emotion-based brain-machine walking intention evoking device, comprising a memory, a processor, and a program stored in the memory, wherein, The processor executes the program to implement the method of any one of claims 1-5.

7. A storage medium having stored thereon a program, characterized by The program is executed to implement the method of any one of claims 1-5.

Images