A multi-degree-of-freedom-based speed imagination dual-mode signal acquisition method

Abstract

The application provides a kind of based on multi-degree of freedom's speed imagination bimodal signal acquisition method, comprising the following steps: S1, constructs two degrees of freedom speed imagination action instruction set, instruction set is based on left elbow joint and finger joint, and the multi-instruction speed imagination of two joint parts is synchronously developed;S2, constructs the bimodal synchronous acquisition system based on electroencephalogram and near infrared spectrum, the EEG is electroencephalogram, and fNIRS is near infrared spectrum, and the synchronous acquisition of EEG and fNIRS signal is completed;S3, constructs the self-induced speed imagination multi-modal signal synchronous acquisition paradigm based on multi-degree of freedom, and the EEG-fNIRS speed imagination signal of elbow joint and finger joint is synchronously acquired using paradigm.The application has the beneficial effects: a kind of based on multi-degree of freedom's speed imagination bimodal signal acquisition method, proposes the speed imagination instruction set of multi-degree of freedom, overcomes the single defect of instruction in the field of brain-computer interface, and enhances the decoding range of brain-computer interface system to continuous neural activity.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of signal acquisition, and particularly relates to a speed imagination bimodal signal acquisition method based on multiple degrees of freedom. BACKGROUND

[0002] The motor imagery (MI) brain-computer interface (BCI) system as a non-invasive brain-computer interface system can take the brain neural activity information as an input signal through imagining limb movement to control external equipment, thereby reducing the dependence of the brain on the body. As a new type of rehabilitation treatment method, the motor imagery brain-computer interface technology greatly improves the possibility of rehabilitation of patients with brain injury and the like, has been widely applied to the field of rehabilitation treatment and the like, and has a broad application prospect. However, the current motor imagery brain-computer interface system mostly adopts single degree of freedom and discrete limb movement (such as hands, feet and tongue, etc.) to analyze subsequent control instructions, lacks a means for capturing multiple degrees of freedom continuous brain neural activity information, and the external controlled equipment of the brain-computer interface often needs to be controlled continuously in multiple degrees of freedom (such as continuously controlling the gripping force of a mechanical hand in two directions), so the existing single degree of freedom and discrete motor imagery brain-computer interface system cannot be applied to real application scenarios.

[0003] The speed, as a physical quantity, has a natural continuous characteristic, and setting speed imagination of different frequencies provides a feasible solution for enhancing the flexibility of the brain-computer interface system. In order to obtain more control instructions, experts and scholars have proposed heuristic speed imagination paradigms, although these methods can improve the number of instructions to a certain extent, but still lack a spontaneous speed imagination induction method.

[0004] Carrying out bimodal speed imagination research based on multiple degrees of freedom is of great significance for overcoming the limitations of the single degree of freedom motor imagination paradigm and increasing the class instruction of the brain-computer interface system. SUMMARY

[0005] Therefore, the application aims to provide a speed imagination bimodal signal acquisition method based on multiple degrees of freedom. Firstly, a two-degree-of-freedom speed imagination motion instruction set is designed, which is based on the left elbow joint and the finger joint, and multiple instruction speed imaginations are simultaneously carried out on the two joint parts. Secondly, a bimodal synchronous acquisition system based on electroencephalogram (EEG) and near-infrared spectrum (fNIRS) is designed, which can complete the synchronous acquisition of EEG and fNIRS signals. Finally, a spontaneous speed imagination multi-modal signal synchronous acquisition paradigm based on multiple degrees of freedom is designed, which synchronously acquires the EEG-fNIRS speed imagination signals of the elbow joint and the finger joint.

[0006] To achieve the above object, the technical scheme of the present application is as follows:

[0007] A multi-degree-of-freedom-based speed imagination bimodal signal acquisition method, comprising the following steps:

[0008] S1, constructing a two-degree-of-freedom speed imagination action instruction set, the instruction set is based on the left elbow joint and the finger joint, and multiple instruction speed imaginations are simultaneously performed on the two joint parts;

[0009] S2, constructing a bimodal synchronous acquisition system based on electroencephalogram (EEG) and near-infrared spectrum (fNIRS), the EEG is electroencephalogram, and the fNIRS is near-infrared spectrum, and the synchronous acquisition of EEG and fNIRS signals is completed;

[0010] S3, constructing a multi-degree-of-freedom-based spontaneous speed imagination multi-modal signal synchronous acquisition paradigm, and utilizing the paradigm to synchronously acquire the EEG-fNIRS speed imagination signals of the elbow joint and the finger joint.

[0011] Further, in step S1, the instruction set is used to adopt two-degree-of-freedom limb parts, including an elbow joint and a finger joint, and multiple speed imaginations are instructed according to two degrees of freedom.

[0012] Further, in step S1, the finger joint is denoted as X, the elbow joint is denoted as Y, and the instruction is denoted as (X, Y), X can select N1 speeds in the range of 0 Hz to positive infinity on the finger joint degree of freedom, and Y can select N2 speeds in the range of 0 Hz to positive infinity on the elbow joint degree of freedom, taking N1 = 3, X selecting 0 Hz, 0.5 Hz and 1 Hz, N2 = 3, and Y selecting 0 Hz, 0.5 Hz and 1 Hz as an example, the instruction set comprises (0 Hz, 0 Hz), (0 Hz, 0.5 Hz), (0 Hz, 1 Hz), (0.5 Hz, 0 Hz), (0.5 Hz, 0.5 Hz), (0.5 Hz, 1 Hz), (1 Hz, 0 Hz), (1 Hz, 0.5 Hz), and (1 Hz, 1 Hz).

[0013] Further, in step S2, the system comprises a photoelectric integrated detection cap, including C1 = 34 EEG channels and C2 = 20 fNIRS channels, containing 4 light sources and 16 detectors, the positions between the light sources and the detectors are fNIRS channels, and the detection cloth is distributed in the sensorimotor area and the supplementary motor area of the scalp.

[0014] Further, in step S2, a marking signal is sent by the subject machine, so as to realize double-mode signal synchronous marking; when the paradigm action training is performed, the near-infrared signal feedback interface is used to help the subject to complete the specified imagination action, and speed imagination capture is realized; the double-mode synchronous acquisition system uses the EEG and fNIRS synchronous acquisition equipment to record and save data.

[0015] Further, in step S2, specifically, the double-mode signal synchronous acquisition includes: when the experiment is performed, a circular color block with adjustable gray value is displayed in the lower right corner of the screen of the subject machine; in order to ensure the accuracy of the recording sample time, the display time of the color block is set to be synchronous with the time of the speed imagination, the gray value of the color block changes with the speed imagination task, so that the photoelectric marking module signal generates different values, the marking values are sent to the fNIRS detection system through the acquisition card, and then are sent to the EEG acquisition system through the wireless serial port, and finally the synchronous sampling of the EEG and fNIRS signals is realized.

[0016] Further, in step S3, N1 arm lifting speed imaginations are performed by using the elbow joints, N2 fist clenching speed imaginations are performed by using the finger joints, and the generated EEG and fNIRS signals are synchronously acquired, and the sampling frequencies are R1 and R2 respectively.

[0017] Further, the scheme discloses an electronic device, including a processor and a memory connected with the processor and used for storing executable instructions of the processor, and the processor is used for executing a multi-degree-of-freedom-based speed imagination double-mode signal acquisition method.

[0018] Further, the scheme discloses a server, including at least one processor and a memory connected with the processor, the memory stores executable instructions of the at least one processor, and the instructions are executed by the processor to make the at least one processor execute a multi-degree-of-freedom-based speed imagination double-mode signal acquisition method.

[0019] Further, the scheme discloses a computer readable storage medium, storing a computer program, and the computer program is executed by the processor to realize a multi-degree-of-freedom-based speed imagination double-mode signal acquisition method.

[0020] Compared with the prior art, the multi-degree-of-freedom-based speed imagination double-mode signal acquisition method has the following beneficial effects:

[0021] (1) The multi-degree-of-freedom-based speed imagination double-mode signal acquisition method provides a multi-degree-of-freedom speed imagination instruction set, overcomes the single instruction defect in the field of brain-computer interface, and enhances the decoding range of the brain-computer interface system on continuous neural activity.

[0022] (2) The speed imagination double-mode signal acquisition method based on multiple degrees of freedom, constructs a double-mode synchronous acquisition system, fully considers the photoelectric combined detection principle, and thus expands the multi-mode acquisition capability of the acquisition system;

[0023] (3) The speed imagination double-mode signal acquisition method based on multiple degrees of freedom, adopts a spontaneous speed imagination multi-mode signal synchronous acquisition paradigm, and takes near-infrared signals as feedback, and further improves the signal quality;

[0024] (4) The speed imagination double-mode signal acquisition method based on multiple degrees of freedom, the proposed double-mode speed imagination signal acquisition method based on multiple degrees of freedom improves the control performance of the brain-computer interface system, and has important significance for obtaining continuous neural intention in the brain and promoting the practical application of the brain-computer interface. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings, which form a part of the disclosure, are intended to provide further understanding of the present disclosure, and the illustrative embodiments of the present disclosure and their description serve to explain the present disclosure, and do not constitute an improper limitation on the present disclosure. In the drawings:

[0026] Figure 1 The double-mode speed imagination signal acquisition flowchart based on multiple degrees of freedom described in the embodiments of the present application;

[0027] Figure 2 The multi-degree-of-freedom instruction schematic diagram described in the embodiments of the present application;

[0028] Figure 3 The multi-mode speed imagination arrangement schematic diagram described in the embodiments of the present application;

[0029] Figure 4 In a complete speed imagination experiment, the task arrangement of each time period described in the embodiments of the present application. DETAILED DESCRIPTION

[0030] It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0031] The speed imagination double-mode signal acquisition method based on multiple degrees of freedom of the present application will be described in detail below in combination with the embodiments and the drawings.

[0032] As shown in the drawings, the speed imagination double-mode signal acquisition method based on multiple degrees of freedom of the present application comprises the following steps: Figure 1

[0033] ​(1) According to the double freedom design speed imagination action instruction set: the instruction set adopts two degrees of freedom of the limb part, including elbow joint and finger joint respectively, according to two degrees of freedom (finger joint is recorded as X, elbow joint is recorded as Y, and instruction is recorded as (X, Y)), a variety of speed imagination is instructed. On the finger joint freedom X, N1 speeds in the range of 0Hz to positive infinity can be selected, and on the elbow joint freedom Y, N2 speeds in the range of 0Hz to positive infinity can be selected. Taking N1 = 3, X selecting 0Hz, 0.5Hz, 1Hz three speeds, N2 = 3, Y selecting 0Hz, 0.5Hz, 1Hz three speeds as an example, the instruction set is constructed as shown in Figure 2 , which includes (0Hz, 0Hz), (0Hz, 0.5Hz), (0Hz, 1Hz), (0.5Hz, 0Hz), (0.5Hz, 0.5Hz), (0.5Hz, 1Hz), (1Hz, 0Hz), (1Hz, 0.5Hz), (1Hz, 1Hz) nine instruction actions.

[0034] (2) A dual-mode synchronous acquisition system based on electroencephalogram (EEG) and near-infrared spectrum (fNIRS) is designed.

[0035] As shown in Figure 3 , in order to realize the combination of synchronous acquisition, considering the physical space configuration of EEG and fNIRS, this chapter sets up an optoelectronic integrated detection cap according to the principle of optoelectronic joint acquisition.

[0036] The system uses C1 = 34 EEG channels and C2 = 20 fNIRS channels, including 4 light sources (red) and 16 detectors (blue), the position between the light source and the detector is the fNIRS channel. Among them, the EEG channel is represented by letters, HEO and BEO represent the upper and lower electrooculogram channels and the left and right electrooculogram channels respectively. The detection layout is distributed in the sensory motor area and the auxiliary motor area of the scalp, and the electrode arrangement mode conforms to the 10 / 20 international arrangement standard. In addition, the test machine sends a marking signal, so as to realize the synchronous marking of dual-mode signals. In order to realize speed imagination capture, near-infrared signal feedback interface is used when performing paradigm action training, helping the subject to complete the specified imagination action. The dual-mode synchronous acquisition system uses EEG and fNIRS synchronous acquisition equipment to record and save data, and at the same time, the system mainly solves the following two main problems:

[0037] First, the dual-mode signal synchronous acquisition. In the experiment, a circular color block with adjustable gray value was displayed in the lower right corner of the subject's screen. To ensure the accuracy of the recorded sample time, the display time of the color block was synchronized with the imagined time of the speed. The gray value of the color block changed with the speed imagination task, so that the photoelectric marking module signal produced different values. The marking value was sent to the fNIRS detection system through the acquisition card, and then sent to the EEG acquisition system through the wireless serial port, finally realizing the synchronous sampling of EEG and fNIRS signals.

[0038] Second, design fNIRS feedback interface to supervise the effectiveness of speed imagination. Speed imagination is an abstract motor imagination task, which requires subjects to complete the specified task according to subjective neural activity, so it is difficult to judge the effectiveness of the subject's speed imagination. To solve this problem, a fNIRS-based neural feedback graphical interface is designed for the dual-mode synchronous acquisition system. This interface can display the blood oxygen concentration changes of the subject in a specific task in real time during the training process, so that the subject can adjust the imagination state according to the feedback information to achieve the ideal training effect, and improve the quality of the collected signal. The host computer of the fNIRS detection system is responsible for the acquisition and display of the imagination signal, and calculates a feedback value according to the collected blood oxygen concentration change and sends it to the subject computer. The subject can adjust the imagination according to the feedback value of the brain activation displayed by the graphical interface of the subject computer.

[0039] (3) as shown in Figure 4 is a self-paced speed imagination multi-modal signal synchronous acquisition paradigm designed:

[0040] The elbow joint is used for N1=3 arm lifting speed imagination, the speed is 0Hz, 0.5Hz and 1Hz respectively, the finger joint is used for N2=3 fist clenching speed imagination, the speed is 0Hz, 0.5Hz and 1Hz respectively, and the generated EEG and fNIRS signals are synchronously acquired (the sampling frequencies are R1=1000Hz and R2=100Hz respectively).

[0041] Each subject's data includes 15 groups, one group includes 180 trials, and one trial includes 7s of speed imagination and 12s of rest. Before starting a formal test module, the screen is in complete darkness. When starting the experiment, there will be a 0.5s voice prompt, and a white dot will appear in the lower right corner of the screen for marking. After 7s, the small dot in the lower right corner of the screen disappears, and the subject rests for 12s.

[0042] The present application uses C1 = 34 EEG channels and C2 = 20 fNIRS channels, which are distributed in the sensorimotor and supplementary motor areas of the scalp. Before the data acquisition experiment, the skin impedance of the EEG electrodes was adjusted to less than 5 kΩ using conductive paste. The fNIRS data acquisition used a fiber-optic near-infrared acquisition system, which included 4 light sources (red) and 16 detectors (blue), with the positions between the light sources and detectors being light path channels. Each light source recording fNIRS signals included two wavelengths (785 nm and 850 nm), with 10 3 cm spaced detector channels in the left and right cerebral regions.

[0043] Those skilled in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0044] In several embodiments provided in the present application, it should be understood that the disclosed method and system can be implemented in other ways. For example, the division of the above-described units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-described units can be or can not be physically separated, and the components displayed as units can be or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0045] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the description of the present application.

[0046] The above merely provides the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A multi-degree-of-freedom based velocity imagination dual-mode signal acquisition method, characterized in that, The method comprises the following steps: S1, constructing a double-degree-of-freedom speed imagination action instruction set, the instruction set being based on an elbow joint and a finger joint, and a plurality of instruction speeds being simultaneously developed for the two joint parts; S2, constructing a double-mode synchronous acquisition system based on EEG and fNIRS, the EEG being electroencephalogram, and the fNIRS being near-infrared spectrum, so as to complete the synchronous acquisition of the EEG and fNIRS signals; S3, constructing a multi-degree-of-freedom spontaneous speed imagination multi-mode signal synchronous acquisition paradigm, and utilizing the paradigm to synchronously acquire the EEG-fNIRS speed imagination signals of the elbow joint and the finger joint; In step S1, the instruction set is utilized to adopt two-degree-of-freedom limb parts, including an elbow joint and a finger joint, and a plurality of speed imaginations are instructed according to the two degrees of freedom; In step S1, the finger joint is marked as X, the elbow joint is marked as Y, and the instruction is marked as (X, Y), X can select N1 speeds in the range of 0 Hz to positive infinity on the elbow joint degree of freedom, and Y can select N2 speeds in the range of 0 Hz to positive infinity on the finger joint degree of freedom, and the instruction set is constructed; In step S2, specifically comprising: Double-mode signal synchronous acquisition: when the experiment is performed, a circular color block with adjustable gray value is displayed in the lower right corner of the screen of the subject, in order to ensure the accuracy of the recorded sample time, the display time of the color block is synchronously set with the time of the speed imagination, the gray value of the color block changes with the speed imagination task, so that the photoelectric marking module signal produces different values, the marking values are sent to the fNIRS detection system through the acquisition card, and then sent to the EEG acquisition system through the wireless serial port, and finally the synchronous sampling of the EEG and fNIRS signals is realized; In step S3, N1 arm lifting speed imaginations are performed by the elbow joint, N2 fist clenching speed imaginations are performed by the finger joint, and the generated EEG and fNIRS signals are synchronously acquired, and the sampling frequencies are R1 and R2 respectively.

2. The method of claim 1, wherein, In step S2, the double-mode synchronous acquisition system based on EEG and fNIRS comprises a photoelectric integrated detection cap, the photoelectric integrated detection cap comprises C1 EEG detection channels and C2 fNIRS detection channels, and simultaneously comprises a light source and a detector, the space between the light source and the detector constitutes an fNIRS detection channel, and the detection channels are distributed in the sensory motor area and the auxiliary motor area of the scalp.

3. The method of claim 1, wherein, In step S2, the marking signal is sent by the subject machine, so as to realize double-mode signal synchronous marking; a near-infrared signal feedback interface is used when the paradigm action training is performed, to help the subject to complete the specified imagination action, realize speed imagination capture; and the double-mode synchronous acquisition system uses an EEG and fNIRS synchronous acquisition device to record and save data.

4. An electronic device, comprising a processor and a memory connected to the processor in communication, and used to store executable instructions of the processor, characterized in that: The processor is configured to execute the speed imagination double-mode signal acquisition method based on the multi-degree-of-freedom as claimed in any one of claims 1-3.

5. A server, characterized by: The application discloses a multi-degree-of-freedom based speed imagination dual-mode signal acquisition method, and relates to the technical field of signal acquisition.

6. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to realize the multi-degree-of-freedom based speed imagination dual-mode signal acquisition method.

Images