Multi-view feature-based motor imagery classification method and device, equipment and storage medium

Abstract

The invention is applicable to the technical field of motor imagery classification, and provides a motor imagery classification method, device and equipment based on multi-view characteristics and a medium, the method comprises the following steps: performing group frequency spectrum filtering on an original electroencephalogram signal through a preset filter to obtain the multi-view characteristics of the original electroencephalogram signal; performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain combined features; dividing the combined features into a time window set according to a time dimension, wherein time windows in the time window set are not overlapped with each other; extracting a target time feature in each time window, and extracting a global dependency relationship of each time window edge time dimension through a Transform module; and completing classification of the motor imagery according to the global dependency relationship.

Description

Technical Field

[0001] The present application belongs to the field of motion recognition technology, and in particular, relates to a method, device, equipment and storage medium for classifying motion imagery based on multi-view features. Background Art

[0002] The brain is the material basis of all advanced human behaviors. It is composed of a large number of nerve cells, synapses and glial cells. These nerve cells are constantly carrying out spontaneous, rhythmic and comprehensive electrical activities. The generated electric field is transmitted through the volume conductor to form a potential distribution on the scalp. This potential signal with time as the axis is the electroencephalogram (EEG) signal. EEG signals are widely used as a non-invasive EEG signal that records voltage fluctuations in brain neurons. Using the motor imagery experimental paradigm to decode EEG signals can achieve the purpose of identifying human movements.

[0003] Neural activity is coherent and EEG signals are continuous signals. In recent years, some deep learning methods have been applied in the field of EEG signal decoding, which has improved the overall accuracy of imaginary action classification using EEG signals. However, in the existing technology, the extracted features are only processed independently, resulting in low accuracy in imaginary action classification. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for classifying motion imagination based on multi-view features, which can solve the problem in the prior art of only processing the extracted features independently, resulting in poor accuracy in imaginary motion classification.

[0005] In a first aspect, an embodiment of the present application provides a method for classifying motion imagery based on multi-view features, comprising:

[0006] Performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal;

[0007] Performing time-domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain a combined feature;

[0008] Dividing the combined features into a time window set according to the time dimension, wherein each time window in the time window set does not overlap with each other;

[0009] Extract the target time features in each time window, and extract the global dependencies of each time window along the time dimension through the Transformer module;

[0010] The classification of motor imagery is completed according to the global dependency.

[0011] Optionally, the step of performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal includes:

[0012] The original electroencephalogram signal is subjected to group spectrum filtering according to a preset frequency range by a filter group consisting of at least two preset filters, and the multi-view features of the original electroencephalogram signal are obtained, wherein the frequency bands of the preset filters do not overlap with each other.

[0013] Optionally, the step of performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal includes:

[0014] The multi-view features are subjected to time-domain convolution processing by at least two one-dimensional convolution kernels of different sizes to obtain the time features output by each one-dimensional convolution kernel, and then the multi-scale time information of the original electroencephalogram signal is obtained according to the time features.

[0015] Optionally, before the step of dividing the combined features into a set of time windows according to the time dimension, the method further includes:

[0016] The combined features are integrated using a preset spatial convolution layer to obtain target combined features, and the target combined features are used to divide into a set of time windows.

[0017] Optionally, the step of extracting target time features in each time window includes:

[0018] Obtaining a time window sequence of each time window, and calculating a variance of each time window sequence, wherein the variance is used to characterize the time series of the time window;

[0019] Determining the target type with the most relevant time characteristics within the time window based on the time series;

[0020] Target time features are extracted in each time window according to the target type.

[0021] Optionally, the step of extracting the global dependency of each time window along the time dimension through the Transformer module includes:

[0022] Inputting the time window into the Transformer module, obtaining query data of the Transformer module and key data of the Transformer module, calculating dot product data of the query data and the key data, and evaluating the correlation between different sequences of the time window according to the dot product data to obtain an evaluation result;

[0023] Obtaining an attention score of the value data of the Transformer module through a Softmax function in the Transformer module according to the evaluation result;

[0024] The dot product data is called to weight the attention score of the value data to obtain the global dependency of the time window along the time dimension.

[0025] Optionally, the step of completing the classification of motor imagery according to the global dependency relationship comprises:

[0026] Inputting the global dependency into the preset flattening layer to obtain flattened data;

[0027] The flattened data is classified through a fully connected layer to obtain a classification result of motor imagery.

[0028] On the other hand, the present application provides a motion imagery classification device based on multi-view features, the device comprising:

[0029] A data acquisition module, used for performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal;

[0030] A time domain convolution module, used for performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original EEG signal, and adding the multi-scale time information and the multi-view features to obtain a combined feature;

[0031] A division module, used for dividing the combined features into a time window set according to the time dimension, wherein each time window in the time window set does not overlap with each other;

[0032] The feature extraction module is used to extract the target time features in each time window and extract the global dependencies of each time window along the time dimension through the Transformer module;

[0033] A classification module is used to complete the classification of motor imagery according to the global dependency.

[0034] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned motion imagery classification method based on multi-view features when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the motion imagery classification method based on multi-view features as described above is implemented.

[0036] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0037] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0038] The original electroencephalogram signal is subjected to group spectrum filtering by a preset filter to obtain multi-view features of the original electroencephalogram signal; the multi-view features are subjected to time domain convolution processing to obtain multi-scale time information of the original electroencephalogram signal, and the multi-scale time information and the multi-view features are added to obtain combined features; the combined features are divided into a set of time windows according to the time dimension, and the time windows in the set of time windows do not overlap with each other; the target time features are extracted from each time window, and the global dependency of each time window along the time dimension is extracted by the Transformer module; and the classification of motor imagery is completed according to the global dependency. The original electroencephalogram signal is subjected to group spectrum filtering by a preset filter to remove the noise in the original signal to a great extent, the time domain features of the original electroencephalogram signal are extracted by performing time domain convolution processing on the multi-view features, the combined features are divided into a set of time windows according to the time dimension, and the target time features are extracted in each time window to locate the frequency band discrimination information related to motor imagery, so as to avoid the extraction of features carried by invalid information, which would waste computing power and affect the accuracy of motor imagery classification, and the global dependency of each time window along the time dimension is extracted, that is, both time domain and spatial domain feature representations are extracted, and the classification accuracy is significantly improved by focusing on the global dependency of features. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is an application scenario diagram of a motion imagery classification method based on multi-view features provided in an embodiment of the present application;

[0041] Figure 2 It is a flowchart of a method for classifying motion imagery based on multi-view features provided by an embodiment of the present application;

[0042] Figure 3 is a flowchart of a method for classifying motor imagery based on multi-view features provided by another embodiment of the present application;

[0043] Figure 4is a structural schematic diagram of a motion imagery classification device based on multi-view features provided by an embodiment of the present application;

[0044] Figure 5 is a structural block diagram of an electronic device in one embodiment;

[0045] Figure 6 FIG. 4 is a structural block diagram of a computer storage medium in another embodiment. DETAILED DESCRIPTION

[0046] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0047] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0048] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0050] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0051] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0052] The motion imagery classification method based on multi-view features provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In an application environment, the EEG signal acquisition device 100 communicates with the server 200, the server 120 obtains the original EEG signal sent by the EEG signal acquisition device 100, and performs group spectrum filtering on the original EEG signal through a preset filter to obtain the multi-view feature of the original EEG signal; the multi-view feature is subjected to time domain convolution processing to obtain multi-scale time information of the original EEG signal, and the multi-scale time information and the multi-view feature are added to obtain the combined feature; the combined feature is divided into a time window set according to the time dimension, and the time windows in the time window set do not overlap with each other; the target time feature is extracted in each time window, and the global dependency of each time window along the time dimension is extracted through the Transformer module; the classification of motor imagery is completed according to the global dependency. The original electroencephalogram signal is subjected to group spectrum filtering by a preset filter to remove the noise in the original signal to a great extent, the time domain features of the original electroencephalogram signal are extracted by performing time domain convolution processing on the multi-view features, the combined features are divided into a set of time windows according to the time dimension, and the target time features are extracted in each time window to locate the frequency band discrimination information related to motor imagery, so as to avoid the extraction of features carried by invalid information, which would waste computing power and affect the accuracy of motor imagery classification, and the global dependency of each time window along the time dimension is extracted, that is, both time domain and spatial domain feature representations are extracted, and the classification accuracy is significantly improved by focusing on the global dependency of features.

[0053] Figure 2 A schematic flowchart of the motion imagery classification method based on multi-view features provided in the present application is shown. As an example but not a limitation, the method can be applied to the above-mentioned server 200, that is, the server 200 is used as a server for motion imagery classification based on multi-view features.

[0054] The present application embodiment provides a method for classifying motion imagery based on multi-view features, including:

[0055] S101, performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal;

[0056] In a possible implementation, the step of performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal includes:

[0057] The original electroencephalogram signal is subjected to group spectrum filtering according to a preset frequency range by a filter group consisting of at least two preset filters, and the multi-view features of the original electroencephalogram signal are obtained, wherein the frequency bands of the preset filters do not overlap with each other.

[0058] Exemplarily, the original EEG signal is spectrally filtered from 4-40 Hz by a filter group consisting of N narrow-band filters to obtain multi-view features. The frequency bands of the narrow-band filters do not overlap. The narrow-band filters use Chebyshev II filters with a transition bandwidth of 2 Hz and a stopband ripple of -30 dB.

[0059] S102, performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain a combined feature;

[0060] In a possible implementation, the step of performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal includes:

[0061] The multi-view features are subjected to time-domain convolution processing by at least two one-dimensional convolution kernels of different sizes to obtain the time features output by each one-dimensional convolution kernel, and then the multi-scale time information of the original electroencephalogram signal is obtained according to the time features.

[0062] Exemplarily, the obtained multi-view features are convolved in the time domain, and one-dimensional convolution kernels of different sizes are used. For example, the one-dimensional convolution kernels are (1, 16), (1, 32), and (1, 64) to learn the time features of the multi-view features to obtain the multi-scale time information of the original EEG signal, and the extracted multi-scale time information and multi-view features are added to obtain the combined features.

[0063] S103, dividing the combined features into a time window set according to the time dimension, wherein each time window in the time window set does not overlap with each other;

[0064] Exemplarily, the process of dividing the combined feature into a set of time windows according to the time dimension can be simply understood as dividing the time axis of the combined feature according to the time window width to obtain a plurality of non-overlapping time windows.

[0065] S104, extracting target time features in each time window, and extracting the global dependency of each time window along the time dimension through the Transformer module;

[0066] In a possible implementation, the step of extracting target time features in each time window includes:

[0067] Obtaining a time window sequence of each time window, and calculating a variance of each time window sequence, wherein the variance is used to characterize the time series of the time window;

[0068] Determining the target type with the most relevant time characteristics within the time window based on the time series;

[0069] Target time features are extracted in each time window according to the target type.

[0070] Exemplarily, the time series is characterized by calculating the variance of each time window sequence, thereby extracting the most relevant time feature (target time feature) for each window and reducing the dimension.

[0071] In a possible implementation, the step of extracting the global dependencies of each time window along the time dimension through the Transformer module includes:

[0072] Inputting the time window into the Transformer module, obtaining query data of the Transformer module and key data of the Transformer module, calculating dot product data of the query data and the key data, and evaluating the correlation between different sequences of the time window according to the dot product data to obtain an evaluation result;

[0073] Obtaining an attention score of the value data of the Transformer module through a Softmax function in the Transformer module according to the evaluation result;

[0074] The dot product data is called to weight the attention score of the value data to obtain the global dependency of the time window along the time dimension.

[0075] Exemplarily, a Transformer module based on a multi-head attention mechanism is used to extract features from all time windows, that is, the correlation between different sequences is evaluated by using dot products on the query (Q) and the key (K), and the result is obtained by a weighted matrix, that is, the attention score, through a Softmax function. The attention score of the value (V) is then weighted by the dot product, thereby extracting the global dependencies of all time windows along the time dimension.

[0076] S105, completing the classification of motor imagery according to the global dependency relationship.

[0077] The original electroencephalogram signal is subjected to group spectrum filtering by a preset filter to remove the noise in the original signal to a great extent, the time domain features of the original electroencephalogram signal are extracted by performing time domain convolution processing on the multi-view features, the combined features are divided into a set of time windows according to the time dimension, and the target time features are extracted in each time window to locate the frequency band discrimination information related to motor imagery, so as to avoid the extraction of features carried by invalid information, which would waste computing power and affect the accuracy of motor imagery classification, and the global dependency of each time window along the time dimension is extracted, that is, both time domain and spatial domain feature representations are extracted, and the classification accuracy is significantly improved by focusing on the global dependency of features.

[0078] In a possible implementation manner, before the step of dividing the combined features into a set of time windows according to the time dimension, the method further includes:

[0079] The combined features are integrated using a preset spatial convolution layer to obtain target combined features, and the target combined features are used to divide into a set of time windows.

[0080] Exemplarily, the combined features are integrated using a spatial convolutional layer with m output channels and a kernel size of (C, 1), where C represents the number of electrode channels of the original EEG signal.

[0081] In a possible implementation, the step of completing the classification of motor imagery according to the global dependency relationship includes:

[0082] Inputting the global dependency into the preset flattening layer to obtain flattened data;

[0083] The flattened data is classified through a fully connected layer to obtain a classification result of motor imagery.

[0084] like Figure 3As shown, the original EEG signal data (original EEG signal) is obtained, the original EEG signal data is spectrally filtered through a filter group to obtain a multi-view representation (spectral feature), the multi-view representation is convolved in the time domain to obtain multi-scale time information (time domain feature), the multi-view representation and the multi-scale time information are combined to obtain a combined feature, the combined feature is convolved in the spatial domain to obtain a target combined feature, the target combined feature is divided into a set of time windows according to the time dimension, the variance of each time window sequence is calculated, and the variance is used to characterize the time series of the time window; the target type of the most relevant time feature in the time window is determined based on the time series; the target time feature is extracted in each time window according to the target type, and the target time feature is input into the Transformer module to extract the global dependency of each time window along the time dimension, and the global dependency is passed through the flattening layer and the fully connected layer to obtain the classification result.

[0085] On the other hand, Figure 4 As shown, an embodiment of the present application provides a motion imagery classification device based on multi-view features, the device comprising:

[0086] The data acquisition module 201 is used to perform group spectrum filtering on the original EEG signal through a preset filter to obtain multi-view features of the original EEG signal;

[0087] A time domain convolution module 202 is used to perform time domain convolution processing on the multi-view features to obtain multi-scale time information of the original EEG signal, and add the multi-scale time information and the multi-view features to obtain a combined feature;

[0088] A division module 203 is used to divide the combined features into a time window set according to the time dimension, and each time window in the time window set does not overlap with each other;

[0089] A feature extraction module 204 is used to extract target time features in each time window and extract the global dependency of each time window along the time dimension through a Transformer module;

[0090] The classification module 205 is used to complete the classification of motor imagery according to the global dependency relationship.

[0091] Thirdly, Figure 5As shown, an embodiment of the present application provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal; performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain combined features; dividing the combined features into a set of time windows according to the time dimension, wherein the time windows in the set of time windows do not overlap with each other; extracting the target time features in each time window, and extracting the global dependency of each time window along the time dimension through the Transformer module; and completing the step of classifying motor imagery according to the global dependency.

[0092] Fourthly, Figure 6 As shown, an embodiment of the present application provides a computer-readable storage medium 400, which stores a computer program 411. When the computer program is executed by a processor, the following steps are implemented: performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal; performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain combined features; dividing the combined features into a time window set according to the time dimension, and the time windows in the time window set do not overlap with each other; extracting the target time feature in each time window, and extracting the global dependency of each time window along the time dimension through the Transformer module; and completing the step of classifying motor imagery according to the global dependency.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0096] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment / terminal equipment and methods can be implemented in other ways. For example, the device / network equipment / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A motion imagery classification method based on multi-view features, characterized in that: include: Performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal; Performing time-domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal, and adding the multi-scale time information and the multi-view features to obtain a combined feature; Dividing the combined features into a time window set according to the time dimension, wherein each time window in the time window set does not overlap with each other; Extract the target time features in each time window, and extract the global dependencies of each time window along the time dimension through the Transformer module; The classification of motor imagery is completed according to the global dependency.

2. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: The step of performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal includes: The original electroencephalogram signal is subjected to group spectrum filtering according to a preset frequency range by a filter group consisting of at least two preset filters, and the multi-view features of the original electroencephalogram signal are obtained, wherein the frequency bands of the preset filters do not overlap with each other.

3. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: The step of performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original electroencephalogram signal comprises: The multi-view features are subjected to time-domain convolution processing by at least two one-dimensional convolution kernels of different sizes to obtain the time features output by each one-dimensional convolution kernel, and then the multi-scale time information of the original electroencephalogram signal is obtained according to the time features.

4. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: Before the step of dividing the combined features into a set of time windows according to the time dimension, the method further includes: The combined features are integrated using a preset spatial convolution layer to obtain target combined features, and the target combined features are used to divide into a set of time windows.

5. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: The step of extracting target time features in each time window includes: Obtaining a time window sequence of each time window, and calculating a variance of each time window sequence, wherein the variance is used to characterize the time series of the time window; Determining the target type with the most relevant time characteristics within the time window based on the time series; Target time features are extracted in each time window according to the target type.

6. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: The step of extracting the global dependency of each time window along the time dimension through the Transformer module includes: Inputting the time window into the Transformer module, obtaining query data of the Transformer module and key data of the Transformer module, calculating dot product data of the query data and the key data, and evaluating the correlation between different sequences of the time window according to the dot product data to obtain an evaluation result; Obtaining an attention score of the value data of the Transformer module through a Softmax function in the Transformer module according to the evaluation result; The dot product data is called to weight the attention score of the value data to obtain the global dependency of the time window along the time dimension.

7. The method for classifying motor imagery based on multi-view features as claimed in claim 1, characterized in that: The step of completing the classification of motor imagery according to the global dependency relationship comprises: Inputting the global dependency into the preset flattening layer to obtain flattened data; The flattened data is classified through a fully connected layer to obtain a classification result of motor imagery.

8. A motion imagery classification device based on multi-view features, characterized in that: The device comprises: A data acquisition module, used for performing group spectrum filtering on the original electroencephalogram signal through a preset filter to obtain multi-view features of the original electroencephalogram signal; A time domain convolution module, used for performing time domain convolution processing on the multi-view features to obtain multi-scale time information of the original EEG signal, and adding the multi-scale time information and the multi-view features to obtain a combined feature; A division module, used for dividing the combined features into a time window set according to the time dimension, wherein each time window in the time window set does not overlap with each other; The feature extraction module is used to extract the target time features in each time window and extract the global dependencies of each time window along the time dimension through the Transformer module; A classification module is used to complete the classification of motor imagery according to the global dependency.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images