Motor imagery model training, prediction method, device, medium and prediction system

Abstract

The application provides a motor imagery model training and prediction method, device, medium and prediction system. The motor imagery prediction model is used to obtain motor imagery prediction results under different fatigue states based on electroencephalogram signals. The motor imagery prediction model training method comprises the following steps: obtaining electroencephalogram signals and corresponding motor imagery results; calculating the fatigue degree corresponding to the electroencephalogram signals; and training a motor imagery prediction model based on the electroencephalogram signals and the motor imagery results, wherein the motor imagery prediction model sets different loss functions for different fatigue degrees, so that different fatigue degrees correspond to different motor imagery prediction models. The motor imagery prediction model obtained by the method takes into account the influence of the fatigue degree on the motor imagery prediction result, and has high prediction accuracy and stability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer control, and relates to a motor imagery prediction model training method, in particular to a motor imagery model training and prediction method, device, medium and prediction system. BACKGROUND

[0002] A brain-computer interface system is a new technology for realizing information interaction between a human brain and an external device through electroencephalogram signals collected from the surface of a human head. Several main implementation modes of the brain-computer interface system can be divided into a brain-computer interface system based on P300 signals, a motor imagery brain-computer interface system and a steady-state visual evoked potential (SSVEP) brain-computer interface system according to the types of the extracted electroencephalogram signals. Among them, the motor imagery brain-computer interface system has a huge application scenario in the medical rehabilitation field and the non-medical field, and thus is concerned by many researchers.

[0003] A motor imagery task is completed by imagining performing a specific task through a target object without actually performing the task. The motor imagery task widely used in current research is the imagination of body parts such as the right hand, the left hand, the right foot, the left foot, both feet and the tongue, and other related movements such as the elbow, the fist and the fingers are also in research. The motor imagery brain-computer interface system relies on the feedback signal of the target object in actual experiments, and the experimental effect is good at the beginning. However, after the target object performs a motor imagery experiment for a period of time, the reaction time and the imagination quality will greatly decrease due to fatigue, and accurate motor imagery prediction results cannot be obtained. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a motor imagery model training and prediction method, device, medium and prediction system, which is used to solve the problem that the reaction time and the imagination quality greatly decrease due to fatigue in the prior art, and accurate motor imagery prediction results cannot be obtained.

[0005] In a first aspect, the present application provides a motor imagery prediction model training method, the motor imagery prediction model being used to obtain motor imagery prediction results under different fatigue states based on electroencephalogram signals, and the motor imagery prediction model training method comprising: acquiring electroencephalogram signals and corresponding motor imagery results; calculating the fatigue degree corresponding to the electroencephalogram signals; and training a motor imagery prediction model based on the electroencephalogram signals and the motor imagery results, wherein the motor imagery prediction model sets different loss functions for different fatigue degrees, so as to make different fatigue degrees correspond to different motor imagery prediction models.

[0006] The motor imagery prediction model training method provided in the first aspect of the application can obtain the corresponding fatigue degree by using the electroencephalogram signal, and then train the motor imagery prediction model, so that different fatigue degrees correspond to different motor imagery prediction models. The motor imagery prediction model obtained in this way takes into account the influence of fatigue degree on the motor imagery prediction result, and has high prediction accuracy and stability.

[0007] In an implementation form of the first aspect, the calculating the fatigue degree corresponding to the electroencephalogram signal comprises: calculating the energy of the electroencephalogram signal in different rhythms; and obtaining the fatigue degree corresponding to the electroencephalogram signal according to the energy of the electroencephalogram signal in different rhythms.

[0008] In an implementation form of the first aspect, the different rhythms comprise a first rhythm and a second rhythm, the energy of the first rhythm is positively correlated with the fatigue degree of the electroencephalogram signal, and the energy of the second rhythm is negatively correlated with the fatigue degree of the electroencephalogram signal.

[0009] In an implementation form of the first aspect, the fatigue degree corresponding to the electroencephalogram signal is a ratio of the energy of the first rhythm to the energy of the second rhythm.

[0010] In an implementation form of the first aspect, the fatigue degree comprises a wake state, a mild fatigue state and a severe fatigue state, which correspond to a wake state motor imagery prediction model, a mild fatigue state motor imagery prediction model and a severe fatigue state motor imagery prediction model respectively.

[0011] In this implementation form, the motor imagery prediction model corresponding to different fatigue degrees can be directly selected according to the fatigue degree, so as to obtain more accurate prediction results.

[0012] In an implementation form of the first aspect, the loss function is:

[0013]

[0014] wherein E represents the fatigue degree, p(x) is the motor imagery result corresponding to the electroencephalogram signal x, q(x) is the motor imagery prediction result corresponding to the electroencephalogram signal x, and a is the weight coefficient corresponding to the fatigue degree.

[0015] In this implementation form, the greater the fatigue degree, the smaller the value of the loss function, and the initial motor imagery prediction model fitting will be towards the characteristics shown by the electroencephalogram signal with lower fatigue degree. The motor imagery prediction model corresponding to different fatigue degrees obtained in this way is more suitable for the prediction of motor imagery under different fatigue degrees.

[0016] In a second aspect, the present application provides a motor imagery prediction method, comprising: acquiring an electroencephalogram of a target object; acquiring a fatigue degree of the target object according to the electroencephalogram of the target object; and acquiring a motor imagery prediction result corresponding to the electroencephalogram based on a motor imagery prediction model corresponding to the fatigue degree; wherein the motor imagery prediction model is trained by the motor imagery prediction model training method according to any one of the first aspect.

[0017] In the present implementation, the fatigue degree of the target object is acquired according to the electroencephalogram, and the motor imagery prediction result corresponding to the electroencephalogram is acquired based on the motor imagery prediction model corresponding to the fatigue degree. The motor imagery prediction result acquired in this way takes into account the influence of the fatigue degree, and can obtain a more accurate motor imagery prediction result.

[0018] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the electronic device executes the motor imagery prediction model training method according to any one of the first aspect and / or the motor imagery prediction method according to the second aspect.

[0019] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the motor imagery prediction model training method according to any one of the first aspect and / or the motor imagery prediction method according to the second aspect.

[0020] In a fifth aspect, the present application provides a motor imagery prediction system, comprising a brain-computer signal acquisition device and the electronic device according to the third aspect, wherein the brain-computer signal acquisition device is used to acquire an electroencephalogram of a target object and provide the electroencephalogram to the electronic device.

[0021] In summary, the motor imagery model training and prediction method, device, medium and prediction system provided by the present application have the following beneficial effects:

[0022] The motor imagery prediction model training method can acquire a corresponding fatigue degree using an electroencephalogram and then train a motor imagery prediction model, so that different fatigue degrees correspond to different motor imagery prediction models. The motor imagery prediction model acquired in this way takes into account the influence of the fatigue degree on the motor imagery prediction result, and has high prediction accuracy and stability.

[0023] The prediction method of the motor imagery obtains the fatigue degree of the target object according to the electroencephalogram, and obtains the motor imagery prediction result corresponding to the electroencephalogram based on the motor imagery prediction model corresponding to the fatigue degree. The motor imagery prediction result obtained in this way takes into account the influence of the fatigue degree, and can obtain a more accurate motor imagery prediction result. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1A A hardware architecture diagram of a brain-computer interface system in some embodiments is shown.

[0025] Figure 1A A structural schematic diagram of an end-cloud interaction scenario in these embodiments is shown.

[0026] Figure 2 A flowchart of a motor imagery prediction model training method according to an embodiment of the application is shown.

[0027] Figure 3 A flowchart of a fatigue degree obtaining method according to an embodiment of the application is shown.

[0028] Figure 4 A flowchart of a fatigue degree obtaining method according to an embodiment of the application is shown.

[0029] Figure 5 A flowchart of a motor imagery prediction method according to an embodiment of the application is shown.

[0030] Figure 6 A structural schematic diagram of an electronic device according to an embodiment of the application is shown.

[0031] Element Number Explanation

[0032] 10 brain-computer interface system

[0033] 11 electroencephalogram acquisition device

[0034] 12 processor

[0035] 13 external device

[0036] 20 terminal

[0037] 21 cloud server

[0038] 600 electronic device

[0039] 610 memory

[0040] 620 processor

[0041] 630 display

[0042] S11-S13 steps

[0043] S21-S22 steps

[0044] S211-S212 steps

[0045] S31-S32 steps

[0046] S41-S43 steps DETAILED DESCRIPTION

[0047] The present application is described herein with reference to particular non-limiting embodiments. Various modifications and changes can be made thereto by those skilled in the art without departing from the spirit and scope of the application as set forth in the claims. The disclosure is not to be limited to the specific embodiments described but only by the claims.

[0048] It should be noted that the drawings provided in the following embodiments are only to illustrate the basic concept of the present application in a schematic manner, and only the components related to the present application are shown in the drawings, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change in type, number and proportion, and the layout pattern of the components may be more complex.

[0049] Motor imagery tasks are completed by imagining performing a specific task without actually performing it. The motor imagery task widely used in current research is the imagination of body parts such as the right hand, left hand, right foot, left foot, both feet, and tongue. Many other tasks are also under study, such as movements related to the elbow, fist, and fingers. The motor imagery brain-computer interface system relies on the feedback signal of the target object in actual experiments, and the experimental effect is good at the beginning. However, after the target object performs a motor imagery experiment for a period of time, the reaction time and imagination quality are greatly reduced due to fatigue, and accurate motor imagery prediction results cannot be obtained.

[0050] At least for the above problems, the embodiment of the present application provides a motor imagery prediction model training method, the motor imagery prediction model is used for obtaining motor imagery prediction results under different fatigue states based on electroencephalogram signals, and the motor imagery prediction model training method comprises: obtaining electroencephalogram signals and corresponding motor imagery results; calculating the fatigue degree corresponding to the electroencephalogram signals; training the motor imagery prediction model based on the electroencephalogram signals and the motor imagery results, wherein the motor imagery prediction model sets different loss functions for different fatigue degrees, so that different fatigue degrees correspond to different motor imagery prediction models.

[0051] In this embodiment, the motor imagery prediction model training method utilizes electroencephalogram (EEG) signals to obtain corresponding fatigue levels, thereby training the motor imagery prediction model so that different fatigue levels correspond to different motor imagery prediction models. The motor imagery prediction model obtained in this way considers the impact of fatigue levels on the prediction results, exhibiting high prediction accuracy and stability.

[0052] Figure 1A The diagram shows the hardware architecture of a brain-computer interface system in some implementations. This brain-computer interface system can be used to implement the motor imagery prediction model training method provided in the embodiments of this application. However, the application scenarios of the motor imagery prediction model training method provided in the embodiments of this application are not limited to these methods. Figure 1A The brain-computer interface system shown. Figure 1A As shown, the brain-computer interface system 10 specifically includes: an EEG acquisition device 11, a processor 12, and an external device 13. The motor imagery prediction model training method provided in this application embodiment can be implemented by the processor 12. In specific applications, the processor 12 can train multiple motor imagery prediction models corresponding to different fatigue levels. The EEG acquisition device 11 is used to acquire the EEG signals of the target object and send the acquired EEG signals to the processor 12. The processor 12 is used to acquire the motor imagery results of the EEG signals, and based on the EEG signals, acquire the fatigue level of the target object. Based on the motor imagery prediction model corresponding to the fatigue level, it acquires the motor imagery prediction results corresponding to the EEG signals, generates corresponding control commands based on the prediction results, and sends them to the external device 13. The external device 13 receives the control commands and performs corresponding operations according to the control commands.

[0053] in, Figure 1A The processor 12 in the text can be a single processor, a processor cluster consisting of multiple processors, or a cloud computing center, etc., and is not specifically limited here. Although Figure 1A Only one EEG acquisition device 11, one processor 12, and one external device 13 are shown in the image, but it should be understood that... Figure 1A The examples provided are for understanding this solution only; the specific number of external devices and processors should be determined flexibly based on the actual situation.

[0054] In other application scenarios, brain-computer interface systems may not include a separate processor, but only external devices with high computing power and EEG acquisition devices. The motor imagery prediction model training method provided in this application embodiment can be applied to external devices. The external devices with high computing power may include smart wheelchairs, robotic arms, tablet computers, PDAs, mobile phones, personal computers (PCs), and voice interaction devices, etc., and are not limited here.

[0055] In some other implementations, the motor imagery prediction model training method described in the present application can be applied to an end-cloud interaction scenario. Figure 1B A structural schematic diagram of an end-cloud interaction scenario in some implementations is shown. As shown in Figure 1B The end-cloud interaction system includes a terminal 20 and a cloud server 21, and the terminal 20 and the cloud server 21 can communicate with each other. The communication mode is not limited to wired or wireless mode. For example, the motor imagery prediction model is deployed on the cloud server 21. The cloud server 21 obtains the electroencephalogram of a target object, obtains the fatigue degree of the target object according to the electroencephalogram of the target object, selects a corresponding motor imagery prediction model based on the fatigue degree, obtains a motor imagery prediction result corresponding to the electroencephalogram by using the motor imagery prediction model, and feeds back the prediction result to the terminal 20.

[0056] The terminal 20 can be mobile or fixed. For example, the terminal 20 can be a wireless terminal or a wired terminal. The wireless terminal can be a device with wireless transceiver function, which can be deployed indoors, outdoors, or wearable devices. The terminal can be a wheelchair, a mechanical arm, a mobile phone, a tablet computer, a notebook computer, etc., which are not limited herein. The cloud server 21 can include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. The cloud server 21 can also be referred to as a server cluster, a management platform, a data processing center, etc., which are not limited by the embodiments of the present application.

[0057] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0058] The embodiment provides a motor imagery prediction model training method, which can be implemented by, for example, Figure 1A a processor 12 shown in Figure 1B a cloud server 21 shown in. Figure 2 A flowchart of the motor imagery prediction model training method described in the embodiments of the present application is shown in Figure 2 As shown in the flowchart, the motor imagery prediction model training method includes the following steps S11-S13.

[0059] In step S11, an electroencephalogram and a corresponding motor imagery result are obtained. The motor imagery result corresponding to the electroencephalogram is the true motor intention of a target object during motor imagery. Optionally, the target object performs motor imagery of left hand clenched, and the motor imagery result of the obtained electroencephalogram is left hand clenched.

[0060] In some possible implementation manners, the target object first imagines holding a ball with the left hand, holding a ball with the right hand, holding the ground with the feet, and resting, each for 100 times, and completes 10 sets, each set including 40 experiments, and a total of 400 experiments (the sampling interval can be set to once every 2 seconds). A plurality of electroencephalogram signals of the target object are acquired during the experiment, and the electroencephalogram signals and the motor imagery results thereof are acquired by means of artificial labeling.

[0061] In step S12, the fatigue degree corresponding to the electroencephalogram signal is calculated. The fatigue degree is used to represent the fatigue degree of the target object during motor imagery, and the fatigue degree can be calculated by using the electroencephalogram signal.

[0062] In step S13, the motor imagery prediction model is trained based on the electroencephalogram signal and the motor imagery result. The motor imagery prediction model sets different loss functions for different fatigue degrees, so that different fatigue degrees correspond to different motor imagery prediction models.

[0063] In some possible implementation manners, the electroencephalogram signal can also be preprocessed before the motor imagery prediction model is trained based on the electroencephalogram signal and the motor imagery result. The preprocessing of the electroencephalogram signal includes downsampling, filtering, deartifacting, and the like. The signal-to-noise ratio (SNR) of the preprocessed electroencephalogram signal is enhanced, and the real label of the electroencephalogram signal and other related features can be extracted by further operation. The electroencephalogram signal feature extraction can be analyzed from different angles of time domain, frequency domain, and space domain. The time domain analysis feature extraction algorithm includes Hjorth parameters, an adaptive autoregressive (AAR) model, particle filtering, and the like. The frequency domain analysis feature extraction algorithm includes fast fourier transform (FFT), spectral features, wavelet transform (WT), and the like. The space domain analysis feature extraction algorithm is common spatial pattern (CSP), which aims to find a spatial filter set that can maximize the variance ratio of different signals. In addition, there are feature extraction algorithms that combine time domain, frequency domain, and space domain analysis methods, such as wavelet packet transform (WPT). Optionally, compared with other algorithms, the wavelet packet transform algorithm can extract information of multiple time windows and frequency bands, is more suitable for non-stationary electroencephalogram signals than fast fourier transform, and can also obtain related features of the electroencephalogram signal, so as to further obtain training data required for training an initial motor imagery prediction model.

[0064] In some other possible implementations, the motor imagery prediction model comprises a classification model and a feature extraction model. The classification algorithm used by the classification model can be a supervised classification algorithm, an unsupervised classification algorithm, or a semi-supervised classification algorithm. Among them, the samples used by the supervised classification algorithm all have labels, the samples used by the unsupervised classification algorithm all have no labels, and only part of the samples used by the semi-supervised classification algorithm have labels. Optionally, a supervised classification algorithm such as Kalman adaptive linear discriminant analysis (KALDA) based on linear discriminant analysis (LDA), a Bayesian classifier, or a support vector machine (SVM) is used, but the present application is not limited thereto.

[0065] The motor imagery prediction model training method in the embodiments of the present application can use the electroencephalogram signal to obtain the corresponding fatigue degree and then train the motor imagery prediction model, so that different fatigue degrees correspond to different motor imagery prediction models. The motor imagery prediction model obtained in this way takes into account the influence of fatigue degree on the motor imagery prediction result, and has high prediction accuracy and stability.

[0066] Figure 3 The flowchart for obtaining the fatigue degree described in the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, obtaining the fatigue degree of the electroencephalogram signal comprises the following steps S21 and S22. Figure 3

[0067] In step S21, the energy of the electroencephalogram signal at different rhythms is calculated. The electroencephalogram (EEG) is some spontaneous rhythmic neural electrical activity, which is the overall reflection of the electrical physiological activity of brain nerve cells on the cerebral cortex or scalp surface. The energy of the electroencephalogram signal at different rhythms can be used to represent different states of the brain.

[0068] ​At step S22, the fatigue degree corresponding to the electroencephalogram is obtained according to the energy of the electroencephalogram at different rhythms. Since the energy of the electroencephalogram at different rhythms changes with the change of the mental state of the human body, the fatigue degree of the electroencephalogram can be obtained according to the energy of the electroencephalogram at different rhythms in the embodiment of the present application. Specifically, in the process of motor imagery, the rhythm signal of the cerebral cortex changes obviously. When motor imagery is performed, the neuron cells are activated and the speed of metabolism is accelerated, the energy of the electroencephalogram rhythm of the contralateral motor sensory area of the cerebral cortex is obviously reduced, and the energy of the electroencephalogram rhythm of the ipsilateral motor sensory area is increased. This phenomenon is called event related desynchronization (ERD) / event related synchronization (ERS). The ERD / ERS phenomenon has a strong correlation with the real motor intention of the brain, and it is generated by the target object through self-motion imagination without performing actual limb movement or external stimulation. Based on this relationship, the electroencephalogram with different rhythms can be generated by actively controlling the amplitude of the left and right brain rhythms, that is, the fatigue degree of the target object can be determined by the energy of the electroencephalogram with different rhythms.

[0069] In an embodiment of the present application, the different rhythms include a first rhythm and a second rhythm, the energy of the first rhythm is positively correlated with the fatigue degree of the electroencephalogram, and the energy of the second rhythm is negatively correlated with the fatigue degree of the electroencephalogram. The greater the energy of the first rhythm is, the higher the fatigue degree is, and the greater the energy of the second rhythm is, the lower the fatigue degree is.

[0070] Alternatively, the first rhythm can be a delta rhythm (1-4 Hz) and a theta rhythm (4-7 Hz), and the second rhythm can be an alpha rhythm (7-13 Hz) and a beta rhythm (13-25 Hz). The electroencephalogram wave of the delta rhythm (1-4 Hz) is the main frequency component when the brain enters a deep sleep state. When the brain enters a deep sleep state, the breathing gradually deepens, the heartbeat slows down, the blood pressure and body temperature decrease, and the energy of the electroencephalogram rhythm of the delta rhythm (1-4 Hz) is greatly increased. The electroencephalogram wave of the theta rhythm (4-7 Hz) is the main frequency component when the brain is in a conscious dim stage. The electroencephalogram wave of the alpha rhythm (7-13 Hz) is the main frequency component when the brain is in a state of high concentration. The electroencephalogram wave of the beta rhythm (13-25 Hz) is the main frequency component when the brain is in a state of anxiety and tension.

[0071] Please continue to refer to Figure 3 In an embodiment of the present application, the energy spectrum density method can be used to obtain the energy of the electroencephalogram at the delta rhythm, the theta rhythm, the alpha rhythm and the beta rhythm, which specifically includes the following steps S211 and S212.

[0072] Step S211, the power spectral density of the signal spectrum is calculated. The calculation formula of the power spectral density is:

[0073] P(w) = |F(w)| 2 ,

[0074] Wherein, P(w) is the power spectral density of the signal x(t), and F(w) is the frequency spectrum of the signal x(t).

[0075] Step S212, the energy of different rhythms is calculated by using the power spectral density of the signal. The energy calculation formula of the brain electrical signal in the delta rhythm (1-4 Hz), theta rhythm (4-7 Hz), alpha rhythm (7-13 Hz) and beta rhythm (13-25 Hz) is:

[0076]

[0077] Wherein, E δ , E θ , E α and E β respectively represent the energy of the signal x(t) in the delta rhythm (1-4 Hz), the theta rhythm (4-7 Hz), the alpha rhythm (7-13 Hz) and the beta rhythm (13-25 Hz), and P(w) is the power spectral density of the signal x(t).

[0078] In an embodiment of the present application, the fatigue degree corresponding to the brain electrical signal is the ratio of the energy of the first rhythm to the energy of the second rhythm. Figure 4 The flowchart for obtaining the fatigue degree according to the embodiment of the present application is shown in FIG. 3. As shown in FIG. 3, obtaining the fatigue degree of the brain electrical signal comprises the following steps S31 and S32. Figure 4

[0079] Step S31, the energy of the brain electrical signal in different rhythms is calculated.

[0080] Step S32, the fatigue degree of the brain electrical signal is obtained according to the energy of the brain electrical signal in different rhythms. The calculation formula of the fatigue degree is:

[0081]

[0082] Wherein, E is the fatigue degree of the brain electrical signal x(t), E δ is the energy of the delta rhythm, E θ is the energy of the theta rhythm, E α is the energy of the alpha rhythm, and E β is the energy of the beta rhythm.

[0083] ​In an embodiment of the present application, the fatigue degree includes a wakeful state, a mild fatigue state and a severe fatigue state, which correspond to a wakeful state motor imagery prediction model, a mild fatigue state motor imagery prediction model and a severe fatigue state motor imagery prediction model respectively.

[0084] The motor imagery prediction models corresponding to the three fatigue degrees are provided in the embodiments of the present application, and in a specific application, a motor imagery prediction model corresponding to a current fatigue degree can be selected according to the current fatigue degree to predict the motor imagery, so as to obtain a more accurate prediction result.

[0085] In an embodiment of the present application, the loss function is:

[0086]

[0087] wherein E represents the fatigue degree, p(x) is a motor imagery result corresponding to the electroencephalogram x, q(x) is a motor imagery prediction result corresponding to the electroencephalogram x, and a is a weight coefficient corresponding to the fatigue degree.

[0088] In some possible implementations, the weight coefficient a can be set artificially or automatically. For example, in some embodiments, the weight coefficients can be respectively assigned as 1, 2 and 3, and the motor imagery prediction models corresponding to the fatigue degrees obtained at this time are respectively a wakeful state motor imagery prediction model, a mild fatigue state motor imagery prediction model and a severe fatigue state motor imagery prediction model.

[0089] In another possible implementation, the weight coefficient of the fatigue degree can be adjusted according to the needs of an actual application scenario, so as to adapt to different fatigue thresholds of different target objects.

[0090] It should be noted that the above is only two possible implementations of the present application, and the present application is not limited thereto.

[0091] In some possible implementations, when the initial motor imagery prediction model is trained, a gradient descent method can be used to find the optimal parameters of the model. Alternatively, the gradient descent formula is:

[0092] θ = θ 0 - η × ▽ H ( θ 0 ),

[0093] Wherein, θ is the current model parameter, H is the loss function about θ, η is the learning rate (i.e. the step length of each gradient descent). The gradient of the loss function is calculated, and the direction of each parameter descent is along the direction of the fastest gradient descent. After iterative processing, the model parameter with the minimum loss function is obtained. For the same experiment, the greater the fatigue degree of the target object, the greater the fatigue degree E calculated according to the different rhythms of the electroencephalogram, the smaller the value of the loss function, and the smaller the value of the gradient of the loss function. It should be noted that the above is only one possible implementation of the present application, and the present application is not limited thereto.

[0094] In the embodiments of the present application, the greater the fatigue degree, the smaller the value of the loss function, and the movement imagination prediction model fitting will tend to the characteristics shown by the electroencephalogram with lower fatigue degree, and the movement imagination prediction model obtained is more suitable for the prediction of movement imagination corresponding to different fatigue degrees.

[0095] The embodiments of the present application also provide a movement imagination prediction method. Figure 5 The flowchart of the movement imagination prediction method described in the embodiments of the present application is shown. As shown in the figure, Figure 5 The movement imagination prediction method includes the following steps S41 to S43.

[0096] Step S41, the electroencephalogram of the target object is obtained.

[0097] Step S42, the fatigue degree of the target object is obtained according to the electroencephalogram of the target object.

[0098] Step S43, the movement imagination prediction result corresponding to the electroencephalogram is obtained based on the movement imagination prediction model corresponding to the fatigue degree. The movement imagination prediction model can be trained by the movement imagination prediction model training method shown in the figure. Figure 2

[0099] ​Next, the motion imagination prediction method described above will be introduced through a specific application example. In the example, the target object controls the wheelchair device through motion imagination. The electroencephalogram signal acquisition device samples every 2 seconds (consistent with the training data) to obtain the electroencephalogram signal of the target object. The processor obtains the fatigue degree of the target object according to the electroencephalogram signal of the target object, and determines the motion imagination prediction model corresponding to the fatigue degree to process the electroencephalogram signal to obtain the prediction result of motion imagination. The target object can imagine the movement of different body parts according to its own needs. For example, when forward movement is needed, the processor processes the electroencephalogram signal collected by the electroencephalogram acquisition device to obtain the fatigue degree of the target object. Then, the motion imagination prediction model corresponding to the fatigue degree is used to process the collected electroencephalogram signal to obtain the prediction result of motion imagination. The processor sends a command to the external device wheelchair to change from the resting state to the forward state according to the prediction result of motion imagination. The wheelchair receiving the forward command will keep moving forward until the target object stops imagining the movement of the legs to restore the resting state, and the external device will also stop moving forward. For another example, when the wheelchair needs to turn, the processor processes the electroencephalogram signal collected by the electroencephalogram acquisition device to obtain the fatigue degree of the target object. Then, the motion imagination prediction model corresponding to the fatigue degree is used to process the collected electroencephalogram signal to obtain the prediction result of motion imagination. The processor sends a command to the external device wheelchair according to the prediction result of motion imagination. After receiving the command, the wheelchair will execute left turn or right turn until the target object stops imagining left turn or right turn to restore the resting state, and the turning stops.

[0100] Those skilled in the art will further appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or both and that the interchangeability of hardware and software methods is within the scope of the present application. In order to clearly illustrate the interchangeability of hardware and software methods, each example has been described in general terms above as being primarily implemented in either hardware or software. The method of implementation is left to the discretion of the designer as is the choice of specific design to implement the described function.

[0101] The embodiments of the present application also provide an electronic device. Figure 6 The structure schematic diagram of the electronic device described in the embodiments of the present application is shown in FIG. 6. As shown in the figure, the electronic device 600 in the embodiment includes a memory 610 and a processor 620. Figure 6 The memory 610 is used for storing a computer program; preferably, the memory 610 includes a ROM, a RAM, a magnetic disc, a U disk, a memory card, or an optical disc, and various media that can store program codes.

[0102] The memory 610 is used for storing a computer program; preferably, the memory 610 includes a ROM, a RAM, a magnetic disc, a U disk, a memory card, or an optical disc, and various media that can store program codes.

[0103] In particular, the memory 610 can include a computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device 600 can further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 610 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0104] The processor 620 is connected with the memory 610, and is configured to execute the computer program stored in the memory 610, so that the electronic device 600 executes the motor imagery prediction model training method provided in the embodiments of the application and / or the motor imagery prediction method described in other embodiments of the application.

[0105] Optionally, the processor 620 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0106] Optionally, the electronic device 600 in the embodiment can further include a display 630. The display 630 is connected with the memory 610 and the processor 620 in communication, and is configured to display the related GUI interaction interface of the motor imagery prediction model training method described in the embodiments of the application and / or the motor imagery prediction method described in other embodiments of the application.

[0107] The embodiments of the application further provide a computer readable storage medium, which stores a computer program. The program is executed by a processor to implement the motor imagery prediction model training method described in any of the embodiments of the application and / or the motor imagery prediction method described in any of the embodiments of the application.

[0108] Those skilled in the art can understand that all or part of the steps in the method of implementing the above-mentioned embodiments can be instructed by a processor through a program, and the program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid state disk, magnetic tape, floppy disk, optical disc, and any combination thereof. The storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, digital video disc (Digital video disc, DVD)), or a semiconductor medium (for example, solid state disk (Solid state disk, SSD)) and the like.

[0109] The embodiment of the present application also provides a motor imagery prediction system, comprising an electroencephalogram signal acquisition device and the electronic device described in the embodiment of the present application, wherein the electroencephalogram signal acquisition device is used to acquire the electroencephalogram signal of a target object and provide the electroencephalogram signal to the electronic device.

[0110] The description of the flow or structure corresponding to each of the above-mentioned figures has its own emphasis, and the parts not described in detail in a certain flow or structure can be referred to the related description of other flows or structures.

[0111] The above-mentioned embodiments are only illustrative of the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical idea of the present application should be covered by the claims of the present application.

Claims

1. A method for training a motion imagery prediction model, characterized in that, The motor imagery prediction model is used to obtain motor imagery prediction results under different fatigue states based on electroencephalogram (EEG) signals. The training method of the motor imagery prediction model includes: Acquire electroencephalogram (EEG) signals and corresponding motor imagery results; Calculating the fatigue level corresponding to the electroencephalogram (EEG) signal includes: calculating the energy of the EEG signal at different rhythms, and obtaining the fatigue level corresponding to the EEG signal based on the energy of the EEG signal at different rhythms; The motor imagery prediction model is trained based on the EEG signals and the motor imagery results; wherein, the motor imagery prediction model sets different loss functions for different fatigue levels, so that different fatigue levels correspond to different motor imagery prediction models. The loss function is: Where E represents the degree of fatigue, p(x) is the motor imagery result corresponding to EEG signal x, q(x) is the motor imagery prediction result corresponding to EEG signal x, and a is the weighting coefficient corresponding to the degree of fatigue.

2. The method for training a motion imagery prediction model according to claim 1, characterized in that, The different rhythms include a first rhythm and a second rhythm. The energy of the first rhythm is positively correlated with the fatigue level corresponding to the EEG signal, and the energy of the second rhythm is negatively correlated with the fatigue level corresponding to the EEG signal.

3. The method for training a motion imagery prediction model according to claim 2, characterized in that, The fatigue level corresponding to the EEG signal is the ratio of the energy of the first rhythm to the energy of the second rhythm.

4. The method for training a motion imagery prediction model according to claim 1, characterized in that, The fatigue level includes a conscious state, a mild fatigue state, and a severe fatigue state, which correspond to the conscious state motor imagery prediction model, the mild fatigue state motor imagery prediction model, and the severe fatigue state motor imagery prediction model, respectively.

5. A method for predicting motion imagery, characterized in that, include: Acquire the electroencephalogram (EEG) signals of the target subject; The fatigue level of the target object is obtained based on the target object's electroencephalogram (EEG) signals. Based on the motor imagery prediction model corresponding to the fatigue level, the motor imagery prediction result corresponding to the EEG signal is obtained, wherein the motor imagery prediction model is trained using the motor imagery prediction model training method according to any one of claims 1 to 4.

6. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the electronic device to perform the motion imagery prediction model training method as described in any one of claims 1 to 4 and / or the motion imagery prediction method as described in claim 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the motion imagery prediction model training method according to any one of claims 1 to 4 and / or the motion imagery prediction method according to claim 5.

8. A predictive system for motion imagery, characterized in that, Includes an electroencephalogram (EEG) signal acquisition device and the electronic device as described in claim 6; The EEG signal acquisition device is used to acquire the EEG signals of the target object and provide them to the electronic device.

Images