Fine feedback method and feedback system based on motor imagery and storage medium

Abstract

The invention discloses a fine feedback method and feedback system based on motor imagery, and a storage medium. The method comprises the following steps: collecting first electroencephalogram data of a user in a first preset time period after a motor imagery starting prompt; performing sliding time window processing on the first electroencephalogram data; calculating amplitude deviation and relative power change percentage for all time windows of the first electroencephalogram data; calculating gradients of all time windows of the first electroencephalogram data; determining whether the user state enters a motor imagery task starting state or not according to the gradient, and if the user state enters the motor imagery task starting state, determining the duration of a motor imagery continuous state according to the gradient; according to the amplitude deviations corresponding to all the sliding time windows of the first electroencephalogram data and the relative power change percentage, whether the user state enters a motor imagery task ending state or not is determined; and generating a feedback signal according to the state of the user. According to the method and the device, finer and richer feedback information can be provided, so that the detection accuracy of each stage of motor imagery is improved.

Description

Technical Field

[0001] This application belongs to the field of motion imagination technology, specifically relating to a fine feedback method, feedback system, and storage medium based on motion imagination. Background Technology

[0002] Currently, brain-computer interface systems based on the motor imagery paradigm mostly provide feedback on motor imagery training by driving external devices to deliver primarily visual motor stimuli to the patient. This feedback is typically designed as a single motor process (e.g., a single fist clenching motion). However, this feedback method has significant limitations. The current feedback provides relatively granular information, reflecting only the overall motor completion status, but failing to describe the patient's specific performance during the imagery execution process, resulting in a gap between the feedback and the actual task execution.

[0003] This makes it difficult for patients to obtain accurate feedback on their imagined task execution process and to understand the details of their performance in each state. Secondly, the current feedback design is fixed and cannot provide further information, which can easily cause patients to lose interest in the feedback. In addition, the single-session motor feedback format limits patients' understanding and mastery of complex motor imagery, which is not conducive to the training effect of multi-step, continuous motor tasks. Summary of the Invention

[0004] The technical objective of this application is to address the aforementioned technical problems existing in current motor imagery feedback mechanisms by providing a refined feedback method, feedback system, and storage medium based on motor imagery, in order to provide more refined and richer feedback information, help patients better understand and adjust their imagery process, thereby improving the effectiveness and relevance of training.

[0005] To achieve the above technical objectives, this application adopts the following technical solution.

[0006] In a first aspect, embodiments of this application provide a fine feedback method based on motion imagery, comprising: Collect the first EEG data of the user during the first preset time period after the prompt to begin motor imagery; The first EEG data is processed using a sliding time window; for all time windows of the first EEG data, the amplitude deviation and the percentage change in relative power are calculated. The gradient is calculated based on the first EEG data; the user's state is determined based on the gradient to determine whether the user has entered the motor imagery task start state. If the user has entered the motor imagery task start state, the duration of the motor imagery state is determined based on the gradient. Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the first EEG data, determine whether the user has entered the end state of the motor imagery task. Feedback signals are generated based on the user's start state of the motion visualization task, the duration of the motion visualization state, and the end state of the motion visualization task.

[0007] Furthermore, the method also includes: collecting second EEG data from the user during a second preset time period before the prompting to begin motor imagery; The second EEG data was processed using a sliding time window; Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the second EEG data, determine whether the user has entered the preparation state for the motor imagery task. If it is determined that the user has entered the preparation state for the motion visualization task, then the gradient is used to determine whether the user's state has entered the start state of the motion visualization task.

[0008] Furthermore, the duration of the second preset time period is shorter than the duration of the first preset time period.

[0009] Furthermore, based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the second EEG data, it is determined whether the user has entered the preparation state for the motor imagery task, including: From all time windows of the second EEG data, the first time window that satisfies the negative direction of the amplitude deviation and the corresponding percentage of relative power change meets the judgment condition corresponding to the first threshold is selected as the first target time window. The first target time window indicates that the user is in the preparation state for the motor imagery task.

[0010] Furthermore, based on the gradient, it is determined whether the user has entered the motion imagery task start state. If the user has entered the motion imagery task start state, the duration of the motion imagery state is determined based on the gradient, including: If the gradient within the time window of the number of consecutive targets meets the judgment condition corresponding to the second threshold, then the first time window in the time window of the number of consecutive targets is taken as the second target time window. The second target time window indicates that the user state has entered the motion imagination task start state. The total duration of the time window of the number of consecutive targets is the duration of the motion imagination continuous state.

[0011] Further, based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the first EEG data, it is determined whether the user state has entered the motor imagery task completion state, including: From all time windows of the first EEG data, the first time window that satisfies the positive direction of the amplitude deviation and the corresponding relative power change percentage meets the judgment condition of the third threshold is selected as the third target time window. The third target time window indicates that the user's state has entered the end state of the motor imagery task.

[0012] Furthermore, the third target time window is used as the start time for feedback reset.

[0013] Furthermore, the fine feedback method also includes: if it cannot be determined that the user's state has entered the preparation state for the motion imagination task, then no feedback signal is generated.

[0014] Secondly, embodiments of this application provide a fine feedback system based on motion imagery, comprising: The EEG data acquisition module is configured to acquire the first EEG data of the user during the first preset time period after the prompting to begin motor imagery. The data preprocessing module is configured to perform sliding time window processing on the first EEG data; and to calculate the amplitude deviation and the percentage of relative power change for all time windows of the first EEG data. The user state determination module is configured to calculate a gradient based on the first EEG data; determine whether the user state has entered the motor imagery task start state according to the gradient; if the user state has entered the motor imagery task start state, determine the duration of the motor imagery state according to the gradient. Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the first EEG data, determine whether the user has entered the end state of the motor imagery task. The status feedback module is configured to generate feedback signals based on the user's motion imagery task start state, motion imagery duration, and motion imagery task end state.

[0015] Thirdly, embodiments of this application provide a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the fine feedback method based on motion imagination as provided in any possible implementation of the first aspect.

[0016] Compared to existing technologies, the fine feedback method based on motor imagery provided in this application collects EEG data after the user is prompted to begin motor imagery and processes it using a sliding time window. By calculating the amplitude deviation and percentage change in relative power within each time window, the system can monitor in real time whether the user has entered the preparation, start, and end stages of motor imagery. This real-time monitoring provides immediate feedback to the user, helping them to self-regulate and optimize the motor imagery process. It allows for more detailed analysis of changes in EEG signals, thereby improving the detection accuracy of each stage of motor imagery (such as the start, duration, and end stages of the motor imagery task). Based on the changes in EEG data of different users, the system can provide personalized feedback and guidance, thereby enhancing the effectiveness of motor imagery training.

[0017] In some embodiments, EEG data prior to the prompting to begin motor imagery is also collected and processed using a sliding time window. The amplitude deviation and percentage change in relative power within each time window are calculated to detect the preparation phase of the motor imagery task. By pre-detecting EEG signals before the prompting to begin motor imagery to filter for valid states, invalid EEG data from when the user is not in a ready state can be accurately avoided. This avoids meaningless feedback interfering with the user's attention and diminishing the experience, while ensuring that feedback focuses only on valid signals after the user is ready for motor imagery. This significantly improves the relevance and accuracy of the feedback, while reducing resource waste from processing invalid data. It helps users establish a connection between motor imagery and feedback more quickly, thereby improving training efficiency or the reliability of brain-computer interface applications.

[0018] Compared with the prior art, the fine feedback system based on motion imagination provided in this application has the same beneficial technical effects as the fine feedback method based on motion imagination described above, and will not be described in detail.

[0019] It should be understood that the summary section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this application in any way. Furthermore, the shapes and scales of the components in the drawings are merely illustrative to aid in understanding this application and do not specifically limit the shapes and scales of the components. Those skilled in the art, guided by the teachings of this application, can select various possible shapes and scales to implement this application according to specific circumstances. In the drawings: Figure 1 A schematic diagram of the fine feedback method based on motion imagery provided in the embodiments of this application; Figure 2 This is a schematic diagram of EEG data acquisition in an embodiment of this application; Figure 3 A schematic diagram of the structure of a fine feedback system based on motion imagination provided in an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0022] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features.

[0023] It should be fully understood that the user information involved in this application (including but not limited to user physiological sensor information, user personal information, etc.) is information and data authorized by the user or fully authorized by all parties. The use of user information shall comply with the privacy policies and practices of the industry that are generally considered to meet or exceed the requirements for maintaining user privacy. The collection, use and processing of related data shall comply with relevant laws, regulations and standards, and provide corresponding operation access points for users to choose to authorize or refuse.

[0024] The fine feedback method based on motor imagery provided in this application includes: collecting first EEG data of the user during a first preset time period after the prompt to begin motor imagery; performing sliding time window processing on the first EEG data; calculating the amplitude deviation and the percentage change in relative power for all time windows of the first EEG data; calculating the gradient based on the first EEG data; determining whether the user has entered the motor imagery task start state based on the gradient; if the user has entered the motor imagery task start state, determining the duration of the motor imagery state based on the gradient; determining whether the user has entered the motor imagery task end state based on the amplitude deviation and the percentage change in relative power corresponding to all sliding time windows of the first EEG data; and generating a feedback signal based on the user's motor imagery task start state, duration of the motor imagery state, and end state.

[0025] In specific embodiments, such as Figure 1 As shown, a fine feedback method based on motion imagery includes: S100. Collect the user's EEG data in a resting state as baseline data for subsequent baseline correction. S200: The user begins to perform a motion visualization task (e.g., imagine making a fist with the left hand once) according to the motion visualization start prompt, and the first preset time after the motion visualization start prompt is collected.

[0026] like Figure 2 As shown in the embodiment, the first preset time, such as 3 seconds, is used for subsequent related signal analysis.

[0027] The data from the first preset time after the motion visualization prompt is used to confirm the ERD (Electronic Resonance Detection) and ERS (Event-related Synchronization) signals.

[0028] After preprocessing, the S300 EEG data is used to calculate relevant brain region energy ERD and ERS related indicators, including ERD start time, ERD gradient information, ERD duration, and ERS start time. Based on these indicators, the motor state is divided into motor imagery task preparation state, motor imagery task start state, motor imagery task duration state, and motor imagery task end state.

[0029] In a specific embodiment, step S300 includes the following steps: S301. Preprocess the first EEG data acquired in S200, including: windowing using a sliding time window of A milliseconds, and calculating the EEG signal amplitude deviation and relative power change percentage for each time window. In this embodiment, the baseline data under resting state is subtracted from the EEG data within the time window to obtain the baseline-corrected first EEG data.

[0030] In the embodiment, the amplitude deviation can be obtained by subtracting the average amplitude of the previous time window from the average amplitude of the current time window; the ratio of the difference between the relative power of the signal in the frequency band of interest in the current time window and the relative power of the signal in the same frequency band of interest in the previous time window to the relative power of the signal in the frequency band of interest in the current time window yields the percentage change in relative power.

[0031] Step 301 can smooth the data and reduce data variability, ensuring stable data quality and reducing interference. S302. ERD gradient information and duration of motor imagery task confirmation: Calculate the corresponding gradient of the EEG data after the motor imagery start prompt. If the gradient does not exceed the set second threshold D within the number of consecutive targets (e.g., C time windows), the first time window (i.e. the second target time window) in the C consecutive time windows is taken as the start state flag of the motor imagery task, and the total length of the C consecutive time windows (i.e., the ERD duration) is taken as the duration of the motor imagery state.

[0032] In some embodiments, gradient calculation includes at least one of the following: (1) Directly calculate the gradient of EEG data between sampling points within a single time window and / or between adjacent time windows in the time domain, and observe the amplitude change pattern; (2) Convert the EEG data to the frequency domain, calculate the gradient information of the EEG data in the frequency domain, and mainly observe the frequency changes; (3) Calculate the gradient information of EEG data between multiple leads and observe the changes in EEG data in different regions.

[0033] In this embodiment, the start of the motion visualization task is determined by: if one or more gradients are calculated, one gradient or several gradients can be selected, and it can be determined that the gradient does not exceed the corresponding set threshold within C consecutive time windows; or all gradients can be selected, and it can be determined that each gradient does not exceed the corresponding set threshold within C consecutive time windows.

[0034] S303, ERS Start Time Confirmation: Analyze the EEG data calculation results after the start of the motor imagery task. If the amplitude deviation direction is positive and the percentage change in relative power meets the judgment condition corresponding to the third threshold E, select the first positive change signal time window (i.e. the third target time window) in the EEG signal segment as the ERS start time. This time indicates that the user is in the state of the end of the motor imagery task. In some embodiments, step 200 further includes: acquiring second EEG data of the user during a second preset time period prior to the prompting of motor imagery. The second EEG data can be used to confirm pre-motor ERD (Event-related Desynchronization) signals.

[0035] Step 301 also includes preprocessing the second EEG data using the same preprocessing method as the first EEG data.

[0036] like Figure 1 As shown, in some embodiments, step 302 further includes ERD start time confirmation: based on the baseline-corrected second EEG data, the data calculation results of a second preset time (the second preset time can be less than or lower than the first preset time, such as 2 seconds) before the start of the motor imagery task (i.e., before the prompt) are analyzed. If the amplitude deviation direction is negative and the percentage of relative power change meets the judgment condition corresponding to the set first threshold B, the signal time window of the first negative change in the signal segment is selected as the ERD start time. This time indicates that the user is in the motor imagery task preparation state before performing motor imagery. If the ERD start time cannot be confirmed, it is considered that the user has not performed the corresponding task preparation.

[0037] If it is determined that the user has entered the preparation state for the motion imagery task, then the gradient is used to determine whether the user has entered the start state of the motion imagery task; if it cannot be determined that the user has entered the preparation state for the motion imagery task, no feedback signal is generated.

[0038] S400. Based on the above results, the ERD start time is used to confirm whether the user is preparing for motion visualization. If it cannot be confirmed, no further feedback is provided. If the ERD start time is confirmed, feedback is provided. The motion visualization task start state flag is used as the start time of the feedback. The same duration of continuous motion feedback is provided as the duration of the motion visualization task. The ERS start time is used as the start of the feedback reset.

[0039] In some embodiments, the feedback signals can be visual signals (such as images, color changes, or motion graphics on a screen), auditory signals (such as sounds, music, or voice prompts), or other forms of sensory stimulation. These feedback signals should be able to visually reflect changes in the user's brain electrical activity and help the user to self-regulate and optimize.

[0040] In some embodiments, the fine feedback method based on motion imagery further includes: outputting a feedback signal to a display device, and using the display device to display an image, color change, or motion graphics to indicate the user's state.

[0041] In some embodiments, the fine feedback method based on motion imagery further includes: outputting a feedback signal to an audio output device, and using the audio output device to output an audio signal to indicate the user's state.

[0042] Audio output devices can include voice prompts, sound systems, speakers, and loudspeakers. For example, after receiving feedback signals, the voice prompts perform signal recognition and processing, and then use control circuitry to decode, amplify, or convert the signals according to preset programs and logic in order to generate corresponding voice prompts.

[0043] The refined feedback method based on motor imagery provided in this application implements an improved feedback mechanism. This mechanism can prevent users from losing attention to single and fixed forms of feedback, and provides rich and diverse feedback information. It can stimulate individual interest, maintain their active participation, and thus improve the continuity and effectiveness of training. By providing detailed and phased feedback information, users can better understand their specific performance in task execution, thereby improving the targeting and effectiveness of training.

[0044] Based on the same inventive concept as the fine feedback method based on motion imagination provided in the above embodiments, this application also provides a fine feedback system based on motion imagination, such as... Figure 3 As shown, it includes an EEG data acquisition module, a data preprocessing module, a user status determination module, and a status feedback module.

[0045] The EEG data acquisition module is configured to collect the first EEG data of the user during the first preset time period after the prompt to begin motor imagery.

[0046] The data preprocessing module is configured to perform sliding time window processing on the first EEG data; for all time windows of the first EEG data, the amplitude deviation and the percentage change in relative power are calculated.

[0047] The user state determination module is configured to calculate gradients based on the first EEG data; determine whether the user has entered the motor imagery task start state based on the gradient; if the user has entered the motor imagery task start state, determine the duration of the motor imagery state based on the gradient. Based on the amplitude deviation and relative power change percentage corresponding to all sliding time windows of the first EEG data, determine whether the user has entered the motor imagery task end state.

[0048] The status feedback module is configured to generate feedback signals based on the user's motion visualization task start state, motion visualization duration, and motion visualization task end state.

[0049] The fine feedback system based on motor imagery provided in this application embodiment can provide richer feedback based on the user's actual performance, which helps the user to train.

[0050] This application also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the fine feedback method based on motion imagination as provided in the above embodiments.

[0051] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, or a tablet computer, or any combination of these devices.

[0052] The above provides a detailed description of the fine feedback method, feedback system, and storage medium based on motion imagination provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the concept of this application and should not be construed as limiting the scope of protection of this application.

Claims

1. A fine feedback method based on motion imagery, characterized in that, include: Collect the first EEG data of the user during the first preset time period after the prompt to begin motor imagery; The first EEG data is processed using a sliding time window; For all time windows of the first EEG data, calculate the amplitude deviation and the percentage change in relative power; The gradient is calculated based on the first EEG data; the user's state is determined based on the gradient to determine whether the user has entered the motor imagery task start state. If the user has entered the motor imagery task start state, the duration of the motor imagery state is determined based on the gradient. Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the first EEG data, determine whether the user has entered the end state of the motor imagery task. Feedback signals are generated based on the user's start state of the motion visualization task, the duration of the motion visualization state, and the end state of the motion visualization task.

2. The fine feedback method based on motion imagery according to claim 1, characterized in that, The method further includes: collecting second EEG data from the user during a second preset time period before the prompting to begin motor imagery; The second EEG data was processed using a sliding time window; Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the second EEG data, determine whether the user has entered the preparation state for the motor imagery task. If it is determined that the user has entered the preparation state for the motion visualization task, then the gradient is used to determine whether the user's state has entered the start state of the motion visualization task.

3. The fine feedback method based on motion imagery according to claim 2, characterized in that, The duration of the second preset time period is less than the duration of the first preset time period.

4. The fine feedback method based on motion imagery according to claim 2, characterized in that, Based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the second EEG data, determine whether the user has entered the preparation state for the motor imagery task, including: From all time windows of the second EEG data, the first time window that satisfies the negative direction of the amplitude deviation and the corresponding percentage of relative power change meets the judgment condition of the first threshold is selected as the first target time window. The first target time window indicates that the user is in the preparation state for the motor imagery task.

5. The fine feedback method based on motion imagery according to claim 1, characterized in that, The gradient is used to determine whether the user has entered the motion imagery task initiation state. If the user has entered the motion imagery task initiation state, the duration of the motion imagery state is determined based on the gradient, including: If the gradient within the time window of the number of consecutive targets meets the judgment condition corresponding to the second threshold, then the first time window in the time window of the number of consecutive targets is taken as the second target time window. The second target time window indicates that the user state has entered the motion imagination task start state. The total duration of the time window of the number of consecutive targets is the duration of the motion imagination continuous state.

6. The fine feedback method based on motion imagery according to claim 1, characterized in that, Based on the amplitude deviation and relative power change percentage corresponding to all sliding time windows of the first EEG data, determine whether the user has entered the end state of the motor imagery task, including: From all time windows of the first EEG data, the first time window that satisfies the positive direction of the amplitude deviation and the corresponding relative power change percentage meets the judgment condition of the third threshold is selected as the third target time window. The third target time window indicates that the user's state has entered the end state of the motor imagery task.

7. The fine feedback method based on motion imagery according to claim 5, characterized in that, The third target time window is used as the start time for feedback reset.

8. The fine feedback method based on motion imagery according to claim 1, characterized in that, The refined feedback method further includes: if it cannot be determined that the user has entered the motion imagination task preparation state, then no feedback signal is generated.

9. A fine feedback system based on motion imagery, characterized in that, include: The EEG data acquisition module is configured to acquire the first EEG data of the user during the first preset time period after the prompting to begin motor imagery. The data preprocessing module is configured to perform sliding time window processing on the first EEG data; For all time windows of the first EEG data, calculate the amplitude deviation and the percentage change in relative power; The user state determination module is configured to calculate a gradient based on the first EEG data; determine whether the user state has entered the motor imagery task start state based on the gradient; if the user state has entered the motor imagery task start state, determine the duration of the motor imagery state based on the gradient; and determine whether the user state has entered the motor imagery task end state based on the amplitude deviation and relative power change percentage corresponding to all the sliding time windows of the first EEG data. The status feedback module is configured to generate feedback signals based on the user's motion imagery task start state, motion imagery duration, and motion imagery task end state.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the fine feedback method based on motion imagery as described in any one of claims 1 to 8.

Images