Virtual reality rehabilitation training method and device based on hierarchical motion intention decoding

Abstract

The application discloses a virtual reality rehabilitation training method and device based on hierarchical motion intention decoding, and relates to the field of human-computer interaction and rehabilitation training. The method comprises the following steps: synchronously collecting electroencephalogram and electromyogram signals of a subject and performing differential pretreatment; a hierarchical motion intention decoding process is constructed, first-level decoding is performed through electroencephalogram and electromyogram weighted fusion to judge a stop or straight-ahead state; when the straight-ahead state is judged, second-level decoding based on the electromyogram signal is triggered to identify acceleration, deceleration or speed maintenance intention; finally, edge triggering and state buffering mechanism are combined to drive a virtual character to perform an action and update path feedback. The application processes basic motion states and speed adjustment intention in layers, effectively avoids control coupling and false triggering problems caused by parallel classification of multiple intentions, and significantly improves the active participation, control stability and interaction naturalness of virtual reality rehabilitation training.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction and rehabilitation training technology, and in particular to a virtual reality rehabilitation training method and device based on graded motion intention decoding. Background Technology

[0002] With the increasing number of patients suffering from lower limb dysfunction due to stroke, spinal cord injury, and other conditions, how to conduct rehabilitation training with active participation characteristics in a safe and controllable task environment to promote the recovery of patients' lower limb motor control function has become an important research question in the field of intelligent rehabilitation. In existing rehabilitation training methods, some training still relies mainly on passive stretching, repetitive movements, or external manipulation, which fails to fully reflect the subject's voluntary motor readiness and residual muscle activation ability, thus limiting the degree of patient active participation and training feedback effects.

[0003] Electroencephalogram (EEG) signals can reflect changes in a subject's central motor readiness and motor intention, while electromyography (EMG) signals can reflect peripheral muscle activation. Using EEG and EMG signals for motor intention recognition helps to describe the subject's active participation process from both central intention and peripheral execution perspectives, thereby improving the physiological relevance, response accuracy, and interaction stability of rehabilitation training control.

[0004] However, existing rehabilitation control solutions still have shortcomings. On the one hand, some systems mainly rely on handles, buttons, voice, or preset procedures to complete scene control, making it difficult to accurately reflect the subject's real-time movement intentions. On the other hand, some decoding methods focus on a single modality or directly recognize different levels of actions such as stopping, moving forward, accelerating, and decelerating as parallel categories, which can easily cause coupling between the basic motor state and speed regulation intentions, leading to false triggering, repeated triggering, or control jitter. Especially in continuous walking tasks, stopping and moving forward belong to basic motor state control, while accelerating and decelerating belong to speed regulation control in the walking state, and the two have different triggering conditions and stability requirements. Therefore, it is necessary to propose a hierarchical movement intention decoding system and method based on joint recognition of EEG and EMG, combined with EMG speed regulation recognition, to improve the active participation, control stability, and natural interaction in virtual scene rehabilitation training. Summary of the Invention

[0005] The purpose of this application is to provide a virtual reality rehabilitation training method and device based on graded motion intention decoding, in order to solve the problems of existing rehabilitation training systems that are difficult to accurately reflect the real-time motion intention of the subject, the easy coupling between basic motion state and speed adjustment intention, and the easy occurrence of false triggering and control jitter during virtual character control.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a virtual reality rehabilitation training method based on graded motion intention decoding, including: Simultaneously, raw electroencephalogram (EEG) and electromyogram (EMG) signals of subjects under different motor intention tasks were collected; the motor intention tasks included a basic motor state task to trigger first-level motor intention decoding and a speed regulation task to trigger second-level motor intention decoding. The raw EEG and raw EMG signals are preprocessed to obtain EEG data and EMG data to be identified. The EEG data and EMG data to be identified are subjected to first-level motor intention decoding to obtain the first-level motor intention decoding result. Based on the first-level motion intent decoding results, it is determined whether the subject is in a stopped state or a straight-moving state; When the first-level motion intent decoding result is a straight-line state, the second-level motion intent decoding is triggered, and the second-level motion intent decoding result is obtained; Based on the secondary motion intent decoding results, the acceleration intent, deceleration intent, or intention to maintain the current speed is identified, and corresponding speed adjustment control results are generated; Based on the first-level motion intent decoding result and the speed adjustment control result, a motion control result is generated to drive the virtual character to perform corresponding actions in the virtual scene, and the path status and training feedback information are updated based on the motion control result.

[0007] Optionally, the basic motor state task includes a motor imagery stage, which guides the subject to generate central motor intentions related to stopping or moving forward, and simultaneously collects EEG and EMG signals to train a first-level motor intention decoding model; the speed regulation task includes a lower limb movement stage, which guides the subject to perform lower limb force exertion or speed regulation movements, and collects EMG signals to train a second-level motor intention decoding model.

[0008] Optionally, preprocessing of the raw EEG signals includes: Selecting effective EEG channels from multi-channel raw EEG signals; The effective EEG channels are subjected to bandpass filtering and notch filtering. The effective EEG channels after bandpass filtering and notch filtering are subjected to common average reference processing to obtain the rereferenced EEG signal. The baseline removal process is performed on the rereferenced EEG signal to obtain the EEG data to be identified.

[0009] Optionally, the specific processes for first-level motion intent decoding and second-level motion intent decoding are as follows: The EEG data to be identified is input into the EEG decoding network to obtain the EEG recognition probability of stopping / going straight. The electromyographic data to be identified is input into the electromyographic decoding network to obtain the electromyographic recognition probability of stopping / going straight. The weighted decision module performs weighted fusion of the EEG recognition probability and the EMG recognition probability to obtain the first-level motor intention decoding probability, and determines the first-level motor intention decoding result based on the first-level motor intention decoding probability. When the first-level motion intent decoding result is in a straight-line state, the electromyographic data to be identified is input into the second-level electromyographic decoding network to obtain the speed regulation intent probability, and the second-level motion intent decoding result is determined based on the speed regulation intent probability to generate the speed regulation control result.

[0010] Optionally, the EEG data to be identified is input into an EEG decoding network to obtain the EEG recognition probability of stopping / going straight, specifically including: Temporal convolution processing is performed on the EEG data to be identified to extract temporal features; The temporal features are input into a spatial convolutional layer to model the spatial relationships between different EEG channels and obtain spatial features. The spatial features are sequentially input into an average pooling layer, a Dropout layer, a separable convolutional layer, and an adaptive global pooling layer for processing to obtain the global EEG feature vector. The global EEG feature vector is input into a fully connected classification layer, and the EEG recognition probability is output.

[0011] Optionally, the electromyographic data to be identified is input into an electromyographic decoding network to obtain the electromyographic recognition probability of stopping / straightening, specifically including: Features are extracted and concatenated for the time window of each electromyography channel to obtain the electromyography feature matrix; After standardizing the electromyographic feature matrix, it is input into the channel self-attention module to calculate the attention weights between different electromyographic channels, thereby obtaining the enhanced electromyographic features. The enhanced electromyographic features of the channel are input into the multi-scale convolutional feature fusion module to obtain the multi-scale fused electromyographic features. The electromyographic features fused from the multi-scale fusion are fused with the temporal features output by the cyclic temporal discriminant branch to obtain the electromyographic spatiotemporal discriminant features; The electromyography spatiotemporal discrimination features are sequentially processed by global average pooling and a fully connected classification layer to output the electromyography recognition probability.

[0012] Optionally, when driving the virtual character to perform corresponding actions in the virtual scene, a command edge triggering mechanism and a state buffering mechanism are adopted; the command edge triggering mechanism means that the corresponding action is triggered only when the current motion control result changes relative to the previous effective motion control result; the state buffering mechanism is used to save the most recent effective motion control result in order to reduce the impact of short-term recognition fluctuations on the motion state of the virtual character.

[0013] Optionally, the update path state and training feedback information specifically include: Determine whether the target node has been reached based on the distance between the virtual character's current position and the target node's position; When the distance is less than or equal to the preset threshold, it is determined that the target node has been reached. The virtual character is then controlled to stop or enter the next training node, and the path prompts, target node status, and map positioning status are updated.

[0014] Optionally, a motion control result is generated based on the first-level motion intent decoding result and the speed adjustment control result to drive the virtual character to perform corresponding actions in the virtual scene, specifically including: When the decoding result of the first-level motion intent is a stopped state, control the virtual character to stop moving; When the first-level motion intent decoding result is a straight-line state, the virtual character is controlled to move along a preset path, and the movement speed of the virtual character is adjusted or maintained according to the preset speed level constraint rules based on the speed adjustment control result. Secondly, this application provides a virtual reality rehabilitation training device based on graded motion intention decoding, comprising: A multimodal signal acquisition module is used to simultaneously acquire raw electroencephalogram (EEG) signals and raw electromyogram (EMG) signals of subjects under different motor intention tasks; the motor intention tasks include a basic motor state task for triggering first-level motor intention decoding and a speed regulation task for triggering second-level motor intention decoding. The preprocessing module is used to preprocess the raw EEG signals and raw EMG signals respectively to obtain the EEG data to be identified and the EMG data to be identified. The first-level motor intention decoding module is used to perform first-level motor intention decoding on the EEG data and EMG data to be identified, and obtain the first-level motor intention decoding result. The subject status judgment module is used to determine whether the subject is in a stopped state or a straight-moving state based on the first-level motion intention decoding result; The secondary motion intent decoding module is used to trigger secondary motion intent decoding and obtain the secondary motion intent decoding result when the primary motion intent decoding result is a straight-line state. The speed adjustment control result generation module is used to identify acceleration intention, deceleration intention or the intention to maintain the current speed based on the secondary motion intention decoding result, and generate the corresponding speed adjustment control result; The virtual reality rehabilitation training module generates motion control results based on the first-level motion intention decoding results and the speed adjustment control results, drives the virtual character to perform corresponding actions in the virtual scene, and updates the path status and training feedback information based on the motion control results.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a virtual reality rehabilitation training method and device based on graded motion intention decoding, which has the following beneficial effects: This application employs a hierarchical motion intention decoding method, processing the recognition of basic motion states (stop / straight-line movement) and the recognition of speed adjustment intentions (acceleration / deceleration) in layers. Level 1 decoding handles the basic state, while level 2 decoding is triggered only in the straight-line state. This avoids control coupling problems caused by parallel classification of intentions at different levels and reduces the risk of false triggering. This application combines EEG and EMG signals for level 1 decoding, comprehensively utilizing central nervous system activity information and peripheral muscle execution information to improve the stability and reliability of basic motion state recognition. In level 2 decoding, this application identifies speed adjustment intentions based on EMG signals, leveraging the sensitivity of EMG signals to changes in muscle force to make the virtual character's speed adjustment more consistent with the subject's actual motion participation. This application stabilizes the virtual character's movement process through control strategies such as command edge triggering, state buffering, and speed level constraints, reducing control jitter caused by continuous repetitive commands and improving control smoothness. This application constructs a virtual reality rehabilitation training module that transforms the decoding results into virtual character movements and simultaneously displays path, distance, and map positioning status, enhancing active participation and task immersion during training. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a virtual reality rehabilitation training method based on graded motion intention decoding, provided as an embodiment of this application; Figure 2 This is a schematic diagram of a data acquisition paradigm for the training phase provided in an embodiment of this application; Figure 3A training data graph provided in one embodiment of this application; Figure 4 This is a motion intent recognition confusion matrix diagram provided in one embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] In one exemplary embodiment, such as Figure 1 As shown, a virtual reality rehabilitation training method based on graded motion intention decoding is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method includes the following steps: Step 101: Simultaneously collect raw EEG and raw EMG signals of the subject under different motor intention tasks; the motor intention tasks include a basic motor state task for triggering first-level motor intention decoding and a speed regulation task for triggering second-level motor intention decoding.

[0021] Subjects wore a 64-channel EEG acquisition device and a 4-channel surface electromyography (EMG) acquisition device, and completed data acquisition tasks during the training phase under visual cues in a virtual reality environment. The system simultaneously acquired raw EEG and EMG signals from subjects under different motor intention tasks, and extracted corresponding synchronous test segment data based on trial labels. Subsequently, the EEG and EMG signals were preprocessed and organized to form datasets for training the primary and secondary motor intention decoding models.

[0022] The specific steps are as follows: In one specific embodiment, the subject wore a 64-channel EEG acquisition device and a 4-channel surface electromyography (EMG) acquisition device, and completed data acquisition tasks during the training phase under the visual cues of a virtual reality scene. The system simultaneously acquired the subject's raw EEG and EMG signals under different motor intention tasks, and recorded the task labels corresponding to each trial for subsequent segment extraction and model training.

[0023] like Figure 2As shown, the data acquisition paradigm during the training phase includes multiple training blocks, each of which consists of several consecutive trials. Rest periods are set between adjacent training blocks to reduce the impact of subject fatigue on the quality of EEG and EMG signals.

[0024] In one specific embodiment, each training block includes multiple consecutive trials, such as Trial-1 to Trial-48. Each trial includes, in a preset time sequence, a fixation phase, a cueing phase, a motor imagery phase, a lower limb movement phase, and a rest phase.

[0025] The process includes the following phases: fixation phase to ensure the subject focuses on the center of the screen and maintains stable attention; cueing phase to present the subject with the motor task corresponding to the current trial; motor imagery phase to guide the subject to generate central motor intentions related to stopping or moving forward, while simultaneously collecting EEG and EMG signals; lower limb movement phase to guide the subject to perform corresponding lower limb force exertion or speed regulation movements according to cues, while also collecting EMG signals; and rest phase to allow the subject to relax and return to a relatively stable state, preparing for the next trial.

[0026] Specifically, during the motor imagery phase, the system simultaneously collects the subject's electroencephalogram (EEG) and electromyogram (EMG) signals to train a primary motor intention decoding model to identify stationary or forward states. During the lower limb movement phase, the system focuses on collecting EMG signals to train a secondary motor intention decoding model to identify acceleration, deceleration, or maintaining current speed intentions during forward movement. This phased data collection approach allows for the acquisition of training data related to basic motor state recognition and speed regulation intention recognition, thereby reducing aliasing between different levels of motor intention labels.

[0027] After data acquisition, the system segments the raw EEG and EMG signals according to the start and end times of each trial and the task label, obtaining EEG and EMG segments corresponding to each trial. Subsequently, the EEG and EMG segments are input into the preprocessing flow and divided into training and validation sets to form datasets for training the EEG decoding network, EMG decoding network, and secondary EMG intention recognition network. The data acquisition paradigm during the training phase is as follows: Figure 3 As shown.

[0028] Step 102: Preprocess the raw EEG signals and raw EMG signals to obtain the EEG data and EMG data to be identified.

[0029] Since EEG and EMG signals differ in physiological origin, spectral distribution, and noise type, this application sets up separate preprocessing procedures for the two types of signals to improve the input quality of subsequent graded motion intent decoding.

[0030] For the raw EEG signal, the first step in preprocessing is to select effective channels. Considering that the EEG acquisition process may include reference electrodes, auxiliary channels, or channels with substandard quality, this application selects 59 effective EEG channels from the 64 channels of raw EEG signal for subsequent preprocessing and decoding. Subsequently, the EEG rhythm components within the effective EEG channels are processed to remove low-frequency drift and high-frequency noise; simultaneously, a notch filter is applied at 50Hz to suppress power line interference. The filtered EEG signal is represented as follows: ; in, Indicates the first Each brainwave channel at any time The raw brain signals, This indicates bandpass filtering from 1 Hz to 40 Hz. This indicates 50 Hz notch filtering. This represents the filtered EEG signal.

[0031] Furthermore, to reduce common-mode interference between different EEG channels, a common average reference processing is performed on the filtered effective EEG channels to obtain the rereferenced EEG signal: ; in Indicates the first The EEG signal after re-reference of an effective EEG channel In this embodiment, the effective number of EEG channels is indicated. .

[0032] Subsequently, baseline removal processing was performed on the rereferenced EEG signals to reduce the impact of intra-segment DC bias and slow baseline fluctuations on subsequent recognition. The baseline-removed EEG signals are represented as follows: ; in, Indicates the first EEG signals after baseline removal of an effective EEG channel Indicates the index of the time sampling point within the baseline interval. This represents the total number of sampling points for the baseline signal.

[0033] For the raw electromyography (EMG) signals, considering that muscle activation signals are mainly distributed in the higher frequency band, a bandpass filter of 20 Hz to 300 Hz was applied to the 4-channel EMG signals during preprocessing to retain the effective components related to muscle contraction and suppress low-frequency motion artifacts and irrelevant noise, resulting in the filtered EMG signals: ; in, Indicates the first Each electromyographic channel at time The original electromyographic signals, This indicates bandpass filtering from 20Hz to 450Hz. This represents the filtered electromyographic signal. .

[0034] Step 103: Perform first-level motor intention decoding on the EEG data and EMG data to be identified to obtain the first-level motor intention decoding result.

[0035] Step 104: Determine whether the subject is in a stopped state or a straight-moving state based on the first-level motion intention decoding result.

[0036] Step 105: When the first-level motion intent decoding result is a straight-line state, the second-level motion intent decoding is triggered to obtain the second-level motion intent decoding result.

[0037] Step 106: Based on the secondary motion intent decoding result, identify the acceleration intent, deceleration intent, or intention to maintain the current speed, and generate the corresponding speed adjustment control result.

[0038] Based on the preprocessed EEG and EMG signals obtained in step 102, this application constructs a hierarchical motor intention decoding process. The hierarchical motor intention decoding process includes primary motor intention decoding and secondary motor intention decoding. Primary motor intention decoding is used to determine whether the subject is currently in a stopped or straight-moving state, while secondary motor intention decoding is used to further determine the subject's speed regulation intention when the current state is straight-moving.

[0039] When the secondary motion intent decoding result is an acceleration intent, the current speed gear is increased by one gear; when the secondary motion intent decoding result is a deceleration intent, the current speed gear is decreased by one gear; when the secondary motion intent decoding result is to maintain the current speed, the speed gear remains unchanged. The virtual character's current movement speed is mapped according to the updated speed tier, and the current movement speed satisfies: ; in, Current movement speed Based on speed, The current speed gear. This represents the speed step size for each gear. The speed gear is limited to a preset range, and the cumulative number of accelerations and decelerations does not exceed the preset maximum number.

[0040] In the first-level motor intention decoding process, the EEG data to be identified is input into the EEG decoding network to obtain the EEG recognition probability of stopping / straightening; the EMG data to be identified is input into the EMG decoding network to obtain the EMG recognition probability of stopping / straightening. Subsequently, the weighted decision module performs weighted fusion of the EEG recognition probability and the EMG recognition probability to obtain the first-level motor intention decoding probability, which is calculated as follows: ; in, This represents the probability of decoding the first-level motion intent. This represents the stopping / straight-line probability output by the EEG decoding network. This represents the stop / straight-line probability output by the electromyography decoding network. The weighting coefficients represent the weighting coefficients of the EEG recognition results, and .

[0041] The first-level motion intent decoding result is determined based on the first-level motion intent decoding probability: ; in, This represents the result of the first-level motion intent decoding. When... When the system is in a stopped state, it outputs a stop control result and sends it to the virtual reality rehabilitation training module, which then controls the virtual character to stop moving. When the system is in a straight-ahead state, it outputs a straight-ahead control result and triggers secondary motion intent decoding. The secondary motion intent decoding result can be expressed as: ; in, This represents the probability of acceleration, deceleration, or maintaining the current speed output by the secondary electromyography decoding network. This indicates the result of the secondary motion intent decoding. Based on the secondary motion intent decoding result, the system generates the corresponding speed adjustment control result. When When the intention is to accelerate, the system outputs the acceleration control result; when When the intention to decelerate is indicated, the system outputs the deceleration control result; when... To maintain the intended current speed, the system outputs a speed maintenance control result. The acceleration control result, deceleration control result, or speed maintenance control result, together with the primary straight-line control result, constitute the motion control result of the virtual character.

[0042] Upon receiving the motion control result, the virtual character is controlled to perform the corresponding action. If the first-level motion intent decoding result is a stop state, the virtual character stops; if the first-level motion intent decoding result is a straight-line state, the virtual character enters a straight-line motion state and performs acceleration, deceleration, or maintaining the current speed action according to the second-level motion intent decoding result.

[0043] Therefore, the basic motion state recognition and speed adjustment intention recognition are processed in layers: the first-level motion intention decoding is responsible for stop / straight-line judgment, and the second-level motion intention decoding is only triggered in the straight-line state for speed adjustment judgment. This hierarchical decoding method avoids the control coupling problem caused by directly classifying stop, straight-line, acceleration, and deceleration in parallel, reduces the risk of false triggering and repeated triggering, and improves the stability of virtual character motion control.

[0044] In one specific embodiment, this application employs an EEG decoding model to output the EEG recognition probability of a stationary or ascending state based on the EEG data to be identified. This EEG decoding model, as the EEG branch in primary motor intention decoding, is mainly used to extract spatiotemporal features related to central motor preparation, motor imagery, and changes in motor intention from the EEG signal, providing EEG-based recognition criteria for primary stationary / ascending state determination. In one specific embodiment, the EEG data to be identified can be represented as: ; in, This represents the EEG data to be identified. Indicates the number of samples. Indicates the number of effective EEG channels. This represents the number of sampling points within each EEG time window. In one specific embodiment, In this context, "1" in the dimension indicates that the input EEG data is organized into a two-dimensional feature map form with a single input channel, so that the subsequent convolutional network can extract features simultaneously along the time and spatial channel dimensions.

[0045] The EEG decoding model first performs temporal convolution processing on the EEG data to be identified in order to extract local dynamic change features of the EEG signal in the time dimension: ; in, Represents the temporal convolution mapping, This represents the extracted temporal features. Temporal convolutional layers are primarily used to capture waveform changes, rhythmic fluctuations, and temporal patterns related to motor preparation within short time windows of EEG signals. Since the stationary and linear states may exhibit different EEG dynamics during central motor preparation, temporal convolutional layers can provide preliminary temporal discriminative features for subsequent identification.

[0046] After obtaining the temporal features, the EEG decoding model further models the spatial relationships between different EEG channels through spatial convolutional layers to obtain spatial features: ; in, Represents spatial convolution mapping, Representing spatial features. Spatial convolutional layers are used to learn the spatial distribution patterns between different EEG channels, enabling the model to capture cross-channel response differences related to motor intentions. For example, when a subject has the intention to move straight or stop, different sensorimotor-related brain regions may show different degrees of activation or rhythmic changes, and spatial convolutional layers can model these inter-channel differences.

[0047] Subsequently, the model reduces the dimensionality of the extracted spatiotemporal features using an average pooling layer, and reduces the risk of overfitting using a Dropout layer. ; in, This indicates the average pooling operation. This indicates a random deactivation operation. The EEG features are represented after pooling and regularization. Average pooling can reduce redundant features and enhance the model's robustness to local perturbations; By randomly masking some feature units, the model's over-reliance on training samples is reduced, thereby improving the model's generalization ability in the validation and online recognition stages.

[0048] Building upon this, the EEG decoding model employs separable convolutional layers to further extract EEG features. Separable convolutional layers decompose the feature extraction process into two parts: depthwise convolution and pointwise convolution. This reduces the number of model parameters while enhancing feature representation capabilities, generalization ability in both the validation and online recognition stages. ; in, This represents a separable convolution mapping. This represents the EEG features after separable convolution. This structure can extract local patterns from different feature channels separately, and it can also fuse different feature channels, which is beneficial for forming a more compact and discriminative EEG feature representation.

[0049] Subsequently, the EEG features are aggregated into a fixed-dimensional global feature vector through an adaptive global pooling layer: ; in, This indicates an adaptive global pooling operation. This represents the global feature vector of the EEG. The adaptive global pooling layer can compress the high-dimensional features after convolution into a fixed-length feature representation, enabling subsequent fully connected classification layers to receive inputs of uniform dimension, while reducing the model's sensitivity to fluctuations at local time points.

[0050] Finally, the global EEG feature vector is input into the fully connected classification layer, and the EEG recognition probability of the stationary or straight-line state is output through the Softmax function: ; in, and These represent the weight parameters and bias parameters of the EEG classification layer, respectively. This represents the stopping / straight-line probability output by the EEG decoding network.

[0051] Therefore, the EEG decoding model sequentially completes the input of the EEG data to be identified, temporal feature extraction, spatial feature extraction, separable convolutional enhancement, global feature aggregation, and classification output. The EEG recognition probability is not directly used as the final control result, but rather as the output of the EEG branch of the first-level motor intention decoding, which, together with the EMG recognition probability output by the EMG decoding network, is input into the weighted decision module. In this way, the system can leverage the advantage of EEG signals reflecting the central motor readiness state to improve the reliability of recognizing the stationary / straight-line basic motor state.

[0052] This application embodiment uses an electromyography (EMG) decoding model to output an EMG recognition probability based on the EMG data to be identified. The EMG decoding model can serve as the EMG branch in the first-level motion intention decoding to assist in determining the stationary or straight-line state, or as a second-level motion intention decoding model to identify acceleration, deceleration, or maintaining the current speed intention in the straight-line state.

[0053] In one specific embodiment, the electromyographic data to be identified can be represented as: ; in, This represents the electromyography data to be identified. Indicates the number of samples. Indicates the number of electromyographic channels. This represents the number of sampling points within each electromyography time window. In this embodiment, Electromyography (EMG) signals directly reflect the activation state of lower limb muscles and are better able to demonstrate changes in peripheral muscle exertion than electroencephalography (EEG) signals. Therefore, they are suitable for describing intentions related to the execution of movements such as straightening, acceleration, and deceleration.

[0054] First, the electromyography (EMG) decoding model extracts features from the time window of each EMG channel to obtain the EMG feature vector corresponding to each channel: ; in, Indicates the first Time window of each electromyographic channel This represents the electromyography feature extraction function. Indicates the first The feature vectors of each electromyographic channel. The electromyographic features may include one or more of the following: time-domain features, frequency-domain features, frequency band energy ratio features, and time-frequency fluctuation features, which are used to describe muscle activation intensity, muscle contraction changes, and spectral distribution from different perspectives.

[0055] In one specific embodiment, 12-dimensional electromyographic features are extracted from each electromyographic channel, and the electromyographic feature matrix is ​​obtained by concatenating the features from each channel: ; in, This represents the feature dimension of each channel, as shown in this embodiment. .when At that time, the electromyographic feature matrix can be represented as This matrix preserves both the independent features of each electromyographic channel and the structural relationships between different electromyographic channels, providing input for subsequent channel co-modeling.

[0056] Subsequently, the electromyography feature matrix was standardized to reduce the impact of differences in feature dimensions and amplitudes on model training. ; in, This represents the standardized electromyographic feature matrix. and Let represent the mean and standard deviation of the electromyographic characteristics in the training set, respectively. To prevent constants with zero denominators, standardization is used to map electromyographic features of different dimensions to similar scales. This helps improve the stability of model training and prevents features with large amplitudes from dominating the training process.

[0057] The standardized electromyography (EMG) feature input channel self-attention module. The channel self-attention module first projects the EMG features to a high-dimensional feature space through a linear mapping and then introduces channel embedding information: ; in, Indicates the input mapping weights. Represents the channel embedding vector. This represents the electromyographic feature representation after channel embedding. The channel embedding vector is used to introduce learnable channel identification information for different electromyographic channels, enabling the model to distinguish the role of different muscle channels in motion intent recognition.

[0058] During the channel self-attention calculation process, the model generates a query matrix, a key matrix, and a value matrix based on the input features, and calculates the attention weights between different electromyographic channels: ; ; in, , , These represent the query matrix, key matrix, and value matrix, respectively. Representing feature dimension, Indicates channel attention weights. This represents the attention-weighted electromyographic (EMG) channel features. Through this process, the model can adaptively adjust the importance of different EMG channels based on the current input sample, highlighting muscle channels more relevant to the current movement intention, and modeling the co-activation relationships between different EMG channels.

[0059] Furthermore, the channel self-attention module enhances feature stability through residual connections and normalization processing: ; in, This indicates a normalization operation. This represents the electromyographic features after channel enhancement. Residual connections can preserve the original electromyographic channel features and avoid losing effective information during attention calculation; normalization can improve network training stability and enhance model convergence.

[0060] Subsequently, the enhanced electromyographic features are input into a multi-scale convolutional feature fusion module. This module includes multiple convolutional branches with different kernel scales, used to extract dynamic electromyographic features from different receptive fields. ; in, , and These represent convolution mappings at different scales. , and The symbols represent the electromyographic features extracted by convolutional branches at different scales. Smaller convolutional kernels are better at capturing short-term muscle activation mutations, while larger convolutional kernels are better at capturing continuous muscle exertion patterns. Therefore, multi-scale convolution can enhance the model's adaptability to electromyographic patterns of different movement intentions.

[0061] The features output from each scale convolutional branch are concatenated and fused to obtain multi-scale electromyographic features: ; in, This indicates a feature concatenation operation. This represents the electromyographic features after multi-scale fusion. By splicing and fusing, local features extracted from convolutional branches at different scales are integrated into a unified feature representation, thereby improving the electromyographic decoding model's ability to express complex muscle activation patterns.

[0062] Furthermore, the model can also incorporate a cyclic temporal discriminant branch to extract dynamic patterns of electromyographic features over time. The temporal features output by the cyclic temporal discriminant branch are denoted as... Then the spatiotemporal discriminant features of electromyography can be expressed as: ; in, This represents the spatiotemporal discriminative features of electromyography (EMG) after fusing multi-scale convolutional features and cyclic temporal features. By fusing multi-scale convolutional features and cyclic temporal features, the EMG decoding model can simultaneously utilize local change information and cross-temporal dynamic evolution information of EMG signals, thereby enhancing its ability to distinguish different movement intentions such as stopping, moving straight, accelerating, and decelerating.

[0063] Subsequently, the spatiotemporal discriminative features of electromyography were subjected to global average pooling and After processing, the data is input into a fully connected classification layer to obtain the electromyography (EMG) recognition probability: ; in, and These represent the weight parameters and bias parameters of the electromyography classification layer, respectively. This represents the probability of electromyography (EMG) recognition. Global average pooling is used to compress high-dimensional EMG features into a fixed-length global representation. To reduce the risk of overfitting, a fully connected classification layer is used to map electromyographic features to the corresponding motion intention category space.

[0064] In primary motor intent decoding, electromyography (EMG) recognition probability Electromyography (EMG) results are used to represent stationary or straight-moving states, and are weighted and fused with EEG recognition probabilities. In the secondary motion intent decoding, the same or similar EMG decoding model structure can be used, setting the output category to acceleration, deceleration, or maintaining the current speed, thereby obtaining the speed regulation intent recognition probability. ; in, This represents the probability of speed modulation intent output by the secondary electromyography decoding network. This represents the spatiotemporal discriminant features of electromyography extracted by the two-level electromyography decoding model. and These represent the weight parameters and bias parameters of the secondary classification layer, respectively.

[0065] Therefore, the electromyography (EMG) decoding model can be used for both stop / straight-line auxiliary discrimination in first-level motion intention decoding and acceleration, deceleration, or maintaining current speed recognition in second-level motion intention decoding. Through EMG feature extraction, channel self-attention modeling, multi-scale convolutional feature fusion, and temporal discrimination, the ability of EMG signals to represent different motion intentions can be enhanced, thereby improving the accuracy and stability of hierarchical motion intention decoding.

[0066] This application further trains the EEG decoding model and the EMG decoding model separately, and records the changes in loss function value, training accuracy, and validation accuracy with the number of iterations during the training process, such as... Figure 3 As shown. Among them, Figure 3 Part (a) in the figure shows the training process curve of the EEG decoding model. In the figure, EEG represents electroencephalogram / brainwave signal. Figure 3 Part (b) in the figure shows the training process curve of the electromyography decoding model. EMG in the figure represents electromyography / electromyography signal.

[0067] During model training, the data obtained in steps 101 and 102 are first divided into training and validation sets according to a preset ratio or cross-validation method. The training set is used to update model parameters, and the validation set is used to evaluate the model's recognition performance on data not used in the training. To reduce the impact of randomness in data partitioning on model performance evaluation, a hierarchical cross-validation method can be used to partition the training samples, ensuring that the distribution of different categories of samples remains relatively consistent between the training and validation sets.

[0068] For the EEG decoding model, the training samples are the EEG data to be identified and their corresponding stop / straight-ahead labels. The EEG decoding model outputs the predicted probability of the stop or straight-ahead state and uses the cross-entropy loss function to measure the difference between the predicted result and the true label. The loss function can be expressed as: ; in, This represents the cross-entropy loss of the EEG decoding model. Indicates the number of training samples. Indicates the number of categories. Indicates the first The sample belongs to the first The true label of the class, The output of the EEG decoding model represents the first... The sample belongs to the first The predicted probability of a class.

[0069] For the electromyography (EMG) decoding model, the training samples are the EMG data to be identified and their corresponding motion intention labels. Considering the potential imbalance in the number of samples for different motion intention categories, this application can use a class-weighted cross-entropy loss function to train the EMG decoding model, and its loss function can be expressed as: ; in, This represents the cross-entropy loss of the EEG decoding model. Indicates the first The class weights corresponding to the class samples. The output of the electromyography decoding model represents the first... The sample belongs to the first The predicted probability of a class. By introducing class weights, the impact of class imbalance on model training results can be reduced, thus improving the model's ability to identify a minority of classes.

[0070] In each iteration of training, the model first calculates the predicted probability and loss function value based on the training set samples, and then updates the model parameters through backpropagation. The model parameter update process can be summarized as follows: ; in, Indicates the first Model parameters at the next iteration Indicates the learning rate. This represents the current training loss function. This represents the gradient of the loss function with respect to the model parameters. For both EEG and EMG decoding models, adaptive gradient optimization algorithms can be used to update the model parameters.

[0071] During training, the system records the training loss, validation loss, training accuracy, and validation accuracy for each iteration. Training accuracy reflects the model's ability to fit the training samples, while validation accuracy reflects the model's ability to generalize to samples not used in the training. Training accuracy and validation accuracy can be expressed as: ; in, This represents the number of samples whose predicted category matches the true label. This represents the total number of samples participating in the evaluation.

[0072] like Figure 3 As shown, with increasing iterations, the training and validation accuracy of both the EEG decoding and EMG decoding models generally increase and then gradually stabilize. This result indicates that the EEG decoding model can learn spatiotemporal discriminative features related to the stop / straight-line state from EEG signals, and the EMG decoding model can learn motor intention features related to muscle activation patterns from EMG signals. The stabilization of the training curves indicates that the model parameters gradually converge, and further iterations offer limited improvement to model performance.

[0073] Furthermore, an early stopping strategy can be implemented during training. If the validation set performance fails to improve over several consecutive iterations, training of the current model is stopped to reduce the risk of overfitting and minimize ineffective training time. Simultaneously, the system saves the model parameters corresponding to the optimal validation set performance, which will then be used as the optimal model for subsequent online motion intent decoding.

[0074] Through the aforementioned training process, this application can obtain the optimal parameters for both the EEG decoding model and the EMG decoding model. The trained EEG decoding model is used to output the EEG recognition probability of stop / straight movement, while the trained EMG decoding model can be used for both stop / straight movement EMG recognition in primary motor intention decoding and acceleration, deceleration, or maintaining current speed recognition in secondary motor intention decoding. This provides a model foundation for real-time motor intention decoding and virtual character control in subsequent virtual reality rehabilitation training.

[0075] After training the EEG and EMG decoding models, this application further uses test data to verify the recognition performance of the trained hierarchical motor intent decoding model and generates a normalized confusion matrix, such as... Figure 4 As shown. Figure 4 Part (a) in the figure is the normalized confusion matrix of the stop / straight-ahead recognition result in the first-level motion intent decoding. Figure 4 Part (b) is the normalized confusion matrix of the deceleration / acceleration recognition results in the secondary motion intent decoding.

[0076] Testing was conducted on EEG and EMG signals collected from multiple subjects. The normalized confusion matrix showed that samples of each category were mainly concentrated in the main diagonal region, indicating that the model has a good ability to distinguish the corresponding motor intention categories. Specifically, in the first-level motor intention decoding results, both the stationary and straight-moving states showed high recognition accuracy, indicating that the weighted decision-making of EEG and EMG results can reliably distinguish the subjects' basic motor states. In the second-level motor intention decoding results, deceleration and acceleration intentions were also mainly distributed in the main diagonal region, indicating that the EMG decoding model can effectively identify speed regulation intentions in the straight-moving state.

[0077] Figure 4 The results show that the hierarchical motion intention decoding method described in this application can provide stable control input for virtual reality rehabilitation training. Specifically, the first-level motion intention decoding result is used to determine whether the virtual character enters a straight-moving state or a stopped state; the second-level motion intention decoding result is used to further adjust the movement speed of the virtual character when it is in a straight-moving state. By processing the basic motion state recognition and speed adjustment intention recognition in a hierarchical manner, the mutual interference between different control intentions can be reduced, and the false triggering and control jitter during the virtual character's movement can be decreased.

[0078] Step 107: Generate motion control results based on the first-level motion intent decoding results and the speed adjustment control results, drive the virtual character to perform corresponding actions in the virtual scene, and update the path status and training feedback information based on the motion control results.

[0079] In the online application phase, the virtual reality rehabilitation training module first loads a virtual training scene built with the Unity engine, and sets up virtual characters, training paths, target nodes, map positioning markers, distance prompt areas, and motion status display areas within the scene. Guided by visual cues, the subject generates corresponding movement intentions, and the system drives the virtual character to perform corresponding actions in the virtual scene based on the motion control results output by the hierarchical motion intention decoding model.

[0080] When a stop control result is received, the virtual reality rehabilitation training module controls the virtual character to stop moving and updates the current motion state in the interface; when a straight-line control result is received, the virtual character moves towards the target node along a preset path; when an acceleration or deceleration control result is received, the virtual reality rehabilitation training module adjusts the virtual character's current movement speed according to speed level constraint rules; when a maintain current speed control result is received, the virtual character maintains its current speed and continues to move. Thus, the subject's motion intention decoding results can be converted into the virtual character's motion behavior in the virtual environment in real time.

[0081] Furthermore, the virtual reality rehabilitation training module continuously updates the path status, remaining target distance, and map positioning status during the virtual character's movement. The path status displays the current path segment and target node information of the virtual character; the remaining target distance indicates the change in distance between the virtual character and the target node; and the map positioning status displays the virtual character's current position on the minimap or path interface, thereby helping the subject understand the training process and motion feedback.

[0082] In one specific embodiment, the virtual reality rehabilitation training module determines whether the target node has been reached based on the distance between the virtual character's current position and the target node's position. When the distance between the virtual character and the target node is less than or equal to a preset threshold, the system determines that the virtual character has reached the target node and controls the virtual character to stop or move to the next training node, while simultaneously updating path prompts, target node status, and training feedback information. In this way, the movement process of the virtual character can be kept consistent with the preset rehabilitation training task.

[0083] To improve the stability of the control process, the virtual reality rehabilitation training module can also be configured with a command edge triggering mechanism and a state buffering mechanism. The command edge triggering mechanism means that the corresponding action is triggered only when the current motion control result changes relative to the previous valid motion control result; when consecutive recognition results remain consistent, the same action is not triggered repeatedly. The state buffering mechanism is used to save the most recent valid motion control result to reduce the impact of short-term recognition fluctuations on the virtual character's motion state.

[0084] Furthermore, the virtual reality rehabilitation training module can display information such as EEG or EMG data streams, current motor control results, speed levels, target distances, and training completion status on the interface, enabling subjects to receive continuous visual feedback in a virtual environment. Through a combination of visual cues, character motion feedback, path feedback, and distance feedback, the system can create an immersive interactive process oriented towards rehabilitation training tasks.

[0085] Therefore, this application is approved. Figure 4 The recognition performance verification shown demonstrates that the hierarchical motor intention decoding model can provide reliable control results for virtual reality rehabilitation training. At the same time, through the reception, execution and feedback display of motor control results by the virtual reality rehabilitation training module, a continuous training process from brain electromyography motor intention recognition to virtual character motor control is realized, which improves the active participation, control stability and naturalness of interaction in rehabilitation training.

[0086] The beneficial effects of this application are mainly reflected in: (1) This application adopts a hierarchical motion intention decoding method, which separates the identification of the basic motion state of stopping / straight-line movement and the identification of the intention to accelerate / decelerate speed adjustment into layers. The first-level motion intention decoding is used to determine whether the subject is in a stopped state or a straight-line state. The second-level motion intention decoding is only triggered in the straight-line state and is used to identify the intention to accelerate, decelerate or maintain the current speed, thereby avoiding the control coupling problem caused by directly classifying stopping, straight-line movement, acceleration and deceleration in parallel, and reducing the risk of false triggering and repeated triggering.

[0087] (2) This application combines EEG and EMG signals for first-level motor intention decoding. The EEG signals reflect the subject's central motor preparation state and changes in motor intention, while the EMG signals reflect the peripheral muscle activation state. By weighting the EEG recognition probability and the EMG recognition probability, the stability and reliability of the recognition of the stop / straight-line basic motor state can be improved by comprehensively utilizing central nervous activity information and peripheral muscle execution information.

[0088] (3) In the second-level motion intention decoding, this application recognizes the speed adjustment intention based on electromyographic signals. It can take advantage of the fact that electromyographic signals are more sensitive to changes in muscle force to recognize acceleration, deceleration or maintaining the current speed in the straight state, so that the speed adjustment of the virtual character is more in line with the actual motion participation state of the subject and improves the naturalness of interaction in the virtual reality rehabilitation training process.

[0089] (4) This application constructs a virtual reality rehabilitation training module, which converts the results of graded motion intention decoding into the virtual character's actions of stopping, moving straight, accelerating, decelerating or maintaining the current speed, and simultaneously displays the path status, distance status, map positioning status and training feedback information, so that the subject can complete rehabilitation training tasks in a visualized and interactive virtual scene, thereby improving the active participation and task immersion in the training process.

[0090] (5) This application uses control strategies such as command edge triggering, state buffering, and speed level constraints to constrain and stabilize the motion process of the virtual character. The corresponding action is triggered only when the motion control result changes, and the cumulative adjustment range of acceleration and deceleration is limited, which can reduce control jitter caused by continuous repetitive commands and improve the smoothness and safety of the motion control of the virtual character.

[0091] (6) This application designs a training data acquisition paradigm for graded motor intention decoding and constructs an EEG decoding model and an EMG decoding model respectively. Through training curves and normalized confusion matrices, the model can effectively distinguish between stationary / straight-line basic motor states and acceleration / deceleration speed adjustment intentions, providing a stable model foundation for subsequent online virtual reality rehabilitation training.

[0092] Based on the same inventive concept, this application also provides a virtual reality rehabilitation training device based on graded motion intention decoding for implementing the virtual reality rehabilitation training method based on graded motion intention decoding described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the virtual reality rehabilitation training device based on graded motion intention decoding provided below can be found in the limitations of the virtual reality rehabilitation training method based on graded motion intention decoding described above, and will not be repeated here.

[0093] In one exemplary embodiment, a virtual reality rehabilitation training device based on graded motion intention decoding is provided, comprising: A multimodal signal acquisition module is used to simultaneously acquire raw electroencephalogram (EEG) signals and raw electromyogram (EMG) signals of subjects under different motor intention tasks; the motor intention tasks include a basic motor state task for triggering first-level motor intention decoding and a speed regulation task for triggering second-level motor intention decoding. The preprocessing module is used to preprocess the raw EEG signals and raw EMG signals respectively to obtain the EEG data to be identified and the EMG data to be identified. The first-level motor intention decoding module is used to perform first-level motor intention decoding on the EEG data and EMG data to be identified, and obtain the first-level motor intention decoding result. The subject status judgment module is used to determine whether the subject is in a stopped state or a straight-moving state based on the first-level motion intention decoding result; The secondary motion intent decoding module is used to trigger secondary motion intent decoding and obtain the secondary motion intent decoding result when the primary motion intent decoding result is a straight-line state. The speed adjustment control result generation module is used to identify acceleration intention, deceleration intention or the intention to maintain the current speed based on the secondary motion intention decoding result, and generate the corresponding speed adjustment control result; The virtual reality rehabilitation training module generates motion control results based on the first-level motion intention decoding results and the speed adjustment control results, drives the virtual character to perform corresponding actions in the virtual scene, and updates the path status and training feedback information based on the motion control results.

[0094] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A virtual reality rehabilitation training method based on hierarchical motion intention decoding, characterized in that, The virtual reality rehabilitation training method based on graded movement intention decoding includes: Simultaneously, raw electroencephalogram (EEG) and electromyogram (EMG) signals of subjects under different motor intention tasks were collected; the motor intention tasks included a basic motor state task to trigger first-level motor intention decoding and a speed regulation task to trigger second-level motor intention decoding. The raw EEG and raw EMG signals are preprocessed to obtain EEG data and EMG data to be identified. The EEG data and EMG data to be identified are subjected to first-level motor intention decoding to obtain the first-level motor intention decoding result. Based on the first-level motion intent decoding results, it is determined whether the subject is in a stopped state or a straight-moving state; When the first-level motion intent decoding result is a straight-line state, the second-level motion intent decoding is triggered, and the second-level motion intent decoding result is obtained; Based on the secondary motion intent decoding results, the acceleration intent, deceleration intent, or intention to maintain the current speed is identified, and corresponding speed adjustment control results are generated; Based on the first-level motion intent decoding result and the speed adjustment control result, a motion control result is generated to drive the virtual character to perform corresponding actions in the virtual scene, and the path status and training feedback information are updated based on the motion control result; The electromyographic data to be identified is input into the electromyographic decoding network to obtain the electromyographic recognition probability of stopping / straightening, specifically including: Features are extracted and concatenated for the time window of each electromyography channel to obtain the electromyography feature matrix; After standardizing the electromyographic feature matrix, it is input into the channel self-attention module to calculate the attention weights between different electromyographic channels, thereby obtaining the enhanced electromyographic features. The enhanced electromyographic features of the channel are input into the multi-scale convolutional feature fusion module to obtain the multi-scale fused electromyographic features. The electromyographic features fused from the multi-scale fusion are fused with the temporal features output by the cyclic temporal discriminant branch to obtain the electromyographic spatiotemporal discriminant features; The electromyography spatiotemporal discrimination features are sequentially processed by global average pooling and a fully connected classification layer to output the electromyography recognition probability. When driving the virtual character to perform corresponding actions in the virtual scene, a command edge triggering mechanism and a state buffering mechanism are adopted. The command edge triggering mechanism means that the corresponding action is triggered only when the current motion control result changes relative to the previous effective motion control result. The state buffering mechanism is used to save the most recent effective motion control result to reduce the impact of short-term recognition fluctuations on the motion state of the virtual character.

2. The virtual reality rehabilitation training method based on hierarchical motion intention decoding according to claim 1, characterized in that, The basic motor state task includes a motor imagery stage, which guides the subject to generate central motor intentions related to stopping or moving forward, and simultaneously collects EEG and EMG signals to train the first-level motor intention decoding model; the speed regulation task includes a lower limb movement stage, which guides the subject to perform lower limb force exertion or speed regulation movements, and collects EMG signals to train the second-level motor intention decoding model.

3. The virtual reality rehabilitation training method based on hierarchical motion intention decoding according to claim 1, characterized in that, Preprocessing of raw EEG signals includes: Selecting effective EEG channels from multi-channel raw EEG signals; The effective EEG channels are subjected to bandpass filtering and notch filtering. The effective EEG channels after bandpass filtering and notch filtering are subjected to common average reference processing to obtain the rereferenced EEG signal. The baseline removal process is performed on the rereferenced EEG signal to obtain the EEG data to be identified.

4. The virtual reality rehabilitation training method based on graded motion intention decoding according to claim 1, characterized in that, The specific processes of primary motion intent decoding and secondary motion intent decoding are as follows: The EEG data to be identified is input into the EEG decoding network to obtain the EEG recognition probability of stopping / going straight. The electromyographic data to be identified is input into the electromyographic decoding network to obtain the electromyographic recognition probability of stopping / going straight. The weighted decision module performs weighted fusion of the EEG recognition probability and the EMG recognition probability to obtain the first-level motor intention decoding probability, and determines the first-level motor intention decoding result based on the first-level motor intention decoding probability. When the first-level motion intent decoding result is in a straight-line state, the electromyographic data to be identified is input into the second-level electromyographic decoding network to obtain the speed regulation intent probability, and the second-level motion intent decoding result is determined based on the speed regulation intent probability to generate the speed regulation control result.

5. The virtual reality rehabilitation training method based on graded motion intention decoding according to claim 4, characterized in that, The EEG data to be identified is input into the EEG decoding network to obtain the EEG recognition probability of stopping / going straight, specifically including: Temporal convolution processing is performed on the EEG data to be identified to extract temporal features; The temporal features are input into a spatial convolutional layer to model the spatial relationships between different EEG channels and obtain spatial features. The spatial features are sequentially input into an average pooling layer, a Dropout layer, a separable convolutional layer, and an adaptive global pooling layer for processing to obtain the global EEG feature vector. The global EEG feature vector is input into a fully connected classification layer, and the EEG recognition probability is output.

6. The virtual reality rehabilitation training method based on graded motion intention decoding according to claim 1, characterized in that, The updated path status and training feedback information specifically include: Determine whether the target node has been reached based on the distance between the virtual character's current position and the target node's position; When the distance is less than or equal to the preset threshold, it is determined that the target node has been reached. The virtual character is then controlled to stop or enter the next training node, and the path prompts, target node status, and map positioning status are updated.

7. The virtual reality rehabilitation training method based on graded motion intention decoding according to claim 1, characterized in that, Based on the first-level motion intent decoding result and the speed adjustment control result, a motion control result is generated to drive the virtual character to perform corresponding actions in the virtual scene, specifically including: When the decoding result of the first-level motion intent is a stopped state, control the virtual character to stop moving; When the first-level motion intent decoding result is a straight-line state, the virtual character is controlled to move along a preset path, and the movement speed of the virtual character is adjusted or maintained according to the preset speed level constraint rules based on the speed adjustment control result.

8. A virtual reality rehabilitation training device based on graded motion intention decoding, characterized in that, The virtual reality rehabilitation training device based on graded motion intention decoding is applied to the virtual reality rehabilitation training method based on graded motion intention decoding as described in any one of claims 1-7, wherein the virtual reality rehabilitation training device based on graded motion intention decoding comprises: A multimodal signal acquisition module is used to simultaneously acquire raw electroencephalogram (EEG) signals and raw electromyogram (EMG) signals of subjects under different motor intention tasks; the motor intention tasks include a basic motor state task for triggering first-level motor intention decoding and a speed regulation task for triggering second-level motor intention decoding. The preprocessing module is used to preprocess the raw EEG signals and raw EMG signals respectively to obtain the EEG data to be identified and the EMG data to be identified. The first-level motor intention decoding module is used to perform first-level motor intention decoding on the EEG data and EMG data to be identified, and obtain the first-level motor intention decoding result. The subject status judgment module is used to determine whether the subject is in a stopped state or a straight-moving state based on the first-level motion intention decoding result; The secondary motion intent decoding module is used to trigger secondary motion intent decoding and obtain the secondary motion intent decoding result when the primary motion intent decoding result is a straight-line state. The speed adjustment control result generation module is used to identify acceleration intention, deceleration intention or the intention to maintain the current speed based on the secondary motion intention decoding result, and generate the corresponding speed adjustment control result; The virtual reality rehabilitation training module generates motion control results based on the first-level motion intention decoding results and the speed adjustment control results, drives the virtual character to perform corresponding actions in the virtual scene, and updates the path status and training feedback information based on the motion control results.

Images