Intelligent rehabilitation training method and system based on artificial intelligence and virtual reality

Abstract

The invention provides an intelligent rehabilitation training method and system based on artificial intelligence and virtual reality, a user wears an intelligent wearable device to collect multi-modal data such as electroencephalogram, myoelectricity, physiological features and motion signals, the multi-modal data is preprocessed and then input into an artificial intelligence training model, and key features are extracted and fused by using a graph convolutional network and an attention mechanism algorithm. And constructing a digital twinborn model by using the fusion features, performing real-time dynamic mapping and predictive simulation, and generating a customized training scheme by means of a reinforcement learning algorithm in combination with a rehabilitation target and a physical state of the user. A user is trained in the virtual reality interaction model, the system monitors actions and physiological states in real time, the digital twin model synchronously acts, and the scene is dynamically adjusted. After training, the rehabilitation effect is evaluated according to the physiological indexes, the motion data and the twinning optimization analysis result, and an optimization training scheme and a digital twinning model are fed back. Precision, individuation and intelligentization of rehabilitation training are achieved, and the training effect and quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent rehabilitation technology, and particularly to an intelligent rehabilitation training method and system based on artificial intelligence and virtual reality. Background Art

[0002] In the modern medical system, rehabilitation training is crucial for the recovery of patients' physical functions. Especially for patients with neurological injuries, muscle dysfunction, etc., scientific and effective rehabilitation training is the key to improving the quality of life and returning to normal life. However, there are many limitations in the traditional rehabilitation training mode and existing technologies in practical applications, making it difficult to meet the growing rehabilitation needs, which has prompted the exploration of new rehabilitation technologies.

[0003] Traditional rehabilitation training mainly relies on the manual operation and experience guidance of rehabilitation therapists. This method not only has low efficiency, but also the treatment effects vary due to the differences in the professional levels of therapists. On the one hand, it is difficult to ensure the accuracy and consistency of movements in manual rehabilitation training, and it is impossible to provide standardized rehabilitation treatment for patients. On the other hand, due to the lack of refined consideration of patients' individual differences, it is difficult to formulate personalized training plans according to the specific conditions, physical conditions and rehabilitation goals of patients, resulting in limited rehabilitation effects. For example, for patients with different degrees of muscle atrophy, traditional training methods may not be able to accurately adjust the training intensity and methods, affecting the rehabilitation process.

[0004] With the development of technology, some technical means have been introduced into rehabilitation training, but there are still obvious deficiencies. In terms of data collection, existing rehabilitation equipment can often only obtain limited physiological data, such as basic indicators like heart rate and blood pressure, and cannot comprehensively collect multi-modal data reflecting the neuro-muscular function state of patients, such as electroencephalogram signals and electromyogram signals. This makes it difficult for rehabilitation therapists to deeply understand the neuro-muscular activities of patients and unable to provide strong data support for precise rehabilitation. For example, it is impossible to accurately judge the movement intention of patients through electroencephalogram signals, thereby affecting the pertinence of rehabilitation training.

[0005] Although some rehabilitation equipment has simple automation functions, most lack intelligent analysis and dynamic adjustment capabilities and cannot optimize the training plan in a timely manner according to the real-time training performance and physical state of patients. Existing rehabilitation training systems also have deficiencies in the interaction with patients, unable to provide an immersive training experience and difficult to stimulate the patients' active participation awareness, resulting in low enthusiasm of patients during the training process and affecting the rehabilitation effect.

[0006] Secondly, the existing evaluation methods are mainly based on simple scales and doctors' subjective judgments, lacking an objective, comprehensive, and accurate evaluation system. This evaluation method cannot accurately quantify the patient's rehabilitation progress, cannot timely detect problems during the rehabilitation process and adjust the training strategy, and is not conducive to the patient obtaining the best rehabilitation effect. For example, for the recovery of complex neuromuscular functions, the existing evaluation methods are difficult to comprehensively reflect the degree of recovery and existing problems. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an intelligent rehabilitation training method and system based on artificial intelligence and virtual reality. By integrating advanced artificial intelligence algorithms, virtual reality technology, and multi-modal data acquisition and analysis technology, it aims to achieve the precision, personalization, and intelligence of rehabilitation training, and improve the effect and quality of rehabilitation training.

[0008] To solve the above technical problem, an embodiment of the present invention provides the following technical solution: An intelligent rehabilitation training method based on artificial intelligence and virtual reality, including:

[0009] S1. The user wears an intelligent wearable device to collect multi-modal data in real time and preprocess the multi-modal data; the multi-modal data includes the user's electroencephalogram signal, electromyogram signal, physiological characteristic signal, and motion signal;

[0010] S2. Input the preprocessed multi-modal data into an artificial intelligence training model, extract the key features in the multi-modal data, and use a graph convolutional network and an attention mechanism algorithm to fuse and process the multi-modal data features;

[0011] S3. Use the fused multi-modal data features extracted to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation;

[0012] S4. According to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goal and physical state, use a reinforcement learning algorithm to generate a customized training plan; the training plan includes training content, difficulty level, and training duration parameters;

[0013] S5. The user enters a virtual reality interaction model and conducts rehabilitation training according to the generated training plan, real-time monitors the user's actions and physiological state, the digital twin model synchronizes the user's actions in real time, and the virtual reality scene dynamically adjusts according to the user's situation;

[0014] S6. According to the user's physiological indicators, motion data after training, and the analysis results of the twin optimization driven by multi-modal data, evaluate the rehabilitation effect, and feedback the rehabilitation evaluation results to the artificial intelligence processing model to further optimize the training plan and the digital twin model.

[0015] Optionally, the user's electroencephalogram signal is acquired using dry electrode technology; the electromyogram signal is collected using a graphene-based flexible electrode; the physiological characteristic signals include heart rate, temperature, and pulse, and are acquired using corresponding sensors; the motion signal is acquired using a pressure sensor and an acceleration sensor on the wearable device.

[0016] Optionally, extract the key features in the multi-modal data, and use the graph convolutional network and the attention mechanism algorithm to fuse the multi-modal data features. Specifically:

[0017] S21. Represent the multi-modal data as a graph structure. Let the graph G=(V, E), where V is the set of nodes and E is the set of edges. The node feature matrix X∈R N×D , where N is the number of nodes and D is the dimension of the node features; the adjacency matrix A∈R N×N , which is used to describe the connection relationship between nodes. A ij indicates whether there is a connection between node i and node j;

[0018] S22. Perform a convolution operation on the graph structure data through the graph convolutional network to extract the feature information of the nodes. The formula is:

[0019]

[0020] represents the node feature matrix of the l-th layer. F (l) is the feature dimension of the l-th layer. Initially, H (0) = X; where I is the identity matrix; is 's degree matrix,

[0021] is the learnable weight matrix of the l-th layer; σ is the activation function; after L layers of GCN processing, the final feature matrix H (L) is obtained;

[0022] S23. Use the features H (L) extracted by the GCN as the input to perform single-head attention calculation. The formula is:

[0023]

[0024] is the query projection matrix, which projects the GCN features into the query space. d q is the dimension of the key;

[0025] is the key projection matrix, which projects the GCN features into the key space. d k is the dimension of the key; is the value projection matrix that projects the GCN features into the value space, d v is the dimension of the key;

[0026] softmax is a function used to normalize the attention scores; is the scaling factor to prevent the dot product result from being too large;

[0027] S24. Process the data in parallel through multiple independent single-head attention calculations to capture multiple features and relationships of the data from different representation subspaces:

[0028] MultiHead(H (L) ) = C(head1,…,head h )W o

[0029] where head i = Attention(H (L) W i Q ,H (L) W i K ,H (L) W i V ), i = 1,…, h, MultiHead(H (L) ) is the final output result of the multi-head attention mechanism, head i is the result of the i-th single-head attention calculation, C(head1,…,head h ) represents the concatenation operation of the h single-head attention calculation results from head1 to head h ; h is the number of heads, is the query, key, and value projection matrix for the i-th head; is the linear transformation matrix for concatenating the results, which maps the output of the multi-head attention back to the same dimensional space as the GCN features;

[0030] S25. Fuse the output of the multi-head attention mechanism with the features extracted by the GCN to obtain the final fused features, and use weighted summation for fusion:

[0031] H f = αH (L) + (1 - α)H Attention

[0032] where H f is the finally obtained fused feature, which is the result of fusing the features extracted by the graph convolutional network GCN and the output features of the multi-head attention mechanism, H Attention = MultiHead(H (L)) is the output of the multi-head attention mechanism, and α is the fusion weight parameter, where α ∈ [0, 1].

[0033] Optionally, in step S3, the multi-modal data features extracted after fusion are used to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation. Specifically:

[0034] S31. Assume that the multi-modal data includes λ modalities, and the set of multi-modal data feature vectors after preprocessing is:

[0035]

[0036] The attention mechanism is used to dynamically allocate weights for each modality and fuse them into a unified feature:

[0037]

[0038] where α i is the weight of the i-th modality, z is the unified feature after fusion, W a is the weight matrix of the attention network, and b a is the bias vector of the attention network;

[0039] S32. The digital twin model needs to map the physiological-motor state of the patient in real time and is described by a state space model:

[0040] s t = f(s t-1 , z t ) + ε1

[0041] where s t is the state vector at time t, f is the state transition function, specifically the LSTM neural network function, Z t is the fused feature vector, and ε1 is the process noise;

[0042] S33. Based on the current state s t , predict the state sequence for the next T steps using the Seq2Seq model:

[0043] The encoder inputs the historical state sequence s 1:t :

[0044] The decoder generates future predictions

[0045]

[0046] where s 1:t is the historical state sequence, that is, the set of state vectors from time 1 to time t; h t encis the hidden state of the encoder at time t; is the predicted state sequence for the next T steps, is the predicted state vector at future time k (k = t + 1, …, t + T); is the hidden state of the decoder at time k, is the predicted state vector at time τ - 1, W p is the decoder weight matrix, b p is the decoder bias vector.

[0047] Optionally, according to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical conditions, a customized training plan is generated using a reinforcement learning algorithm, specifically:

[0048] S41. Define the state space S as the fused feature vector and the user's rehabilitation goals, the action space A as the adjustment options of various training parameters, and the reward function R includes the action completion situation and the change of physiological goals;

[0049] S42. Construct a policy network using the Actor-Critic architecture, input the state s t output the action probability distribution or mean variance: u t , σ t = MLP θ (s t ), the continuous action sampling is a t ~ N(u t , σ t ); θ is the parameter of the Actor network, which determines the structure and behavior of the network; MLP θ is a multi-layer perceptron with θ as the parameter, which is used to process the input state and generate action-related outputs; u t is the mean of the continuous action distribution, representing the central tendency of the action; σ t is the standard deviation of the continuous action distribution, measuring the uncertainty of the action;

[0050] The Critic network is used to evaluate the state value and guide the Actor to update. The formula is is the parameter of the Critic network, which is continuously adjusted through learning to accurately evaluate the state value, is a multi-layer perceptron with as the parameter, which is used to calculate the value of the state;

[0051] Reinforcement learning training, maximizing the constrained policy improvement. The formula is

[0052]

[0053] where, is the importance sampling ratio, π θ is the current policy, π old is the old policy, used to measure the difference between the new and old policies;

[0054] is the advantage function estimate, r t is the future reward, γ is the discount factor, ranging from [0, 1], used to measure the importance of future rewards; ε is the clipping parameter, used to limit the magnitude of policy updates; E t represents the expectation over time step t, used to average the objective function over multiple time steps;

[0055] S43. Collect trajectory data η = {(s t , a t , r t )}, obtained by interacting the current policy with the environment. These data contain state, action, and reward information at different time steps, used for subsequent learning and updates;

[0056] Calculate the advantage estimate and the return where T is the final time step in the trajectory data, and the return R t is the cumulative discounted reward from time step t to the final time step, t represents the current time step; r τ is the immediate reward obtained at time step τ;

[0057] Update the Critic network to minimize the value error, and the objective function is E t represents the expectation operation with respect to time step t. By adjusting the parameters of the Critic network make the estimated state value t equivalent to the actual return R;

[0058] Update the Actor network to maximize the objective function ζ CLIP (θ). By optimizing the parameters θ of the Actor network, make the current policy obtain the highest reward;

[0059] S44. During user training, the policy network generates an action a t according to the current state s t , dynamically adjusts the training plan, and at the same time adds hard constraint conditions to ensure that the action is within the safe range, that is, a t ← Clip(a t , a min , a max ), where a min is the minimum value of the action, and a max is the maximum value of the action.

[0060] Optionally, the user enters the virtual reality interaction model and performs rehabilitation training according to the generated training plan. Specifically:

[0061] S51. Generate a corresponding virtual reality training scenario according to the training plan, and the scenario should match the action type, rehabilitation goal, etc. in the training plan;

[0062] The training scenario consists of multiple elements, and each element is represented by a three-dimensional vector P i =[x i , y i , z i to represent its position in the virtual space, the attribute vector of the element is represented by a i The position vector is represented by p i The scenario generation function S is expressed as:

[0063] S ={(p i , a i )|i = 1, 2,..., n}

[0064] Use the rendering algorithm to render the elements in the scenario onto the screen. For each pixel point (u, v), its color value C(u, v) is calculated by the following formula:

[0065] C(u, v)=R(S, r(u, v))

[0066] where r(u, v) is the light vector starting from the camera position and passing through the pixel point (u, v), and R is the ray tracing algorithm model function;

[0067] S52. During the training process, monitor the user's rehabilitation training actions in real time, compare them with the requirements in the training plan, and calculate the matching degree, where:

[0068] Action type matching. Let the set of action types specified in the training plan be A =[A1, A2,..., A m , m be the number of action types, A i is a binary variable indicating whether the i-th action type is included. The set of action types actually performed by the user obtained through the action recognition model is Use the Hamming distance to measure the matching degree, and the formula is:

[0069]

[0070] The action type matching degree M A is:

[0071] The intensity range of the j-th action specified in the training plan The intensity of the user's actual execution of the action measured by the sensor is I j , the action intensity matching index M I is:

[0072]

[0073] The target frequency of the action specified in the training plan is f t , within the time interval [t1, t2], the number of times the user executes the action is N t , then the actual action frequency The action frequency matching error E f is:

[0074]

[0075] The action frequency matching degree M f : M f = 1 - E f

[0076] The total training duration specified in the training plan is T t , the currently trained duration is T e , the training duration matching index M T is:

[0077]

[0078] S53. According to the action type, action intensity, action frequency, and training duration matching results, when it is found that there is a deviation between the user's training action and the training plan, a control instruction or an intervention signal is sent to the smart wearable device to provide auxiliary force or feedback information to compensate for the deviation of the training control data.

[0079] Optionally, the digital twin model synchronizes the user's actions in real time, and the virtual reality scene is dynamically adjusted according to the user's situation. Specifically:

[0080] S54. The user's action data and physiological data are monitored in real time and transmitted to the digital twin model. The multi-modal data is fused and processed, and the digital twin model is updated using the fused feature vector. The digital twin model is described by a state space model, and the formula is:

[0081] S t = f(S t-1 , Z t ) + ε1

[0082] where S t is the state vector at time t, f is the state transition function, Z tis the fused feature vector, and ε1 is the process noise; by continuously inputting new fused data, the digital twin model updates its own state in real time to synchronize with the user's actions and physiological states;

[0083] S55. Synthesize the user's action features and physiological features, evaluate the user's training status and fatigue level, set the action completion index C, and use the dynamic time warping algorithm DTW to calculate the actual action trajectory T a with the standard action trajectory T s , and the formula is where len represents the length of the trajectory, and the user's fatigue level is evaluated based on heart rate variability, electromyogram signal characteristics, and electroencephalogram signal characteristics;

[0084] S56. According to the evaluation results of the user's situation, formulate a virtual reality scene adjustment strategy: if the action completion is low and the user is not fatigued, reduce the training difficulty; if the user is fatigued, pause the training or switch to an easy relaxation scene;

[0085] S57. According to the scene adjustment decision, adjust the parameters of the objects in the virtual scene or switch the scene through 3D modeling technology.

[0086] Optionally, evaluate the rehabilitation effect according to the user's physiological indicators, motion data, and the twin optimization analysis results driven by multi-modal data, specifically:

[0087] S61. According to the normal range of physiological indicators and the rehabilitation goal, evaluate each physiological state of the user to obtain the comprehensive physiological index score S p ; use the dynamic time warping algorithm DTW to calculate the similarity between the user's actual action trajectory and the standard action trajectory, and calculate the comprehensive score S of the motion data M ; calculate the mean square error between the predicted value and the actual value of the digital twin model, and set the score S for the twin optimization analysis result according to the evaluation result T ;

[0088] S62. According to the emphasis of the training goal, set the corresponding weights ω p , ω M , ω T for the comprehensive physiological index score, the comprehensive motion data score, and the twin optimization analysis result score respectively, and calculate the comprehensive effect score S:

[0089] S = ω P S p + ω M S M + ω T S T ;

[0090] S63. Set corresponding score thresholds according to the comprehensive score S, and divide the rehabilitation effect into grades of excellent, good, medium, and poor.

[0091] Optionally, for issuing a control instruction or an intervention signal to the smart wearable device, the intervention signal generation steps are as follows:

[0092] S531. Use dry electrodes on the smart wearable device to collect electroencephalogram (EEG) signals, and use flexible electrodes based on graphene to collect electromyogram (EMG) signals. Perform preprocessing on the collected original EEG and EMG signals, including but not limited to filtering and amplification.

[0093] S532. Apply algorithms such as graph convolutional network and attention mechanism in the artificial intelligence training model to extract features from the preprocessed EEG and EMG signals respectively. The extracted features include but are not limited to the frequency, amplitude, phase, and time-frequency characteristics of the signals. Calculate the energy distribution and correlation coefficient indicators of the EEG and EMG signals in different frequency bands and draw data maps.

[0094] S533. Combine the standard database of EEG-EMG signal characteristics under normal physiological conditions established in advance, compare the current extracted EEG-EMG signal characteristics of the patient with the standard database, and combine the patient's clinical symptoms, medical history, and physiological examination results to determine the specific nerve root position and fault type where the nerve control circuit fails.

[0095] S534. According to the type, position, and severity of the nerve control circuit failure, determine the parameters of the spatio-temporal electrical stimulation signal. Adopt micro-current stimulation technology and send the spatio-temporal electrical stimulation signal to the specific nerve root through the electrode array on the smart wearable device according to the set parameters. The parameters of the spatio-temporal electrical stimulation signal include but are not limited to the stimulation frequency, intensity, duration, pulse waveform, and spatial distribution of the stimulation.

[0096] The present invention also proposes an intelligent rehabilitation training system based on artificial intelligence and virtual reality, including:

[0097] A smart wearable device module for real-time collection of multimodal data and preprocessing of the multimodal data. The multimodal data includes user EEG signals, EMG signals, physiological feature signals, and motion signals.

[0098] A signal feature extraction and fusion module that inputs the preprocessed multimodal data into the artificial intelligence training model, extracts key features from the multimodal data, and performs fusion processing on the multimodal data features using graph convolutional network and attention mechanism algorithms.

[0099] A digital twin model construction module that uses the multimodal data features extracted after fusion to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation.

[0100] The training program generation module generates a customized training program using a reinforcement learning algorithm based on the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical condition; the training program includes training content, difficulty level, and training duration parameters;

[0101] The rehabilitation training module performs rehabilitation training according to the generated training program, monitors the user's movements and physiological state in real time, synchronizes the user's movements with the digital twin model in real time, and dynamically adjusts the virtual reality scene according to the user's situation;

[0102] The rehabilitation results evaluation module evaluates the rehabilitation effect based on the user's physiological indicators, motion data and multimodal data-driven twin optimization analysis results after training, and feeds the rehabilitation evaluation results back to the artificial intelligence processing model to further optimize the training plan and digital twin model.

[0103] The beneficial effects of the above technical solution of the present invention are as follows:

[0104] 1. The present invention can comprehensively obtain information such as user's EEG, EMG, physiological characteristics and motion signals through multimodal data acquisition technology. Through graph convolutional networks and attention mechanism algorithms, these data can be deeply integrated, key features can be accurately extracted, and individual differences between different patients can be explored. On this basis, combined with the real-time simulation and reinforcement learning algorithms of the digital twin model, a training plan is tailored for each patient. During the training process, the digital twin model synchronizes the user status in real time, and the reinforcement learning algorithm adjusts the plan in time according to the changes to ensure that the training accurately meets the patient's needs and improves the rehabilitation effect.

[0105] 2. The present invention creates a rich and realistic training scene for patients through the virtual reality interactive model, making patients feel as if they are in a real environment during training, greatly improving the fun and immersion of training. At the same time, the system monitors the movements in real time and gives timely feedback. When patients are affirmed and assisted by correct movements, they will feel progress, thereby enhancing their confidence in rehabilitation, improving their enthusiasm and compliance in training, increasing the duration and frequency of training, and further promoting the rehabilitation process.

[0106] 3. During the training process, the system of the present invention monitors the user's movements in real time, compares and calculates the degree of matching with the training plan, and comprehensively monitors the movement type, intensity, frequency and training duration. Once a deviation is found, the smart wearable device will immediately issue a control instruction or intervention signal. For example, when the user's strength training movements are not standardized, it will promptly remind and correct them to avoid muscle and joint injuries due to wrong movements. In addition, by comprehensively analyzing the user's movements and physiological characteristics, the system can accurately assess the training status and fatigue level. When it is judged that the user is tired, the virtual reality scene will automatically pause the training or switch to the relaxation scene, effectively preventing overtraining and ensuring the safety of patients during rehabilitation training.

[0107] 4. The present invention evaluates the rehabilitation effect from three dimensions: physiological indicators, motion data, and digital twin model optimization analysis results. Physiological indicators reflect the recovery of body functions, motion data reflect the degree of standardization of action execution, and twin optimization analysis results show the fit between model prediction and reality. This multi-dimensional evaluation avoids the limitations of a single indicator, making the evaluation results more comprehensive and objective, and setting weights according to training goals to derive a comprehensive effect score, dividing the rehabilitation effect into different levels, achieving quantitative evaluation, and reducing the subjectivity of human evaluation. According to the evaluation results, the problem can be identified, the training plan can be adjusted in a targeted manner, the digital twin model can be optimized, a virtuous circle of rehabilitation training can be formed, and the quality of rehabilitation can be continuously improved.

[0108] 5. Aiming at the situation where limb diseases are mostly caused by nervous system problems, the present invention collects EEG-EMG signals, uses artificial intelligence models to extract signal frequency, amplitude and other characteristics, and combines clinical information with standard databases to accurately determine the location and type of neural control circuit faults. According to the fault situation, the spatiotemporal electrical stimulation signal parameters are determined, and stimulation signals are sent to specific nerve roots through smart wearable devices to achieve accurate nervous system rehabilitation therapy. This process can promote the recovery of limb function from the root, improve the accuracy and effectiveness of rehabilitation treatment, and provide more powerful support for the rehabilitation of patients with damaged nervous systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 This is a flow chart of the intelligent rehabilitation training method based on artificial intelligence and virtual reality of the present invention;

[0110] Figure 2 This is a functional block diagram of the intelligent rehabilitation training system module based on artificial intelligence and virtual reality of the present invention;

[0111] Figure 3 This is a hardware principle block diagram of the intelligent rehabilitation training system based on artificial intelligence and virtual reality of the present invention. DETAILED DESCRIPTION

[0112] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0113] like Figure 1 As shown, the present invention provides an intelligent rehabilitation training method based on artificial intelligence and virtual reality, comprising:

[0114] S1. The user wears a smart wearable device, collects multimodal data in real time, and pre-processes the multimodal data; the multimodal data includes the user's EEG signals, EMG signals, physiological characteristic signals, and motion signals. The user wears a smart wearable device with integrated advanced sensors, such as an EEG cap, EMG patch, smart bracelet, etc. The device collects physiological characteristic signals such as EEG, EMG, heart rate, body temperature, and joint motion signals in real time. At the same time, in addition to collection, smart wearable devices also include traction equipment, power equipment, and massage equipment based on moving parts such as motors, lead screws, and reducers to assist rehabilitation training.

[0115] S2. Input the preprocessed multimodal data into the artificial intelligence training model, extract the key features in the multimodal data, and use the graph convolutional network and attention mechanism algorithm to fuse the multimodal data features. The model uses a deep learning algorithm to extract key features, then uses a graph convolutional network to capture the topological relationship between data, and combines the attention mechanism algorithm to focus on important features, achieving deep fusion of multimodal data and tapping the potential value of data.

[0116] S3. Use the multimodal data features extracted after fusion to build a digital twin model of the patient for real-time dynamic mapping and predictive simulation. This model can map the patient's physiological-motor state in real time, and dynamically predict the patient's future state through the state space model and Seq2Seq model to assist in the analysis of the rehabilitation process.

[0117] S4. Based on the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical condition, a customized training plan is generated using a reinforcement learning algorithm; the training plan includes training content, difficulty level, and training duration parameters. Based on the fused data features and the analysis results of the digital twin model, the user's rehabilitation goals and current physical condition are comprehensively considered, and the strategy is continuously optimized using a reinforcement learning algorithm. A customized training plan containing targeted training content, adaptive difficulty levels, and reasonable training duration parameters is generated to ensure the scientificity and effectiveness of the training.

[0118] S5. The user enters the virtual reality interactive model and performs rehabilitation training according to the generated training program. The user's movements and physiological state are monitored in real time. The digital twin model synchronizes the user's movements in real time, and the virtual reality scene is dynamically adjusted according to the user's situation. The system monitors the user's movements and physiological state in real time through sensors, and the digital twin model synchronizes the movements. At the same time, the virtual reality scene is dynamically adjusted according to the user's performance, which increases the fun and adaptability of the training.

[0119] S6. Evaluate the rehabilitation effect based on the user's physiological indicators, motion data, and the results of the twin optimization analysis driven by multimodal data after training, and feed back the rehabilitation evaluation results to the artificial intelligence processing model to further optimize the training plan and digital twin model. After the user completes the training, collect their physiological indicators and motion data, and combine them with the results of the twin optimization analysis driven by multimodal data to comprehensively evaluate the rehabilitation effect. Feedback the evaluation results to the artificial intelligence processing model to further optimize the training plan and digital twin model to achieve continuous improvement of rehabilitation training.

[0120] In this embodiment, the user's EEG signal is acquired using dry electrode technology; the electromyographic signal is acquired using graphene-based flexible electrodes; the physiological characteristic signals include heart rate, temperature, and pulse, which are acquired using corresponding sensors; and the motion signal is acquired using pressure sensors and acceleration sensors on the wearable device. The electromyographic signal is acquired using graphene-based flexible electrodes. Graphene has excellent electrical properties and good flexibility, so that the flexible electrode can fit the skin closely without causing too much burden on the skin. At the same time, its high sensitivity can accurately capture the weak electromyographic signals generated during muscle contraction and relaxation, providing an accurate basis for evaluating muscle function status and muscle strength changes, and helping rehabilitation therapists understand the patient's muscle recovery. Physiological characteristic signals include heart rate, temperature, pulse, etc., which are acquired using corresponding sensors respectively. The heart rate sensor uses the principle of photoelectric induction to monitor the changes in the absorption and reflection of light by blood, track the heart beat frequency in real time, and reflect the heart function and exercise intensity. The temperature sensor uses a high-precision thermistor element, which can quickly sense the changes in skin surface temperature and assist in judging the body's metabolic state and fatigue level. The pulse sensor obtains pulse information by sensing the pulsation of arterial blood vessels, providing data support for a comprehensive assessment of the health of the cardiovascular system. The motion signal is obtained using the pressure sensor and accelerometer on the wearable device. The pressure sensor can keenly capture the pressure distribution and changes at the contact point between the user's limbs and the device during exercise, and analyze the force conditions during exercise, which is of great significance for judging whether the exercise posture is correct and whether the exercise intensity is appropriate. The accelerometer can measure the acceleration of the object's movement, and accurately record information such as the user's movement speed changes, movement direction changes, and movement amplitude and rhythm.

[0121] In this embodiment, the key features in the multimodal data are extracted, and the multimodal data features are fused using a graph convolutional network and an attention mechanism algorithm, specifically:

[0122] S21. Represent multimodal data as a graph structure. Let G = (V, E), where V is the node set, E is the edge set, and the node feature matrix X∈R N×D , where N is the number of nodes, D is the dimension of node features; the adjacency matrix A∈R N×N, used to describe the connection relationship between nodes, A ij Indicates whether there is a connection between node i and node j;

[0123] S22. Perform a convolution operation on the graph structure data through a graph convolutional network to extract the feature information of the nodes. The formula is:

[0124]

[0125] Represents the node feature matrix of the l-th layer, F (l) Is the feature dimension of the l-th layer. Initially, H (0) = X; Where I is the identity matrix; Is The degree matrix of

[0126] Is the learnable weight matrix of the l-th layer; σ is the activation function; After L layers of GCN processing, the final feature matrix H (L) ;

[0127] S23. Take the feature H (L) extracted by GCN as the input and perform single-head attention calculation. The formula is:

[0128]

[0129] Is the query projection matrix that projects the GCN feature into the query space, d q Is the dimension of the key;

[0130] Is the key projection matrix that projects the GCN feature into the key space, d k Is the dimension of the key;

[0131] Is the value projection matrix that projects the GCN feature into the value space, d v Is the dimension of the key;

[0132] softmax is a function used to normalize the attention scores; Is the scaling factor to prevent the dot product result from being too large;

[0133] S24. Process the data in parallel through multiple independent single-head attention calculations to capture multiple features and relationships of the data from different representation subspaces:

[0134] MultiHead(H (L) ) = C(head1,…,head h )W o

[0135] Among them, head i = Attention(H (L) W i Q , H (L) W i K , H (L) W i V ), i = 1, …, h, MultiHead(H (L) ) is the final output result of the multi - head attention mechanism, head i is the result of the i - th single - head attention calculation, C(head1, …, head h ) represents the concatenation operation of the h single - head attention calculation results head1 to head h ; h is the number of heads, is the query, key, and value projection matrix of the i - th head; is the linear transformation matrix for the concatenated result, which maps the output of the multi - head attention back to the same dimensional space as the GCN features;

[0136] S25. Fuse the output of the multi - head attention mechanism with the features extracted by the GCN to obtain the final fused features, and use weighted summation for fusion:

[0137] H f = αH (L) + (1 - α)H Attention

[0138] Among them, H f is the finally obtained fused feature, which is the result of fusing the features extracted by the graph convolutional network GCN with the output features of the multi - head attention mechanism, H Attention = MultiHead(H (L) ) is the output of the multi - head attention mechanism, α is the fusion weight parameter, and α ∈ [0, 1].

[0139] Multimodal data includes various types such as electroencephalogram (EEG), electromyogram (EMG), physiological characteristics, and motion signals. Each type of data reflects the user's physical state from different perspectives. By extracting key features, redundant information can be removed, and the focus can be on the information that is of great value for rehabilitation training. For example, specific frequency band signals in EEG may be closely related to the motion intention. Extracting these key features can capture the user's motion intention more accurately. The application of graph convolutional network and attention mechanism algorithms can explore the potential relationships between different modal data, improve the feature expression ability, and enable the model to understand the user's physical state more comprehensively and deeply. By analyzing the fused features, combined with the prediction results of the digital twin model, the user's rehabilitation goals, and the physical state, the training plan generated using the reinforcement learning algorithm can better meet the actual needs of the patient. For example, if the fused features show that a certain muscle of the patient is weak and the motion coordination is poor, the training plan can specifically increase the strength training and coordination training content of this muscle, and reasonably adjust the difficulty level and training duration.

[0140] The rehabilitation needs and physical conditions of different patients vary greatly. Multimodal data fusion processing can fully consider these individual differences. Through the unique analysis of the multimodal data of each patient, the system can customize a rehabilitation training plan for them, avoiding the "one-size-fits-all" training mode. The fused multimodal data features provide richer and more accurate information for constructing the digital twin model, enabling the model to more precisely map the physiological-motor state of the patient in real time and conduct more reliable predictive simulations.

[0141] In this embodiment, step S3 uses the multimodal data features extracted after fusion to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation, specifically as follows:

[0142] S31. Assume that the multimodal data includes λ modalities, and the set of multimodal data feature vectors after preprocessing is:

[0143]

[0144] Use the attention mechanism to dynamically allocate weights for each modality and fuse them into a unified feature:

[0145]

[0146] where α i is the weight of the i-th modality, z is the unified feature after fusion, W a is the weight matrix of the attention network, and b a is the bias vector of the attention network;

[0147] S32. The digital twin model needs to map the physiological-motor state of the patient in real time and is described using the state space model:

[0148] st = f(s t-1 , z t ) + ε1

[0149] where s t is the state vector at time t, f is the state transition function, specifically the LSTM neural network function, Z t is the fused feature vector, and ε1 is the process noise;

[0150] S33. Based on the current state s t , predict the state sequence for the next T steps using the Seq2Seq model:

[0151] The encoder inputs the historical state sequence s 1:t :

[0152] The decoder generates the future prediction

[0153]

[0154] where s 1:t is the historical state sequence, i.e., the set of state vectors from time 1 to time t; is the hidden state of the encoder at time t; is the predicted state sequence for the next T steps, is the predicted state vector at future time k (k = t + 1,..., t + T); is the hidden state of the decoder at time k, is the predicted state vector at time τ - 1, W p is the weight matrix of the decoder, b p is the bias vector of the decoder.

[0155] The rehabilitation process of patients is complex and variable, involving coordinated changes in multiple physiological systems such as the nervous, muscular, and cardiovascular systems. The multi-modal data fusion features cover rich information such as electroencephalogram, electromyogram, physiological features, and motion signals, and can comprehensively present the patient's physical state from multiple dimensions. The digital twin model constructed based on these features can achieve real-time dynamic mapping of the patient's physiological-motor state, just like creating a "clone" of the patient in the virtual world, accurately reflecting the patient's current real situation, enabling rehabilitation therapists and medical staff to understand the patient's real-time state in a timely and intuitive manner.

[0156] Predictive simulation is an important function of the digital twin model. By continuously monitoring and analyzing the current state of the patient, the model can predict the trend of the patient's state change in the future based on the fused data features. This helps to detect potential problems in the rehabilitation process in advance, such as predicting the possible fatigue or overtraining risk of a certain muscle of the patient, providing a basis for timely adjustment of the rehabilitation training plan, avoiding secondary injuries caused by unreasonable training, and ensuring the safety and effectiveness of the patient's rehabilitation process.

[0157] Secondly, the rehabilitation path of each patient is unique, and the simulation function of the digital twin model can conduct personalized analysis according to individual differences. Combining the patient's rehabilitation goals, the model can simulate the impact of different training plans on the patient's physical state, helping the rehabilitation team find the most suitable training content, difficulty level, and training duration for the patient. As the rehabilitation process progresses, the model is continuously optimized according to the real-time dynamically mapped data, providing a scientific reference for adjusting the rehabilitation plan, ensuring that the rehabilitation training always meets the actual needs of the patient, and improving the rehabilitation effect.

[0158] In this embodiment, generating a customized training plan by using the reinforcement learning algorithm according to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical state, specifically:

[0159] S41. Define the state space S as the fused feature vector and the user's rehabilitation goal, the action space A as the adjustment options of various training parameters, and the reward function R includes the action completion situation and the change of physiological goals;

[0160] S42. Construct a policy network using the Actor-Critic architecture, input the state s t Output the action probability distribution or mean variance: u t , σ t = MLP θ (s t ), the continuous action sampling is a t ~ N(u t , σ t ); θ is the parameter of the Actor network, which determines the structure and behavior of the network; MLP θ is a multi-layer perceptron with θ as the parameter, used to process the input state and generate action-related outputs; u t is the mean of the continuous action distribution, representing the central tendency of the action; σ t is the standard deviation of the continuous action distribution, measuring the uncertainty of the action;

[0161] The Critic network is used to evaluate the state value and guide the Actor to update. The formula is are the parameters of the Critic network, which are continuously adjusted through learning to accurately evaluate the state value. is a multi-layer perceptron with as parameters, used to calculate the value of the state;

[0162] For reinforcement learning training, to maximize the constrained policy improvement, the formula is

[0163]

[0164] where is the importance sampling ratio, π θ is the current policy, π old is the old policy, used to measure the difference between the new and old policies;

[0165] is the advantage function estimate, r t is the future reward, γ is the discount factor, ranging between [0, 1], used to measure the importance of future rewards; ε is the clipping parameter, used to limit the amplitude of policy updates; E t represents the expectation over time step t, used to average the objective function over multiple time steps;

[0166] S43. Collect trajectory data η = {(s t , a t , r t )}, obtained by interacting the current policy with the environment. These data contain state, action, and reward information at different time steps, used for subsequent learning and updates;

[0167] Calculate the advantage estimate and the return where T is the final time step in the trajectory data, and the return R t is the cumulative discounted reward from time step t to the final time step, t represents the current time step; r τ is the immediate reward obtained at time step τ;

[0168] Update the Critic network to minimize the value error, and the objective function is E t represents the expectation operation with respect to time step t. By adjusting the parameters of the Critic network, make the estimated state value t equivalent to the actual return R;

[0169] Update the Actor network to maximize the objective function ζ CLIP (θ). By optimizing the parameters θ of the Actor network, make the current policy obtain the highest reward;

[0170] S44. During user training, the policy network generates an action a based on the current state s t and dynamically adjusts the training plan. At the same time, hard constraint conditions are added to ensure that the action is within the safe range, that is, a t ← Clip(a t , a t , a min , a max ), where a min is the minimum value of the action and a max is the maximum value of the action.

[0171] The fused data features include multi - aspect information such as the user's EEG, EMG, physiological characteristics, and motion signals. The digital twin model can reflect the user's state changes in real - time and accurately. Combining this information and the user's rehabilitation goals, a training plan can be customized for each user using the reinforcement learning algorithm. For example, for a user recovering from a fracture and a user recovering from a neurological disease, different training contents (such as strength training, coordination training, etc.), difficulty levels, and training durations can be set according to their respective unique situations, truly achieving individualized training and improving the pertinence and effectiveness of rehabilitation training.

[0172] The user's physical state is constantly changing during the rehabilitation process. The digital twin model can monitor and feedback these changes in real - time, and the reinforcement learning algorithm can adjust the training plan in a timely manner according to the new data. For example, when a user's strength of a certain muscle has improved after a period of training, the training plan can appropriately increase the training difficulty or adjust the training focus to ensure that the training plan always fits the user's current physical state and continuously and effectively promotes rehabilitation.

[0173] Through continuous trial - and - error and learning, the reinforcement learning algorithm can find the optimal solution among numerous possible training plans. It can comprehensively consider various factors, such as training safety, effectiveness, and the user's tolerance level, based on the analysis results of the fused data features and the digital twin model, and generate a training plan with the goal of maximizing the rehabilitation effect. For example, on the premise of avoiding injuries caused by over - training, it can improve the user's rehabilitation speed and quality as much as possible, making the training resources more reasonably utilized.

[0174] In this embodiment, the user enters the virtual reality interaction model and conducts rehabilitation training according to the generated training plan. Specifically:

[0175] S51. Generate a corresponding virtual reality training scenario according to the training plan, and this scenario should match the action type, rehabilitation goal, etc. in the training plan;

[0176] The training scenario consists of multiple elements, and each element is represented by a three - dimensional vector P i =[x i , yi , z i represents its position in the virtual space, and the attribute vector of the element is represented by a i The position vector is represented by p i , and the scene generation function S is expressed as:

[0177] S = {(p i , a i ) | i = 1, 2, …, n}

[0178] Use the rendering algorithm to render the elements in the scene onto the screen. For each pixel point (u, v), its color value C(u, v) is calculated by the following formula:

[0179] C(u, v) = R(S, r(u, v))

[0180] where r(u, v) is the light vector starting from the camera position and passing through the pixel point (u, v), and R is the ray tracing algorithm model function;

[0181] S52. During the training process, the user's rehabilitation training actions are monitored in real time, compared with the requirements in the training plan, and the matching degree is calculated, where:

[0182] Action type matching. Let the set of action types specified in the training plan be A = [A1, A2, …, A m , m is the number of action types, and A i is a binary variable indicating whether it includes the i-th action type. The set of action types actually executed by the user obtained through the action recognition model is Use the Hamming distance to measure the matching degree, and the formula is:

[0183]

[0184] The action type matching degree M A is:

[0185] The intensity range of the j-th action specified in the training plan The intensity of the action actually executed by the user measured by the sensor is I j , and the action intensity matching index M I is:

[0186]

[0187] The target frequency of the action specified in the training plan is f t , and within the time interval [t1, t2], the number of times the user executes the action is N t , then the actual action frequency The action frequency matching error Ef is:

[0188]

[0189] Motion frequency matching degree M f : M f = 1 - E f

[0190] The total training duration specified in the training plan is T t , and the current training duration is T e , training duration matching index M T is:

[0191]

[0192] S53. According to the matching results of motion type, motion intensity, motion frequency and training duration, when it is found that the user's training motion deviates from the training plan, a control instruction or an intervention signal is sent to the smart wearable device to provide auxiliary force or feedback information to compensate for the deviation of the training control data.

[0193] Traditional rehabilitation training is often monotonous and boring, which easily makes patients bored and affects the enthusiasm and compliance of training. The virtual reality interaction model can create rich, diverse and vivid virtual scenes, such as simulating outdoor walking, sports competitions, etc. Patients can train in such scenes as if they were in a real environment, which can greatly improve the fun of training, attract patients to actively participate in training, thereby increasing the duration and frequency of training and enhancing the rehabilitation effect. According to the customized training plan generated in the early stage, the virtual reality interaction model can accurately adjust the training content and difficulty. For example, for patients who need to perform fine hand movement training, various hand operation tasks with different difficulties can be set in the virtual scene, such as picking up small objects, screwing screws, etc., and the difficulty and requirements of the tasks can be adjusted in real time according to the actual performance of the patients to ensure that the training plan can be accurately executed and personalized and precise rehabilitation training can be achieved. Since the customized rehabilitation training plan is carefully designed based on various factors such as the user's physical condition and rehabilitation goals. It also monitors and compares in real time to timely discover whether the user performs actions according to the requirements of the plan during training. For example, if the training plan requires the user to perform limb stretching actions at a specific speed and amplitude, the monitoring system can judge in real time whether the user's actions meet the standards. If not, the user can be reminded to adjust in time, so as to ensure that the training plan can be accurately executed.

[0194] The user's physical condition and rehabilitation progress are changing dynamically. The data on the degree of match between the monitored movements and the requirements of the training program can reflect whether the current training program is still suitable for the user. If it is found that the user cannot meet the movement requirements in the training program for a long time, it may mean that the difficulty of the program is too high; on the contrary, if the user can easily complete the movements and the degree of match is very high, the training difficulty may need to be appropriately increased. Based on these feedbacks, the rehabilitation team can adjust the training program in a timely and scientific manner so that it always fits the actual situation of the user and better promotes the rehabilitation process. At the same time, incorrect training movements may cause injuries to users, especially in rehabilitation training, when the user's body is often more fragile. Real-time monitoring and comparison of the degree of movement matching can timely detect errors or irregularities in the user's movements. For example, when the user is doing strength training, if the movement posture is incorrect, the monitoring system can detect and remind corrections in time to avoid unnecessary damage to muscles, joints, etc. due to wrong movements, and ensure the safety of users during training.

[0195] By monitoring the matching results of these aspects in real time and intervening in time once deviations are found, it can ensure that the user's training is always carried out in the predetermined direction. For example, if the training plan requires the user to perform hand grasping movements of a specific frequency, when the actual movement frequency is detected to be inconsistent, the assistance or feedback provided by the smart wearable device can help the user adjust, making the training more accurate and effective, and improving the rehabilitation effect. Wrong or inappropriate movement types, excessive movement intensity, unreasonable movement frequency, and inappropriate training duration may increase the risk of injury to users during rehabilitation training. Timely issuance of control instructions or intervention signals can quickly adjust when user movements deviate, avoiding injuries such as muscle strain and joint sprain caused by wrong movements. For example, when the user's movement intensity exceeds the plan setting and may cause a burden on the body, the smart wearable device provides auxiliary force to limit the intensity and protect the user's physical safety. Each user's physical condition and rehabilitation process are different, and the physical state will continue to change during the training process. This mechanism can dynamically adjust according to the user's real-time movement situation and provide personalized support. For example, as rehabilitation progresses, the user's physical functions improve, and the original training plan may need to be fine-tuned. At this time, by monitoring the matching of movements and plans, it can be discovered in time and adjusted accordingly, so that the training plan always fits the user's actual situation and continuously promotes rehabilitation. When users receive timely and accurate feedback and assistance during training, they can complete training movements more easily and feel their own progress. This positive experience helps to enhance users' confidence and enthusiasm for rehabilitation training and enhance their willingness to persist in training. For example, for users whose movement ability is limited due to injuries, the auxiliary force provided by smart wearable devices can help them complete movements that were originally difficult to complete, allowing users to see that their efforts are effective, so that they can participate in training more actively.

[0196] In this embodiment, the digital twin model synchronizes the user's actions in real time, and the virtual reality scene is dynamically adjusted according to the user's situation. Specifically:

[0197] S54. Real-time monitor the user's action data and physiological data and transmit them to the digital twin model. Perform fusion processing on the multi-modal data, and update the digital twin model using the fused feature vector. The digital twin model is described by a state space model, and the formula is:

[0198] S t = f(S t-1 , Z t ) + ε1

[0199] where S t is the state vector at time t, f is the state transition function, Z t is the fused feature vector, and ε1 is the process noise. By continuously inputting new fused data, the digital twin model updates its own state in real time to achieve synchronization with the user's actions and physiological states;

[0200] S55. Synthesize the user's action features and physiological features, evaluate the user's training status and fatigue level, set the action completion index C, and use the dynamic time warping algorithm DTW to calculate the actual action trajectory T a and the standard action trajectory T s , and the formula is where len represents the length of the trajectory, and evaluate the user's fatigue level based on heart rate variability, electromyogram signal characteristics, and electroencephalogram signal characteristics;

[0201] S56. According to the evaluation results of the user's situation, formulate a virtual reality scene adjustment strategy: if the action completion degree is low and the user is not fatigued, reduce the training difficulty; if the user is fatigued, pause the training or switch to a relaxing scene;

[0202] S57. According to the scene adjustment decision, adjust the parameters of the objects in the virtual scene or switch the scene through 3D modeling technology.

[0203] Each user has different physical conditions, rehabilitation progress, and reaction abilities. The digital twin model synchronizes user actions in real time and can accurately capture these individual differences. The virtual reality scene is dynamically adjusted accordingly, and customized training environments and tasks can be provided for different users, so that the training program is more in line with the actual situation of the individual, meets personalized rehabilitation needs, and improves the rehabilitation effect. Real-time synchronization and dynamic adjustment make the user's experience in the virtual reality scene more realistic and smooth. When the scene can respond to its own actions and state changes in a timely manner, users are more likely to immerse themselves in it and reduce their resistance to rehabilitation training. For example, the simulated outdoor mountaineering scene can adjust the slope and scenery according to the user's fatigue level, increase the fun of training, and improve the user's enthusiasm for active participation in training. By monitoring the user's actions and physical condition in real time, the digital twin model can promptly detect potential dangerous situations, such as excessive movements and physical fatigue. The virtual reality scene is dynamically adjusted accordingly, such as reducing the difficulty of training, pausing training, or providing auxiliary prompts to prevent users from being injured due to improper training, providing safety guarantees for rehabilitation training.

[0204] In this embodiment, the rehabilitation effect is evaluated based on the user's physiological indicators, motion data, and twin optimization analysis results driven by multimodal data after training, specifically:

[0205] S61. According to the normal range of physiological indicators and rehabilitation goals, the user's various physiological states are evaluated to obtain a comprehensive score of physiological indicators S p ; Dynamic time warping algorithm DTW is used to calculate the similarity between the user's actual motion trajectory and the standard motion trajectory, and the comprehensive score S of the motion data is calculated M ; Calculate the mean square error between the predicted value and the actual value of the digital twin model, and set a score S for the twin optimization analysis result based on the evaluation result T ;

[0206] S62. According to the training goal focus, set corresponding weights ω for the comprehensive score of physiological indicators, the comprehensive score of sports data, and the score of twin optimization analysis results. p ,ω M ,ω T , calculate the comprehensive effect score S:

[0207] S=ω P S p +ω M S M +ω T S T ;

[0208] S63. According to the comprehensive score S, a corresponding score threshold is set to divide the rehabilitation effect into excellent, good, medium, and poor levels.

[0209] This step starts from multiple dimensions including physiological indicators, motion data, and the optimization analysis results of the digital twin model. Physiological indicators reflect the recovery of physical functions, motion data reflects the standardization of action execution, and the twin optimization analysis results show the degree of fit between the model's prediction of the rehabilitation process and the actual situation. The multi-dimensional evaluation avoids the limitations of single indicators, making the evaluation results more comprehensively and objectively reflect the effects of rehabilitation training. Specific algorithms such as the Dynamic Time Warping (DTW) algorithm and the calculation of the mean square error are used to calculate the comprehensive scores for different data respectively, and the comprehensive effect score is obtained by setting weights. This quantitative method makes the evaluation more scientific and accurate, reducing the subjectivity and ambiguity of manual evaluation. The scores and grade divisions are clearly defined. If the rehabilitation effect is poor, it is possible to judge whether there are problems such as slow recovery of physiological functions, non-standard action execution, or deviation in model prediction based on the scores of each dimension, so as to adjust the training plan targeted and improve the effectiveness of subsequent rehabilitation training. The evaluation of the mean square error between the predicted value and the actual value of the digital twin model helps to discover the defects of the model, and the model can be optimized and improved through continuous comparative analysis.

[0210] In this embodiment, when issuing a control instruction or an intervention signal to the smart wearable device, the intervention signal generation step is as follows:

[0211] S531. Use dry electrodes on the smart wearable device to collect electroencephalogram (EEG) signals, and use flexible electrodes based on graphene to collect electromyogram (EMG) signals. Perform preprocessing on the collected original EEG and EMG signals, including but not limited to filtering and amplification.

[0212] S532. Apply algorithms such as graph convolutional network and attention mechanism in the artificial intelligence training model to extract features from the preprocessed EEG and EMG signals respectively. The extracted features include but are not limited to the frequency, amplitude, phase, and time-frequency characteristics of the signals. Calculate the energy distribution and correlation coefficient indexes of the EEG and EMG signals in different frequency bands and draw data maps.

[0213] S533. Combine the pre-established standard database of EEG-EMG signal characteristics under normal physiological conditions, compare the currently extracted patient's EEG-EMG signal characteristics with the standard database, and combine the patient's clinical symptoms, medical history, and physiological examination results to judge the specific nerve root position and fault type where the nerve control circuit fails.

[0214] S534. According to the type, position, and severity of the nerve control circuit failure, determine the parameters of the spatio-temporal electrical stimulation signal. Adopt the micro-current stimulation technology, and through the electrode array on the smart wearable device, send the spatio-temporal electrical stimulation signal to the specific nerve root according to the set parameters. The parameters of the spatio-temporal electrical stimulation signal include but are not limited to the stimulation frequency, intensity, duration, pulse waveform, and spatial distribution of the stimulation.

[0215] In human body rehabilitation training and recovery, not only limb rehabilitation should be achieved through movement training and muscle training, but also deeper rehabilitation training of the nervous system should be realized, because the sources of many limb diseases lie in the nervous system. Based on collecting electroencephalogram-electromyogram signals, this invention uses an artificial intelligence model to extract features from the preprocessed electroencephalogram and electromyogram signals respectively. The extracted features include but are not limited to the frequency, amplitude, phase, and time-frequency features of the signals. Combining with a standard database of electroencephalogram-electromyogram signal features under normal physiological conditions, comparing the currently extracted electroencephalogram-electromyogram signal features of the patient with the standard database, and combining the patient's clinical symptoms, medical history, and physiological examination results, the specific nerve root position and fault type where the nerve control circuit fails can be judged. According to the type, position, and severity of the nerve control circuit fault, a spatio-temporal electrical stimulation signal can be sent to a specific nerve root directionally, and precise rehabilitation physiotherapy of the nervous system can be achieved.

[0216] As Figure 2 shown, this invention also proposes an intelligent rehabilitation training system based on artificial intelligence and virtual reality, including:

[0217] An intelligent wearable device module 101, which is used to collect multi-modal data in real time and preprocess the multi-modal data; the multi-modal data includes the user's electroencephalogram signal, electromyogram signal, physiological feature signal, and motion signal;

[0218] A signal feature extraction and fusion module 102, which inputs the preprocessed multi-modal data into an artificial intelligence training model, extracts the key features in the multi-modal data, and uses a graph convolutional network and an attention mechanism algorithm to fuse and process the multi-modal data features;

[0219] A digital twin model construction module 103, which uses the fused multi-modal data features extracted to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation;

[0220] A training plan generation module 104, which generates a customized training plan using a reinforcement learning algorithm according to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical conditions; the training plan includes training content, difficulty level, and training duration parameters;

[0221] A rehabilitation training module 105, which conducts rehabilitation training according to the generated training plan, monitors the user's actions and physiological conditions in real time, the digital twin model synchronizes the user's actions in real time, and the virtual reality scene is dynamically adjusted according to the user's situation;

[0222] The rehabilitation result evaluation module 106 evaluates the rehabilitation effect based on the user's physiological indicators, motion data and multimodal data-driven twin optimization analysis results after training, and feeds the rehabilitation evaluation results back to the artificial intelligence processing model to further optimize the training plan and digital twin model.

[0223] like Figure 3 As shown, it is a hardware framework diagram of the present invention, wherein multimodal data such as EEG, EMG, physiological characteristics and motion signals of the human body are collected through intelligent wearable devices, and the signals are sent to the computer through a data acquisition card or other communication modules, and the computer extracts and analyzes the multimodal data through an artificial intelligence algorithm model, and then implements the above processing process through a digital twin model, a virtual reality model, etc., and finally the computer sends control instructions and electrical stimulation control signals corresponding to rehabilitation training to the traction device, the power-assisting device, the nerve stimulation device, and the massage device, and performs rehabilitation training on the patient through the corresponding strategy of the training program. Among them, the components for collecting EEG, EMG, physiological characteristics and motion signals of the human body are worn on the corresponding parts of the human body, and the traction device, the power-assisting device, the nerve stimulation device, and the massage device are external devices. During rehabilitation training, people need to fix their limbs on their training components.

[0224] In summary, the present invention can comprehensively obtain information such as user's EEG, EMG, physiological characteristics and motion signals through multimodal data acquisition technology. Through graph convolutional networks and attention mechanism algorithms, these data can be deeply integrated, key features can be accurately extracted, and individual differences between different patients can be explored. On this basis, combined with the real-time simulation and reinforcement learning algorithms of the digital twin model, a training plan is tailored for each patient. For patients recovering from fractures, the plan will focus on limb function recovery training; while patients recovering from neurological diseases will focus on neural function training. During the training process, the digital twin model synchronizes the user status in real time, and the reinforcement learning algorithm adjusts the plan in time according to the changes, such as increasing the training difficulty in time according to the improvement of muscle strength, to ensure that the training accurately meets the needs of patients and improves the rehabilitation effect.

[0225] The virtual reality interactive model creates rich, diverse and realistic training scenarios for patients, such as simulated outdoor walks and sports events, so that patients feel as if they are in a real environment during training, which greatly enhances the fun and immersion of training. Traditional rehabilitation training is often boring and patients are prone to boredom, but the virtual reality scenario of the present invention can effectively reduce this negative emotion and allow patients to participate in training more actively. At the same time, the system monitors the movements in real time and gives timely feedback. When patients are affirmed and assisted for correct movements, they will feel progress, thereby enhancing their confidence in rehabilitation, improving their enthusiasm and compliance in training, increasing the duration and frequency of training, and further promoting the rehabilitation process.

[0226] During the training process, the system monitors the user's movements in real time, compares them with the training plan to calculate the degree of matching, and comprehensively monitors the movement type, intensity, frequency, and training duration. Once a deviation is found, the smart wearable device will immediately issue a control command or intervention signal. For example, when the user's strength training movements are not standardized, it will promptly remind and correct them to avoid muscle and joint injuries due to incorrect movements. In addition, by comprehensively analyzing the user's movements and physiological characteristics, the system can accurately assess the training status and fatigue level. When the user is judged to be tired, the virtual reality scene will automatically pause the training or switch to the relaxation scene, effectively preventing overtraining and ensuring the safety of patients during rehabilitation training.

[0227] The present invention evaluates the rehabilitation effect from three dimensions: physiological indicators, motion data, and digital twin model optimization analysis results. Physiological indicators reflect the recovery of body functions, motion data reflect the degree of standardization of action execution, and twin optimization analysis results show the fit between model prediction and reality. This multi-dimensional evaluation avoids the limitations of a single indicator and makes the evaluation results more comprehensive and objective. Through algorithms such as dynamic time warping (DTW) and mean square error calculation, comprehensive scores are calculated for different data, and weights are set according to training goals to obtain comprehensive effect scores. The rehabilitation effects are divided into different levels, and quantitative evaluation is achieved, reducing the subjectivity of human evaluation. According to the evaluation results, the problem can be identified, the training plan can be adjusted in a targeted manner, the digital twin model can be optimized, a virtuous circle of rehabilitation training can be formed, and the quality of rehabilitation can be continuously improved.

[0228] In view of the fact that most limb diseases are caused by problems in the nervous system, the present invention collects EEG-EMG signals, uses artificial intelligence models to extract signal frequency, amplitude and other characteristics, and combines clinical information with standard databases to accurately determine the location and type of neural control circuit faults. According to the fault situation, the spatiotemporal electrical stimulation signal parameters are determined, and stimulation signals are sent to specific nerve roots through smart wearable devices to achieve accurate nervous system rehabilitation therapy. This process can promote the recovery of limb function from the root, improve the accuracy and effectiveness of rehabilitation treatment, and provide more powerful support for the rehabilitation of patients with damaged nervous systems.

[0229] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent rehabilitation training method based on artificial intelligence and virtual reality, characterized in that, Including: S1. The user wears a smart wearable device to collect multimodal data in real time and preprocess the multimodal data; the multimodal data includes the user's electroencephalogram signal, electromyogram signal, physiological characteristic signal, and motion signal; S2. Input the preprocessed multimodal data into an artificial intelligence training model, extract the key features in the multimodal data, and use a graph convolutional network and an attention mechanism algorithm to perform fusion processing on the multimodal data features; S3. Utilize the multimodal data features extracted after fusion to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation; S4. According to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical status, use a reinforcement learning algorithm to generate a customized training plan; the training plan includes training content, difficulty level, and training duration parameters; S5. The user enters the virtual reality interaction model, conducts rehabilitation training according to the generated training plan, monitors the user's actions and physiological status in real time, the digital twin model synchronizes the user's actions in real time, and the virtual reality scene dynamically adjusts according to the user's situation; S6. Evaluate the rehabilitation effect according to the user's physiological indicators, motion data after training, and the twin optimization analysis results driven by multimodal data, and feedback the rehabilitation evaluation results to the artificial intelligence processing model to further optimize the training plan and the digital twin model.

2. The intelligent rehabilitation training system and method based on artificial intelligence and virtual reality according to claim 1, characterized in that The user's electroencephalogram signal is obtained by using dry electrode technology; the electromyogram signal is collected by using a graphene-based flexible electrode; the physiological characteristic signals include heart rate, temperature, and pulse, which are obtained by using corresponding sensors; the motion signal is obtained by using a pressure sensor and an acceleration sensor on the wearable device.

3. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, characterized in that The extraction of the key features in the multimodal data and the use of a graph convolutional network and an attention mechanism algorithm to perform fusion processing on the multimodal data features are specifically as follows: S21. Represent the multimodal data as a graph structure. Let the graph be \(G=(V, E)\), where \(V\) is the set of nodes and \(E\) is the set of edges. The node feature matrix \(X\in\mathbb{R}\) N×D , where \(N\) is the number of nodes and \(D\) is the dimension of node features; the adjacency matrix \(A\in\mathbb{R}\) N×N , which is used to describe the connection relationships between nodes. \(A\) ij indicates whether there is a connection between node \(i\) and node \(j\). S22. Perform a convolution operation on the graph structure data through a graph convolutional network to extract the feature information of the nodes, and the formula is: H (l) ∈R N×F(l) : represents the node feature matrix of the l-th layer, F (l) is the feature dimension of the l-th layer. Initially, H (0) = X; where I is the identity matrix; is the degree matrix of, is the learnable weight matrix of the l-th layer; σ is the activation function; after being processed by the GCN with L layers, the final feature matrix H is obtained (L) ; S23. Take the feature H extracted by GCN (L) as the input and perform single-head attention calculation. The formula is as follows: For querying the projection matrix to project GCN features into the query space, d q is the dimension of the key; is the key projection matrix that projects GCN features into the key space, and d k is the dimension of the key; is the value projection matrix that projects the GCN features into the value space, and d v is the dimension of the key; The softmax function is used to normalize the attention scores; is the scaling factor to prevent the dot product result from being too large; S24. Parallel process the data through multiple independent single-head attention calculations to capture various features and relationships of the data from different representation subspaces: MultiHead(H (L) ) = C(head1, …, head h )W o Among them, head i = Attention(H (L) W i Q , H (L) W i K , H (L) W i V ), i = 1, …, h, MultiHead(H (L) ) is the final output result of the multi - head attention mechanism, head i is the result of the i - th single - head attention calculation, C(head1, …, head h ) represents the concatenation operation of the h single - head attention calculation results from head1 to head h ; h is the number of heads, is the query, key, and value projection matrix for the i - th head; is the linear transformation matrix for the concatenated result, mapping the output of the multi - head attention back to the same dimensional space as the GCN features; S25. Fuse the output of the multi-head attention mechanism with the features extracted by the GCN to obtain the final fused features, and use the weighted sum method for fusion: H f = αH (L) + (1 - α)H Attention Among them, H f is the finally obtained fused feature, which is the result of fusing the features extracted by the graph convolutional network GCN and the output features of the multi-head attention mechanism. H Attention = MultiHead(H (L) ) is the output of the multi-head attention mechanism, and α is the fusion weight parameter, where α ∈ [0, 1].

4. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, wherein The step S3 utilizes the multimodal data features extracted after fusion to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation, specifically as follows: S31. Assume that the multimodal data includes λ modalities, and the set of multimodal data feature vectors after preprocessing is: Use the attention mechanism to dynamically allocate the weights of each modality and fuse them into a unified feature: Among them, α i is the weight of the i-th modality, z is the fused unified feature, and W a is the weight matrix of the attention network, and b a is the bias vector of the attention network; S32. The digital twin model needs to map the physiological-motor state of the patient in real time, and is described by a state space model: s t = f(s t-1 , z t ) + ε1 where s t is the state vector at time t, f is the state transition function, specifically the LSTM neural network function, Z t is the fused feature vector, and ε1 is the process noise; S33. Based on the current state s t , predict the state sequence for the next T steps using the Seq2Seq model: Encoder input historical state sequence s 1:t : The decoder generates future predictions where s 1:t is the historical state sequence, i.e., the set of state vectors from time 1 to time t; is the hidden state of the encoder at time t; is the predicted state sequence for the next T steps, is the predicted state vector at future time k (k = t + 1, …, t + T); is the hidden state of the decoder at time k, is the predicted state vector at time τ - 1, W p is the decoder weight matrix, b p is the decoder bias vector.

5. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, characterized in that The generation of a customized training plan by using a reinforcement learning algorithm according to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical status, is specifically as follows: S41. Define the state space S as the fused feature vector and the user's rehabilitation goal, the action space A as the adjustment options for various training parameters, and the reward function R to include the action completion and the changes in physiological goals; S42. Construct a policy network using the Actor-Critic architecture, with the input state s t Output the action probability distribution or mean variance: u t , σ t = MLP θ (s t ), and sample the continuous action as a t ~ N(u t , σ t ); θ is the parameter of the Actor network, which determines the structure and behavior of the network; MLP θ is a multi-layer perceptron with θ as the parameter, used to process the input state and generate action-related outputs; u t is the mean of the continuous action distribution, representing the central tendency of the action; σ t is the standard deviation of the continuous action distribution, measuring the uncertainty of the action; The Critic network is used to evaluate the state value and guide the Actor to update. The formula is are the parameters of the Critic network, which are continuously adjusted through learning to accurately evaluate the state value. is is a multi-layer perceptron with as parameters, which is used to calculate the value of the state; Reinforcement learning training, maximizing the constrained policy improvement, with the formula Among them, is the importance sampling ratio, π θ is the current policy, π old is the old policy, which is used to measure the difference between the new and old policies; For advantage function estimation, r t is the future reward, γ is the discount factor, ranging between [0, 1], used to measure the importance of future rewards; ε is the clipping parameter, used to limit the magnitude of policy updates; E t denotes the expectation over time step t, used to average the objective function over multiple time steps; S43. Collect trajectory data η = {(s t , a t , r t )}, obtained by interacting the current policy with the environment. These data contain state, action, and reward information at different time steps and are used for subsequent learning and updating; Calculating Advantage Estimation and Returns where T is the final time step in the trajectory data, and the return R t is the cumulative discounted reward from time step t to the final time step, where t represents the current time step; r τ is the immediate reward obtained at time step τ; Update the Critic network to minimize the value error, with the objective function being E t denoting the expected operation with respect to time step t. By adjusting the parameters of the Critic network make the estimated state value equivalent to the actual return R t ; Update the Actor network to maximize the objective function ζ CLIP (θ), by optimizing the parameters θ of the Actor network, so that the current policy obtains the highest reward; S44. When the user is training, the policy network generates an action a according to the current state s t and dynamically adjusts the training plan. At the same time, hard constraints are added to ensure that the action is within the safe range, that is, a t ← Clip(a t , a t , a min , a max ), where a min is the minimum value of the action and a max is the maximum value of the action.

6. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, characterized in that The user enters the virtual reality interaction model and conducts rehabilitation training according to the generated training plan. Specifically: S51. Generate a corresponding virtual reality training scenario according to the training plan. The training scenario consists of multiple elements, and each element is represented by a three-dimensional vector P i = [x i , y i , z i indicating its position in the virtual space. The attribute vector of the element is represented by a i , and the position vector is represented by p i . The scenario generation function S is expressed as: S = {(p i , a i ) | i = 1, 2, …, n} Use the rendering algorithm to render the elements in the scene onto the screen. For each pixel point (u, v), its color value C(u, v) is calculated by the following formula: C(u, v) = R(S, r(u, v)) where r(u, v) is the light vector starting from the camera position and passing through the pixel point (u, v), and R is the ray tracing algorithm model function; S52. During the training process, continuously monitor the user's rehabilitation training actions, compare them with the requirements in the training plan, and calculate the matching degree, where: Action type matching. Let the set of action types specified in the training scheme be \(A = [A_1, A_2, \ldots, A m \), where \(m\) is the number of action types, and \(A i \) is a binary variable indicating whether the \(i\)-th action type is included. The set of action types actually performed by the user is obtained through the action recognition model as The Hamming distance is used to measure the matching degree, and the formula is: Action type matching degree M A is as follows: The intensity range of the j-th action specified in the training plan The intensity of the user's actual execution of this action measured by the sensor is I j , the action intensity matching index M I is as follows: The action target frequency specified in the training plan is f t , within the time interval [t1, t2], the number of times the user performs the action is N t , then the actual action frequency Action frequency matching error E f is: Action frequency matching degree M f : M f = 1 - E f The total training duration specified in the training plan is T t , and the currently trained duration is T e , and the training duration matching index M T is as follows: S53. According to the matching results of the action type, action intensity, action frequency, and training duration, when it is found that the user's training actions deviate from the training plan, issue a control instruction or an intervention signal to the smart wearable device to provide auxiliary force or feedback information to compensate for the deviation of the training control data.

7. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, wherein The digital twin model synchronizes the user's actions in real time, and the virtual reality scene is dynamically adjusted according to the user's situation. Specifically: S54. Continuously monitor the user's action data and physiological data and transmit them to the digital twin model, perform fusion processing on the multi-modal data, and update the digital twin model using the fused feature vector. The digital twin model is described by a state space model, with the formula: S t = f(S t-1 , Z t ) + ε1 Among them, S t is the state vector at time t, f is the state transition function, Z t is the fused feature vector, and ε1 is the process noise; by continuously inputting new fused data, the digital twin model updates its own state in real time to achieve synchronization with the user's actions and physiological states; S55. Integrate the user's motion characteristics and physiological characteristics, evaluate the user's training status and fatigue level, set the action completion index C, and use the dynamic time warping algorithm DTW to calculate the actual motion trajectory T a with the standard motion trajectory T s , and the formula is where len represents the length of the trajectory, and the user's fatigue level is evaluated based on heart rate variability, electromyogram signal characteristics, and electroencephalogram signal characteristics; S56. According to the evaluation results of the user's situation, formulate a virtual reality scene adjustment strategy: if the action completion degree is low and the user is not fatigued, reduce the training difficulty; if the user is fatigued, pause the training or switch to an easy relaxation scene; S57. According to the scene adjustment decision, adjust the parameters of the objects in the virtual scene or switch the scene through 3D modeling technology.

8. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 1, characterized in that, Evaluating the rehabilitation effect according to the user's physiological indicators, motion data after training, and the results of twin optimization analysis driven by multi-modal data. Specifically: S61. Evaluate each physiological state of the user according to the normal range of physiological indicators and the rehabilitation goal to obtain the comprehensive physiological index score S p ; Use the dynamic time warping algorithm DTW to calculate the similarity between the actual movement trajectory of the user and the standard movement trajectory, and calculate the comprehensive score S of the movement data M ; Calculate the mean square error between the predicted value and the actual value of the digital twin model, and set the score S for the twin optimization analysis result according to the evaluation result T ; S62. According to the training goal focus, set corresponding weights ω for the comprehensive score of physiological indicators, the comprehensive score of sports data, and the score of twin optimization analysis results. p ,ω M ,ω T , calculate the comprehensive effect score S: S = ω P S p + ω M S M + ω T S T ; S63. Set the corresponding score threshold according to the comprehensive score S, and divide the rehabilitation effect into grades of excellent, good, medium, and poor.

9. The intelligent rehabilitation training method based on artificial intelligence and virtual reality according to claim 6, characterized in that, Issuing a control instruction or an intervention signal to the smart wearable device. The steps for generating the intervention signal are: S531. Use the dry electrodes on the smart wearable device to collect electroencephalogram (EEG) signals, and use the flexible electrodes based on graphene to collect electromyogram (EMG) signals. Perform preprocessing on the collected original EEG and EMG signals, including but not limited to filtering and amplification; S532. Apply algorithms such as graph convolutional networks and attention mechanisms in the artificial intelligence training model to extract features from the preprocessed EEG and EMG signals respectively. The extracted features include but are not limited to the frequency, amplitude, phase, and time-frequency features of the signals. Calculate the energy distribution and correlation coefficient indicators of the EEG and EMG signals in different frequency bands and draw data graphs; S533. Combine with the pre-established standard database of electroencephalogram-electromyogram signal characteristics under normal physiological conditions, compare the currently extracted electroencephalogram-electromyogram signal characteristics of the patient with the standard database, and combine the patient's clinical symptoms, medical history, and physiological examination results to determine the specific nerve root position and fault type where the nerve control circuit fails; S534. Determine the parameters of the spatio-temporal electrical stimulation signal according to the type, position, and severity of the nerve control circuit failure, and use the microcurrent stimulation technology to send the spatio-temporal electrical stimulation signal to the specific nerve root through the electrode array on the intelligent wearable device according to the set parameters; the parameters of the spatio-temporal electrical stimulation signal include but are not limited to stimulation frequency, intensity, duration, pulse waveform, and spatial distribution of the stimulation.

10. An intelligent rehabilitation training system based on artificial intelligence and virtual reality, characterized in that, Including: An intelligent wearable device module for real-time collection of multimodal data and preprocessing of the multimodal data; the multimodal data includes user electroencephalogram signals, electromyogram signals, physiological characteristic signals, and motion signals; A signal feature extraction and fusion module that inputs the preprocessed multimodal data into an artificial intelligence training model, extracts key features from the multimodal data, and uses a graph convolutional network and an attention mechanism algorithm to perform fusion processing on the multimodal data features; A digital twin model construction module that uses the multimodal data features extracted after fusion to construct a digital twin model of the patient for real-time dynamic mapping and predictive simulation; A training plan generation module that generates a customized training plan using a reinforcement learning algorithm according to the fused data features and the analysis results of the digital twin model, combined with the user's rehabilitation goals and physical status; The training plan includes training content, difficulty level, and training duration parameters; A rehabilitation training module that performs rehabilitation training according to the generated training plan, real-time monitors the user's movements and physiological status, the digital twin model synchronizes the user's movements in real time, and the virtual reality scene dynamically adjusts according to the user's situation; A rehabilitation result evaluation module that evaluates the rehabilitation effect according to the physiological indicators, motion data, and twin optimization analysis results driven by multimodal data after the user's training, and feeds back the rehabilitation evaluation results to the artificial intelligence processing model to further optimize the training plan and the digital twin model.

Images