Virtual fusion personalized upper limb rehabilitation training cloud platform and method

Abstract

The invention relates to the technical field of medical rehabilitation, and discloses a virtual fusion personalized upper limb rehabilitation training cloud platform and method. The platform is composed of a multi-source biological signal acquisition module, a motion function feature modeling module, a heterogeneous data fusion module, a rehabilitation knowledge graph construction module, a dynamic training planning module and the like. The method comprises the steps of collecting multi-source biological signals, carrying out hierarchical feature modeling, carrying out data fusion, constructing an atlas database, generating a personalized rehabilitation scheme and the like. In this way, individuation and precision of upper limb rehabilitation training are achieved. Patient data can be comprehensively collected, fusion rehabilitation state representation is generated, a double-layer atlas database is constructed to assist training planning, training safety is guaranteed, the rehabilitation effect is predicted, pertinence, scientificity and efficiency of rehabilitation training are effectively improved, and high-quality rehabilitation service is provided for upper limb dysfunction patients.

Description

Technical Field

[0001] The present invention relates to the field of medical rehabilitation technology, and in particular to a virtual fusion personalized upper limb rehabilitation training cloud platform and method. Background Art

[0002] In the field of medical rehabilitation, upper limb rehabilitation training is crucial for patients with impaired upper limb function due to various reasons. It is directly related to the restoration of patients' ability to care for themselves and their quality of life. However, traditional upper limb rehabilitation training methods currently face many difficulties that cannot be ignored.

[0003] From the perspective of personalized training, traditional rehabilitation training programs are mostly developed based on universal standards and the experience of medical staff, failing to fully account for the significant differences between individual patients. The cause of each patient's upper limb injury, the degree of injury, basic physical conditions, and psychological state are different. This "one-size-fits-all" training model cannot accurately match the patient's actual needs. For example, for patients with upper limb muscle weakness caused by neurological diseases, traditional training programs may not be able to strengthen muscle strength in a targeted manner and are prone to secondary injury due to improper training intensity. For patients recovering from fractures, if the training movements and intensity are not adjusted according to the fracture healing stage, it may affect the normal healing of the bones. This not only reduces the effectiveness of rehabilitation training, but also may prolong the patient's recovery period, increase the patient's pain and financial burden.

[0004] When it comes to rehabilitation assessment, existing methods are relatively simple and subjective. They typically rely on direct observation by medical staff and simple scale tests, making it difficult to comprehensively, accurately, and in real time monitor a patient's upper limb motor function. For example, traditional methods are unable to accurately capture subtle changes in muscle activity during training, precisely measure the trajectory and angle of joint movement, or comprehensively analyze the distribution of tactile pressure. This makes it difficult for medical staff to accurately monitor a patient's rehabilitation progress in a timely and accurate manner, and they are unable to adjust training plans based on actual conditions, thus affecting rehabilitation effectiveness.

[0005] From the perspective of the integration and application of rehabilitation knowledge, the knowledge system of rehabilitation medicine is vast and complex, covering multiple aspects such as basic medical theory, rehabilitation treatment techniques, and clinical experience. However, this knowledge is currently scattered across various medical literature, medical records from different medical institutions, and the personal experience of medical staff, lacking an efficient integration and sharing mechanism. When formulating rehabilitation plans, medical staff often struggle to quickly and accurately obtain the required knowledge, resulting in rehabilitation training programs that are not scientifically sound. For example, when selecting rehabilitation exercises, they may not be able to base their choices on the latest clinical research results, or when adjusting training intensity, they lack effective data support and scientific basis.

[0006] With the continuous advancement of science and technology, the application of virtual reality technology, biosignal acquisition technology, and artificial intelligence technology in the medical field has gradually increased. However, in upper limb rehabilitation training, the integration and application of existing technologies still have many shortcomings. For example, although some rehabilitation devices can collect multiple biosignals, these signals lack effective fusion processing, making it impossible to form a unified and comprehensive rehabilitation assessment index. When using virtual reality technology for rehabilitation training, it cannot dynamically adjust according to the patient's real-time status and personalized needs, resulting in less targeted training. Therefore, the development of a virtual fusion personalized upper limb rehabilitation training cloud platform and method that can address these issues is of great practical significance and urgent need. Summary of the Invention

[0007] The purpose of the present invention is to provide a virtual fusion personalized upper limb rehabilitation training cloud platform and method to solve the problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solutions: a virtual fusion personalized upper limb rehabilitation training cloud platform, the platform comprising:

[0009] Multi-source bio-signal acquisition module: used to collect multi-source bio-signals for upper limb rehabilitation training in real time, including surface electromyography signals, joint motion trajectory data, tactile pressure distribution matrix, virtual reality interaction logs, and physiological indicator time series data;

[0010] Movement function feature modeling module: performs hierarchical feature modeling on the multi-source biological signals to generate functional feature vectors corresponding to each modality. The functional feature vectors include: electromyographic activation pattern features, joint kinematic chain topology features, pressure distribution symmetry features, virtual environment interaction intention features, and cardiopulmonary metabolic load index;

[0011] Heterogeneous data fusion module: A domain adaptation algorithm based on transfer learning maps the functional feature vectors to a unified rehabilitation assessment space to generate a fused rehabilitation status representation;

[0012] Rehabilitation knowledge graph construction module: Constructs a dual-layer graph library consisting of a rehabilitation medicine ontology graph and a personalized training model graph. The dual-layer graph library stores the patient's musculoskeletal function injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategies;

[0013] Dynamic training planning module: Based on the fused rehabilitation state representation and the dual-layer atlas library, a decision engine driven by reinforcement learning is used to generate personalized rehabilitation action sequences and intensity adaptive adjustment plans.

[0014] Preferably, performing hierarchical feature modeling on the multi-source biological signals includes:

[0015] The surface electromyographic signal is decomposed by wavelet packet combined with convolutional recursive network to extract the time-frequency domain muscle co-activation pattern;

[0016] Using Lie group space modeling algorithm to analyze the rigid body transformation parameters of the three-dimensional kinematic chain for the joint motion trajectory data;

[0017] The tactile pressure distribution matrix is ​​subjected to a graph convolutional network combined with an attention mechanism to identify the pressure center migration path characteristics.

[0018] Preferably, the motor function feature modeling module further includes:

[0019] A spatiotemporal graph convolutional network is used to extract action intention sequences from the virtual reality interaction logs, and a causal reasoning algorithm is used to annotate the logical dependencies of behavioral decisions.

[0020] A dynamic mode decomposition algorithm is used to separate the steady-state and transient components of the cardiopulmonary metabolic load from the physiological indicator time series data.

[0021] Preferably, the construction of a dual-layer atlas library of the rehabilitation medicine ontology atlas and the personalized training model atlas includes:

[0022] Based on the International Classification of Functional Disabilities standard entities and the evidence-based relationship between rehabilitation actions and therapeutic effects, a knowledge graph embedding algorithm is used to construct an extensible rehabilitation medicine ontology graph;

[0023] Based on the patient's historical training data and functional evaluation indicators, a federated learning framework is used to generate node embedding representations of the distributed personalized training model graph.

[0024] Preferably, the reinforcement learning-driven decision engine adopts a curriculum learning framework, including:

[0025] The state space is defined as the joint encoding of the current rehabilitation state representation and the medical ontology subgraph, and the action space is defined as the candidate set of rehabilitation training actions;

[0026] The training sequence policy is generated by a double-delayed deep deterministic policy gradient network, and the reward function is designed based on biomechanical safety constraints.

[0027] Preferably, the decision engine further includes:

[0028] An evolutionary strategy algorithm is used to explore the diversity of training action sequences, and a Bayesian optimization algorithm is used to calculate the risk boundary of the intensity adjustment scheme.

[0029] Action combinations that violate medical guidelines are filtered through logical consistency constraints of rehabilitation stage transitions.

[0030] Preferably, the platform further comprises:

[0031] Performing abnormal pattern detection on the virtual reality interaction log, reconstructing a normal interaction pattern using a variational autoencoder, and calculating a reconstruction error threshold;

[0032] When an abnormal interaction is detected, the trajectory replanning mechanism of the decision engine is triggered to generate an alternative training plan that is adapted to the current rehabilitation progress.

[0033] Preferably, the trajectory replanning mechanism uses a memory-enhanced neural network to model historical training sequences, including:

[0034] Extracting feature templates of successful training patterns through memory readout gating mechanism;

[0035] Based on the similarity matching between the feature template and the current state, a training action topology structure after local fine-tuning is generated.

[0036] Preferably, the platform further comprises:

[0037] Construct a rehabilitation effect prediction model based on the patient's training frequency and musculoskeletal function recovery curve

[0038] Line, using neural differential equations to model the dynamic evolution of the rehabilitation process;

[0039] The rehabilitation effect prediction model also includes:

[0040] Based on the dynamic graph neural network, the rehabilitation stage division criteria and individual recovery rate differences are aligned to generate an adaptive training cycle adjustment strategy and embed it into the training sequence strategy.

[0041] Preferably, the present invention further includes a virtual fusion personalized upper limb rehabilitation training method, the method comprising the following steps:

[0042] Step 1: Real-time collection of multi-source biological signals of the patient's upper limb rehabilitation training, wherein the multi-source biological signals include surface electromyography signals, joint motion trajectory data, tactile pressure distribution matrix, virtual reality interaction log and physiological indicator time series data;

[0043] Step 2: Perform hierarchical feature modeling on the multi-source biological signals to generate functional feature vectors corresponding to each modality, wherein the functional feature vectors include myoelectric activation pattern features, joint kinematic chain topology features, pressure distribution symmetry features, virtual environment interaction intention features, and cardiopulmonary metabolic load index;

[0044] Step 3: Mapping the functional feature vector to a unified rehabilitation assessment space using a domain adaptation algorithm based on transfer learning to generate a fused rehabilitation status representation;

[0045] Step 4: Pre-build a double-layer atlas library of rehabilitation medicine ontology and personalized training mode atlas, wherein the double-layer atlas library stores the patient's musculoskeletal function injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategy;

[0046] Step 5: Based on the fused rehabilitation state representation and the two-layer atlas library, a decision engine driven by reinforcement learning is used to generate a personalized rehabilitation action sequence and an intensity adaptive adjustment plan.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] In terms of personalized training, the multi-source biosignal acquisition module comprehensively collects a variety of biosignals during upper limb rehabilitation training, such as surface electromyography (EMG) signals and joint motion trajectory data. This rich data provides a precise basis for the motor function feature modeling module. The functional feature vector generated through hierarchical feature modeling accurately reflects the patient's individual motor function status. Based on this foundation, the dynamic training planning module, by integrating rehabilitation state representation with a two-layer atlas library and leveraging a reinforcement learning-driven decision engine, generates highly personalized rehabilitation exercise sequences and adaptive intensity adjustment plans. This means that training plans can be dynamically optimized based on the patient's real-time status and individual differences, avoiding the drawbacks of traditional, uniform training plans and significantly improving the targetedness and effectiveness of rehabilitation training. For example, for patients with weak muscle strength, the system can automatically adjust the difficulty and intensity of training exercises to gradually increase muscle strength while avoiding injuries caused by overtraining. For patients with poor joint mobility, targeted joint range of motion exercises can be precisely designed to gradually improve joint range of motion.

[0049] During the rehabilitation assessment phase, the heterogeneous data fusion module uses a domain adaptation algorithm based on transfer learning to map multiple functional feature vectors into a unified rehabilitation assessment space, generating a fused rehabilitation status representation. This fused assessment approach overcomes the one-sidedness of traditional assessment methods and comprehensively considers information from multiple dimensions, making rehabilitation assessments more comprehensive, accurate, and scientific. Based on these precise assessment results, medical staff can more clearly understand the patient's rehabilitation progress, promptly identify potential problems, and make more reasonable treatment decisions. For example, by analyzing the fused rehabilitation status representation, medical staff can accurately determine the patient's strengths and weaknesses at a specific stage of rehabilitation, thereby specifically adjusting training priorities and optimizing rehabilitation plans.

[0050] The two-layer graph library constructed by the rehabilitation knowledge graph construction module is of great significance to the management and application of rehabilitation knowledge. The rehabilitation medicine ontology graph is constructed based on the entities of the International Classification of Functional Disabilities and the evidence-based relationship between rehabilitation movement and efficacy, providing a standardized and normalized knowledge framework for rehabilitation training. The personalized training model graph is generated based on the patient's historical training data and functional assessment indicators, recording each patient's unique rehabilitation experience and optimization strategy. This not only facilitates the storage, query, and sharing of rehabilitation knowledge, promotes the inheritance and accumulation of rehabilitation medicine knowledge, but also provides valuable reference and reference for the rehabilitation training of new patients. Medical staff can quickly obtain relevant knowledge, develop more scientific and reasonable rehabilitation plans, and improve the efficiency and quality of rehabilitation training.

[0051] In terms of intelligence and safety assurance during the training process, the platform detects abnormal patterns in virtual reality interaction logs. Once an abnormal interaction is detected, it immediately triggers the trajectory replanning mechanism of the decision engine to generate an alternative training plan adapted to the current rehabilitation progress, ensuring the smooth progress of the training process and effectively avoiding the risk of patient injury due to abnormal situations. At the same time, when generating training sequence strategies, the reinforcement learning-driven decision engine fully considers biomechanical safety constraints to design the reward function, and uses evolutionary strategy algorithms, Bayesian optimization algorithms, etc. for optimization. It also filters out action combinations that violate medical guidelines through logical consistency constraints of rehabilitation stage transitions, ensuring the safety and rationality of the training process from multiple angles. For example, during training, if a patient's action exceeds the safe range, the system will promptly issue a reminder and adjust the training action to prevent accidental injuries caused by incorrect actions.

[0052] Furthermore, the development of a rehabilitation effect prediction model further enhances the scientific nature of rehabilitation training. Based on the patient's training frequency and musculoskeletal function recovery curve, this model employs neural differential equations to model the dynamic evolution of the rehabilitation process. Furthermore, using a dynamic graph neural network, it aligns the rehabilitation stage division criteria with individual recovery rate differences to generate an adaptive training cycle adjustment strategy. This enables medical staff to predict rehabilitation outcomes in advance, rationally arrange training plans, optimize resource allocation, and provide patients with more proactive rehabilitation services. For example, based on the prediction results, medical staff can adjust training intensity and duration in advance, or recommend more appropriate rehabilitation assistive devices for patients, thereby improving the effectiveness and efficiency of rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a diagram showing the working principle of the upper limb rehabilitation training cloud platform of the present invention;

[0054] Figure 2 A diagram showing how the curriculum learning framework works for the decision engine;

[0055] Figure 3A diagram showing the working principle of training action sequence exploration and intensity adjustment in the decision engine;

[0056] Figure 4 A diagram showing the working principle of anomaly detection and trajectory replanning in virtual reality interaction logs. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] See also Figure 1-Figure 4 The present invention provides a technical solution: a virtual fusion personalized upper limb rehabilitation training cloud platform and method to achieve accurate assessment and personalized planning of patients' upper limb rehabilitation training. The specific implementation scheme is as follows:

[0059] Multiple sensors are used to collect multi-source biological signals from patients during upper limb rehabilitation training in real time, including surface electromyography (EMG) signals, joint motion trajectory data, tactile pressure distribution matrices, virtual reality interaction logs, and time-series data of physiological indicators. For example, surface electromyography (EMG) sensors are attached to relevant muscle parts of the patient's upper limbs to obtain electrical signals generated during muscle activity; motion capture systems, such as optical or inertial sensors, are used to record joint motion trajectories; pressure sensor arrays are installed at the contact points of the training equipment to collect tactile pressure distribution information; virtual reality interaction logs are generated using the interactive recording function of virtual reality devices; and physiological monitoring equipment, such as heart rate sensors and respiratory monitors, are used to obtain time-series data of physiological indicators.

[0060] Hierarchical feature modeling is performed on the collected multi-source biosignals to generate functional feature vectors corresponding to each modality. Surface electromyography signals are analyzed for muscle coactivation patterns; joint motion trajectory data are analyzed for topological features of the three-dimensional kinematic chain; pressure distribution symmetry is identified from the tactile pressure distribution matrix; interaction intent features are extracted from virtual reality interaction logs; and the cardiopulmonary metabolic load index is calculated based on time-series data of physiological indicators. These functional feature vectors comprehensively reflect the motor function status of the patient's upper limb.

[0061] A domain adaptation algorithm based on transfer learning maps the generated functional feature vectors into a unified rehabilitation assessment space, generating a fused rehabilitation status representation. This process integrates information from different modalities, overcoming data heterogeneity and providing a unified and effective data foundation for subsequent rehabilitation assessment and training planning.

[0062] A dual-layered atlas database consisting of a rehabilitation medicine ontology and a personalized training model atlas was constructed. The rehabilitation medicine ontology was constructed based on the International Classification of Functional Disabilities (ICDM) entities and the evidence-based relationship between rehabilitation movement and efficacy, storing general rehabilitation medicine knowledge. The personalized training model atlas was generated based on the patient's historical training data and functional assessment indicators, containing patient-specific training model information, such as musculoskeletal injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategies.

[0063] By integrating rehabilitation status representation with a dual-layer graph library, a reinforcement learning-driven decision engine generates personalized rehabilitation movement sequences and adaptive intensity adjustment plans. The decision engine comprehensively considers the patient's current rehabilitation status and information in the knowledge graph to intelligently plan training movements and adjust training intensity to achieve optimal rehabilitation results.

[0064] The present invention will be further described below in conjunction with Examples 1 to 6:

[0065] Example 1:

[0066] In this embodiment, the specific processing methods of various biological signals in the motor function feature modeling module are described in detail.

[0067] For surface electromyographic signals, wavelet packet decomposition combined with convolutional recursive network is used to extract the muscle co-activation pattern in the time-frequency domain. Wavelet packet decomposition is a multi-resolution analysis method that can decompose the signal at different frequency scales, thereby analyzing the time-frequency characteristics of the signal in more detail. Assume that the collected surface electromyographic signal is , it can be decomposed into multiple sub-band signals through wavelet packet decomposition ,in represents the number of decomposition levels, Represents the subband index at this layer. These subband signals contain information about muscle activity in different frequency ranges. Next, the decomposed subband signals are input into a convolutional recurrent network. The convolutional layer automatically extracts local features from the signal, while the recurrent network captures the signal's time series characteristics. Combining the two effectively extracts muscle coactivation patterns in the time-frequency domain. For example, parameters such as the size and number of convolutional kernels in the convolutional layer, as well as the number of layers and hidden units in the recurrent network, can be adjusted and optimized based on actual conditions to achieve better feature extraction.

[0068] For the joint motion trajectory data, the Lie group space modeling algorithm is used to analyze the rigid body transformation parameters of the three-dimensional kinematic chain. In three-dimensional space, joint motion can be regarded as the motion of a rigid body, and Lie group space provides an effective mathematical framework for describing this rigid body motion. Assume that the joint motion trajectory is composed of a series of space points It means that through the Lie group space modeling algorithm, the parameters describing the rigid body transformation, such as the rotation matrix, can be calculated. and translation vectors These parameters can accurately describe the motion state of the joint in three-dimensional space, including information such as the rotation angle and translation distance of the joint, thereby generating the topological characteristics of the joint kinematic chain.

[0069] For the tactile pressure distribution matrix, a graph convolutional network (GCN) combined with an attention mechanism is used to identify the characteristics of the pressure center migration path. GCNs can perform convolution operations on graph-structured data, effectively extracting graph features. The tactile pressure distribution matrix can be viewed as a graph, where nodes represent the locations of pressure sensors and edges represent the relationships between them. GCNs can learn both local and global characteristics of the pressure distribution. The attention mechanism allows the model to focus more on the migration path of the pressure center. During the calculation process, the attention mechanism assigns weights based on the importance of different nodes and edges, enabling the model to more accurately identify the characteristics of the pressure center migration path and thus obtain the symmetry characteristics of the pressure distribution.

[0070] Furthermore, for VR interaction logs, a spatiotemporal graph convolutional network is used to extract action intention sequences, and a causal inference algorithm is used to annotate the logical dependencies of behavioral decisions. The spatiotemporal graph convolutional network can simultaneously process information in both temporal and spatial dimensions, extracting the spatiotemporal features of actions from VR interaction logs to form action intention sequences. The causal inference algorithm is used to analyze the causal relationships between actions and annotate the logical dependencies of behavioral decisions, such as which previous actions determine whether a particular action is based on its results.

[0071] For physiological indicator time series data, the dynamic mode decomposition algorithm is used to separate the steady-state and transient components of cardiopulmonary metabolic load. This algorithm decomposes complex time series data into distinct modes, with the steady-state component reflecting the steady state of cardiopulmonary metabolic load over time and the transient component reflecting short-term changes. This decomposition provides a clearer understanding of the changing patterns of cardiopulmonary metabolic load during rehabilitation training and allows the calculation of a cardiopulmonary metabolic load index.

[0072] Example 2:

[0073] When constructing the rehabilitation medicine ontology graph, a knowledge graph embedding algorithm is used based on the entities of the International Classification of Functional Disabilities and the evidence-based relationship between rehabilitation actions and efficacy. The International Classification of Functional Disabilities provides a unified concept and classification system for rehabilitation medicine, which includes various entities related to functional disabilities, such as physical function, activities and participation, environmental factors, etc. The evidence-based relationship between rehabilitation actions and efficacy is derived through a large number of clinical studies and practical summaries, which clarifies the effects of different rehabilitation actions on the recovery of various functions. The purpose of the knowledge graph embedding algorithm is to map these entities and relationships into a low-dimensional vector space for computer processing and analysis. For example, the TransE algorithm is used to represent each entity in the rehabilitation medicine ontology as a vector , the relationship is represented as a vector For a triple ,in is the head entity, It's a relationship. is the tail entity, and the model training makes In this way, the semantic information of entities and relationships is preserved in the low-dimensional vector space, and knowledge reasoning and query can be easily performed. The rehabilitation medicine ontology graph constructed in this way has good scalability and can be continuously updated and improved as new rehabilitation knowledge emerges.

[0074] When generating a personalized training pattern map, a federated learning framework is employed based on the patient's historical training data and functional evaluation metrics. This framework allows multiple participants to jointly train models without sharing the original data. In this scenario, the historical training data for different patients is stored on their respective local devices. First, each local device calculates a node embedding representation of the personalized training pattern map based on the local patient's historical training data and functional evaluation metrics. For example, for each patient, data such as their historical training action sequence, training intensity, training time, and corresponding functional evaluation results are used as input, and a node embedding vector is calculated using a local neural network model. Then, using a federated learning algorithm, such as the federated averaging algorithm, each local device uploads the calculated node embedding vector to a central server, which aggregates and averages these vectors to obtain a global node embedding representation of the personalized training pattern map. These global node embedding representations are then distributed to each local device, which then updates its local model based on the global information. In this way, the patient's privacy data is protected, and the data of all patients can be fully utilized to generate a more accurate and personalized training pattern map, which stores information such as each patient's specific musculoskeletal function injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategy.

[0075] Example 3:

[0076] This embodiment describes in detail the specific implementation of the reinforcement learning-driven decision engine using the curriculum learning framework.

[0077] First, the state space is defined as the joint encoding of the current rehabilitation state representation and the medical ontology subgraph. The current rehabilitation state representation is a fused rehabilitation state representation generated by the heterogeneous data fusion module. It integrates the feature information of multi-source biological signals and comprehensively reflects the patient's current rehabilitation status. The medical ontology subgraph is a portion of the rehabilitation medicine ontology graph related to the current rehabilitation training, which contains relevant rehabilitation knowledge and concepts. Jointly encoding these two can provide richer information for the decision engine. For example, a neural network can be used to concatenate the feature vectors of the current rehabilitation state representation and the medical ontology subgraph. This can then be encoded using a multi-layer perceptron to obtain a high-dimensional state vector, which serves as the input to the decision engine.

[0078] The action space is defined as a candidate set of rehabilitation training actions. This set includes a variety of possible rehabilitation training actions, such as arm flexion and extension, rotation, and grasping, as well as different combinations of training intensities and durations. The decision engine needs to select the appropriate action sequence and intensity adjustment plan from this candidate set.

[0079] The training sequence policy is generated by a double-delayed deep deterministic policy gradient network (TD3). TD3 is an algorithm based on deep reinforcement learning that learns the optimal training policy through two Q networks and a policy network. Let the policy network be ,in It's a state. are the parameters of the policy network; the two Q networks are and ,in It's action. and is the parameter of the Q network. During the training process, first according to the current state Sample an action from the policy network , then execute this action to get the next state and rewards By updating the parameters of the Q network, and Estimate the value of state-action pairs as accurately as possible. Simultaneously, by updating the policy network's parameters, the actions generated by the policy network maximize the value of the Q network. To avoid overfitting and improve algorithm stability, TD3 employs a double-delayed update mechanism, updating the policy network at a fixed interval and smoothing the target Q value.

[0080] The reward function is designed based on biomechanical safety constraints. The design of the reward function directly impacts the learning effect of the decision engine. To ensure the safety of the training process, biomechanical safety constraints are considered. For example, based on the biomechanical characteristics of the human upper limb, a safe range of joint movement angles and a reasonable threshold for muscle strength are set. If the training movements generated by the decision engine cause joint movement to exceed the safe range or excessive muscle strength, a negative reward is given. If the training movements effectively promote recovery and comply with biomechanical safety constraints, a positive reward is given. This way, the decision engine will tend to choose training movement sequences and intensity adjustment plans that are both safe and effective during the learning process.

[0081] Example 4:

[0082] This embodiment further illustrates the evolutionary strategy algorithm and Bayesian optimization algorithm used in the decision engine, as well as the specific implementation of the logical consistency constraints of the rehabilitation stage transition.

[0083] An evolutionary strategy algorithm is used to explore the diversity of training action sequences. This algorithm simulates the process of biological evolution, searching for optimal training scenarios by performing mutation and selection operations on training action sequences. In the initial stage, a set of training action sequences is randomly generated as the initial population. Each training action sequence is treated as an individual, and its fitness is calculated based on the training sequence strategy generated by the decision engine. Fitness can be measured based on the value of a reward function; higher rewards indicate higher fitness. Then, mutation operations are performed on the individuals, such as randomly changing certain actions in the action sequence or adjusting the intensity of the actions. These mutated individuals form a new population, and fitness is calculated again. Through multiple iterations, individuals with higher fitness are selected and those with lower fitness are eliminated, allowing the population to gradually evolve towards more optimal training action sequences. This prevents the decision engine from becoming trapped in local optimal solutions, allowing it to explore more possible training scenarios and improving the diversity and effectiveness of training.

[0084] The Bayesian optimization algorithm is used to calculate the risk bounds of the intensity adjustment plan. The Bayesian optimization algorithm is an optimization method based on a probabilistic model. It gradually finds the optimal solution by continuously updating the estimate of the objective function. When calculating the risk bounds of the intensity adjustment plan, the intensity adjustment plan is used as the optimization variable, and the risk indicators during training (such as joint injury risk and muscle fatigue risk) are used as the objective function. First, a probabilistic model is established based on prior knowledge to describe the relationship between the intensity adjustment plan and the risk indicators. Then, in each iteration, the probabilistic model is updated based on the existing sample data, and an intensity adjustment plan with the highest expected improvement is selected for trial. Through multiple iterations, the optimal intensity adjustment plan is gradually approached and its risk bounds are determined. For example, if the risk bounds exceed the acceptable range, the decision engine adjusts the intensity adjustment plan to ensure the safety of the training process.

[0085] Action combinations that violate medical guidelines are filtered out through logical consistency constraints for rehabilitation stage transitions. During rehabilitation training, different rehabilitation stages have different training requirements and contraindications. Logical rules for rehabilitation stage transitions are established based on rehabilitation medicine guidelines. For example, in the early stages of rehabilitation, patients have limited muscle strength and joint range of motion, so high-intensity, large-amplitude movements should be avoided. In the later stages of rehabilitation, the intensity and difficulty of training can be gradually increased. When generating a training action sequence, the decision engine checks whether the action combination complies with these logical rules based on the patient's current rehabilitation stage. If there is an action combination that violates medical guidelines, such as choosing an overly strenuous exercise in the early stages of rehabilitation, the decision engine will filter it out and regenerate a training action sequence that complies with logical consistency constraints to ensure the scientific nature and rationality of the training process.

[0086] Example 5:

[0087] When detecting abnormal patterns in VR interaction logs, a variational autoencoder is used to reconstruct normal interaction patterns and calculate a reconstruction error threshold. A variational autoencoder is a generative model based on deep learning, consisting of an encoder and a decoder. First, normal VR interaction logs are fed into the variational autoencoder as training data. The encoder maps the interaction logs into a low-dimensional latent space, obtaining a latent vector representation. The decoder reconstructs the original interaction logs from the latent vectors. During training, the variational autoencoder parameters are optimized by minimizing the reconstruction error (such as mean squared error). After training is complete, new VR interaction logs are encoded into latent vectors by the encoder and reconstructed by the decoder. The reconstruction error between the reconstructed interaction logs and the original interaction logs is calculated. Using a large amount of normal interaction log data, the distribution of the reconstruction error can be statistically analyzed, allowing an appropriate reconstruction error threshold to be set. When the reconstruction error of a new interaction log exceeds this threshold, an abnormal interaction is detected.

[0088] When an abnormal interaction is detected, the decision engine's trajectory replanning mechanism is triggered. This trajectory replanning mechanism uses a memory-augmented neural network to model historical training sequences. The memory-augmented neural network consists of a memory module and a neural network module. The memory module stores feature information from historical training sequences, such as the characteristic templates of successful training patterns. When the trajectory replanning mechanism is triggered, the characteristic templates of successful training patterns are extracted through a memory access gating mechanism. The memory access gating mechanism selects an appropriate characteristic template for access based on the current interaction state and the information in the memory module. For example, the similarity between the current interaction state and each characteristic template in memory can be calculated, and the characteristic template with the highest similarity can be selected. Then, based on the similarity between the characteristic template and the current state, a locally fine-tuned training action topology is generated. For example, if the current state has a high degree of similarity to a characteristic template of a successful training pattern, but there are some differences, the training actions in the characteristic template can be locally fine-tuned based on these differences, such as adjusting the amplitude and sequence of the movements. This generates a new training action topology as an alternative training plan adapted to the current rehabilitation progress. This allows for timely response to abnormal interactions and ensures the smooth progress of rehabilitation training.

[0089] Example 6:

[0090] In the trajectory replanning mechanism, the memory read gating mechanism of the memory-augmented neural network plays a key role. When an abnormal interaction is detected and trajectory replanning is required, the memory read gating mechanism starts to work. Assume that the memory module stores multiple feature templates of successful training patterns, denoted as , each feature template Contains various characteristic information of training movements, such as the sequence, amplitude, time interval, etc. The current rehabilitation status is represented by , by calculating the current state With each feature template Similarity , for example, using the cosine similarity calculation method, ,in represents the dot product operation of vectors, Represents the norm of the vector. Select the one with the highest similarity Feature templates ( According to the actual situation), these feature templates represent the successful training mode that is most similar to the current state. The feature templates are fused, for example, a weighted average method can be used to obtain a comprehensive feature template. , weights are assigned according to the similarity, the higher the similarity, the greater the weight. Based on this comprehensive feature template With the current state Based on the differences in the training movements in the feature template, local fine-tuning is performed on the training movements in the feature template. For example, if the range of joint motion in the current state is smaller than the expected range in the feature template, the amplitude of the movement is reduced accordingly; if muscle strength is weak in the current state, the intensity of the movement can be appropriately reduced. Through these adjustments, a locally fine-tuned training movement topology is generated as a new training plan to meet the rehabilitation training needs after abnormal interactions.

[0091] In terms of constructing a rehabilitation effect prediction model, based on the patient's training frequency and musculoskeletal function recovery curve, a neural differential equation is used to model the dynamic evolution of the rehabilitation process. Assume that the patient's training frequency is , the musculoskeletal function recovery index is ,in represents the training time. The neural differential equation can be expressed as ,in are the parameters of the neural differential equation, learned from the training data. This equation describes the relationship between the rate of change of musculoskeletal function recovery indicators over time, the current musculoskeletal function status, and the training duration. By solving this neural differential equation, we can predict the patient's musculoskeletal function recovery at different training durations.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0093] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A virtual fusion personalized upper limb rehabilitation training cloud platform, characterized by: include: Multi-source bio-signal acquisition module: used to collect multi-source bio-signals for upper limb rehabilitation training in real time, including surface electromyography signals, joint motion trajectory data, tactile pressure distribution matrix, virtual reality interaction logs, and physiological indicator time series data; Movement function feature modeling module: performs hierarchical feature modeling on the multi-source biological signals to generate functional feature vectors corresponding to each modality. The functional feature vectors include: electromyographic activation pattern features, joint kinematic chain topology features, pressure distribution symmetry features, virtual environment interaction intention features, and cardiopulmonary metabolic load index; Heterogeneous data fusion module: A domain adaptation algorithm based on transfer learning maps the functional feature vectors to a unified rehabilitation assessment space to generate a fused rehabilitation status representation; Rehabilitation knowledge graph construction module: Constructs a dual-layer graph library consisting of a rehabilitation medicine ontology graph and a personalized training model graph. The dual-layer graph library stores the patient's musculoskeletal function injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategies; Dynamic training planning module: Based on the fused rehabilitation state representation and the dual-layer atlas library, a decision engine driven by reinforcement learning is used to generate personalized rehabilitation action sequences and intensity adaptive adjustment plans.

2. The personalized upper limb rehabilitation training cloud platform according to claim 1, characterized in that: The performing hierarchical feature modeling on the multi-source biological signals includes: The surface electromyographic signal is decomposed by wavelet packet combined with convolutional recursive network to extract the time-frequency domain muscle co-activation pattern; Using Lie group space modeling algorithm to analyze the rigid body transformation parameters of the three-dimensional kinematic chain for the joint motion trajectory data; The tactile pressure distribution matrix is ​​subjected to a graph convolutional network combined with an attention mechanism to identify the pressure center migration path characteristics.

3. The personalized upper limb rehabilitation training cloud platform according to claim 2, characterized in that: The motion function feature modeling module also includes: The virtual reality interaction log is extracted using a spatiotemporal graph convolutional network to extract the action intention sequence, and the behavior decision logic dependency is annotated using a causal reasoning algorithm; A dynamic mode decomposition algorithm is used to separate the steady-state and transient components of the cardiopulmonary metabolic load from the physiological indicator time series data.

4. The personalized upper limb rehabilitation training cloud platform according to claim 1, characterized in that: The dual-layer atlas library for constructing the rehabilitation medicine ontology atlas and the personalized training model atlas includes: Based on the International Classification of Functional Disabilities standard entities and the evidence-based relationship between rehabilitation actions and therapeutic effects, a knowledge graph embedding algorithm is used to construct an extensible rehabilitation medicine ontology graph; Based on the patient's historical training data and functional evaluation indicators, a federated learning framework is used to generate node embedding representations of the distributed personalized training model graph.

5. The personalized upper limb rehabilitation training cloud platform according to claim 1, characterized in that: The reinforcement learning-driven decision engine adopts a curriculum learning framework, including: The state space is defined as the joint encoding of the current rehabilitation state representation and the medical ontology subgraph, and the action space is defined as the candidate set of rehabilitation training actions; The training sequence policy is generated by a double-delayed deep deterministic policy gradient network, and the reward function is designed based on biomechanical safety constraints.

6. The personalized upper limb rehabilitation training cloud platform according to claim 5, characterized in that: The decision engine also includes: An evolutionary strategy algorithm is used to explore the diversity of training action sequences, and a Bayesian optimization algorithm is used to calculate the risk boundary of the intensity adjustment scheme. Action combinations that violate medical guidelines are filtered through logical consistency constraints of rehabilitation stage transitions.

7. The personalized upper limb rehabilitation training cloud platform according to claim 1, characterized in that: The platform also includes: Performing abnormal pattern detection on the virtual reality interaction log, reconstructing a normal interaction pattern using a variational autoencoder, and calculating a reconstruction error threshold; When an abnormal interaction is detected, the trajectory replanning mechanism of the decision engine is triggered to generate an alternative training plan that is adapted to the current rehabilitation progress.

8. The personalized upper limb rehabilitation training cloud platform according to claim 7, characterized in that: The trajectory replanning mechanism uses a memory-augmented neural network to model historical training sequences, including: Extracting feature templates of successful training patterns through memory readout gating mechanism; Based on the similarity matching between the feature template and the current state, a training action topology structure after local fine-tuning is generated.

9. The personalized upper limb rehabilitation training cloud platform according to claim 1, characterized in that: Place The platform also includes: Construct a rehabilitation effect prediction model based on the patient's training frequency and musculoskeletal function recovery curve Line, using neural differential equations to model the dynamic evolution of the rehabilitation process; The rehabilitation effect prediction model also includes: Based on the dynamic graph neural network, the rehabilitation stage division criteria and individual recovery rate differences are aligned to generate an adaptive training cycle adjustment strategy and embed it into the training sequence strategy.

10. A virtual fusion personalized upper limb rehabilitation training method, comprising the following steps: Step 1: Real-time collection of multi-source biological signals of the patient's upper limb rehabilitation training, wherein the multi-source biological signals include surface electromyography signals, joint motion trajectory data, tactile pressure distribution matrix, virtual reality interaction log and physiological indicator time series data; Step 2: Perform hierarchical feature modeling on the multi-source biological signals to generate functional feature vectors corresponding to each modality, wherein the functional feature vectors include myoelectric activation pattern features, joint kinematic chain topology features, pressure distribution symmetry features, virtual environment interaction intention features, and cardiopulmonary metabolic load index; Step 3: Mapping the functional feature vector to a unified rehabilitation assessment space using a domain adaptation algorithm based on transfer learning to generate a fused rehabilitation status representation; Step 4: Pre-build a double-layer atlas library of rehabilitation medicine ontology and personalized training mode atlas, wherein the double-layer atlas library stores the patient's musculoskeletal function injury level, rehabilitation movement-efficacy association rules, and historical training trajectory optimization strategy; Step 5: Based on the fused rehabilitation state representation and the two-layer atlas library, a decision engine driven by reinforcement learning is used to generate a personalized rehabilitation action sequence and an intensity adaptive adjustment plan.

Images