Bed-in-bed complete-period rehabilitation system for leg fracture based on hybrid real force and touch interaction and control method of bed-in-bed complete-period rehabilitation system

Abstract

The invention relates to the technical field of medical rehabilitation equipment control, in particular to a leg fracture bedridden complete-cycle rehabilitation system based on hybrid real force touch interaction and a control method, by adopting the method provided by the invention, a multi-mode rehabilitation state feature set is constructed, a coupled feature set is formed through feature coupling, and the rehabilitation effect of a patient is improved. The method comprises the following steps: firstly, dynamically updating a local dynamic transition threshold value and a global dynamic transition threshold value, setting a trigger condition, judging whether to transition from a current rehabilitation stage to a next stage or not according to the trigger condition, and dynamically updating the local dynamic transition threshold value and a global accumulation threshold value; and a control signal for virtual display is sent to the MR interaction unit according to the current rehabilitation stage and a preset condition, so that real-time adaptive switching of multi-stage rehabilitation training is realized, a mixed reality function is realized in combination with the MR interaction unit, lightweight, immersive and personalized rehabilitation experience is provided for a patient, leg function recovery is accelerated, and the rehabilitation effect of the patient is improved. The rehabilitation efficiency and quality are improved.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation equipment control technology, and more specifically, to a full-cycle rehabilitation system and control method for leg fractures in bed based on mixed reality force-tactile interaction. Background Technology

[0002] In existing technologies, traditional rehabilitation equipment primarily relies on mechanical movements, providing only passive limb activity and lacking multimodal feedback such as visual and tactile feedback. Existing VR rehabilitation systems construct virtual environments that are independent of the real physical space, making it difficult to accurately match movement commands in the virtual environment with the patient's actual limb movement. For example, if the virtual environment requires a leg lift of 20cm, but the patient's actual leg lift is insufficient, the device cannot adjust its feedback in real time, leading to distorted training movements. Long-term use may result in compensatory movement patterns, affecting rehabilitation outcomes and potentially causing secondary injuries.

[0003] The training parameters of existing rehabilitation equipment (such as resistance level and range of motion) cannot be automatically adjusted according to the patient's real-time status during training (such as muscle fatigue level and changes in joint range of motion). If the patient experiences fatigue or discomfort during training and medical staff do not detect it in time, it may lead to training interruption or increased risk of injury, affecting the continuity and safety of rehabilitation.

[0004] Current rehabilitation techniques primarily develop training plans based on the patient's current condition, lacking the ability to predict rehabilitation trends. They cannot identify potential rehabilitation risks (such as early signs of joint stiffness) in advance and adjust plans accordingly. Often, responses are reactive, occurring only after problems arise, missing optimal intervention opportunities, prolonging the rehabilitation period, and increasing patient suffering and medical costs. Summary of the Invention

[0005] The purpose of this invention is to provide a full-cycle rehabilitation system and control method for leg fractures based on mixed reality tactile interaction, in order to solve the above-mentioned problems in the prior art.

[0006] This invention is achieved through the following technical solution:

[0007] In a first aspect, the present invention provides a control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction, comprising: Acquire the current patient's multimodal data and perform data preprocessing on the multimodal data; Feature extraction is performed on the multimodal data after data preprocessing, and the obtained features are fused to construct a multimodal rehabilitation state feature set. A coupled feature set is then formed through feature coupling. An adaptive neural network model is constructed. The adaptive neural network model takes a multimodal feature set and a coupling feature set as inputs and outputs the coupling step value of the rehabilitation stage and the cumulative coupling comprehensive rehabilitation level. A local dynamic transition threshold is constructed based on the coupling step value of the rehabilitation stage, and a global dynamic transition threshold is constructed based on the cumulative coupling comprehensive rehabilitation level. The model parameters are optimized by using a loss function, the local dynamic transition threshold and the global dynamic transition threshold are dynamically updated, trigger conditions are set, and it is determined whether to transition from the current recovery stage to the next stage based on the trigger conditions. Several rehabilitation training scenarios are constructed based on different rehabilitation stages, and the adaptive switching of rehabilitation training scenarios is automatically triggered based on the updated local dynamic transition threshold and global dynamic transition threshold.

[0008] Several rehabilitation training scenarios were constructed based on different stages of rehabilitation, including: The system obtains the current patient's rehabilitation stage and sets preset conditions for each stage. Based on the current rehabilitation stage and the corresponding preset conditions, it sends control signals to the MR interactive unit for virtual display.

[0009] Preferably, the multimodal data includes basic physiological data, skeletal structure and morphology data, biomechanical data, functional and motor data, and feedback data.

[0010] Preferably, the types of rehabilitation stages include a bed rest period, a transition period from bed to lodging, and a preparatory period for independent walking; Obtain the start time node of the patient's surgery and the current target time node. Based on the start time node and the target time node, obtain the current postoperative time and output the current rehabilitation stage type based on the postoperative time.

[0011] Preferably, if the current rehabilitation stage is a bed rest period, a three-dimensional virtual skeleton model of the affected lower limb is projected, a safe activity area is set, and it is displayed in a color different from that of the three-dimensional virtual skeleton model. Set alarm thresholds and safety thresholds, and issue different colored prompts and control signals based on the maximum hip joint range of motion, alarm thresholds, and safety thresholds; When a control signal for the first training is received, a virtual trajectory is projected to move at a set speed, and the current patient's angular fluctuation rate is collected in real time. A first angular fluctuation rate threshold is set. When the angular fluctuation rate is greater than the first angular fluctuation rate threshold, a control signal is issued to stop the projection of the virtual trajectory and to activate the airbag feedback component.

[0012] Preferably, if the current rehabilitation stage is a transitional period from bed rest, the preset conditions include a first judgment threshold and a second judgment threshold, wherein the first judgment threshold is greater than the second judgment threshold, and further include: When a control signal for the second training is received, several virtual obstacles of N meters in height are projected, and the maximum range of motion of the patient's hip joint during training is obtained. If the maximum range of motion is less than the first judgment threshold, the height of the virtual obstacles is adjusted by the height adjustment model and then redisplayed. If the maximum range of motion of the hip joint is less than the second judgment threshold, a signal is issued to pause the second training session and require the first training session to be performed again before the second training session can begin.

[0013] Preferably, if the current rehabilitation stage is the preparatory stage for independent walking, the preset conditions include a step length judgment threshold based on the maximum range of motion of the hip joint, and also include: Based on the initial step length setting, a virtual footprint that the patient needs to move to next is projected according to the current patient position. The patient's actual step length is obtained based on the maximum range of motion of the hip joint and the stepping time. A second angular volatility is set. When the angular volatility is greater than the second angular volatility, a standard demonstration action is issued for display. Generate a virtual step, set the ratio between the height of the virtual step and the maximum range of motion of the hip joint, and adjust the height of the virtual step based on the value of the maximum range of motion of the hip joint; The resistance value of the SMA filament bundle is adjusted according to the height of the virtual step. Secondly, the present invention also provides a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction, comprising: The wearable device includes an outer skin-friendly layer, a middle sensing layer, and an inner feedback layer arranged sequentially. The middle sensing layer contains a pressure sensor, a triaxial accelerometer, and an inertial measurement unit for data acquisition. The inner feedback layer contains an SMA filament bundle and an airbag feedback assembly. MR display equipment is used to receive control signals and display virtual images; The main control device is connected to the wearable device and the MR display device, and is used to execute the above-mentioned control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction.

[0014] The technical solution of the present invention has at least the following advantages and beneficial effects: Using the method provided by this invention, a multimodal rehabilitation state feature set is constructed, and a coupled feature set is formed through feature coupling; a hierarchical memory dynamic transition model is constructed, and based on the feature space and the coupled feature set, the hierarchical memory dynamic transition model outputs the coupled step value of the rehabilitation stage to obtain the result of whether to transition to the next rehabilitation stage; an adaptive neural network model is established to dynamically update the local dynamic transition threshold and the global cumulative threshold in real time, and finally realize the real-time adaptive switching of multi-stage rehabilitation training, providing patients with a lightweight, immersive, and personalized rehabilitation experience, accelerating the recovery of leg function, and improving rehabilitation efficiency and quality. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the control method of the present invention; Figure 2 This is a schematic diagram of the adaptive neural network model structure of the present invention; Figure 3 This is a schematic diagram of the control system configuration of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The naming or numbering of steps in this application does not imply that the steps in the method flow must be executed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical objective, as long as the same or similar technical effect is achieved.

[0019] Please refer to Figures 1-3 The present invention provides a control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction, comprising: S101: Obtain the current patient's multimodal data and perform data preprocessing on the multimodal data; S102: Extract features from the preprocessed multimodal data, fuse the obtained features to generate a multimodal rehabilitation state feature set, and form a coupled feature set through feature coupling; S103: Construct an adaptive neural network model, using the multimodal feature set and the coupling feature set as input, calculate and output the coupling step value of the rehabilitation stage and the cumulative coupling comprehensive rehabilitation level through the model, generate a local dynamic transition threshold based on the coupling step value of the rehabilitation stage, and generate a global dynamic transition threshold based on the cumulative coupling comprehensive rehabilitation level. S104: Optimize the parameters of the adaptive neural network model through a loss function to dynamically update the local dynamic transition threshold and the global dynamic transition threshold; set trigger conditions based on the updated local and global dynamic transition thresholds; when the trigger conditions are met, generate a control command to switch from the current rehabilitation stage to the next stage.

[0020] S105: Pre-construct multiple rehabilitation training scenarios corresponding to different rehabilitation stages; in response to the stage switching control command, perform adaptive switching of rehabilitation training scenarios.

[0021] The multimodal data are as follows: Basic physiological data: demographic data, basic medical history, metabolic data, baseline bone mineral density, limb swelling, local soft tissue tightness, joint effusion grade, body surface temperature gradient, skin impedance, local nerve conduction velocity, local blood flow, and inflammatory factor levels; Bone structure and morphology data: dynamic bone stress imaging, callus collagen fiber arrangement direction, three-dimensional closure rate of fracture line, callus maturity, and bone structure symmetry; Biomechanical data: bone strength, bone stiffness, lower limb extensor strength, muscle resistance, muscle endurance, dynamic stress data, muscle-skeleton synergistic load distribution, strain distribution uniformity, MR force feedback error rate, tendon-skeleton insertion stress distribution, and muscle fatigue dynamic threshold. Functional and motion data: joint range of motion (hip, knee, ankle), joint stability, gait data (stride length, cadence, stride width), dynamic stress data, plantar pressure distribution, proportion of weight-bearing phase in gait cycle, balance ability, limb weight-bearing ratio, balance maintenance time, body center of gravity trajectory, virtual and real trajectory synchronization coefficient, multi-joint phase difference in gait cycle, environmental adaptability score. Subjective feedback data: pain assessment, tactile sensitivity, functional impairment score, quality of life level, MR immersion score, perceptual bias in achieving stage goals, training enjoyment, and rehabilitation confidence score.

[0022] The entire period of bed rest rehabilitation for leg fractures is divided into: A progressive stage. Let the stage index be denoted as . , Each stage Corresponding to a multimodal feature set To balance information integrity and computational efficiency, It consists of two parts: ,in: It is the preliminary stage ( A compressed memory representation of the feature set (stage). It uses a memory operator. The goal of this generation is to retain only the core rehabilitation goals predicted for the current stage. Features that make significant contributions and avoid the curse of dimensionality. Specifically, from Selecting from those that satisfy mutual information The feature subset, where From Stage feature set The extracted first One characteristic, It is a preset empirical threshold; then, dimensionality reduction is performed through principal component analysis to form... .

[0023] It is a stage The newly added feature sets are derived from newly enabled or newly collected multimodal data in this phase. ,in Representation phase The One new feature, For the stage The newly added feature dimension.

[0024] Existing technologies, when processing multi-stage rehabilitation features, typically analyze individual features independently, such as assessing muscle endurance and joint range of motion separately, without considering the synergistic or restrictive relationships between features. For example, when muscle endurance is high but joint range of motion is extremely low, a single analysis might misjudge the rehabilitation status as good, when in reality, joint stiffness is causing functional limitations. Similarly, when both pain scores and swelling are high, a single analysis can only reflect the abnormalities of each indicator, failing to capture the amplified risk effect of their combined effect. This isolated feature analysis leads to biased stage evaluations, thus affecting the accuracy of transition decisions. Therefore, a mechanism to quantify the interactions between features, i.e., coupled features, is needed to address the problem of insufficient feature correlation capture in existing technologies.

[0025] To quantify the correlation between features, this invention proposes to do so at each stage. Generate a coupling feature set Used to capture feature space The synergistic, restrictive, or weakest link effect between any two features is defined as follows: (1) in, Representation phase The complete feature space, including newly added features and historical embedded features; express Any two distinct features, Ensure that no double counting occurs; It is an open, coupled type collection that does not require a pre-fixed number of elements and can be dynamically expanded according to the needs of the actual rehabilitation scenario, for example: Cooperative coupling Such as the synergistic coupling of callus growth and protective weight-bearing; complementary coupling. For example, dual verification of inflammatory indicators such as joint effusion level and body surface temperature gradient; extreme value coupling. For example, when gait symmetry is high but balance ability is low, the minimum term focuses on the bottleneck of balance ability; causal coupling Such as the causal relationship between muscle resistance and joint stability; dynamic coupling. For example, real-time linkage between stride length and gait characteristics of the body's center of gravity trajectory; temporal coupling. Such as the correlation between preoperative indicators and postoperative recovery time; spatial coupling. Such as the symmetry of movements of the left and right limbs. For any , Is it a coupling type? The corresponding function, its form is... It is determined by its physical nature.

[0026] The Coupling Feature Set Proposed in this Invention This is a general framework that can be specifically constructed to capture key feature associations based on the core goals of the rehabilitation stage. In this embodiment, four typical coupling indicators are designed for the four core dimensions of bone healing, neural response, balance ability, and motor control, as follows: The Bone-Force Synergy (BFS) index quantifies the degree of match between the biomechanical strength of bone healing and the functional load applied by the patient through a force-tactile interaction system. Its calculation formula is as follows: (2) in, The value is the bone-strength synergy compatibility index, and the closer the value is to 1, the better the synergy.

[0027] and These represent the maximum force that the healthy and affected limbs can stably apply in a standardized MR virtual task (such as "virtual stepping"), respectively. By analyzing the raw signal from the force sensor Peak detection is performed, and the average of the strongest values ​​from three consecutive successful tasks is obtained. Indicates the callus strength ratio, This represents the elastic modulus of the callus extracted from images (such as CT scans). This represents the elastic modulus of the healthy side bone. Indicates the force feedback error rate. The preset target force or force range in the virtual task. To actually exert force on the patient and The average force feedback deviation. , , For dynamic weighting coefficients, satisfying The system has built-in initial weight values, and adjusts them according to the current recovery stage. The rules are dynamically adjusted in accordance with the doctor's preset rules.

[0028] Neuro-Force Feedback Entropy (NFFE) is used to quantify the ordered response of the nervous system to diverse force-tactile stimuli, reflecting the level of recovery of sensorimotor integration. Its calculation is based on the principle of information entropy. (3) in, The entropy is the neural-mechanical feedback response entropy. A decrease in entropy indicates that the neural response tends to be more ordered. This indicates the number of types of force-tactile stimuli, such as SMA resistance, vibration, and pneumatic pulses, applied pseudo-randomly by the MR system within a single evaluation period. For the first The probability that a stimulus elicits an effective neural response, defined as: within a 300ms time window after stimulus application, the amplitude of the surface electromyography (EMG) signal of the target muscle group exceeds the resting threshold. (Usually set to twice the root mean square of resting electromyography) events. , For the first The number of times a stimulus elicits a valid response. This represents the total number of stimulations.

[0029] The Load-Balance Dynamic Coupling (LBDC) index is used to assess the real-time synergistic relationship between the weight-bearing capacity of the affected limb and overall balance control in a MR (Magnetic Resonance) environment. Its calculation formula is as follows: (4) in, This is the load-balance dynamic coupling index. A higher value indicates a stronger load-bearing capacity while maintaining balance. The time integral ratio of the sum of pressures borne by the affected side to the total pressure of the whole body (including the affected side, healthy side, support points, etc.) during the task cycle is collected by a high-resolution pressure distribution sensing system. The standard deviation of the fluctuation of the body's center of pressure within the same task cycle is a key indicator for measuring the stability of static and dynamic equilibrium. It is obtained by calculating the standard deviation of the Euclidean distance between the trajectory point of the center of pressure and its average position. This is a complexity coefficient for MR rehabilitation scenarios, used to standardize index values ​​for scenarios of different difficulty, such as flat ground. slope Disturbance field .

[0030] The Virtual-Real Trajectory Consistency (VRTC) index is used to evaluate the degree of similarity between a patient's actual movement trajectory and a preset or personalized ideal trajectory when performing MR virtual rehabilitation tasks. It directly reflects motor control ability. The preferred calculation formula is as follows: (5) in, The consistency index of virtual and real motion trajectories, range The higher the value, the better the consistency. and These are the patient's actual movement trajectory collected by the MR positioning system and the standard healthy trajectory or personalized trajectory based on the healthy side mirror image stored in the system. It is a dynamic time warping algorithm used to calculate the minimum cumulative distance between two variable-length time series data. It is not sensitive to differences in motion speed and focuses on the similarity of trajectory shape. The normalization scaling factor is usually taken as the maximum value of the standard trajectory amplitude. T is the theoretical duration of the task.

[0031] The above indicators are merely exemplary implementations of feature coupling. In practical applications, more coupling indicators can be expanded and incorporated into the coupling feature set according to rehabilitation needs (such as joint range of motion, pain scores, etc.). .

[0032] based on Constructing a coupled feature set In this case, these exemplary metrics can further participate in cross-dimensional coupling to form more complex correlation features, such as cross-dimensional product terms. (The linkage between skeletal adaptability and load-bearing-balance synergy reflects the overall state of "the skeleton being able to bear weight and the body being able to maintain balance"); cross-dimensional minimum term (The shortcomings in trajectory consistency and the orderliness of neural response reflect the bottleneck of motor control precision being limited by neural regulation).

[0033] To quantify the stages of fracture rehabilitation In view of the overall state, this invention constructs the Rehabilitation Stage Coupling Step Value (RSCSV), denoted as... By integrating the absolute level of the feature space with the coupling relationship between features, it is possible to achieve stage-specific control. The comprehensive quantification of recovery status is calculated using the following formula: (6) The first item is the basic characteristics of rehabilitation, which is included in the stage. Core rehabilitation indicators Representation phase feature The importance weight is set through regression models or expert experience, with core features receiving higher weights; the second term is a coupling effect modifier, incorporating the correlation between features (e.g., ...). , (cross-dimensional coupling), through weights Quantitative Coordination (Positive Regulation) ) or risk coupling (negative adjustment) The stronger the coupling, the better the rehabilitation coordination, and the larger this item is. Coupling characteristics In the stage The interaction effect coefficient, These are elements within the coupling feature set.

[0034] To quantify the progress from the initial stage of rehabilitation to the current stage To assess the overall rehabilitation level, this invention constructs a Cumulative Stage Coupling Comprehensive Rehabilitation Level (CSCCRL), denoted as... By dynamically integrating the coupling characteristics of each historical stage with the evaluation of the current stage, a global decision-making basis is provided for the dynamic transition of hierarchical memory. The calculation formula is as follows: (7) in, Indicates as of The comprehensive rehabilitation state at each stage is a historical memory carrier of coupled information from all previous stages; Indicates the current stage RSCSV; It contributes a decay coefficient to the historical stage, with a value range of [value range missing]. This is used to balance the contribution of historically accumulated information with the new characteristics of the current stage. By assigning higher weight to recent stages, it ensures that new recovery states have a priority in influencing the overall evaluation, while retaining the memory of historical trends (the earlier the stage, the greater the impact). (Power-law decay), to avoid short-term fluctuations interfering with overall judgment.

[0035] Based on the coupling step value in the rehabilitation stage (RSCSV) Constructing Local Dynamic Transition Thresholds As a way to determine the patient's current stage Leap to the next stage The threshold is a phased quantitative standard. This threshold is dynamically adjusted based on the performance in the previous stage, adapting to the individual's recovery pace. The specific calculation is as follows: (8) in, Representation phase The local dynamic transition threshold reflects the threshold history memory of the preceding stage, ensuring the continuity of threshold adjustment. The initial threshold... Based on population baseline data. This represents the historical weighting coefficient, with a value range of [value range missing]. This controls the influence of the preceding threshold on the current threshold, preventing sudden changes. Representation phase The baseline target value is derived from the minimum achievement requirements of the core needs at each stage of rehabilitation as specified in clinical rehabilitation guidelines, as well as the average achievement of patients at that stage based on a large sample of historical data. Value statistics results. This represents the relative deviation between an individual's current step value and the stage's baseline target value. This is the deviation correction factor, and its value range is... ,when When (performing well), Appropriately raising the threshold is to verify the stability and sustainability of progress; conversely, lowering the threshold avoids unreasonably prolonged rehabilitation cycles due to excessive stringency.

[0036] Based on cumulative coupling comprehensive rehabilitation level (CSCCRL) Constructing a global dynamic transition threshold As a way to determine the stage of the patient's condition This is a quantitative basis for determining whether the overall cumulative rehabilitation level meets the requirements for stage progression. This threshold varies with stage. Dynamic increment (i.e.) ,in To achieve the final global threshold for rehabilitation, the system integrates group-level baselines and individual cumulative trends to achieve phased individualized calibration. The calculation formula is as follows: (9) in, Representation phase The corresponding global cumulative threshold is the minimum global cumulative requirement that needs to be achieved in this stage; Representation phase The global threshold for the population baseline is based on a large sample of patients at the same stage. Values ​​were obtained through statistical analysis; Indicates patient baseline deviation, quantifying individual initial state. The difference from the average initial state of the group at the same stage, The baseline influence coefficient controls the magnitude of the initial state's correction to the threshold, and its value range is [value range missing]. ; Indicates the patient's current stage The CSCCRL reflects the historical cumulative trend; This indicates the initial CSCCRL value at the start of the patient's rehabilitation (such as the first assessment after a fracture or before the rehabilitation program is initiated). This represents the cumulative trend coefficient, with a value range of [value range missing]. The threshold is fine-tuned based on the historical cumulative growth rate.

[0037] From the stage Leap to stage The triggering conditions are as follows: (10) in, Representation phase Achieve specific goals; Ensure the stability of the jump; This indicates that the overall cumulative trend has met the target at the current stage; The requirement for the new stage is to change the characteristics from "unmeasurable" to "measurable," such as the ability to stand, which allows for the collection of gait features; Representation feature set The transition is triggered when all four conditions are met: "extension compatibility" (meaning the next stage feature can be extended based on the current feature); ... and "extension compatibility" (meaning the next stage feature can be extended based on the current feature).

[0038] The adaptive neural network model is a modular network designed specifically for multi-stage rehabilitation processes. Its core function is to comprehensively process the multimodal features of the current stage and historical rehabilitation information, and output a series of key intermediate calculation parameters through dynamic learning, which are ultimately used to generate control commands for switching rehabilitation stages.

[0039] Its inputs include: Current recovery stage index Maintained by the system state machine, and initialized by the doctor; the current stage's standardized multimodal feature set. and coupling feature set The system stores the cumulative comprehensive rehabilitation level of the preceding period. .

[0040] Its output includes: the coupling step value used to calculate the current rehabilitation phase. Dynamic weights Coupling effect coefficient Used to calculate the cumulative coupled comprehensive rehabilitation level Historical attenuation coefficient ; Coefficients used to dynamically adjust local and global transition thresholds , , , ; and the stage switching control instructions generated based on the above parameters and preset trigger conditions logic.

[0041] The model, from input to output, includes the following levels: Input and preprocessing layer: receives raw input and indexes the stages. Converted to one-hot encoding and used as a globally gating signal. and Standardize the process.

[0042] Multi-stage dynamic weighted branch layer, determined by the total number of rehabilitation stages. This consists of multiple parallel fully connected subnetworks. Each subnetwork is dedicated to learning the importance distribution of various features at the corresponding rehabilitation stage. Through the aforementioned one-hot encoding gating, only features relevant to the current stage are activated. The corresponding subnetwork outputs the dynamic weights of each feature in the current stage. .

[0043] The coupled feature interaction layer employs a network module containing a multi-head attention mechanism to process the coupled feature set. Modeling is performed. This mechanism can capture different coupling features (such as...). , The inherent connections and mutual influences among (etc.) are mapped to the effect coefficients of each coupled feature through a fully connected layer. .

[0044] The cumulative memory fusion layer is a small, fully connected network. and As input, the output is an adaptive attenuation coefficient. . Then it is calculated by equation (7).

[0045] The threshold parameter learning layer consists of two sub-modules: a local threshold parameter module that learns the coefficients. and Global threshold parameter module learning coefficients and . and Then, according to equations (8) and (9) respectively, the system-stored data is called. , It is calculated from the benchmark value.

[0046] The instruction generation layer will calculate the... , , , Substitute the preset trigger condition (10) into the equation for comparison. If all conditions are met, generate a stage switching control instruction pointing to the next rehabilitation stage (k+1); otherwise, generate an instruction to maintain the current stage.

[0047] During the recovery phase During a particular evaluation period, the model workflow was as follows: After the input data is preprocessed, the activation phase begins. A dedicated dynamic weighting branch calculates the contribution of each indicator to the current stage. Simultaneously, the coupling feature interaction layer analyzes the complex relationships between features, and the cumulative memory fusion layer integrates historical progress; the threshold parameter learning layer outputs threshold coefficients that adapt to the current individual state; finally, the instruction generation layer generates stage switching control instructions based on all calculation results and preset condition rules, thereby driving the automatic switching between mixed reality scenes and force haptic training.

[0048] The adaptive neural network model was trained based on historical fracture rehabilitation case data. Before application, all case data were anonymized and de-identified, and the continuous rehabilitation records were divided into structured samples according to their actual rehabilitation stage, with each sample corresponding to a specific rehabilitation stage. Status data.

[0049] A single patient's data sample can be represented as ,in This represents the total number of stages experienced by the patient. , Stages Multimodal feature sets and coupled feature sets. , , , All values ​​are labeled true values ​​given by clinical experts based on relevant information, and are used as optimization targets for the Rehabilitation Stage Coupled Step Value (RSCSV), Cumulative Coupled Comprehensive Rehabilitation Level (CSCCRL), and Local and Global Dynamic Transition Thresholds, respectively.

[0050] Total loss of a single patient sample It consists of a weighted average of three loss terms, each corresponding to a key output variable:

[0051] in, , , The weights for each loss term are determined through cross-validation. The optimization objective is to maximize the model's overall performance on the independent validation set. This performance is measured by both the accuracy of stage transition prediction and a safety metric. The safety metric is quantified by the proportion of samples where the patient's actual recovery level is below the clinical safety threshold when the model suggests a stage transition. The specific loss terms are as follows: Local threshold loss term Local dynamic transition thresholds used to constrain model generation Approaching the expert's annotation The mean squared error (MSE) is used for calculation:

[0052] in, The total number of stages. Minimize... Optimize threshold accuracy to reduce the risk of premature or late switching due to threshold deviation from clinical standards at the algorithm level.

[0053] Global threshold loss term Compared with cumulative level loss item Both were calculated using MSE and used to constrain the global dynamic transition thresholds generated by the model. Comprehensive rehabilitation level coupled with cumulative duration Approximating its corresponding expert-annotated true value ( and The calculation formulas are as follows:

[0054]

[0055] minimize Ensure that the model, when making personalized adjustments to the global threshold, does not deviate from a reasonable baseline established based on group rehabilitation data; and minimizes... Then the driving model optimizes its internal parameters (such as the historical decay coefficient). ),make The calculations can adapt to the recovery pace of different patients: For patients who recover quickly More sensitive to recent progress; for patients with slow recovery, Greater emphasis is placed on the stability of historical trends to avoid distorted assessments.

[0056] Model training is performed in batches, incorporating the loss function of all patient samples within each batch. The average value is used as the total batch loss. Through collaborative learning of batch samples, the model parameters are adapted to the rehabilitation characteristics of different patients.

[0057] During training, gradient descent algorithms (such as Adam) are used to calculate... Gradients are applied to all learnable parameters of the model, and the parameters are iteratively updated until the loss function converges on the validation set.

[0058] For patients with long recovery periods, whose recovery process may exhibit individual-specific trends, such as sudden acceleration, stagnation, or fluctuations, the model incorporates an online personalized fine-tuning mechanism. Using recent patient data and the latest status assessment, the pre-trained model is lightly fine-tuned, and the loss function is simplified to:

[0059] This fine-tuning process ignores global threshold loss. Only parameters relevant to recent dynamics are updated, and the adjustment magnitude is constrained to prevent the model from deviating excessively from the pre-training state and introducing bias.

[0060] Through this mechanism, the model can dynamically capture personalized trends in the long-term rehabilitation of patients, such as the slowdown in the later stages of recovery for elderly patients and the surge in the middle stages for young patients, thus achieving optimization from group benchmarks to precise individual assessments.

[0061] An exemplary embodiment of the present invention is illustrated using a proximal femoral fracture as an example. The entire rehabilitation cycle is divided into three progressive stages. The system processes multimodal data from each stage to drive adaptive switching between mixed reality (MR) scenes and force-tactile interactions, thereby achieving intelligent rehabilitation training control from bed rest and getting out of bed to preparation for independent walking.

[0062] Corresponding to the bed rest period (For example, 1-4 weeks after surgery), the rehabilitation goal is to control pain and swelling, maintain joint range of motion within a safe range, and prevent complications such as muscle atrophy and pressure sores.

[0063] Corresponding to the transition period from bed to bed (For example, 5-8 weeks after surgery), the rehabilitation goal is to increase the range of motion of the active joints, carry out partial weight-bearing training under close monitoring, and rebuild basic balance ability.

[0064] Preparatory period for independent walking (For example, 9-12 weeks after surgery), the rehabilitation goal is to restore near-normal joint mobility and weight-bearing capacity, and to carry out gait correction training to prepare for fully independent walking.

[0065] During system initialization, medical staff input patient information and set the initial stage index. The system then loads the corresponding MR rehabilitation scene and force-tactile interaction parameters.

[0066] During the bed rest period, the system assists patients in maintaining joint movement within the set safety boundaries and prevents complications through multimodal feedback.

[0067] The MR system projects a three-dimensional model of the affected limb and generates a semi-transparent safe range of motion (such as hip flexion) based on fracture stability data. When the joint angle is collected in real time When approaching the safety boundary (e.g.) ), the edge of the area turns into a yellow alert; if Beyond the boundary (e.g.) The area flashes red and a warning text pops up. Simultaneously, the SMA tow actuator integrated into the flexible rehabilitation pants generates an increasing resistance torque, the magnitude of which is proportional to the degree of overextension, physically limiting excessive movement.

[0068] The system collects and analyzes pressure sensor array data in real time, calculating the pressure-time integral of key bony prominences. When the system determines that the PTI of a certain area has reached 80% of the preset risk threshold, it immediately executes a multimodal prompt: playing a turning-over guidance animation in the MR scene and triggering the pneumatic pulse component to generate a prompting vibration.

[0069] Healthcare professionals can initiate a "virtual trajectory following" task, where a virtual hip flexion trajectory is presented in MR space. The system calculates the consistency index between the patient's actual movement trajectory and this trajectory in real time. .when When the value is below the threshold, the system pauses the virtual trajectory and provides a text prompt, while simultaneously controlling the pneumatic airbag to apply tactile cues related to the error to assist motion control.

[0070] During the transition period from bed rest, the system expands the patient's range of mobility by introducing controlled weight-bearing and dynamic balance training.

[0071] Using SLAM technology, the MR system integrates the real ward environment with virtual elements. A virtual weight-bearing scale marked with a percentage of body weight is projected onto the floor. When the patient attempts to stand or shift their weight, the system calculates and displays the weight-bearing percentage on the affected side in real time based on plantar pressure data. If the weight-bearing percentage exceeds the safe upper limit (e.g., 30%), the scale line turns red, the virtual floor exhibits a "sunken" visual effect, and SMA fibers provide counter-assistance to help the patient reduce the load. If the weight-bearing percentage stabilizes within the target range (e.g., 20-30%), positive visual feedback (e.g., green ripples) is provided.

[0072] MR generates several virtual obstacles on a real ground surface, and patients need to practice walking around these obstacles. The system dynamically adjusts the training difficulty based on two key indicators: If the maximum range of motion of the affected hip joint If the target remains below the current stage, the system will proceed according to the formula. Reduce virtual obstacle height , At the current altitude, To adjust the coefficients and reduce the difficulty of taking the steps.

[0073] If joint angle fluctuation rate If the height is too high (indicating poor balance control), the virtual obstacle will emit flashing and sound warnings, while the airbag on the back of the lower leg will provide tactile cues to remind the patient to slow down and control their posture.

[0074] During the preparatory phase for independent walking, the system aims to restore patients' near-normal functional activity abilities through refined gait correction and training to adapt to complex environments.

[0075] The MR system analyzes gait cycles in real time, overlaying and displaying a gait comparison line between the affected and healthy sides onto the real ground. If stride asymmetry is detected, the system projects an elongated virtual footprint at the expected landing point of the affected foot, providing clear spatial guidance. If gait tremors (angular fluctuations) are detected... If the gait is too high, a virtual mirror image of the patient will be displayed to demonstrate the standard gait in real time for the patient to imitate.

[0076] The MR system constructs a rehabilitation corridor that includes virtual steps and ramps. The step height can be adaptively adjusted according to the patient's ability, for example, based on the patient's current hip joint range of motion. According to the formula The initial step height is set. As the patient lifts their leg to step, the SMA fiber bundle provides the step height. Proportional simulated resistance ( This is to enhance muscle proprioception and strength training load.

[0077] Throughout the rehabilitation process, when calculated , , , When the preset transition trigger conditions are met, the system immediately generates a stage switching control command. This command triggers seamless adaptive switching of rehabilitation training scenarios and updates force and tactile parameters. For example, it automatically switches from the "ward obstacle avoidance training" scenario in the transition period to the "rehabilitation corridor gait training" scenario in the preparation period for independent walking, and updates all force and tactile control parameters.

[0078] This invention also provides a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction, the system being used to execute the control method described in any of the preceding claims. The system includes: a wearable device, an MR display device, and a main control device.

[0079] The wearable device includes a one-piece molded flexible carrier, which has a layered structure and comprises, from the inside out: The inner skin-friendly layer is made of a flexible fabric with antibacterial and conductive properties, designed for direct contact with the skin and to ensure wearing comfort.

[0080] The middle sensing layer integrates a multimodal data acquisition module. Specifically, this module includes: a pressure sensing array consisting of multiple flexible pressure sensors distributed at key bony prominences such as the hip joint and the greater trochanter of the femur, configured to monitor body pressure distribution with high precision; and an inertial measurement unit (IMU) fixed to the outer thigh, configured to capture the angle and angular velocity of the hip joint in three-dimensional space at high frequency.

[0081] The outer feedback layer integrates a forceful tactile feedback module. Specifically, this module includes: a shape memory alloy (SMA) filament actuator arranged along the femoral direction, configured to generate adjustable linear tension via electronic control to simulate motion resistance or provide assistive force; and a pneumatic tactile feedback component consisting of multiple micro-airbags distributed in the hip and thigh, configured to generate pulsed tactile stimulation through rapid adjustment of air pressure.

[0082] The wearable device features an adjustable wrap-around design, making it easy to put on and take off and adapt to patients of different body types, and is equipped with a wireless charging unit.

[0083] The MR display device is a mixed reality head-mounted display, configured to receive graphics rendering instructions from the main control device and present the patient with a three-dimensional virtual training scene and real-time biomechanical visualization information adapted to the rehabilitation stage.

[0084] The main control device is communicatively connected to the wearable device and the MR display device, and is configured to run a computer program to execute the above-mentioned control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction.

[0085] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction, characterized in that, include: Acquire the current patient's multimodal data and perform data preprocessing on the multimodal data; Feature extraction is performed on the multimodal data after data preprocessing, and the obtained features are fused to construct a multimodal rehabilitation state feature set. A coupled feature set is then formed through feature coupling. An adaptive neural network model is constructed. The adaptive neural network model takes a multimodal feature set and a coupling feature set as inputs and outputs the coupling step value of the rehabilitation stage and the cumulative coupling comprehensive rehabilitation level. A local dynamic transition threshold is constructed based on the coupling step value of the rehabilitation stage, and a global dynamic transition threshold is constructed based on the cumulative coupling comprehensive rehabilitation level. The model parameters are optimized by using a loss function, the local dynamic transition threshold and the global dynamic transition threshold are dynamically updated, trigger conditions are set, and it is determined whether to transition from the current recovery stage to the next stage based on the trigger conditions. Several rehabilitation training scenarios are constructed based on different rehabilitation stages, and the adaptive switching of rehabilitation training scenarios is automatically triggered based on the updated local dynamic transition threshold and global dynamic transition threshold.

2. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 1, characterized in that, The adaptive neural network model includes an input layer, a multi-stage dynamic feature weight branch layer, a coupled feature interaction layer, a cumulative memory layer, a threshold learning layer, and an output layer. The input layer is used to receive a set of multimodal feature data that can currently be collected from the patient; The coupling feature interaction layer is used to receive a standardized set of coupling features. Through a multi-head attention mechanism and a fully connected layer, the coupling feature interaction strength is quantified and converted into coupling effect coefficients. The cumulative memory fusion layer is used to balance the contributions of historical memory and the current state through a learnable historical stage contribution decay coefficient. The threshold parameter learning layer is used to dynamically adjust the transition threshold by learning the threshold parameter. The output layer substitutes the calculated rehabilitation stage coupling step value, local dynamic transition threshold, cumulative coupling comprehensive rehabilitation level, and global dynamic transition threshold into the triggering conditions. When all conditions are met, it outputs 1 to trigger the transition; otherwise, it outputs 0.

3. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 2, characterized in that, The triggering conditions include: First condition: Data collection for the next stage of characteristics can be carried out based on the current patient's condition; Second condition: In the formula, This represents the coupled step value during the recovery phase. This is the threshold for local dynamic transitions. To achieve a comprehensive rehabilitation level over a long period, This is the global dynamic transition threshold; When both the first and second conditions are met, the patient can advance from the current rehabilitation stage to the next stage.

4. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 1, characterized in that, The several rehabilitation training scenarios constructed based on different rehabilitation stages include: The system obtains the current patient's rehabilitation stage and sets preset conditions for each stage. Based on the current rehabilitation stage and the corresponding preset conditions, it sends control signals to the MR interactive unit for virtual display.

5. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 1, characterized in that, The multimodal data includes basic physiological data, skeletal structure and morphology data, biomechanical data, functional and motor data, and feedback data.

6. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 1, characterized in that, The types of rehabilitation stages include the bed rest period, the transition period from bed to walk, and the preparation period for independent walking. Obtain the start time node of the patient's surgery and the current target time node. Based on the start time node and the target time node, obtain the current postoperative time and output the current rehabilitation stage type based on the postoperative time.

7. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 4, characterized in that, It also includes projecting a three-dimensional virtual skeleton model of the affected lower limb if the current rehabilitation stage is a bed rest period, setting a safe activity area, and displaying it with a different color than the three-dimensional virtual skeleton model; Set alarm thresholds and safety thresholds, and issue different colored prompts and control signals based on the maximum hip joint range of motion, alarm thresholds, and safety thresholds; When a control signal for the first training is received, a virtual trajectory is projected to move at a set speed, and the current patient's angular fluctuation rate is collected in real time. A first angular fluctuation rate threshold is set. When the angular fluctuation rate is greater than the first angular fluctuation rate threshold, a control signal is issued to stop the projection of the virtual trajectory and to activate the airbag feedback component.

8. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 7, characterized in that, It also includes, if the current rehabilitation stage is the transition period from bed rest, the preset conditions include a first judgment threshold and a second judgment threshold, wherein the first judgment threshold is greater than the second judgment threshold, and further includes: When a control signal for the second training is received, several virtual obstacles of N meters in height are projected, and the maximum range of motion of the patient's hip joint during training is obtained. If the maximum range of motion is less than the first judgment threshold, the height of the virtual obstacles is adjusted by the height adjustment model and then redisplayed. If the maximum range of motion of the hip joint is less than the second judgment threshold, a signal is issued to pause the second training session and require the first training session to be performed again before the second training session can begin.

9. The control method for a full-cycle rehabilitation system for leg fractures based on mixed reality tactile interaction according to claim 8, characterized in that, It also includes, if the current rehabilitation stage is the preparatory period for independent walking, the preset conditions include a step length judgment threshold based on the maximum range of motion of the hip joint, and further include: Based on the initial step length setting, a virtual footprint that the patient needs to move to next is projected according to the current patient position. The patient's actual step length is obtained based on the maximum range of motion of the hip joint and the stepping time. A second angular volatility is set. When the angular volatility is greater than the second angular volatility, a standard demonstration action is issued for display. Generate a virtual step, set the ratio between the height of the virtual step and the maximum range of motion of the hip joint, and adjust the height of the virtual step based on the value of the maximum range of motion of the hip joint; The resistance value of the SMA filament bundle is adjusted according to the height of the virtual step.

10. A full-cycle rehabilitation system for leg fractures based on mixed reality force-tactile interaction, characterized in that, include: The wearable device includes an outer skin-friendly layer, a middle sensing layer, and an inner feedback layer arranged sequentially. The middle sensing layer contains a pressure sensor, a triaxial accelerometer, and an inertial measurement unit for data acquisition. The inner feedback layer contains an SMA filament bundle and an airbag feedback assembly. MR display equipment is used to receive control signals and display virtual images; A main control device, which is connected to the wearable device and the MR display device, is used to execute the control method of the full-cycle rehabilitation system for leg fractures based on mixed reality force-tactile interaction as described in any one of claims 1-9.

Images