A method and system for tracking the rehabilitation progress of orthopedic patients
By fusing high-frame-rate optical imaging with multimodal perception from millimeter-wave radar and combining it with a biomechanical constraint model, the problem of incomplete joint and muscle motion capture in traditional rehabilitation monitoring has been solved, enabling continuous and accurate assessment and report generation of orthopedic patients' rehabilitation progress.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHENGDU MILITARY GENERAL HOSPITAL OF PLA
- Filing Date
- 2026-01-13
- Publication Date
- 2026-05-12
AI Technical Summary
Traditional rehabilitation monitoring methods rely on physician assessments at discrete time points or inconveniently worn inertial sensors, making it difficult to continuously and accurately capture subtle daily movements (such as joint flexion and muscle activation status), resulting in incomplete assessment of rehabilitation progress.
A non-contact multimodal sensing fusion architecture is adopted, which combines high frame rate optical imaging and millimeter-wave radar signals to simultaneously collect patient limb movement data. A time-series analytical model with biomechanical constraints is introduced to achieve fine-grained quantification of joint angle changes, muscle activation intensity and motor coordination.
It enables continuous and non-intrusive capture of patients' daily rehabilitation activities, improves the accuracy of joint movement trajectory and muscle activation state estimation, and generates structured and traceable rehabilitation progress reports to support doctors in making precise interventions.
Smart Images
Figure CN121506508B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer technology, specifically relating to a method and system for tracking the rehabilitation process of orthopedic patients. Background Technology
[0002] With the increasing aging population and frequent sports injuries, orthopedic rehabilitation has become a crucial part of the clinical medical system. Modern rehabilitation medicine emphasizes individualized, dynamic, and data-driven intervention strategies, with its core being the continuous quantitative assessment of patients' joint range of motion, muscle exertion patterns, and daily functional movements. However, current mainstream rehabilitation monitoring methods still heavily rely on periodic outpatient follow-ups or video-based motion analysis systems. The former can only obtain static indicators at discrete time points and cannot reflect the true functional recovery status in a home environment; the latter is limited by fixed scenarios and equipment deployment, making it difficult to achieve all-weather, non-intrusive natural behavior capture. In addition, some wearable solutions use rigid inertial measurement units (IMUs), which, while possessing certain continuous monitoring capabilities, are bulky, uncomfortable to wear, and susceptible to skin slippage, leading to distortion or loss of key biomechanical signals such as micro-joint flexion and extension and muscle group synergistic activation, seriously affecting the accuracy and timeliness of rehabilitation progress assessment.
[0003] The precise rehabilitation tracking of orthopedic patients urgently requires a sensing mechanism that can seamlessly conform to the human body and highly sensitively perceive subtle physiological movements. This sensing mechanism should be able to stably capture low-amplitude, high-frequency signals such as 0.5° slight flexion of the knee joint, changes in ankle dorsiflexion torque, or the timing of local electromyographic activation, while maintaining long-term wearing comfort and signal reliability during daily activities. Flexible electronics technology, with its skin-like mechanical properties and high extensibility, offers new possibilities for constructing such biomechanical sensing interfaces; however, how to effectively couple them with clinical assessment standards in rehabilitation medicine still lacks a systematic solution.
[0004] Rehabilitation monitoring systems, whether based on optical motion capture, pressure insoles, or traditional IMUs, generally suffer from problems such as limited sensory dimensions, insufficient spatial resolution, and poor user compliance. Especially in home or community rehabilitation settings, patients' movements are small in range, low in frequency, and unstructured. Existing equipment struggles to distinguish between therapeutic micro-movements and random limb swaying, and cannot simultaneously analyze multi-joint coupled movements and muscle responses. This leads to rehabilitation physicians being unable to promptly identify compensatory movements, motor inhibition, or early signs of functional decline, resulting in missed intervention windows. Summary of the Invention
[0005] This invention provides a method and system for tracking the rehabilitation process of orthopedic patients. It aims to address the technical problem that traditional rehabilitation monitoring relies on discrete time-point physician assessments or inconveniently worn inertial sensors, making it difficult to continuously and accurately capture subtle daily movements (such as joint flexion and muscle activation status), leading to incomplete assessments of rehabilitation progress. This invention constructs a non-contact multimodal perception fusion architecture, combining high-frame-rate optical imaging and millimeter-wave radar signals to collect limb movements of patients in natural life scenarios around the clock and without physical contact. It also introduces a biomechanically constrained temporal motion analysis model to achieve fine-grained quantification of joint angle changes, muscle activation intensity, and motor coordination, thereby generating a dynamic, continuous, and traceable digital profile of the rehabilitation process.
[0006] This invention provides a method for tracking the rehabilitation process of orthopedic patients, comprising:
[0007] A high frame rate optical imaging unit and a millimeter-wave radar sensing unit are deployed in the patient's home environment to simultaneously acquire optical image sequences and millimeter-wave echo signals of the patient in a free-moving state.
[0008] The optical image sequence is used to perform three-dimensional reconstruction of human key points to obtain the spatial coordinate temporal trajectory of six major joints: shoulder, elbow, wrist, hip, knee and ankle.
[0009] Micro-Doppler feature extraction is performed on the millimeter-wave echo signal to analyze the micro-vibration spectrum of muscle tissue during movement, and then the time function of muscle activation state is derived.
[0010] The temporal trajectory of joint spatial coordinates and the time function of muscle activation state are input into a preset biomechanical constraint temporal analysis model. This biomechanical constraint temporal analysis model is based on the human kinematic chain structure and muscle-skeleton coupling dynamic equation. It performs physical consistency verification and noise suppression on the original perception data and outputs standardized joint flexion and extension angle sequences, muscle activation intensity sequences and motion smoothness indicators.
[0011] Based on standardized sequences and indicators, a multidimensional rehabilitation process trajectory map is constructed with time as the horizontal axis and functional recovery as the vertical axis. According to the clinical rehabilitation staging standards, the current rehabilitation stage is automatically divided, and a structured assessment report including functional improvement rate, symmetry deviation, and compensatory behavior identification is generated.
[0012] In one embodiment of the present invention, the high frame rate optical imaging unit consists of at least two infrared supplementary cameras with a frame rate of not less than 120 frames per second and a resolution of not less than 1.28 million pixels. The installation height is between 1.5 meters and 2 meters above the ground, with a horizontal angle of 60 degrees to 90 degrees, ensuring full-body vision coverage of the patient's daily activity area. The millimeter-wave radar sensing unit operates in the 77 GHz frequency band, with a transmission power of not more than 10 milliwatts. It adopts a frequency-modulated continuous wave system, achieving a distance resolution of 5 centimeters and a speed resolution of 0.1 meters per second. The installation position is coplanar or adjacent to the optical imaging unit to ensure spatiotemporal alignment.
[0013] As one embodiment of the present invention, the three-dimensional reconstruction of human body key points specifically includes: segmenting the human body in each frame of optical image and extracting the foreground human body mask; using a convolutional neural network to perform heat map regression on the pixels within the mask to locate two-dimensional key points; mapping the two-dimensional key points to three-dimensional coordinates in the world coordinate system by triangulation and combining binocular or multi-view parallax information; applying Kalman filtering to the three-dimensional coordinate sequence to eliminate jump noise caused by occlusion or sudden changes in illumination, forming a smooth joint trajectory.
[0014] As one embodiment of the present invention, the micro-Doppler feature extraction specifically includes: performing a range-Doppler transform on the original millimeter-wave echo signal to generate a range-velocity matrix; extracting the velocity spectrum of the corresponding region within a preset distance threshold between the human torso and limbs; performing a short-time Fourier transform on the velocity spectrum to obtain a time-frequency distribution; identifying low-frequency components with a frequency shift of less than 5 Hz in the distribution, the low-frequency components corresponding to the micro-vibrations generated by muscle fiber contraction; and calculating the energy integral value of the low-frequency component as a quantitative indicator of muscle activation intensity.
[0015] As one embodiment of the present invention, the biomechanically constrained temporal analytical model is a recurrent neural network embedded in the topology of the human kinematic chain. Its hidden state update equation explicitly introduces coupling constraint terms between the rate of change of joint angles and the motion of adjacent joints, as well as linear relationship constraint terms between muscle activation intensity and joint torque. The loss function of this biomechanically constrained temporal analytical model consists of three parts: the first part is the joint trajectory prediction error, measured by mean square error; the second part is the muscle-joint dynamic consistency error, defined as the Euclidean distance between the predicted joint torque and the torque derived from the muscle activation intensity; the third part is the motion smoothness regularization term, which penalizes the second derivative of the joint angle through integration. The model training employs a backpropagation algorithm with physical constraints, performing end-to-end optimization on a labeled dataset containing healthy individuals and patients at different rehabilitation stages.
[0016] As one embodiment of the present invention, the multidimensional rehabilitation process trajectory map includes three core dimensions: the first dimension is the range of motion of the joint, represented by the average daily maximum flexion and extension angle; the second dimension is the motion symmetry, defined as the dynamic time regularization distance of the angle trajectory of the same joint on the affected side and the healthy side under the same task; the third dimension is the compensation index, which calculates the proportion of time exceeding the baseline threshold by detecting abnormal activation patterns of non-target joints when performing a specified action; the structured assessment report is automatically generated at three time granularities: daily, weekly, and monthly, and supports longitudinal comparison with historical assessment results.
[0017] This invention provides a tracking system for the rehabilitation process of orthopedic patients, comprising:
[0018] The multimodal sensing layer is used to deploy high frame rate optical imaging units and millimeter-wave radar sensing units to simultaneously acquire optical image sequences and millimeter-wave echo signals.
[0019] The data preprocessing layer is used for 3D reconstruction of human key points in optical image sequences and micro-Doppler feature extraction of millimeter-wave echo signals.
[0020] The biomechanical analysis layer is used to input the reconstructed key point trajectories and extracted muscle activation signals into the biomechanical constraint temporal analysis model, and output a standardized functional recovery sequence.
[0021] The rehabilitation assessment layer is used to construct a multidimensional rehabilitation process trajectory map based on standardized sequences, divide rehabilitation stages, and generate structured assessment reports.
[0022] In one embodiment of the present invention, the optical imaging unit and the millimeter-wave radar sensing unit in the multimodal perception layer are connected to a local edge computing node via gigabit Ethernet. The local edge computing node has a built-in time synchronization module and uses a precise time protocol to achieve microsecond-level hardware-level timestamp alignment. The data preprocessing layer runs on the graphics processor of the edge computing node and adopts a pipelined parallel architecture to ensure that the end-to-end latency from data acquisition to feature output does not exceed 200 milliseconds.
[0023] In one embodiment of the present invention, the rehabilitation assessment layer is connected to the hospital's electronic medical record system through a secure encrypted channel to automatically obtain the patient's basic diagnostic information, surgical records and initial functional scores, and pushes the generated structured assessment report to the attending physician's workstation; at the same time, the system is equipped with a patient interaction terminal to display the rehabilitation performance of the day in the form of visual charts and provide action correction prompts based on the current status.
[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0025] 1. This invention abandons the traditional monitoring paradigm that relies on the patient's active cooperation or wearable devices, and adopts a non-contact multimodal sensing fusion architecture to achieve continuous capture of daily rehabilitation activities without the patient's awareness.
[0026] 2. By combining high frame rate optical imaging with millimeter-wave radar, high-precision joint spatial motion trajectories were obtained, and muscle micro-vibration information that cannot be detected by inertial sensors was captured, solving the problem that single-mode is easily affected by shading, lighting or clothing interference in complex home environments.
[0027] 3. By introducing a time-series analytical model that incorporates prior biomechanical knowledge, physical consistency constraints are imposed on the original sensor data, effectively suppressing sensor noise and outliers and improving the accuracy of joint angle and muscle activation state estimation.
[0028] 4. The system can output multi-dimensional quantitative indicators, including joint range of motion, motion symmetry, and recognition of compensatory behaviors, and automatically generate structured and traceable rehabilitation progress reports, enabling doctors to make precise interventions based on continuous and objective data, which significantly improves the comprehensiveness, timeliness, and individualization of rehabilitation assessment. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the overall technical solution architecture of a tracking method and system for the rehabilitation process of orthopedic patients proposed in this invention;
[0030] Figure 2 This is a schematic diagram of the core principle framework of the biomechanical constraint temporal analytical model in this invention;
[0031] Figure 3 This is a flowchart illustrating the data acquisition and spatiotemporal alignment logic of the multimodal perception layer in this invention.
[0032] Figure 4 This is a flowchart illustrating the logical process of 3D reconstruction of human key points and micro-Doppler feature extraction in the data preprocessing layer of this invention.
[0033] Figure 5 This is a flowchart illustrating the logical process framework for constructing a multidimensional rehabilitation process trajectory and generating a structured report in the rehabilitation assessment layer of this invention.
[0034] Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the multimodal perception layer, data preprocessing layer, biomechanical analysis layer and rehabilitation assessment layer in this invention. Detailed Implementation
[0035] Please refer to the attached document. Figures 1 to 6This invention provides a method and system for tracking the rehabilitation process of orthopedic patients. It aims to solve the technical problem that traditional rehabilitation monitoring relies on physician assessments at discrete time points or inconveniently worn inertial sensors, making it difficult to continuously and accurately capture subtle daily movements (such as joint flexion and muscle activation), leading to incomplete assessment of rehabilitation progress. This embodiment details the specific execution flow of the method and discloses the necessary system architecture supporting its operation.
[0036] The method includes the following steps:
[0037] S1, a high frame rate optical imaging unit and millimeter-wave radar sensing unit deployed in the patient's home environment, simultaneously acquires optical image sequences and millimeter-wave echo signals of the patient in a free-moving state.
[0038] S2, Perform three-dimensional reconstruction of human key points on the optical image sequence to obtain the spatial coordinate temporal trajectory of six major joints including shoulder, elbow, wrist, hip, knee and ankle;
[0039] S3, perform micro-Doppler feature extraction on the millimeter-wave echo signal, analyze the micro-vibration spectrum of muscle tissue during movement, and then derive the time function of muscle activation state;
[0040] S4, input the temporal trajectory of joint spatial coordinates and the time function of muscle activation state into the preset biomechanical constraint temporal analysis model. This biomechanical constraint temporal analysis model is based on the human kinematic chain structure and muscle-skeleton coupling dynamic equation. It performs physical consistency verification and noise suppression on the original perception data and outputs standardized joint flexion and extension angle sequence, muscle activation intensity sequence and motion smoothness index.
[0041] S5, based on standardized sequences and indicators, constructs a multidimensional rehabilitation process trajectory map with time as the horizontal axis and functional recovery dimension as the vertical axis. According to the clinical rehabilitation staging standards, it automatically divides the current rehabilitation stage and generates a structured assessment report that includes functional improvement rate, symmetry deviation, and compensatory behavior identification.
[0042] In step S1, the high frame rate optical imaging unit consists of two infrared supplementary cameras with a frame rate of 120 frames per second and a resolution of 1.28 million pixels. It is installed 1.6 meters above the ground at a horizontal angle of 75 degrees to ensure full-body coverage of the patient's daily activity area. The millimeter-wave radar sensing unit operates in the 77 GHz band with a transmit power of 8 milliwatts, employing a frequency-modulated continuous wave (FMCV) system. It has a distance resolution of 5 centimeters and a velocity resolution of 0.1 meters per second. The unit is installed coplanar with the optical imaging unit, and both are connected to the local edge computing node via gigabit Ethernet. This edge computing node has a built-in time synchronization module that uses a precise time protocol to achieve microsecond-level hardware-level timestamp alignment, ensuring strict time synchronization between each frame of optical image and the corresponding millimeter-wave echo signal. During acquisition, the optical imaging unit continuously outputs the raw image stream, and the millimeter-wave radar sensing unit continuously outputs the intermediate frequency echo signal stream. Both are written to the circular buffer of the edge computing node at a fixed sampling period, awaiting subsequent processing.
[0043] In step S2, the process of 3D reconstruction of human key points in the optical image sequence includes several sub-steps. First, semantic segmentation is performed on each frame of the optical image, and a pre-trained fully convolutional network model is used to extract the foreground human mask and remove background interference. Second, the mask region is input into a heatmap regression network, which is a convolutional neural network with a stacked hourglass structure. The network outputs a heatmap of the probability distribution of each key point on the image plane, and the peak position of the heatmap is taken as the coordinates of the 2D key point. All 12 key points (symmetrically arranged on the left and right sides) of the six joints are located in this way. Subsequently, using the principle of binocular vision geometry, combined with the intrinsic and extrinsic parameter matrices of the two cameras, triangulation is used to map each pair of corresponding 2D key points to 3D coordinates in the world coordinate system. To improve reconstruction stability, a Kalman filter is applied to the 3D coordinate sequence of consecutive frames. The state vector includes joint positions and their first derivatives (velocities). The observation model is a linear constant-velocity model. The process noise covariance and observation noise covariance are set according to the actual motion characteristics, thereby effectively eliminating coordinate jumps caused by clothing obstruction, rapid movement, or sudden changes in lighting, forming a smooth and continuous temporal trajectory of joint spatial coordinates. This temporal trajectory of joint spatial coordinates is output at a fixed frequency (e.g., 30 Hz), and each time point contains the 3D coordinate values of 12 joints.
[0044] In step S3, the process of micro-Doppler feature extraction of the millimeter-wave echo signal also includes several sub-steps. The original echo signal is first converted from analog to digital to form a discrete-time series. A range-Doppler transform is then performed on this discrete-time series, that is, a fast Fourier transform is performed on the sampling points in each period to obtain the range dimension, and then a fast Fourier transform is performed on the continuous results under the same range threshold to obtain the velocity dimension, finally generating a two-dimensional range-velocity matrix.
[0045] Based on human anatomy, typical distance ranges for the trunk and limbs within the radar field of view are predefined. For example, the distance threshold for upper limb activity is 0.5 to 1.5 meters, and for lower limbs, it is 1.0 to 2 meters. Within these distance thresholds, the corresponding velocity spectrum is extracted. A short-time Fourier transform is performed on the velocity spectrum along the time axis with a window length of 250 milliseconds and an overlap rate of 50%, resulting in a time-frequency distribution map. In this time-frequency distribution map, low-frequency components with an absolute frequency shift of less than 5 Hz are identified. These low-frequency components originate from the micrometer-level vibrations generated by muscle fiber contraction and relaxation, exhibiting minimal Doppler frequency shift. The time integral value of the signal energy within this low-frequency band is calculated as a quantitative indicator of muscle activation intensity. This quantitative indicator is output segmented by limb, such as the left upper arm and right thigh, forming a time function of muscle activation state aligned with the joint trajectory time. To enhance robustness, a moving average filter is applied to the energy integral value, and a dynamic threshold is set to distinguish between resting and active states.
[0046] In step S4, the temporal trajectory of joint spatial coordinates and the time function of muscle activation state are input into a preset biomechanical constraint temporal analysis model. This biomechanical constraint temporal analysis model is a recurrent neural network embedded in the topology of the human kinematic chain, and its basic unit is a gated recurrent unit. The input layer of the model receives the three-dimensional coordinate sequence of 12 joints (after normalization) and the muscle activation intensity sequence of the corresponding limb segment. During the hidden state update process, two physical constraints are explicitly introduced: the first is the coupling constraint between the rate of change of joint angle and the motion of adjacent joints, which reflects the transmission relationship between the motion of proximal joints and the pose of distal joints in the kinematic chain; the second is the linear relationship constraint between muscle activation intensity and joint torque, which is based on a simplified human musculoskeletal model and assumes that the joint torque and the activation intensity of the main antagonistic muscle groups have a linear combination relationship. The output layer of the model predicts the standardized joint flexion and extension angle sequence, the corrected muscle activation intensity sequence, and the motion smoothness index, respectively. The motion smoothness index is defined as the reciprocal of the standard deviation of the first derivative of the joint angle time series, reflecting the smoothness of the motion.
[0047] The loss function of this biomechanically constrained temporal analytical model consists of three parts. The first part is the joint trajectory prediction error. The mean square error is used to measure the difference between the predicted joint coordinates and the input coordinates, expressed as:
[0048] ;
[0049] in, The input is the three-dimensional coordinates of the joint. The coordinates for model reconstruction For sequence length, For time step indexing.
[0050] Part Two: Muscle-Joint Dynamics Consistency Error Defined as predicting joint torque With the intensity of muscle activation input , is represented as:
[0051] ;
[0052] in, , This is the pre-calibrated muscle torque mapping matrix.
[0053] The third part is the motion smoothness regularization term. Regarding joint angles The integral penalty applied to the second derivative is expressed as:
[0054] ;
[0055] for The joint angle at any moment, for The joint angle at any given moment.
[0056] Total loss function ,in , , The weights are set to 1, 0.5, and 0.1, respectively. The model is trained end-to-end on a labeled dataset containing healthy volunteers and patients at different stages of postoperative rehabilitation. A backpropagation algorithm with physical constraints is used to ensure that the learned mapping conforms to the laws of human biomechanics. After training, the model is deployed on the graphics processor of an edge computing node, using a pipelined parallel architecture, with an end-to-end latency of no more than 200 milliseconds from data input to standardized sequence output.
[0057] In step S5, a multidimensional rehabilitation progress trajectory map is constructed based on the standardized sequence and indicators output in step S4. This multidimensional rehabilitation progress trajectory map contains three core dimensions. The first dimension is the range of motion of the joints. For each major joint (such as the knee and shoulder joints), the maximum daily flexion and extension angles are calculated and smoothed using a seven-day moving average to form a range of motion curve that evolves over time. The second dimension is the symmetry of movement. A specific rehabilitation task is selected (such as standing leg raises or forward arm raises), and the angle trajectories of the corresponding joints on the affected and healthy sides are extracted when performing the task. The dynamic time warp distance between the two is calculated, and the smaller the dynamic time warp distance, the better the symmetry. The third dimension is the compensation index. By analyzing the activation patterns of non-target joints when performing the specified movements, for example, in training that requires only knee joint movement, if the hip joint angle change is detected to exceed the baseline threshold (derived from data from healthy individuals) and the duration exceeds 20% of the total movement time, it is determined to be a compensatory behavior. The compensation index is the percentage of compensatory behaviors that occur per unit time.
[0058] Based on internationally accepted orthopedic rehabilitation staging standards (such as the acute phase, subacute phase, functional recovery phase, and intensive phase of orthopedic rehabilitation), the system automatically compares current multidimensional indicators with the thresholds of each phase to classify the patient's current rehabilitation stage. Structured assessment reports are automatically generated at daily, weekly, and monthly granularities. Content includes historical trends in the range of motion of each joint, quantitative values of symmetry deviations, specific descriptions of compensatory behaviors (such as "excessive forward tilting of the left hip during right knee flexion training"), the rate of functional improvement (expressed as weekly increases in range of motion), and longitudinal comparison charts with historical assessment results. The report is pushed to the hospital's electronic medical record system via a secure, encrypted channel and simultaneously displayed on the patient's interactive terminal's visual interface. The interface displays the day's performance in the form of line graphs, heat maps, etc., and provides text prompts such as "Please try to reduce lumbar swaying and focus on independent knee joint movement" when abnormal compensation or insufficient activity is detected.
[0059] The system comprises a multimodal perception layer, a data preprocessing layer, a biomechanical analysis layer, and a rehabilitation assessment layer. The multimodal perception layer, composed of the aforementioned high-frame-rate optical imaging unit and millimeter-wave radar sensing unit, is responsible for raw data acquisition and spatiotemporal alignment. The data preprocessing layer, running on edge computing nodes, includes modules for human body segmentation, heatmap regression, triangulation, Kalman filtering, distance-Doppler transformation, short-time Fourier transform, and energy integration, converting raw signals into joint trajectories and muscle activation functions. The biomechanical analysis layer deploys the inference engine of the aforementioned biomechanical constraint temporal analysis model, responsible for generating standardized functional recovery sequences. The rehabilitation assessment layer includes modules for trajectory construction, staging, and report generation, responsible for multidimensional indicator calculation, stage division, and structured output. Data is transferred between layers via memory sharing or message queues to ensure real-time processing and consistency. The patient interaction terminal is a tablet or smart TV application, providing a user-friendly interface, supporting video demonstrations and voice prompts, but does not participate in core data processing.
[0060] This embodiment achieves seamless, continuous, and multi-dimensional tracking of the rehabilitation process of orthopedic patients through the above-described methods and systems, overcoming the limitations of traditional methods and providing objective, dynamic, and quantifiable assessment data for clinical practice.
Claims
1. A method for tracking the rehabilitation process of orthopedic patients, characterized in that, include: A high frame rate optical imaging unit and a millimeter-wave radar sensing unit are deployed in the patient's home environment to simultaneously acquire optical image sequences and millimeter-wave echo signals of the patient in a free-moving state. The optical image sequence is used to perform three-dimensional reconstruction of human key points to obtain the spatial coordinate temporal trajectory of six major joints: shoulder, elbow, wrist, hip, knee and ankle. Micro-Doppler feature extraction is performed on the millimeter-wave echo signal to analyze the micro-vibration spectrum of muscle tissue during movement, thereby deriving the time function of muscle activation state, including: Perform range-Doppler transformation on the raw millimeter-wave echo signal to generate a range-velocity matrix; Within a preset distance threshold between the human torso and limbs, the velocity spectrum of the corresponding region is extracted; A short-time Fourier transform is performed on the velocity spectrum to obtain the time-frequency distribution; Identify low-frequency components in the distribution with a frequency shift of less than 5 Hz, which correspond to the micro-vibrations generated by muscle fiber contraction. Calculate the energy integral value of this low-frequency component as a quantitative indicator of muscle activation intensity; The temporal trajectory of the joint spatial coordinates and the time function of the muscle activation state are input into a preset biomechanical constraint temporal analysis model. This model, based on the human kinematic chain structure and muscle-skeleton coupling dynamic equations, performs physical consistency verification and noise suppression on the original perceived data, and outputs standardized joint flexion-extension angle sequences, muscle activation intensity sequences, and motion smoothness indices, including: The normalized joint 3D coordinate sequence and muscle activation intensity sequence are input into a recurrent neural network embedded with the human kinematic chain topology. During the hidden state update process, explicit constraints are introduced to couple the joint angle change rate with the motion of adjacent joints, as well as constraints on the linear relationship between muscle activation intensity and joint torque. End-to-end optimization is performed using a composite loss function that includes joint trajectory prediction error, muscle-joint dynamic consistency error, and motion smoothness regularization term, resulting in a standardized functional recovery sequence. The joint trajectory prediction error in the composite loss function is measured by mean square error to measure the difference between the predicted joint coordinates and the input coordinates; the muscle-joint dynamics consistency error is defined as the Euclidean distance between the predicted joint torque and the torque derived from the muscle activation intensity; the motion smoothness regularization term applies an integral penalty to the second derivative of the joint angle. Based on the standardized sequences and indicators, a multidimensional rehabilitation process trajectory map is constructed with time as the horizontal axis and functional recovery dimension as the vertical axis. According to the clinical rehabilitation staging standards, the current rehabilitation stage is automatically divided, and a structured assessment report including functional improvement rate, symmetry deviation, and compensatory behavior identification is generated.
2. The method for tracking the rehabilitation progress of an orthopedic patient according to claim 1, wherein, The optical image sequence is used for three-dimensional reconstruction of human key points to obtain the spatial coordinate temporal trajectory of six major joints: shoulder, elbow, wrist, hip, knee, and ankle, including: Perform human body segmentation on each frame of optical image and extract the foreground human body mask; Using a convolutional neural network to perform heatmap regression on pixels within the mask, two-dimensional key points are located. By using triangulation and combining binocular or multi-view parallax information, two-dimensional key points are mapped to three-dimensional coordinates in the world coordinate system. Kalman filtering is applied to the three-dimensional coordinate sequence to eliminate jump noise caused by occlusion or sudden changes in lighting, resulting in a smooth joint trajectory.
3. The method for tracking the rehabilitation progress of an orthopedic patient according to claim 2, wherein, Construct a multidimensional rehabilitation progress trajectory map with time as the horizontal axis and functional recovery as the vertical axis, including: The range of motion of the joint is represented by the average daily maximum flexion and extension angle. Motion symmetry is represented by the dynamic time-normalized distance of the angular trajectories of the same joints on the affected and healthy sides under the same task. By detecting abnormal activation patterns of non-target joints when performing a specified action, the percentage of time that exceeds the baseline threshold is calculated as a compensation index.
4. The method for tracking the rehabilitation progress of an orthopedic patient according to claim 3, wherein, The structured evaluation report is automatically generated at three time granularities: daily, weekly, and monthly, and supports longitudinal comparison with historical evaluation results.
5. The method for tracking the rehabilitation progress of an orthopedic patient according to claim 4, wherein, The high frame rate optical imaging unit consists of at least two infrared supplementary cameras; the millimeter-wave radar sensing unit adopts a frequency-modulated continuous wave system.
6. The method for tracking the rehabilitation process of orthopedic patients according to claim 5, characterized in that, The high frame rate optical imaging unit and the millimeter-wave radar sensing unit are connected to the local edge computing node via gigabit Ethernet. The node has a built-in time synchronization module and uses a precise time protocol to achieve microsecond-level hardware-level timestamp alignment.
7. A tracking system for the rehabilitation process of orthopedic patients, characterized in that, The system for tracking the rehabilitation process of orthopedic patients using the method described in any one of claims 1 to 6 includes: The multimodal sensing layer is used to deploy high frame rate optical imaging units and millimeter-wave radar sensing units to simultaneously acquire optical image sequences and millimeter-wave echo signals. The data preprocessing layer is used for 3D reconstruction of human key points in optical image sequences and micro-Doppler feature extraction of millimeter-wave echo signals. The biomechanical analysis layer is used to input the reconstructed key point trajectories and extracted muscle activation signals into the biomechanical constraint temporal analysis model, and output a standardized functional recovery sequence. The rehabilitation assessment layer is used to construct a multidimensional rehabilitation process trajectory map based on the standardized sequence, divide the rehabilitation stages, and generate a structured assessment report.