Ankle joint muscle-bone dynamic coupling digital twin modeling method, system, device and storage medium

Abstract

The present application belongs to the technical field of biomedical engineering, sports biomechanics, medical image processing and artificial intelligence, and particularly relates to a kind of ankle muscle-bone dynamic coupling digital twin modeling method, system, equipment and storage medium.The method synchronously acquires three-dimensional motion capture, foot / ground reaction force, musculoskeletal ultrasonic elastography and magnetic resonance imaging data, through gait phase registration, individualized three-dimensional reconstruction and soft tissue elasticity mapping, constructs muscle-bone dynamic coupling model, coupling graph and coupling tensor, and trains deep learning agent model for prediction, virtual disturbance and explainability analysis, for evaluating ankle stability and intervention effect.

Description

Technical Field

[0001] This invention belongs to the fields of biomedical engineering, sports biomechanics, medical image processing and artificial intelligence, and particularly relates to a method, system, device and storage medium for dynamic coupling digital twin modeling of ankle joint tendons and bones. Background Technology

[0002] The ankle joint is a crucial joint structure in the human lower limb for weight-bearing, walking propulsion, and postural stability. Its stability depends not only on the spatial alignment of bony structures such as the tibia, fibula, talus, and calcaneus, but also on the constraint and regulation of bony movements by soft tissues such as the anterior talofibular ligament, calcaneofibular ligament, deltoid ligament, and peroneus longus and brevis tendons. Clinically, chronic ankle instability, functional ankle instability, post-ankle sprain sequelae, and sports injury rehabilitation assessments often manifest as a combination of multiple factors, including abnormal bony alignment, ligament laxity, delayed tendon response, plantar pressure path deviation, and decreased gait stability. Traditional imaging examinations can display the state of skeletal and soft tissue structures, gait analysis can reflect external kinematic and dynamic changes during movement, and ultrasound elastography can provide local soft tissue elasticity parameters. However, these test results are mostly presented as independent data, making it difficult to directly reveal the dynamic coupling relationship between ankle tendons and bones as the gait phase changes.

[0003] In the prior art, there are already individualized simulation modeling schemes for foot and ankle structures. For example, the international publication document WO2019072875A1 discloses a foot and / or ankle simulation method, which generates individualized foot and ankle models based on relevant subject information and uses a combination of multibody models and finite element models to simulate the foot and ankle under static or dynamic conditions. The model can include tissues such as bone, muscle, tendon, ligament, and cartilage, and can obtain information such as stress and strain under load. This scheme helps to improve the completeness of foot and ankle models in terms of structural simulation and load analysis, but its focus is still on establishing a hybrid simulation model based on imaging and mechanical conditions. It does not address the design for gait phase synchronization registration between three-dimensional motion capture, plantar or ground reaction force, musculoskeletal ultrasound elastography, and magnetic resonance imaging, nor does it dynamically map the gait phase-related soft tissue elastic field as a material parameter into the ankle joint model, and it does not form a tensor quantification characterization and interpretable analysis process for quantifying the contribution of tendon-bone coupling.

[0004] Other existing technologies focus on gait analysis based on sensor data and deep learning. For example, US Patent Publication No. US20190150793A1 discloses a method and system for human gait analysis. This method uses external measurement systems such as pressure pads and motion capture systems to annotate gait features, and employs a deep learning system including convolutional layers, fully connected layers, and readout layers to learn the relationship between sensor data and parameters such as stride length and gait features. While this approach can improve the automation of gait feature recognition or estimation, it primarily deals with the mapping relationship between gait sensor data and gait features. It struggles to reflect the interactions between ligaments, tendons, articular cartilage, and bony structures within the ankle joint, and it also fails to provide the mechanical contribution of soft tissue elasticity changes to bony stability.

[0005] Furthermore, musculoskeletal finite element modeling and medical image 3D reconstruction technologies have been applied to scenarios such as joint contact pressure, bone stress, implant matching, and surgical planning; machine learning and deep learning methods have also been used for gait classification, motion parameter prediction, and disease state identification. However, finite element models typically involve large computational costs, making rapid iteration difficult under different gait phases, intervention parameters, and individual conditions; while deep learning models possess strong nonlinear fitting capabilities, without mechanical constraints and interpretability analysis, they can easily become black-box models relying solely on statistical correlations. For diseases such as chronic ankle instability, clinicians are more concerned with the mechanisms underlying which changes in the mechanical properties of ligaments or tendons lead to decreased bony stability, whether a particular rehabilitation training or orthotic constraint can improve abnormal talar rotation or varus peak, soft tissue laxity, and how neuromuscular delay and joint contact pressure transfer interact. Existing methods such as simple image modeling, simple gait recognition, or simple finite element simulation are insufficient to fully meet these needs.

[0006] Therefore, there is an urgent need to establish a digital twin modeling technology for the ankle joint that can integrate three-dimensional motion capture, plantar or ground reaction force, musculoskeletal ultrasound elastography, and magnetic resonance imaging. This technology would enable the individualized ankle joint geometry, soft tissue elastic parameters, bony motion boundary conditions, and dynamic stability indices to be expressed in a unified gait phase. Furthermore, through deep learning surrogate models, biomechanical residual constraints, generative adversarial calibration, and interpretability analysis, it would be possible to achieve rapid prediction, contribution quantification, and virtual intervention assessment of the dynamic coupling relationship between the ankle joint tendons and bones. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for dynamic coupling digital twin modeling of the ankle joint's musculoskeletal structure based on multimodal data and deep learning. By integrating 3D motion capture, plantar or ground reaction force, musculoskeletal ultrasound elastography, and magnetic resonance imaging data, a digital twin model of the ankle joint is constructed that reflects the individualized bony structure, soft tissue elastic properties, and dynamic changes in gait phase. This addresses the problems in existing ankle joint stability assessments, such as the difficulty in unifying the registration of multimodal data, the difficulty in quantifying the mechanical contribution of soft tissues, the low computational efficiency of finite element simulation, and the insufficient interpretability of deep learning models. The model output is used for non-diagnostic auxiliary assessment of the ankle joint's biomechanical state, engineering simulation, or orthotic parameter design, and is not directly used as a disease diagnosis result or treatment plan.

[0008] Firstly, in order to achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0009] A method for dynamic coupling digital twin modeling of ankle joint tendons and bones, comprising the following steps:

[0010] S1. Simultaneously acquire three-dimensional motion capture data, plantar or ground reaction force data, and ankle perimuscular ultrasound elastography data of the subject during walking, single-leg standing, or closed-chain weight-bearing movements. Separately acquire ankle joint magnetic resonance imaging data, and perform time registration of the three-dimensional motion capture data, plantar or ground reaction force data, and ankle perimuscular ultrasound elastography data using gait phase as a unified time index.

[0011] S2. Based on the magnetic resonance imaging data, establish an individualized three-dimensional geometric model of the ankle joint including the tibia, fibula, talus, calcaneus, articular cartilage, anterior talofibular ligament, calcanofibular ligament and peroneus longus and shortus tendons, and map the musculoskeletal ultrasound elastography data onto the individualized three-dimensional geometric model of the ankle joint to generate a phase-labeled soft tissue elastic field.

[0012] S3. Based on three-dimensional motion capture data, foot or ground reaction force data and soft tissue elastic field, construct a musculoskeletal dynamic coupling finite element-multibody dynamics model that includes bony movement, soft tissue deformation and joint contact relationship;

[0013] S4. Based on the aforementioned musculoskeletal dynamic coupling finite element-multibody dynamics model, construct a musculoskeletal coupling diagram and generate a musculoskeletal coupling tensor characterizing the contribution of soft tissue constraints to bony stability.

[0014] S5. Construct a deep learning proxy model that includes a temporal coding branch, a spatial coding branch, and a graph attention coupling branch, and introduce biomechanical residual constraints and generative adversarial calibration for training.

[0015] S6. Based on the trained deep learning agent model, establish an individualized digital twin model of the ankle joint, and virtually perturb the elastic modulus of ligaments, elastic modulus of tendons, neuromuscular delay, plantar reaction path or orthotic constraint parameters to obtain the change in dynamic stability index. Combine the SHAP value or gradient class activation heatmap to output key soft tissues, key bony regions and their contribution.

[0016] As a further improvement, in step S1, the gait phase includes at least the initial contact phase, weight-bearing reaction phase, mid-term support phase, end-term support phase, pre-swing phase, initial swing phase, mid-term swing phase, and end-term swing phase; the time registration includes uniformly resampling the three-dimensional motion capture sampling sequence, the plantar or ground reaction force sampling sequence, and the musculoskeletal ultrasound elastography sampling sequence to the normalized gait cycle coordinates, so that the bony motion, external load, and soft tissue elastic parameters under the same gait phase have a corresponding relationship.

[0017] As a further improvement, in step S2, the soft tissue elastic field of the phase marker is generated in the following manner: using the lateral process of the talus, the tip of the lateral malleolus, the lateral tubercle of the calcaneus, the attachment point of the anterior talofibular ligament, and the attachment point of the calcanofibular ligament as registration reference points, the elastic modulus distribution measured by musculoskeletal ultrasound elastography is interpolated to the corresponding ligament or tendon grid unit, and each grid unit is assigned a gait phase label, a spatial coordinate label, and an elastic modulus label.

[0018] As a further improvement, in step S3, the bony motion boundary conditions of the ankle joint are established based on the three-dimensional motion capture data and the plantar or ground reaction force data, and the phase-related material parameters of ligaments and tendons are established based on the phase-marked soft tissue elastic field, thus constructing a tendon-bone dynamic coupling finite element-multibody dynamics model that includes bony motion, soft tissue deformation and joint contact relationship.

[0019] As a further improvement, in step S3, the musculoskeletal dynamic coupling finite element-multibody dynamics model satisfies the following dynamic relationship:

[0020] ;

[0021] In the formula, Ankle joint angle vector; This is the ankle joint angular velocity vector; This is the angular acceleration vector of the ankle joint; This is the quality matrix; For the Coriolis and eccentricity terms matrix; This is the term related to gravity. For soft tissue strain Phase-dependent elastic modulus The jointly determined soft tissue constraint torque; For muscle active torque; This is the external reaction torque.

[0022] As a further improvement, in step S4, the musculoskeletal dynamic coupling finite element-multibody dynamics model outputs the bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index under different phase states, and the bony pose nodes, soft tissue attachment point nodes and joint contact area nodes constitute a musculoskeletal coupling diagram.

[0023] As a further improvement, in step S4, the musculoskeletal coupling tensor includes changes in bony pose, changes in soft tissue strain, soft tissue elastic modulus, changes in joint contact pressure, and spatial distance weights between nodes, which are used to characterize the constraint contribution of ligaments or tendons on the relative motion of the talus, calcaneus, tibia, or fibula in the same gait phase.

[0024] As a further improvement, in step S5, a deep learning proxy model is constructed with gait phase, bony motion boundary conditions, plantar or ground reaction force, soft tissue elastic field and muscle-bone coupling tensor as inputs, and bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index as outputs. The deep learning proxy model includes a temporal encoding branch for extracting bony rigid body motion features, a spatial encoding branch for extracting soft tissue elastic field features, and a graph attention coupling branch for fusing the interaction between bony nodes and soft tissue nodes.

[0025] As a further improvement, in step S5, the deep learning proxy model is trained using measured kinematic data, musculoskeletal ultrasound elastography data, and finite element simulation output. Biomechanical residual constraints and generative adversarial calibration are introduced during the training process to ensure that the bony pose, soft tissue stress and strain, and joint contact pressure output by the deep learning proxy model simultaneously satisfy both measured consistency and mechanical balance consistency.

[0026] As a further improvement, in step S5, the graph attention coupling branch uses bony nodes and soft tissue nodes as graph nodes, and soft tissue attachment relationships, joint contact relationships and spatial proximity relationships as graph edges, and outputs the contribution characteristics of different ligaments, tendons and joint contact areas to ankle joint stability according to attention weights.

[0027] As a further improvement, in step S5, the biomechanical residual constraints include at least joint torque balance residuals, soft tissue strain continuity residuals, joint contact non-penetration residuals, and gait cycle continuity residuals; the generative adversarial calibration includes using a deep learning proxy model as a generator and using the joint features of measured kinematic data, musculoskeletal ultrasound elastography data, and finite element simulation output as the discrimination object to train the discriminator.

[0028] As a further improvement, in step S5, the training loss function includes a supervision error term, an adversarial error term, a biomechanical residual term, and a regularization error term. The supervision error term is used to characterize the error between the predicted output and the measured data or simulation annotations. The adversarial error term is used to characterize the error in the adversarial calibration process. The biomechanical residual term is used to characterize the mechanical consistency constraint error. The regularization error term is used to constrain the model complexity. Each error term is trained by weighting it with corresponding weight coefficients.

[0029] As a further improvement, in step S6, the dynamic stability index includes at least one of the following: peak ankle inversion, peak talus external rotation, peak stress of the anterior talofibular ligament, peak stress of the calcaneofibular ligament, peak joint contact pressure, center of pressure migration, ligament-bone stress transmission efficiency, neuromechanical delay time, and dynamic stability margin.

[0030] Secondly, the present invention also provides a dynamic coupling digital twin modeling system for ankle joint tendons and bones, the system being used to implement the method, comprising:

[0031] The data acquisition and registration module is used to simultaneously acquire and register three-dimensional motion capture data, plantar or ground reaction force data, musculoskeletal ultrasound elastography data and magnetic resonance imaging data, and generate a multimodal data sequence indexed by gait phase;

[0032] The individualized model building module is used to build a three-dimensional geometric model of the ankle joint based on magnetic resonance imaging and to map musculoskeletal ultrasound elastography data into a phase-labeled soft tissue elastic field.

[0033] The dynamic coupling simulation module is used to construct a finite element-multibody dynamic model of muscle-bone dynamic coupling that includes bony movement, soft tissue deformation and joint contact relationship, and outputs bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index.

[0034] The proxy model training module is used to build and train a deep learning proxy model that includes a temporal encoding branch, a spatial encoding branch, and a graph attention coupling branch.

[0035] The adversarial calibration module is used to generate adversarial calibration for the deep learning agent model based on measured data and simulation output, and correct the model output through biomechanical residual constraints.

[0036] The interpretable analysis and virtual perturbation module is used to output key soft tissues, key bony regions and their contributions based on SHAP values, gradient-type activation heatmaps or sensitivity analysis, and to simulate the effects of rehabilitation training, orthotic constraints or changes in soft tissue parameters on the dynamic stability of the ankle joint.

[0037] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method described above.

[0038] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.

[0039] This invention uses gait phase as a unified index to synchronously register 3D motion capture data, plantar or ground reaction force data, musculoskeletal ultrasound elastography data, and magnetic resonance imaging data. This allows the ankle joint's bony motion boundary, external load, and soft tissue elastic parameters to be expressed correspondingly under the same dynamic process. By mapping ultrasound elastography data into a phase-labeled soft tissue elastic field, the individualized mechanical properties of soft tissues such as the anterior talofibular ligament, calcaneofibular ligament, and peroneus longus and brevis tendons can participate in finite element-multibody dynamics simulation, improving the model's ability to express the real musculoskeletal coupling relationship of the ankle joint. By constructing musculoskeletal coupling diagrams and musculoskeletal coupling tensors, the contribution of soft tissue constraints to bony stability can be quantitatively characterized, overcoming the limitations of traditional models that only output displacement, stress, or contact pressure. The limitations of explaining the mechanisms of action in soft tissues can be addressed by employing a deep learning proxy model that incorporates temporal encoding, spatial encoding, and graph attention coupling branches, combined with biomechanical residual constraints and generative adversarial calibration. This approach improves simulation prediction speed and generalization ability while maintaining mechanical consistency. Furthermore, by virtually perturbing ligament elastic modulus, tendon elastic modulus, neuromuscular delay, plantar reaction path, or orthotic constraint parameters, and combining this with SHAP or gradient-based activation heatmap output contribution, the identification of key soft tissues, key bony regions, and key intervention parameters in the rehabilitation process of chronic ankle instability, functional ankle instability, or ankle injury can be achieved. This provides quantifiable and interpretable technical evidence for individualized rehabilitation training, orthotic design, and surgical plan evaluation. Attached Figure Description

[0040] Figure 1 This is the overall flowchart of the ankle joint tendon-bone dynamic coupling digital twin modeling method based on multimodal data and deep learning according to the present invention.

[0041] Figure 2 This is a schematic diagram of the system structure of the present invention.

[0042] Figure 3 This is a schematic diagram illustrating the synchronization and registration of multimodal data using gait phase as a unified time index according to the present invention.

[0043] Figure 4This is a schematic diagram illustrating how the present invention establishes an individualized three-dimensional geometric model of the ankle joint based on magnetic resonance imaging and maps musculoskeletal ultrasound elastography data to form a phase-labeled soft tissue elastic field.

[0044] Figure 5 This is a schematic diagram of the structure of the ligament-bone dynamic coupling finite element-multibody dynamics model of the present invention.

[0045] Figure 6 This is a schematic diagram illustrating the construction of the tendon-bone coupling diagram and tendon-bone coupling tensor of the present invention.

[0046] Figure 7 This is a schematic diagram of the network structure of the deep learning agent model of the present invention.

[0047] Figure 8 This is a flowchart of the biomechanical residual constraint and generative adversarial calibration training process of the present invention.

[0048] Figure 9 This is a schematic diagram of the interpretability analysis and virtual perturbation experiment of the present invention.

[0049] Figure 10 This is a visualization of the results of the present invention.

[0050] Figure 11 The bar chart shows the prediction error comparison between Example 1 and Comparative Examples 1 to 4.

[0051] Figure 12 The graph shows the changes in dynamic stability index under different orthodontic constraint parameters.

[0052] Figure 13 This is a graph showing the results of the ablation test. Detailed Implementation

[0053] The technical solutions in the embodiments of the present invention will be clearly and completely described below. It should be understood that the following embodiments are intended to enable those skilled in the art to implement the present invention, and are not intended to limit the scope of protection of the present invention. Without departing from the core technical concept of the present invention, the three-dimensional motion capture device, the foot / ground reaction force acquisition device, the musculoskeletal ultrasound elastography device, the magnetic resonance imaging device, the finite element solver, the multibody dynamics solver, and the deep learning framework can all be replaced with equivalent devices or software.

[0054] I. Terminology Explanation

[0055] The "tendon-bone" dynamic coupling of the ankle joint, as referred to in this invention, refers to the interaction between the spatial positional changes of bony structures such as the tibia, fibula, talus, and calcaneus during walking, single-leg standing, closed-chain weight-bearing, or simulated external torque application, and the elastic constraints, traction direction, stress-strain changes, and joint contact pressure migration of soft tissues such as the anterior talofibular ligament, calcaneofibular ligament, posterior talofibular ligament, deltoid ligament, intercalculotal ligament, peroneus longus tendon, and peroneus brevis tendon. Here, "tendon" in engineering modeling refers broadly to ligaments, tendons, joint capsules, cartilage, and related constraint tissues, and is not limited to a single tissue type.

[0056] In this invention, gait phase refers to a time index formed by normalizing a complete gait cycle using gait events such as heel strike, plantar weight-bearing, heel lift-off, toe lift-off, and the next heel strike as references. Through gait phase, three-dimensional motion capture data, plantar / ground reaction force data, musculoskeletal ultrasound elastography data, and subsequent model output data obtained at different sampling frequencies and trigger times can be mapped to a unified time reference.

[0057] The phase-marked soft tissue elastic field referred to in this invention refers to the distribution of soft tissue elastic modulus at a spatial location corresponding to the gait phase. This elastic field includes not only soft tissue material parameters, but also the spatial location, tissue type, and gait phase information corresponding to those material parameters. Unlike traditional finite element models that assign a fixed average elastic modulus to a ligament or tendon, this invention can express the spatial-temporal changes in soft tissue elastic parameters as the gait phase changes.

[0058] The musculoskeletal coupling diagram referred to in this invention is a data representation method that abstracts the anatomical structure and mechanical connections of the ankle joint into a graphical structure. This graphical structure uses bony nodes, soft tissue nodes, and joint contact nodes as graph nodes, and bony-soft tissue connections, soft tissue-soft tissue connections, soft tissue-joint contact connections, and bony-joint contact connections as graph edges, to express the anatomical connections and mechanical transmission relationships between ligaments, tendons, bones, and contact surfaces.

[0059] The musculoskeletal coupling tensor referred to in this invention is a high-order data structure used to describe the correspondence between bony nodes, soft tissue nodes, joint contact nodes, and multiple mechanical features under different time-phase states. This tensor may include elements such as changes in bony pose, changes in soft tissue strain, elastic modulus, changes in contact pressure, and distance weights, and is used to provide structured input with anatomical constraints and mechanical meaning to deep learning surrogate models.

[0060] The deep learning proxy model referred to in this invention is a rapid predictive model that uses multimodal measured data and finite element-multibody dynamics simulation data as supervision or reference, and learns the mapping relationship of ankle joint dynamic mechanical response through neural networks. After training, this proxy model can replace the computationally intensive complete simulation process and output bony pose, soft tissue stress and strain, joint contact pressure, and dynamic stability indicators in a short time.

[0061] The biomechanical residual constraint referred to in this invention refers to the mechanical consistency constraint introduced during the training of the surrogate model, including joint moment balance residuals, soft tissue strain continuity residuals, contact non-penetration residuals, and gait cycle continuity residuals. This constraint is used to prevent the surrogate model from deviating from biomechanical laws by only learning statistical correlations.

[0062] The virtual perturbation referred to in this invention refers to the controllable alteration of ligament elastic modulus, tendon elastic modulus, neuromuscular delay, plantar reaction path, or orthotic constraint parameters in a trained individualized digital twin model of the ankle joint, in order to predict the impact of the above changes on the dynamic stability and injury risk of the ankle joint.

[0063] The SHAP value, referred to in this invention, is the Shapley additive interpretation value, used to characterize the marginal contribution of input features to the model output.

[0064] II. System Structure

[0065] like Figure 2 As shown, the system of the present invention includes an input data module, a system core module, and an output result module.

[0066] The input data module receives four types of data: the first type is three-dimensional motion capture data, including three-dimensional angles, angular velocities, angular accelerations of the ankle joint, trajectories of key lower limb markers, and gait event information; the second type is plantar / ground reaction force data, including ground reaction force components, resultant forces, center of pressure trajectories, and plantar zone loads in the vertical, anterior-posterior, and medial directions; the third type is musculoskeletal ultrasound elastography data, including shear wave velocities, elastic modulus distributions, or relative elastograms of regions such as the anterior talofibular ligament, calcaneofibular ligament, peroneus longus tendon, and peroneus brevis tendon; and the fourth type is magnetic resonance imaging data, including high-resolution tomographic images or three-dimensional sequence images of the ankle joint.

[0067] The core modules of the system include a data acquisition and registration module, an individualized model construction module, a dynamic coupling simulation module, a musculoskeletal coupling characterization module, a surrogate model training module, an adversarial calibration module, and an interpretable analysis and virtual perturbation module. The data acquisition and registration module is used to perform time synchronization of multimodal data, gait phase normalization, and multi-source signal alignment; the individualized model construction module is used to perform three-dimensional geometric reconstruction of the ankle joint based on magnetic resonance imaging and to map ultrasound elastic data to ligament and tendon regions; the dynamic coupling simulation module is used to construct a finite element-multibody dynamics coupled model; the musculoskeletal coupling characterization module is used to construct musculoskeletal coupling maps and musculoskeletal coupling tensors; the surrogate model training module is used to construct and train a deep learning surrogate model; the adversarial calibration module is used to combine the generation of adversarial training and biomechanical residual constraint correction model outputs; and the interpretable analysis and virtual perturbation module is used to output key contributing factors and intervention prediction results.

[0068] The output results module includes bony posture, soft tissue stress and strain, joint contact pressure, dynamic stability indices, key contributions, and intervention prediction results. Bony posture can include the relative position, posture, rotation angle, and angular velocity of the tibia, fibula, talus, and calcaneus; soft tissue stress and strain can include the equivalent stress, principal strain, and peak region of the anterior talofibular ligament, calcanofibular ligament, posterior talofibular ligament, deltoid ligament, and peroneus longus and brevis tendons; joint contact pressure can include the contact pressure distribution at the tibiotalar joint, subtalar joint, or lateral articular surface; dynamic stability indices can include peak ankle inversion, peak talar external rotation, dynamic stability margin, and center of pressure migration; key contributions can include the SHAP value, attention weight, or heatmap of sensitive areas; intervention prediction results can include stability changes under different orthotic constraints, rehabilitation training, or soft tissue parameter adjustment schemes.

[0069] III. Structured Disclosure of Datasets and Data Usage

[0070] To enable those skilled in the art to implement this invention, the dataset structure is described below. In one embodiment, one effective gait cycle of each subject forms a sample, which can be represented as:

[0071] ;

[0072] In the formula, For the first One sample; For 3D motion capture data; This refers to the plantar / ground reaction force data; This is musculoskeletal ultrasound elastography data; For magnetic resonance imaging data or individualized geometric models derived from it; For finite element-multibody dynamics simulation outputs or actual measurement reference labels; Provides clinical and demographic information for the subjects.

[0073] 3D motion capture data may include Ankle angle, angular velocity, and key marker coordinates at each gait phase sampling point; plantar / ground reaction data may include the same... The data includes triaxial ground reaction force, pressure center coordinates, and plantar pressure at each gait phase sampling point; musculoskeletal ultrasound elastography data may include local elastic modulus matrices at multiple phases or segmented ligament / tendon elastic characteristics; magnetic resonance imaging data may be segmented and converted into three-dimensional meshes, point clouds, or voxel data; output labels may include bony pose, soft tissue stress, soft tissue strain, joint contact pressure, and dynamic stability indices.

[0074] During training, the dataset can be divided into training, validation, and test sets according to the subject dimension to avoid different gait cycles of the same subject appearing in both the training and test sets simultaneously. In one implementation, the dataset can be divided into training, validation, and test sets at a ratio of 70%, 15%, and 15%, respectively; alternatively, a 5-fold cross-validation method can be used to evaluate generalization ability. If the sample size is small, the training samples can be expanded using finite element-multibody dynamics simulation output, followed by calibration with real measurement data; if the sample size is large, both measured and simulated data can be used for joint training.

[0075] To improve model robustness, data cleaning can be performed before training. This includes removing gait cycles with a marker loss exceeding a preset proportion, removing data with significantly abnormal plantar / ground reaction force peaks, and removing data with substandard region of interest quality in the ultrasound elastogram. The input sequence is then normalized. Preferably, gait cycles with a marker loss exceeding 10% are removed; data with plantar / ground reaction force peaks deviating from the group mean by more than three standard deviations are removed; and data with unclear region of interest boundaries, less than 80% effective pixels, or a signal-to-noise ratio below a preset threshold in the ultrasound elastogram are removed. Continuous variables can be normalized using mean-standard deviation, image or grid attributes can be normalized to the range of 0 to 1, and classification or phenotypic labels can use one-hot encoding or ordered encoding.

[0076] IV. Specific Technical Approach

[0077] S1: Multimodal data acquisition and gait phase registration

[0078] like Figure 1 and Figure 3As shown, in one embodiment of the present invention, the subject completes natural walking, single-leg standing, closed-chain weight-bearing, or external torque simulation movements. A three-dimensional motion capture device records the coordinates of key lower limb markers and calculates ankle joint angles, angular velocities, and angular accelerations; a plantar / ground reaction force acquisition device records the three-dimensional ground reaction forces and the trajectory of the center of pressure; a musculoskeletal ultrasound elastography device acquires elastograms of the anterior talofibular ligament, calcaneofibular ligament, peroneus longus tendon, and peroneus brevis tendon region; and a magnetic resonance imaging device acquires static ankle joint structural images.

[0079] Because different devices have different sampling frequencies and trigger times, this invention uses gait phase as a unified time index to register various types of data. A complete gait cycle can be determined by the current heel strike time and the next heel strike time. The normalized gait phase is calculated as follows:

[0080] ;

[0081] In the formula, This represents the normalized gait phase. This is the current sampling time; For the first The heel strike time of each gait cycle; For the next heel strike.

[0082] During registration, heel strike (HS) can be identified by sudden increases in the vertical component of the plantar / ground reaction force, changes in the center of plantar pressure, or changes in the velocity of the heel marker point during motion capture; plantar weight-bearing (FF) can be identified by the plantar contact area or peak forefoot pressure; heel lift (HO) can be identified by the disappearance of pressure in the heel area or changes in the ankle plantar flexion angle; and toe lift (TO) can be identified by the disappearance of pressure in the toe area. After the above events are identified, the data of each modality are resampled to a unified [database / database]. A gait phase sampling point, for example It can be set to 101 sampling points, corresponding to 0% to 100% of the gait cycle.

[0083] For samples with different gait speeds or varying gait cycle lengths, dynamic time warping or event-segmented linear resampling can be used to align key gait events under a unified phase coordinate system. Each resampled sample includes ankle kinematics, plantar / ground reaction forces, and soft tissue elastic features under the same gait phase. Furthermore, 3D motion capture data can be low-pass filtered at 6Hz to 8Hz, preferably 6Hz; plantar / ground reaction force data can be low-pass filtered at 20Hz to 50Hz, preferably 30Hz; and ultrasound elastography data can be median-filtered on the region of interest, with a filtering window of 3 to 5 pixels, preferably 3 pixels. This step primarily aims to unify image, elastic, and dynamic mechanical data under the same motion phase reference, providing a foundation for subsequent phase-labeled soft tissue elastic field and muscle-bone dynamic coupling modeling.

[0084] S2: Individualized 3D geometric reconstruction of the ankle joint and construction of phase-labeled soft tissue elastic field

[0085] Traditional ankle finite element models typically establish their geometry based on static images and assign fixed material parameters to ligaments and tendons, making it difficult to reflect the dynamic changes in the elastic state of soft tissues under different gait phases. For example... Figure 4 As shown, this invention constructs an individualized three-dimensional geometric model using magnetic resonance imaging and maps musculoskeletal ultrasound elastography data to ligament and tendon regions to generate a phase-labeled soft tissue elastic field.

[0086] Specifically, the magnetic resonance imaging (MRI) images are segmented to extract tissue structures such as the tibia, fibula, talus, calcaneus, articular cartilage, anterior talofibular ligament, calcanofibular ligament, posterior talofibular ligament, deltoid ligament, intercalculotal ligament, peroneus longus tendon, and peroneus brevis tendon. Segmentation methods can include manual segmentation, semi-automatic thresholding, region growing, active contour models, or deep learning segmentation networks. After segmentation, surface reconstruction, mesh smoothing, hole repair, and mesh quality checks are performed on each tissue structure to create an individualized three-dimensional geometric model.

[0087] For bony structures, the tibia, fibula, talus, and calcaneus can be modeled as rigid or elastic bodies; for articular cartilage, ligaments, and tendons, finite element meshing can be used. The mesh elements of soft tissue regions should correspond as closely as possible to the region of interest in ultrasound elastography. To facilitate mapping, anatomical landmarks such as the tip of the lateral malleolus, the lateral process of the talus, the lateral tubercle of the calcaneus, the attachment points of the anterior talofibular ligament, the attachment points of the calcaneofibular ligament, and the peroneal tendon groove can be selected as registration reference points. Spatial correspondence between the ultrasound imaging plane coordinates and the coordinates of the 3D magnetic resonance model is established through rigid registration, affine registration, or thin-plate spline transformation.

[0088] For soft tissue mesh elements Its gait phase The elastic modulus can be obtained by interpolation using surrounding ultrasonic elastic sampling points:

[0089] ;

[0090] In the formula, For soft tissue mesh elements In gait phase The elastic modulus below; For the first One ultrasonic elastography sampling point; Sampling points In gait phase The elastic modulus measured below; To participate in the grid cell The set of sampling points for interpolation; Sampling points For grid cells Interpolation weights.

[0091] Interpolation weights can comprehensively consider spatial distance, organizational type consistency, and registration confidence, and are expressed as:

[0092] ;

[0093] In the formula, For interpolation weights; Sampling points With grid cells Spatial distance; For grid cells Spatial scale parameters of the organizational region; This is the organizational consistency coefficient. For registration confidence coefficients.

[0094] In one implementation, The sampling point should be between 3mm and 8mm, preferably 5mm; With grid cells When belonging to the same ligament or tendon region, the tissue consistency coefficient Use 1.00; when the two belong to adjacent soft tissue regions and are anatomically continuous, Take 0.30; when the two belong to different organizations and have no continuous relationship, Set to 0. Registration confidence coefficient The determination can be based on ultrasound image quality, the sharpness of the region of interest boundary, and registration error; when the ultrasound image quality is high and the registration error is less than 2mm, Use 1.00; when the image quality is medium or the registration error is 2mm to 4mm, Use 0.70; when the image quality is low or the registration error is 4mm to 6mm, Take 0.40; when the registration error is greater than 6mm, this sampling point will not be used for interpolation.

[0095] Regarding model parameter selection, the soft tissue elastic modulus can be either the absolute elastic modulus output by ultrasound shear wave elastography or a relative elastic modulus. If there are differences in elastic modulus calibration between different devices, the elastic data of the same subject or the same batch of subjects can be normalized to a relative modulus first. For subsequent deep learning input, the phase-labeled soft tissue elastic field can be voxelized, meshed, or point cloudified to form a data structure that can be read by spatial coding branches.

[0096] This step yields two key benefits: firstly, an individualized 3D geometric model reflects the differences in bony structure, soft tissue attachment locations, and joint morphology among different subjects; secondly, phase-labeled soft tissue elastic fields allow soft tissue material parameters to dynamically change with gait phase, avoiding the problem that traditional static material parameters cannot express the contributions of ligaments and tendons under functional weight-bearing conditions. Therefore, subsequent dynamic coupling simulations and surrogate model training have a basis for more closely approximating the actual tendon-bone interaction process in the ankle joint.

[0097] S3: Construction of a multibody dynamics model for dynamic coupling of musculoskeletal system.

[0098] like Figure 5 As shown, this step integrates the individualized ankle joint geometry model, soft tissue elastic field, plantar / ground reaction force, and bony motion boundary conditions to form a finite element-multibody dynamics coupled model. The contribution of this step lies in unifying bony structure motion, soft tissue deformation, and joint contact pressure within the same dynamic framework, providing the surrogate model with mechanically meaningful training labels.

[0099] The multibody sub-model is used to describe the relative motion between bony structures such as the tibia, fibula, talus, and calcaneus. These bony structures can be considered rigid bodies, and their spatial pose is described using joint constraints, kinematic pairs, and generalized coordinates. The finite element sub-model is used to describe the deformation, stress, and strain of soft tissues such as articular cartilage, ligaments, tendons, and joint capsules. The origin and insertion points of ligaments and tendons are bound to their corresponding bone attachment regions based on magnetic resonance imaging (MRI) segmentation results. Contact relationships are established between articular cartilages, and plantar or ground reaction forces act as external loads on the plantar region or the foot segment of the multibody model.

[0100] The dynamic coupling model satisfies the following dynamic relationship:

[0101] ;

[0102] In the formula, The generalized angle vector of the ankle joint; This is the ankle joint angular velocity vector; This is the angular acceleration vector of the ankle joint; This is the quality matrix; For the Coriolis and eccentricity terms matrix; This is the term related to gravity. For soft tissue strain Phase-dependent elastic modulus The jointly determined soft tissue constraint torque; For muscle / tendon active torque; This is the external reaction torque.

[0103] In one embodiment, the soft tissue constraint torque can be determined by the elastic modulus, equivalent cross-sectional area, lever arm, strain, and constraint coefficient of each ligament or tendon unit. The constraint coefficient for the anterior talofibular ligament can be 1.00; the constraint coefficient for the calcaneofibular ligament can be 0.90 to 1.00, preferably 0.95; the constraint coefficient for the posterior talofibular ligament can be 0.80 to 0.95, preferably 0.90; the constraint coefficient for the deltoid ligament can be 0.85 to 1.05, preferably 1.00; the constraint coefficients for the peroneus longus and peroneus brevis tendons can be 1.10 to 1.30, preferably 1.20; and the constraint coefficient for the joint capsule can be 0.50 to 0.70, preferably 0.60. These coefficients can be fine-tuned based on the individualized anatomical structure of the subject, the quality of ultrasound elastography, and the convergence of the finite element model.

[0104] Joint contact relationships can be represented using non-penetrating contact conditions:

[0105] ;

[0106] In the formula, This is a function of the normal clearance between the joint contact surfaces; Generalized coordinates for bony structures; The normal contact pressure.

[0107] To couple the finite element sub-model with the multibody sub-model, the following process can be performed at each phase sampling point: First, the bony motion boundary and external loads are determined using 3D motion capture data and plantar / ground reaction force data; second, phase-related material parameters are assigned to the ligament and tendon finite element elements; third, soft tissue deformation, stress-strain, and joint contact pressure are solved; finally, soft tissue reaction forces and contact reaction forces are fed back into the multibody model to correct the bony pose and joint reaction forces. This iteration can be performed once or multiple times until the bony pose, soft tissue reaction forces, and contact pressures meet the convergence criteria. In one embodiment, the convergence criteria can be set to a change in bony pose of less than 0.5°, a relative change in contact pressure of less than 5%, and a relative change in soft tissue reaction forces of less than 5%.

[0108] The model output includes bony posture, soft tissue stress, soft tissue strain, joint contact pressure, and dynamic stability indices. Among them, the dynamic stability indices can include peak ankle inversion, peak talus external rotation, peak stress of the anterior talofibular ligament, peak stress of the calcaneofibular ligament, center of pressure migration, peak joint contact pressure, ligament-bone stress transfer efficiency, and dynamic stability margin.

[0109] S4: Construction of Muscle-Skeleton Coupling Diagram and Coupling Tensor

[0110] While traditional finite element models can output stress contour maps and contact pressure distributions, the output results are usually unstructured or weakly structured numerical fields, making them difficult for deep learning models to directly interpret as the contribution of a ligament to the stability of a bony structure at a certain phase. For example... Figure 6 As shown, this invention transforms anatomical relationships, connectivity relationships, and mechanical variables into structured inputs by constructing a musculoskeletal coupling diagram and a musculoskeletal coupling tensor.

[0111] First, the nodes in the ankle joint model are classified. Bony nodes can be representative nodes, centroid nodes, attachment point nodes, or key anatomical nodes of the tibia, fibula, talus, and calcaneus; soft tissue nodes can be attachment point nodes or grid representative nodes of the anterior talofibular ligament, calcanofibular ligament, posterior talofibular ligament, deltoid ligament, peroneus longus and brevis tendons; joint contact nodes can be from the tibiotalar joint contact area, the subtalar joint contact area, and the lateral articular surface contact area. Then, graph edges are established based on anatomical connections, spatial proximity, contact transmission, and mechanical relationships.

[0112] The muscle-bone coupling diagram can be represented as: In the formula, This is a diagram of muscle-bone coupling. A set of bony nodes; A set of soft tissue nodes; For the set of joint contact nodes; It is the set of graph edges that connect bony nodes, soft tissue nodes, and joint contact nodes.

[0113] Edge types can include: bony-soft tissue connections, representing ligament or tendon attachments; soft tissue-soft tissue connections, representing continuous relationships between different grid regions of the same ligament or tendon; soft tissue-joint contact connections, representing the effect of soft tissue traction changes on pressure migration in the contact area; and bony-joint contact connections, representing the effect of bony pose changes on contact pressure. Edges can be weighted, and the weights can be determined by spatial distance, attachment relationship, tissue type, and biomechanical correlation.

[0114] For bony nodes and soft tissue nodes The distance weight can be expressed as:

[0115] ;

[0116] In the formula, For nodes With nodes Distance weights between them; bony nodes Spatial location; soft tissue nodes Spatial location; The spatial distance between the two nodes; For scale parameters; This represents the anatomical connectivity coefficient.

[0117] In one implementation, the scale parameter The thickness ranges from 5mm to 15mm, with 10mm being preferred; when there is a direct attachment between bony and soft tissue nodes, the anatomical connectivity coefficient... Set to 1.00; when there is a contact transmission relationship between the two, Take 0.80; when the two are only spatially adjacent, Take 0.50; when there is no anatomical or mechanical connection between the two, Take 0.

[0118] Based on the musculoskeletal coupling diagram, a coupling tensor is constructed. This tensor can have dimensions for bony nodes, soft tissue nodes, joint contact nodes, features, and relationships. For gait phase... bony nodes Soft tissue nodes and joint contact nodes The coupled tensor element can be represented as:

[0119] ;

[0120] In the formula, Gait phase Inferior bony nodes Soft tissue nodes and joint contact nodes Coupled tensor elements between them; For characteristic combination functions; bony nodes The change in pose; soft tissue nodes The amount of strain change; soft tissue nodes The elastic modulus at the corresponding location; This represents the change in joint contact pressure. For distance weights.

[0121] In practice, feature stitching, weighted summation, or normalized multi-channel stacking can be used. Feature dimensions can include displacement, rotation, principal strain, equivalent stress, elastic modulus, contact pressure, spatial distance, phase encoding, and tissue category encoding. Relationship dimensions can include adhesion relationships, contact relationships, proximity relationships, and mechanical transfer relationships. Thus, not only numerical features are preserved, but also the anatomical and mechanical relationships from which the features originate.

[0122] In one implementation, when weighted normalized superposition is used as the feature combination function, the weights for bony pose change, soft tissue strain change, soft tissue elastic modulus, joint contact pressure change, and inter-node distance are all set at 0.25. If the focus of the study is on the contribution of soft tissue elasticity, the weight of soft tissue elastic modulus can be increased to 0.25, and the weights of bony pose change or inter-node distance can be decreased by 0.05 accordingly.

[0123] This step transforms the finite element-multibody dynamics output into structured data suitable for graph attention network learning. This allows the model to learn the contributions of different ligaments, tendons, and bony regions to ankle joint stability during training, rather than simply concatenating multimodal features as ordinary vectors. This structured representation provides the foundation for subsequent interpretable analysis and virtual intervention prediction.

[0124] S5: Structured Disclosure of Deep Learning Agent Models

[0125] like Figure 7 As shown, the deep learning proxy model of this invention includes a temporal coding branch, a spatial coding branch, a graph attention coupling branch, a feature fusion module, and an output decoding layer. This model is used to learn the mapping relationship between multimodal inputs and the musculoskeletal coupling tensor to the mechanical output.

[0126] The inputs to the temporal coding branch include gait phase, joint motion sequence, and plantar / ground reaction force. In one embodiment, the temporal coding branch may include an input embedding layer, a temporal feature extraction layer, and a temporal pooling layer. The input embedding layer maps the kinematic and dynamic features of each phase sampling point to a fixed-dimensional vector; the temporal feature extraction layer may employ a combination of 1D-CNN and BiLSTM, where 1D-CNN is used to extract local phase change features, and BiLSTM is used to extract preceding and following phase dependencies; the temporal pooling layer may employ global average pooling or attention pooling to output temporal features.

[0127] The spatial encoding branch takes into account an individualized geometric model and a soft tissue elastic field as input. If the geometric model is represented as a point cloud, PointNet++ or a local neighborhood aggregation network can be used to extract geometric features; if the soft tissue elastic field is represented as a voxel, 3D-CNN can be used to extract spatial elastic distribution features; if a grid representation is used, a grid convolutional network or a graph convolutional network can be used to extract spatial features. The spatial encoding branch outputs spatial features.

[0128] The input to the graph attention coupling branch includes a musculoskeletal coupling graph and tissue properties / connectivity features. For nodes... and neighboring nodes The graph attention weights can be expressed as:

[0129] ;

[0130] In the formula, For nodes For nodes Attention weights; These are learnable attention parameters; The learnable feature transformation matrix; For nodes Input features; For nodes Input features; For nodes With nodes In gait phase The coupling characteristics under; Indicates feature splicing; For nodes The set of neighboring nodes.

[0131] Node updates can be represented as:

[0132] ;

[0133] In the formula, For nodes Updated features; This is a non-linear activation function; the meanings of the other symbols are the same as before.

[0134] The feature fusion module receives temporal features, spatial features, and graph features, where the graph features are the output features of the graph attention coupling branch. Fusion methods can include cross-modal attention, feature concatenation, and gated fusion. Gated fusion can be represented as:

[0135] ;

[0136] In the formula, Features of fusion; For gating weights; This is the representation after feature concatenation; For cross-modal attention output; This is element-wise multiplication.

[0137] The output decoding layer can adopt a structure of shared decoder plus multiple output heads. The shared decoder consists of a fully connected layer, a normalization layer, an activation layer and a dropout layer, and the output heads predict bony pose, soft tissue stress, soft tissue strain, joint contact pressure and dynamic stability indices, respectively.

[0138] Regarding parameter selection, in one implementation, the number of temporal sampling points can be set to 101, the temporal coding dimension to 64 or 128 (preferably 128), the spatial coding dimension to 128, the number of graph attention layers to 2 to 4 (preferably 3), the number of attention heads to 4 or 8 (preferably 4), the fusion feature dimension to 256, and the dropout ratio to 0.1 to 0.3 (preferably 0.2). If the training sample size is small, the dropout ratio can be increased to 0.3; if the training sample size is large and the validation set loss is stable, the dropout ratio can be decreased to 0.1. The above parameters can be adjusted according to the amount of data and hardware conditions, but the basic structure of three-branch coding, cross-modal fusion, and multi-output decoding should be maintained.

[0139] S6: Training Methods, Biomechanical Residual Constraints, and Generative Adversarial Calibration

[0140] like Figure 8 As shown, this step addresses the issue that ordinary deep learning proxy models may deviate from biomechanical principles. This invention uses the proxy model as a generator, employs measured data and high-fidelity finite element simulation results as the real or reference distribution, performs generative adversarial calibration through a discriminator, and introduces biomechanical residual constraints.

[0141] The training samples can be divided into three categories: the first category is measured kinematic data and corresponding clinical / phenotypic labels; the second category is musculoskeletal ultrasound elastography data and corresponding soft tissue elastic characteristics; the third category is finite element-multibody dynamics simulation output, including bony pose, soft tissue stress and strain, and contact pressure. During training, the generator receives multimodal inputs and outputs predicted mechanical responses, while the discriminator determines whether the input comes from real data / simulation references or from the generator's predictions.

[0142] The total loss function is:

[0143] ;

[0144] In the formula, This is the total loss function; To monitor losses; To combat the losses; Loss due to physical constraints; This is the regularization loss; , , These are the corresponding weighting coefficients.

[0145] In one implementation, The value should be between 0.05 and 0.20, preferably 0.10. The value should be between 0.20 and 0.60, with 0.50 being preferred. The value should be between 0.0001 and 0.001, with 0.0001 being preferred. If the adversarial loss fluctuates significantly in the early stages of training, it can be initially set to... Set it to 0.05, and gradually increase it to 0.10 after the model converges; if physical consistency is insufficient, it can be adjusted. Increase to 0.60; if overfitting occurs, it can be... Increased to 0.001.

[0146] Supervisory loss may include multiple output errors:

[0147] ;

[0148] In the formula, , , , , These are reference bony posture, soft tissue stress, soft tissue strain, joint contact pressure, and dynamic stability indicators. , , , , These are the model predictions; , , , , The weights for each output item.

[0149] In one implementation, Take 0.25, Take 0.20, Take 0.15, Take 0.20, Take 0.20. If the primary goal is to identify stress-induced thermal zones, then... Increased to 0.25, and Reduced to 0.20; if the primary objective is to predict dynamic stability indicators, then... Increased to 0.25, and Reduced to 0.10.

[0150] The loss of resistance can be expressed as:

[0151] ;

[0152] In the formula, To combat the losses; Output as real data or simulation reference; Predict the output for the generator; This represents the discriminator's judgment result on the reference output; This represents the discriminator's judgment result on the predicted output; This indicates the expected operation.

[0153] Physical constraint losses include joint moment balance residuals, soft tissue strain continuity residuals, contact non-penetration residuals, and gait cycle continuity residuals. The joint moment balance residuals can be expressed as:

[0154] ;

[0155] In the formula, This represents the joint torque balance residual; the meanings of the other symbols are the same as those in the aforementioned dynamic relationships.

[0156] The residual of soft tissue strain continuity can be expressed as:

[0157] ;

[0158] In the formula, For soft tissue strain continuity residuals; The soft tissue strain is the strain of the adjacent subsequent gait phase; The soft tissue strain at the current gait phase; This is the phase increment.

[0159] The contact non-penetration residual can be calculated based on the contact condition; the gait period continuity residual can constrain the state differences at the start and end points of the same period. The physical constraint loss can be expressed as:

[0160] ;

[0161] In the formula, Loss due to physical constraints; For contact non-penetrating residuals; For gait period continuity residuals; , , , The weights are the residual terms.

[0162] In one implementation, Take 0.40, Take 0.20, Take 0.25, Take 0.15. If the model exhibits joint torque imbalance, it can be adjusted. Increase to 0.50; if the strain curve output by the model shows a jump, it can be... Increase to 0.30; if unreasonable penetration or interruption occurs in the contact pressure, it can be... Increased to 0.35.

[0163] The training method may include the following steps: first, pre-train the surrogate model using the output of finite element-multibody dynamics simulation to enable the model to learn basic mechanical responses; then, fine-tune it by introducing measured kinematic data and ultrasonic elastic data; subsequently, add a discriminator for adversarial training; finally, use a combined training approach with full supervised loss, adversarial loss, physical constraint loss, and regularization loss. The optimizer can be Adam or AdamW, preferably AdamW; the initial learning rate can be set to 0.001 or 0.0005, preferably 0.001; the batch size can be set to 8, 16, or 32, preferably 16; the number of training epochs can be determined based on the convergence of the validation set loss, preferably 200 epochs. An early stopping strategy can be adopted, stopping training when the validation set loss does not decrease for 20 consecutive epochs.

[0164] This step integrates the mechanical consistency of the high-fidelity simulation model, the authenticity of the measured data, and the rapid prediction capability of the deep learning surrogate model, so that the output of the surrogate model not only closely matches the reference data, but also meets the requirements of joint torque balance, continuous soft tissue deformation, non-penetrating contact, and gait cycle continuity.

[0165] S7: Personalized Digital Twin Models, Virtual Perturbations, and Interpretability Analysis

[0166] like Figure 9 As shown, the trained surrogate model, together with the individualized geometric model, phase-labeled soft tissue elastic field, muscle-bone coupling map, and coupling tensor, constitutes an individualized digital twin model of the ankle joint. This model can simulate dynamic stability changes under different soft tissue states, asynchronous conditions, and different intervention protocols for a single subject.

[0167] Virtual perturbation inputs include ligament elastic modulus, tendon elastic modulus, neuromuscular delay, plantar reaction path, and orthotic constraint parameters. Perturbations can be single-factor, multi-factor combined, or parameterized based on rehabilitation protocols. For example, decreasing the elastic modulus of the anterior talofibular ligament can simulate ligament laxity; increasing the activation delay of the peroneal tendon can simulate neuromuscular control lag; adjusting the plantar reaction path can simulate outward shift of the center of pressure; and increasing orthotic constraint parameters can simulate the supporting effect of a semi-rigid ankle-foot orthosis.

[0168] In one embodiment, the ligament elastic modulus perturbation amplitude is reduced by 10% to 40% from the baseline value, preferably by 20%; the tendon elastic modulus perturbation amplitude is reduced by 10% to 30% from the baseline value, preferably by 15%; the neuromuscular delay perturbation amplitude is reduced by 20ms to 80ms, preferably by 40ms; the outward displacement of the plantar reaction path is reduced by 5mm to 15mm, preferably by 10mm; the orthotic constraint parameters are expressed using normalized values, with 0.30 for low constraint, 0.60 for medium constraint, and 0.90 for high constraint.

[0169] The change in dynamic stability before and after the disturbance can be expressed as:

[0170] ;

[0171] In the formula, This refers to the change in the dynamic stability index. It is a dynamic stability index under disturbance conditions; This is a dynamic stability index under baseline conditions.

[0172] To quantify the ligament-bone stress transmission relationship, we can define:

[0173] ;

[0174] In the formula, Gait phase The efficiency of stress transmission between the ligaments and bones; This represents the change in joint contact pressure. This represents the change in ligament stress. To avoid stable terms with a denominator of zero, preferably, Take 0.001.

[0175] In one implementation, the dynamic stability index can be obtained by weighting the normalized risk of peak ankle inversion, peak talus external rotation, peak ligament stress, peak joint contact pressure, and center of pressure migration. Specifically, the weight of the normalized risk of peak ankle inversion is 0.25, the weight of the normalized risk of peak talus external rotation is 0.20, the weight of the normalized risk of peak ligament stress is 0.20, the weight of the normalized risk of peak joint contact pressure is 0.20, and the weight of the normalized risk of center of pressure migration is 0.15. For lateral ligament instability risk analysis, the weight of the normalized risk of peak ligament stress can be increased to 0.25, and the weight of the normalized risk of center of pressure migration can be decreased to 0.10.

[0176] Interpretability analysis can employ SHAP contribution, graph attention weights, gradient-type activation heatmaps, or sensitivity analysis. Taking SHAP as an example, the model can output a ranking of the contributions of each input feature to dynamic stability indices, ankle inversion peak, talus external rotation peak, or ligament peak stress. If the anterior talofibular ligament elastic modulus, plantar reaction path, or peroneal tendon delay has a high contribution, it indicates that the subject's instability risk is closely related to the corresponding soft tissue or control parameters. Gradient-type activation heatmaps can map sensitive areas of the model back to the ankle joint geometry model, allowing clinicians to visually observe key ligaments, key tendons, or key articular surface areas.

[0177] like Figure 10 As shown, this invention can also output ROC curves, SHAP feature importance, confusion matrix, and learning curves to evaluate the model's classification, generalization, and interpretability; it can also output the changes in equivalent stress distribution at key nodes of the ankle joint under different external torque conditions to illustrate the migration or expansion of high stress areas as torque increases.

[0178] V. Specific Implementation Examples and Comparative Examples

[0179] (a) Test subjects, equipment and data processing conditions

[0180] To verify the technical effectiveness of the ankle joint tendon-bone dynamic coupling digital twin modeling method based on multimodal data and deep learning, the laboratory followed the... Figure 1 The procedure shown was used for a pre-validation experiment. The subjects included 30 subjects with chronic ankle instability and 20 healthy controls. The subjects with chronic ankle instability had a history of recurrent ankle sprains, instability during movement, or lateral ankle instability in closed-chain weight-bearing tests; the healthy controls had no history of ankle injury within the past 6 months. Each subject completed natural walking, single-leg standing, closed-chain weight-bearing, and graded external rotation torque simulation tests. Each movement was recorded 5 times. After excluding data with missing motion capture markers, abnormal plantar reaction forces, and substandard ultrasound elastography quality, a total of 238 valid gait cycles were obtained.

[0181] Data acquisition equipment includes: an infrared 3D motion capture system with a sampling frequency of 100Hz; an embedded force table or plantar pressure acquisition system with a sampling frequency of 1000Hz; a musculoskeletal ultrasound shear wave elastography device with a linear array probe frequency of 4MHz to 15MHz; and a 3.0T magnetic resonance imaging system to acquire high-resolution 3D sequence images of the ankle joint with a slice thickness of 1mm. A finite element-multibody dynamics model is used to generate a high-fidelity reference output, and a deep learning proxy model is used... Figure 7 The structure shown is established, and according to... Figure 8 The biomechanical residual constraint and generative adversarial calibration training are performed in the manner shown.

[0182] The dataset is divided into training, validation, and test sets based on the subject dimension, with the training set comprising 70%, the validation set 15%, and the test set 15%. To avoid data leakage from the same subject, different gait cycles of the same subject do not appear simultaneously in the training and test sets. Each gait cycle is arranged according to... Figure 3 The method shown normalizes to 101 gait phase sampling points.

[0183] (II) Example 1: Complete Multimodal Muscle-Bone Dynamic Coupling Digital Twin Modeling Method

[0184] Example 1 uses the complete modeling method described above for modeling, and its process is the same as... Figure 1 Consistent, system structure and Figure 2 Consistent.

[0185] First, such as Figure 3 As shown, three-dimensional motion capture data, plantar / ground reaction force data, and musculoskeletal ultrasound elastography data were collected simultaneously during the subject's walking process. The heel strike HS, plantar weight-bearing FF, heel lift HO, toe lift TO, and the next heel strike HS were used as gait events, and each gait cycle was normalized into a unified gait phase.

[0186] Secondly, such as Figure 4 As shown, an individualized three-dimensional geometric model of the ankle joint was established by segmenting the tibia, fibula, talus, calcaneus, articular cartilage, anterior talofibular ligament, calcaneofibular ligament, posterior talofibular ligament, deltoid ligament, intercalculotal ligament, peroneus longus tendon, and peroneus brevis tendon based on magnetic resonance imaging. Using the lateral malleolus tip, lateral talus process, lateral calcaneal tuberosity, anterior talofibular ligament attachment point, and calcaneofibular ligament attachment point as registration references, the elastic modulus in the ultrasound elastogram was mapped onto the three-dimensional ligament and tendon mesh region, forming a phase-labeled soft tissue elastic field.

[0187] In this embodiment, the spatial scale parameter for soft tissue elastic field interpolation is 5 mm; the tissue consistency coefficient for the same ligament or tendon region is 1.00, the tissue consistency coefficient for adjacent soft tissue regions is 0.30, and the tissue consistency coefficient for tissues without anatomical continuity is 0; the registration confidence coefficient is 1.00 when the registration error is less than 2 mm, 0.70 when the registration error is 2 mm to 4 mm, 0.40 when the registration error is 4 mm to 6 mm, and the corresponding sampling point is not used for interpolation when the registration error is greater than 6 mm.

[0188] Then, as Figure 5As shown, the phase-labeled soft tissue elastic field is input into the fascio-bone dynamic coupling finite element-multibody dynamics model. The bony structure is set as a multibody dynamics sub-model, and ligaments, tendons, articular cartilage, and joint capsules are set as finite element sub-models. Foot / ground reaction force is used as the external load input. The model outputs bony pose, soft tissue stress and strain, joint contact pressure, and dynamic stability indices. In this embodiment, the constraint coefficients are set to 1.00 for the anterior talofibular ligament, 0.95 for the calcaneofibular ligament, 0.90 for the posterior talofibular ligament, 1.00 for the deltoid ligament, 1.20 for the peroneus longus and peroneus brevis tendons, and 0.60 for the joint capsule. The convergence conditions for the finite element-multibody dynamics coupling iteration are: bony pose change less than 0.5°, relative change in contact pressure less than 5%, and relative change in soft tissue reaction force less than 5%.

[0189] Furthermore, such as Figure 6 As shown, a musculoskeletal coupling graph is constructed by bony nodes, soft tissue nodes, and joint contact nodes, and a coupling tensor is built based on changes in bony pose, soft tissue strain, elastic modulus, contact pressure, and distance weights. This tensor serves as the input to the graph attention coupling branch, used to learn the contribution of soft tissue constraints to bony stability. In this embodiment, the anatomical connectivity coefficient for direct attachment is set to 1.00, the anatomical connectivity coefficient for contact transmission is set to 0.80, and the anatomical connectivity coefficient for spatial proximity is set to 0.50; the scale parameter for the distance weights is set to 10 mm. In the coupling tensor, the weights for changes in bony pose, soft tissue strain, soft tissue elastic modulus, joint contact pressure, and inter-node distance are set to 0.10.

[0190] Finally, as Figure 7 and Figure 8 As shown, a deep learning proxy model is constructed, comprising temporal encoding, spatial encoding, and graph attention coupling branches, and jointly trained using supervised loss, adversarial loss, physical constraint loss, and regularization loss. After training, an individualized ankle joint digital twin model is built based on the trained model, and then... Figure 9 Perform virtual perturbation and interpretability analysis.

[0191] In this embodiment, the initial learning rate is 0.001, the batch size is 16, the training epochs are 200, the optimizer is AdamW, the weight decay coefficient is 0.0001, the number of graph attention heads is 4, the number of graph attention layers is 3, the fusion feature dimension is 256, and the dropout ratio is 0.2. If the validation set loss does not decrease for 20 consecutive epochs, training is stopped early. In the total training loss, the adversarial loss weight is 0.10, the physical constraint loss weight is 0.50, and the regularization loss weight is 0.0001. In the supervised loss, the weights for bony pose, soft tissue stress, soft tissue strain, joint contact pressure, and dynamic stability index are 0.20. In the physical constraint loss, the weights for joint moment balance residual, soft tissue strain continuity residual, contact non-penetration residual, and gait cycle continuity residual are 0.40, 0.20, 0.25, and 0.15.

[0192] (III) Example 2: Prediction of Orthotics Constraint Parameters Based on Virtual Perturbation

[0193] Example 2, based on the individualized ankle joint digital twin model trained in Example 1, further simulates the orthotic constraint parameters. The effects on subjects with chronic ankle instability, specifically the process is as follows: Figure 9 As shown.

[0194] Four conditions are set: no orthotics, low-constraint orthotics, medium-constraint orthotics, and high-constraint orthotics. Orthotics constraint parameters. This includes lateral anti-inversion stiffness, talus external rotation limitation coefficient, and plantar reaction path correction coefficient. These parameters are input into an individualized digital twin model, which outputs peak ankle inversion stress, peak talus external rotation stress, peak anterior talofibular ligament stress, peak joint contact pressure, and dynamic stability margin. .

[0195] The change in the dynamic stability index is calculated using the following formula:

[0196] ;

[0197] In the formula, This refers to the change in the dynamic stability index. It is a dynamic stability index under disturbance conditions; This is a dynamic stability index under baseline conditions.

[0198] This embodiment is used to demonstrate that the present invention can not only predict the dynamic biomechanical response of the ankle joint, but also screen personalized intervention programs.

[0199] (iv) Example 3: Phenotypic Identification and Interpretability Analysis of Chronic Ankle Instability

[0200] Example 3 uses the multimodal features and muscle-bone coupling tensor obtained in Example 1. and dynamic stability index Phenotypic identification was performed on subjects with chronic ankle instability. Phenotyps included ligament laxity-dominant, neurological delay-dominant, and mixed types.

[0201] The model input includes the elastic modulus of the anterior talofibular ligament, the elastic modulus of the calcaneofibular ligament, the elastic modulus of the peroneal tendon, neuromuscular delay, plantar reaction path, peak ankle inversion, peak talar external rotation, peak joint contact pressure, and tendon-bone coupling tensor features. The output consists of three phenotypic labels. Figure 10 The ROC curve, confusion matrix, and SHAP analysis results can be used to determine the classification performance and key contribution characteristics of the model.

[0202] In this embodiment, the main explanatory features of the model for the ligament laxity-dominant type are decreased elastic modulus of the anterior talofibular ligament and stress concentration in the anterior talofibular ligament; the main explanatory feature for the neuromuscular delay-dominant type is neuromuscular delay. Increased and outward shift of the plantar reaction path; the main explanatory features of the mixed type are the coexistence of decreased ligament elastic modulus, delayed peroneal tendon response, and increased lateral joint contact pressure.

[0203] (v) Proportional Setting

[0204] Comparative Example 1: Static finite element model based solely on magnetic resonance imaging

[0205] Comparative Example 1 uses only magnetic resonance imaging to establish a three-dimensional geometric model of the ankle joint, assigning a fixed average elastic modulus to the ligaments and tendons, without introducing three-dimensional motion capture data, plantar / ground reaction force data, or musculoskeletal ultrasound elastography data, and without constructing a phase-labeled soft tissue elastic field. This model is used to verify the impact of the dynamic phase elastic field on prediction accuracy.

[0206] Comparative Example 2: Multimodal Feature Direct Concatenation Model

[0207] Comparative Example 2 uses the same multimodal data as Example 1, but does not construct a musculoskeletal coupling map. and coupling tensor Instead, the kinematics, reaction force, elastic modulus, and geometric features are directly vectorized, concatenated, and then input into a conventional multilayer perceptron. This comparative model is used to verify the effectiveness of structured muscle-bone coupling representation.

[0208] Comparative Example 3: Surrogate Model Without Biomechanical Residual Constraints

[0209] Comparative Example 3 uses the same three-branch deep learning proxy model as Example 1, but only uses supervised loss during training. No physical constraint loss is introduced. It also does not employ joint moment balance, strain continuity, contact non-penetration, or gait period continuity residuals. This comparative example is used to verify the effect of biomechanical residual constraints.

[0210] Comparative Example 4: Proxy Model without Generative Adversarial Calibration

[0211] Comparative Example 4 uses the same three-branch surrogate model and biomechanical residual constraints as Example 1, but without a discriminator. Do not use counter-loss This comparative example is used to verify the effect of generative adversarial calibration on model generalization and experimental consistency.

[0212] Comparative Example 5: Predictive models without virtual perturbations and interpretability analysis

[0213] Comparative Example 5 uses the training process of Example 1 to obtain a prediction model, but only outputs bony pose, soft tissue stress and strain, and joint contact pressure, without performing any further analysis. , , , and The virtual perturbation does not output SHAP contribution or sensitive area heatmaps. This comparative example is used to verify the effectiveness of the present invention in mechanism explanation and intervention prediction.

[0214] (vi) Testing methods and evaluation indicators

[0215] 1. Evaluation of prediction error

[0216] Using measured kinematic data from the test set and high-fidelity finite element-multibody dynamics simulation results as references, the prediction errors of ankle inversion angle, talus external rotation angle, peak stress of anterior talofibular ligament, and peak joint contact pressure are calculated.

[0217] The root mean square error is calculated using the following formula:

[0218] ;

[0219] In the formula, For variables The root mean square error; This represents the number of test samples; For the first Predicted values ​​for each sample; For the first Reference values ​​for each sample.

[0220] The relative error is calculated using the following formula:

[0221] ;

[0222] In the formula, For variables The relative error; This is a predicted value; For reference only.

[0223] 2. Mechanical consistency evaluation

[0224] Using joint torque to balance residuals Soft tissue strain continuity residual Contact non-penetrating residual and gait period continuity residual Evaluate whether the model output conforms to biomechanical principles. The smaller the residual, the more the model's predictions conform to mechanical constraints.

[0225] 3. Risk hotspot identification and assessment

[0226] Using the high-stress region of the anterior talofibular ligament and the high-value contact pressure region of the lateral joint output by the finite element-multibody dynamics simulation as a reference, the overlap rate between the predicted thermal zone and the reference thermal zone is calculated. The higher the overlap rate of hot zones, the more accurate the model's location of key risk areas.

[0227] 4. Intervention prediction and evaluation

[0228] For some subjects, a short-term semi-rigid ankle-foot orthosis test was conducted. The measured peak reduction rate of ankle inversion, peak reduction rate of talus external rotation, and improvement rate of dynamic stability margin were compared with the prediction results of the digital twin model to verify the reliability of the virtual intervention prediction.

[0229] (vii) Test Results

[0230] Table 1 Comparison of dynamic prediction accuracy of different models

[0231]

[0232] As shown in Table 1, the prediction errors of Example 1 are all lower than those of the comparative example. Comparative Example 1, however, lacks a phase-labeled soft tissue elastic field. This cannot reflect the dynamic elastic changes of ligaments and tendons under different gait phases; although Comparative Example 2 uses multimodal data, it does not construct... and This makes it difficult for the model to learn the structured relationship between soft tissue constraints and bone stability; Comparative Examples 3 and 4 show that both biomechanical residual constraints and generative adversarial calibration help improve prediction accuracy.

[0233] Table 2 Results of Mechanical Consistency and Identification of Key Risk Areas

[0234]

[0235] As shown in Table 2, Example 1 exhibits lower residuals in terms of joint torque balance, strain continuity, non-penetrating contact, and gait cycle continuity, indicating that its output results better conform to the dynamic coupling law of muscle and bone. Combined with... Figure 10 As can be seen from the changes in equivalent stress distribution, Example 1 can more accurately identify the high stress area that expands with the increase of external torque, and can locate the stress concentration in the anterior talofibular ligament and lateral articular surface area.

[0236] Table 3 Ablation Test Results

[0237]

[0238] As shown in Table 3, the elastic field of soft tissue after removing the phase markers... The most significant performance degradation was observed later, indicating that phase mapping of soft tissue elastic parameters plays a crucial role in identifying chronic ankle instability phenotypes and predicting dynamic stability. and The performance also decreased significantly, indicating that the musculoskeletal coupling diagram and coupling tensor can effectively express the relationship between soft tissue nodes, bony nodes and joint contact nodes.

[0239] Table 4. Prediction Results of Virtual Intervention for Orthotics

[0240]

[0241] As shown in Table 4, the medium-restraint orthosis reduced the peak ankle inversion stress by approximately 30.4%, the peak talar external rotation stress by approximately 29.9%, the peak anterior talofibular ligament stress by approximately 36.6%, and the peak joint contact pressure by approximately 19.5% compared to the baseline state, while increasing the dynamic stability margin by approximately 39.0%. Although the high-restraint orthosis further limited inversion and external rotation, the dynamic stability margin did not continue to improve, indicating that excessive restraint may lead to a redistribution of local forces. These results demonstrate that the present invention can be used for individualized selection of orthotic restraint parameters.

[0242] Table 5 Comparison of Model Running Efficiency

[0243]

[0244] As shown in Table 5, after training, Example 1 has a single gait cycle prediction time of 0.43s, which is significantly shorter than the high-fidelity finite element-multibody dynamics complete simulation. At the same time, it can output key contribution and virtual perturbation results, indicating that the present invention has both computational efficiency, mechanical consistency and interpretability.

[0245] Table 6 Phenotypic Identification Results and Figure 10 Corresponding data

[0246]

[0247] Figure 10 The confusion matrix results for the test set are as follows: in category 0, 4 cases were correctly identified as category 0, and 7 cases were identified as category 1; in category 1, all 19 cases were correctly identified as category 1; in category 2, 1 case was correctly identified as category 2, and 5 cases were identified as category 1. These results indicate that the model is relatively stable in identifying intermediate phenotypes. For significantly unstable phenotypes, the sample size still needs to be increased under small sample conditions. However, the macro-average AUC reaches 0.798, demonstrating good multi-class discrimination ability.

[0248] Table 7 Ranking of Key Feature Contribution

[0249]

[0250] From Table 7 and Figure 10 The SHAP feature importance results show that features such as gait deviation index, muscle strength related indicators, double support phase, and TUGT have a high contribution to the model output, indicating that the present invention can jointly interpret multimodal gait features, clinical functional indicators, and local mechanical response of the ankle joint.

[0251] Figure 11 The bar chart shows the difference in prediction error between Example 1 and Comparative Examples 1 to 4, displaying the differences in error for ankle inversion angle, talus external rotation angle, peak stress of anterior talofibular ligament, and peak joint contact pressure.

[0252] Figure 12 Constraint parameters for different orthotics The following dynamic stability index change curves show the effects of low-constraint, medium-constraint, and high-constraint schemes on... The impact.

[0253] Figure 13 The image shows the results of the ablation test, indicating the removal of... Remove and Remove Remove Changes in post-model accuracy, macro-average F1, and macro-average AUC.

[0254] The above embodiments and comparative examples show that the technical effects of the present invention are mainly reflected in the following aspects.

[0255] First, this invention enables 3D motion capture data, plantar / ground reaction force data, musculoskeletal ultrasound elastography data, and magnetic resonance imaging reconstruction models to be expressed in a unified phase through gait phase registration, thus avoiding the problem that traditional multimodal data are independent and difficult to couple directly.

[0256] Second, this invention uses phase-labeled soft tissue elastic fields. The soft tissue biomechanical parameters measured by ultrasound elastography are mapped onto an individualized three-dimensional geometric model, allowing the elastic parameters of ligaments and tendons to vary with gait phase and spatial position. As shown in Table 3, removing... The accuracy of the model decreased from 86.7% to 72.8%, indicating that this feature made a substantial contribution to the technical effect.

[0257] Third, this invention utilizes a musculoskeletal coupling diagram. and coupling tensor This invention provides a structured representation of the relationships between bony nodes, soft tissue nodes, and joint contact nodes. Compared to the simple feature stitching in Comparative Example 2, this invention more accurately identifies the anterior talofibular ligament thermal zone and the lateral contact pressure thermal zone, demonstrating that this structured representation can help the model learn the intrinsic connection between soft tissue constraints, bony pose, and contact pressure.

[0258] Fourth, this invention, through biomechanical residual constraints and generative adversarial calibration, enables the surrogate model not only to fit the training data but also to satisfy mechanical constraints such as joint torque balance, continuous soft tissue strain, non-penetrating contact, and continuous gait cycle. Table 2 shows that the residuals of Example 1 are all lower than those of the comparative example, indicating that its output results are closer to the actual mechanical laws.

[0259] Fifth, this invention, through virtual perturbation and interpretability analysis, not only outputs prediction results but also identifies key contributing factors and predicts intervention effects. Table 4 shows that the moderately restrained orthosis has a good comprehensive effect in reducing peak ankle inversion, peak talus external rotation, and peak stress in the anterior talofibular ligament; Table 7 shows that the model can output the contribution ranking of gait and biomechanical characteristics, providing a quantitative basis for clinical rehabilitation training, orthotic design, and individualized intervention.

[0260] The foregoing description of embodiments of the present invention, through which those skilled in the art are able to implement or use the present invention, will be readily apparent to those skilled in the art. Various modifications to these embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novelty disclosed herein.

[0261] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0262] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure One One or more processes and / or boxes Figure One A device that provides the functions specified in one or more boxes.

[0263] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure One One or more processes and / or boxes Figure One The function specified in one or more boxes.

[0264] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure One One or more processes and / or boxes Figure One Figure One The steps of the function specified in one or more boxes.

[0265] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0266] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0267] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

Claims

1. A method for dynamic coupling digital twin modeling of ankle joint tendons and bones, characterized in that, The method includes the following steps: S1. Simultaneously acquire three-dimensional motion capture data, plantar or ground reaction force data, and ankle perimuscular ultrasound elastography data of the subject during walking, single-leg standing, or closed-chain weight-bearing movements. Separately acquire ankle joint magnetic resonance imaging data, and perform time registration of the three-dimensional motion capture data, plantar or ground reaction force data, and ankle perimuscular ultrasound elastography data using gait phase as a unified time index. S2. Based on the magnetic resonance imaging data, establish an individualized three-dimensional geometric model of the ankle joint including the tibia, fibula, talus, calcaneus, articular cartilage, anterior talofibular ligament, calcanofibular ligament and peroneus longus and shortus tendons, and map the musculoskeletal ultrasound elastography data onto the individualized three-dimensional geometric model of the ankle joint to generate a phase-labeled soft tissue elastic field. S3. Based on three-dimensional motion capture data, foot or ground reaction force data and soft tissue elastic field, construct a musculoskeletal dynamic coupling finite element-multibody dynamics model that includes bony movement, soft tissue deformation and joint contact relationship; S4. Based on the aforementioned musculoskeletal dynamic coupling finite element-multibody dynamics model, construct a musculoskeletal coupling diagram and generate a musculoskeletal coupling tensor characterizing the contribution of soft tissue constraints to bony stability. S5. Construct a deep learning proxy model that includes a temporal coding branch, a spatial coding branch, and a graph attention coupling branch, and introduce biomechanical residual constraints and generative adversarial calibration for training. S6. Based on the trained deep learning agent model, establish an individualized digital twin model of the ankle joint, and virtually perturb the elastic modulus of ligaments, elastic modulus of tendons, neuromuscular delay, plantar reaction path or orthotic constraint parameters to obtain the change in dynamic stability index. Combine the SHAP value or gradient class activation heatmap to output key soft tissues, key bony regions and their contribution.

2. The method according to claim 1, characterized in that, In step S1, the gait phase includes at least the initial contact phase, weight-bearing reaction phase, mid-stage support phase, end-stage support phase, pre-swing phase, initial swing phase, mid-stage swing phase, and end-stage swing phase; the time registration includes uniformly resampling the three-dimensional motion capture sampling sequence, the plantar or ground reaction force sampling sequence, and the musculoskeletal ultrasound elastography sampling sequence to the normalized gait cycle coordinates, so that the bony movements, external loads, and soft tissue elastic parameters under the same gait phase have a corresponding relationship; And / or, in step S2, the phase-marked soft tissue elastic field is generated as follows: using the lateral process of the talus, the tip of the lateral malleolus, the lateral tubercle of the calcaneus, the attachment point of the anterior talofibular ligament, and the attachment point of the calcanofibular ligament as registration reference points, the elastic modulus distribution measured by musculoskeletal ultrasound elastography is interpolated to the corresponding ligament or tendon grid unit, and each grid unit is assigned a gait phase label, a spatial coordinate label, and an elastic modulus label.

3. The method according to claim 1, characterized in that, In step S3, the boundary conditions for bony motion of the ankle joint are established based on the three-dimensional motion capture data and the plantar or ground reaction force data. The phase-related material parameters of ligaments and tendons are established based on the phase-marked soft tissue elastic field. A tendon-bone dynamic coupling finite element-multibody dynamics model containing bony motion, soft tissue deformation and joint contact relationship is constructed. And / or, in step S3, the musculoskeletal dynamic coupling finite element-multibody dynamics model satisfies the following dynamic relationship: ； In the formula, Ankle joint angle vector; This is the ankle joint angular velocity vector; This is the angular acceleration vector of the ankle joint; This is the quality matrix; For the Coriolis and eccentricity terms matrix; This is the term related to gravity. For soft tissue strain Phase-dependent elastic modulus The jointly determined soft tissue constraint torque; For muscle active torque; This is the external reaction torque.

4. The method according to claim 1, characterized in that, In step S4, the musculoskeletal dynamic coupling finite element-multibody dynamics model outputs the bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index under different time phases, and the bony pose nodes, soft tissue attachment point nodes and joint contact area nodes constitute a musculoskeletal coupling diagram. And / or, in step S4, the musculoskeletal coupling tensor includes changes in bony pose, changes in soft tissue strain, soft tissue elastic modulus, changes in joint contact pressure, and spatial distance weights between nodes, used to characterize the constraint contribution of ligaments or tendons on the relative motion of the talus, calcaneus, tibia, or fibula in the same gait phase.

5. The method according to claim 1, characterized in that, In step S5, a deep learning proxy model is constructed with gait phase, bony motion boundary conditions, plantar or ground reaction force, soft tissue elastic field and muscle-bone coupling tensor as inputs, and bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index as outputs. The deep learning proxy model includes a temporal encoding branch for extracting bony rigid body motion features, a spatial encoding branch for extracting soft tissue elastic field features, and a graph attention coupling branch for fusing the interaction between bony nodes and soft tissue nodes. And / or, in step S5, the deep learning proxy model is trained by using measured kinematic data, musculoskeletal ultrasound elastography data and finite element simulation output, and biomechanical residual constraints and generative adversarial calibration are introduced during the training process so that the bony pose, soft tissue stress and strain and joint contact pressure output by the deep learning proxy model simultaneously satisfy the consistency of measured data and the consistency of mechanical equilibrium. And / or, in step S5, the graph attention coupling branch uses bony nodes and soft tissue nodes as graph nodes, soft tissue attachment relationships, joint contact relationships and spatial proximity relationships as graph edges, and outputs the contribution characteristics of different ligaments, tendons and joint contact areas to ankle joint stability according to attention weights.

6. The method according to claim 1, characterized in that, In step S5, the biomechanical residual constraints include at least joint moment balance residuals, soft tissue strain continuity residuals, joint contact non-penetration residuals, and gait cycle continuity residuals; the generative adversarial calibration includes using a deep learning proxy model as a generator and using the joint features of measured kinematic data, musculoskeletal ultrasound elastography data, and finite element simulation output as the discrimination object to train the discriminator. And / or, in step S5, the training loss function includes a supervision error term, an adversarial error term, a biomechanical residual term, and a regularization error term. The supervision error term is used to characterize the error between the predicted output and the measured data or simulation annotations. The adversarial error term is used to characterize the error in the adversarial calibration process. The biomechanical residual term is used to characterize the mechanical consistency constraint error. The regularization error term is used to constrain the model complexity. Each error term is trained by weighting it with corresponding weight coefficients.

7. The method according to claim 1, characterized in that, In step S6, the dynamic stability index includes at least one of the following: peak ankle inversion, peak talus external rotation, peak anterior talofibular ligament stress, peak calcaneofibular ligament stress, peak joint contact pressure, center of pressure migration, ligament-bone stress transmission efficiency, neuromechanical delay time, and dynamic stability margin.

8. A dynamic coupling digital twin modeling system for ankle joint tendons and bones, characterized in that, The system is used to implement the method according to any one of claims 1 to 7, comprising: The data acquisition and registration module is used to simultaneously acquire and register three-dimensional motion capture data, plantar or ground reaction force data, musculoskeletal ultrasound elastography data and magnetic resonance imaging data, and generate a multimodal data sequence indexed by gait phase; The individualized model building module is used to build a three-dimensional geometric model of the ankle joint based on magnetic resonance imaging and to map musculoskeletal ultrasound elastography data into a phase-labeled soft tissue elastic field. The dynamic coupling simulation module is used to construct a finite element-multibody dynamic model of muscle-bone dynamic coupling that includes bony movement, soft tissue deformation and joint contact relationship, and outputs bony pose, soft tissue stress and strain, joint contact pressure and dynamic stability index. The proxy model training module is used to build and train a deep learning proxy model that includes a temporal encoding branch, a spatial encoding branch, and a graph attention coupling branch. The adversarial calibration module is used to generate adversarial calibration for the deep learning agent model based on measured data and simulation output, and correct the model output through biomechanical residual constraints. The interpretable analysis and virtual perturbation module is used to output key soft tissues, key bony regions and their contributions based on SHAP values, gradient-type activation heatmaps or sensitivity analysis, and to simulate the effects of rehabilitation training, orthotic constraints or changes in soft tissue parameters on the dynamic stability of the ankle joint.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images