Biomechanical simulation system for child musculoskeletal system disease analysis

Abstract

The invention relates to the technical field of biomechanical simulation, in particular to a biomechanical simulation system for child musculoskeletal system disease analysis, comprising: an acquisition module for acquiring a gait video for attitude estimation to obtain attitude estimation data; the calculation module is used for performing inverse kinematics calculation so as to determine muscle driving force and joint torque and generate plantar pressure distribution and a gravity center moving track; and the classification module generates a classification result according to the dynamic data, the kinematics data and the clinical information. In order to solve the problem of poor simulation effect of a biological simulation method for children in the prior art, inverse kinematics calculation and sole finite element analysis are respectively performed according to attitude estimation data; therefore, the muscle driving force in the walking process, the joint torque on the corresponding joint, the plantar pressure distribution and the corresponding gravity center moving track are estimated, and the gait modes of the to-be-evaluated object are effectively classified in combination with clinical information, so that a doctor can make related judgment conveniently.

Description

Technical Field

[0001] The present invention relates to the technical field of biomechanics simulation, and in particular to a biomechanics simulation system for analyzing diseases of the musculoskeletal system of children. Background Art

[0002] Currently, biomechanical modeling and analysis techniques for adult musculoskeletal disorders have mature systems and are widely used in fields such as sports medicine and rehabilitation. Biomechanics is a branch of biophysics that applies mechanical principles and methods to quantitatively study mechanical problems in living organisms. Its scope ranges from organisms as a whole to systems and organs (including blood, body fluids, internal organs, and bones). It studies human movement using the fundamental principles of statics, kinematics, and dynamics, combined with anatomy and physiology.

[0003] For example, Chinese patent CN202210444795.X discloses a biomechanical simulation analysis method for cervical spine rehabilitation training based on Opensim, comprising the following steps: Step S1: Building a resistance training model and an isometric training model at different angles based on a simulation model of an external rehabilitation training device combined with the Opensim head and neck musculoskeletal model; Step S2: Based on the obtained resistance training model and the isometric training model at different angles, simulation analysis is performed based on a preset simulation scheme to obtain the degree of muscle fatigue; Step S3: Using Matlab to update the model's muscle parameters, and using the model with updated muscle parameters to perform simulation, the generated results are compared with the simulation results without adding muscle fatigue, and the impact of muscle fatigue on the training effect is analyzed from aspects such as muscle activation patterns and the contribution of each muscle group before and after muscle fatigue. This method explores indicators such as muscle activation, muscle force, and intervertebral pressure during cervical resistance training and isometric training in a safe and reliable non-invasive manner.

[0004] For example, Chinese patent CN201310268647.8 discloses a femoral biomechanical finite element analysis system based on force feedback, including: a femoral CT image grayscale value extraction module for importing CT images in various formats; a femoral elastic modulus assignment module for obtaining the inner core coordinates of the femoral body grid unit; a femoral biomechanical finite element analysis module for calculating the stiffness matrix of each femoral body grid unit through the elastic modulus of each femoral body grid unit; a force feedback module for simulating the mechanical properties of the test point based on the above strain or stress data: the magnitude and direction of the force. The force feedback module is also used to transmit the magnitude and direction of the force to the operating handle of an external force sensing device. After data conversion, the force feedback device can output the force generated when the femur is actually deformed, and can truly perceive the biomechanical properties of the femoral model.

[0005] However, during actual implementation, the inventors discovered that the application of existing technologies to the analysis of pediatric musculoskeletal diseases still faces numerous challenges. During their growth and developmental stages, the characteristics of children's skeletal and muscular systems differ significantly from those of adults. Existing adult models are unable to effectively adapt to children's anatomical changes and movement characteristics, resulting in low accuracy in disease screening and diagnosis. Summary of the Invention

[0006] In view of the above problems existing in the prior art, a biomechanical simulation system for analyzing diseases of the musculoskeletal system of children is provided.

[0007] The specific technical solutions are as follows:

[0008] A biomechanical simulation system for analyzing pediatric musculoskeletal system diseases, comprising:

[0009] An acquisition module, wherein the acquisition module collects a gait video of the subject to be evaluated, and performs posture estimation based on the gait video to obtain posture estimation data;

[0010] The posture estimation data includes 3D skeleton trajectories and corresponding joint angle sequences;

[0011] A calculation module, the calculation module is connected to the acquisition module;

[0012] The calculation module performs inverse kinematics calculation based on the posture estimation data to determine the muscle driving force and joint torque on the corresponding joints of the subject to be evaluated during walking as dynamic data, and generates plantar pressure distribution and corresponding center of gravity movement trajectory based on the posture estimation data to generate kinematic data;

[0013] a classification module, the classification module being connected to the calculation module;

[0014] The classification module generates a classification result based on the kinetic data, the kinematic data and clinical information.

[0015] On the other hand, the acquisition module includes:

[0016] A video acquisition module, wherein the video acquisition module collects the gait video of the subject to be evaluated and performs background separation on the gait video to obtain a preprocessed video;

[0017] The pre-processed video includes the frontal gait and the side gait of the subject to be evaluated;

[0018] A two-dimensional processing module, the two-dimensional processing module is connected to the video acquisition module;

[0019] The two-dimensional processing module extracts two-dimensional skeleton key points from the pre-processed video to form a two-dimensional skeleton key point sequence;

[0020] a three-dimensional processing module, the three-dimensional processing module being connected to the two-dimensional processing module;

[0021] The three-dimensional processing module constructs three-dimensional skeleton key points according to the two-dimensional skeleton key point sequence, thereby predicting and obtaining the posture estimation data.

[0022] On the other hand, the acquisition module further includes:

[0023] a first discrimination module, the first discrimination module being connected to the video acquisition module;

[0024] The first discrimination module detects the image clarity of the pre-processed video using a Laplace algorithm and re-captures the image when the image clarity does not reach a preset clarity threshold;

[0025] a second discrimination module, the second discrimination module being connected to the first discrimination module;

[0026] The second discrimination module performs preliminary posture estimation on the pre-processed video and re-collects the pre-processed video when the pre-processed video does not cover a length of more than two walking cycles.

[0027] On the other hand, the two-dimensional processing module includes:

[0028] a skeleton detection module, which sequentially detects skeleton key points on a plurality of video frames in the gait video;

[0029] an occlusion enhancement module, the occlusion enhancement module being connected to the skeleton detection module;

[0030] The occlusion enhancement module performs posture comparison based on the skeleton key points and the reference model to determine the occlusion key points occluded by clothing, and performs data enhancement on the occluded joint points using the joint area occlusion mask to obtain enhanced data;

[0031] a frame interpolation module, the frame interpolation module being connected to the occlusion enhancement module;

[0032] The frame completion module compares the enhanced data according to the reference model to determine missing frames, and interpolates and completes the skeleton key points according to the enhanced data of adjacent frames for the missing frames to form a complete two-dimensional skeleton key point sequence.

[0033] On the other hand, the three-dimensional processing module includes:

[0034] a three-dimensional prediction module, which obtains physiological parameters of the object to be evaluated and predicts the three-dimensional skeleton key points based on the physiological parameters and the two-dimensional skeleton key point sequence;

[0035] a joint configuration module, the joint configuration module being connected to the three-dimensional prediction module;

[0036] The joint configuration module constructs a human body model according to the physiological parameters, and performs joint angle constraints according to the human body model and the three-dimensional skeleton key points, thereby calculating the joint angle sequence.

[0037] On the other hand, the calculation module includes:

[0038] a first configuration module, configured to configure joint connections and tendon attachment points according to the human body model to form a preconfigured model;

[0039] a second configuration module, the second configuration module being connected to the first configuration module;

[0040] The second configuration module adjusts the physiological characteristic parameters of the pre-configured model to obtain an actual configuration model;

[0041] The physiological characteristic parameters include bone density and muscle elastic modulus;

[0042] an inverse kinematics calculation module, wherein the inverse kinematics calculation module is connected to the second configuration module;

[0043] The inverse kinematics calculation module performs inverse kinematics calculation according to the actual configuration model and the posture estimation data, thereby obtaining the dynamics data;

[0044] A finite element analysis module, the finite element analysis module is connected to the inverse kinematics calculation module;

[0045] The finite element analysis module performs finite element analysis on the plantar soft tissue according to the actual configuration model and the posture estimation data, and estimates the kinematic data.

[0046] On the other hand, the finite element analysis module uses the Mooney-Rivlin hyperelastic model to simulate the plantar soft tissue and uses the anisotropic elastic model to simulate the metatarsal bones.

[0047] On the other hand, the classification module includes:

[0048] a gait classification module, wherein the gait classification module inputs the kinetic data and the kinematic data into a spatiotemporal attention neural network and uses a 3D convolution kernel to capture the spatial features of the human body model, thereby predicting abnormal gait patterns;

[0049] a risk classification module, the risk classification module being connected to the gait classification module;

[0050] The risk classification module performs multimodal classification based on the abnormal gait pattern and the clinical information to generate a risk level.

[0051] On the other hand, it also includes:

[0052] A display module generates a three-dimensional skeleton animation, a joint load thermal map, and a plantar pressure distribution cloud map according to the dynamic data and the kinematic data.

[0053] The above technical solution has the following advantages or beneficial effects:

[0054] In response to the problem that the biosimulation methods in the existing technology are not effective in simulating children, in this embodiment, posture estimation is first performed through gait video, and inverse kinematics calculation and plantar finite element analysis are performed according to the posture estimation data, so as to estimate the muscle driving force during walking and the joint torque on the corresponding joints, the plantar pressure distribution and the corresponding center of gravity movement trajectory. Then, the gait pattern of the subject to be evaluated is effectively classified in combination with clinical information, which facilitates doctors to make relevant judgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0056] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of an acquisition module in an embodiment of the present invention;

[0058] Figure 3 Schematic diagram of the first discrimination module in an embodiment of the present invention;

[0059] Figure 4 Schematic diagram of a two-dimensional processing module in an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of a three-dimensional processing module in an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram of a calculation module in an embodiment of the present invention;

[0062] Figure 7 This is a schematic diagram of a classification module in an embodiment of the present invention;

[0063] Figure 8 Schematic diagram of a display module in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0065] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0066] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0067] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0068] The present invention comprises:

[0069] A biomechanical simulation system for the analysis of pediatric musculoskeletal system diseases, such as Figure 1 Shown, including:

[0070] Acquisition module 1, which collects gait videos of the subject to be evaluated and performs posture estimation based on the gait videos to obtain posture estimation data;

[0071] The pose estimation data includes 3D skeleton trajectories and corresponding joint angle sequences;

[0072] Calculation module 2, calculation module 2 is connected to acquisition module 3;

[0073] The calculation module 2 performs inverse kinematics calculation based on the posture estimation data to determine the muscle driving force and joint torque on the corresponding joints of the subject to be evaluated during walking as dynamic data, and generates plantar pressure distribution and corresponding center of gravity movement trajectory based on the posture estimation data to generate kinematic data;

[0074] Classification module 3, classification module 3 is connected to calculation module 2;

[0075] The classification module 3 generates classification results based on the kinetic data, kinematic data and clinical information.

[0076] Specifically, to address the problem that the biosimulation methods in the prior art are not very effective in simulating children, in this embodiment, posture estimation is first performed through gait video, and inverse kinematics calculation and plantar finite element analysis are performed according to the posture estimation data, so as to estimate the muscle driving force during walking and the joint torque on the corresponding joints, the plantar pressure distribution and the corresponding center of gravity movement trajectory, and then combine the clinical information to effectively classify the gait pattern of the subject to be evaluated, so that doctors can make relevant judgments.

[0077] In one embodiment, Figure 2 As shown, the acquisition module 1 includes:

[0078] The video acquisition module 11 collects gait videos of the subject to be evaluated and performs background separation on the gait videos to obtain pre-processed videos;

[0079] A two-dimensional processing module 12, the two-dimensional processing module 12 is connected to the video acquisition module 11;

[0080] The two-dimensional processing module 12 extracts two-dimensional skeleton key points from the pre-processed video to form a two-dimensional skeleton key point sequence;

[0081] A three-dimensional processing module 13, the three-dimensional processing module 13 is connected to the two-dimensional processing module 12;

[0082] The three-dimensional processing module 13 constructs three-dimensional skeleton key points according to the two-dimensional skeleton key point sequence, thereby predicting and obtaining posture estimation data.

[0083] Specifically, in order to achieve effective estimation of the posture of the subject to be evaluated, in this embodiment, a camera device is first used to capture gait video of the subject to be evaluated, including fixed front and side perspectives, while also capturing some clinical data, such as height, weight, age, gender and other physiological data.

[0084] Subsequently, for the collected gait video, the ViBe algorithm is first enabled to segment the human body and the background in real time, and interference such as clothing wrinkles and ground textures are removed through morphological filtering to ensure that the skeleton key point area is clear.

[0085] The content of human body segmentation using Vibe algorithm includes background model initialization [M(x,y)={v_1,v_2,…,v_N},N=20] and pixel condition classification

[0086] Then, a pre-trained child-optimized version of the HRNet network is used to detect the 17 skeletal key points of the object to be evaluated. The skeletal key points mainly include multiple joints.

[0087] Among them, the HRNet network adds a feature extraction branch for immature joints such as children's carpal bones and tarsal bones based on the public HRNet-V2. The training data set contains 20,000 gait videos of children aged 3-15 years old (covering daily walking, running, jumping and other movements, 30% of which include scenes blocked by clothing).

[0088] Among them, for common data sources, a 10%-30% joint area occlusion mask can be randomly generated to simulate actual scenarios such as winter clothing covering, thereby improving the robustness of the model.

[0089] After acquiring 17 skeletal keypoints from a single viewpoint, the 2D keypoint time series (x, y, t) and physiological parameters (height, weight) are input into the MotionBert model. The 3D skeleton coordinates are then restored using a 12-layer spatiotemporal Transformer architecture (8-head attention, 1024 hidden layer dimensions). This results in a 3D coordinate sequence of multiple skeletal keypoints that changes over time, serving as the 3D skeletal trajectory. Simultaneously, a personalized human mesh model is generated based on a pediatric body database, embedding anatomical constraints such as the knee flexion range and ankle dorsiflexion range. Within this range, vector calculations are performed on the multiple identified skeletal joints, combining the 3D coordinate positions of adjacent points, to produce a joint angle sequence.

[0090] The joint angle sequence is aligned with the 3D bone trajectory.

[0091] In one embodiment, Figure 3 As shown, the acquisition module 1 also includes:

[0092] A first discrimination module 14, the first discrimination module 14 is connected to the video acquisition module 11;

[0093] The first discrimination module 14 uses the Laplace algorithm to detect the image clarity of the pre-processed video and re-captures the image if the image clarity does not reach a preset clarity threshold;

[0094] A second discrimination module 15, the second discrimination module 15 is connected to the first discrimination module 14;

[0095] The second discrimination module 15 performs preliminary posture estimation on the pre-processed video and re-collects the pre-processed video when the pre-processed video does not cover the length of more than two walking cycles.

[0096] Specifically, in order to achieve better video acquisition effects, in this embodiment, the clarity of the video and the integrity of the data are also checked during the acquisition process.

[0097] Specifically, the first discrimination module 14 uses the Laplace gradient algorithm to calculate the clarity of the video frame. The Laplace clarity detection algorithm is specifically as follows:

[0098]

[0099] On this basis, the clarity of the image is determined by a pre-calibrated threshold. If it is not clear, the image is re-captured.

[0100] At the same time, since the children are collected through manual control during the collection process, the number of samples may be missing during the collection process. The second discrimination module 15 performs preliminary posture estimation on the pre-processed video to determine whether a complete walking cycle has appeared, and re-collects when the pre-processed video does not cover the length of more than two walking cycles.

[0101] In one embodiment, Figure 4 As shown, the two-dimensional processing module 12 includes:

[0102] The skeleton detection module 121 performs skeleton key point detection on multiple video frames in the gait video in sequence to obtain pre-detection data;

[0103] A frame interpolation module 122 , the frame interpolation module 122 is connected to the skeleton detection module 121 ;

[0104] The frame supplementation module 122 compares the pre-detection data with the reference model to determine the missing frames, and interpolates the skeleton key points of the missing frames with adjacent frames to form a complete two-dimensional skeleton key point sequence.

[0105] Specifically, in order to achieve better fault tolerance and acquisition effect, in this embodiment, the skeleton detection module 121 is first used to perform skeleton key point detection on multiple video frames in the gait video in sequence.

[0106] The skeleton detection module 121 uses a pre-trained child-optimized version of the HRNet network to detect 17 skeleton key points of the object to be evaluated. During the detection process, the skeleton key points appearing in the image are output and marked.

[0107] When there are skeletal key points that cannot be directly extracted from the image, the first thing we think is that the key points are blocked by clothing due to perspective issues.

[0108] At this time, for the missing occluded joint points, the frame supplementation module 122 first compares the pre-detection data according to the reference model to determine the missing frames and the corresponding key points that need to be predicted on the missing frames.

[0109] For missing frames, the skeleton key points are interpolated and completed using the Kalman filter method according to the adjacent frames of the previous and next three frames to form a complete two-dimensional skeleton key point sequence.

[0110] The 2D skeleton keypoint sequence contains 17 sets of 2D coordinate data for each video frame, corresponding to the locations of the skeleton keypoints in the current video frame. This sequence allows for the characterization of the motion posture of the object being evaluated on the 2D plane.

[0111] In one embodiment, Figure 5 As shown, the three-dimensional processing module 13 includes:

[0112] The three-dimensional prediction module 131 obtains physiological parameters of the object to be evaluated, and predicts three-dimensional skeleton key points based on the physiological parameters and the two-dimensional skeleton key point sequence;

[0113] The joint configuration module 132 is connected to the three-dimensional prediction module 131;

[0114] The joint configuration module 132 constructs a human body model according to physiological parameters, and performs joint angle constraints according to the human body model and three-dimensional skeleton key points, thereby calculating a joint angle sequence.

[0115] Specifically, to construct 3D pose estimation data based on a two-dimensional skeleton keypoint sequence, in this embodiment, the 3D prediction module 131 first obtains the physiological parameters of the subject to be evaluated, including height and weight, from the electronic medical record system. It then predicts 3D skeleton keypoints based on the physiological parameters and the 2D skeleton keypoint sequence.

[0116] Specifically, this step first inputs the two-dimensional skeleton key point sequence (x, y, t) and physiological parameters into the MotionBert model, and predicts the forward and backward spatiotemporal relationship through a 12-layer spatiotemporal Transformer structure to restore the three-dimensional skeleton coordinates.

[0117] The input layer of the model fuses the 2D key point time series (x, y, t) with physiological parameters (height and weight normalized to the [0, 1] interval) as position encoding, and then inputs it into the spatiotemporal Transformer structure.

[0118] The spatiotemporal Transformer structure includes a multi-head attention mechanism: And its feedforward network: FFN(x)=ReLU(xW1+b1)W2+b2, thereby outputting frame-level 3D skeleton coordinates and assembling them to obtain 3D skeleton key points.

[0119] On this basis, since the physiological parameters have been obtained in advance, by searching the corresponding children's body shape database, an individualized human body Mesh model can be generated based on the standard body shape.

[0120] Combined with this personalized human mesh model, anatomical constraints are pre-embedded for multiple joints, such as limiting knee flexion to 0-150° and ankle dorsiflexion to 0-30°, to avoid unreasonable posture output. Finally, the joint angle sequence is calculated based on the 3D skeleton key points.

[0121] As an optional implementation, this model uses FP16 quantization technology to compress the model size to 1.2GB when deployed on the edge, supporting inference at ≤150ms / frame on an Nvidia Jetson Nano device, a 40% increase in inference speed compared to the original model. The output includes a 3D skeleton trajectory (.json format, recording the changes in joint coordinates over time), a joint angle sequence (.csv format, with 0.1° accuracy), and a mesh model (.obj format), which are transmitted to subsequent modules in real time via a REST API.

[0122] In one embodiment, Figure 6 As shown, the calculation module 2 includes:

[0123] A first configuration module 21, which configures joint connections and tendon attachment points according to the human body model to form a preconfigured model;

[0124] A second configuration module 22, which is connected to the first configuration module 21;

[0125] The second configuration module 22 adjusts the physiological characteristic parameters of the pre-configured model to obtain an actual configuration model;

[0126] Physiological property parameters include bone density and muscle elastic modulus;

[0127] an inverse kinematics calculation module 23 , the inverse kinematics calculation module 23 being connected to the second configuration module 22 ;

[0128] The inverse kinematics calculation module 23 performs inverse kinematics calculation based on the actual configuration model and the posture estimation data to obtain dynamic data;

[0129] Finite element analysis module 24, the finite element analysis module 24 is connected to the inverse kinematics calculation module 23;

[0130] The finite element analysis module 24 performs finite element analysis on the plantar soft tissue according to the actual configuration model and the posture estimation data, and estimates kinematic data.

[0131] Specifically, in order to achieve a better analysis of gait, in this embodiment, based on the construction of posture estimation data, modeling is first performed based on the child's anatomical structure to determine the dynamic process of each muscle during walking.

[0132] Specifically, based on the age-specific growth and development database, joint connections, 128 tendon attachment points (refer to Gray's Anatomy for Children) and center of mass positions are automatically generated to form a pre-configured model.

[0133] Subsequently, the second configuration module 22 dynamically adjusts the quality parameters of each link through the body surface area formula (Mosteller formula: body surface area = √(height × weight) / 60), simulating physiological characteristics such as bone density Z value, muscle elastic modulus (E = 5 MPa in infancy → E = 15 MPa in adolescence), and articular cartilage thickness (the average annual increase of the knee joint is 0.2 mm), thereby serving as the actual configuration model.

[0134] This method dynamically calculates the quality parameters of each link, with the error controlled within 2%, which significantly improves the accuracy compared with direct scaling of traditional adult models.

[0135] Then, the inverse kinematics calculation module 22 performs inverse kinematics calculation according to the actual configuration model and the posture estimation data, thereby obtaining dynamics data.

[0136] Specifically, the inverse kinematics calculation module 23 uses the augmented Lagrangian method (ALM) to solve the muscle driving force of the child's personalized Mesh model configured with the above-mentioned physiological characteristics. The constraints include the joint torque balance equation, muscle activation degree 0≤u≤1 and maximum contraction force, etc., and finally outputs the activation sequence of 12 major muscle groups such as the quadriceps femoris and gastrocnemius and the torque data of 6 major joints such as the knee joint and ankle joint.

[0137] The above-mentioned dynamic data can be used to effectively measure the force exerted by each muscle in each video frame during movement based on the posture estimation data, which is convenient for further prediction of whether there are any problems with gait.

[0138] To further enhance analysis accuracy, finite element simulation is also used to simulate plantar forces. Specifically, the finite element analysis module 24 uses the Mooney-Rivlin hyperelastic model to simulate plantar soft tissue and an anisotropic elastic model for the metatarsals. Ground reaction forces during the gait cycle are applied as boundary conditions, and the friction coefficient is automatically adjusted based on the sole material. The simulation results generate a plantar pressure distribution cloud map and center of gravity offset trajectory, visually demonstrating areas of abnormal force during the gait cycle.

[0139] In one embodiment, Figure 7 As shown, the classification module 3 includes:

[0140] The gait classification module 31 inputs the dynamic data and kinematic data into the spatiotemporal attention neural network and uses a 3D convolution kernel to capture the spatial features of the human body model to predict abnormal gait patterns.

[0141] The risk classification module 32 is connected to the gait classification module 31;

[0142] The risk classification module 32 performs multimodal classification based on abnormal gait patterns and clinical information to generate a risk level.

[0143] Specifically, in order to achieve effective judgment of the abnormal gait of the object to be evaluated, in this embodiment, the dynamic data and kinematic data are first input into the spatiotemporal attention neural network through the gait classification module 31, and then the spatial features of the human body model are captured by using a 3D convolution kernel to find 12 pre-marked and trained abnormal gait patterns such as dragging steps and knee hyperextension, and determine whether there is a similar motion trajectory and output the judgment result.

[0144] On this basis, the risk classification module 32 performs multimodal classification according to abnormal gait patterns and clinical information to generate a risk level.

[0145] Specifically, the classification process is implemented using a multimodal fusion model. In the early fusion stage, kinematic, dynamic, and anatomical features are spliced ​​into a 72-dimensional vector and input into two fully connected layers (with 128 and 64 neurons, respectively). In the late fusion stage, weighted voting is performed on the independent prediction results of each modality (the weights are determined by leave-one-out cross-validation), supporting the assessment of five diseases including cerebral palsy (CP) and developmental dysplasia of the hip (DDH), and outputting a 0-1 risk value (a threshold value >0.6 triggers an orange warning, and >0.8 triggers a red warning).

[0146] Multimodal fusion includes early fusion and late fusion:

[0147]

[0148] [p_{"{final}}=αp_k+βp_d+γp_a,α+β+γ=1]

[0149] Where (p_k, p_d, p_a) represents the predicted probability of kinematics and dynamics based on anatomical modalities, and (α, β, γ) represents their corresponding fusion weights.

[0150] In one embodiment, Figure 8 As shown, it also includes:

[0151] Display module 5 generates a three-dimensional skeleton animation, a joint load thermal map and a plantar pressure distribution cloud map based on the dynamic data and kinematic data.

[0152] Specifically, in order to achieve better output effects, a display module 5 is introduced in this embodiment. The display module 5 generates three-dimensional skeleton animation, joint load thermal map and plantar pressure distribution cloud map based on dynamic data and kinematic data. It can also convert the analysis results into three-dimensional animation, thermal map and other visual content, generate standardized reports and connect to the hospital information system.

[0153] Those skilled in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, may be embodied as systems, methods, or computer program products. Thus, various aspects of the present invention, or possible implementations of various aspects, may take the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuits," "modules," or "systems." Furthermore, various aspects of the present invention, or possible implementations of various aspects, may take the form of computer program products, which refer to computer instructions stored in a memory.

[0154] The memory may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).

[0155] The processor in the computer reads the computer instructions stored in the memory, so that the processor can perform the functional actions specified in each step or the combination of steps in the flowchart; and generate a device that implements the functional actions specified in each block or the combination of blocks in the block diagram.

[0156] It should be understood that the processor in the computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components implemented to execute the aforementioned computer instructions.

[0157] Computer instructions can be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in certain alternative embodiments, the functions noted in each step of the flow chart or each block in the block diagram may not occur in the order noted in the figure. For example, depending on the functions involved, two steps or two blocks shown in succession may actually be executed approximately simultaneously, or the blocks may sometimes be executed in reverse order.

[0158] In practice, the various components of a computer system are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0159] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A biomechanical simulation system for analyzing pediatric musculoskeletal system diseases, characterized in that: include: An acquisition module, wherein the acquisition module collects a gait video of the subject to be evaluated, and performs posture estimation based on the gait video to obtain posture estimation data; The posture estimation data includes 3D skeleton trajectories and corresponding joint angle sequences; A calculation module, the calculation module is connected to the acquisition module; The calculation module performs inverse kinematics calculation based on the posture estimation data to determine the muscle driving force and joint torque on the corresponding joints of the subject to be evaluated during walking as dynamic data, and generates plantar pressure distribution and corresponding center of gravity movement trajectory based on the posture estimation data to generate kinematic data; a classification module, the classification module being connected to the calculation module; The classification module generates a classification result based on the kinetic data, the kinematic data and clinical information.

2. The biomechanics simulation system according to claim 1, characterized in that: The acquisition module includes: A video acquisition module, wherein the video acquisition module collects the gait video of the subject to be evaluated and performs background separation on the gait video to obtain a preprocessed video; A two-dimensional processing module, the two-dimensional processing module is connected to the video acquisition module; The two-dimensional processing module extracts two-dimensional skeleton key points from the pre-processed video to form a two-dimensional skeleton key point sequence; a three-dimensional processing module, the three-dimensional processing module being connected to the two-dimensional processing module; The three-dimensional processing module constructs three-dimensional skeleton key points according to the two-dimensional skeleton key point sequence, thereby predicting and obtaining the posture estimation data.

3. The biomechanics simulation system according to claim 2, characterized in that: The acquisition module also includes: a first discrimination module, the first discrimination module being connected to the video acquisition module; The first discrimination module detects the image clarity of the pre-processed video using a Laplace algorithm and re-captures the image when the image clarity does not reach a preset clarity threshold; a second discrimination module, the second discrimination module being connected to the first discrimination module; The second discrimination module performs preliminary posture estimation on the pre-processed video and re-collects the pre-processed video when the pre-processed video does not cover a length of more than two walking cycles.

4. The biomechanics simulation system according to claim 2, characterized in that: The two-dimensional processing module includes: A skeleton detection module, wherein the skeleton detection module sequentially performs skeleton key point detection on a plurality of video frames in the gait video to obtain pre-detection data; a frame interpolation module connected to the skeleton detection module; The frame supplementation module compares the pre-detection data according to a reference model to determine missing frames, and interpolates and supplements skeleton key points according to adjacent frames for the missing frames to form a complete two-dimensional skeleton key point sequence.

5. The biomechanics simulation system according to claim 2, characterized in that: The three-dimensional processing module includes: a three-dimensional prediction module, which obtains physiological parameters of the object to be evaluated and predicts the three-dimensional skeleton key points based on the physiological parameters and the two-dimensional skeleton key point sequence; a joint configuration module, the joint configuration module being connected to the three-dimensional prediction module; The joint configuration module constructs a human body model according to the physiological parameters, and performs joint angle constraints according to the human body model and the three-dimensional skeleton key points, thereby calculating the joint angle sequence.

6. The biomechanics simulation system according to claim 5, characterized in that: The calculation module includes: a first configuration module, configured to configure joint connections and tendon attachment points according to the human body model to form a preconfigured model; a second configuration module, the second configuration module being connected to the first configuration module; The second configuration module adjusts the physiological characteristic parameters of the pre-configured model to obtain an actual configuration model; The physiological characteristic parameters include bone density and muscle elastic modulus; an inverse kinematics calculation module, wherein the inverse kinematics calculation module is connected to the second configuration module; The inverse kinematics calculation module performs inverse kinematics calculation according to the actual configuration model and the posture estimation data, thereby obtaining the dynamics data; A finite element analysis module, the finite element analysis module is connected to the inverse kinematics calculation module; The finite element analysis module performs finite element analysis on the plantar soft tissue according to the actual configuration model and the posture estimation data, and estimates the kinematic data.

7. The biomechanics simulation system according to claim 6, characterized in that: The finite element analysis module uses the Mooney-Rivlin hyperelastic model to simulate the plantar soft tissue and uses the anisotropic elastic model to simulate the metatarsal bones.

8. The biomechanics simulation system according to claim 1, characterized in that: The classification module includes: a gait classification module, wherein the gait classification module inputs the kinetic data and the kinematic data into a spatiotemporal attention neural network and uses a 3D convolution kernel to capture the spatial features of the human body model, thereby predicting abnormal gait patterns; a risk classification module, the risk classification module being connected to the gait classification module; The risk classification module performs multimodal classification based on the abnormal gait pattern and the clinical information to generate a risk level.

9. The biomechanics simulation system according to claim 1, characterized in that: Also includes: A display module generates a three-dimensional skeleton animation, a joint load thermal map, and a plantar pressure distribution cloud map according to the dynamic data and the kinematic data.

Images