A foot deformity correction surgery planning and implementation method

Abstract

The application relates to the technical field of surgical assistance, and discloses a foot deformity correction surgery planning and implementation method, which comprises the following steps: fusing multi-modal medical images and dynamic function data to construct a high-fidelity foot digital twin model; intelligently quantitatively evaluating and mechanismally analyzing deformities by using a deep learning network combined with a knowledge graph; automatically optimizing in a virtual space by using a multi-agent reinforcement learning algorithm to generate a personalized surgery plan verified by biomechanical simulation; converting the optimized scheme into augmented reality navigation information and robot control instructions to guide a surgical robot to perform adaptive and accurate osteotomy and reduction operations; dynamically evaluating execution deviations and guiding correction by using real-time three-dimensional scanning and registration technology during the operation, and intelligently matching and implanting internal fixators; immediately performing function verification after the operation, and generating a personalized digital rehabilitation scheme and long-term curative effect prediction for the patient based on the operation result. The application constructs an intelligent surgery ecological system throughout the whole process of "diagnosis, planning, implementation, evaluation and management", and significantly improves the accuracy, safety, predictability and scientific level of curative effect management of foot deformity correction surgery.

Description

Technical Field

[0001] This invention patent relates to the field of surgical assistance technology, specifically a method for planning and implementing foot deformity correction surgery. Background Technology

[0002] Foot deformity correction surgery is one of the most challenging areas of foot and ankle surgery, aiming to restore the normal biomechanical structure and function of the foot. Traditional surgical methods rely heavily on the surgeon's personal experience, spatial imagination, and manual manipulation skills, and have significant limitations in diagnosis, planning, execution, and evaluation.

[0003] During the preoperative planning stage, doctors mainly rely on static images such as two-dimensional X-rays, CT scans, and MRI for qualitative or semi-quantitative assessments, which makes it difficult to accurately quantify complex three-dimensional composite deformities (such as those involving rotation and displacement of multiple planes and joints simultaneously). Surgical plans are mostly based on empirical rules and the doctor's personal "mental ideas," lacking systematic and quantitative analytical tools. This makes it impossible to predict and compare the long-term biomechanical effects of different surgical strategies, resulting in significant subjectivity and uncertainty in plan formulation.

[0004] During the surgical procedure, manual manipulation is the main bottleneck for precision control. Whether it's the angle and location of the osteotomy or the spatial transformation required for bone fragment repositioning, precise execution is difficult, especially when complex three-dimensional adjustments are needed. Accumulated errors can lead to undercorrection, overcorrection, and joint mismatch. Intraoperative assessment relies primarily on repeated X-ray fluoroscopy, which involves significant radiation exposure and only provides two-dimensional information, making it difficult to assess the quality of three-dimensional repositioning in real time and comprehensively.

[0005] In the internal fixation process, the selection, shaping, and implantation of internal fixation devices (such as bone plates) often rely on experience. Problems such as non-adherence of bone surfaces, insufficient screw holding force, and impact with soft tissues frequently occur, affecting the fixation effect and potentially leading to complications.

[0006] In the postoperative management phase, rehabilitation programs are generally generic, lacking personalized guidance that is precisely linked to specific surgical procedures and individual healing responses. Long-term efficacy assessments rely heavily on patients' subjective feelings and static imaging, lacking objective and continuous functional biomechanical data, making it difficult to scientifically and quantitatively optimize surgical techniques.

[0007] In view of this, we propose a method for planning and implementing foot deformity correction surgery.

[0008] Invention Patent Content The purpose of this invention is to provide a method for planning and implementing foot deformity correction surgery to solve the problems existing in the background art.

[0009] To achieve the above objectives, this invention provides the following technical solution: A method for planning and implementing foot deformity correction surgery, including: S1. Using a variety of medical imaging and sensing devices, collect multi-dimensional data of the target foot under static and dynamic conditions, register and fuse the multi-dimensional data in a unified spatiotemporal coordinate system, and construct a comprehensive high-dimensional multimodal digital twin model that includes bony structure, soft tissue morphology, joint kinematics, ground reaction force distribution and gait phase. S2. Input the comprehensive digital twin model into a pre-trained multimodal deep learning network, and combine it with the foot and ankle biomechanics and pathology knowledge graph to automatically identify the main driving forces and compensatory links of the deformity, quantitatively assess the degree of composite deviation of the deformity in the coronal, sagittal and horizontal planes, and output a quantitative assessment report covering the displacement of the rotation center of each joint, abnormal areas of joint surface contact stress, ligament tension imbalance and abnormal muscle force vector. S3. Based on the quantitative assessment report, with the comprehensive goals of restoring physiological force lines, rebuilding joint stability, and optimizing load transfer, within the preset biomechanical constraints and clinical feasibility boundaries, a multi-agent reinforcement learning algorithm is used to automatically explore and optimize in the virtual surgical space to generate one or more personalized surgical plans that include osteotomy techniques, osteotomy surface spatial pose, bone block displacement vectors, fusion joint selection, internal fixation device configuration and implantation planning. S4. In the finite element and multibody dynamics joint simulation environment, each generated surgical plan is virtually executed and mechanically loaded. The biomechanical response of the foot in each phase of the gait cycle after surgery is simulated. The initial stability of the bone-internal fixation system, the peak and distribution of contact stress in key joints, the improvement of the trajectory deviation of the plantar pressure center are quantitatively calculated. The probabilistic assessment of potential reduction loss, internal fixation failure, and adjacent joint degeneration risks is also performed. S5. The final surgical plan, which has been verified and optimized through simulation, is decomposed into executable augmented reality navigation information and robot control instructions. The navigation information includes virtual osteotomy lines, osteotomy amounts, expected bone block positions, and internal fixation models superimposed on the patient's actual anatomical structure. The robot control instructions precisely plan the motion path, speed, force, and dynamic registration logic of the surgical robot's end effector with the intraoperative navigation system. S6. During the operation, the system integrates optical navigation, intraoperative cone-beam CT scanning and force feedback sensing to achieve real-time, high-precision six-degree-of-freedom tracking of the bone, instruments and robot end effector. The surgical robot, under the supervision of the doctor, adapts to perform bone grinding, precise osteotomy and preliminary reduction operations based on the planned instructions and real-time sensing data. S7. At key operational nodes such as osteotomy and repositioning, real-time three-dimensional point clouds of the bone surface are obtained by intraoperative three-dimensional scanning, and rapid non-rigid registration and comparison are performed with the target geometry planned in the preoperative procedure. The overall deviation matrix and local error heat map between the actual shape and the target shape are calculated in real time. If the deviation exceeds the preset tolerance, a correction guidance plan is generated immediately. S8. Based on the actual bone geometry and mechanical environment during the operation, the system calls up the biomechanical simulation model from the internal fixation device database, automatically matches the optimal model and specification of the internal fixation device, and calculates the best fit position with the bone surface and the safe implantation channel of the screw; through robot or augmented reality navigation, the system guides the doctor to complete the precise implantation, pre-bending and final fixation of the internal fixation device. S9. After the initial internal fixation is completed, a portable sensor is used to apply standardized passive movement and simulated load to the affected foot under anesthesia, and the joint range of motion, ligament tension changes and plantar micro-pressure distribution are collected in real time. The real-time data is compared with the preoperative functional goals to provide a quantitative basis for decision-making on whether the internal fixation needs to be fine-tuned (such as re-tightening or angle fine-tuning). S10. Based on the final confirmed surgical results and individual patient characteristics, automatically generate a phased and quantifiable digital rehabilitation training plan; establish a complete data chain spanning preoperative, intraoperative, and postoperative periods; construct a digital twin file exclusive to the patient; and use time series prediction models to dynamically predict and warn of medium- and long-term functional recovery and complication risks.

[0010] Preferably, the multi-agent reinforcement learning algorithm is used to collaboratively optimize multiple surgical decision variables, and its global reward function is designed as follows: ; Among them, R i The immediate reward representing the i-th optimization sub-objective (such as force line correction, joint fit, surgical invasiveness) is calculated by the corresponding evaluation function; w i λ represents the weight coefficients of each sub-objective, reflecting clinical priority; C is the penalty term for surgical complexity, positively correlated with surgical time and osteotomy difficulty; λ is the complexity penalty coefficient. The algorithm learns to maximize R through exploration and collaboration among multiple agents (representing different surgical operations) in a virtual environment. G The combined strategy generates a comprehensive and optimal surgical plan.

[0011] Preferably, the "Dynamic Stability Index" (DSI) used to quantitatively assess the stability of the surgical plan integrates strain energy under load with interface micro-motion in its calculation formula: ; Among them, K s To simulate the overall equivalent stiffness of the bone-internal fixation system during the gait cycle; E totalμ represents the total strain energy stored in the system during a single gait cycle, reflecting its ability to resist deformation. avg δ represents the average micromotion amplitude of the bone-internal fixation interface under dynamic load; δ is a small constant to prevent the denominator from being zero. A higher DSI value indicates better mechanical stability provided by the design under dynamic load.

[0012] Preferably, the calculation of the overall deviation matrix is ​​based on the non-rigid registration results of the point cloud, and its elements reflect the deviation characteristics of the local region: ; Where D is the overall deviation matrix, P actual With P plan For point clouds, D ij T reflects the deviation characteristics of a local area. local For acting on the local point set P actual_i The optimal non-rigid transformation; ||·|| is the Euclidean distance, used to calculate the positional deviation; N angle The angle between the normal vectors at the corresponding points is used to evaluate the surface angular deviation; Φ is a comprehensive function that integrates the position and angular deviations. This matrix is ​​used to generate an error heatmap to guide corrections.

[0013] Preferably, the intelligent matching of the internal fixation device is based on a "biomechanical fit score": ; Among them, S interface S represents the average contact stress between the candidate internal fixation device and the bone surface under simulated load. max To determine the acceptable maximum stress threshold, this assessment evaluates fit and pressure distribution; ΔV collision F represents the predicted potential collision volume between the internal fixation device and surrounding critical soft tissues (such as nerves and tendons). screw_variance The variance of the pull-out force of each screw under simulated load reflects the fixation equilibrium. α, β, and γ are weighting coefficients.

[0014] Preferably, the pre-trained multimodal deep learning network adopts a cross-modal attention fusion mechanism, which can simultaneously process the voxel features of three-dimensional medical images, the spatiotemporal features of motion sequences, and the features of force signals. It also uses graph neural networks to perform relational reasoning on the biomechanical graph structure composed of joints, bones, and ligaments, thereby automatically extracting deep features related to the malformation mechanism from multi-source data.

[0015] Preferably, the generated augmented reality navigation information is rendered and overlaid using a mixed reality-based head-mounted device, and the virtual information and the real surgical scene are locked in real time with millimeter-level high precision through simultaneous localization and mapping (SLAM) technology, ensuring the spatial consistency of the navigation information.

[0016] Preferably, the control of the surgical robot integrates a model-based predictive control algorithm, which can proactively and dynamically adjust the motion trajectory and output parameters based on real-time intraoperative bone impedance, instrument force, and navigation pose information to cope with anatomical variations and uncertainties in tissue mechanics, ensuring the accuracy and safety of the operation.

[0017] Preferably, when using portable sensors for passive activity testing, the tension change spectrum of ligaments and joint capsules is collected simultaneously. By calculating the symmetry and uniformity index of the tension distribution, the recovery of soft tissue balance is evaluated, providing supplementary evidence for functional verification.

[0018] Preferably, the time series prediction model combines recurrent neural networks with an attention mechanism, using multi-time point data (images, functional scores, gait parameters) from the patient's digital twin archive as input to predict joint function scores, reoperation risk probability, and gait parameter recovery curves at specific future time points, thereby achieving individualized and prospective management of treatment efficacy.

[0019] By employing the above technical solution, this invention patent provides a method for planning and implementing foot deformity correction surgery. It possesses at least the following beneficial effects: This invention patent achieves a complete, high-fidelity digital mapping of a patient's foot from morphology to function by constructing a comprehensive digital twin model that integrates static anatomy, dynamic function, and biomechanical information. It overcomes the limitations of traditional preoperative assessments that rely on two-dimensional, static, and isolated information, and solves the problem of the inability to comprehensively and accurately quantify complex three-dimensional deformities and their biomechanical consequences, providing a solid and unified data foundation for all subsequent intelligent decisions.

[0020] By developing a malformation mechanism analysis and intelligent assessment based on knowledge graphs and reinforcement learning, and by adopting a deep learning model that integrates cross-modal attention and graph neural networks, intelligent and in-depth analysis of the causes of malformations has been achieved. This solves the problem that traditional diagnosis relies on experience and is difficult to reveal the intrinsic biomechanical mechanisms of complex malformations. It upgrades diagnosis from appearance-based "description" to mechanism-based "analysis", providing a scientific basis for formulating surgical strategies that address the root cause rather than just the symptoms.

[0021] By designing multi-objective collaborative optimization and personalized solution generation for surgical strategies, and applying multi-agent reinforcement learning algorithms and global reward functions, surgical planning is transformed from an "artistic creation" process relying on the doctor's personal experience into a "scientific computing" process based on multi-objective optimization and machine learning. It can automatically explore a massive number of possible surgical combinations in virtual space, weigh multiple objectives such as force line correction, joint stability, and surgical invasiveness, and generate a personalized Pareto optimal solution set. This solves the core problem of subjective, rough, and inability to achieve optimal multi-objective trade-offs in surgical planning, and assists doctors in making more scientific and comprehensive decisions.

[0022] By introducing biomechanical simulation verification and risk quantification prediction of surgical plans, as well as quantitative evaluation indicators such as the Dynamic Stability Index (DSI), it becomes possible to conduct "virtual trial and error" and "stress testing" of the plan before the surgery. The joint simulation of finite element method and multibody dynamics can accurately predict the mechanical environment after surgery and quantitatively evaluate stability and risk. This solves the dilemma of "unpredictable results" in traditional surgery, where the efficacy can only be passively observed after surgery. It achieves a major leap from "experience-based surgery" to "predictive surgery" and significantly reduces the risk of surgical failure due to flawed plan design.

[0023] By achieving collaborative planning of augmented reality navigation and robotic surgical paths, the optimal digital solution is seamlessly transformed into executable augmented reality guidance and robot control commands, building a "bridge" connecting virtual planning and real surgery. This solves the "execution gap" problem in traditional surgery, where detailed preoperative planning is difficult to accurately reproduce during surgery. Augmented reality provides intuitive intraoperative visualization navigation, while the robot provides stability and precision that surpasses the limits of human hands. The collaboration between the two ensures the accurate implementation of the plan.

[0024] By integrating intraoperative multimodal real-time perception and adaptive precision execution, and fusing model-based predictive control algorithms, the surgical execution process possesses a closed-loop intelligence of "perception-decision-execution". The robot can dynamically adjust its operation based on real-time perceived information such as bone impedance and navigation pose, and cope with the differences in individual anatomy and tissues. This solves the problems of insufficient precision and poor adaptability of traditional manual operation or simple programmed robot operation, and greatly improves the accuracy, safety and adaptability of surgical execution.

[0025] By establishing an online measurement of intraoperative three-dimensional morphology and dynamic assessment of protocol compliance, and generating an error heatmap based on the overall deviation matrix D, a real-time "quality detection" and "feedback correction" mechanism was established at key surgical nodes. This solved the problems of traditional intraoperative reliance on two-dimensional fluoroscopy, incomplete assessment, and delayed correction. It enabled real-time, quantitative, and visual monitoring and guidance of surgical execution quality, ensuring that the final result conforms to the optimal preoperative plan to the greatest extent possible.

[0026] By developing intelligent matching and robot-assisted implantation of internal fixation systems, and making intelligent recommendations based on the Biomechanical Fit Score (BMS), the selection and implantation of internal fixation devices has been upgraded from "experience-based fitting" to "model optimization." This ensures optimal fit between the internal fixation device and the bone surface, balanced mechanical distribution, and avoidance of soft tissue. It solves the problems of poor fit, unstable fixation, and soft tissue irritation that are common in traditional internal fixation procedures, thus improving the reliability, safety, and surgical efficiency of internal fixation.

[0027] By adding steps for immediate functional verification and fine-tuning decision support during surgery, and conducting standardized biomechanical testing under anesthesia, "functional closed-loop verification" was achieved before the end of the surgery. This changed the traditional surgical approach of only pursuing "anatomical reduction" while neglecting immediate "functional reduction." It provided a final opportunity for intraoperative optimization based on objective data to address potential problems such as micro-movement and ligament imbalance caused by improper fixation, thereby improving the immediate functional efficacy of the surgery.

[0028] By constructing a full-cycle digital rehabilitation and intelligent efficacy prediction system, and using time series prediction models for dynamic management, postoperative patient management is incorporated into a data-driven precision medicine system. Rehabilitation plans are personalized and dynamic, and efficacy is predictable and predictable. This solves the problems of traditional extensive postoperative rehabilitation and subjective and one-sided follow-up assessment, forming a closed loop of digital health management covering the entire patient cycle. It provides valuable data assets for improving long-term efficacy and optimizing surgical techniques.

[0029] In summary, this invention constructs an intelligent surgical ecosystem that spans the entire process of foot deformity correction, encompassing diagnosis, planning, implementation, evaluation, and management. This not only addresses the inherent pain points of traditional surgical models at each stage but also achieves a qualitative leap in the scientific nature of surgical decisions, the precision of execution, the predictability of results, and the refinement of efficacy management through intelligent enhancement and closed-loop linkage throughout the entire process. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of the invention, form part of this application: Figure 1 This is a schematic diagram of the overall process of this invention patent. Detailed Implementation

[0031] The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0032] Please see Figure 1 The present invention provides a method for planning and implementing foot deformity correction surgery, comprising: S1. Using a variety of medical imaging and sensing devices, collect multi-dimensional data of the target foot under static and dynamic conditions, register and fuse the multi-dimensional data in a unified spatiotemporal coordinate system, and construct a comprehensive high-dimensional multimodal digital twin model that includes bony structure, soft tissue morphology, joint kinematics, ground reaction force distribution and gait phase. It should be noted that various medical imaging and sensing devices include at least: computed tomography (CT, providing high-resolution three-dimensional morphology of bones), magnetic resonance imaging (MRI, clearly showing soft tissues such as cartilage, tendons, and ligaments), and weight-bearing X-rays (reflecting static force lines). Dynamic data is acquired simultaneously through an optical motion capture system (collecting the trajectory of skin reflective markers to calculate the six degrees of freedom of joint motion) and an embedded plantar pressure plate (measuring the trajectory of the pressure center and pressure in various regions during the gait cycle). The core technical challenge of data fusion lies in spatiotemporal alignment: by extracting common bony anatomical landmarks from various data sources, point cloud registration algorithms (such as the iterative nearest point algorithm) are used to unify all images into the same three-dimensional coordinate system; at the same time, the time axis of dynamic motion capture data is synchronized with gait events (such as heel strike), thereby achieving seamless integration of static anatomical structure and dynamic functional information, ultimately forming a "digital twin" that can realistically and comprehensively reflect the patient's foot morphology and function.

[0033] S2. Input the comprehensive digital twin model into a pre-trained multimodal deep learning network, and combine it with the foot and ankle biomechanics and pathology knowledge graph to automatically identify the main driving forces and compensatory links of the deformity, quantitatively assess the degree of composite deviation of the deformity in the coronal, sagittal and horizontal planes, and output a quantitative assessment report covering the displacement of the rotation center of each joint, abnormal areas of joint surface contact stress, ligament tension imbalance and abnormal muscle force vector. It should be noted that the pre-trained multimodal deep learning network employs an advanced architecture: First, a 3D convolutional neural network branch processes voxel data from CT / MRI to extract spatial features of bones and soft tissues; simultaneously, a temporal convolutional network or recurrent neural network branch processes motion sequences and force signals to extract dynamic features. The key lies in the "cross-modal attention fusion mechanism," which automatically learns and weights the contribution of different modal features to the assessment of specific deformities. Furthermore, the system's embedded "foot and ankle biomechanics and pathology knowledge graph" encodes, in a graph structure, the topological connections and mechanical constraints between bones, joints, ligaments, and tendons under normal and common deformities. By combining this prior knowledge, the network uses graph neural networks to perform relational reasoning on the biomechanical system of the foot. This not only outputs geometric parameters such as the talus tilt angle and calcaneal valgus angle, but also infers the "primary factors" (such as posterior tibial tendon failure) and "secondary compensations" (such as calcaneal valgus and forefoot abduction) that cause deformities. Furthermore, it quantifies the abnormal contact stress distribution on the articular surfaces, providing a deeper basis for developing surgical strategies that address the root cause rather than just the symptoms.

[0034] S3. Based on the quantitative assessment report, with the comprehensive goals of restoring physiological force lines, rebuilding joint stability, and optimizing load transfer, within the preset biomechanical constraints and clinical feasibility boundaries, a multi-agent reinforcement learning algorithm is used to automatically explore and optimize in the virtual surgical space to generate one or more personalized surgical plans that include osteotomy techniques, osteotomy surface spatial pose, bone block displacement vectors, fusion joint selection, internal fixation device configuration and implantation planning. It's important to note that traditional planning involves single-path thinking, while this invention models surgical decisions (such as choosing between medial calcaneal osteotomy or lateral column extension, determining the osteotomy angle, and whether to fuse a joint) as a multi-agent collaborative decision-making problem. Each "agent" is responsible for a sub-decision (e.g., one agent is responsible for "choosing the osteotomy technique," and another for "adjusting the osteotomy angle"). They explore in parallel within a virtual surgical environment, trying different combinations of actions. The quality of their collaboration is judged by the global reward function R_G. The surgeon sets weight coefficients w_i to reflect clinical priorities (e.g., prioritizing gravity line correction versus preserving joint range of motion). Through tens of thousands of simulated trials, the algorithm allows the agents to learn the joint strategy that yields the highest reward (i.e., best aligns with multi-objective optimization). The final output is not just a single solution, but a set of "Pareto optimal" solutions, demonstrating the trade-offs between different objectives, allowing the surgeon to make the final choice based on the patient's specific situation.

[0035] S4. In the finite element and multibody dynamics joint simulation environment, each generated surgical plan is virtually executed and mechanically loaded. The biomechanical response of the foot in each phase of the gait cycle after surgery is simulated. The initial stability of the bone-internal fixation system, the peak and distribution of contact stress in key joints, the improvement of the trajectory deviation of the plantar pressure center are quantitatively calculated. The probabilistic assessment of potential reduction loss, internal fixation failure, and adjacent joint degeneration risks is also performed. It should be noted that finite element analysis is used to accurately simulate the mechanical behavior of bone, internal fixation devices (such as bone plates and screws), and their interfaces, calculate the stress and strain distribution under load, and evaluate the strength of the internal fixation system and the mechanical environment for bone healing. Multibody dynamics simulation is used to calculate the interaction forces and movements between joints during gait. The coupling of the two can predict the comprehensive biomechanical performance of the foot during walking after surgery. The Dynamic Stability Index (DSI) is a core quantitative indicator that integrates the system's stiffness (resistance to deformation), dissipated energy, and micromotions at the bone-implant interface. Its value directly reflects whether the scheme can provide sufficient mechanical stability for early bone healing. In addition, simulation can also predict the stress redistribution of adjacent joints (such as the ankle and subtalar joints) due to changes in force lines, thereby assessing the potential risk of secondary arthritis and achieving proactive early warning.

[0036] S5. The final surgical plan, which has been verified and optimized through simulation, is decomposed into executable augmented reality navigation information and robot control instructions. The navigation information includes virtual osteotomy lines, osteotomy amounts, expected bone block positions, and internal fixation models superimposed on the patient's actual anatomical structure. The robot control instructions precisely plan the motion path, speed, force, and dynamic registration logic of the surgical robot's end effector with the intraoperative navigation system. It's important to note that augmented reality navigation information is presented through a head-mounted device based on mixed reality technology. Utilizing Simultaneous Localization and Mapping (SLAM) technology, virtual osteotomy lines and planned bone plate models are precisely and in real-time "locked" onto the patient's actual anatomical structures. Doctors can see the surgical plan through the headset, greatly enhancing the intuitiveness of spatial perception. Furthermore, the instruction files generated for the surgical robot not only include spatial path points but also integrate force-position hybrid control strategies. For example, the instructions might specify: position control to ensure path precision during cortical bone incision; switching to force control when penetrating cancellous bone to prevent slippage or overcutting; and automatically triggering deceleration and protection zone settings when instruments approach important nerves and blood vessels. This ensures the precision and safety of the robot's operation.

[0037] S6. During the operation, the system integrates optical navigation, intraoperative cone-beam CT scanning and force feedback sensing to achieve real-time, high-precision six-degree-of-freedom tracking of the bone, instruments and robot end effector. The surgical robot, under the supervision of the doctor, adapts to perform bone grinding, precise osteotomy and preliminary reduction operations based on the planned instructions and real-time sensing data. It's worth noting that the optical navigation system tracks the tracer fixed to the bone and instruments in real time, providing sub-millimeter-level spatial positioning. Intraoperative cone-beam computed tomography (CBCT) can quickly scan as needed, updating the bone model to handle potential unforeseen circumstances. Force feedback sensors allow the robot to "sense" the stiffness of the bone and the forces applied to the instruments. This multimodal sensing data is fed into a model-based predictive control (MPC) algorithm. This algorithm not only adjusts commands based on current errors but also predicts the state several steps ahead, making smooth and optimized control decisions in advance. This enables the robot to cope with uncertainties such as differences in bone texture and soft tissue traction, achieving more intelligent and compliant operation than pre-programmed paths.

[0038] S7. At key operational nodes such as osteotomy and repositioning, real-time three-dimensional point clouds of the bone surface are obtained by intraoperative three-dimensional scanning, and rapid non-rigid registration and comparison are performed with the target geometry planned in the preoperative procedure. The overall deviation matrix and local error heat map between the actual shape and the target shape are calculated in real time. If the deviation exceeds the preset tolerance, a correction guidance plan is generated immediately. It should be noted that a dense point cloud data of the bone surface is acquired using an intraoperative 3D scanner (such as a handheld device based on structured light or laser scanning). Since the bone fragment may undergo slight deformation after osteotomy, a non-rigid registration algorithm (such as thin-plate spline transformation) is used to compare the real-time scanning model with the preoperative planning model, which reflects the actual situation more accurately than rigid registration. The calculated overall deviation matrix D and its visualization product—the error heatmap—can intuitively show which areas deviate from the plan and by how much. For example, the heatmap uses color coding to show areas with errors ranging from millimeters (green) to exceeding a threshold (red). Based on this, the system can automatically analyze the causes of deviations and generate correction suggestions, such as "the medial osteotomy surface needs to be ground down by another 0.5mm" or "the bone fragment needs to be rotated inward by 2 degrees," guiding the surgeon or robot to make precise adjustments.

[0039] S8. Based on the actual bone geometry and mechanical environment during the operation, the system calls up the biomechanical simulation model from the internal fixation device database, automatically matches the optimal model and specification of the internal fixation device, and calculates the best fit position with the bone surface and the safe implantation channel of the screw; through robot or augmented reality navigation, the system guides the doctor to complete the precise implantation, pre-bending and final fixation of the internal fixation device. It should be noted that the established internal fixation device database includes not only three-dimensional geometric models but also their finite element mechanical models. During matching, the system virtually matches the actual three-dimensional bone model during surgery with the internal fixation device models in the database and quickly performs finite element analysis to calculate the Biomechanical Fit Score (BMS). This score comprehensively considers fit (avoiding high-point stress), implantation safety (avoiding neurovascular structures), and fixation balance (uniform holding force of each screw). The system recommends the internal fixation device model with the highest BMS and projects its optimal placement position, contour, and the "safe implantation channel" (three-dimensional axis) of each screw onto the bone surface using AR navigation. Surgeons or robots can then perform precise implantation based on this information, eliminating the need for repeated trial molding and manual bending during surgery, thus improving efficiency and effectiveness.

[0040] S9. After the initial internal fixation is completed, a portable sensor is used to apply standardized passive movement and simulated load to the affected foot under anesthesia, and the joint range of motion, ligament tension changes and plantar micro-pressure distribution are collected in real time. The real-time data is compared with the preoperative functional goals to provide a quantitative basis for decision-making on whether the internal fixation needs to be fine-tuned (such as re-tightening or angle fine-tuning). It should be noted that while the patient is still under anesthesia and with muscles relaxed, the doctor passively moves the affected foot and ankle joint (dorsiflexion / plantar flexion, inversion / eversion), while simultaneously measuring pressure distribution changes using a thin-film plantar pressure pad and assessing the tension of key ligaments (such as the deltoid ligament and spring ligament) using a probe with a tension sensor. Indicators such as the pressure distribution symmetry index are calculated in real time and compared with data from the "expected good functional state" obtained based on preoperative digital twin simulation. If limited joint movement, abnormal pressure distribution, or ligament tension imbalance is found due to minor displacement of internal fixation devices or uneven tightening force, the system will provide quantitative prompts, such as "excessive pressure in the lateral joint space; it is recommended to loosen the third lateral screw by half a turn," providing objective evidence for the doctor to make final optimizations before closing the incision.

[0041] S10. Based on the final confirmed surgical results and individual patient characteristics, automatically generate a phased and quantifiable digital rehabilitation training plan; establish a complete data chain spanning preoperative, intraoperative, and postoperative periods; construct a digital twin file exclusive to the patient; and use time series prediction models to dynamically predict and warn of medium- and long-term functional recovery and complication risks. It should be noted that the rehabilitation plan is automatically generated by the system, and its intensity, frequency, and progression standards are closely linked to the specific surgical procedure, internal fixation stability, and individual factors such as the patient's age and bone quality. More importantly, the patient's digital twin profile built by the system will be continuously updated, incorporating new data such as X-rays, CT scans, and gait analysis at each follow-up visit. Utilizing a time-series prediction model combining recurrent neural networks and attention mechanisms, the system can learn the patient's unique healing pattern and dynamically predict their future functional recovery curve (such as changes in AOFAS score), bone healing probability, and risks such as internal fixation loosening and adjacent joint degeneration, achieving individualized and proactive management of treatment efficacy and early warning of complications.

[0042] Multi-agent reinforcement learning algorithms are used to collaboratively optimize multiple surgical decision variables, and their global reward function is designed as follows: ; Among them, R i The immediate reward representing the i-th optimization sub-objective (such as force line correction, joint fit, surgical invasiveness) is calculated by the corresponding evaluation function; w i λ represents the weight coefficients of each sub-objective, reflecting clinical priority; C is the penalty term for surgical complexity, positively correlated with surgical time and osteotomy difficulty; λ is the complexity penalty coefficient. The algorithm learns to maximize R through exploration and collaboration among multiple agents (representing different surgical operations) in a virtual environment. G The combined strategy generates a comprehensive and optimal surgical plan; It is worth noting that the global reward function R GThe complex surgical planning objectives are broken down into multiple quantifiable sub-objectives R. i (such as the degree of force line correction, the reduction in joint contact stress, and the size of surgical trauma), and weighted by a coefficient w. i This allows for flexible consideration of priorities in different clinical scenarios (e.g., joint preservation may be more important for younger patients, while immediate stability may be more important for older patients). Introducing a "surgical complexity" penalty term C effectively avoids the algorithm recommending theoretically excellent but extremely complex and high-risk procedures, ensuring the clinical applicability of the generated plans. Multiple agents maximize R through exploration and collaboration. G Essentially, it allows machines to simulate the thought process of expert doctors weighing the pros and cons of various surgical combinations, but the scope and depth of exploration far exceed human capabilities.

[0043] The Dynamic Stability Index (DSI) is used to quantitatively assess the stability of surgical plans. Its calculation formula integrates strain energy under load and interface micromotion. ; Among them, K s To simulate the overall equivalent stiffness of the bone-internal fixation system during the gait cycle; E total μ represents the total strain energy stored in the system during a single gait cycle, reflecting its ability to resist deformation. avg δ represents the average micromotion amplitude of the bone-internal fixation interface under dynamic load; δ is a small constant to prevent the denominator from being zero. A higher DSI value indicates better mechanical stability provided by the design under dynamic load. It is worth noting that the Dynamic Stability Index (DSI) is a comprehensive evaluation indicator with clear physical meaning, and the numerator K... s / E total This reflects the overall "stiffness efficiency" of the bone-internal fixation system, that is, the stiffness provided per unit strain energy consumed. A higher value indicates a more rational and efficient system structure. The denominator is 1 / (μ avg +δ) focuses on the interfacial mechanical environment of bone healing, and interfacial micromotion μ avg Excessive stress can hinder bone healing and lead to internal fixation failure. DSI (Displacement Stress Injection) cleverly combines the overall deformation resistance of the system with the local micromechanical environment of the interface; its value directly predicts whether the bone can achieve stable healing in a favorable mechanical environment in the early postoperative period. This is more comprehensive and reliable than simply looking at single indicators such as maximum stress or displacement.

[0044] The overall deviation matrix is ​​calculated based on the non-rigid registration results of the point cloud, and its elements reflect the deviation characteristics of local regions: ; Where D is the overall deviation matrix, P actual With Pplan For point clouds, D ij T reflects the deviation characteristics of a local area. local For acting on the local point set P actual_i The optimal non-rigid transformation; ||·|| is the Euclidean distance, used to calculate the positional deviation; N angle The angle between the normal vectors at the corresponding points is used to evaluate the surface angular deviation; Φ is a comprehensive function that integrates position and angular deviation. This matrix is ​​used to generate an error heatmap to guide corrections. It is worth noting that bones, especially bone fragments after osteotomy, are not rigid bodies and may undergo slight deformation. Non-rigid registration can capture this deformation, making the comparison more realistic. Each element D in the deviation matrix D... ij The function Φ integrates positional error and normal angle error, which more comprehensively reflects the differences in three-dimensional morphology than simply calculating point-to-point distance (for example, a parallel offset and an angular offset, even at the same distance, have different clinical significance and correction methods). Error heatmaps are generated through matrix operations and visualization, providing surgeons with an extremely intuitive and accurate "construction drawing deviation report," making intraoperative corrections evidence-based and targeted.

[0045] Intelligent matching of internal fixation devices is based on a "biomechanical fit score": ; Among them, S interface S represents the average contact stress between the candidate internal fixation device and the bone surface under simulated load. max To determine the acceptable maximum stress threshold, this assessment evaluates fit and pressure distribution; ΔV collision F represents the predicted potential collision volume between the internal fixation device and surrounding critical soft tissues (such as nerves and tendons). screw_variance The variance of the pull-out force of each screw under simulated load reflects the fixation equilibrium. α, β, and γ are weighting coefficients. It is worth noting that the Biomechanical Fit Score (BMS) model comprehensively evaluates candidate internal fixation devices from multiple dimensions. The first dimension assesses biomechanical compatibility, requiring good contact between the internal fixation device and the bone surface to avoid stress concentration leading to pain or plate fatigue fracture. The second dimension assesses surgical safety, actively avoiding collisions between the implant and surrounding important soft tissues to reduce complications such as nerve injury and tendinitis. The third dimension assesses fixation reliability, measuring the balance of fixation through the variance of screw pull-out force; excessive variance indicates a risk of stress shielding or insufficient local holding force. The BMS model transforms clinical experience into calculable parameters, enabling the selection of internal fixation devices to move from "approximately suitable" to "optimal match."

[0046] The pre-trained multimodal deep learning network adopts a cross-modal attention fusion mechanism, which can simultaneously process the voxel features of three-dimensional medical images, the spatiotemporal features of motion sequences, and the features of force signals. It also uses graph neural networks to perform relational reasoning on the biomechanical graph structure composed of joints, bones, and ligaments, thereby automatically extracting deep features related to the malformation mechanism from multi-source data. It is worth noting that the cross-modal attention fusion mechanism is the core of this network's ability to achieve deep understanding. It allows the network to dynamically focus on the modal information most relevant to the current analysis task when processing data. For example, when assessing ligament tension, the network might pay more attention to the soft tissue imaging features of MRI and the joint stability data captured by motion capture; while when assessing bony deformities, it would focus more on the skeletal morphological features of CT scans. This mechanism mimics the process by which expert physicians comprehensively review different examination reports. Combined with graph neural networks for modeling the foot's biomechanical system, the network can not only identify abnormalities but also understand the abnormal interactions between bones, joints, and ligaments, thus achieving true "mechanism diagnosis."

[0047] The generated augmented reality navigation information is rendered and overlaid using a mixed reality-based head-mounted device, and the virtual information and the real surgical scene are locked in real time with millimeter-level high precision through simultaneous localization and mapping technology, ensuring the spatial consistency of the navigation information. It is worth noting that SLAM technology enables devices to build maps and self-locate in the complex operating room environment in real time, thus stably "anchoring" virtual information to real-world coordinates. This overcomes the problems of distraction and spatial disorientation caused by the surgeon's repeated looking up and down and switching of gaze required by traditional monitor navigation. While focusing on the surgical field, the doctor can perceive the virtual planning lines with their peripheral vision, achieving coordination and unity of "eye-hand-information," greatly improving the intuitiveness and smoothness of the surgery.

[0048] The control of the surgical robot integrates model-based predictive control algorithms, which can proactively and dynamically adjust the motion trajectory and output parameters based on real-time intraoperative bone impedance, instrument force, and navigation pose information to cope with anatomical variations and uncertainties in tissue mechanics, ensuring the precision and safety of the operation. It is worth noting that, unlike traditional proportional-integral-derivative (PI-DI) control or pre-programmed path control, MPC is an optimization control strategy. In each control cycle, it not only predicts the system's behavior over a future period based on the current error but also on the dynamic model of the robot and its environment, and solves for a series of optimal control inputs to make the predicted trajectory best match the desired trajectory. This allows the robot to "think" in advance, responding to changes in resistance during bone cutting and disturbances caused by soft tissue traction, achieving smoother, more adaptive, and interference-resistant operations. It exhibits stronger robustness, especially when dealing with tissues with ambiguous boundaries or uneven textures.

[0049] When conducting passive activity tests using portable sensors, the tension change spectrum of ligaments and joint capsules is collected simultaneously. By calculating the symmetry and uniformity index of the tension distribution, the recovery of soft tissue balance is assessed, providing supplementary evidence for functional verification. It is important to note that the correction of foot deformities involves not only skeletal alignment but also the reconstruction of the balance of surrounding soft tissues. By measuring the tension curves of key ligaments (such as the deltoid and calcaneofibular ligaments) during passive joint movement using miniature tension sensors and comparing them with the healthy side or an ideal model, it is possible to quantitatively assess whether soft tissue contractures have been released or whether new tension imbalances exist. This provides direct evidence for determining whether the surgery has achieved a "dual balance of bone and soft tissues," avoiding postoperative joint stiffness, pain, or recurrence of deformity due to neglecting soft tissue issues.

[0050] The time series prediction model combines recurrent neural networks with attention mechanisms. It takes multi-time point data (images, functional scores, gait parameters) from the patient's digital twin archive as input to predict joint function scores, reoperation risk probability, and gait parameter recovery curves at specific future time points, thereby achieving individualized and prospective management of treatment efficacy. It is worth noting that models combining recurrent neural networks (RNNs) with attention mechanisms are well-suited for processing time-series data from patient follow-ups. RNNs can memorize long-term dependencies (e.g., healing progress at 3 months post-surgery can affect function at 6 months), while attention mechanisms allow the model to focus on the most informative data from past time points during prediction (e.g., an early radiolucent line from a follow-up may be a better predictor of loosening risk than a standard image from another time). This type of model can learn complex healing patterns and complication evolution from massive amounts of historical patient data, thus providing personalized, dynamically updated prognostic predictions for current patients and serving as a powerful tool for achieving precise postoperative management.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0052] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for planning and implementing foot deformity correction surgery, characterized in that, The method includes: S1. Using a variety of medical imaging and sensing devices, collect multi-dimensional data of the target foot under static and dynamic conditions, register and fuse the multi-dimensional data in a unified spatiotemporal coordinate system, and construct a comprehensive high-dimensional multimodal digital twin model that includes bony structure, soft tissue morphology, joint kinematics, ground reaction force distribution and gait phase. S2. Input the comprehensive digital twin model into a pre-trained multimodal deep learning network, and combine it with the foot and ankle biomechanics and pathology knowledge graph to automatically identify the main driving forces and compensatory links of the deformity, quantitatively assess the degree of composite deviation of the deformity in the coronal, sagittal and horizontal planes, and output a quantitative assessment report covering the displacement of the rotation center of each joint, abnormal areas of joint surface contact stress, ligament tension imbalance and abnormal muscle force vector. S3. Based on the quantitative assessment report, with the comprehensive goals of restoring physiological force lines, rebuilding joint stability, and optimizing load transfer, within the preset biomechanical constraints and clinical feasibility boundaries, a multi-agent reinforcement learning algorithm is used to automatically explore and optimize in the virtual surgical space to generate one or more personalized surgical plans that include osteotomy techniques, osteotomy surface spatial pose, bone block displacement vectors, fusion joint selection, internal fixation device configuration and implantation planning. S4. In the finite element and multibody dynamics joint simulation environment, each generated surgical plan is virtually executed and mechanically loaded. The biomechanical response of the foot in each phase of the gait cycle after surgery is simulated. The initial stability of the bone-internal fixation system, the peak and distribution of contact stress in key joints, the improvement of the trajectory deviation of the plantar pressure center are quantitatively calculated. The probabilistic assessment of potential reduction loss, internal fixation failure, and adjacent joint degeneration risks is also performed. S5. The final surgical plan, which has been verified and optimized through simulation, is decomposed into executable augmented reality navigation information and robot control instructions. The navigation information includes virtual osteotomy lines, osteotomy amounts, expected bone block positions, and internal fixation models superimposed on the patient's actual anatomical structure. The robot control instructions precisely plan the motion path, speed, force, and dynamic registration logic of the surgical robot's end effector with the intraoperative navigation system. S6. During the operation, the system integrates optical navigation, intraoperative cone-beam CT scanning and force feedback sensing to achieve real-time, high-precision six-degree-of-freedom tracking of the bone, instruments and robot end effector. The surgical robot, under the supervision of the doctor, adapts to perform bone grinding, precise osteotomy and preliminary reduction operations based on the planned instructions and real-time sensing data. S7. At key operational nodes such as osteotomy and repositioning, real-time three-dimensional point clouds of the bone surface are obtained by intraoperative three-dimensional scanning, and rapid non-rigid registration and comparison are performed with the target geometry planned in the preoperative procedure. The overall deviation matrix and local error heat map between the actual shape and the target shape are calculated in real time. If the deviation exceeds the preset tolerance, a correction guidance plan is generated immediately. S8. Based on the actual bone geometry and mechanical environment during the operation, the system calls up the biomechanical simulation model from the internal fixation device database, automatically matches the optimal model and specification of the internal fixation device, and calculates the best fit position with the bone surface and the safe implantation channel of the screw; through robot or augmented reality navigation, the system guides the doctor to complete the precise implantation, pre-bending and final fixation of the internal fixation device. S9. After the initial internal fixation is completed, a portable sensor is used to apply standardized passive movement and simulated load to the affected foot under anesthesia, and the joint range of motion, ligament tension changes and plantar micro-pressure distribution are collected in real time. The real-time data is compared with the preoperative functional goals to provide a quantitative basis for decision-making on whether the internal fixation needs to be fine-tuned (such as re-tightening or angle fine-tuning). S10. Based on the final confirmed surgical results and individual patient characteristics, automatically generate a phased and quantifiable digital rehabilitation training plan; establish a complete data chain spanning preoperative, intraoperative, and postoperative periods; construct a digital twin file exclusive to the patient; and use time series prediction models to dynamically predict and warn of medium- and long-term functional recovery and complication risks.

2. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The multi-agent reinforcement learning algorithm is used to collaboratively optimize multiple surgical decision variables, and its global reward function is designed as follows: ； Among them, R i The immediate reward representing the i-th optimization sub-objective (such as force line correction, joint fit, surgical invasiveness) is calculated by the corresponding evaluation function; w i λ represents the weight coefficients of each sub-objective, reflecting clinical priority; C is the penalty term for surgical complexity, positively correlated with surgical time and osteotomy difficulty; λ is the complexity penalty coefficient. The algorithm learns to maximize R through exploration and collaboration among multiple agents (representing different surgical operations) in a virtual environment. G The combined strategy generates a comprehensive and optimal surgical plan.

3. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The "Dynamic Stability Index" (DSI), used to quantitatively assess the stability of surgical plans, integrates strain energy under load with interface micro-motion in its calculation formula. ； Among them, K s To simulate the overall equivalent stiffness of the bone-internal fixation system during the gait cycle; E total μ represents the total strain energy stored in the system during a single gait cycle, reflecting its ability to resist deformation. avg δ represents the average micromotion amplitude of the bone-internal fixation interface under dynamic load; δ is a small constant to prevent the denominator from being zero. A higher DSI value indicates better mechanical stability provided by the design under dynamic load.

4. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The calculation of the overall deviation matrix is ​​based on the non-rigid registration results of the point cloud, and its elements reflect the deviation characteristics of local regions: ； Where D is the overall deviation matrix, P actual With P plan For point clouds, D ij T reflects the deviation characteristics of a local area. local For acting on the local point set P actual_i The optimal non-rigid transformation; ||·|| is the Euclidean distance, used to calculate the positional deviation; N angle The angle between the normal vectors at the corresponding points is used to evaluate the surface angular deviation; Φ is a comprehensive function that integrates the position and angular deviations. This matrix is ​​used to generate an error heatmap to guide corrections.

5. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The intelligent matching of the internal fixation device is based on a "biomechanical fit score": ； Among them, S interface S represents the average contact stress between the candidate internal fixation device and the bone surface under simulated load. max To determine the acceptable maximum stress threshold, this assessment evaluates fit and pressure distribution; ΔV collision F represents the predicted potential collision volume between the internal fixation device and surrounding critical soft tissues (such as nerves and tendons). screw_variance The variance of the pull-out force of each screw under simulated load reflects the fixation equilibrium. α, β, and γ are weighting coefficients.

6. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The pre-trained multimodal deep learning network adopts a cross-modal attention fusion mechanism, which can simultaneously process the voxel features of three-dimensional medical images, the spatiotemporal features of motion sequences, and the features of force signals. It also uses graph neural networks to perform relational reasoning on the biomechanical graph structure composed of joints, bones, and ligaments, thereby automatically extracting deep features related to the malformation mechanism from multi-source data.

7. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The generated augmented reality navigation information is rendered and overlaid using a mixed reality-based head-mounted device, and achieves millimeter-level high-precision, real-time spatial locking between virtual information and the real surgical scene through simultaneous localization and mapping (SLAM) technology, ensuring spatial consistency of the navigation information.

8. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The control of the surgical robot integrates a model-based predictive control algorithm, which can proactively and dynamically adjust the motion trajectory and output parameters based on real-time intraoperative information such as bone impedance, instrument force, and navigation posture. This addresses anatomical variations and uncertainties in tissue mechanics, ensuring the precision and safety of the operation.

9. The method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, When conducting passive activity tests using portable sensors, the tension change spectrum of ligaments and joint capsules is collected simultaneously. By calculating the symmetry and uniformity index of the tension distribution, the recovery of soft tissue balance is assessed, providing supplementary evidence for functional verification.

10. A method for planning and implementing foot deformity correction surgery according to claim 1, characterized in that, The time series prediction model combines recurrent neural networks with an attention mechanism. It takes multi-time point data (images, functional scores, gait parameters) from the patient's digital twin file as input to predict joint function scores, reoperation risk probability, and gait parameter recovery curves at specific future time points, thereby achieving individualized and prospective management of treatment efficacy.

Images