A surgical robot control system based on digital twinning
By constructing a surgical robot control system using digital twin technology, the problem of insufficient control precision in surgical robots has been solved, enabling high-precision and adaptive surgical operations, reducing reliance on doctors' experience, and improving the safety and smoothness of surgery.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNION MEDICAL COLLEGE HOSPITAL
- Filing Date
- 2026-06-05
- Publication Date
- 2026-07-24
AI Technical Summary
Existing surgical robot control systems suffer from insufficient precision and operational risks when translating the surgeon's intentions into robotic arm movements. They require extensive training and rely heavily on the surgeon's experience, which affects the smoothness and safety of the surgery.
A surgical robot control system based on digital twins is adopted. Through a digital twin simulation training platform, a hierarchical skill model library, an online perception synchronization module, a skill reasoning and decision-making module, and a control decision-making module, a digital twin is constructed and task motion planning is performed. Multimodal input signals are collected in real time, and surgical plans are dynamically selected to reduce reliance on the doctor's subjective experience.
It improves the control precision of surgical robots, reduces unexpected risks, enhances the adaptability to sudden situations during surgery, and ensures the accuracy and safety of surgical actions.
Smart Images
Figure CN122440323A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical device technology, and more specifically, to a surgical robot control system based on digital twins. Background Technology
[0002] Surgical robots are advanced intelligent surgical platforms that are led and controlled by doctors throughout the entire process. By integrating high-precision robotic arms, three-dimensional high-definition vision systems, and intelligent control algorithms, they accurately translate the doctor's operational intentions into delicate movements on the operating table.
[0003] Currently, surgical robot control systems primarily rely on surgeons transmitting their intentions to the robotic arm in real time. Surgeons need to establish a mental mapping between 3D vision and a 2D interface, while adapting to the lack or delay of feedback. This requires hundreds or even thousands of hours of simulation and clinical training to reach a proficient level. Furthermore, when encountering unexpected situations during actual operation, surgeons must rely on their experience and on-the-spot judgment, constantly repositioning themselves. This not only disrupts the flow of the surgery but also significantly increases operational risks. Summary of the Invention
[0004] The problem addressed by this invention is how to improve the control precision of surgical robots.
[0005] To address the above problems, this invention provides a surgical robot control system based on digital twins.
[0006] In a first aspect, the present invention provides a surgical robot control system based on digital twins, characterized in that it includes a digital twin simulation training platform, a hierarchical skill model library, an online perception synchronization module, a skill reasoning and decision-making module, and a control decision-making module; The digital twin simulation training platform is used to construct a digital twin using preoperative medical imaging data and send the digital twin to the hierarchical skill model library; The hierarchical skill model library is used to perform task motion planning on the digital twin to obtain at least one complete surgical plan, and to send all the complete surgical plans to the skill reasoning and decision module; The online perception synchronization module is used to obtain current surgical information through multimodal input signals and send the current surgical information to the skill reasoning and decision-making module; The skill reasoning and decision-making module is used to filter all the complete surgical plans based on the current surgical information to obtain a target surgical decision reference plan; The control decision module is used to receive the target surgical decision reference scheme and control the robotic arm to perform actions according to the target surgical decision reference scheme.
[0007] Optionally, the online sensing and synchronization module further includes a doctor intent recognition unit, which is used to obtain doctor operation instructions by extracting modal features from the multimodal input signal and send the doctor operation instructions to the control decision module.
[0008] Optionally, controlling the surgical robot according to the target surgical decision reference scheme includes: By assigning operational weights to the doctor's operational instructions and the target surgical decision reference scheme, a dynamic autonomous weight coefficient is obtained; The target pose command is obtained based on the dynamic autonomous weight coefficient, the doctor's operation command, and the target surgical decision reference scheme; The robotic arm is controlled to perform actions according to the target pose command.
[0009] Optionally, obtaining the doctor's operation instructions by extracting modal features from the multimodal input signal includes: The multimodal input signals are preprocessed to obtain the processed signals, wherein the multimodal input signals include surgical robot pose, surgical robot speed, surgical robot applied force, endoscopist's operating area, and heart rate variability; The processed signal is input into the intent recognition model for modal feature extraction to obtain the doctor's operation instructions.
[0010] Optionally, the dynamic autonomous weighting coefficients include: , in, C_task is the preset autonomy weight of the current task of the target surgical decision reference scheme, C_model is the confidence of the current execution skill, I_intervene is the doctor intervention intensity measure, I_confidence is the doctor state confidence of the doctor operation instruction, and f() is the weight calculation function.
[0011] Optionally, the target pose command includes: , Wherein, X_cmd is the target pose command. Xd is the dynamic autonomous weight coefficient, Xa is the doctor's expected pose in the doctor's operation command, K is the target surgical suggestion pose in the target surgical decision reference scheme, Fd is the doctor's applied interaction force in the doctor's operation command, and Fa is the ideal interaction force in the target surgical decision reference scheme.
[0012] Optionally, the step of performing task motion planning on the digital twin to obtain at least one complete surgical plan includes: The digital twin is input into the basic operation skill model to obtain multiple atomic-level operation units; All the atomic-level operation units are input into the composite task skill model and combined to obtain multiple composite tasks; Based on the task motion planning algorithm, all the composite tasks are used as nodes in the planning graph to obtain at least one complete surgical plan.
[0013] Optionally, the online sensing synchronization module is further configured to update the digital twin based on the multimodal input signal to obtain an updated digital twin.
[0014] Optionally, the surgical robot control system further includes a safety verification module, which is used to verify and correct the target surgical decision reference scheme by mapping it to the digital twin, thereby obtaining a corrected target surgical decision reference scheme.
[0015] Optionally, the construction of a digital twin using preoperative medical imaging data includes: Feature extraction is performed on the preoperative medical imaging data to obtain image feature data; Based on the image feature data, a model is constructed to obtain an initial geometric 3D model; Based on the finite element method and the hybrid model of a point mass and a spring, physical properties are assigned to the initial three-dimensional geometric model to obtain the digital twin.
[0016] The beneficial effects of the digital twin-based surgical robot control system of this invention are as follows: The digital twin simulation training platform utilizes preoperative medical imaging data to construct a digital twin, enabling the pre- and post-operative simulation and verification of surgical plans in a virtual environment, effectively reducing the unexpected risks of traditional surgery. The hierarchical skill model library uses a hierarchical reinforcement learning framework to perform task motion planning on the digital twin, training multiple complete surgical plans from basic operations to the entire procedure. The online perception and synchronization module collects multimodal input signals in real time and obtains current surgical information. The skill reasoning and decision-making module dynamically filters complete surgical plans based on the current surgical information, obtaining a target surgical decision reference plan, enabling the robot to adapt to unexpected situations during surgery and ensuring the accuracy of surgical actions. The control decision-making module controls the robotic arm to perform actions according to the target surgical decision reference plan, reducing excessive reliance on the surgeon's subjective experience and thus improving the control accuracy of the surgical robot. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of a surgical robot control system based on digital twin according to an embodiment of the present invention; Figure 2 This is a schematic diagram of another surgical robot control system based on digital twin according to an embodiment of the present invention. Detailed Implementation
[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.
[0019] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.
[0020] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.
[0021] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0022] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.
[0023] like Figure 1 , Figure 2 As shown in the figure, an embodiment of the present invention provides a surgical robot control system based on digital twin, including a digital twin simulation training platform, a hierarchical skill model library, an online perception synchronization module, a skill reasoning decision module, and a control decision module; The digital twin simulation training platform is used to construct a digital twin using preoperative medical imaging data and send the digital twin to the hierarchical skill model library.
[0024] Specifically, by utilizing artificial intelligence technology to transform preoperative medical imaging data into a three-dimensional, visualized model, the system assigns realistic physical properties to different parts of the 3D model. By introducing physical modeling techniques to simulate real physiological functions, a digital twin is obtained. For example, blood vessels are given elasticity and resilience, soft tissues are given softness, and tumors are given a hard texture.
[0025] The hierarchical skill model library is used to perform task motion planning on the digital twin to obtain at least one complete surgical plan, and to send all the complete surgical plans to the skill reasoning and decision module.
[0026] Specifically, the hierarchical skill model library is trained using a three-layer progressive skill architecture. First, atomic-level operational units are defined through a basic operational skill model to construct basic skill sequences. Then, a meaningful sub-task composed of these basic skill sequences is generated using a composite task skill model trained with hierarchical reinforcement learning. Finally, the corresponding standard surgical procedure is obtained as a complete surgical plan based on a task motion planning algorithm.
[0027] The online perception synchronization module is used to obtain current surgical information through multimodal input signals and send the current surgical information to the skill reasoning and decision-making module.
[0028] Specifically, the multimodal input signals include real-time surgical images, real-time status data, and physician operation data. Real-time surgical images represent an endoscopic video stream acquired via the endoscope at a rate of tens of frames per second, providing texture and color information of the tissue surface. Real-time status data and physician operation data include patient physiological data, pose, velocity, and applied force in the master hand control signals, the inferred fixation point of the physician's operating area from the endoscopic video stream, and optional physiological signals such as heart rate variability. Based on the multimodal input signals, physician operation instructions and an updated digital twin are obtained, thus yielding current surgical information, including the current surgical status and task progress.
[0029] The skill reasoning and decision-making module is used to filter all the complete surgical plans based on the current surgical information to obtain a target surgical decision reference plan.
[0030] Specifically, the current surgical information includes the current surgical status and task progress. Based on the current surgical status and task progress, the most matching skill model is dynamically retrieved from the hierarchical skill model library to obtain a complete surgical plan. Multimodal input signals are input into the activated skill policy network, simultaneously generating autonomous action proposals and ideal interaction force predictions, and quantifying the uncertainty of the prediction results. The final output includes: a synthesized autonomous action proposal Xa, containing the target pose and expected velocity of the instrument; a predicted ideal interaction force Fa, containing three-dimensional force and three-dimensional torque; and a comprehensive confidence score C_model, reflecting the overall reliability of the prediction. These outputs are passed to the context memory and history fusion unit for further processing. The context memory and history fusion unit is responsible for using historical information to perform temporal smoothing and consistency enhancement on the current inference results. This unit maintains an action memory buffer, storing autonomous action proposals and corresponding execution result feedback from multiple past time steps. Smoothing the action sequence through a temporal filter can effectively eliminate jitter caused by single-frame inference, making the motion more natural and coherent.
[0031] The control decision module is used to receive the target surgical decision reference scheme and control the robotic arm to perform actions according to the target surgical decision reference scheme.
[0032] Specifically, the virtual coordinate system is precisely aligned with the base coordinate system of the robotic arm in the physical world, ensuring that the two are completely coincident in space. Each node in the target surgical decision reference scheme is converted into a corresponding action, and the generated actions are smoothed before being used as input signals to the main hand controller to control the robotic arm to perform actions.
[0033] In this embodiment, the digital twin simulation training platform utilizes preoperative medical imaging data to construct a digital twin, enabling the pre- and post-operative simulation and verification of surgical plans in a virtual environment, effectively reducing the unexpected risks associated with traditional surgery. A hierarchical skill model library employs a hierarchical reinforcement learning framework to perform task motion planning on the digital twin, training multiple complete surgical plans ranging from basic operations to the entire procedure. An online perception and synchronization module collects multimodal input signals in real time and obtains current surgical information. A skill reasoning and decision-making module dynamically filters complete surgical plans based on the current surgical information, obtaining a target surgical decision reference plan. This enables the robot to adapt to unexpected intraoperative situations, ensuring the accuracy of surgical actions. The control and decision-making module controls the robotic arm to perform actions based on the target surgical decision reference plan, reducing over-reliance on the surgeon's subjective experience and thus improving the control precision of the surgical robot.
[0034] Optionally, the online sensing and synchronization module further includes a doctor intent recognition unit, which is used to obtain doctor operation instructions by extracting modal features from the multimodal input signal and send the doctor operation instructions to the control decision module.
[0035] Optionally, such as Figure 2 As shown, controlling the surgical robot according to the target surgical decision reference scheme includes: By assigning operational weights to the doctor's operational instructions and the target surgical decision reference scheme, a dynamic autonomous weight coefficient is obtained; The target pose command is obtained based on the dynamic autonomous weight coefficient, the doctor's operation command, and the target surgical decision reference scheme; The robotic arm is controlled to perform actions according to the target pose command.
[0036] Optionally, obtaining the doctor's operation instructions by extracting modal features from the multimodal input signal includes: The multimodal input signals are preprocessed to obtain the processed signals, wherein the multimodal input signals include surgical robot pose, surgical robot speed, surgical robot applied force, endoscopist's operating area, and heart rate variability; The processed signal is input into the intent recognition model for modal feature extraction to obtain the doctor's operation instructions.
[0037] Specifically, the endoscopist's operating area represents the inferable fixation point of the endoscopic video stream, while heart rate variability represents the physiological signals collected by the wearable device to assess the doctor's fatigue state. The multimodal input signals are first preprocessed, with low-pass filtering to remove high-frequency noise and extraction of statistical features such as mean, variance, and acceleration. A design combining a multimodal temporal Transformer and a bidirectional LSTM is employed. The multimodal temporal Transformer layer is responsible for cross-modal attention fusion of the manipulation feature sequence (surgical robot pose, surgical robot speed, and applied force), visual feature sequence (endoscopist's operating area), and contextual features (heart rate variability), capturing the correlation between different modalities. A self-attention mechanism is used to calculate the correlation weights between different modalities, achieving cross-modal feature alignment and fusion. For example, the system uses the attention mechanism to identify changes in the correlation strength between visual fixation points and hand gesture intentions under specific fatigue states. Subsequently, a bidirectional LSTM layer performs temporal modeling on the fused feature sequence, extracting pattern features of the operation as it changes over time. This effectively captures the long-term dependencies and dynamic patterns of the operation behavior over time, extracting deep intent features embedded in continuous actions. Finally, the network generates different types of intent information through multiple output heads. For example, one output head is used to predict the next instrument movement trajectory; another output head is used to identify the current surgical stage (such as separation, suturing, hemostasis); and a third output head is used to assess the operational risk level in real time.
[0038] In this optional embodiment, the doctor's intent recognition unit infers the doctor's operational intent and collaborative state in real time through multimodal signals, providing an understanding basis for human-machine collaboration. It can not only understand the doctor's current actions but also predict their deeper intentions by combining their physiological state and visual focus, thereby providing a more intelligent and safer auxiliary control strategy for the surgical robot.
[0039] Optionally, the dynamic autonomous weighting coefficients include: , in, C_task is the preset autonomy weight of the current task of the target surgical decision reference scheme, C_model is the confidence of the current execution skill, I_intervene is the doctor intervention intensity measure, I_confidence is the doctor state confidence of the doctor operation instruction, and f() is the weight calculation function.
[0040] Specifically, generate a scalar coefficient in the range [0, 1]. Where 0 represents fully manual and 1 represents fully autonomous. C_task is the preset autonomy weight for the current task of the target surgical decision reference scheme; for example, the "tissue traction" task has a high weight, and the "important structure separation" task has a low weight. C_model is the confidence level of the currently executed skill, such as the output of the policy network value function or uncertainty estimation. I_intervene is a measure of the intensity of physician intervention; this value increases when the force Fd applied by the physician or the direction of movement deviates from the autonomously suggested direction beyond a threshold, leading to… Reduce. I_confidence is the doctor's confidence level in the doctor's operation instructions, and f() is the weight calculation function. If the system determines that the doctor's intention is clear and the operation is smooth, then the weight is appropriately reduced. If the doctor is deemed hesitant or fatigued, the workload can be appropriately increased under safe conditions. .
[0041] In this optional embodiment, the solution dynamically allocates control between the doctor and the robot by calculating the dynamic autonomous weight coefficient in real time. By comprehensively considering the doctor's situation and adjusting the dynamic autonomous weight coefficient, the solution fully leverages the respective advantages of both humans and machines, maximizing the efficiency of human-machine collaboration.
[0042] Optionally, the target pose command includes: , Wherein, X_cmd is the target pose command. Xd is the dynamic autonomous weight coefficient, Xa is the doctor's expected pose in the doctor's operation command, K is the target surgical suggestion pose in the target surgical decision reference scheme, Fd is the doctor's applied interaction force in the doctor's operation command, and Fa is the ideal interaction force in the target surgical decision reference scheme.
[0043] Specifically, admittance control allows the robot to respond compliantly to additional forces applied by the doctor, achieving a sense of physical cooperation. The robot's motion speed and acceleration are based on... The rate of change is smoothed using a filtering method to avoid abrupt changes.
[0044] Optionally, such as Figure 2 As shown, the process of performing task motion planning on the digital twin to obtain at least one complete surgical plan includes: The digital twin is input into the basic operation skill model to obtain multiple atomic-level operation units; All the atomic-level operation units are input into the composite task skill model and combined to obtain multiple composite tasks; Based on the task motion planning algorithm, all the composite tasks are used as nodes in the planning graph to obtain at least one complete surgical plan.
[0045] Specifically, a three-layer progressive skill architecture is used for training. First, atomic-level operational units are defined through basic operational skill models, such as "grasping the suture needle," "tying a surgical knot," "blunt dissection," and "precise point coagulation." Each basic skill trains an independent deep reinforcement learning policy network. Its state space S_b contains the local field of view image, the device pose / force information, and the target position; the motion space A_b is the incremental motion of the robotic arm end effector or the device joints; the reward function R_b focuses on operational accuracy, stability, and safety.
[0046] All atomic-level operation units are input into the composite task skill model, which consists of meaningful sub-tasks composed of basic skill sequences, such as "completing a continuous suture" and "completely dissecting the gallbladder artery." A hierarchical reinforcement learning method is used to train a high-level meta-policy network. The system learns which basic skill (or low-level skill) to invoke and combine under what environmental state S_m, and sets a sub-goal (e.g., "the next stitch should be placed at point A"). S_m refers to the surgical scenario; in different surgical scenarios, some require suturing, while others require hemostasis, requiring the determination of the optimal operation. Sub-goals are derived from basic skills and specifically include end-effector position, instrument posture, contact force, etc. For example, when suturing is required, the end-effector position and posture must be controlled. The low-level controller (i.e., the basic skill model) is responsible for executing to achieve the sub-goal. State S_m contains more global field of view and task progress information, obtained from endoscopic video, with optimized video image processing. The current observation is compared with the defined surgical task goal to obtain the task progress. Finally, based on the task motion planning algorithm, a corresponding standard surgical procedure (e.g., "laparoscopic cholecystectomy") is performed. Through task and motion planning, the composite task is used as a node in the planning graph. The system retrieves the standardized surgical skill matching each task node from a predefined skill model library. Then, based on the specific anatomical structure of the patient's twin (such as the precise location of blood vessels and the thickness of tissues), it parameterizes these skills, for example, setting the movement trajectory of the robotic arm, the magnitude of the applied force, and the cutting depth. Finally, the system connects these parameterized task nodes in a logical order to form a complete, executable surgical motion sequence as a complete surgical plan.
[0047] In this optional embodiment, a three-step progressive strategy is employed to achieve intelligent and structured generation of surgical plans. A hierarchical planning strategy is used, first decomposing and then combining the solutions, avoiding the high-dimensional state space search problem caused by performing a one-time global plan for the entire surgical process. By breaking down a large problem into multiple smaller problems, the computational burden of the algorithm is significantly reduced, improving the real-time performance and feasibility of the generated solutions.
[0048] Optionally, the online sensing synchronization module is further configured to update the digital twin based on the multimodal input signal to obtain an updated digital twin.
[0049] Specifically, the digital twin is updated based on real-time surgical images and real-time status data to obtain the updated digital twin.
[0050] Optionally, the surgical robot control system further includes a safety verification module, which is used to verify and correct the target surgical decision reference scheme by mapping it to the digital twin, thereby obtaining a corrected target surgical decision reference scheme.
[0051] Specifically, the security verification module is responsible for performing multi-level security verification on the action suggestions generated by inference to ensure the safety of the output actions. Spatial boundary verification maps the action suggestions to the three-dimensional space of the patient's digital twin and performs collision detection with the pre-marked danger areas. If it detects that the action is about to enter a danger area, it triggers a safety correction algorithm to project the action vector into the safe space.
[0052] Optionally, the construction of a digital twin using preoperative medical imaging data includes: Feature extraction is performed on the preoperative medical imaging data to obtain image feature data; Based on the image feature data, a model is constructed to obtain an initial geometric 3D model; Based on the finite element method and the hybrid model of a point mass and a spring, physical properties are assigned to the initial three-dimensional geometric model to obtain the digital twin.
[0053] Specifically, such as Figure 2 As shown, preoperative medical imaging data (such as CT / MRI) is input for feature data extraction. Organ segmentation and model building are then performed based on the feature data. Biomechanical-based physical properties (such as nonlinear elasticity, viscoelasticity, and anisotropy) are added to the geometric model. A hybrid model combining finite element method (FEM) and mass-spring method is used to improve the accuracy of deformation calculations while ensuring real-time performance, resulting in a digital twin. For example, the critical area (surgical operating area) uses a finite element model, while the surrounding tissue uses a mass-spring model.
[0054] In some more specific embodiments, such as Figure 2As shown, a large amount of abdominal CT data from patients is imported into the digital twin simulation training platform to generate digital twins of the gallbladder and liver with variability. Using a basic operational skill model, a "blunt dissection" skill model is trained, with a reward function encouraging thorough dissection and avoiding liver tissue damage. A composite task skill model is used to train the "gallbladder bed dissection" skill. A high-level strategy learning system plans the dissection path (e.g., advancing from the bottom of the gallbladder towards the neck) and continuously invokes the "blunt dissection" basic skill; simultaneously, the "electrocoagulation hemostasis" basic skill is trained, and during simulated bleeding, the high-level strategy learning system interrupts the dissection and invokes the hemostasis skill. An online perception synchronization module collects multimodal input signals and, based on the intraoperative laparoscopic view, registers and updates the patient's current digital twin in real time. When the surgeon begins the dissection operation, the online perception synchronization module obtains the surgeon's operational instructions. The skill reasoning and decision-making module identifies the current task as "gallbladder bed dissection," invokes the relevant complete surgical plan from the hierarchical skill model, and generates an autonomous dissection action suggestion Xa. The control decision-making module calculates the dynamic autonomous weight coefficient based on the following information. Currently in the "stripping" phase (C_task=0.7), model confidence is high (C_model=0.9), and doctors are operating stably (low I_intervene, high I_confidence). Calculated... =0.6, the system enters semi-autonomous cooperative mode. The hybrid controller presses... =0.6 Integrating physician input and autonomous suggestions, the robot performs dissection with 60% autonomy. Physicians can easily control the process through minor guiding forces or directional adjustments (reflected in the admittance control parameters). When encountering critical vascular areas, the system automatically reduces the weight of C_task; or the physician actively increases the grip to indicate takeover, resulting in an increase in I_intervene. The reading dynamically drops below 0.2, at which point the system reverts to auxiliary mode, primarily relying on the doctor's input. Once the stripping process is complete, the system's identification task ends. Reset to zero and await the doctor's next instructions.
[0055] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.
Claims
1. A surgical robot control system based on digital twin, characterized in that, It includes a digital twin simulation training platform, a hierarchical skill model library, an online perception synchronization module, a skill reasoning and decision-making module, and a control decision-making module; The digital twin simulation training platform is used to construct a digital twin using preoperative medical imaging data and send the digital twin to the hierarchical skill model library; The hierarchical skill model library is used to perform task motion planning on the digital twin to obtain at least one complete surgical plan, and to send all the complete surgical plans to the skill reasoning and decision module; The online perception synchronization module is used to obtain current surgical information through multimodal input signals and send the current surgical information to the skill reasoning and decision-making module; The skill reasoning and decision-making module is used to filter all the complete surgical plans based on the current surgical information to obtain a target surgical decision reference plan; The control decision module is used to receive the target surgical decision reference scheme and control the robotic arm to perform actions according to the target surgical decision reference scheme.
2. The surgical robot control system based on digital twin according to claim 1, characterized in that, The online sensing and synchronization module also includes a doctor intent recognition unit, which is used to obtain doctor operation instructions by extracting modal features from the multimodal input signal and send the doctor operation instructions to the control decision module.
3. The surgical robot control system based on digital twin according to claim 2, characterized in that, The step of controlling the surgical robot according to the target surgical decision reference scheme includes: By assigning operational weights to the doctor's operational instructions and the target surgical decision reference scheme, a dynamic autonomous weight coefficient is obtained; The target pose command is obtained based on the dynamic autonomous weight coefficient, the doctor's operation command, and the target surgical decision reference scheme; The robotic arm is controlled to perform actions according to the target pose command.
4. The surgical robot control system based on digital twin according to claim 2, characterized in that, The step of obtaining doctor's operation instructions by extracting modal features from the multimodal input signal includes: The multimodal input signals are preprocessed to obtain the processed signals, wherein the multimodal input signals include surgical robot pose, surgical robot speed, surgical robot applied force, endoscopist's operating area, and heart rate variability; The processed signal is input into the intent recognition model for modal feature extraction to obtain the doctor's operation instructions.
5. The surgical robot control system based on digital twin according to claim 3, characterized in that, The dynamic autonomous weighting coefficients include: ， in, C_task is the preset autonomy weight of the current task of the target surgical decision reference scheme, C_model is the confidence of the current execution skill, I_intervene is the doctor intervention intensity measure, I_confidence is the doctor state confidence of the doctor operation instruction, and f() is the weight calculation function.
6. The surgical robot control system based on digital twin according to claim 3, characterized in that, The target pose command includes: ， Wherein, X_cmd is the target pose command. Xd is the dynamic autonomous weight coefficient, Xa is the doctor's expected pose in the doctor's operation instruction, K is the target surgical suggestion pose in the target surgical decision reference scheme, Fd is the doctor's applied interaction force in the doctor's operation instruction, and Fa is the ideal interaction force in the target surgical decision reference scheme.
7. The surgical robot control system based on digital twin according to claim 1, characterized in that, The process of performing task motion planning on the digital twin to obtain at least one complete surgical plan includes: The digital twin is input into the basic operation skill model to obtain multiple atomic-level operation units; All the atomic-level operation units are input into the composite task skill model and combined to obtain multiple composite tasks; Based on the task motion planning algorithm, all the composite tasks are used as nodes in the planning graph to obtain at least one complete surgical plan.
8. The surgical robot control system based on digital twin according to claim 1, characterized in that, The online sensing and synchronization module is also used to update the digital twin based on the multimodal input signal to obtain the updated digital twin.
9. The surgical robot control system based on digital twin according to claim 1, characterized in that, The surgical robot control system also includes a safety verification module, which is used to verify and correct the target surgical decision reference scheme by mapping it to the digital twin, thereby obtaining a corrected target surgical decision reference scheme.
10. The surgical robot control system based on digital twin according to claim 1, characterized in that, The construction of a digital twin using preoperative medical imaging data includes: Feature extraction is performed on the preoperative medical imaging data to obtain image feature data; Based on the image feature data, a model is constructed to obtain an initial geometric 3D model; Based on the finite element method and the hybrid model of a point mass and a spring, physical properties are assigned to the initial three-dimensional geometric model to obtain the digital twin.